Riverbend Institute for Responsible AI Software

Words in [brackets] are blanks you fill in with your own details when you adopt this sample in Supreme Plan AI.

Executive Summary

This nonprofit research institute in Massachusetts is seeking a grant of $120,000 from the Open Commons Research Foundation to build and openly release evaluation tools, datasets and reference software that help public-benefit organizations test AI systems before they use them. Our work is applied research with a clear public purpose: many schools, libraries, clinics, housing agencies and small nonprofits are being offered AI products they cannot independently check, and they need tools that show, in plain terms, what a system does well, where it fails and for whom.

The problem is practical. A public-benefit organization that considers an AI system for intake, translation, summarizing case notes or sorting requests often has no staff who can test it, no safe way to try it on real records and no shared standard to compare vendors. Our project will produce an open evaluation toolkit, a set of consent-based and synthetic test datasets, and a training program that teaches staff to run the tests themselves.

In Year 1 we will complete the first toolkit release, run three community evaluation studies with partner organizations and publish the methods and results. The grant funds the core research team, the compute and evaluation datasets, the community research panel and the ethics review that governs every study. Our Year 1 budget is about $997,000, with about $220,000 from research contracts, $60,000 from public-benefit licensing and service fees and $60,000 from donations, memberships and corporate partners. We expect 7 paid staff in Year 1 and 10 in Year 5. Grant money is planned to fall from about 66% of revenue in Year 1 to about 37% in Year 5.

We do not claim that our toolkit will make any system safe or unbiased. Our role is to make testing possible and to publish what the tests show, including the limits of the tests themselves.

I am Dr. Helen Okafor, the Executive Director, a researcher in machine learning evaluation. The numbers in this proposal are estimates based on the assumptions listed at the end, not promises.

Company Description

Riverbend Institute for Responsible AI Software is a nonprofit research institute organized as a 501(c)(3) public charity in Massachusetts, governed by an independent board of directors. We will provide our IRS determination letter, our articles of organization, our board list, our conflict-of-interest and research ethics policies and our most recent financial statements with this proposal. We will confirm eligibility with the Open Commons Research Foundation before we accept any award.

Our mission. To help public-benefit organizations understand and test the AI systems they are offered, so that the systems that serve people are tested in the open, against standards the public can see.

What we believe. Testing is a public good. A tool that is not tested can harm the people it serves, and a tool that is tested in private can hide its failures. We publish our methods, our datasets where the consent and privacy rules allow, and our results, including the limits of our tests. We work with the people affected by a system, not only with the organizations that buy it. We do not accept funding that requires us to change a finding.

How we started. The institute grew from a seminar on evaluation in public services. Its founders, two researchers and a former county social services director, found that agencies were asking the same questions about vendor tools and getting very different answers. A small pilot with three public-benefit partners produced the first working of our evaluation checklist, which was reviewed by a community panel and revised. That checklist is the starting point for the toolkit described in this proposal.

Who we serve. Public-benefit organizations, including public libraries, school districts, community health centers, housing agencies, legal aid organizations and small nonprofits, and the communities they serve. Our tools are free to use. Organizations pay for training, for custom evaluations and for membership if they wish.

What we are not. We are not a vendor, we do not sell AI systems, and we do not certify any product as safe, fair or compliant. Our evaluations are research, and every report says what was tested and what was not. We do not conduct surveillance, we do not identify individuals and we do not share participant data.

What we offer. An open evaluation toolkit with test plans, scoring methods and reporting templates. Consent-based and synthetic evaluation datasets. Training for staff who will run tests in their own organization. Custom evaluations for member organizations, with a written report and a public summary where the partner agrees. A community research panel of people who use public services, who review our methods and our findings before release.

How we are run. A board of seven independent directors, including a public-interest lawyer, a community representative, an academic researcher and a former public agency leader, meets quarterly. An Institutional Review Board or equivalent research ethics committee, independent of management, approves each study that involves people. An outside accountant reviews our books each year.

Where things stand today. We have a working evaluation checklist tested with three partner organizations, letters of support from a public library system, a school district, a community health center and a regional housing authority, and a research agreement under negotiation with a state university's ethics board for our community panel. We have not yet released a public toolkit, and this grant funds the first release.

Problem & Opportunity

More public-benefit organizations are being offered AI systems than ever before. These systems promise to translate documents, summarize casework, sort requests, answer questions for residents and help staff write letters and reports. Some of these tools may help a great deal. Others may make mistakes that are hard to see, such as misreading a name in one language, treating a common phrase as a risk signal or giving confident answers that are wrong.

Most public-benefit organizations cannot test these systems themselves. They lack technical staff, they do not have a safe way to run a system on real records, and they have no shared standard to compare one vendor against another. A vendor's own results are often the only evidence a buyer sees, and those results are rarely described in terms a non-specialist can use. A decision to adopt an AI tool can therefore be made on a demonstration, a sales claim or a conference booth.

The people most affected are often the least able to check. Residents who use a library, a clinic or a housing agency may not know that an AI system has touched their file. People who speak a language other than English, have a disability or have less access to a lawyer or an advocate may bear the cost of a system that works poorly for them. Testing has to look at those groups, not only at average performance.

Existing research tools are built for researchers and for large companies. They are often expensive, technical or tied to a single model. Public-benefit organizations need methods they can run, reports they can read and datasets they are allowed to use.

We do not know how much harm current systems cause in public services, and we do not claim to. Our project is designed to create the measurement tools that would let organizations and the public find out. We will report our own limits, including the fact that a test is only as good as the situations it covers and that no test can prove a system safe.

Products & Services

Riverbend Institute for Responsible AI Software will release an open evaluation toolkit and run community evaluation studies with partner organizations. The work proceeds in three parts: building and checking the tools, running studies with partners and training organizations to run the tests themselves.

How it works, step by step:
1. Scoping. A partner organization describes the task it wants to use an AI system for, the people affected and the risks it worries about. We agree on a written evaluation plan with the partner and the community panel.
2. Ethics review. Any study that involves real records or people is reviewed by our independent research ethics committee before it begins. Consent is obtained where needed, and records are minimized and de-identified.
3. Testing. We test the system on consent-based or synthetic data across the tasks and groups the plan covers, measuring accuracy, consistency, error types and differences in performance across language, disability and other relevant groups where the data allows and the ethics review permits.
4. Reporting. We write a report that describes what was tested, what was not tested, the results and the uncertainty. The partner decides what to publish, and we publish a public summary where the partner agrees and the results are safe to share.
5. Training. We teach the partner's staff to run the toolkit's core tests, so the organization can repeat the evaluation when a system changes.
6. Release. The toolkit, templates and methods are published under an open license, with a version history, so other organizations and researchers can use them and improve them.

What the grant will pay for. Part of one Principal Research Scientist's salary, the research engineers and postdoctoral researcher who build and test the toolkit, the shared compute and licensed evaluation datasets, the community research panel stipends and the research ethics review. The grant does not pay for lobbying or for any activity that would require us to change a finding.

Community panel. A group of people who use public services reviews our methods, questions and reports before release. Panel members are paid stipends by check or payroll, never cash, and they may stop at any time. Their feedback is recorded and published in summary form.

Independence and conflicts. We accept no funding from a vendor whose product we are evaluating in the same study, and we disclose all funders of each study. Any conflict of interest for staff or board members is disclosed and managed under written policy.

Why this approach is strong. Public-benefit organizations get tools they can run, partners and residents help shape what is tested, and results are published openly. We do not promise that a system will pass or fail. We promise to measure it honestly and to say what we could and could not learn.

Market Analysis

Who benefits: public-benefit organizations and the communities they serve, and, through open publication, other researchers, vendors who want honest feedback and the public.

Our primary users are staff at public libraries, school districts, community health centers, housing agencies, legal aid organizations and small nonprofits that are considering or already using AI systems. We expect our first users to be organizations with a partner relationship and a specific system to test. Over time, we expect to serve a wider group through the toolkit and training.

Community members are the people whose lives are affected. They gain a voice in what is tested, through the community panel and through the plain-language public summaries.

Researchers and funders gain open methods and datasets they can use and check. Vendors gain honest, published feedback, which can help them improve, even when the results are not flattering.

We will use the most recent public data from the Census Bureau, federal and state agencies and sector surveys to describe need in our service area, and we will not overstate it. Our planning assumes the research contracts and service fees described in the funding section, with a growing number of member organizations and training clients. We do not guarantee demand from any organization.

We accept payment for services only on written terms, and we never accept payment in exchange for a particular result.

Competitive Landscape

Other choices public-benefit organizations have for testing AI, and what makes us different.

- Vendor documentation and demonstrations. Vendors publish results, and these are useful. They are written by the seller, often on tests the seller chose, and they rarely show how a system performs for the groups a public agency serves. We do not replace vendor documentation. We check it and add the tests vendors do not run.
- University research groups. They produce excellent work, often as papers. Their methods can be hard for a public agency to run, and their projects end when a grant ends. We work with universities and publish in forms public agencies can use.
- Large technology companies' safety and evaluation programs. These are valuable and tied to their own products. Our tests are designed to be independent of any product.
- Commercial evaluation consultancies. Some offer evaluations for a fee, which can be costly for a small agency. Our toolkit is free to use, and our custom work is priced on a sliding scale.
- Government guidance and standards. Federal and state agencies publish frameworks for AI risk. They are helpful, and they do not give an agency a test it can run. We translate such frameworks into tests.

What makes us different.
- Open methods, datasets where permitted and published results, including limitations.
- Community panel review before any release.
- Training so that partner staff can run the tests themselves.
- Independent ethics review of each study involving people.
- No vendor funding for evaluations of that vendor's product.

What we do not claim. We are new, and our first toolkit has not yet been used across many organizations. Our tests are only as good as the situations they cover, and we will say so.

Marketing & Sales Strategy

How we will reach the organizations, researchers and communities we serve once the project is funded.

Partner organizations. We will work first with the three partners who helped develop the checklist, and then recruit a wider group through library systems, school district networks, state associations of public agencies, community health networks and housing agency associations. Each partner signs a written research agreement that describes the study, the data, the publication terms and the right to withdraw.

Communities. We will hold open sessions in libraries and community centers to explain what testing is, what we are testing and how residents can take part in the panel. We will publish plain-language summaries in several languages where our partners have the capacity to translate them.

Researchers. We will publish our methods in open-access venues and present at workshops of public-interest technology groups. We will share our code and datasets under open licenses, and invite comment and reuse.

Funders and the public. We will report our activities, results and finances to funders and the public each year. We will disclose all funders for each study and publish our conflict-of-interest policy.

Vendors. We will tell any vendor we evaluate what we tested before we publish, and we will offer them a chance to respond. We will not accept any payment from a vendor for a result, and we will not allow a vendor to approve or edit a finding.

Media and public briefings. We will brief journalists on our methods and limits as well as our results, and we will avoid language that implies more certainty than the work supports.

Operations Plan

How the work will be carried out.

Timeline for Year 1:
- Months 1 to 3: Recruit the research engineers and postdoctoral researcher, set up the ethics review committee and the community panel, sign research agreements with the three partners, set up secure storage for consent-based data and confirm the open license and publication policy.
- Months 4 to 6: Build the first toolkit release from the working checklist, test it on synthetic data and revise it with the community panel.
- Months 7 to 9: Run three community evaluation studies with partners, each reviewed by the ethics committee. Begin training partner staff.
- Months 10 to 12: Publish the toolkit version 1.0 and the study reports and public summaries that partners approve. Plan Year 2 with the board and the panel.

Research infrastructure. Shared compute and research workstations run our evaluations, with the cloud and research compute costs planned at about $60,000 in Year 1. Dataset licensing and evaluation tools are about $20,000. Data is stored in encrypted, access-controlled systems, with logs of every access, and is deleted on the schedule in each research agreement.

Ethics and consent. Every study that involves real records or people is reviewed by the independent research ethics committee before it starts, and the committee may require changes, may approve a study with conditions or may decline it. Participants give informed consent in plain language, in the language they choose, and may withdraw. We minimize and de-identify data, use synthetic data where it answers the question, and do not re-identify anyone. We follow the federal rules for research with human subjects that apply to our work, and our attorney confirms which rules apply to each study.

Community panel. The panel meets monthly for a stipend, with a set number of hours per meeting. Panel members review methods, questions and working reports, and their comments are recorded in the study file. Stipends are about $24,000 a year in Year 1, paid by check or payroll.

Publication and open release. We publish our toolkit under an open license and our methods in open-access venues. Study reports are published with the partner's agreement, with a summary for the public. We do not publish personal data, and we check every report for re-identification risk before release.

Independence. We do not evaluate a product for a vendor that funds our work for that product. We disclose funders of every study, we manage conflicts of interest under written policy, and no one outside the research team may change a finding.

Safety and security. Staff and panel members follow data security rules, with multi-factor authentication, encryption, device controls and training. We will notify each partner of any security incident as its agreement requires.

Legal and insurance. Our attorney reviews each research agreement, data-use term and open license. We carry general liability, professional liability, cyber liability, and directors and officers coverage, and workers' compensation once we hire. Insurance is about $14,000 a year in Year 1. Grant money is tracked separately, and an outside accountant reviews our books every year.

Management & Organization

The people who will carry out the project.

Dr. Helen Okafor, Executive Director. A researcher in machine learning evaluation with more than twelve years of experience in testing systems for public-sector use, and a former co-chair of a state technology advisory group. She leads the institute, its partnerships and its governance, reports to the board and is the main contact for funders. The Principal Research Scientist, a separate hire described below, leads the research program day to day. (Replace this paragraph with your own director's background.)

Principal Research Scientist (one, full time). Designs the evaluation methods, leads the toolkit, supervises the research team and co-signs study reports. A doctorate or equivalent experience in evaluation or measurement is expected.

Research Engineers (two in Year 1, three in Year 3, four in Year 4). Build the toolkit, the testing harnesses and the secure data environment, and run evaluations.

Postdoctoral Researcher (one in Year 1, two from Year 3). Conducts studies, leads analysis and writes publications.

Community Engagement and Ethics Lead. Coordinates the community panel, the partner relationships, public sessions and the research ethics committee's administration. Trained in research ethics and community-based research.

Research Coordinator (part-time). Manages research agreements, consent records, the study file and grant reporting.

Board of directors. Seven independent directors who meet quarterly: a public-interest lawyer, a community representative, an academic researcher, a former public agency leader and three others with relevant expertise. The board approves the budget and policies and does not receive pay. Directors may not be employed by a vendor whose product we evaluate.

Research ethics committee. An independent committee of at least five members, including a community member, that reviews every study involving people, records its decisions and meets monthly.

Community research panel. About twelve people who use public services, recruited through partner organizations and open calls, and paid stipends for their time.

Advisors. Academic and practitioner advisors in machine learning evaluation, public administration and data protection advise on methods and the open release. Advisors disclose any relationships with vendors.

Staff pay in this plan includes payroll taxes and benefits. Salaries are Executive Director $130,000, Principal Research Scientist $160,000, Research Engineer $125,000, Postdoctoral Researcher $72,000, Community Engagement and Ethics Lead $90,000 and part-time Research Coordinator $45,000. Replace them with the pay you can offer where you live.

Milestones Narrative

Our goals for each year and how we will measure them.

- Year 1: Release toolkit version 1.0, complete three community evaluation studies with partners, train partner staff and publish the methods and study summaries. A revenue of about $997,000 in revenue, with surplus of about $23,800. 7 paid staff.
- Year 2: Run six studies, release toolkit version 1.1 with community panel feedback and train staff from at least ten partner organizations. A revenue of about $1,020,000 in revenue, with surplus of about $23,800. 7 paid staff.
- Year 3: Add a second research engineer and a second postdoctoral researcher, run a multi-language evaluation program and publish the first annual evaluation report. A revenue of about $1,246,500 in revenue, with surplus of about $27,000. 9 paid staff.
- Year 4: Support member organizations with custom evaluations on a sliding-scale fee and expand the community panel to more regions. A revenue of about $1,275,500 in revenue, with surplus of about $26,700. 9 paid staff.
- Year 5: Sustain the toolkit with member and service revenue, keep the community panel active and reduce the share of revenue from grants. A revenue of about $1,436,000 in revenue, with surplus of about $28,800. 10 paid staff.

What we measure:
- Partner organizations and studies completed, and training participants.
- Toolkit downloads, reuse by other organizations and contributions from outside the institute.
- Study quality, including the completeness of each report's limitations section and reviews by the ethics committee.
- Community panel participation, and the share of panel comments that change a method or report.
- Publications, open datasets and public summaries released.
- Revenue by source, and the share of studies with no vendor funding for the evaluated product.
- Participant satisfaction and whether participants understood the consent process, collected anonymously.

How we review results. The Principal Research Scientist and the Executive Director review study progress every two weeks, the ethics committee reviews each study before it begins and at its end, and the board reviews results, finances and conflicts quarterly. Every number in this plan is a goal or an estimate, and we will report what really happens, including when we fall short.

Funding Request

What we are asking for, and how the money will be used. Our request: $120,000 from the Open Commons Research Foundation for the first public toolkit release, three community evaluation studies with partner organizations, the community research panel and the research ethics review, for July 2027 through June 2028.

The request is spread across these budget lines:
- Personnel: Research Engineer salary, $60,000 (other funds add $65,000).
- Other: Community research panel stipends (paid by check or payroll, never cash), $24,000.
- Other: Shared compute and licensed evaluation datasets, $26,000 (other funds add $24,000).
- Contractual: Research ethics committee review and legal review of research agreements, $10,000 (other funds add $8,000).

Other money in the project. Federal and state research contracts, public-benefit service fees, donations and memberships pay the other $97,000 of those lines, plus the principal research scientist, postdoctoral researcher, coordinator and ethics lead salaries, office rent and insurance in the budget.

Year 1 budget: about $997,000. Year 1 money coming in: grants $657,000 (this request plus other grants and awards we are applying for), federal and state research contracts $220,000 and public benefit licensing and service fees $60,000 and donations, memberships and corporate partners $60,000. Year 1 costs: Executive Director $130,000; Principal Research Scientist $160,000; two Research Engineers $250,000; Postdoctoral Researcher $72,000; Community Engagement and Ethics Lead $90,000; Research Coordinator $45,000; cloud computing and research compute about $60,000; dataset licensing and evaluation tools about $20,000; research participant and community panel stipends about $24,000; office rent and utilities about $30,000; conference, publication and open-source release costs about $12,000; security, compliance and data protection tooling about $14,000; insurance about $14,000; legal, ethics review and accounting about $24,000; travel and community convenings about $12,000; recruiting and training about $10,000; and about $10,200 a year for the wear on equipment.

How we planned the budget. Most of our Year 1 cost is pay, because careful evaluation is skilled, slow work. Compute and evaluation data cost about $80,000 in Year 1 and grow as we run more studies. Research contracts and service fees grow as partners and members join, and donations and memberships grow more slowly. We plan for a small surplus each year, which we hold as a reserve for the research program, not for distribution. Grant money is a planning estimate: the grant line in the plan is the balance needed after commercial or other income, and it is not a promise from any funder.

Research and products in the Year 1 plan depend on the success of the work, and if an award is delayed or not made, we will reduce hiring and scope before we reduce safety, testing or review.

About other funding. We are also applying to federal and state research programs, foundations focused on public-interest technology and university research partnerships. Our plan counts a research contract only at the amount in a signed agreement. These sources are competitive, are awarded for set periods and have strict rules about eligibility, reporting and intellectual property. The amounts in this plan are our planning estimates, not promises, and funders can change their budgets, rules and awards.

One-time startup costs. Startup costs of $78,000 are paid once from startup money, not from the yearly budget: $31,000 for equipment, $5,000 for leasehold improvements, $7,500 for deposits, $9,000 for opening supplies and inventory, $1,500 for licenses and permits, $14,000 for legal and professional help, $4,000 for signage, brochures and a website, $6,000 for background checks, training and first insurance.

How we got our startup number. The Funding Need Calculator in this plan adds the startup costs ($78,000), the cash we need to cover the wait for the first payments (about $29,800) and a cash cushion of about $241,800, which is about three months of Year 1 costs. That comes to $350,000. We start with $40,000 of our own cash and $320,000 in a founding donor gift and a bridge grant that the board has approved in writing, $360,000 in all, which covers that amount.

The next five years. Revenue grows from about $997,000 in Year 1 to about $1,436,000 in Year 5. Grant money is planned at $657,000, $535,000, $606,500, $500,500, $526,000 in Years 1 to 5, and the grant share of revenue falls from about 66% in Year 1 to about 37% in Year 5 as research contracts, service fees and memberships grow. Our staff grows from 7 in Year 1 to 10 in Year 5.

Savings. We plan to keep several months of costs in cash. The lowest year-end balance in the plan is about $299,200, at the end of Year 1. Customers and funders often pay after we invoice or submit reports, so we plan for about $59,820 owed to us at the end of Year 1.

Risks and how we will handle them:
- A study finds that a system performs poorly or unfairly. We report the finding fully and fairly, give the partner and vendor a chance to respond and publish the limits of the test.
- A study harms a participant or exposes personal information. We minimize data, use synthetic data where possible, follow an independent ethics review, maintain security controls and notify affected people and partners as the rules require.
- Partners withdraw or fund changes are made under pressure. We have written research agreements, a diversified funding base and a policy that no funder may change a finding.
- Staff or panel turnover slows the work. We document methods, cross-train staff and keep the panel and the partner network active.
- Limited demand for custom evaluations. We keep the toolkit free, build training around it and price custom work on a sliding scale.
- Government and grant funding is delayed or not awarded. We plan with a cash cushion, stage hiring so costs follow funding and apply to other funders.

If you are just starting. A new research institute can begin as a project inside a fiscal sponsor, run three partner studies with a small budget and apply for larger grants once it has published results. Strong partner results and a published toolkit make larger awards easier to win.

Appendix Notes

Documents to attach with this proposal:

- Your IRS determination letter (501(c)(3)), articles of organization and bylaws.
- Your most recent financial statements, Form 990 and current year budget.
- A list of board members, their roles and any conflicts of interest.
- Resumes for the Executive Director and Principal Research Scientist.
- Your research ethics committee charter and its membership list.
- Your working research agreement and data-use terms for a partner study.
- Letters of support from the public library system, school district, community health center and housing authority.
- Your open license, publication policy and conflict-of-interest policy.
- Your security policies, including data encryption and access controls.
- Proof of general liability, professional liability, cyber liability and directors and officers insurance, or quotes.
If you don't have any of this yet, no worries - let's continue.

The assumptions behind our numbers (change them in the Financials section to match your own):
- Grant Revenue: $657,000 in Year 1, then $535,000, $606,500, $500,500, $526,000.
- Federal and State Research Contracts: $220,000 in Year 1, growing to $500,000 in Year 5.
- Public Benefit Licensing and Service Fees: $60,000 in Year 1, growing to $290,000 in Year 5.
- Donations, Memberships and Corporate Partners: $60,000 in Year 1, growing to $120,000 in Year 5.
- Pay (a full year, per person, including payroll taxes and benefits): Executive Director $130,000 (1, 1, 1, 1, 1 people in Years 1 to 5); Principal Research Scientist $160,000 (1, 1, 1, 1, 1 people in Years 1 to 5); Research Engineer $125,000 (2, 2, 3, 3, 4 people in Years 1 to 5); Postdoctoral Researcher $72,000 (1, 1, 2, 2, 2 people in Years 1 to 5); Community Engagement and Ethics Lead $90,000 (1, 1, 1, 1, 1 people in Years 1 to 5); Research Coordinator (part-time) $45,000 (1, 1, 1, 1, 1 people in Years 1 to 5).
- Other yearly costs in Year 1: Cloud Computing and Research Compute $60,000, Dataset Licensing and Evaluation Tools $20,000, Research Participant and Community Panel Stipends $24,000, Office Rent and Utilities $30,000, Conference, Publication and Open-Source Release Costs $12,000, Security, Compliance and Data Protection Tooling $14,000, Insurance (General, Professional and Cyber) $14,000, Legal, Ethics Review and Accounting $24,000, Travel and Community Convenings $12,000, Recruiting and Training $10,000. The other costs grow between 4% and 25% a year.
- Equipment: $31,000 at the start, then $4,000 to $6,000 a year.
- Startup costs: $78,000, paid once from $360,000 in startup money.
- About 6% of each year's revenue is still owed to us at year end.
- Tax: none, as a 501(c)(3) public charity.
- Every dollar amount is an estimate. Replace them with your own quotes and real budget.

Startup Costs