Home - Human Feedback Foundation

Mission

There’s a race for human feedback. When AI became a new paradigm for modern software, human feedback has emerged as an essential element of the AI tech stack. We are here to build public input into AI models used in critical domains like healthcare, and act as an independent, third-party custodian to create a global database of human feedback – for there to be an authoritative and democratic source of human feedback data for AI builders everywhere.

Ways We Work

Human Input

Advance the integration of public input into AI through working with the nonprofit sector, technology partners, and human-centric open-source AI projects.

Open Source AI

Mentor the next generation of open source AI contributors, focusing on working with human feedback datasets and inputs across the model training and development stack to accelerate the development of resilient and better aligned open-source AI systems.

Education

Educate newcomers to the ML field, and larger business, policy, and public institutions on the advancements in cutting-edge AI safety research and open science, making AI more accessible to all.

Work Streams

Democratic
input to AI

We work with civic groups, technology partners, academia, and the open-source AI community to support and advance the use of AI in the public interest.

Data Trust

We work with our partners to create an authoritative and democratic global database of human feedback, by generating and curating relevant datasets, and providing front-end tools to AI companies to gather users’ feedback about LLM output.

AI Education x Community

We actively educate newcomers to the ML field, making AI more accessible to all. We have built an engaged community through our LLM Reading Group and AI Tinkerers meetups, and work closely with the nonprofit sector to bring more nonprofits into the age of AI.

Paper Club by Human Feedback

Our work wouldn’t be the same without Paper Club by Human Feedback Foundation x AI Tinkerers. Grateful to the paper authors who come to present and discuss their research with thousands of AI/ML builders. We’re excited to feature more amazing research in 2025. Please add your name to the Paper Club list to be notified of our future sessions. In 2024, we’re proud to have shared with the world:

Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning, Marzieh Fadaee, Ahmet Ustun – Cohere For AI
Consent in Crisis: The Rapid Decline of the AI Data Commons, Shayne Longpre and team – MIT
Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data, Rylan Schaeffer, Matthias Gerstgrasser – Stanford University
Learning from Naturally Occurring Feedback, Shachar Don-Yehiya, Leshem Choshen, and Omri Abend – Hebrew University of Jerusalem, IBM, MIT
Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback, Stephen Casper – MIT CSAIL
Preference Learning Algorithms Do Not Learn Preference Rankings, Sadhika Malladi – Princeton University
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, Patrick Lewis – Cohere
The Curse of Recursion: Training On Generated Data Makes Models Forget, Ilia Shumailov – University of Oxford
The Future of Human Feedback, Shachar Don-Yehiya, Leshem Choshen, and Omri Abend – Hebrew University of Jerusalem, IBM, MIT, Cohere, etc.
Theory of Mind May Have Spontaneously Emerged in Large Language Models, Michal Kosinski – Stanford University
Training language models to follow instructions with human feedback, Long Ouyang – OpenAI

Community

Read: Introduction to Human Feedback Foundation, currently the best general overview of the Human Feedback Foundation and our work streams.
Join an active 1,500+ members community through an LLM Reading Group where we host scholars from Google DeepMind, McGill University, Yale University, Stanford University, University of Toronto, and Cohere For AI.
Join the local 2,000+ AI Tinkerers community and attend our meetups for AI builders and entrepreneurs, hosted and supported by Shopify, Shopify Ventures, Accenture, Drive Capital, and other key partners.
Catch up on OpenAI: Democratic Input to AI, Team submission and white paper — a deep dive into some of the team’s thinking about the future of democracy and governance.
Let us be your resource. Ask us to speak at your event! We’re grateful to MaRS, Data Universe, Business Council of Canada, FITC, and others who have hosted our speakers. Please reach out to hello [at] humanfeedback.io.