RLHF Companies

The vendors that supply human preference data, rubric scoring and expert feedback for model training, compared on what they publish.

RLHFreinforcement learning from human feedback
People compare or score model outputs, a reward model learns their judgments, and the model is tuned to score well against it. The human work is the comparisons.
RLAIFreinforcement learning from AI feedback
A model applies written principles to produce the preference labels instead of people. Humans still write the principles and check the model's labels against their own.
DPOdirect preference optimization
Trains the model straight from pairs of a chosen and a rejected response, with no separate reward model. It needs the same pairwise data that RLHF does.
Preference datanoun
Records of which of two or more responses a rater preferred for a prompt, often with a strength score and a reason. It is the raw material for reward models and DPO.
Rubricnoun
The written guide raters score against: criteria in priority order, a defined scale and worked examples. A vague rubric is the usual reason two careful raters disagree.
Get a quote

Who supplies RLHF and human feedback data?

15 vendors that sell preference data, post-training data or expert feedback work, grouped by how they deliver it. Most quote on request. The full vendor table adds what each says it supplies, with a source for every line.

Last verified November 7, 2026

RLHF and human feedback vendors, from their own pages, read November 7, 2026
VendorDeliveryPublished pricing
Surge AIManaged serviceNot published
TolokaManaged serviceNot published
MercorManaged serviceNot published
micro1Managed serviceNot published
TuringManaged serviceNot published
TELUS DigitalManaged serviceNot published
Handshake AIManaged serviceNothing published for buyers; fellows are told pay differs from project to project.
AppenManaged serviceNot published
InnodataManaged serviceNot published
Scale AIPlatform plus servicesNot published
LabelboxPlatform plus servicesMetered in Labelbox Units: 20 per data row on Live Multi Modal or LLM projects, and 500 free each month on free accounts. No dollar value per unit is listed.
Kili TechnologyPlatform plus servicesA $0 trial (2 to 5 seats, 100-asset cap); paid Grow and Enterprise plans are quoted.
SuperAnnotatePlatform plus servicesThree tiers (Starter, Pro, Enterprise), all without listed prices.
CentificPlatform plus servicesNot published
ProlificSelf-serve marketplaceParticipants: $8 an hour floor, $12 suggested. Prolific's own fee is extra and its rate is not given in that article.

A vendor in this table can pay RLHF Companies when an introduction turns into a contract. Who appears, the grouping by delivery model and every description are not for sale.

Sources and dates

What does RLHF data cost?

Human feedback is priced on time, so the first number to pin down is how many minutes one task takes.

Vendors charge per task, per rater hour, through platform fees with labor on top, or for a fixed statement of work. Few publish rates. The public floor comes from marketplaces: Prolific sets participant pay at a minimum of $8 an hour and recommends $12, with its platform fee on top. Expert raters, overlap between raters and review layers multiply from there.

Two thousand comparisons at six minutes each, with a fifth of them sent to a second rater, is 240 rater hours before anyone has reviewed a sample. The pricing models page has a calculator for your own numbers.

Sources and dates

How do you brief an RLHF vendor?

Write the brief before the first call. Vendors quote on the same handful of facts, and briefs that leave them out get quotes that cannot be compared.

  1. Name the judgment

    Pairwise choice, ranking, rubric score, written demonstration or red teaming, and the criteria in priority order.

  2. Say who can make it

    Generalists for tone and instruction following; people who can verify the answer for code, math, law or science.

  3. Set the quality bar

    Agreement on shared items, accuracy on your own gold tasks, and what happens to work that falls short.

  4. Size a paid pilot

    Several hundred to around two thousand items on your real task, timed, before any production volume.

The brief builder turns these choices into text you can send. For the reasoning behind rubrics, see how to write rubrics for a human feedback project.

How is the quality of human feedback checked?

Give some items to two raters and measure how often they agree, corrected for chance. Cohen's kappa is the standard figure for two raters:

κ = (po − pe) / (1 − pe)

Preference judgments agree less than object labels do. In OpenAI's InstructGPT work, trained labelers agreed with each other about 73% of the time. Alongside agreement, ask for accuracy on gold tasks you write, a reviewer's overturn rate on a random sample, and a rubric version on every record. The quality controls page lists what to request each week.

Sources and dates

Questions to ask before you sign

  • What do raters on a project like ours earn, and is pay per task or per hour?
  • How are raters qualified for our subject, and what share pass?
  • What agreement did your last comparable pilot reach, by criterion?
  • What share of items go to a second rater, and who reviews disagreements?
  • Can we supply our own gold tasks, hidden in the normal stream?
  • Does every delivered record carry a rater ID and rubric version?
  • Are raters allowed to use AI tools, and how is that checked?
  • Who owns the data, and can you reuse prompts or responses for other clients?

Questions about RLHF companies

What do RLHF companies do?

They supply the human judgments used to train and tune language models: pairwise preferences, rubric scores, written demonstrations, red teaming and expert review. Most run the rater workforce for you; some sell the platform and let you bring raters.

Who are the main RLHF vendors?

Vendors whose own pages describe this work include Scale AI, Surge AI, Toloka, Mercor, micro1, Turing, TELUS Digital, Handshake AI, Centific, Appen and Innodata, plus platforms such as Labelbox, Kili Technology and SuperAnnotate and the Prolific marketplace. The vendor table sets out what each publishes.

How much does RLHF data cost?

Managed vendors quote after scoping. The public floor is marketplace participant pay: Prolific's minimum is $8 an hour and it recommends $12, plus a platform fee. Expert raters, overlap and review raise the price. See pricing models.

Is RLAIF cheaper than RLHF?

It needs fewer human labels, but the remaining work (writing principles, building a human-labeled check set and auditing the AI judge) falls to senior, expensive people. The total is often smaller; the hourly rate is not.

How do I compare RLHF vendors fairly?

Send each the same written brief, ask for a paid pilot on your own task, and compare agreement on shared items, accuracy on your gold tasks, time per task and price per accepted item. Our brief builder writes the brief.

Do you see or handle our data?

No. We are an introduction service. The form asks for a description of the work, never prompts, outputs or data, and samples go to a vendor only under an agreement you sign with them.

Get a quote

Sketch the feedback work in about a minute. What comes back is one vendor name, not a round of cold calls.

RLHF Companies introduces teams to vendors, and the vendor can pay us for that introduction. Sending a brief costs your team nothing.

Keep prompts, model outputs, datasets and samples out of this form; describe the work in words instead.

Part 1 of 3

Your company's location

For a team outside the US, use the ZIP code of your US office or entity.

Get a quote