OpenAI, according to research by 404 Media, allows hundreds of external contractors to read real ChatGPT conversations and evaluate the chatbot’s responses. The so-called prompt reviewers sometimes see personal details from the conversations of more than 900 million weekly users. They are paid through the training platform Mercor with more than 50 dollars per hour.
Reviewers are to eliminate flattery and clichés from ChatGPT
The external contractors work in three clearly separated steps. They read an anonymized excerpt of a real conversation and summarize what the inquiring person wanted. They then evaluate several response variants generated by ChatGPT on a scale from one to seven. The reviewers pay attention to exaggerated agreement, unnecessary emojis, and contrived “AI language” – ChatGPT is supposed to sound more reserved and professional.
One participant said that users would “hardly imagine that somewhere a contractor is analyzing their conversations.” This shows how unknown the process is so far. The reviewers are recruited through the mediation platform Crossing Hurdles and paid through Mercor. OpenAI itself conducts the project internally under the code name “Lily” and has not publicly commented on it so far.
The program has existed for several months, but internally it is largely kept under wraps. Internal training documents provided to the editorial team of 404 Media show concrete example evaluations. Exuberant exclamations and reflexive praise phrases are considered unwanted clichés there and are to disappear from the responses.
Data protection filter has gaps, business accounts remain excluded
OpenAI stated to 404 Media that conversations undergo an automatic data protection filter before the review, which is supposed to remove usernames and other identifiers. However, the company admitted that sensitive details such as health or financial information could occasionally slip through. After the publication of the report, OpenAI adjusted its own help page without explicitly mentioning human reviewers.
Those who do not want contractors to read along can deactivate the option “Improve model for everyone” in the data controls. In the Free, Plus, and Pro tiers, it is activated by default. The objection does not have retroactive effect: conversations already conducted remain available for evaluation, only new chats are excluded. For ChatGPT Business, ChatGPT Enterprise, and the API, OpenAI claims not to train with customer data by default anyway – companies would have to agree to this separately.
OpenAI is not alone in human chat review: Anthropic also confirmed to 404 Media that it uses similar reviews under certain account settings, as does Google with Gemini. Just in August, researchers showed that encrypted reasoning logs from OpenAI, Anthropic, and Google can be read through indirect means. This shows how many attack surfaces AI conversation data now offers.
It will be crucial whether OpenAI communicates more transparently after the publication of the report that humans can read along. So far, the company has only retroactively adjusted a help page without explicitly naming human reviewers. A direct answer to the question of where exactly users are informed about this was not provided. For all those who use ChatGPT professionally, it is worth taking a look at their own data controls in the meantime.
