Microsoft's contractors read Copilot prompts and see the photos people upload

When you ask Copilot to edit a photo, the picture does not stop at the model. According to internal documents and reviewer accounts obtained by 404 Media, hundreds of contractors are paid to look at what people send Microsoft's assistant: the original prompt, the image the user uploaded, and two versions Copilot produced from it. Their job is to say which edit is better.

The reviewers describe a queue that is heavily sexual. One told 404 Media that "faces are always uncensored, and many of the prompts are sexual in nature and dubiously consensual". Another asked the obvious question out loud: "Who is writing these prompts and who is deciding that basically generating porn is what Copilot is now focused on?"

What the reviewer actually sees

The task is mundane by design. A worker is shown the user's request, the source picture and two candidate results, and has to pick the better one: does it follow the instruction, does it leave untouched parts untouched, are there artefacts. The instruction quoted in the documents is as casual as it sounds: "Trust your intuition - when you glance at the two edited images side by side, which one immediately feels like the better edit?"

What passes in front of that intuition is the part people did not picture. Reviewers describe sexualised edits of real women, upskirt photographs, pro-anorexia material and fetish imagery built around cartoon characters. One said simply: "I recoiled." The recruiting goes through Prolific, which advertises "human feedback from representative populations" for preference tuning and safety evaluations; the company did not answer 404 Media's questions.

Microsoft's response was a single sentence: "Microsoft uses customer data as described in our terms of use, including to improve our products and enforce our code of conduct." That is not a denial. It is a pointer to the paperwork, and the paperwork says the same thing.

All of this is already written down

The uncomfortable detail is that none of this was hidden. Microsoft's own privacy FAQ for Copilot states that "some Copilot conversations are subject to both automated and human review for product improvement and digital safety purposes", and that "limited human review is required as part of the investigation process when a violation of the Code of Conduct is suspected". The same page puts default retention of conversation activity at 18 months, and says an uploaded file is stored "no longer than 18 months" before automatic deletion.

So the story is not that Microsoft broke a promise. It is that the promise most people think they have, and the one they actually have, are different documents. Nobody reads a privacy FAQ before dropping a holiday photo into a chat window, and the gap between "an AI is processing this" and "a stranger on a crowdsourcing platform will look at this" is where the surprise lives. We went through the same gap for other assistants in our practical guide to using AI assistants safely.

Why the training switch does not help here

Microsoft does give consumers a toggle. On copilot.com it sits under your profile, then Privacy, then Training on conversation activity; in the Windows and macOS apps it is under Settings, Privacy, the same entry. It is worth turning off if you do not want your chats feeding model training, and it is worth knowing exactly how narrow it is.

Microsoft's own help page spells out the limit: opting out "will exclude your future conversation activities from being used for training these AI models", but "will not exclude your conversations from being used for other general product or system improvements nor from use for advertising, digital safety, security, and compliance purposes". Quality review of image edits sits squarely in "product improvement". Switching off training does not switch off the human in the queue.

There is one narrower carve-out worth knowing: images captured with Copilot Vision are, per the FAQ, "processed only to respond to your request and aren't used to train AI models or personalize your experience". That is a different pipeline from uploading a file and asking for an edit, which is the one the reviewers were working in.

What to do with this in practice

The honest rule is the old one, restated for a new interface: treat anything you upload to a consumer AI assistant as something a stranger may read. Not because a breach might happen, but because review by people is part of how the product is built and policed. That does not make the assistant useless; it makes some inputs a bad idea.

• Assume a human reviewer may see the prompt and the picture together, faces included
• Do not upload other people's photos, documents, medical images or anything that identifies a third party who did not agree to this
• Turn the training toggle off if you want, but do not treat it as a promise of privacy: it excludes model training, not product review
• Remember the default retention: conversation activity and uploaded files stay up to 18 months
• For anything sensitive, edit the image locally with an offline tool instead of sending it to a cloud assistant
• At work, check whether you are in the enterprise tenant or signed in with a personal account, because the rules differ

One more thing worth saying plainly, because this site is about network privacy and it would be easy to sell the wrong answer here: a VPN changes nothing in this story. It hides which network your connection comes from, and the data in question is what you deliberately hand over inside your own account. The line that matters is not where the traffic goes, it is what you choose to put into the box. That distinction also runs through the prosecutions of sexualised image sites, such as when Manhattan prosecutors seized twelve deepfake porn services: the harm followed the images, not the connection.