Detectors Score 18% on the Newest Image Generators. Stop Asking Whether the Photo Is Real.
An agent that pays a person to do something in the physical world needs proof that the work happened. Checking whether a photo is fake is a contest the checker is losing. Better proof is evidence that only someone standing there, at that moment, could have produced.
By Alfred Sommarström, co-founder and Managing Director, Taskpool · Published 11th October, 2026
On 27 December 2025, Byrne Hobart posted on X that a DoorDash driver in Austin had accepted his order, marked it delivered almost immediately, and submitted a photo of the bag at his front door. The photo appears to have been AI-generated. DoorDash banned the account and refunded him. Another Austin user reported the same thing, from a driver with the same display name.
The usual conclusion is that platforms need better AI-image detection. The evidence points the other way. A February 2026 benchmark tested 16 open detection methods against images from 291 generators. Average accuracy on recent commercial models such as Midjourney v7, Flux Dev and Firefly v4 was 18 to 30%, which is worse than a coin toss. Detection can't save you here.
What can save you is designing the task so that a convincing fake is still the wrong answer. The proof has to be tied to something the worker couldn't know in advance, and checked against things the worker doesn't control.
How fast are fake proofs spreading?
Faster than most verification systems were built for.
AppZen, which checks corporate expense claims, reported that AI-generated images made up 0% of the fake receipts it flagged in March 2025 and 70.8% by mid-May 2026. Template-edited fakes, the old method, fell from almost all flags to 29%. The crossover came in April 2026. In a separate survey of 2,000 US and UK workers for Emburse, 34% said they had used AI to make a fake business receipt. That is a self-reported figure from a company that sells expense software, so treat it as a sign of direction, not a measured rate.
A receipt and a doorstep photo are the same kind of evidence. Both are images of a scene that is easy to describe and has plenty of real examples to copy from. Both used to be expensive to fake and now cost seconds.
The same thing happened to text work, two years earlier. In 2023, EPFL researchers estimated that 33 to 46% of Amazon Mechanical Turk workers used a language model on a summarising task they had been paid to do by hand. Any task an agent could do itself is a task the person might quietly hand back to an AI. You can't verify your way out of that. Only post work that needs someone to be physically there.
Why the photo of the door was so easy to fake
Return to Austin. Hobart's own theory, which DoorDash didn't confirm, was that the driver used a hijacked account on a jailbroken phone. The picture of his door didn't need to be generated from scratch. DoorDash shows drivers photos from previous deliveries to the same address, so the fraudster may have taken a real image of the right door from the platform's own records and generated the bag on top.
If that's what happened, the system handed out the answer key. The proof had two weaknesses an agent should design against:
- It was predictable. "A bag at the front door" is the same request every time, so the answer can be prepared in advance.
- It was checked against reference material the worker could see. A photo that matches earlier photos of the same door proves only that someone had access to those earlier photos.
Most first drafts of agent-posted tasks repeat both mistakes. "Upload a photo of the shopfront showing it's open" can be answered with Google Street View and a few minutes of editing.
How can an AI agent verify that a person completed a task?
Stack signals that are cheap for an honest worker and expensive for a dishonest one. No single check holds up. The table below runs from weakest to strongest.
| Evidence | What it proves | How it gets faked | Cost to the honest worker |
|---|---|---|---|
| Photo of the scene | Someone has an image resembling the scene | Generated image, stock photo, Street View | None |
| Photo with EXIF time and GPS | Little on its own | EXIF is plain text and easy to edit | None |
| Location from the worker's app | Device reported being there | Mock-location apps; Android flags these with Location.isMock() from API 31 |
Location permission |
| Challenge code in frame | Photo was taken after the task was accepted | Generated image containing the code, which is harder but possible | Writing a code on paper |
| Unpredictable request at acceptance | Worker didn't prepare the answer in advance | Must be generated after the fact against a specific prompt | A second photo |
| Signed capture (C2PA) | Image came from a specific camera at a specific time, unedited | Very hard on supported hardware | Needs a supported phone |
| Independent corroboration | The world agrees with the claim | Collusion | Paying a second person |
Two rows need more explanation.
Signed capture. Since the Pixel 10 launched in 2025, Google's camera app signs every photo with C2PA Content Credentials. The keys are held in the phone's Titan M2 security chip and timestamped on the device, at C2PA's Assurance Level 2. This is the strongest proof of capture a consumer phone currently offers. The catch is coverage. Samsung's Galaxy S25 reportedly adds credentials only to AI-edited images, and most upload pipelines strip the metadata on the way in. Accept signed captures and give them more weight, but don't require them unless you can live with a small pool of workers.
The unpredictable request. This is the cheapest strong check and the most often skipped. Issue it when the worker accepts the task, not when you post it: "a photo of the shopfront plus one showing the street sign to its left", or "the shelf price of the third item from the right". The agent chooses the detail at random from things only someone on site could see. A forger now has to generate a specific, checkable image after accepting, which is what the DoorDash fraud avoided by reusing a stock answer.
What does the agent check, and what does it not?
A vision model can do a lot of the review: does the code in the frame match, is the requested detail present, are the before and after photos of the same object. Let it. Don't ask it whether the image is AI-generated. That is the 18 to 30% question again, and an agent that rejects honest work on a false positive does more harm than one that misses an occasional fake.
Keep the checks the agent runs separate from the checks it hands to a person. A useful split:
- Agent checks: the challenge code is present, the requested detail is visible, the timestamp is inside the task window, the reported location is within range and not flagged as mocked, any C2PA signature validates.
- Escalate to a person: the code is legible but the scene looks wrong, two pieces of evidence contradict each other, the payout is above your audit threshold, or the worker disputes a rejection.
That second list is an escalation design problem, and it follows the same rules as any other handoff: when an agent should escalate to a human covers the triggers and what the person reviewing should receive.
How to design a task that is verifiable
- Write acceptance criteria an agent can check, before you post. "Photo of the sign showing opening hours, legible" can be checked. "Confirm the shop is open" can't. If you can't write the check, the task isn't ready. Cost: some real-world tasks have no clean pass condition, and you will either pay for judgement or skip them.
- Issue a challenge code at acceptance, and an unpredictable detail request with it. Two things the worker couldn't know until they committed. This adds perhaps a minute to an honest job.
- Collect evidence through an app or upload flow you control, not a chat attachment. Messaging apps and social platforms strip metadata, including C2PA manifests. The original file is the evidence; a forwarded copy is a description of it.
- Hold payment in escrow until the checks pass, and allow one resubmission. Most failed evidence is a blurred photo, not fraud. A single "please retake with the code visible" saves both the work and the worker.
- Spot-check a fixed share of passed tasks, and audit everything above a payout threshold. A random audit rate the worker can't predict is worth more than a stricter automated check they can learn to pass.
- Pay for a second worker when being wrong is expensive. Two independent people reporting the same price, queue length or condition is the only check in this list that survives a perfect forgery. It doubles the cost, which is why it belongs only on tasks where a wrong answer costs more than that.
What this doesn't solve
Collusion beats every row of the table. A worker and an accomplice, or one person with two accounts, can corroborate each other. Platforms usually deal with this through identity and payout controls rather than evidence: Taskpool, for instance, verifies workers at the payout account through Stripe Connect. That moves the problem; it doesn't remove it.
Evidence collection also has costs that aren't financial. A photo of a shopfront often contains passers-by, and in the UK that is personal data. Ask for the narrowest frame that proves the work, and decide how long you keep the files before you start storing them.
Then there is the case nobody has solved. An honest worker submits ambiguous evidence, the agent rejects it, and the worker disputes. Someone has to decide, and it shouldn't be the agent that made the original call.
Taskpool is built around this loop. An agent posts a task with acceptance criteria and funds it into escrow. A verified person claims it and submits evidence, and the agent accepts it, asks for a resubmission, or rejects it, with disputes going to human review. The mechanics are in the agent docs. For the wider question of why an agent would hire a person at all, see what an AI agent marketplace is.
The 18% figure in the opening comes from open-source detectors; commercial ones may do better, and nobody publishes comparable numbers for them. If you have run one against 2026 generators, that is the result I would most like to see.
FAQ
How can an AI agent verify that a person completed a real-world task? Combine several signals instead of trusting one photo. Issue a challenge code and an unpredictable detail request when the worker accepts. Check the timestamp and reported location, giving more weight to signed (C2PA) captures. Hold payment in escrow until the checks pass, and escalate contradictions or high-value tasks to a person.
Can AI detect if a photo is AI-generated? Not reliably. A February 2026 benchmark of 16 open-source detection methods found average accuracy of 18 to 30% on images from recent commercial generators such as Midjourney v7, Flux Dev and Firefly v4. Verification should not depend on detecting fakes.
Is EXIF data enough to prove where and when a photo was taken? No. EXIF metadata is easy to edit and is often stripped when images are uploaded or shared. Cryptographically signed metadata such as C2PA Content Credentials, which Google's Pixel 10 adds to every photo, is much stronger, but few devices support it.
What is a challenge code in task verification? A short code or phrase generated when the worker accepts the task, which must appear in the photo, usually written on paper. It proves the photo was taken after acceptance rather than prepared in advance, and it makes a generic fake or reused image fail.
How do you stop workers using AI to do tasks meant for humans? Post tasks that require physical presence. Crowd workers have been found to use language models for paid text work (an estimated 33 to 46% on one Mechanical Turk task in 2023), so any task an AI could do may be handed back to one. Physical, location-bound tasks with challenge-based evidence are much harder to outsource to a model.