← All posts

Every Marketplace Pays Out If Nobody Checks

Upwork releases the money after 14 days of silence, Fiverr after three, TaskRabbit charges the card within 24 hours, and Mechanical Turk approved by default after 30 days until Amazon closed it in September 2026. Proof of completion is mostly settled before any photo arrives: by the acceptance criteria you wrote, and by whether your agent is awake when the review clock runs out.

Finley Jones, co-founder and CCO/CMO, TaskPool International Ltd · @finjonesceo · Published 11 November 2026 · Platform rules checked against each platform's own help pages and documentation on 11 November 2026.

Every work marketplace whose rules I checked pays the worker if the buyer says nothing. That is the finding that should shape how an AI agent asks for proof that a real-world task was done. The agent settles the proof when it writes down what counts as done, well before any evidence lands, and that decision has a deadline, after which the platform decides for it.

The usual way to frame verification is as a detection problem: is this photo real, is this location spoofed, did the person actually go. Those questions matter, and we covered them in Detectors Score 18% on the Newest Image Generators. Forgery is one way proof fails. Another is duller: The task said "confirm the shop is open", the worker sent a photo of a door, and nobody can say whether that passes.

To reject a submission, an agent needs something specific to point to. If the acceptance criteria don't name the evidence, the agent has nothing to rule against, and if it doesn't rule in time, the money moves anyway.

How long does a buyer have to reject work before it pays out?

Here are the published defaults, from each platform's own pages. Amazon closed Mechanical Turk on 30 September 2026 (MTurk closure FAQ), so its row shows how the service worked until then, from Amazon's own API reference.

Platform What happens if the buyer does nothing Window
Upwork, fixed-price milestone "That's considered approval", and funds are released 14 days from submission
Upwork, hourly Hours are paid on Friday unless disputed 4 days from the Monday invoice
Fiverr Order completes automatically 3 days from delivery (14 for gigs that ship)
TaskRabbit Card on file is charged Within 24 hours of the invoice
Amazon Mechanical Turk, closed 30 September 2026 Assignment was approved and paid "unless the Requester explicitly rejects it" Configurable, 0 to 30 days; 30 if unset
TaskPool Submission auto-accepted and paid 7 days after the worker's completion deadline

Two details in those pages matter more than the windows.

The first is what happens after the clock. Fiverr says that once an order is complete, "its status cannot be changed or canceled via the Resolution Center." Upwork will mediate for 30 days after funds are released but says it "can't guarantee or enforce any refund or work" (Upwork Help). Once the window has passed, a complaint about quality depends on the other side's goodwill.

The second is that the platforms want short windows, and they say so. Amazon's own guidance told requesters to set the auto-approval delay "as low as possible so that workers don't have to wait" (MTurk HIT attributes, archived copy), and its getting-started example used three days, 259,200 seconds (AWS getting-started guide, archived copy). The timer protects the worker from a buyer who never answers, which is fair, but it also gives an agent's verification step a service-level target, whether or not anyone wrote one down.

Why proof of completion fails before the photo arrives

A lot of arguments about whether work was done are really about what was asked for.

In 2017, Meng-Han Wu and Alexander Quinn scraped 135 live task designs from Mechanical Turk and had an expert check them against requester best practice. Most followed most guidelines. The exception was the one this post is about: 77% didn't explain the criteria for accepting or rejecting the work (Wu and Quinn, HCOMP 2017). They were mostly decent instructions about how to do the job, with nothing about how it would be judged.

Workers notice. A year earlier, Cornell researchers coded 1,092 comments Turkers left on Amazon's participation agreement and found 437 about the risk of having work rejected. The causes they named were "poorly-designed tasks, unclear instructions, technical errors, and malicious Requesters" (McInnis et al., CHI 2016). The strategies Turkers described included avoiding new requesters and choosing low-risk tasks. A new agent posting its first task is a new requester in exactly that sense.

Agents have the same problem among themselves. Berkeley's MAST study annotated 200 execution traces from seven open-source multi-agent frameworks and sorted the failures into three groups. Specification issues, meaning flawed instructions or roles, made up 41.77%, and task verification failures, such as stopping early or checking badly, another 21.30% (MAST project page). Those were agents briefing agents, and a person in a car park reading a brief on a phone won't find it any easier.

What does a good acceptance criterion for a real-world task look like?

It is something a reviewer who has never seen the task could rule on in a minute, using only what was submitted. In practice that means five parts:

  • The observable. What must be visible or recorded: "the opening-hours sign, legible", not "confirm it's open".
  • The evidence type and count. "Two photos, at least one including the street number", or "a PDF receipt".
  • The window. When the evidence must have been captured: "between 12:00 and 14:00 on the task date".
  • The tolerance. What doesn't count against the worker: "a closed shop is a valid result if the sign is photographed".
  • The rejection reasons. The short list of things that will fail, written in advance.

Take the example in TaskPool's own agent docs, a task to photograph a café's specials board, with the submission spec "Upload 3+ sharp photos: full board, close-up of prices, shopfront" (agent docs). It names the evidence and the count, which plenty of briefs don't. It says nothing about what happens if the board has been wiped, if the café is shut, or if "sharp" is borderline. Each of those becomes an argument after the work is done, when the money is already committed and the clock is already running.

The tolerance line is the one people skip, and it is aimed squarely at unfair rejections. A worker who travels to a closed shop and photographs the sign has done the part you could check, and criteria that don't allow for that leave them unpaid for it.

What this doesn't prove

Longer criteria cost you workers. In the second half of the Wu and Quinn study, the version of the task with full accept and reject criteria ran to 199 words, against 89 for the control, and fewer workers who previewed it took it on (Wu and Quinn). The criteria didn't measurably improve accuracy either; the authors' lesson was that named tools and concrete examples did more for that. Each version was done by only 15 workers, so treat the result as a direction rather than a measurement. It still suggests criteria are mainly for the reviewer, and that they should be short.

The studies are about screen work. The model task in the Wu and Quinn experiment was a short exercise with a known right answer. A real-world errand has weather, opening hours and other people in it, and no clean answer key. That is a reason to write criteria down, though none of the numbers above were measured on physical tasks.

Some tasks can't be fully specified. "Tell me whether the venue feels safe for a client meeting" is a judgement, and the honest acceptance criterion is "a written answer with reasons and two photos". You're paying for an opinion, so what you can check is that one was given.

The auto-release timers exist so a buyer can't keep the work and never answer. An agent that needs longer should use whatever extension the platform offers (Fiverr lets buyers extend review by up to five days at a time, per its order guide), not count on going quiet.

How to write acceptance criteria for a real-world task

  1. Write the rejection reasons before the task, and nothing else counts. List the three or four things that will fail a submission. If a problem isn't on the list, accept the work and fix the list for next time. It takes about ten minutes per task type, and it gives every rejection a reason the worker can see coming.
  2. Add a tolerance line for the world not cooperating. Closed, sold out, cancelled, inaccessible: say which of those still earns the payout and what evidence proves it. It is cheap goodwill with people who can see your agent's public reputation.
  3. Keep the spec short enough to read on a phone. One line per criterion. Put background in the description, and keep the acceptance part to what a reviewer checks. Use an example of a passing submission rather than a paragraph about one.
  4. Treat the platform's review window as your deadline, and set your own at half of it. Trigger review on the submission event, not on a daily sweep. If the agent can't decide in time, page a person. On a 3-day Fiverr window or a 24-hour TaskRabbit charge, a weekly batch job is already too late.
  5. Alert on auto-accepts. Any payout that went through on a timer means your verification didn't run. Count them, and treat a non-zero number the way you'd treat a failed health check.
  6. Version your specs and keep the rejected ones. When a submission is rejected, store the criterion it failed. After twenty tasks you will know which line causes the arguments. Submissions that hit those lines are the ones to route to a person.

Where the review clock fits

Verifying real-world work is usually pictured as a model squinting at a photo, but in practice it looks more like scheduling: the criteria you wrote, the evidence they ask for, and a decision made before a timer you didn't set runs out. An agent that runs for hours without supervision, which is where duration starts to break agents, needs that decision to fire on an event, because if it waits to check later, the timer can release the money first.

TaskPool is built around that loop. A task carries a written submissionSpec stating "exactly what evidence counts as done", and opening it moves the payout plus a 10% fee into a held balance. A filed submission fires a submission.filed webhook, and the worker gets 60 seconds to fix an upload mistake before the agent should judge it. The agent accepts and pays, or rejects with a note the worker sees, and the Terms require a reason for every rejection; while a retry remains, the worker can fix and resubmit. A submission left in review 7 days past the worker's deadline is auto-accepted, and if a worker misses the deadline the slot expires and the money returns to the operator's balance. A final rejection can be disputed within 7 days and goes to a platform moderator, with escalation to an admin, handled through the signed-in console rather than the API key (agent docs; Terms, clauses 12 and 13). How the money is held is covered in Every Agent Payment Rail Assumes the Payee Is a Shop, and why a submission's text should never be treated as an instruction to your agent is in How TaskPool keeps people and AI agents safe.

Common questions

What are acceptance criteria for a real-world task? The written conditions a submission must meet to be paid, short enough that a reviewer who has never seen the task can apply them in about a minute.

What happens if I don't approve a freelancer's work in time? On most platforms the worker is paid automatically. Upwork releases fixed-price milestones after 14 days, Fiverr completes orders 3 days after delivery, TaskRabbit charges within 24 hours of the invoice, and Mechanical Turk approved after the requester's chosen delay, 30 days if none was set, until Amazon closed it on 30 September 2026.

How long should an AI agent take to review submitted work? Well inside the platform's window. Set your own deadline at about half of it.

What counts as proof of completion for a physical task? Whatever the acceptance criteria named before the task started, usually photos of a specific detail, a time window and sometimes a receipt or a location. For how to make that evidence hard to fake, see Detectors Score 18% on the Newest Image Generators.

Can an AI agent reject work fairly? It can apply written criteria consistently, which is what the Turkers in the Cornell study said was often missing. It should give a reason with every rejection, allow a resubmission, and leave disputes to a person rather than the agent that made the original call.

If your agent went offline for a week from today, work out how much it would pay out without checking anything. That is the amount your review step is there to protect.

← Back to the blog