AI Helped Kill MTurk. What Comes Next for Survey Research?

What the end of Mechanical Turk may tell us about AI, online surveys, and the increasingly blurry line between human and synthetic respondents.

Key Takeaways

• Amazon Mechanical Turk permanently closed on September 30, 2026.

• MTurk had problems well before generative AI, but AI increasingly undermined the basic premise of a platform built around humans doing things computers could not.

• Agentic AI creates a different challenge for online survey research because nonhuman respondents can increasingly navigate surveys and produce plausible responses.

• New detection methods are promising, but they can also incorrectly flag real people.

• At the same time researchers are trying to keep AI out of surveys, others are studying whether AI can intentionally serve as synthetic respondents.

Amazon Mechanical Turk permanently closed on September 30 after more than 20 years. Amazon Mechanical Turk closure FAQ

For behavioral and social science researchers, MTurk was many things, but one of its most important uses was simple: it gave researchers a relatively fast and inexpensive way to recruit people for studies and surveys.

Its closure feels oddly appropriate for this moment because I think AI helped kill MTurk.

AI is not the whole story. MTurk had significant problems before generative AI arrived. Researchers documented a substantial deterioration in data quality beginning in 2018, including more failed validity indicators, reduced reliability and validity, and failures to replicate established findings. Chmielewski and Kucker's MTurk data-quality study

But AI created a deeper problem for MTurk's original premise.

When Amazon launched Mechanical Turk in 2005, the company explicitly described it as “Artificial Artificial Intelligence.” The idea was to let computer programs call on humans to perform tasks that computers were not yet good at. Amazon's original 2005 Mechanical Turk announcement A recent AI & Society article even uses that original framing in its title: Artificial artificial intelligence: The case of Amazon's Mechanical Turk. Bloor's AI & Society article

Then computers got much better at the work humans had been needed for. And eventually something stranger happened: AI began showing up on the human side of the transaction too.

In 2023, Veselovsky and colleagues reran an abstract-summarization task on MTurk and estimated that 33% to 46% of workers in that particular task used large language models when completing it. The authors were appropriately cautious about generalizing beyond that task, but the irony is difficult to miss: a platform built to provide human intelligence when computers were insufficiently capable was increasingly operating in a world where the humans themselves could turn to AI. Veselovsky et al., Artificial Artificial Artificial Intelligence

The Survey Problem Is Changing

This matters beyond MTurk because online survey research has a related problem. Over roughly the past year, I have become increasingly reluctant to post open survey links on social media, particularly when there is an incentive attached.

Researchers have worried about bots and questionable respondents for years. We use CAPTCHAs, attention checks, completion-time rules, open-ended questions, and other techniques to identify suspicious responses. But many of those defenses assume that a bot will be worse than a human at completing the survey.

Agentic AI changes that assumption. An AI agent can potentially read instructions, navigate a website, remember earlier answers, interpret an open-ended question, and generate a plausible response.

Felipe Affonso recently tested this directly in the Journal of Consumer Research. He deployed 526 AI agents to complete an online survey. Conventional attention checks detected only 2.3% of them. His specially designed “cognitive traps” performed dramatically better, detecting 97.1% of the agents. Affonso, A Framework for Detecting AI Agents in Online Research

But there was a tradeoff. The cognitive traps also flagged 41 of 1,007 presumed-human respondents, or 4.1%. In other words, better agent detection can also mean excluding some real people. Affonso's 526 agent deployments were conducted in November 2025, another reminder that any detection method is being evaluated against technology that continues to change quickly.

That makes this look less like a solved problem than an arms race. A detection method may work because current models have particular limitations. As models and agent architectures change, researchers will have to keep testing whether those distinctions still hold.

So I am increasingly uncomfortable with a basic assumption online researchers have often been able to make: a completed survey is evidence that a person completed a survey.

And Then It Gets Stranger

At the same time researchers are trying to keep AI respondents out of studies, other researchers are intentionally putting them in.

There is now a growing literature on synthetic respondents: using language models to simulate people with particular demographics, backgrounds, attitudes, or other characteristics. Argyle and colleagues found that language models conditioned on demographic information could reproduce some patterns found in human attitude data, while Aher and colleagues showed that language-model simulations could reproduce findings from several classic human-subject experiments while also revealing systematic distortions. Argyle et al., Out of One, Many Aher et al., Using Large Language Models to Simulate Multiple Humans

Then, on September 30—the same day MTurk closed—Pew Research Center published a major investigation with a wonderfully direct title: Can AI Stand In for Human Survey-Takers? Not Really. Read Pew's full report

Pew created synthetic versions of real members of its American Trends Panel and compared their AI-generated responses with the answers provided by the actual people. In its main three-wave analysis, which used Claude Opus 4.6 with a low reasoning setting, the synthetic estimates differed from the human results by an average of 12.4 percentage points. Pew concluded that “at this time” AI models are not an adequate replacement for rigorous traditional polling.

There was another important problem: the results depended on which AI model was used. Pew also compared Claude Opus 4.6 with GPT-5.1 on a subset of the data and found that changing the model could produce what Pew itself called “large, systematic differences” between otherwise similarly constructed synthetic samples. Pew also emphasizes that this is a fast-evolving field. Pew's analysis of model differences

So Pew's answer is not “not yet.” It is “not really,” at least based on what they tested now. Whether future systems change that conclusion remains an empirical question.

What Does This Mean for Survey Researchers?

For now, I think online researchers need to take respondent provenance much more seriously. Where did participants come from? How were they recruited? What evidence do we have that they are human? Are we relying too heavily on a single CAPTCHA or attention check? What happens to our conclusions when questionable cases are excluded?

These questions are moving from methodological housekeeping toward core research design.

Survey methodology has traditionally asked whether we sampled the right people, whether respondents understood the questions, and whether they answered honestly. Increasingly, we may need to add another question very close to the top of that list:

Was the respondent actually a person?

And almost immediately, another question appears:

Did the respondent need to be?

That is the fascinating contradiction. In one study, researchers may work extremely hard to keep AI respondents out. In another, they may deliberately create thousands of synthetic respondents to see whether they can learn something useful without surveying people at all.

Mechanical Turk was created for a world in which human intelligence filled the gaps left by computers.

That distinction is no longer nearly as clean.

RIP, Mechanical Turk.

Sources and Further Reading

Amazon Web Services. (2005). Announcing Amazon Mechanical Turk. Original AWS announcement

Affonso, F. M. (2026). Brief commentary: A framework for detecting AI agents in online research. Journal of Consumer Research, 53(3), 619–633. https://doi.org/10.1093/jcr/ucag006

Aher, G. V., Arriaga, R. I., & Kalai, A. T. (2023). Using large language models to simulate multiple humans and replicate human subject studies. Proceedings of the 40th International Conference on Machine Learning, 202, 337–371. PMLR article

Argyle, L. P., Busby, E. C., Fulda, N., Gubler, J. R., Rytting, C., & Wingate, D. (2023). Out of one, many: Using language models to simulate human samples. Political Analysis, 31(3), 337–351. https://doi.org/10.1017/pan.2023.2

Bloor, S. (2026). Artificial artificial intelligence: The case of Amazon's Mechanical Turk. AI & Society, 41, 1207–1217. Springer article

Chapekis, A., Lau, A., Bestvater, S., Shah, S., Mercer, A., & Smith, A. (2026, September 30). Can AI stand in for human survey-takers? Not really. Pew Research Center. Pew Research Center report

Chmielewski, M., & Kucker, S. C. (2020). An MTurk crisis? Shifts in data quality and the impact on study results. Social Psychological and Personality Science, 11(4), 464–473. https://doi.org/10.1177/1948550619875149

Veselovsky, V., Ribeiro, M. H., & West, R. (2023). Artificial artificial artificial intelligence: Crowd workers widely use large language models for text production tasks. arXiv. https://doi.org/10.48550/arXiv.2306.07899

Amazon Mechanical Turk. Closure FAQs. MTurk closure information

AI Disclosure

Generative AI was used to assist with drafting and editing this post and to create the accompanying image. I reviewed and revised the content and take responsibility for its accuracy and final form.