Two numbers are circulating right now and they look like a contradiction. One says people identify machine-written text at roughly the rate of a coin flip. The other says a small panel of readers missed exactly one article out of 300. Both are real and the results are interesting. The studies are not measuring the same skill. They're measuring two skills.
Detection = can a reader tell that a text was produced by a machine rather than a person. Recognition = can a reader tell that a text carries the default output style of one particular model family. Nearly everything published as the first is actually the second. In other words: the good scores aren't evidence that humans got better at spotting machines; they're evidence that a specific vendor's default voice became a recognizable dialect and some readers learned it.
The timeline
2021. The baseline study puts trained annotators at 57.9% on GPT-2 output and 49.9% on GPT-3; the second figure is a coin flip with extra steps. Training the annotators (guidelines, worked examples, side-by-side comparisons) moved them to about 55%.
2022 through 2024. Chance, with variance by domain. Six students familiar with AI text averaged 59%. ESL teachers grading student essays hit 61%, improving to 67% after minimal self-training. On scientific abstracts, annotators drifted toward calling everything human-written. One study found people identified the exact sentence where a text switched from human to machine only 23.4% of the time.
2023 through 2025, the conversational track, which runs the other direction. In a public online Turing test, the best GPT-4 prompt passed 49.7% of games against a 66% human baseline. A year later, in a controlled two-party version, GPT-4 was judged human 54% of the time (humans: 67%). Then the three-party version: GPT-4.5, prompted to adopt a humanlike persona, was judged human 73% of the time, more often than the actual humans it was competing against. The same prompt on LLaMa-3.1-405B produced 56%. Unprompted GPT-4o scored 21%, below 1966's ELIZA at 23%.
Sit with that spread for a second, because it's useful. Same underlying models, same five minutes, same judges; the delta between 21% and 73% is a persona prompt. GPT-4o's formal, hedging, relentlessly balanced register was the tell. Not the reasoning, not the knowledge, the register.
January 2025, the high number. Annotators read 300 nonfiction articles produced by GPT-4o, Claude 3.5 Sonnet, and o1-pro. Annotators who rarely use LLMs performed poorly, as usual. But five annotators who use LLMs daily for writing missed one article out of 300 by majority vote, with no training and no feedback, and held up when the text was paraphrased and run through "humanizers." They beat almost every commercial detector tested. Worth reading the method: 70-plus annotators were screened and only the ones already scoring above 90% were promoted to "expert."
2026. The general population is where it was. A Scientific Reports study on research abstracts found individual accuracy ranging from 44% to 76% and concluded human judgment alone is insufficient. A study of 2,587 adults couldn't reliably separate ChatGPT short stories from published literary fiction, and rated the machine versions slightly higher. Being well-read predicted nothing. Familiarity with AI predicted everything: each point on the AI-literacy scale raised the odds of a correct guess by 33%, and the tells people named were em dashes and the "it's not just X, it's Y" construction.
Why the expert number is fragile
The experts are reading the coating and not running an assay. Pharma taught me this distinction and it transfers: you can identify a tablet by color, debossing, and shape, right up until the manufacturer changes the press. Vocabulary, punctuation habits, and rhetorical scaffolding are the coating. They are properties of a default configuration, and a default is the single easiest thing in the stack to change.
Three consequences follow, and my working assumption is that all three are in motion. The signal is model-specific (detectors trained on GPT output degrade on Claude and Gemini text). The signal decays as house styles shift (the em dash has already faded from a reliable tell to a folk belief). And the signal collapses on demand, because the persona prompt that moved a model from 21% to 73% is one sentence.
If you're writing policy on this, stop building it on detectors
The vendor accuracy claims run 99% to 100%, and they are almost all vendor-published. Independent benchmarking puts the leading tool near 85% average accuracy on RAID; one 2026 analysis found detectors catching 31.7% of GPT-5 output; seven tools falsely flagged 61.3% of TOEFL essays written by non-native English speakers as machine-generated. That last figure should end the conversation on its own. A control with a 61% false-positive rate against one population isn't a control.
The position: detection is a losing arms race and was never interesting. The interesting question is what you're actually trying to enforce. Provenance (where did this come from) is a signing and metadata problem. Accountability (who is answerable for this) is a policy problem. Quality (is this any good) is the problem we already had before any of this, and it is the only one where reading the text carefully still works.
Sources
- Clark et al., "All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Text," ACL 2021. GPT-2 57.9% / GPT-3 49.9%; training to ~55%.
- Köbis & Mossink (2021), incentivized detection of GPT-2 poetry; warnings reduced trust without improving accuracy.
- Jakesch et al., PNAS (2023), AI-generated self-presentations and flawed detection heuristics.
- Dugan et al., AI-transition-point identification (23.4% exact, 72.3% any AI sentence).
- Verma et al. (2024), Ghostbuster; six familiar annotators at 59%.
- Liu et al. (2023b), ESL teachers on student essays, 61% to 67%.
- Uchendu et al. (2021) and Sarvazyan et al. (2023), near-chance human baselines. Surveyed in "Detecting AI-Generated Text: Factors Influencing Detectability with Current Methods".
- Jones & Bergen, "Does GPT-4 Pass the Turing Test?" (49.7% vs. 66% human).
- Jones & Bergen, "People Cannot Distinguish GPT-4 from a Human in a Turing Test" (54% vs. 67%).
- Jones & Bergen, "Large Language Models Pass the Turing Test," PNAS (GPT-4.5 at 73%, LLaMa-3.1 at 56%, GPT-4o at 21%, ELIZA at 23%). See also the UC San Diego summary.
- Russell, Karpinska & Iyyer, "People Who Frequently Use ChatGPT for Writing Tasks Are Accurate and Robust Detectors of AI-Generated Text," ACL 2025 (1 of 300 misclassified).
- "Humans Can Learn to Detect AI-Generated Texts, or at Least Learn When They Can't" (feedback training; overconfidence without feedback).
- Scientific Reports (March 2026), young academics identifying ChatGPT-generated research abstracts, 44%–76% accuracy range.
- Weisberg et al., Judgment and Decision Making (August 2026), Villanova University; 2,587 adults, AI-literacy effect of +33% odds per point. Summary.
- International AI Safety Report 2026, on AI content becoming harder to distinguish.
- Liang et al., Patterns / Cell Press (2023), 61.3% false-positive rate on TOEFL essays by non-native speakers.
- RAID benchmark (ACL 2024, University of Pennsylvania) and 2026 detector comparisons, including the 31.7% GPT-5 recall figure.