There is a category error at the centre of this entire topic, and almost no page ranking for the term names it. AI detectors do not detect AI. No detector examines a document for a watermark, a signature, or any trace left behind by a language model. There is nothing to find — generated text is ordinary text.
What detectors do is measure statistical properties of writing and estimate how likely a language model would have been to produce it. That estimate is then presented as a percentage, which users read as a verdict. Understanding the gap between those two things explains every failure mode that follows.
The Mechanism: Perplexity and Burstiness
Nearly every detector rests on two measures:
- Perplexity — how surprised a language model is by each successive word. Low perplexity means the text follows the most probable path. Language models are optimised to produce probable text, so their output tends to score low.
- Burstiness — the variance in sentence length and structure across a passage. Human writing tends to be uneven: a long complex sentence, then a short one. Generated text tends toward uniformity.
A detector combines these into a probability. Some add a classifier trained on labelled human and AI samples, which shifts the problem rather than solving it: the classifier learns the characteristics of the training data, and generalises poorly to models, prompts, or writing styles it has not seen.
Why This Design Fails — And Who It Fails
The core problem is that low perplexity and low burstiness are not properties unique to machines. They are properties of clear, conventional, formulaic writing — which humans produce constantly, and which some humans produce far more than others.
Non-Native English Speakers
This is the most serious and best-documented failure. Writers working in a second language tend to use a narrower vocabulary, more conventional sentence construction, and more predictable phrasing — precisely the profile detectors read as machine-generated. Published research on this bias has found detectors flagging large proportions of essays written by non-native speakers as AI-generated while classifying comparable native-speaker essays correctly. The discrimination is structural, not incidental: it follows directly from what the tools measure.
Neurodivergent Writers and Formulaic Genres
Writers who use consistent structural patterns — a common feature of some autistic writing styles — trip the same signal. So does any genre that rewards uniformity: technical documentation, legal drafting, lab reports, and the rigidly structured five-paragraph essay that schools explicitly teach and then flag as suspicious.
Heavily Edited Human Writing
Grammar and style assistants push text toward conventional, low-perplexity phrasing. Using a mainstream writing assistant on your own work can raise its AI score. Several documented false-accusation cases trace to exactly this.
The False Positive Arithmetic
Vendors advertise accuracy figures around 98–99% and false positive rates around 1%. Even taking those at face value — and independent testing generally finds them optimistic — the numbers do not mean what people assume.
A 1% false positive rate applied to a 500-student course generating 10 assignments each produces 5,000 documents and roughly 50 false accusations per term. Scaled to an institution, it is thousands. Each one is a named individual facing an academic integrity process, with the burden of proof effectively inverted.
Two further problems compound this:
- Accuracy is measured against the wrong distribution. Benchmarks typically use unedited output from a specific model version. Real submissions are edited, paraphrased, hybrid human-AI, and produced by models the detector was never tuned against. Performance degrades sharply outside the benchmark.
- Short texts are close to unusable. Statistical measures need volume. Under roughly 300 words, detector outputs approach noise, yet they still return confident-looking percentages.
The Vendor Retreat
The most telling evidence is what the industry itself has done. OpenAI — which had the strongest possible commercial motive for a working detector, and unmatched access to the model — released an AI Text Classifier in early 2023 and withdrew it within six months, citing low accuracy. That withdrawal is the single most important fact in this topic and is routinely omitted from articles recommending detectors.
Major detection vendors have correspondingly softened their language, repositioning outputs as "indicators" requiring human review rather than determinations. Several institutions have disabled AI detection in their learning management systems outright, citing bias and unreliability. Threads on r/Professors show the split among educators clearly: some treat scores as actionable, while others have abandoned them entirely after being burned by a false accusation, and increasingly rely on process evidence — draft history, in-class writing, oral defence — instead.
Watermarking: The Approach That Would Work, and Why It Doesn't
Statistical watermarking embeds a detectable pattern at generation time by subtly biasing token selection. Unlike post-hoc detection, this is real evidence rather than inference. It fails in practice for structural reasons:
- Paraphrasing, translation round-trips, and moderate editing destroy the signal.
- It only works for models that implement it. Open-weight models can simply have it removed.
- It requires industry-wide coordination that has not happened and is not obviously in any vendor's commercial interest.
The general reliability problems of language models are relevant context here too — see our explainer on why AI models hallucinate answers.
If You've Been Falsely Accused
- Ask what the detector score actually is. A percentage is a probability estimate, not a finding of fact. Ask which tool, which threshold, and what its documented false positive rate is.
- Produce process evidence. Version history in a cloud document, draft files with timestamps, research notes, browser history, handwritten planning. This is far stronger evidence than any detector score in either direction.
- Ask for a second sample. Offer to write comparable material under supervision.
- Cite the known bias. If you are a non-native English speaker or write in a structurally consistent style, the documented bias against those profiles is directly relevant and institutions are increasingly obliged to consider it.
- Point to OpenAI's withdrawal. The company best placed to build a working detector concluded it could not.
- Use your institution's appeals process. Many policies now explicitly bar detector output as sole evidence.
Preventatively, the most effective habit is simply writing in a platform that retains version history. A visible drafting trail settles the question in a way no detector can.
What Actually Works Instead
Institutions that have handled this well moved away from detection entirely, toward assessment design that makes the question less decisive: in-class and handwritten components, oral defence of submitted work, iterative drafts reviewed at stages, assignments requiring personal or local specifics a model cannot know, and explicit disclosure policies stating permitted AI use rather than pretending it is absent. None of these require a detector, and none of them falsely accuse anyone.
Frequently Asked Questions
Are ChatGPT detectors accurate?
Not reliably. They estimate statistical properties of text rather than detecting anything left behind by a model, and their accuracy degrades sharply on edited text, short passages, and models they were not tuned against. OpenAI withdrew its own detector in 2023 for low accuracy.
Can a ChatGPT detector be wrong?
Yes, in both directions. False positives disproportionately affect non-native English speakers, some neurodivergent writers, and anyone writing in a formal, structurally consistent style. Lightly paraphrasing generated text is usually enough to produce a false negative.
Why do AI detectors flag human writing?
Because they measure predictability and sentence-length variation, not authorship. Clear, conventional, formulaic writing scores as machine-like regardless of who wrote it — which is why second-language writing, technical documentation and taught essay structures are flagged so often.
Can teachers actually prove I used ChatGPT?
A detector score alone is not proof, and a growing number of institutional policies say so explicitly. Stronger evidence comes from process — document version history, drafts, and the ability to discuss the work — which cuts both ways and is the fairer standard.
Does paraphrasing beat AI detectors?
Generally yes, which is itself an argument against relying on them. Paraphrasing raises perplexity and burstiness, the two measures detectors depend on. That the tools are easy to defeat while still falsely accusing honest writers is the core problem with using them punitively.
Did OpenAI make a ChatGPT detector?
Yes. OpenAI released an AI Text Classifier in early 2023 and withdrew it roughly six months later, citing a low rate of accuracy. It remains the strongest available evidence that reliable post-hoc detection is not currently achievable.