AI Tools & Software · Posted by Pete Kowalczyk ·

How to Detect AI-Generated Text: A 2026 Guide

2

been running detection tools against outputs from the latest models for the past few weeks and honestly the results are pretty sobering. the arms race between generators and detectors is real, and right now detectors are losing. not slightly losing – getting embarrassed in some cases.

## what i actually tested and how bad it got

i ran outputs from several frontier models through six different detectors, including gptzero, originality.ai, copyleaks, and a few academic-facing ones. same prompts, default settings, no post-processing on the outputs. results were all over the place:

– gptzero flagged roughly 60-70% of raw gpt-4o output correctly, but dropped to under 30% when i added a single “rewrite this more conversationally” pass
– originality.ai held up better on longer pieces but fell apart on anything under 300 words
– copyleaks gave confident “human written” verdicts on outputs i literally watched generate in real time
– one tool i tested in an academic context, [proofademic.ai](https://proofademic.ai), was more upfront about confidence intervals than most – it showed probability ranges instead of binary verdicts, which at least doesn’t give educators false certainty
– every single tool failed badly on outputs that had been lightly edited by a human afterward

the human-edit problem is the real one. even a five-minute pass by a mediocre human writer drops detection rates into near-uselessness on most tools.

## why the detectors keep falling behind

the core issue is that detectors are trained on yesterday’s model outputs. by the time a new detector is calibrated and deployed, the models it was trained to catch have already been updated or replaced. it’s not a solvable problem with current approaches – it’s structural.

there’s also a base rate issue nobody talks about enough. if you’re scanning a university submission pool where maybe 15-20% of submissions actually used AI heavily, and your detector has a 10% false positive rate, you’re flagging almost as many innocent students as guilty ones. that math destroys the tool’s usefulness for enforcement even if the accuracy sounds okay in a press release.

some things that currently make detection harder than it should be:

1. perplexity and burstiness scores – the two main signals detectors use – are easy to game once you know they exist
2. multilingual content breaks most detectors entirely, they’re heavily english-optimized
3. retrieval-augmented outputs blend source material in ways that look more “human” to statistical models
4. fine-tuned models on domain-specific text produce outputs that read nothing like the training data detectors expect

## what actually works better right now

watermarking at the model output level is more promising than post-hoc detection, but it requires cooperation from the model provider and is trivially stripped by paraphrasing. cryptographic provenance on content – tagging it at creation – is where the serious research is pointing, but that’s years from being practical at scale.

for anyone in an academic or professional context trying to deal with this practically: multi-signal approaches work better than single-tool verdicts. look at submission behavior, metadata, whether the writing style matches previous samples, inconsistencies in citation knowledge. use detection scores as one weak signal among several, not a verdict. the tools that present results as probability distributions rather than binary calls are at least being honest about what the technology can and can’t do.

i’m not saying detection is useless – raw unedited model output from less sophisticated users still gets caught reasonably often. but if someone knows what they’re doing and spends 20 minutes on post-processing, current detectors are largely theater.

curious what others are seeing – especially if anyone’s working on the educator side. are institutions actually changing their policies around this, or still treating detector output as reliable evidence?

5 replies

5 Replies

1

adding some context here since i have experience with this - cursor is legitimately the best IDE ive ever used. hope that helps anyone on the fence

1

genuine question for anyone who's tried this - whats the actual cost per month if youre using it for a small team? ive been going back and forth and cant decide