How AI Detectors Work: Perplexity, Burstiness and Accuracy

AI detectors use multiple approaches to identify AI-generated text. Most modern detectors combine several methods for higher accuracy:

1. Perplexity Analysis

Perplexity measures how "surprised" a language model is by a piece of text. AI models tend to generate text that's highly predictable to other AI models—meaning low perplexity. Human writers, especially creative or non-native speakers, produce text with higher perplexity.

Why this causes false positives: Non-native English speakers often use simpler, more predictable language structures. A 2023 study by Liang et al. in Patterns found that the non-native essays every detector flagged had significantly lower perplexity. Technical writing, business communications, and academic papers also follow predictable patterns. All of these can trigger low perplexity scores despite being genuinely human-written.

2. Burstiness Measurement

Burstiness refers to variation in sentence length and complexity. Human writers naturally vary their sentences—mixing short, punchy statements with longer, complex ones. AI tends to produce more uniform sentence structures.

Why this causes false positives: Writers who've been taught to write clearly and concisely, especially in corporate or academic settings, may produce uniform sentence structures that resemble AI output.

3. Embedding Analysis

More sophisticated detectors analyze the semantic embeddings (vector representations) of your text and compare them to known patterns in AI-generated content. They look for clustering patterns, relationships between concepts, and stylistic fingerprints specific to particular AI models.

This method examines how concepts relate to each other in multi-dimensional space. AI models tend to organize concepts in predictable ways, creating detectable patterns in the embedding space.

4. Stylistic Fingerprinting

AI models have characteristic "tells"—phrases they overuse, punctuation patterns, transition words, and structural quirks. For example:

• ChatGPT frequently uses phrases like "it's worth noting," "dive into," "delve into," and "it's important to remember"
• Claude tends toward more formal structuring with numbered lists and systematic breakdowns
• GPT-3/3.5 often uses "ultimately" and "additionally" as transition words

Detectors maintain databases of these stylistic patterns and flag text that exhibits multiple model-specific characteristics.

Each major detector uses a slightly different approach. Understanding these differences helps explain why text might be flagged by one detector but not another.

Vendors often advertise very high accuracy (GPTZero's Canvas page, for example, cites 99%+ accuracy). Independent research and the vendors' own documentation paint a more cautious picture.

The Accuracy Problem

• Independent testing: Weber-Wulff et al. (2023) tested 14 detection tools, including Turnitin, and concluded they are “neither accurate nor reliable,” with a bias toward classifying AI output as human-written. Content obfuscation techniques made them significantly worse.
• Non-native English writers: In Liang et al. (2023), seven widely used detectors labeled more than half of 91 TOEFL essays by non-native writers as AI-generated (an average false positive rate of 61.3%), while classifying US eighth-grade essays accurately. The authors write that “GPT detectors frequently misclassify non-native English writing as AI generated, raising concerns about fairness and robustness.”
• A vendor's own trade-off: Turnitin says it keeps its false positive rate under 1% for documents with over 20% AI writing, and that this means it may miss some AI text: a document scored at 50% could contain as much as 65% AI writing. Its AI Writing Report hides scores between 1% and 19% behind an asterisk because false positives are more common in that range.

You can check a draft with our free AI detector. Like every detector's, its score is an estimate, not proof.

Why False Positives Occur

The Cat-and-Mouse Game

AI detection is fundamentally a moving target. As AI models improve and humanization techniques evolve, detectors must constantly update. This creates several problems:

1. Detection lag: Newly released AI models may not be in a detector's training data until the vendor updates it
2. Overfitting: Detectors trained too specifically on old models create false positives on human writing
3. Adversarial adaptation: As humanization improves, detection becomes less reliable
4. No ground truth: In real-world use, there's no definitive way to verify accuracy

What Detectors Cannot Do (Yet)

Current AI detectors have clear limitations:

• Cannot detect future AI models they haven't been trained on
• Cannot distinguish between AI-assisted and AI-generated content reliably
• Cannot detect AI-generated ideas rewritten by humans
• Cannot account for legitimate language diversity across cultures and contexts
• Cannot provide definitive proof—only probabilistic assessment

For Students and Writers

Schools connect detectors such as Turnitin, Copyleaks or GPTZero to their learning platforms, and the tool can vary by course. Our guide to which AI detector your school uses explains how to find out and whether you can see the result.

If you're concerned about false positives on your authentic writing:

• Document your process: Save drafts, notes, and research to prove your work
• Understand your institution's policy: Some allow AI with disclosure, others prohibit it entirely
• Add personal voice: Include specific examples, personal anecdotes, and course-specific references
• Vary your sentence structure: Mix short and long sentences intentionally
• Consider humanization: If you're a non-native speaker getting false positives, tools like ours can help add natural variation while maintaining your authentic content

For Educators and Institutions

If you're implementing AI detection policies:

• Never use detectors as sole evidence: They're probabilistic tools, not definitive proof
• Account for false positives: Especially with non-native speakers and neurodiverse students
• Create clear policies: Define acceptable AI use rather than blanket bans
• Focus on assessment design: Create assignments that require authentic demonstration of understanding
• Use multiple indicators: Detection + voice consistency + specific knowledge + process evidence

For Content Publishers

If you're verifying content authenticity:

• Use detectors as screening tools: Not definitive judgments
• Value originality over detection scores: Unique insights matter more than word choice
• Accept AI-assisted content: With appropriate editing and expertise
• Focus on value: Does the content serve readers regardless of authorship method?

AI detection technology is evolving rapidly, but so are AI models and humanization techniques. Here's where the technology is headed:

Emerging Approaches

• Watermarking: AI companies like OpenAI are exploring imperceptible watermarks in generated text
• Provenance tracking: Blockchain-based systems to verify content origin
• Behavioral analysis: Analyzing writing patterns over time rather than single documents
• Multimodal detection: Analyzing not just text but images, formatting, and metadata

The Philosophical Question

As AI becomes more sophisticated and human-like, we're approaching a fundamental question: If AI-generated text is indistinguishable from human writing, does the distinction matter?

Perhaps the focus should shift from how content was created to what value it provides. In education, this means designing assessments that require authentic demonstration of understanding. In publishing, it means valuing originality, insight, and helpfulness over authorship method.

Understanding how AI detectors work empowers you to use them appropriately, challenge false positives when they occur, and make informed decisions about AI use in your context. Whether you're a student, educator, writer, or publisher, knowledge of detection technology helps you navigate this rapidly evolving landscape.