Why length matters so much
Text detectors look for patterns: how predictable each word is to a language model, and how that predictability varies across sentences. Those patterns only become measurable over enough text. A 40-word caption, a reply or a headline gives a detector almost nothing to work with, so its score swings on a single word choice. Sadasivan and colleagues argued that as language models improve, even long texts become hard to separate reliably; short ones are harder still.
What independent testing found
Weber-Wulff and colleagues tested fourteen detection tools on human-written, machine-written, edited and translated texts and concluded that none was accurate or reliable enough to support accusations, with editing and paraphrasing reducing detection sharply. Liang and colleagues found that several detectors flagged a large share of essays by non-native English writers as machine-generated. Both studies used longer texts than a social post, which makes their warnings stronger, not weaker, for captions and replies.
What to rely on instead
| Question | Better evidence than a detector score |
|---|---|
| Is this account a real person? | History, consistency, replies, links to a known site |
| Is this image real? | Provenance data, earlier copies, other sources for the event |
| Is this claim true? | The original source the post cites, or its absence |
| Did this writer use a tool? | Their disclosure, and in a dispute their drafts and history |
For images there are better tools than for text: provenance records and reverse image search, covered in checking Content Credentials and spotting AI images in your feed. For text, the useful questions are about the source, not the style.
An AI detector on social media posts: a worked example
Suppose an editor collects 60 posts of about 40 words each: 30 written by staff without any tool and 30 produced by a language model from the same prompts. The numbers are invented for this example, but they resemble the pattern in published tests. Run through a detector, 11 of the 30 human posts are flagged as likely machine-written, 14 of the 30 generated posts are flagged, and the rest of both groups are called human or uncertain. On this length of text the detector's answer is closer to a coin toss than to a verdict.
The same 60 texts extended to 400 words by the same authors and the same model do better in the example: 4 of 30 human texts flagged, 22 of 30 generated texts flagged. That is still 4 honest writers accused, and any light edit of the generated texts drops the second number sharply. For captions and replies the first set of numbers is the relevant one, and a moderation rule built on them would be wrong about a third of the time. A rule that instead asks for one piece of behavioural evidence, such as an account older than a month or a link to a known site, removes far fewer honest posts in the same example.
Pitfalls
- Running one post through several detectors and reporting the worst score. Each tool is unstable on short text; picking the maximum manufactures a false certainty.
- Treating a formal or non-native writing style as a signal. Published tests found exactly that style flagged most often.
- Applying a threshold tuned on essays to captions. Scores on 40 words swing with a single word choice.
- Auto-moderating on a score. Behaviour, links, account history and reports are stronger evidence and produce fewer wrongful removals.
- Publishing a score as if it were a finding. A number invites readers to treat an estimate as a fact.
- Re-running the same text until the score changes. Some tools vary between runs on short input, which says more about the tool than the text.
What to record
If you must run detectors at all, keep a log that makes the limits visible: the text length, the tool, the score and what other evidence you had. A log like the examples below usually persuades a team to drop scores from moderation decisions after a few weeks, because the mismatches pile up. The other signals worth logging are covered across the provenance section.
| Post | Words | Detector said | Other evidence | Decision |
|---|---|---|---|---|
| Reply on a news thread | 38 | Likely AI, 84 percent | Account 6 years old, consistent history | Leave up |
| Product caption | 44 | Likely human, 71 percent | New account, identical text on 30 other pages | Remove as spam |
| Comment from a regular | 52 | Likely AI, 91 percent | Non-native writer, known to moderators | Leave up |
| Guest post excerpt | 410 | Uncertain, 55 percent | Author supplied drafts and sources | Publish |
For publishers
If you run a site with comments or community posts, do not auto-moderate on a text detector's score. Moderate on behaviour, links and reports. And if your own writing is ever questioned, the guide to being accused of posting AI content sets out the evidence that settles it. The wider provenance section covers the signals you can add to your own posts.
Sources
- Weber-Wulff et al., Testing of Detection Tools for AI-Generated Text (2023)
- Sadasivan et al., Can AI-Generated Text be Reliably Detected? (2023)
- Liang et al. (2023), detectors of machine-written text are biased against non-native English writers
Questions
Can AI detectors tell if a tweet was written by AI?
Not reliably. Short posts give detectors too little text to measure, so scores are unstable and often wrong.
Are AI detectors accurate?
Independent tests found none of the tools studied accurate or reliable enough to support accusations, especially on edited or paraphrased text.
Why do detectors flag non-native English writers?
Simpler vocabulary and common constructions make text more predictable, which pushes statistical detectors towards a machine verdict.
How can I tell if a social account posts AI content?
Look at the account's history, sources and behaviour over time rather than scoring individual posts.
What length of text does an AI detector need?
Published tests used essays of several hundred words and still found detectors unreliable. Below about 100 words the scores become unstable enough that a single edit changes the verdict.
Hannah Voss, Editor. Checks every guide against a working WordPress install and the networks' current documentation. Last reviewed September 2026.
Related reading
- Accused of posting AI content: how to show your workWhat to do when readers, a client or a platform say your post or article was written by AI: the evidence that settles it and what does not help.5 min read
- Spotting AI images in your feed: checks that hold upWhich checks for AI-generated images in a social feed actually hold up: platform labels, provenance data, earlier copies and the source, and which tells fail.5 min read
- AI disclosure lines that survive a shareWhere to put an AI assistance disclosure so it reaches readers who only see the share card: titles, descriptions, bylines and the first lines of a post.5 min read
- AI-generated content in social feeds: what publishers should doHow AI-generated images and text change what happens when your posts are shared, and the practical steps publishers can take: disclosure, provenance, bylines.7 min read