What an AI detector measures
A detector estimates how closely the wording of a text resembles the AI-written text it was trained on.
Language models tend to pick the most likely words and build even, well-formed sentences. People are less predictable: they repeat themselves, change pace and choose words no model would expect. A detector learns this statistical difference from a large number of examples.
Two designs are common. The first asks another language model how predictable each word of the text is and treats very predictable text as suspicious. That measure is called perplexity.
The second is a trained classifier. An encoder turns the text into a set of numbers, and a classifier trained on human and AI examples scores those numbers. Textumi works this way, with detector models we train, tune and run on our own servers. Both designs rely on the text alone.
Watermarking is a separate idea: a model leaves a statistical mark in its text as it writes, and whoever holds the key can check for it. It only works for models that add such a mark, so a detector meant to read any text cannot rely on it.
A detector does not check facts, does not judge the argument and does not look your text up in an archive of chatbot answers. Two texts with the same meaning and different wording can get very different results.
Why the result is a signal and not a verdict
Human and AI writing overlap. Some people write evenly and predictably, and a model can be prompted to write unevenly.
Wherever a threshold is drawn through that overlap, some human writing ends up on the AI side and some AI writing on the human side. Moving the threshold only trades one mistake for the other.
The result itself is easy to misread. Textumi names it with one of five labels: “Human-written”, “Mostly human”, “Mixed: human and AI”, “Mostly AI” and “AI-generated”. The first two sit below the threshold, the other three above it. A label describes how the style of the whole text reads, not the probability of AI authorship or the share of the text that was generated.
If another detector shows a percentage, that is also a model's estimate and deserves the same caution.
These limits are well documented. In 2023 OpenAI withdrew its own AI text classifier, citing its low accuracy, and Stanford researchers reported the same year that detectors flagged essays by non-native English writers more often than essays by native speakers.
What paraphrasing and editing do to a result
A detector judges the text it receives. When the wording changes, the result changes too.
When a person rewrites an AI-generated text sentence by sentence, fewer of the patterns the detector knows remain. When an AI assistant polishes a person's text, more of them appear.
The outcome is mixed text. A detector trained on purely human and purely AI examples can place it on either side.
The same goes for paraphrasing tools, our own Humanizer included. The Humanizer aims at natural wording that keeps the facts and logic of the original, not at a score. A rewrite is new wording, and the detector's result is about that wording, not about where the text came from, so a text one detector leaves alone can still be flagged by another.
Heavy grammar correction and translation move results as well, sometimes toward the AI side.
Why new models are a moving target
A detector knows the models whose texts were in its training data, as those models wrote at the time. Text from a newer or different model may resemble them or may not, and only testing can tell.
Textumi's detectors for languages other than English and Ukrainian were trained on text from two widely used AI models. Text from other models is outside those tests, which is one reason we keep improving the detectors and release them in versions.
Why languages other than English are harder
A model trained on English text does not start working equally well in other languages by itself. Each language needs its own human texts, its own AI texts and its own test.
The hardest part is collecting text that is known to be written by people. Smaller languages have less of it, so a training set often covers only a few genres.
Textumi trains a separate classifier for each language. The English and Ukrainian detectors are the most thoroughly tested. The detectors for the other languages were trained and tested on prose from Wikimedia projects and on text from two AI models.
On other genres some of these detectors miss many AI texts, among them French, Vietnamese, Turkish, Indonesian and Dutch. Translated text is a further source of error in any language.
Languages and the state of eachHow Textumi's AI detector is built and tested
How to read a result responsibly
- Read the highlighted passages before you look at the label, because they show where the signal comes from and let you judge those lines yourself.
- Mind the length. Under 50 words Textumi shows “Too little text to judge” instead of a label, and short texts deserve caution in any detector.
- Mind the genre and the language. A flag on a formal letter, or a result in a language tested on a narrower set of texts, carries less weight than the same result on a long English essay.
- Treat “Human-written” and “Mostly human” as estimates. They mean the detector found little that looks like AI, not that a person definitely wrote the text.
- Gather other evidence before you decide anything: drafts, version history and a conversation with the writer say more about authorship than any detector.
Common questions
Are AI detectors accurate?
It depends on the text, and no single figure describes a detector honestly. Results vary with the language, the genre, the length of the text and the model that wrote it. Every detector sometimes flags human writing and misses AI writing. That is why we publish how we test our detector but do not advertise an accuracy figure.
Can AI detectors detect ChatGPT?
A detector reacts to wording, not to a particular model. Text from ChatGPT, Claude, Gemini or any other model can be flagged if it resembles the AI writing the detector was trained on, and missed if it does not. Textumi does not identify which model wrote a text and has not been tested model by model.
Does paraphrasing or editing change a detector's result?
Yes. A rewrite changes the wording, and the result follows the wording. This is a limit of the method, not a guarantee either way: edited AI text can still be flagged, a rewrite can be flagged by another detector, and the result never shows that the origin of a text has changed.
Why do two AI detectors judge the same text differently?
Each was trained on different human and AI texts, scores wording in its own way and sets its own threshold. A text close to the line can fall on either side of it. Disagreement between detectors is normal, and it is one more reason to treat every result as a signal rather than a finding.
Do AI detectors work on short texts?
Poorly. A few sentences hold too little evidence for a stable result, and one unusual phrase can move it. Textumi accepts texts from 5 words, but under 50 words it shows “Too little text to judge” instead of a label. Whenever you can, check a longer passage from the same document.
Do AI detectors work in languages other than English?
Some do, with more caveats. Each language needs its own training texts and its own test, and those are harder to collect outside English. Textumi's detector works in 20 languages. English and Ukrainian are the most thoroughly tested; the others were tested on a narrower set of texts, and some are weaker on other genres.
What are perplexity and burstiness in AI detection?
Perplexity measures how predictable a text is to a language model: the lower it is, the more predictable the text. Burstiness describes unevenness, how long and short, simple and complex sentences alternate. Some detectors treat low perplexity and an even rhythm as signs of AI. Both measures are blunt, since formulaic human writing is predictable too. Textumi uses a trained classifier rather than these measures directly.