arXiv study finds AI humanizers alter 35–65% of fact units in documents
A new paper on arXiv examines how AI ‘humanizers’ rewrite machine-generated text to evade detection, but finds they often drop or change factual details. Researchers tested 11 commercial tools in the October 2026 leaderboard cycle and found all altered at least one of four fact units—numbers, dates, names, or quoted spans—in 35–65% of documents containing them. Most errors occurred in quoted…
Key points
- 11 commercial AI humanizers altered 35–65% of fact units (numbers, dates, names, quotes) in October 2026 tests
- An open 12B humanizer kept 97% of numbers and names but changed 67–78% of quoted spans
In contrast, perplexity-based detectors failed to distinguish AI-styled rewrites from human text when line breaks were removed. The authors released a rule-based checker and per-item results to validate their findings.
Model page: Jev →
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Research
All →- Researchers introduce ResMem to turn retrieved knowledge into parametric memory · 1 src
- Circuit Oracle automates attribution graph analysis · 1 src
- Researchers propose gavagai to improve Vision-Language model alignment · 1 src
- Gemini 3.6 Flash shows language-dependent negotiation behavior in study · 1 src
- Researchers outline framework for unifying embedding models across modalities · 1 src
Comments
via GitHub Discussions