Hyptor Lab Pre-registration: AI Detector Accuracy Index

This is the test plan for Hyptor Lab’s first hands-on test, published before any sample is run. Results will be judged against this plan, not the other way round.

The question

How often do consumer AI detectors flag human writing as AI-written, and how often do they catch AI writing, on the same set of texts? Each detector’s own accuracy claim will be quoted from its website and graded against the result.

Detectors

GPTZero, ZeroGPT, Grammarly’s AI detector, QuillBot’s AI detector, Copyleaks and Scribbr, each on the free tier available in the test window. If a free tier is withdrawn or capped below the sample size before the test, the detector is marked “not tested: access” rather than dropped silently. Tiers and versions are recorded on the day of each run.

Samples: 90 texts

• 40 human-written: public-domain essays and articles published before 2020, plus the author’s own pre-2022 writing. Each has a recorded source and date. Mixed formality; English only.
• 40 AI-written: 10 fixed prompts, each answered by four current chatbots (ChatGPT, Claude, Gemini and one open-weight model), default settings, no instruction to evade detection.
• 10 lightly edited AI: 10 of the AI samples, edited by a person for 5 minutes each with no tool, to represent ordinary editing rather than deliberate evasion.

Every sample is between 300 and 600 words. The full set, with sources and provenance notes, is published with the results.

Procedure

• Each sample is submitted to each detector once, in randomised order, within a 14-day window.
• The detector’s headline output is recorded exactly (percentage, label or both), with a screenshot.
• A sample counts as “flagged” when the detector’s own label says AI-written or likely AI-written, or when its score is 50% or above where only a score is given. The rule is fixed before the first submission.
• Any sample a detector refuses or errors on is recorded as “no result”, not excluded.

What we will report

• False positive rate on human writing: flagged human samples out of 40, per detector. This is the headline number.
• True positive rate on AI writing: flagged AI samples out of 40, per detector.
• Edited-AI rate: flagged edited samples out of 10, per detector.
• Each detector’s quoted accuracy claim, with the archive link, next to its measured result and a verdict label.
• The full results as a CSV under a CC BY 4.0 licence.

Limits, stated in advance

90 samples is enough to show large differences between detectors and to test a “99% accurate” claim; it is not enough to resolve differences of a few percentage points, and results will say so. English only. Free tiers only, which may differ from paid tiers. One run per sample, so detectors that give variable results on repeat submissions are not characterised. Non-native English writing is not separately sampled in this first run; it is planned for the second.

Repeat schedule

The same 90 samples will be re-run every quarter, so the index shows how each detector changes over time. Any change to the sample set or scoring rule will be announced on this page before it takes effect, with the reason.

Change log

15 Oct 2026: plan published. No test has been run.