How we know the checker is right — and where it can be wrong.

A checker is wrong in two directions, and they cost different things. A false positive wastes an afternoon and teaches you to distrust the next finding. A false negative is worse: it tells you a page is clean when it is not, and you find out from a client. Almost nothing in this industry publishes either number, so here is what we actually do, with figures and dates.

1. Every finding is re-checked against the raw HTML

We have a second reader — separate code, written to look for the exact thing each finding claims, without using the engine at all. On 21 October 2026 it re-checked 299 of 389 findings across 20 pages of 15 sites, including our own demo site, university sites and news sites. Every one of the 299 agreed with the source.

The other 90 findings could not be re-checked independently, and we say so rather than counting them as verified: they are the checks where the judgement (not the markup) is the hard part.

One disagreement did appear on the first run of that check, on a page that turned out to be an OpenResty welcome screen. We chased it: the engine was right and our verifier was wrong, because of a mangled regular expression. That is the whole point of having a second reader — it catches us either way, and the disagreement is fixed and recorded rather than argued away.

2. Faults planted on purpose, on a public site

demo.accessaudit.dev is a small site we built to be wrong. Twenty three specific faults were planted in it — images with no alt, links with no text, skipped heading levels, unlabelled inputs, duplicate ids, blocked zoom, an untitled frame, a table with no headers, a dangling aria-labelledby. Every health check of our own systems re-scans that site and fails if the engine stops finding any of them.

There is a second set of 22 planted problems in the engine's own test suite, which asserts both directions: that each planted fault is found, and that nothing is invented on the clean control pages. The list of what is planted, and why, is public in the repository. Three faults on the demo site are deliberately not on the expected list, because we do not check for them — they are there so that anyone comparing our output with a browser extension can see what we do not claim.

3. Every check says when it might be wrong

Each of the 25 checks has its own page with the failing markup, the WordPress fix, and a section called when we might be wrong — the cases where the check can fire on something that is actually fine. The index is here. If you get a finding you disagree with, that section is written for exactly that argument.

4. We compared ourselves with axe-core

We ran axe-core, the most widely used accessibility engine, over the same pages and compared the results. The exercise found one real miss on our side — accessible names made only of non-breaking spaces, which we were treating as a name when a screen reader hears nothing — and it was fixed, with fixture cases added so it stays fixed. We are not claiming to be better than axe; we are claiming to have looked, and to have published what we found.

5. What we could not read, we say we could not read

Automated checkers are silent about pages they never saw, and that silence is easy to present as cleanliness. In our own weekly study, 485 of 1,298 homepages could not be read at all in one week — a third of the top million. Our reports and our audits list the pages that could not be read, separately, and never count them as clean.

What none of this proves

• It does not prove a site is accessible, or compliant with WCAG, or anything else. Nobody can prove that from an automated scan, including us, and we do not sell it.
• It does not prove we find every mechanical failure. It proves that the ones we report are real, and that we keep finding the ones we know about.
• It says nothing about colour contrast, whether alt text is useful, keyboard behaviour, reading order, or how a page sounds in a screen reader. Those need a person. Our ACT rules coverage page says which rules we fully implement, which we partly implement, and which we do not attempt.

How to argue with us

The engine is open source (GPL), the rules are readable, and the checks pages name their own failure cases. If you find a finding that is wrong, or a failing page we miss, send it: hello@accessaudit.dev. Corrections get published here rather than quietly fixed, because a checker that hides its errors is asking to be trusted on nothing.