a11y-auditor

What it is

The agent gets four tools and uses them together to produce WCAG 2.2 AA findings with reproducible evidence.

• **sr-driver** owns the headed browser and VoiceOver (macOS) or Orca (Linux), exposed over CDP so other tools share the session.
• **collect.ts** runs a baseline evidence sweep in one command: axe results, screen-reader announcements, screenshots.
• **audit.ts** runs axe-core and returns the accessibility tree over HTTP.
• **agent-browser** handles clicks, typing, and snapshots through CDP.

How an audit works

The auditor walks a page through a fixed phase structure. Each phase has a defined goal and produces structured evidence the next phase can build on.

1. Boot. Launch the screen-reader driver and a paired Chromium instance. Both run as long-lived daemons exposing CDP + HTTP so the other tools can connect.
2. Discover. Sweep baseline evidence in one pass: axe-core results, screen-reader announcements, screenshots, and a first read of the accessibility tree.
3. Baseline. Audit document structure: headings, landmarks, images, language, page title, and other criteria that don’t require interaction.
4. Keyboard. Walk the page via Tab, Shift+Tab, arrow keys, and Escape. Check tab order, focus traps, focus indicators, and skip links.
5. Visual. Cross-reference screenshots against the accessibility tree. Catches issues where rendered DOM and exposed semantics disagree (a visible button that’s not in the a11y tree, a heading that looks like a heading but isn’t marked as one).
6. Report. Produce findings with reproducible evidence: criterion, severity, annotated screenshot, the screen-reader transcript that produced it, suggested remediation.

Heavy phases (baseline, keyboard, visual) delegate to dedicated sub-agents in .claude/agents/. Each sub-agent gets a focused system prompt and a tight tool allowlist, so a long screen-reader transcript doesn’t poison the orchestrator’s context and a keyboard walk doesn’t have to reason about visual cross-referencing. The orchestrator stays clean, the sub-agents stay specialized.

A companion skill, acr, takes a completed audit and emits a VPAT 2.5 Accessibility Conformance Report from the same evidence. That’s the document a vendor sends to a procurement team to claim WCAG conformance.

Eval harness

The repo includes an evaluation pipeline that runs the agent against W3C ACT Rules test cases and scores precision and recall on its findings. The point is to make the auditor measurable: “did the agent catch the violation?” becomes a number you can move when you change the prompt or rearrange the tools.

Dogfooded on this site

I ran the auditor against this portfolio while building it. Scope: 5 routes across 2 themes (light, dark) = 10 page-mode combinations, full WCAG 2.0 / 2.1 / 2.2 AA + AAA + best-practice rule sets, plus a VoiceOver walkthrough on the homepage.

First pass found 4 violations, 52 AAA contrast failures, and 44 incomplete contrast nodes. Some of the findings:

• An opacity-60 modifier on top of an already-muted color dropped one label to 2.61:1 against a near-white background. Fails AA.
• Section labels were <p> elements, breaking the H1 to H2 to H3 progression.
• No <main> landmark; six nodes of content sat outside any landmark.
• The “linked card” pattern used an absolutely-positioned overlay anchor that prevented axe from computing the background color of every element underneath it. 44 contrast nodes flagged as incomplete because of one architectural choice. The tool caught it because axe distinguishes “incomplete” from “violation” instead of silently passing.
• Four “repo” links shared the same accessible name. To a screen reader they were indistinguishable.

After fixes: 0 violations across all 10 combinations. The remaining “incomplete” findings are decorative arrow glyphs already wrapped in aria-hidden="true" spans. Those are false positives the tool correctly defers on.

The VoiceOver walkthrough announces landmarks, headings, article boundaries, tags, and distinctly named action links cleanly:

main → heading 1 ed-tech / applied ai engineer → ...
  region 01 — recent projects → heading 2 → 4 projects
    article a11y-auditor → heading 3 link → in dev → tagline → tags →
    link "a11y-auditor — repository" → end of article
    article Euchre Reasoning Arena → ...

This is the loop the eval harness is built to scale: the auditor produces findings, the findings drive fixes, the fixes get re-audited.

What’s next

The auditor today is “a Claude Code session reads SKILL.md and drives four tools.” This isn’t deterministic and is tough to visualize, so the next iteration moves orchestration to a LangGraph agent graph and adds a web dashboard so the same audit runs as a launchable, observable workflow. The full plan lives in [docs/web-dashboard-plan.md](https://github.com/estern1011/a11y-auditor/blob/main/docs/web-dashboard-plan.md); the short version:

Runner. Replace the read-the-skill orchestration with a typed LangGraph StateGraph. Nodes are tools (boot, discover, report) or agents (baseline, keyboard, visual, auth). Conditional edges branch on signals each agent returns (needsAuth, hasInteractive, treeEmpty). Durability via a SQLite checkpointer; resume by run-id.

Hybrid execution. VoiceOver can’t be driven remotely (Apple’s accessibility APIs block it), so VoiceOver runs stay on the user’s Mac and Orca runs dispatch to a sprites.dev sandbox. Same runner binary, two environments, one UI.

Three surfaces. A Configure view (URL + WCAG level + viewport + SR engine + tool toggles), a Run view (graph + phase rail + event timeline + live VNC into the browser the agents are driving), and a Findings view (filterable list with Overview / Evidence / SR Transcript / Remediation tabs).

Phased delivery. Phase A is replay-first: the UI reads completed runs from disk. Phase B adds sprite dispatch so Configure → Start actually launches Orca runs without a terminal. Phase C adds SSE event streaming and CDP screencast for a live observer.

The point of moving to a declared graph is that the agent graph definition in code IS the graph in the UI. graph.getGraph().drawMermaid() emits the same shape that renders on screen, so the UI cannot lie about what the runner is doing.

Tech

TypeScript, Bun, Playwright, Chrome DevTools Protocol, axe-core, VoiceOver/Orca AT bridges, Claude Code skill spec.