EAA Testing & Audit Guide 2026: Automated + Manual Accessibility Testing Stack for EU Compliance

Post #1430 in the sota.io EU Compliance Series — EU-EAA-DEVELOPER-2026 #5/6

The European Accessibility Act (Directive 2019/882) entered enforcement on June 28, 2025. If your SaaS or web service is in scope, Art.13 places concrete obligations on you as a service provider: you must ensure your service conforms to the accessibility requirements in Art.4, the harmonized standard is EN 301 549, and for web content that embeds WCAG 2.1 AA (Chapter 9).

The uncomfortable truth: most accessibility testing done today catches only 20-30% of actual barriers. Automated scanners are fast but blind to context. Screen-reader testing finds what scanners miss. And EAA market surveillance authorities will expect documentation — not just a clean Lighthouse score.

This guide covers the complete testing stack for EAA compliance in 2026: what tools to use, how to configure your CI/CD pipeline, how to run manual tests, and what audit documentation you need to survive an enforcement inquiry.

Why Testing Matters for EAA Art.13 Compliance

Art.13 of Directive 2019/882 requires service providers to:

• Ensure their service meets the accessibility requirements in Art.4
• Draw up information on the service's compliance under Art.13(1)
• Ensure procedures are in place for in-scope services to remain in conformity
• Notify market surveillance authorities of non-conformity when detected

Art.15 establishes EN 301 549 as the harmonized technical standard. Conformance with EN 301 549 creates a presumption of conformity with the EAA accessibility requirements. For web services, the relevant chapter is Chapter 9 (WCAG 2.1 AA). For mobile apps, Chapter 11.

This is not optional: Art.19 gives market surveillance authorities the power to inspect, test, and demand documentation. Art.24 enables enforcement measures including market withdrawal. Article 24(4) allows national authorities to fine for non-conformity.

What "conformance" means practically:

1. You can demonstrate that your service meets WCAG 2.1 Level AA (for web) or EN 301 549 Chapter 11 (for native apps)
2. You have a documented testing process showing ongoing compliance verification
3. You have an Accessibility Statement (required in most national transpositions)
4. You have a feedback mechanism for users to report barriers

Without testing documentation, you cannot make any of these claims.

Layer 1: Automated Testing (Catches 20-30% of Issues)

Automated testing is fast, cheap, and integrable into CI/CD. Use it for every commit. But understand its limits: it can reliably detect contrast failures, missing alt text, form label gaps, and focus order problems — but it cannot assess whether an alt text is meaningful, whether a modal is keyboard-operable in context, or whether a chart is understandable by screen reader users.

axe-core (Best Coverage for WCAG 2.1 AA)

axe-core is the most widely used open-source accessibility engine. It powers browser extensions (axe DevTools), Playwright integration, and the axe-cli. Coverage: ~57% of WCAG 2.1 AA success criteria have automated checks.

Installation:

npm install --save-dev @axe-core/playwright
# or for CLI
npm install -g axe-cli

Playwright + axe-core (CI-ready):

import { test, expect } from '@playwright/test';
import AxeBuilder from '@axe-core/playwright';

test('Homepage meets WCAG 2.1 AA', async ({ page }) => {
  await page.goto('/');
  
  const accessibilityScanResults = await new AxeBuilder({ page })
    .withTags(['wcag2a', 'wcag2aa', 'wcag21a', 'wcag21aa'])
    .exclude('#cookie-banner') // exclude third-party widgets if needed
    .analyze();
  
  expect(accessibilityScanResults.violations).toEqual([]);
});

Key axe rules for EAA compliance:

Running on a URL:

axe https://yourapp.com/dashboard --tags wcag2a,wcag2aa,wcag21aa --reporter json > axe-report.json

Lighthouse Accessibility Audit

Lighthouse is built into Chrome DevTools and available via CLI. It provides a 0-100 accessibility score based on audits powered by axe-core plus additional checks.

npm install -g lighthouse
lighthouse https://yourapp.com --only-categories=accessibility --output json --output-path ./lighthouse-report.json

In CI (GitHub Actions):

- name: Run Lighthouse CI
  uses: treosh/lighthouse-ci-action@v10
  with:
    urls: |
      https://staging.yourapp.com/
      https://staging.yourapp.com/dashboard
    budgetPath: .lighthouserc.json
    uploadArtifacts: true

# .lighthouserc.json
{
  "ci": {
    "assert": {
      "categories:accessibility": ["error", {"minScore": 0.9}]
    }
  }
}

Note on Lighthouse scores: A score of 90/100 does not mean 90% of WCAG criteria pass. It means the weighted score of Lighthouse's audits hits 90. You can have a 90+ score and still have real barriers. Use Lighthouse as a directional signal, not a compliance certificate.

Pa11y (Batch Testing at Scale)

Pa11y is designed for bulk URL testing — useful for auditing all pages of your app, not just the homepage.

npm install -g pa11y pa11y-ci

pa11y-ci config for EAA (WCAG 2.1 AA):

{
  "standard": "WCAG2AA",
  "runners": ["axe", "htmlcs"],
  "timeout": 30000,
  "wait": 2000,
  "urls": [
    "https://yourapp.com",
    "https://yourapp.com/login",
    "https://yourapp.com/dashboard",
    "https://yourapp.com/settings/profile",
    "https://yourapp.com/checkout"
  ],
  "threshold": 0,
  "ignore": [
    "WCAG2AA.Principle1.Guideline1_4.1_4_3.G18.Fail"
  ]
}

Run:

pa11y-ci --config .pa11yci.json --reporter json > pa11y-report.json

WAVE (Web Accessibility Evaluation Tool)

WAVE is a visual debugging tool from WebAIM. It's not CI-automatable in the same way, but it provides uniquely readable visual output — errors, alerts, and structural markers overlaid directly on the rendered page. Use it during development, not just CI.

The WAVE browser extension works in Firefox and Chrome. WAVE API is available for programmatic access (paid, but useful for thorough pre-release audits).

Contrast Checker Tools

EAA Art.4 / WCAG 2.1 SC 1.4.3 requires:

• Normal text: contrast ratio ≥ 4.5:1
• Large text (18pt or 14pt bold): ratio ≥ 3:1
• WCAG 2.1 SC 1.4.11 (Non-text contrast): UI components and graphics ≥ 3:1

For design-time checking:

• Colour Contrast Checker (EU-hosted)
• Figma plugin: Contrast (free, checks all text layers)
• Stark (paid, but excellent for component-level contrast and color blindness simulation)

For coded UI:

# axe catches most contrast failures automatically
# For custom color schemes, use:
npm install --save-dev jest-axe

Layer 2: Manual Testing with Assistive Technology (Catches the Rest)

Automated tools cannot test what users experience. Manual testing with real assistive technology catches the other 70-80% of issues: focus traps in modals, screen-reader announcements that don't make sense in context, keyboard navigation that technically works but is confusing, dynamic content updates that go unannounced.

For EAA compliance, EN 301 549 Chapter 9 (web) and Chapter 11 (software) both require that you test with assistive technology — because the standard tests "functional performance criteria," not just code-level attributes.

Testing with NVDA (Windows Screen Reader)

NVDA (NonVisual Desktop Access) is the most widely used free screen reader. It's Windows-only, open-source, and your first choice for Windows-based manual testing.

Download: nvaccess.org — free, no registration.

Key NVDA + Chrome test procedure:

1. Enable NVDA, open Chrome
2. Navigate to your app URL
3. Press Tab — verify focus lands on skip-to-content link
4. Activate skip link — verify focus jumps to main content
5. Browse by heading (H key) — verify heading hierarchy makes sense
6. Browse by form field (F key) — verify each field has a label
7. Navigate into a modal (Escape) — verify focus trap works, Escape closes
8. Submit a form with errors — verify error messages are announced
9. Test dynamic content: data tables, live regions, pagination

Critical failure patterns that NVDA finds:

• "Button" announced with no name (icon-only buttons with no aria-label)
• Form error messages not announced (error injected into DOM but no live region)
• Modal does not trap focus (Tab cycles out of modal to background page)
• Data table cells read without row/column headers
• Dynamic page updates (SPA route changes) not announced

Testing with JAWS (Windows, Enterprise)

JAWS (Job Access With Speech) is the dominant commercial screen reader in enterprise. Many accessibility audits specify JAWS testing because it is the most common AT in corporate and government settings.

Recommended pairing: JAWS 2024 + Chrome or Edge (Chromium).

JAWS testing procedure is similar to NVDA. Key differences:

• JAWS handles some ARIA patterns differently — especially aria-live and combobox patterns
• JAWS virtual cursor vs. application mode switching is a common failure point

Testing with VoiceOver (macOS / iOS)

VoiceOver is Apple's built-in screen reader — zero install required.

macOS VoiceOver:

• Enable: Cmd+F5 (or System Settings → Accessibility → VoiceOver)
• Navigate: VO+Right Arrow (next element), VO+Space (activate)
• Web Rotor: VO+U — navigate by headings, links, form controls, landmarks

iOS VoiceOver (critical for EAA Art.13 + EN 301 549 Chapter 11.x for mobile web):

• Enable: Triple-tap Home/Side button (Settings → Accessibility → Accessibility Shortcut)
• Single-tap to explore, double-tap to activate, three-finger swipe to scroll
• Swipe right/left to navigate elements

VoiceOver-specific failure patterns:

• <title> is the first thing VoiceOver announces on page load — empty or duplicated titles
• iOS touch targets: minimum 44×44pt (WCAG 2.1 SC 2.5.5, EN 301 549 §5.9)
• accessibilityLabel on custom UIKit components (React Native bridge leaks)

Testing with TalkBack (Android)

TalkBack is Google's screen reader for Android.

• Enable: Settings → Accessibility → TalkBack
• Navigate: Swipe right (next element), double-tap (activate), swipe up+right (back)

For React Native / Flutter apps targeting EAA: test TalkBack on Android 12+ (API 31+), the most common version in the enterprise Android fleet.

Keyboard-Only Navigation Testing

This is the fastest, most reliable manual test you can do without a screen reader. Every interactive element on every page must be reachable and operable by keyboard alone (WCAG 2.1 SC 2.1.1, EN 301 549 §9.2.1).

Procedure:

1. Disconnect mouse
2. Tab through entire page from top to bottom
3. Verify: every interactive element receives focus (visible focus indicator per WCAG 2.4.7)
4. Verify: you can activate every control with Enter/Space
5. Verify: modals trap focus; Escape closes them
6. Verify: dropdown menus are operable with arrow keys
7. Verify: date pickers, rich text editors, and carousels have keyboard support

Common failures:

• onClick handlers on <div> elements with no keyboard equivalent
• Focus indicator removed with outline: none in CSS
• Tooltips only triggered on hover (not keyboard focus)
• Autocomplete dropdowns not operable with arrow keys

Layer 3: EN 301 549 Coverage Mapping

EN 301 549 v3.2.1 (the current version, harmonized under EAA via Commission Implementing Decision C(2021) 7167) maps WCAG 2.1 AA into its structure. For web services (Chapter 9), WCAG 2.1 AA is embedded verbatim. Here is a practical mapping to testing approach:

Critical gap: Automated tools have zero coverage for SC 1.4.4 (Resize Text), SC 1.4.10 (Reflow), SC 2.4.11 (Focus Appearance WCAG 2.2), and SC 1.4.13 (Content on Hover/Focus). These require manual testing.

Layer 4: CI/CD Integration Architecture

Here is a complete CI/CD testing architecture that gives you defense-in-depth coverage for EAA:

┌─────────────────────────────────────────────────────────────────┐
│  Pull Request Pipeline                                           │
│                                                                 │
│  1. Unit Tests ─────────────────────────┐                       │
│  2. axe-core + Playwright (per page) ───┤─── Block merge on fail│
│  3. Pa11y batch (10 key URLs) ──────────┘                       │
│                                                                 │
│  Staging Deploy Pipeline                                        │
│                                                                 │
│  4. Lighthouse CI (score gate ≥90) ─────┐                      │
│  5. Full Pa11y crawl (all pages) ───────┤─── Alert on fail      │
│  6. axe API report (JSON artifact) ─────┘                      │
│                                                                 │
│  Weekly Manual Testing                                          │
│                                                                 │
│  7. NVDA + Chrome (10 user flows)                               │
│  8. VoiceOver + Safari (5 flows)                                │
│  9. Keyboard-only nav (all pages)                               │
│  10. 200% zoom reflow test                                      │
│  11. Update VPAT / Accessibility Statement                      │
└─────────────────────────────────────────────────────────────────┘

GitHub Actions implementation:

# .github/workflows/accessibility.yml
name: Accessibility Tests

on:
  push:
    branches: [main, staging]
  pull_request:
    branches: [main]

jobs:
  axe-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
      
      - name: Install dependencies
        run: npm ci
      
      - name: Start app
        run: npm run start &
        env:
          NODE_ENV: test
      
      - name: Wait for app
        run: npx wait-on http://localhost:3000 --timeout 30000
      
      - name: Run axe accessibility tests
        run: npx playwright test accessibility/
      
      - name: Run Pa11y batch
        run: |
          npx pa11y-ci \
            --config .pa11yci.json \
            --reporter json > pa11y-results.json
      
      - name: Upload accessibility report
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: accessibility-reports
          path: |
            pa11y-results.json
            playwright-report/

Fail fast vs. warn: For EAA compliance, configure axe to fail the build on critical and serious violations. Downgrade moderate and minor to warnings — they surface in your audit trail but don't block delivery.

// Playwright test helper
const BLOCK_VIOLATIONS = ['critical', 'serious'];
const WARN_VIOLATIONS = ['moderate', 'minor'];

async function checkAccessibility(page: Page, url: string) {
  await page.goto(url);
  const results = await new AxeBuilder({ page })
    .withTags(['wcag2a', 'wcag2aa', 'wcag21aa'])
    .analyze();
  
  const blocking = results.violations.filter(v => 
    BLOCK_VIOLATIONS.includes(v.impact)
  );
  const warnings = results.violations.filter(v => 
    WARN_VIOLATIONS.includes(v.impact)
  );
  
  if (warnings.length > 0) {
    console.warn(`⚠️  ${warnings.length} moderate/minor violations on ${url}`);
    warnings.forEach(v => console.warn(`  - ${v.id}: ${v.description}`));
  }
  
  expect(blocking).toEqual([]);
}

Layer 5: Audit Documentation for EAA Art.13

Market surveillance under EAA Art.19 means authorities can request your compliance documentation. Having a clean Lighthouse score is not enough — you need a paper trail.

Accessibility Statement (Required)

Most EU member states require an Accessibility Statement as part of national EAA transposition. The statement must cover:

• Which standard you conform to (EN 301 549 v3.2.1, which embeds WCAG 2.1 AA)
• Parts of your service that do not fully conform, and why
• Accessible alternatives where barriers exist
• Contact mechanism for users to report barriers
• Enforcement procedure (how users can escalate)

Template structure (following Web Accessibility Directive model, applicable to EAA):

Accessibility Statement for [Your Service Name]

Conformance status: [Fully conformant / Partially conformant / Non-conformant]
Standard: EN 301 549 v3.2.1 (which incorporates WCAG 2.1 Level AA)

We are committed to making [Service] accessible in accordance with Directive 2019/882 
(European Accessibility Act) and [national transposition law].

Non-accessible content:
- [Feature]: [Reason — disproportionate burden / content outside scope / pending fix]

Contact: accessibility@yourcompany.com
Enforcement procedure: [Link to national authority complaint mechanism]

This statement was last reviewed: [Date]

Voluntary Product Accessibility Template (VPAT / ACR)

A VPAT (Voluntary Product Accessibility Template) — or its output, an ACR (Accessibility Conformance Report) — is a structured declaration of which WCAG success criteria your product meets, partially meets, or does not support.

Format: VPAT 2.5 WCAG (international) or VPAT 2.5 EU (specifically for EU standards including EN 301 549).

VPAT ACRs are required by some enterprise customers as part of their own EAA supply chain compliance. If you're selling B2B to EU enterprises, expect to produce one.

Where to get the template: itic.org/policy/accessibility/vpat — free download.

Testing Log / Audit Trail

Your testing log should capture, for each release or audit cycle:

## Accessibility Audit — [Date] — [Version/Release]

### Automated Testing
- Tool: axe-core 4.9.x via Playwright
- Pages tested: [list]
- Critical violations: 0
- Serious violations: 0
- Report: [link to CI artifact]

### Manual Testing
- Screen reader: NVDA 2024.x + Chrome 124
- Keyboard testing: [tester name], [duration]
- Flows tested: Login, Dashboard, Settings, Checkout
- Findings: [list — or "No new findings"]

### Outstanding Issues
- [Issue ID]: [Description] — [Impact] — [Target resolution date]

### Sign-off
- Tester: [Name]
- Date: [Date]
- Scope: [What is/isn't covered]

Store these in your repo under /docs/accessibility-audits/. They are your evidence in an enforcement inquiry.

EU-Native vs. US-Native Testing Tools: CLOUD Act Consideration

A pragmatic note on tooling choice for EAA compliance. Most testing tools are open-source and self-hostable, so CLOUD Act exposure is not a concern. However, if you are using SaaS-based accessibility audit platforms (Deque axe DevTools Pro, Level Access Platform, Siteimprove, Crownpeak), you are uploading page screenshots, rendered DOM content, and potentially PII from test environments.

EU-native options:

• axe-core (Deque, US-based) — but the open-source engine runs locally, no data upload
• Pa11y (open-source, self-hosted) — no external data transfer
• Monsido (Copenhagen, Denmark) — EU-based SaaS audit platform, GDPR-native
• Silktide (UK-based, post-Brexit) — European but outside EU legal jurisdiction

If your test environment contains real user data or production-equivalent data, prefer self-hosted or EU-based audit tooling to avoid GDPR Art.44 cross-border transfer issues.

Common EAA Testing Failures by Component Type

Modals and Dialogs

• Focus not trapped: Tab escapes modal to background page → WCAG 2.1 SC 2.1.2 failure
• Focus not sent to modal on open: aria-modal="true" present but focus stays behind → Screen readers browse background
• Escape doesn't close: Requirement per ARIA modal pattern, not directly WCAG but expected behavior
• No accessible name: role="dialog" without aria-label or aria-labelledby

<!-- Correct -->
<div role="dialog" aria-modal="true" aria-labelledby="modal-title" tabindex="-1">
  <h2 id="modal-title">Confirm deletion</h2>
  <!-- content -->
  <button>Cancel</button>
  <button>Delete</button>
</div>

Data Tables

• Missing headers: <table> without <th> elements → WCAG 1.3.1 failure
• No scope attribute: Column/row headers without scope="col" or scope="row" → axe td-headers-attr violation
• Complex tables: Multi-row/column headers need id + headers attributes — axe won't catch if partially wrong

Forms

• Error messages not associated: Error text injected into DOM without aria-describedby link to input → NVDA won't announce errors
• Placeholder as label: placeholder is not a label — disappears on input, fails WCAG 1.3.5 and 2.4.6
• Required fields: required attribute present but not announced → add aria-required="true" for legacy AT compatibility
• Autofill: No autocomplete attributes where applicable → fails WCAG 1.3.5 (Identify Input Purpose)

Single Page App Route Changes

SPA route changes are invisible to screen readers by default — the URL changes, content updates, but nothing is announced.

// React: announce route changes to screen readers
import { useEffect, useRef } from 'react';
import { useLocation } from 'react-router-dom';

function RouteAnnouncer() {
  const location = useLocation();
  const announcerRef = useRef<HTMLDivElement>(null);
  
  useEffect(() => {
    if (announcerRef.current) {
      // Update page title first, then announce
      const pageTitle = document.title;
      announcerRef.current.textContent = `Navigated to ${pageTitle}`;
    }
  }, [location]);
  
  return (
    <div
      ref={announcerRef}
      role="status"
      aria-live="polite"
      aria-atomic="true"
      className="sr-only"
    />
  );
}

EAA Testing Checklist (Pre-Release)

Before each release that changes user-facing content:

Automated (CI gate — must pass):

• axe-core: 0 critical violations, 0 serious violations
• Pa11y batch: 0 WCAG2AA errors on all key user flows
• Lighthouse accessibility score ≥ 90 on homepage + 3 key pages

Manual (weekly or per-release):

• Keyboard-only navigation through all primary user flows
• NVDA + Chrome: Login, Dashboard, Settings, Checkout flows
• VoiceOver + Safari: same flows on macOS
• 200% zoom: no content clipping or horizontal scroll
• 320px viewport: no horizontal scroll (reflow — WCAG 1.4.10)
• Color-blind simulation: Deuteranopia, Protanopia (Chrome DevTools rendering)

Documentation:

• Accessibility Statement updated with date
• Audit log entry created with findings
• Outstanding issues tracked with target dates
• VPAT/ACR current (B2B customers may request)

What Market Surveillance Authorities Will Check

EAA Art.19 gives national authorities the right to inspect your service. Based on how enforcement under the Web Accessibility Directive (2016/2102) has played out in the public sector, expect:

1. Accessibility Statement exists and is published — typically linked from your footer
2. Statement contains required elements — conformance status, exceptions, contact, complaint mechanism
3. Feedback mechanism works — they will likely test your accessibility contact email
4. Basic automated scan clean — they will run an automated scan; critical violations are immediate flags
5. Manual spot check — keyboard navigation, screen-reader navigation of key flows
6. Documentation on request — audit logs, remediation plans for known gaps

The Art.22 microenterprise exemption (fewer than 10 employees AND annual turnover/balance sheet below EUR 2 million) may exempt you from the statement requirement. But if you are above those thresholds, the Accessibility Statement and feedback mechanism are the minimum bar before any technical assessment.

Key Takeaways

1. No tool covers everything. Use axe-core + NVDA + keyboard testing as your minimum three-layer stack.
2. Automate what you can, but test manually. Automated tools catch ~20-30% of WCAG issues. The rest requires human testing with AT.
3. EN 301 549 Chapter 9 = WCAG 2.1 AA for web. If you pass WCAG 2.1 AA, you have a strong presumption of EAA conformity for web services.
4. Document everything. Accessibility Statement + audit logs + VPAT is your evidence package for EAA Art.19 market surveillance.
5. CI/CD gates protect you. Blocking on critical and serious axe violations means you don't regress.
6. EAA Art.13 is ongoing, not a one-time audit. You need procedures to ensure continued conformity — CI/CD integration plus quarterly manual testing is a defensible program.

EU-EAA-DEVELOPER-2026 Series:

• Post #1/6: European Accessibility Act (Directive 2019/882) — SaaS & Web App Guide
• Post #2/6: WCAG 2.1 AA Technical Implementation for EAA
• Post #3/6: Complete SaaS Developer Guide WCAG EN 301 549
• Post #4/6: EAA Mobile App Accessibility — iOS, Android & PWA
• Post #5/6: EAA Testing & Audit Guide (this post)
• Post #6/6: EAA Compliance Finale — Complete Developer Toolkit & Enforcement Timeline (coming soon)