Playwright and axe: a check that can fail

Source code is not the rendered page

A template may look semantically correct while the browser shows low-contrast text or a button without a name. The CSS cascade, menu and cookie banner affect the result. Check the served page with the stylesheets and fonts actually loaded.

The small check below visits the same URL in two viewports, runs axe and exits with failure on serious or critical findings. It also checks whether page content extends beyond the viewport. These widths are test conditions, not a claim about visitors’ devices.

An executable minimal check

Save the code as axe-check.mjs. Install playwright and axe-core, then Chromium with npx playwright install chromium. Run node axe-check.mjs http://127.0.0.1:8769/engineering/. The URL must serve a real HTML page; HTTP errors also stop the check.

import { chromium } from 'playwright';
import { createRequire } from 'node:module';

const require = createRequire(import.meta.url);
const browser = await chromium.launch();
let failed = false;
try {
  for (const width of [1440, 390]) {
    const page = await browser.newPage({ viewport: { width, height: 900 } });
    const response = await page.goto(process.argv[2], { waitUntil: 'load' });
    if (!response?.ok()) throw new Error(`HTTP ${response?.status()}`);
    await page.evaluate(() => document.fonts.ready);
    await page.addScriptTag({ path: require.resolve('axe-core/axe.min.js') });
    const results = await page.evaluate(() => axe.run(document));
    const blocking = results.violations.filter(
      issue => ['serious', 'critical'].includes(issue.impact)
    );
    const overflow = await page.evaluate(
      () => document.documentElement.scrollWidth > innerWidth
    );
    console.log(JSON.stringify({ width, blocking, overflow,
                                incomplete: results.incomplete }));
    failed ||= blocking.length > 0 || overflow;
    await page.close();
  }
} finally {
  await browser.close();
}
process.exitCode = failed ? 1 : 0;

Deliberately trigger the failure path

The local functional test serves a small accessible fixture, then the same fixture with a button lacking an accessible name. The first command must succeed; the second must fail and report button-name. This proves that the process does not merely print axe findings while remaining green. Another fixture returns an HTTP error.

For Engineering, the verifier extends this minimal check to all language editions, open and closed banners, links, SEO and code contrast. Automated checks and deliberate mutations run only locally. They do not modify any publicly served page.

Automated checks have limits

Zero serious or critical findings does not mean complete WCAG conformance. Review incomplete results and check keyboard interaction, focus order, zoom, link clarity and alternative-text quality. axe cannot determine whether a technical explanation is understandable or a translation is idiomatic.

The minimal check does not open menus or close banners. Interactive states need additional steps and fresh scans. Pin tool versions in the lockfile and retain findings alongside screenshots; an image alone proves neither contrast nor usability.