AI Agents for Accessibility Audits

Beyond Rule-Based Scanning

AI agents can go beyond rule-based scanning to perform qualitative accessibility assessments that approach human auditor capability. While axe-core checks "is alt text present?", an AI agent can evaluate "is this alt text actually descriptive?"

What AI Agents Add Beyond axe-core

AI Accessibility Audit Prompt Pattern

You are an accessibility auditor reviewing a web page. You have access to:
1. A screenshot of the page
2. The page's accessibility tree (from Playwright)
3. The page's HTML source

Evaluate the page for WCAG 2.1 AA compliance. For each issue found:
- Identify the WCAG success criterion violated
- Rate severity: Critical / Major / Minor
- Describe the user impact (which users are affected and how)
- Provide a specific remediation recommendation

Focus on issues that automated tools typically miss:
- Is the reading order logical?
- Do error messages clearly explain what went wrong and how to fix it?
- Are interactive elements predictable in their behavior?
- Is equivalent information available to users who cannot see the visual layout?
- Are animations respectful of users who prefer reduced motion?

Enhanced Prompt for Specific Assessments

Review the following alt text for accuracy and quality. For each image,
evaluate whether the alt text:

1. Accurately describes the image content
2. Conveys the same information a sighted user would get
3. Is neither too brief (losing meaning) nor too verbose (annoying to listen to)
4. Avoids redundant phrases like "image of" or "picture of"
5. Is appropriate for the context (e.g., a product image needs different
   alt text than a decorative hero image)

Images and their current alt text:
- Image 1: [product photo of red running shoes] alt="shoe"
- Image 2: [team photo of 12 people] alt="team photo"
- Image 3: [graph showing sales growth from $1M to $5M] alt="graph"
- Image 4: [decorative wave pattern] alt="wave"

For each, rate the current alt text as GOOD, NEEDS IMPROVEMENT, or POOR,
and provide a suggested replacement.

Automated AI Accessibility Crawl

# scripts/ai_accessibility_audit.py
import asyncio
from playwright.async_api import async_playwright

async def crawl_and_audit(base_url: str, max_pages: int = 50):
    """Crawl an application and run accessibility audits on each page."""
    async with async_playwright() as p:
        browser = await p.chromium.launch()
        page = await browser.new_page()

        visited = set()
        to_visit = [base_url]
        all_results = []

        while to_visit and len(visited) < max_pages:
            url = to_visit.pop(0)
            if url in visited:
                continue
            visited.add(url)

            try:
                await page.goto(url, timeout=10000)
                await page.wait_for_load_state('networkidle')

                # 1. Run axe-core for rule-based checks
                axe_results = await page.evaluate("""
                    async () => {
                        await new Promise(r => {
                            const s = document.createElement('script');
                            s.src = 'https://cdn.jsdelivr.net/npm/axe-core@4/axe.min.js';
                            s.onload = r;
                            document.head.appendChild(s);
                        });
                        return await axe.run();
                    }
                """)

                # 2. Get accessibility tree for AI analysis
                a11y_tree = await page.accessibility.snapshot()

                # 3. Take screenshot for AI vision analysis
                screenshot = await page.screenshot(full_page=True)

                # 4. Collect links for crawling
                links = await page.evaluate(f"""
                    () => Array.from(document.querySelectorAll('a[href]'))
                        .map(a => a.href)
                        .filter(href => href.startsWith('{base_url}'))
                """)
                to_visit.extend([l for l in links if l not in visited])

                all_results.append({
                    'url': url,
                    'axe_violations': axe_results.get('violations', []),
                    'a11y_tree': a11y_tree,
                    'screenshot': screenshot,
                })

            except Exception as e:
                all_results.append({
                    'url': url,
                    'error': str(e),
                })

        await browser.close()
        return all_results

async def generate_ai_report(results, ai_client):
    """Send collected data to an AI model for qualitative analysis."""
    for result in results:
        if 'error' in result:
            continue

        # Combine axe-core findings with accessibility tree
        context = {
            'url': result['url'],
            'automated_violations': [
                {'rule': v['id'], 'impact': v['impact'], 'count': len(v['nodes'])}
                for v in result['axe_violations']
            ],
            'accessibility_tree_summary': summarize_tree(result['a11y_tree']),
        }

        # Send to AI for qualitative review
        response = ai_client.messages.create(
            model="claude-sonnet-4-20250514",
            max_tokens=2000,
            messages=[{
                "role": "user",
                "content": [
                    {
                        "type": "image",
                        "source": {
                            "type": "base64",
                            "media_type": "image/png",
                            "data": base64.b64encode(result['screenshot']).decode(),
                        },
                    },
                    {
                        "type": "text",
                        "text": f"""Review this page for accessibility issues.
                        URL: {context['url']}
                        Automated findings: {context['automated_violations']}
                        Accessibility tree: {context['accessibility_tree_summary']}

                        Focus on qualitative issues that axe-core cannot detect.""",
                    },
                ],
            }],
        )

        result['ai_review'] = response.content[0].text

def summarize_tree(tree, depth=0, max_depth=3):
    """Summarize an accessibility tree for AI consumption."""
    if not tree or depth > max_depth:
        return ""
    summary = f"{'  ' * depth}{tree.get('role', 'unknown')}"
    if tree.get('name'):
        summary += f": {tree['name']}"
    children = tree.get('children', [])
    child_summaries = [summarize_tree(c, depth + 1, max_depth) for c in children]
    return summary + '\n' + '\n'.join(filter(None, child_summaries))

Combining Automated and AI Audits in CI

# .github/workflows/a11y-audit.yml
name: Accessibility Audit
on:
  schedule:
    - cron: '0 6 * * 1'  # Weekly Monday
  workflow_dispatch:

jobs:
  audit:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci && npm run build && npm run start &

      - name: Run automated + AI audit
        run: |
          python scripts/ai_accessibility_audit.py \
            --base-url http://localhost:3000 \
            --max-pages 50 \
            --output audit-report.json

      - name: Generate HTML report
        run: python scripts/generate_a11y_report.py audit-report.json

      - name: Upload report
        uses: actions/upload-artifact@v4
        with:
          name: accessibility-audit
          path: audit-report.html

AI accessibility audits complement, not replace, rule-based tools. Use axe-core for fast, deterministic checks on every PR. Use AI agents for deeper qualitative analysis on a weekly or per-release schedule. Use human experts for the full-spectrum audit that neither automation nor AI can fully replicate.