Browser Testing Strategy

The Three-Tier Approach

Testing every feature on every browser is neither practical nor necessary. The key is knowing what to test everywhere, what to spot-check, and what to skip. A three-tier strategy gives you maximum coverage with minimum effort.

Browser Usage Statistics (Reference)

Key Insight: Engine Matters More Than Browser

Chrome, Edge, Opera, Samsung Internet, and Brave all use the Blink engine. If your app works in Chrome, it will work in Edge with rare exceptions. This means your three actual rendering engines to test against are:

• Blink (Chrome and derivatives) -- ~75% of all users
• WebKit (Safari) -- ~20% of all users
• Gecko (Firefox) -- ~3% of all users

The Three-Tier Strategy

+------------------------------------------------------+
|  Tier 1: Full Regression (every PR)                  |
|  Chrome (latest) + Firefox (latest) + Safari (latest)|
|  These 3 cover ~92% of desktop users                 |
+------------------------------------------------------+
|  Tier 2: Critical Path (weekly / release)            |
|  Chrome Android + Safari iOS + Edge                  |
|  Samsung Internet + Firefox Android                  |
+------------------------------------------------------+
|  Tier 3: Spot Check (monthly / quarterly)            |
|  Older Safari (2 versions back)                      |
|  Chrome on low-end Android                           |
|  UCBrowser / Opera Mini (if emerging market users)   |
+------------------------------------------------------+

Tier 1: Full Regression on Every PR

Run your entire test suite against the latest versions of Chrome, Firefox, and Safari. These three browsers represent the three rendering engines and cover the vast majority of users.

// playwright.config.ts -- Tier 1 configuration
import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
    projects: [
        {
            name: 'chrome',
            use: { ...devices['Desktop Chrome'] },
        },
        {
            name: 'firefox',
            use: { ...devices['Desktop Firefox'] },
        },
        {
            name: 'safari',
            use: { ...devices['Desktop Safari'] },
        },
    ],
});

Tier 2: Critical Paths on Release

Test the most important user journeys (login, search, checkout, payment) on mobile browsers and less common desktop browsers:

// playwright.config.tier2.ts
export default defineConfig({
    projects: [
        {
            name: 'chrome-android',
            use: { ...devices['Pixel 7'] },
        },
        {
            name: 'safari-ios',
            use: { ...devices['iPhone 15'] },
        },
        {
            name: 'edge',
            use: { channel: 'msedge' },
        },
    ],
    // Only run critical path tests
    testMatch: '**/critical-path/**',
});

Tier 3: Spot Checks

Manual or semi-automated checks on older browsers and niche platforms. These do not need to run in CI:

# Quarterly checklist for Tier 3 browsers
# Run manually on BrowserStack/Sauce Labs:
# - Safari 2 versions back: login + main page + checkout
# - Chrome on Android 12 (low-end): page load under 5s
# - Samsung Internet: form submission + file upload
# - Opera Mini: basic content rendering (no JS-heavy features)

Configuring Cross-Browser CI

# .github/workflows/cross-browser.yml
name: Cross-Browser Tests
on:
  pull_request:
    # Tier 1: every PR
  schedule:
    - cron: '0 6 * * 1'  # Tier 2: weekly on Monday
    - cron: '0 6 1 * *'  # Tier 3: monthly on 1st

jobs:
  tier-1:
    if: github.event_name == 'pull_request'
    runs-on: ubuntu-latest
    strategy:
      matrix:
        browser: [chromium, firefox, webkit]
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npx playwright install --with-deps ${{ matrix.browser }}
      - run: npx playwright test --project=${{ matrix.browser }}

  tier-2:
    if: github.event_name == 'schedule' || github.event_name == 'pull_request'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npx playwright install --with-deps
      - run: npx playwright test --config=playwright.config.tier2.ts

  tier-3:
    if: github.event.schedule == '0 6 1 * *'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - name: Run on BrowserStack
        env:
          BROWSERSTACK_USER: ${{ secrets.BS_USER }}
          BROWSERSTACK_KEY: ${{ secrets.BS_KEY }}
        run: npx playwright test --config=playwright.config.tier3.ts

Browser-Specific CI Gotchas

When to Test on Real Browsers vs Engine Equivalents

Playwright's WebKit engine is a close but not identical approximation of Safari. For most web testing, it is sufficient. For Safari-specific features (media playback, WebKit-only CSS, Share Sheet), test on real Safari via BrowserStack.

The three-tier strategy balances thoroughness with practicality. Tier 1 catches 95% of cross-browser bugs. Tier 2 catches the mobile-specific issues. Tier 3 is insurance against long-tail edge cases.