2.2The Tiered Device Strategy

Tier	Selection Criteria	Coverage Target	Testing Depth
Tier 1 (3-5 devices)	Top devices from analytics, latest OS	50-60% of users	Full regression, every release
Tier 2 (5-10 devices)	Popular mid-range, one version behind	25-30% of users	Critical paths, weekly
Tier 3 (10-20 devices)	Long-tail devices, oldest supported OS	10-15% of users	Smoke tests, monthly
Edge cases (as needed)	Foldables, tablets, RTL locales	<5% of users	Targeted testing for specific features

Building Your Matrix

The process starts with analytics, not assumptions:

# Example: building your device matrix from analytics data
import analytics

def build_device_matrix(min_coverage=0.90):
    """Select minimum device set covering 90% of users."""
    devices = analytics.get_device_breakdown(days=30)
    # Sort by user share descending
    devices.sort(key=lambda d: d["user_share"], reverse=True)

    selected = []
    cumulative = 0.0
    for device in devices:
        selected.append(device)
        cumulative += device["user_share"]
        if cumulative >= min_coverage:
            break

    return selected

# Output example:
# [
#   {"model": "iPhone 15", "os": "iOS 18", "share": 0.18},
#   {"model": "iPhone 14", "os": "iOS 17", "share": 0.12},
#   {"model": "Samsung Galaxy S24", "os": "Android 14", "share": 0.09},
#   {"model": "iPhone 13", "os": "iOS 17", "share": 0.08},
#   {"model": "Samsung Galaxy A54", "os": "Android 14", "share": 0.07},
#   ... # ~12 devices to reach 90% coverage
# ]

Enriched Device Matrix

def build_enriched_matrix(min_coverage=0.90):
    """Build matrix with testing metadata for each device."""
    base_matrix = build_device_matrix(min_coverage)

    for device in base_matrix:
        device["tier"] = assign_tier(device["share"])
        device["test_scope"] = get_test_scope(device["tier"])
        device["frequency"] = get_test_frequency(device["tier"])
        device["farm_available"] = check_device_farm_availability(device["model"])

    return base_matrix

def assign_tier(share):
    if share >= 0.05:
        return 1
    elif share >= 0.02:
        return 2
    else:
        return 3

def get_test_scope(tier):
    scopes = {
        1: "full_regression",
        2: "critical_paths",
        3: "smoke_test",
    }
    return scopes.get(tier, "smoke_test")

def get_test_frequency(tier):
    frequencies = {
        1: "every_pr",
        2: "weekly",
        3: "monthly",
    }
    return frequencies.get(tier, "monthly")