3.2What to Automate vs What to Keep Manual

The Decision Framework

For each test scenario, evaluate it against four criteria:

Criterion	Automate If...	Keep Manual If...
Repetition	Executed more than 3 times per release cycle	One-time or rare execution
Stability	The feature is stable and unlikely to change frequently	The feature is in active flux (UI redesign, requirements changing)
Determinism	The expected result is objectively verifiable	The result requires human judgment (visual appeal, "feels right")
Risk	High-risk area where regression would be costly	Low-risk area where a missed regression is tolerable

What to Automate First (High Priority)

Category	Why	Examples
Smoke tests	Verify the application is up and core features work after every deployment	Login, homepage loads, main navigation works
Critical user journeys	These paths generate revenue or affect the most users	Checkout, registration, search + purchase
Regression tests for fixed bugs	Ensure resolved bugs do not return	Every P1/P2 bug should get a regression test
Data validation	Repetitive, error-prone when done manually	API response schemas, database constraints, calculations
Cross-browser/device matrix	Impractical to test manually across all combinations	Top 5 browser-device combinations

What to Keep Manual (Low Automation Priority)

Category	Why	Examples
Exploratory testing	Requires creativity, intuition, and context-switching	"What if I do something unexpected here?"
Usability assessment	Requires human judgment about user experience	"Is this workflow confusing?"
Visual design review	Subtle visual differences require human perception	"Does this layout feel balanced?"
One-time setup verification	Automating costs more than doing it once manually	First-time database migration
Rapidly changing features	Automation will break immediately and need constant rewriting	Feature in active prototype phase

The Gray Zone

Some tests fall between clear-cut automate and clear-cut manual. Use the ROI calculation below to decide.