16.3AI Test Generation (LLM-Based)

Large language models (as of July 2026: Claude Opus 4.8, GPT-5.5, Gemini 3.1 Pro) can generate Playwright tests from natural language descriptions, user stories, or application specifications.

What Works Well

• Generating boilerplate: "Write a Playwright test for user login with valid and invalid credentials" produces working, well-structured test code
• Converting test cases to code: Given a list of test steps, LLMs produce reasonable automation code
• Explaining and debugging: "Why is this locator flaky?" or "Optimize this test" gets useful analysis
• Refactoring: Converting linear scripts into page objects, extracting fixtures, improving locator strategies

What Does Not Work Yet

• Generating comprehensive test suites from scratch: LLMs miss edge cases, boundary conditions, and domain-specific requirements
• Understanding application state: an LLM alone cannot see the actual DOM or application behavior — closing this gap is exactly what the agent integrations below exist for
• Reliable locator generation for unknown apps: without seeing the real HTML, generated selectors are guesses
• Replacing QA judgment: Knowing what to test still requires human understanding of the product