65 / 95 · 05 Performance & Chaos Engineering · Load Testing LLM Endpoints← prev⊞ allnext →☰ Read as one page
10.6CI Integration for LLM Load Tests
# .github/workflows/llm-performance.yml
name: LLM Performance Gate
on:
push:
branches: [main]
paths:
- 'src/ai/**' # Only run when AI-related code changes
- 'prompts/**'
jobs:
llm-load-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Run LLM load test
uses: grafana/k6-action@v0.4.0
with:
filename: tests/performance/k6-llm-load-test.js
flags: --out json=llm-results.json
env:
LLM_API_KEY: ${{ secrets.LLM_API_KEY }}
- name: Check cost threshold
run: |
python scripts/check_llm_test_cost.py llm-results.json --max-cost 10.00
- name: Upload results
if: always()
uses: actions/upload-artifact@v4
with:
name: llm-load-test-results
path: llm-results.json
LLM load testing requires a mindset shift from "how many requests per second" to "how does cost, latency, and quality behave as concurrency increases." The tools are the same (k6, Locust), but the metrics and constraints are fundamentally different.