Atlas › 05 Performance & Chaos Engineering › LLM Performance Metrics☰ Read as one page
LLM Performance Metrics
9.1Why LLM Performance Is DifferentLLM-backed features introduce performance characteristics that traditional load testing does not account for. Unlike a REST API that…9.2Key Metrics for LLM Performance9.3Understanding TTFT vs Total Generation TimeTTFT is what determines perceived responsiveness. A user staring at a blank screen for 3 seconds feels slow, even if the total response…9.4Measuring LLM MetricsFor non-streaming endpoints, you can only measure total generation time directly. TTFT must be estimated:9.5Context Window EconomicsThe size of the prompt directly affects latency and cost. Understanding this relationship is critical for performance optimization:9.6Setting SLOs for LLM FeaturesLLM SLOs should be defined separately from traditional API SLOs:9.7Benchmarking Across ProvidersWhen evaluating LLM providers, run standardized benchmarks: