9.3Understanding TTFT vs Total Generation Time

Request sent                      Response complete
    |                                    |
    |--[TTFT]-->|                        |
    |           |---[Token streaming]--->|
    |           |                        |
    |           First token              Last token
    |                                    |
    |--------[Total Generation Time]---->|

TTFT is what determines perceived responsiveness. A user staring at a blank screen for 3 seconds feels slow, even if the total response arrives in 5 seconds. With streaming, a 500ms TTFT followed by 4.5 seconds of token-by-token output feels much faster than a 5-second wait for a complete response.

Rule of thumb:

• TTFT < 1s: Users perceive the response as "instant"
• TTFT 1-3s: Acceptable with a loading indicator
• TTFT > 3s: Users begin to disengage