Tokens per second buys you nothing until the answer is 2,000 tokens long

Google Gemini 3.8 Flash generates 311 tokens per second. NVIDIA Nemotron 3.5 Lightning generates 281. If throughput is how you pick a serving model, those are the same model — a 10 percent difference, well inside what a re-measurement moves. Ask each for a 300-token answer and one returns in 1.5 seconds and the other in 16.

The difference is entirely time to first token: 0.47s for Nemotron 3.5 Lightning against 15.10s for Gemini 3.8 Flash, both measured by Artificial Analysis. Prices and listings in this piece come from OpenRouter; the latency and throughput figures do not, and it is worth keeping the two straight.

Wall-clock time for one answer is not a speed, it is a sum: TTFT + output_tokens / rate. The first term is fixed per request. The second scales with how much the model has to say. Which term dominates is decided by the length of the answer, and for most production answers it is the first one — the term that throughput rankings do not contain.

Here is the same set of models at two answer lengths. Both columns are arithmetic on the measured TTFT and tokens-per-second figures; nothing here is a fresh benchmark.

The ordering inverts. Mercury 2 is the fastest generator in the measured catalogue at 958 tokens per second, and on a 300-token answer it places seventh of ten — four times slower than Nemotron 3.5 Lightning. At 3,000 tokens it is first.

So the useful question is not which model is faster. It is how long the answer has to be before the fast generator wins.

Mercury 2 pays off its 5.86s start after 2,143 output tokens. Ling 3.0 Flash after 1,938. Trinity Large Thinking after 1,209. Those are reachable — a long document summary, a full generated file. Gemini 3.5 Flash Lite needs 8,808 tokens. Gemini 3.7 Flash needs 17,786. Gemini 3.8 Flash needs 42,617, which is not an answer, it is a corpus. Against Nemotron 3.5 Lightning, its higher throughput never pays for itself on anything a person is waiting for.

That is the part you can act on, and it depends on your traffic rather than on a leaderboard. Chat turns, tool-call arguments, classification labels, routing decisions, structured extraction — these run to the low hundreds of output tokens, and at that length the ranking is almost purely a TTFT ranking. Agent loops are worse, because they pay the fixed term once per step. Ten tool calls of 300 tokens each against Gemini 3.8 Flash is 151 seconds of pure startup before any of the generation. The same loop on Nemotron 3.5 Lightning costs 4.7 seconds of startup.

Two things this does not say.

It does not say the slow-starting models are badly served. Some of what reads as latency is the model thinking. I left the top of the measured TTFT range out of this arithmetic deliberately: the entries up there are extended-reasoning configurations where the measured first-token time covers an entire deliberation phase, and the listing does not tell you which effort setting was measured. Treating two minutes of reasoning as a serving problem is how you end up changing providers to fix a model setting.

It also does not say tokens per second is worthless. It says it is the second term. If you generate long-form output — a translated document, a large diff, a full report — the crossover runs the other way and Mercury 2 is the right answer. The error is using one number for both jobs.

One caveat on the data itself. Only 61 of the 444 models listed on OpenRouter carry a published TTFT at all, 13 percent, and across the measured set time to first token spans 0.60s to 55.1s from the tenth to the ninetieth percentile around a median of 1.70s. The other 383 listings are unmeasured, not fast. And this is not a stable table: 24 of the measured models moved their TTFT by more than 10 percent in the three days to 16 September. A first-token figure you read last month is not one you can plan against.

The check is cheap and takes a minute. Take the median output length of your actual traffic — the median, not the longest — and put it through TTFT + n / rate for the two or three models you are choosing between. If your median answer is 300 tokens, you are buying a first-token time, and the throughput column is decoration.

Both projects are on GitHub and PyPI. Install them.