What is a token, and what does tokens per second mean?
How a model splits text, reading your prompt versus writing the answer, and what measured speeds feel like.
Short answer
A token is a piece of text, usually a word or part of one. Tokens per second is how fast a model writes them.
- Example: The 10-word sentence below is 13 tokens.
- Reading: A person reads about 5 tokens a second.
- Measured: A Mac mini M4 writes 24.1 tokens/s with a 7B model, faster than you read.
Published llama.cpp result (llama.cpp Apple Silicon summary · community result, read Oct 25, 2026)
The idea
Think of a typesetter with a box of ready-made pieces: whole common words, pieces of longer words, punctuation. To print a sentence they pick one piece at a time. A language model works the same way. Its tokenizer has a fixed box of pieces, the tokens, and the model reads and writes text only as a row of them, one piece at a time.
So speed is counted in tokens, not words: tokens per second (tok/s) is how many pieces the model adds to its answer each second.
A sentence in tokens
- Localid 7469
- space, modelsid 4211
- space, writeid 3350
- space, oneid 832
- space, tokenid 4037
- space, atid 520
- space, aid 264
- space, timeid 892
- ,id 11
- space, unbelieid 40037
- vablyid 89234
- space, fastid 5043
- .id 13
Short common words are one token each, and so is the comma. The long word splits in two. The same idea in Portuguese, “Modelos locais escrevem um token por vez, incrivelmente rápido.”, is 16 tokens for 9 words: this tokenizer learned more whole English words than Portuguese ones, so the count per word depends on the language. Here: 1.3 tokens per English word and 1.8 per Portuguese word.
Tokens also set how much a model can keep in mind at once: the context length is counted in tokens, and each token of the conversation takes memory (context length and the KV cache).
Reading vs writing
Every answer has two speeds. llama.cpp's benchmark tool names them: prompt processing (pp), “processing a prompt in batches”, and text generation (tg), “generating a sequence of tokens” (llama.cpp · llama-bench, read Oct 25, 2026).
- 1 · Reading your prompt (pp)
- All tokens at once, in batchesSets the wait before the first word. Grows with the length of what you paste; limited mostly by the chip's compute.
- 2 · Writing the answer (tg)
- One token after anotherThe speed you watch as the text appears. Each token needs the previous one, so it cannot run in parallel; limited mostly by memory speed.
Waiting time ≈ prompt tokens ÷ prompt speed, then writing time ≈ answer tokens ÷ writing speed. Reading a prompt is many times faster per token than writing, because the tokens are processed together. The benchmarks this site cites publish both: llama-bench as pp512 and tg128, and the mlx-lm table as a “Prompt (2048)” and a “Generation (128)” column (mlx-lm · BENCHMARKS.md, read Oct 25, 2026). This site stores only the writing speed so far, so every tokens/s figure on it, here and on model and hardware pages, is writing speed.
What tokens per second feels like
Adults read English silently at about 238 words a minute (non-fiction, average of 190 studies) (Brysbaert, 2019 · reading rate, read Oct 25, 2026). With 1.3 tokens per English word from the sentence above, that is roughly 5 tokens per second: a rough line, not a measurement. A model writing above it is faster than you can read; well below it feels like waiting.
The same 7B file, measured on seven machines
Writing speed in tokens/s; the line is reading pace (about 5 tokens/s)
Published llama.cpp results, read Oct 25, 2026: llama.cpp Apple Silicon summary · community result · llama.cpp CUDA scoreboard · community result
| Machine | Writing · tokens/s (measured) | 500-word answer (s, computed) | Backend | Build |
|---|---|---|---|---|
| Mac mini M1 16GB | 14.15 | 46 | Metal | 8e672ef |
| Mac mini M4 16GB | 24.11 | 27 | Metal | 8e672ef |
| MacBook Air 15-inch M5 16GB | 31.88 | 20 | Metal | c1d0e7a |
| RTX 4060 Ti 8 GB | 64.03 ± 0.53 | 10 | CUDA | 89d10295 |
| Mac mini M5 Pro 24GB | 67.59 | 10 | Metal | 60081bb |
| Mac Studio M5 Max 128GB | 119.92 | 5 | Metal | c1d0e7a |
| RTX 4090 24 GB | 188.96 ± 0.05 | 3 | CUDA | 2241453 |
A 500-word answer is about 650 tokens. At the measured speeds it takes about 46 seconds on Mac mini M1 · 16 GB unified and 3 on RTX 4090 · 24 GB VRAM. Every row is one published llama-bench run of one file, with the build named in the table; older Macs in the Apple table were run on an older build to keep them comparable, so they read lower than a current build would. Each machine's page lists its measured runs.
Why memory speed matters
To write each token, the computer reads every weight of the model from memory once. The arithmetic is quick; moving the file is what takes the time. So a smaller file writes faster on the same machine. The mlx-lm maintainers measured one model, Qwen3 4B, at several bit widths on one Apple M4 Max · 64 GB (GPU cores not stated) (mlx 0.29.2.dev20251008+85a8824a8 · mlx-lm 0.28.2 · macOS 26.1): 134.5 tokens/s as MLX q4 and 52.5 as MLX bf16 (mlx-lm BENCHMARKS.md · maintainer result, read Oct 25, 2026).
| File | Writing · tokens/s (measured) |
|---|---|
| MLX q4 | 134.52 |
| MLX q5 | 116.39 |
| MLX q6 | 104.68 |
| MLX q8 | 86.907 |
| MLX bf16 | 52.47 |
Across machines, the one with more memory bandwidth (GB/s) writes faster, whether it is a Mac or a graphics card. How close each machine comes to its bandwidth, in a table: LLM speed and memory bandwidth. Which file format to pick on a Mac: GGUF vs MLX.
Questions
Is a token the same as a word?
Often, not always. Common short words are one token; long or rare words split into two or three. In the example above, “unbelievably” is two tokens. Portuguese text split by this tokenizer takes more tokens per word (1.8 against 1.3 in English), because its vocabulary has fewer whole Portuguese words.
Do all models use the same tokens?
No. Each model family ships its own tokenizer and vocabulary; this one has 128,256 entries. The same sentence can be a few tokens longer or shorter in another model, so compare speeds of the same model only.
Why does the answer start late with a long document?
Because the whole prompt is read before the first word appears. Reading is much faster per token than writing, but a long document has many thousands of tokens, so the wait grows with its length.
Does this site estimate tokens per second for my computer?
No. A speed appears only where a source measured that machine, with the model, file, program and build it used. Model and hardware pages show those rows under Measured speed.