Skip to content
Local AI Wizard
Your machine
Theme

What is a token, and what does tokens per second mean?

How a model splits text, reading your prompt versus writing the answer, and what measured speeds feel like.

Speed · updated

Short answer

A token is a piece of text, usually a word or part of one. Tokens per second is how fast a model writes them.

  • Example: The 10-word sentence below is 13 tokens.
  • Reading: A person reads about 5 tokens a second.
  • Measured: A Mac mini M4 writes 24.1 tokens/s with a 7B model, faster than you read.

Published llama.cpp result (llama.cpp Apple Silicon summary · community result, read Oct 25, 2026)

The idea

Think of a typesetter with a box of ready-made pieces: whole common words, pieces of longer words, punctuation. To print a sentence they pick one piece at a time. A language model works the same way. Its tokenizer has a fixed box of pieces, the tokens, and the model reads and writes text only as a row of them, one piece at a time.

So speed is counted in tokens, not words: tokens per second (tok/s) is how many pieces the model adds to its answer each second.

A sentence in tokens

  1. Localid 7469
  2. space, modelsid 4211
  3. space, writeid 3350
  4. space, oneid 832
  5. space, tokenid 4037
  6. space, atid 520
  7. space, aid 264
  8. space, timeid 892
  9. ,id 11
  10. space, unbelieid 40037
  11. vablyid 89234
  12. space, fastid 5043
  13. .id 13
10 words, 13 tokens. Each chip is one token with its number in the vocabulary; a dot marks the space that belongs to the token. Split by the Llama 3 tokenizer (128,256 tokens, used by Llama 3.1 8B) with (llama3-tokenizer-js 1.2.0, read Oct 25, 2026)

Short common words are one token each, and so is the comma. The long word splits in two. The same idea in Portuguese, “Modelos locais escrevem um token por vez, incrivelmente rápido.”, is 16 tokens for 9 words: this tokenizer learned more whole English words than Portuguese ones, so the count per word depends on the language. Here: 1.3 tokens per English word and 1.8 per Portuguese word.

Tokens also set how much a model can keep in mind at once: the context length is counted in tokens, and each token of the conversation takes memory (context length and the KV cache).

Reading vs writing

Every answer has two speeds. llama.cpp's benchmark tool names them: prompt processing (pp), “processing a prompt in batches”, and text generation (tg), “generating a sequence of tokens” (llama.cpp · llama-bench, read Oct 25, 2026).

1 · Reading your prompt (pp)
All tokens at once, in batchesSets the wait before the first word. Grows with the length of what you paste; limited mostly by the chip's compute.
2 · Writing the answer (tg)
One token after anotherThe speed you watch as the text appears. Each token needs the previous one, so it cannot run in parallel; limited mostly by memory speed.

Waiting time ≈ prompt tokens ÷ prompt speed, then writing time ≈ answer tokens ÷ writing speed. Reading a prompt is many times faster per token than writing, because the tokens are processed together. The benchmarks this site cites publish both: llama-bench as pp512 and tg128, and the mlx-lm table as a “Prompt (2048)” and a “Generation (128)” column (mlx-lm · BENCHMARKS.md, read Oct 25, 2026). This site stores only the writing speed so far, so every tokens/s figure on it, here and on model and hardware pages, is writing speed.

What tokens per second feels like

Adults read English silently at about 238 words a minute (non-fiction, average of 190 studies) (Brysbaert, 2019 · reading rate, read Oct 25, 2026). With 1.3 tokens per English word from the sentence above, that is roughly 5 tokens per second: a rough line, not a measurement. A model writing above it is faster than you can read; well below it feels like waiting.

The same 7B file, measured on seven machines

Writing speed in tokens/s; the line is reading pace (about 5 tokens/s)

Mac mini M1 16GB14.2 tokens/s
Mac mini M4 16GB24.1 tokens/s
RTX 4060 Ti 8 GB64 tokens/s
RTX 4090 24 GB189 tokens/s

Published llama.cpp results, read Oct 25, 2026: llama.cpp Apple Silicon summary · community result · llama.cpp CUDA scoreboard · community result

Writing speed (llama.cpp tg128, batch 1) of the same Llama 7B Q4_0 file, as published; the time column is computed: 500 words ≈ 650 tokens with the tokenizer above, divided by the measured speed.
MachineWriting · tokens/s (measured)500-word answer (s, computed)BackendBuild
Mac mini M1 16GB14.1546Metal8e672ef
Mac mini M4 16GB24.1127Metal8e672ef
MacBook Air 15-inch M5 16GB31.8820Metalc1d0e7a
RTX 4060 Ti 8 GB64.03 ± 0.5310CUDA89d10295
Mac mini M5 Pro 24GB67.5910Metal60081bb
Mac Studio M5 Max 128GB119.925Metalc1d0e7a
RTX 4090 24 GB188.96 ± 0.053CUDA2241453

A 500-word answer is about 650 tokens. At the measured speeds it takes about 46 seconds on Mac mini M1 · 16 GB unified and 3 on RTX 4090 · 24 GB VRAM. Every row is one published llama-bench run of one file, with the build named in the table; older Macs in the Apple table were run on an older build to keep them comparable, so they read lower than a current build would. Each machine's page lists its measured runs.

Why memory speed matters

To write each token, the computer reads every weight of the model from memory once. The arithmetic is quick; moving the file is what takes the time. So a smaller file writes faster on the same machine. The mlx-lm maintainers measured one model, Qwen3 4B, at several bit widths on one Apple M4 Max · 64 GB (GPU cores not stated) (mlx 0.29.2.dev20251008+85a8824a8 · mlx-lm 0.28.2 · macOS 26.1): 134.5 tokens/s as MLX q4 and 52.5 as MLX bf16 (mlx-lm BENCHMARKS.md · maintainer result, read Oct 25, 2026).

Qwen3 4B on one machine, generation speed over 128 tokens after a 2,048-token prompt (mlx_lm.benchmark), as published.
FileWriting · tokens/s (measured)
MLX q4134.52
MLX q5116.39
MLX q6104.68
MLX q886.907
MLX bf1652.47

Across machines, the one with more memory bandwidth (GB/s) writes faster, whether it is a Mac or a graphics card. How close each machine comes to its bandwidth, in a table: LLM speed and memory bandwidth. Which file format to pick on a Mac: GGUF vs MLX.

Questions

Is a token the same as a word?

Often, not always. Common short words are one token; long or rare words split into two or three. In the example above, “unbelievably” is two tokens. Portuguese text split by this tokenizer takes more tokens per word (1.8 against 1.3 in English), because its vocabulary has fewer whole Portuguese words.

Do all models use the same tokens?

No. Each model family ships its own tokenizer and vocabulary; this one has 128,256 entries. The same sentence can be a few tokens longer or shorter in another model, so compare speeds of the same model only.

Why does the answer start late with a long document?

Because the whole prompt is read before the first word appears. Reading is much faster per token than writing, but a long document has many thousands of tokens, so the wait grows with its length.

Does this site estimate tokens per second for my computer?

No. A speed appears only where a source measured that machine, with the model, file, program and build it used. Model and hardware pages show those rows under Measured speed.