What do total and active parameters mean in a MoE model?
Why a “35B A3B” model needs the memory of 35B and not of 3B.
Short answer
In a mixture-of-experts (MoE) model, total is every parameter; active is the few it uses for each word. Memory follows the total.
- Total: Sets the memory, because every expert must be loaded.
- Active: Sets the work done for each word.
- Example: Qwen3.5 35B A3B needs about 35 GB in Q4, planned from 35B, not 3B.
Planning estimate with the site's formula, not a measurement. Methodology
How MoE works
In a dense model every parameter takes part in every word. In a MoE model, some layers are split into many “experts”, and a small router picks a few of them for each word. The rest wait in memory for the words they are good at.
- Total parameters
- Everything that has to be loadedSets the memory and the download size.
- Active parameters
- What is used for one wordSets the work per word, and so most of the speed.
Three MoE models
Q4 file, short conversations, on a Mac or PC with 32 GB or more (8 GB reserve; smaller machines reserve less). Planning estimates in whole GB.
| Model | Total (B) | Active (B) | Memory from total (GB) | If planned from active (GB) |
|---|---|---|---|---|
| Qwen3.5 35B A3B | 35 | 3 | 35 | 11 |
| Nemotron 3 Super 120B A12B | 120 | 12 | 98 | 17 |
| Ling 3.0 Flash | 124 | 5.1 | 101 | 12 |
Counts as published in each model card: (Qwen3.5 35B A3B, reviewed Sep 30, 2026) · (Nemotron 3 Super 120B A12B, reviewed Sep 30, 2026) · (Ling 3.0 Flash, reviewed Sep 30, 2026)
Less work per word
Each word reads the active weights from memory, not all of them, so Qwen3.5 35B A3B does about the work of a 3B model per word while holding 35B in memory. The site does not estimate speed: hardware pages show tokens per second only when someone measured that exact machine (fitting is not speed).
Reading “A3B”
Many publishers put both numbers in the name: in “Qwen3.5 35B A3B”, 35B is the total and A3B means about 3B active. When the name has only one number, the model card says whether the model is dense or MoE. On this site, model pages show both counts and plan memory with the total.