What happens when a model doesn't fit in VRAM?
Offloading layers to the CPU and PC RAM: when it works, what it costs and how the site marks it.
Short answer
It still runs, much more slowly: the layers that don't fit stay in PC RAM and run on the processor.
- Which apps: Those built on llama.cpp, such as Ollama and LM Studio, can split a model this way.
- How far: An 8 GB card with 32 GB of RAM reaches models up to 32 GB, marked “runs slowly”.
- On a Mac: There is no separate graphics memory to offload from; the model must fit in unified memory.
Planning estimate with the site's fit rule, not a measurement. Methodology
How offloading works
A model is a stack of layers. When the file is bigger than the graphics card's memory, llama.cpp and the apps built on it (LM Studio, Ollama) can put as many layers as fit on the card and leave the rest in the computer's normal memory, where the processor runs them. The answer is the same; only the speed changes. (llama.cpp README, read Oct 17, 2026)
It is slower because each new word reads every layer, and the layers in PC RAM are read at the speed of the system memory. The site's profiles list 448 GB/s for the memory on an RTX 5060 Ti 8 GB card and 152 GB/s for the fastest system memory they record (a Snapdragon X2 laptop's LPDDR5x). The part of the work that runs from PC RAM sets the pace (RTX 5060 Ti 8 GB, Surface Laptop X2 Plus). The site never turns this into a speed figure: “runs slowly” is a tier, not a number.
An 8 GB card, model by model
Q4 planning requirement on the card (2 GB reserve included) and the site's tier, with 32 GB of PC RAM assumed or 16 GB listed.
| Model | Parameters (B) | Q4 need on the card (GB) | RTX 4060 8 GB · 32 GB RAM | LOQ 15IRX10 i7-14700HX RTX 5060 16GB 83JE000QUS |
|---|---|---|---|---|
| Qwen3 4B Instruct 2507 | 4 | 5 | Runs well | Runs well |
| Qwen3 8B | 8.2 | 9 | Runs slowly | Runs slowly |
| Qwen3 14B | 14.8 | 14 | Runs slowly | Runs slowly |
| Mistral Small 3.2 24B | 24 | 21 | Runs slowly | Doesn't run |
| Qwen3 30B A3B Instruct 2507 | 30.5 | 25 | Runs slowly | Doesn't run |
| Olmo 3 32B Think | 32 | 26 | Runs slowly | Doesn't run |
Where the limit is
- Runs well
- Up to 8 GB on the card13 reviewed text models on RTX 4060 · 8 GB VRAM.
- Runs slowly
- Up to 32 GB with 32 GB of RAMThe card's 8 GB plus PC RAM minus the system's own 8 GB (25% of the RAM, 2 to 8 GB). 24 reviewed models fall here.
- Less PC RAM, less room
- Up to 20 GB with 16 GB of RAMLOQ 15IRX10 · MTM 83JE000QUS lists 16 GB of RAM, so its slow tier ends sooner: 10 reviewed models.
- When the RAM is unknown
- 32 GB assumed and said soA bare graphics-card profile does not say how much RAM the PC has; the page states the assumption next to every slow row.
On a Mac
A Mac has no separate graphics memory, so there is nothing to offload to. What it has is a default cap on how much memory the GPU may use; the site plans with 75% of the memory as that share. On Mac Studio · M5 Max · 18-core CPU / 40-core GPU · 64 GB unified the site's “runs well” tier ends at 56 GB (the GPU share plus the reserve) and “runs slowly” covers the rest up to 64 GB: the model fits in memory, but above the share it needs a changed system setting or runs partly on the CPU. (Apple · recommendedMaxWorkingSetSize, read Oct 17, 2026)
Every hardware page lists these models apart, under “Also runs slowly”, so they never mix with the ones that run well.