Skip to content
Local AI Wizard
Your machine
Theme

What happens when a model doesn't fit in VRAM?

Offloading layers to the CPU and PC RAM: when it works, what it costs and how the site marks it.

Speed · updated

Short answer

It still runs, much more slowly: the layers that don't fit stay in PC RAM and run on the processor.

  • Which apps: Those built on llama.cpp, such as Ollama and LM Studio, can split a model this way.
  • How far: An 8 GB card with 32 GB of RAM reaches models up to 32 GB, marked “runs slowly”.
  • On a Mac: There is no separate graphics memory to offload from; the model must fit in unified memory.

Planning estimate with the site's fit rule, not a measurement. Methodology

How offloading works

A model is a stack of layers. When the file is bigger than the graphics card's memory, llama.cpp and the apps built on it (LM Studio, Ollama) can put as many layers as fit on the card and leave the rest in the computer's normal memory, where the processor runs them. The answer is the same; only the speed changes. (llama.cpp README, read Oct 17, 2026)

It is slower because each new word reads every layer, and the layers in PC RAM are read at the speed of the system memory. The site's profiles list 448 GB/s for the memory on an RTX 5060 Ti 8 GB card and 152 GB/s for the fastest system memory they record (a Snapdragon X2 laptop's LPDDR5x). The part of the work that runs from PC RAM sets the pace (RTX 5060 Ti 8 GB, Surface Laptop X2 Plus). The site never turns this into a speed figure: “runs slowly” is a tier, not a number.

An 8 GB card, model by model

Q4 planning requirement on the card (2 GB reserve included) and the site's tier, with 32 GB of PC RAM assumed or 16 GB listed.

Reviewed models on an 8 GB graphics card with 32 GB or 16 GB of PC RAM.
ModelParameters (B)Q4 need on the card (GB)RTX 4060 8 GB · 32 GB RAMLOQ 15IRX10 i7-14700HX RTX 5060 16GB 83JE000QUS
Qwen3 4B Instruct 250745Runs wellRuns well
Qwen3 8B8.29Runs slowlyRuns slowly
Qwen3 14B14.814Runs slowlyRuns slowly
Mistral Small 3.2 24B2421Runs slowlyDoesn't run
Qwen3 30B A3B Instruct 250730.525Runs slowlyDoesn't run
Olmo 3 32B Think3226Runs slowlyDoesn't run

Where the limit is

Runs well
Up to 8 GB on the card13 reviewed text models on RTX 4060 · 8 GB VRAM.
Runs slowly
Up to 32 GB with 32 GB of RAMThe card's 8 GB plus PC RAM minus the system's own 8 GB (25% of the RAM, 2 to 8 GB). 24 reviewed models fall here.
Less PC RAM, less room
Up to 20 GB with 16 GB of RAMLOQ 15IRX10 · MTM 83JE000QUS lists 16 GB of RAM, so its slow tier ends sooner: 10 reviewed models.
When the RAM is unknown
32 GB assumed and said soA bare graphics-card profile does not say how much RAM the PC has; the page states the assumption next to every slow row.

On a Mac

A Mac has no separate graphics memory, so there is nothing to offload to. What it has is a default cap on how much memory the GPU may use; the site plans with 75% of the memory as that share. On Mac Studio · M5 Max · 18-core CPU / 40-core GPU · 64 GB unified the site's “runs well” tier ends at 56 GB (the GPU share plus the reserve) and “runs slowly” covers the rest up to 64 GB: the model fits in memory, but above the share it needs a changed system setting or runs partly on the CPU. (Apple · recommendedMaxWorkingSetSize, read Oct 17, 2026)

Every hardware page lists these models apart, under “Also runs slowly”, so they never mix with the ones that run well.