Skip to content
Local AI Wizard
Your machine
Theme

Local AI basics: what do 9B, Q4 and VRAM mean?

The ideas behind every number on this site, one short section each.

Getting started · updated

Short answer

9B is the model's size, Q4 is how compactly its file is stored, and VRAM is the graphics card's memory, where the model has to fit.

  • 9B, 27B: Billions of parameters. All of them sit in memory while the model runs.
  • Q4: About 0.625 byte per parameter, so a 27B model's weights take about 17 GB.
  • VRAM: A graphics card uses only its own; a Mac shares one pool of unified memory.
  • The rule: Memory, not processor speed, decides what you can run.

Planning estimate with the site's formula, not a measurement. Methodology

What “9B” or “27B” means

A model is a very large file of numbers it learned during training, called parameters or weights. “B” means billion: a 27B model has 27 billion of them. More parameters usually means a more capable model, but every one of them has to be in memory while the model runs. That is why memory, not processor speed, decides what you can run.

Compare Qwen3.5 9B with Qwen3.8 27B: three times the parameters, roughly three times the memory.

Q4, Q8 and FP16: bytes per parameter

Each parameter is stored with a precision. Lower precision makes the file smaller with a small loss of quality. Local apps such as Ollama, LM Studio and llama.cpp usually download Q4 versions. A 27B model takes:

FP16
~54 GB2 bytes per parameter · the original
Q8
~30 GBabout 1.1 bytes · near-original quality
Q4
~17 GBabout 0.6 byte · the usual local choice

How the site estimates whether a model fits

Weights are not the whole story: the program running the model needs working room, and the computer needs memory for itself. The site adds both:

memory ≈ Q4 weights + 20% working room + reserve
27B on a 32 GB Mac or PC
~29 GB16.9 + 3.4 + 8 GB reserve
27B on a 16 GB Mac or PC
~25 GB16.9 + 3.4 + 4 GB reserve
27B on a graphics card
~23 GB2 GB reserve per GPU: the system uses its own RAM

The reserve scales with the machine: 25% of its memory, at least 2 GB and at most 8 GB (8 GB → 2, 16 GB → 4, 32 GB or more → 8). Pages without a chosen machine show the 8 GB figure.

This is a planning floor, not a test. Image, video and audio models use the publisher's documented minimum when there is one; otherwise a 16-bit estimate from the parameter count, labelled “16-bit est.”:

memory = ceil( parameters × 2 × 1.2 + 2 )

Total and active parameters (MoE)

Some models are a team of “experts” and use only a few of them for each word. Qwen3.5 35B A3B has 35B parameters in total and about 3B active per word. The active part sets the speed; the total sets the memory, because every expert must be loaded. The site always plans with the total.

Why image models come in parts

An image generator is an assembly line: a text encoder reads the prompt, a main network (UNet or transformer) draws, and a VAE turns the result into pixels. Each part has its own parameters. Publishers often give the size of each part and no total: the Stable Diffusion 1.5 authors write “860M UNet and 123M text encoder”.

When that happens the site shows the sum of the published parts, labelled as such, lists the parts and says what the publisher leaves out (here, the VAE). It does not use the Hugging Face file count for these models: for GLM-Image the publisher states 9B + 7B, while the Hub counts 6.9B because it reads only some of the files.

VRAM, unified memory and the macOS limit

A PC graphics card has its own memory, VRAM. Models load there, and the computer's RAM does not add to it. RTX 4090 · 24 GB can hold about 24 GB of model, even in a PC with 64 GB of RAM.

A Mac with Apple silicon has one pool, unified memory, shared by the processor and the GPU. By default macOS lets the GPU use about 75% of it (less on some versions and small machines). On a 128 GB Mac that is about 96 GB, so models run well up to 96 GB plus the 8 GB reserve, 104 GB; between that and 128 GB they still load but run slowly, because the part beyond the GPU share runs on the processor. With the 75% default the site plans with, the 25% reserve covers the difference on 8 to 24 GB Macs, so the whole memory counts; where macOS gives the GPU less, a model near the top of that memory can run slowly.

Advanced users can raise the limit in Terminal; it resets when the Mac restarts, and macOS still needs room for itself:

sudo sysctl iogpu.wired_limit_mb=110000

Long conversations need more memory

While it answers, a model keeps a working memory of the conversation (the KV cache). It grows with the length of the text: a short chat adds little, a long document or a long chat can add several GB. The site's 20% working room covers short use; the planner on each hardware page lets you add a budget for long context.

When a model does not fit in VRAM

Tools such as llama.cpp and Ollama can keep part of a model in the computer's RAM when it does not fit in VRAM. It runs, but much more slowly, because RAM is far slower to reach from the GPU. Hardware pages list what fits in VRAM as “runs well” and, apart, what “runs slowly” with offload: up to the VRAM plus the PC's RAM minus the RAM's own reserve. When a graphics card page lists no RAM, the site assumes 32 GB and says so. “Slowly” is never a speed figure.

Fitting is not speed

Memory decides whether a model fits; memory bandwidth (GB/s) mostly decides how fast it writes. Every word reads the active weights from memory once, so a machine with twice the bandwidth is roughly twice as fast on the same model. Hardware pages show bandwidth in the specifications and list measured tokens per second only when someone has published a run on that exact machine. The site never estimates speed.

Published, estimated and measured

Published
stated by the publisher or benchmark, with source and date
Estimated
computed here from published inputs with the formulas above
Measured
a named run on named hardware, shown exactly as published