Skip to content
Local AI Wizard
Your machine
Theme

Can I run AI with 8 GB of memory?

What fits on an 8 GB graphics card, an 8 GB Mac and an 8 GB PC, and what to do when it does not.

Memory · updated

Short answer

~8B

Yes, with small models: up to about 8B parameters in Q4 on an 8 GB graphics card, with no margin.

  • On an 8 GB Mac or a PC without a graphics card, the estimate stops near 8B.
  • Conversations: Keep them short; long chats need more memory.
  • Bigger models: They run with part in system RAM, much more slowly.

Planning estimate with the site's formula, not a measurement. Methodology

Size by size

Q4 file, short conversations. Estimates rounded up to whole GB.

Each estimate includes a reserve: a graphics card keeps 2 GB aside for its runtime and display, because the system runs from the PC's own RAM; a Mac or a PC without a card shares its memory with the system, so the site keeps 25% of it aside, at least 2 GB: 2 GB on 8 GB (VRAM and unified memory).

Memory each size needs on an 8 GB machine, by machine type.
Model sizeQ4 weights, est. (GB)Needs on an 8 GB graphics card (GB)Needs on an 8 GB Mac or PC (GB)
1B0.633
3B1.955
4B2.555
8B588
9B5.699
12B7.51111
14B8.81313

8 GB machines

The largest common model size whose Q4 estimate stays within what each machine offers.

RTX 4060 · 8 GB VRAM
Up to 8B in Q4Offers 8 GB to a model
RTX 5060 · 8 GB VRAM
Up to 8B in Q4Offers 8 GB to a model
Radeon RX 9060 XT · 8 GB VRAM
Up to 8B in Q4Offers 8 GB to a model
Mac mini M2 · 8 GB unified
Up to 8B in Q4Offers 8 GB to a model

A real 9B file

The Q4_K_M file of Qwen3.5 9B is 5.7 GB (unsloth/Qwen3.5-9B-GGUF, read Oct 17, 2026). It looks like it fits an 8 GB card, but the program needs working room on top of the file: the site's estimate is 5.6 GB of weights, 1.1 GB of working room and 2 GB of reserve, about 9 GB in total.

That is more than 8 GB, so by the site's estimate a 9B model in Q4 does not fit an 8 GB card. The options are below: a smaller file, a smaller model, or splitting the work with system RAM. Qwen3.5 9B

When it is too big

Pick a smaller file
Q3 and Q2 files trade more quality for sizellama.cpp measures Q3_K_M at 4 and Q2_K at 3.16 bits per weight, against 4.89 for Q4_K_M.llama.cpp · quantize
Pick a smaller model
A 4B model leaves room for a longer conversationIn Q4 it needs about 5 GB on a graphics card, 3 GB less than an 8 GB card offers.
Split it with system RAM
llama.cpp and the apps built on it can run the rest on the processorIt works, much more slowly; hardware pages list these models apart, under “Also runs slowly”.llama.cpp README
Keep conversations short
Long chats and documents need extra memoryThe working room covers short use only.