Q4_K_M, Q8_0 or F16: which file should I download?
The same model in three precisions: file size, memory and what you give up.
Short answer
Start with Q4_K_M. Move up to Q8_0 only if it fits with room to spare.
- Q4_K_M: The usual pick: 16.8 GB for Qwen3.6 27B, 31% of the original.
- Q8_0: Closer to the original, and bigger: 28.6 GB for the same model.
- F16 · BF16: Full precision (53.8 GB), for converting or fine-tuning, not for everyday use.
Published file sizes (unsloth/Qwen3.6-27B-GGUF, read Oct 16, 2026)
What the names mean
A GGUF file stores every weight of the model with a chosen precision. The name says which one (Hugging Face · GGUF, read Oct 17, 2026):
- Q4_K_M
- 4-bit “k-quant”, medium variantWeights in blocks with their own scales; some layers kept at higher precision. llama.cpp measures 4.89 bits per weight.
- Q8_0
- 8-bit, one scale per block of 32 weightsAn older, simpler scheme that stays close to the original. 8.5 bits per weight.
- F16 · BF16
- 16-bit floating point, no quantizationHow most publishers release the weights. F16 measures 16 bits per weight.
The letter after K is the size within the same bit count: S (small), M (medium), L (large). Files starting with IQ use an importance matrix to keep quality at even fewer bits. Bits per weight from llama.cpp's table for Llama 3.1 8B (llama.cpp · quantize, read Oct 17, 2026).
Three real files
Qwen3.6 27B on Hugging Face. Memory is the site's planning estimate for short conversations.
| File | Size (GB) | 32 GB+ Mac or PC (GB, est.) | Graphics card (GB, est.) |
|---|---|---|---|
| Q4_K_M | 16.8 | 29 | 23 |
| Q8_0 | 28.6 | 45 | 39 |
| BF16 | 53.8 | 73 | 67 |
The memory columns use the site's Q4, Q8 and FP16 formula (how memory is estimated). The model also has a separate vision file of about 0.9 GB, needed only to read images.
Largest file per machine
The most precise of the three files whose 27B estimate stays within what the machine offers to a model.
- RTX 4090 · 24 GB VRAM
- Q4_K_MOffers 24 GB to a model
- RTX 5090 · 32 GB VRAM
- Q4_K_MOffers 32 GB to a model
- Mac mini M4 Pro · 48 GB unified
- Q4_K_MOffers 44 GB to a model
- Mac Studio · M5 Max · 18-core CPU / 40-core GPU · 64 GB unified
- Q8_0Offers 56 GB to a model
- Mac Studio M5 Max · 128 GB unified
- BF16Offers 104 GB to a model
How to choose
- Most people
- Q4_K_MOllama picks it by default when a Hugging Face repository has it.Hugging Face · Ollama
- Memory to spare
- Q8_0 · Q6_K · Q5_K_MCloser to the original, larger and slower to read.
- Not enough memory
- Q3_K_M · IQ3 · IQ2Smaller, with a larger quality loss; compare with a smaller model in Q4.
Smaller files have less to read for every word. In llama.cpp's example run on one machine, Q8_0 generated text more slowly than Q4_K_M (llama.cpp · quantize, read Oct 17, 2026). The site does not estimate speed. Benchmark scores on this site are for the publisher's original weights; a quantized file can score a little lower.