Skip to content
Local AI Wizard
Your machine
Theme

GGUF vs MLX: which file to download on a Mac (and in LM Studio)?

What each format is, which apps open it, and what the real file sizes and measured speeds show.

Files and versions · updated

Short answer

It depends on your app, not the size. On a Mac with an M chip using LM Studio or mlx-lm, MLX; everywhere else, GGUF.

  • GGUF: One file that Ollama, LM Studio and llama.cpp open on Windows, Linux and any Mac.
  • MLX: Apple's format, built for the Mac's unified memory. Only on Macs with an M chip.
  • Size: Almost the same: Qwen3 4B at 4 bits is 2.5 GB as GGUF and 2.3 GB as MLX.

Published file sizes, read Oct 23, 2026: unsloth/Qwen3-4B-Instruct-2507-GGUF · mlx-community/Qwen3-4B-Instruct-2507-4bit

The idea

Think of one film sold on two discs: a disc that plays in almost any player, and a disc made for one brand of player that uses its features. The film is the same. GGUF is the disc for any player: one file that llama.cpp and the apps built on it read on Windows, Linux and Macs. MLX is the disc for one player: a folder of files for Apple's own MLX framework, which runs on Macs with Apple silicon (M1 and later).

GGUF was designed for “single-file deployment” (GGUF specification, read Oct 25, 2026), and llama.cpp loads nothing else (llama.cpp · models, read Oct 25, 2026). MLX is “an array framework for machine learning on Apple silicon” whose arrays “live in shared memory”, the unified memory of a Mac (Apple · MLX README, read Oct 25, 2026). Both formats usually hold a quantized copy, so the same Q4 idea applies to both.

Which one to pick

  1. Is it a Windows or Linux PC, a graphics card, or an Intel Mac?

    YesGGUFMLX apps run on Apple silicon; llama.cpp also runs on NVIDIA, AMD and the CPU.

    No

  2. Do you use Ollama, or want one file that also works on a PC?

    YesGGUFOllama imports GGUF files; one GGUF file runs in every app below.

    No

  3. Do you use LM Studio or mlx-lm on a Mac with Apple silicon?

    YesMLXBuilt for the Mac's unified memory. LM Studio lists both, so you can also try the GGUF of the same size.

    No

  4. GGUFThe file every app on this page can open.

Read top to bottom: the first yes decides. Sources for each app are in the table below.

Size is not the deciding factor: the two copies of the same model are within a few percent of each other (Qwen3 4B at 4 bits: 2.5 GB as GGUF and 2.3 GB as MLX). Pick the bit width for your memory first, then the format for your app.

Which app opens which

What each app's own documentation says, read 25 Oct 2026.
AppGGUFMLXRuns onSource
LM StudioYes, through llama.cppYes, on Apple silicon MacsMac (Apple silicon only), Windows, LinuxLM Studio docs
OllamaYes (a Modelfile with FROM file.gguf)An MLX engine for safetensors models, on by default on Apple silicon; downloading mlx-community folders is not documentedMac, Windows, LinuxOllama · importing a model
llama.cppYes, the only format it loadsNoApple silicon (Metal), NVIDIA (CUDA), AMD (HIP), Vulkan, CPUllama.cpp · models
mlx-lmNoYes, MLX is its formatApple silicon MacsApple · mlx-lm README

LM Studio's documentation says it runs models “using llama.cpp” on Mac, Windows and Linux, and “on Apple Silicon Macs, LM Studio also supports running LLMs using Apple's MLX” (LM Studio docs, read Oct 25, 2026); on a Mac it needs Apple silicon (LM Studio · system requirements, read Oct 25, 2026). Ollama imports a GGUF file with a one-line Modelfile (Ollama · importing a model, read Oct 25, 2026); its developer notes add an MLX engine for safetensors models, on by default on Apple silicon (Ollama · development (MLX engine), read Oct 25, 2026). To start with either app, see getting started with Ollama or LM Studio.

Mac vs PC and NVIDIA

On a Mac with Apple silicon both formats run on the same chip and the same unified memory: llama.cpp treats Apple silicon as “a first-class citizen” through Metal (llama.cpp README, read Oct 25, 2026), and MLX is Apple's own framework for it. That is the one machine where the choice is real.

On a Windows or Linux PC, and on any NVIDIA or AMD graphics card, download GGUF: llama.cpp has CUDA kernels for NVIDIA, HIP for AMD and a Vulkan backend, and LM Studio and Ollama use it there. MLX does have a CUDA backend for Linux (MLX · installation, read Oct 25, 2026), but the MLX folders on Hugging Face are made for Macs, and LM Studio uses MLX only on Apple silicon. Whether a model fits is the same question in both formats: see Mac or graphics card and VRAM, RAM or unified memory.

File sizes

Qwen3 4B: GGUF and MLX side by side

Download size in GB at 4, 6 and 8 bits

  • GGUF
  • MLX
2.5 GBGGUF Q4_K_MGGUF Q4_K_M2.3 GBMLX 4-bitMLX 4-bit3.3 GBGGUF Q6_KGGUF Q6_K3.3 GBMLX 6-bitMLX 6-bit4.3 GBGGUF Q8_0GGUF Q8_04.3 GBMLX 8-bitMLX 8-bit

Hub file sizes, read Oct 23, 2026: unsloth/Qwen3-4B-Instruct-2507-GGUF and the MLX repositories named in the table

GB = 10⁹ bytes, summed over a repository's weight files; read Oct 23, 2026. A GGUF file of this size is one file; an MLX copy is a folder of safetensors parts.
ModelBitsGGUF file (GB)MLX folder (GB)MLX partsMLX repository
Qwen3 4B42.5 (Q4_K_M)2.31mlx-community/Qwen3-4B-Instruct-2507-4bit
Qwen3 4B63.3 (Q6_K)3.31lmstudio-community/Qwen3-4B-Instruct-2507-MLX-6bit
Qwen3 4B84.3 (Q8_0)4.31lmstudio-community/Qwen3-4B-Instruct-2507-MLX-8bit
Qwen3 30B A3B418.6 (Q4_K_M)17.24lmstudio-community/Qwen3-30B-A3B-Instruct-2507-MLX-4bit
Qwen3 30B A3B625.1 (Q6_K)24.85lmstudio-community/Qwen3-30B-A3B-Instruct-2507-MLX-6bit
Qwen3 30B A3B832.5 (Q8_0)32.47lmstudio-community/Qwen3-30B-A3B-Instruct-2507-MLX-8bit

At the same bit width the two formats are within a few percent. The larger Qwen3 30B A3B shows the other difference: its GGUF Q4_K_M is one 18.6 GB file, while its MLX 4-bit copy is 17.2 GB split into 4 files that must stay in one folder. The table has both models. Memory to run either is the file plus room for the conversation; the Qwen3 4B model page lists every file with the memory it needs.

Measured speed

The question people ask most is which one is faster. This site has no run of the same model in both formats on the same Mac, so it does not say. What it holds: the mlx-lm maintainers measured MLX copies of these two models on one Apple M4 Max · 64 GB (GPU cores not stated), and llama.cpp's Apple silicon table measured one GGUF reference file (Llama 7B Q4_0) on 27 Mac configurations. Different models and different machines, so the two sets cannot be compared with each other. (mlx-lm · BENCHMARKS.md, read Oct 25, 2026)

Generation speed, mlx_lm.benchmark --model <repo> -p 2048 -g 128 (maintainers' benchmark table); Apple M4 Max · 64 GB (GPU cores not stated); mlx 0.29.2.dev20251008+85a8824a8 · mlx-lm 0.28.2 · macOS 26.1. Measured by the mlx-lm maintainers; prompt speed is a separate column of the source that this site does not store.
ModelFileWriting speed · tokens/s (measured)
Qwen3 4BMLX q4134.52
Qwen3 4BMLX q5116.39
Qwen3 4BMLX q6104.68
Qwen3 4BMLX q886.907
Qwen3 4BMLX bf1652.47
Qwen3 30B A3BMLX q4113.33
Qwen3 30B A3BMLX q5101.00
Qwen3 30B A3BMLX q694.14
Qwen3 30B A3BMLX q883.16

Within one format the rule from the file sizes holds: fewer bits, smaller folder, faster writing. Qwen3 4B writes 134.5 tokens per second as MLX q4 and 52.5 as MLX bf16, because each new token reads the whole model from memory. What a token is and why memory speed sets this pace: tokens and tokens per second.

Questions

Is an MLX 4-bit file the same quality as GGUF Q4_K_M?

Close, not identical. Both keep about 4 bits per number, but they round differently: Q4_K_M keeps a few sensitive parts at 6 bits, and MLX 4-bit rounds groups of 64 numbers with one scale each. Neither is scored on this site; scores belong to the original model.

Can I use MLX on Windows or with an NVIDIA card?

Not in LM Studio, which uses MLX only on Apple silicon Macs. MLX itself has a CUDA backend for Linux, but the usual apps on Windows and NVIDIA cards load GGUF.

Can I turn one into the other?

Both are made from the original 16-bit weights, not from each other. mlx-lm's convert command makes an MLX copy, and llama.cpp's convert scripts make a GGUF. For popular models someone has usually done it already.

Why does LM Studio show both for the same model?

Because on a Mac with Apple silicon it can run both engines. Community accounts such as lmstudio-community and mlx-community publish an MLX folder next to the GGUF files of the same model.