Skip to content
Local AI Wizard
Your machine
Theme

RTX 3060 · 12 GB VRAM

NVIDIA · RTX 3060

Graphics cards12 GB VRAM

569 text models run well · 360 GB/s bandwidth

Models on this hardware

Start here

Text

Within 12 GB: 569 run well · 25 tight · 18 reviewed · 175 more run slowly (39 of them run well with 16 GB)

Text · LMArena

rating · higher is better

  • Color = publisher

Source: LMArena 2026-10-22 · collected 2026-10-25 · dataset · CC BY 4.0 · Models without a rating are left out, not scored as zero.

Rows run well first, then tight fits; within each, reviewed models, then Hub downloads over 30 days, not quality. Sort a score column to rank by that source alone. All 39 text models that would run well with 16 GB but not here already run here, slowly.
Text models that run well within 12 GB
Granite 4.2 3BIBM · 3B · LMArena 1,290Q4 · published params · source1,290 ± 12~5 GBQ4 · published paramssource
Granite 4.2 8BIBM · 8B · LMArena 1,290Q4 · published params · source1,290 ± 11~8 GBQ4 · published paramssource
Llama 3.1 8B InstructMeta · 8B · LMArena 1,211Q4 · published params · source1,211 ± 4~8 GBQ4 · published paramssource
Llama 3.2 3B InstructMeta · 3.21B · LMArena 1,167Q4 · published params · source1,167 ± 8~5 GBQ4 · published paramssource
Llama 3.2 1B InstructMeta · 1.23B · LMArena 1,111Q4 · published params · source1,111 ± 8~3 GBQ4 · published paramssource
Qwen3.5 9BAlibaba · 9BQ4 · published params · sourceNot on the LMArena text leaderboard of 2026-10-22~9 GBQ4 · published paramssource
Qwen3 8BAlibaba · 8.2BQ4 · published params · sourceNot on the LMArena text leaderboard of 2026-10-22~9 GBQ4 · published paramssource
Gemma 4 E4BGoogle · 8BQ4 · published params · sourceNot on the LMArena text leaderboard of 2026-10-22~8 GBQ4 · published paramssource
8 of 594

No model within 12 GB has a LiveBench score, so the table has no LiveBench column. LMArena ratings in this table: LMArena 2026-10-22 · collected 2026-10-25 · dataset · CC BY 4.0

Tight fit: it runs, but leaves less than 1.2 GB free beyond the 2 GB the system keeps (the memory figure already includes that reserve).

Every reviewed text model here 37

Every one that runs here, well or slowly, scored or not (18 run well, 19 slowly); 20 of 37 have a LiveBench or LMArena result. Run well first, then tight fits, then slow; within each, by Hub downloads over 30 days, read 2026-11-19, not by quality.

Reviewed text models that run on RTX 3060 · 12 GB VRAM
Qwen3.5 9BAlibaba · 9BNot in LiveBench 2026-06-25Not on the LMArena text leaderboard of 2026-10-22~9 GB5.7Apache 2.06.5M
Qwen3 8BAlibaba · 8.2BNot in LiveBench 2026-06-25Not on the LMArena text leaderboard of 2026-10-22~9 GB5.0Apache 2.06.1M
Gemma 4 E4BGoogle · 8BNot in LiveBench 2026-06-25Not on the LMArena text leaderboard of 2026-10-22~8 GB5.0Apache 2.04.4M
Llama 3.1 8B InstructMeta · 8BNot in LiveBench 2026-06-251,211~8 GB4.9Llama 3.14.1M
Gemma 4 E2BGoogle · 5.1BNot in LiveBench 2026-06-25Not on the LMArena text leaderboard of 2026-10-22~6 GB3.1Apache 2.02.8M
Qwen3 4B Instruct 2507Alibaba · 4BNot in LiveBench 2026-06-25Not on the LMArena text leaderboard of 2026-10-22~5 GB2.5Apache 2.02.8M
Qwen3 1.7BAlibaba · 1.7BNot in LiveBench 2026-06-25Not on the LMArena text leaderboard of 2026-10-22~4 GB1.1Apache 2.02.7M
Llama 3.2 1B InstructMeta · 1.2BNot in LiveBench 2026-06-251,111~3 GB0.8Llama 3.21.3M
8 of 37
Also runs slowly156 models

Above the 12 GB of VRAM, the layers that do not fit run on the CPU from PC RAM, up to 36 GB in all. 32 GB of PC RAM assumed: this configuration lists none. Planning estimate, not a speed figure. The 19 reviewed models that run slowly are in the table above.

6 of 156

Browse Text models

Adjust the estimate: quantization, context, reserveloads the live catalog

Vision

Within 12 GB: 892 run well · 29 tight · 4 reviewed · 134 more run slowly (27 of them run well with 16 GB) · 4 more with 16 GB

Vision · LMArena

rating · higher is better

  • Color = publisher

Source: LMArena 2026-10-19 · collected 2026-10-25 · dataset · CC BY 4.0 · Models without a rating are left out, not scored as zero.

Rows run well first, then tight fits; within each, reviewed models, then Hub downloads over 30 days, not quality. Sort a score column to rank by that source alone.
Vision models that run well within 12 GB
Molmo2-8Ballenai · 8.66B · Image to text · LMArena 1,112Q4 · Hugging Face count · source1,112 ± 20~9 GBQ4 · Hugging Face countsource
Qwen2-VL-7B-InstructQwen · 8.29B · Image to text · LMArena 1,031Q4 · Hugging Face count · source1,031 ± 11~9 GBQ4 · Hugging Face countsource
Molmo-7B-D-0924allenai · 8.02B · Image to text · LMArena 994Q4 · Hugging Face count · source994 ± 14~9 GBQ4 · Hugging Face countsource
Llama-3.2-11B-Vision-Instructmeta-llama · 10.67B · Image to text · LMArena 993Q4 · Hugging Face count · source993 ± 11~11 GBTight fit: Needs ~11 GB: runs well, about 1 GB left beyond the system reserveQ4 · Hugging Face countsource
MiniCPM-V-2_6openbmb · 8.1B · Image to text · LMArena 962Q4 · Hugging Face count · source962 ± 16~9 GBQ4 · Hugging Face countsource
Phi-3.5-vision-instructmicrosoft · 4.15B · Image to text · LMArena 919Q4 · Hugging Face count · source919 ± 16~6 GBQ4 · Hugging Face countsource
Qwen3.5 9BAlibaba · 9B · Image to textQ4 · published params · sourceNot on the LMArena vision leaderboard of 2026-10-19~9 GBQ4 · published paramssource
Gemma 4 E4BGoogle · 8B · Image to textQ4 · published params · sourceNot on the LMArena vision leaderboard of 2026-10-19~8 GBQ4 · published paramssource
8 of 921

LMArena ratings in this table: LMArena 2026-10-19 · collected 2026-10-25 · dataset · CC BY 4.0

Tight fit: it runs, but leaves less than 1.2 GB free beyond the 2 GB the system keeps (the memory figure already includes that reserve).

Also runs slowly134 models

Above the 12 GB of VRAM, the layers that do not fit run on the CPU from PC RAM, up to 36 GB in all. 32 GB of PC RAM assumed: this configuration lists none. Planning estimate, not a speed figure.

6 of 134

With 16 GB you also run… 4

The next memory size on this ladder runs these well. Figures are the requirement on this hardware type.

471 more vision models in the catalog have no usable memory figure or parameter count for this profile. Browse Vision models

Image

89 within 12 GB · 9 more with 16 GB

Image · Prompt alignment · GenEval

0–1 · higher is better

Source: GenEval · 2023 · reviewed 2026-10-05 · Published quality in the original benchmark preset; not measured on this hardware.

Image models within 12 GB
Stable Diffusion XL 1.0Stability AI · 2.6B · Text to imagePlatform path documented · stable-diffusion.cpp · source0.55Not on the LMArena text-to-image leaderboard of 2026-10-07Documented path without a memory figurePlatform path documentedstable-diffusion.cpp · source
Stable Diffusion 1.5Stability AI · 0.98B · Text to imagePlatform path documented · stable-diffusion.cpp · source0.43Not on the LMArena text-to-image leaderboard of 2026-10-07Documented path without a memory figurePlatform path documentedstable-diffusion.cpp · source
BagelByteDance · 14B · Text to image · LMArena 898Conditional path · CUDA · sourceNo comparable published result898 ± 6~12 GBConditional pathCUDA · source
FOFPredSalesforce · 4B · Image to image16-bit est. · params · sourceNo comparable published resultNot reviewed for LMArena~12 GB16-bit est. · paramssource
OmniGen-v1BAAI · 3.9B · Text to image16-bit est. · params · sourceNo comparable published resultNot reviewed for LMArena~12 GB16-bit est. · paramssource
Stable Diffusion 3.5 LargeStability AI · 8B · Text to image · LMArena 938Conditional path · CUDA · sourceNo comparable published result938 ± 5~11 GBConditional pathCUDA · source
Stable Diffusion 3.5 MediumStability AI · 2.47B · Text to imageConditional path · sourceNo comparable published resultNot on the LMArena text-to-image leaderboard of 2026-10-07~10 GBConditional pathsource
CogView3-Plus-3Bzai-org · 2.8B · Text to image16-bit est. · params · sourceNo comparable published resultNot reviewed for LMArena~9 GB16-bit est. · paramssource
8 of 89

LMArena ratings in this table: LMArena 2026-10-07 · collected 2026-10-25 · dataset · CC BY 4.0

With 16 GB you also run… 9

The next memory size on this ladder. Figures are the requirement on this hardware type.

6 of 9

231 more image models in the catalog have no usable memory figure or parameter count for this profile. Browse Image models

Video

49 within 12 GB · 7 more with 16 GB
Video models within 12 GB
Kandinsky-6.0-Lite-5s-Diffuserskandinskylab · 3.2B · Image to video16-bit est. · params · source~10 GB16-bit est. · paramssource
Kandinsky-6.0-Lite-distill-5s-Diffuserskandinskylab · 3.2B · Image to video16-bit est. · params · source~10 GB16-bit est. · paramssource
Kandinsky-6.0-Lite-pretrain-5s-Diffuserskandinskylab · 3.2B · Image to video16-bit est. · params · source~10 GB16-bit est. · paramssource
Allegrorhymes-ai · 2.8B · Text to video16-bit est. · params · source~9 GB16-bit est. · paramssource
Allegro-T2V-40x360Prhymes-ai · 2.8B · Text to video16-bit est. · params · source~9 GB16-bit est. · paramssource
Allegro-T2V-40x720Prhymes-ai · 2.8B · Text to video16-bit est. · params · source~9 GB16-bit est. · paramssource
Allegro-TI2Vrhymes-ai · 2.8B · Image to video16-bit est. · params · source~9 GB16-bit est. · paramssource
Wan2.1 T2V 1.3BWan-AI · 1.3B · Text to videoPlatform path documented · Wan2.1 / PyTorch · source~8 GBPlatform path documentedWan2.1 / PyTorch · source
8 of 49

How this list works Documented execution paths on this hardware first, then publisher minimums (CUDA or Apple MPS).

With 16 GB you also run… 7

The next memory size on this ladder. Figures are the requirement on this hardware type.

6 of 7

124 more video models in the catalog have no usable memory figure or parameter count for this profile. Browse Video models

Audio

296 within 12 GB · 21 more with 16 GB

Audio · Word error rate · Open ASR

% · lower is better

Source: Word error rate · Open ASR · reviewed 2026-10-20 · Published quality in the original benchmark preset; not measured on this hardware.

Audio models within 12 GB
Whisper large-v3-turboOpenAI · 0.81B · Speech to textPlatform path documented · whisper.cpp / CUDA · source7.83Documented path without a memory figurePlatform path documentedwhisper.cpp / CUDA · source
Whisper large-v3OpenAI · 1.55B · Speech to textPlatform path documented · whisper.cpp / CUDA · source7.44Documented path without a memory figurePlatform path documentedwhisper.cpp / CUDA · source
3b-de-pretrain-research_releasecanopylabs · 3.8B · Text to speech16-bit est. · params · sourceNo comparable published result~12 GB16-bit est. · paramssource
3b-es_it-pretrain-research_releasecanopylabs · 3.8B · Text to speech16-bit est. · params · sourceNo comparable published result~12 GB16-bit est. · paramssource
3b-fr-pretrain-research_releasecanopylabs · 3.8B · Text to speech16-bit est. · params · sourceNo comparable published result~12 GB16-bit est. · paramssource
3b-ko-pretrain-research_releasecanopylabs · 3.8B · Text to speech16-bit est. · params · sourceNo comparable published result~12 GB16-bit est. · paramssource
3b-zh-pretrain-research_releasecanopylabs · 3.8B · Text to speech16-bit est. · params · sourceNo comparable published result~12 GB16-bit est. · paramssource
Breeze TTS 2BreezeBlue · 3.47B · Text to speechPublisher minimum · CUDA · sourceNo comparable published result~12 GBPublisher minimumCUDA · source
8 of 296

With 16 GB you also run… 21

The next memory size on this ladder. Figures are the requirement on this hardware type.

6 of 21

1,984 more audio models in the catalog have no usable memory figure or parameter count for this profile. Browse Audio models

Embeddings

634 run well within 12 GB · 14 more with 16 GB

Embeddings · MTEB (Multilingual, v2) · mean(task)

0–100 · higher is better

  • Color = publisher

Source: MTEB results (curated rows, sourced per model)

Embeddings models that run well within 12 GB
Qwen3-Embedding 8BAlibaba (Qwen)8-bit · published params · source70.6~12 GB8-bit · published paramssource
Qwen3-Embedding 4BAlibaba (Qwen)8-bit · published params · source69.5~7 GB8-bit · published paramssource
Harrier OSS v1 (0.6B)Microsoft8-bit · published params · source69.0~3 GB8-bit · published paramssource
Qwen3-Embedding 0.6BAlibaba (Qwen)8-bit · published params · source64.3~3 GB8-bit · published paramssource
Multilingual E5 Large Instructintfloat8-bit · published params · source63.2~3 GB8-bit · published paramssource
EmbeddingGemma 2Google8-bit · published params · source61.4~3 GB8-bit · published paramssource
EmbeddingGemma (300M)Google8-bit · published params · source61.1~3 GB8-bit · published paramssource
BGE-M3BAAI8-bit · published params · source59.6~3 GB8-bit · published paramssource
8 of 634

With 16 GB you also run… 14

The next memory size on this ladder runs these well. Figures are the requirement on this hardware type.

6 of 14

Browse Embeddings models

Runs well within 12 GB of VRAM · planning estimates, not measured speed live catalog · 19,444 repositories · checked

Measurements

Published tokens per second, exactly as measured; never estimated.

Published measurements on this hardware
Llama 2 7B Q4_0TheBloke/Llama-2-7B-GGUF · llama-2-7b.Q4_0.…NVIDIA GeForce RTX 3060 · 12 GB GDDR676.92 ± 0.03Q4_0
0 / 128 · Not reported by llama-bench
CUDA
llama.cpp commit baa9255
Flash Attention enabled
llama.cpp CUDA scoreboard · community result
2026-09-25
Qwen3.5 9B Q4_K_Munsloth/Qwen3.5-9B-GGUF/Qwen3.5-9B-Q4_K_M.g…NVIDIA GeForce RTX 3060 · 12 GB (the dataset id names 12 GB; rented host)55.28 ± 0.08Q4_K_M
0 / 128 · Single llama-bench process
llama.cpp CUDA
llama.cpp commit 3e4dfa8f3 (b11658)
Flash Attention enabled
Data: Local LLM Advisor (localllmadvisor.com) · CC BY 4.0
2026-10-25
Gemma 4 12B Q4_K_Munsloth/gemma-4-12b-it-GGUF/gemma-4-12b-it-…NVIDIA GeForce RTX 3060 · 12 GB (the dataset id names 12 GB; rented host)39.71 ± 0.04Q4_K_M
0 / 128 · Single llama-bench process
llama.cpp CUDA
llama.cpp commit 3e4dfa8f3 (b11658)
Flash Attention enabled
Data: Local LLM Advisor (localllmadvisor.com) · CC BY 4.0
2026-10-25
Qwen3.5 9B Q8_0unsloth/Qwen3.5-9B-GGUF/Qwen3.5-9B-Q8_0.ggufNVIDIA GeForce RTX 3060 · 12 GB (the dataset id names 12 GB; rented host)36.6 ± 0.04Q8_0
0 / 128 · Single llama-bench process
llama.cpp CUDA
llama.cpp commit 3e4dfa8f3 (b11658)
Flash Attention enabled
Data: Local LLM Advisor (localllmadvisor.com) · CC BY 4.0
2026-10-25

Specifications

11 sourced fields; the source is named once per group, and a field from another source links to it. Official specifications

GPU VRAM12 GBDedicated · system RAM does not add · source
Memory bandwidth360 GB/sManufacturer spec · not model speed · source
Launch price (MSRP)USD 329.00US · launched · source · not a tracked offer

Marker: the lightest text model that does not run well here but runs well with 16 GB. No text model that runs well here has a LiveBench score. Q4 planning estimate, not a measured fit.

Selected configuration

Latest check · Manufacturer source

Memory capacity
12 GB
Memory type
12 GB GDDR6 · 192-bit
Chip / processor
Ampere · RTX 3060
Memory bandwidth
360 GB/s

Family specification

Latest check · Manufacturer source

CUDA cores
3,584
GPU power (manufacturer specification; OEM may vary)
170 W
Memory interface
192 bit
CUDA compute capability
8.6
Release date
2021-02-25
Introduced
2021
Display connectors (reference design; OEM may vary)
3 × DisplayPort 1.4a · HDMI 2.1

NVIDIA specifies 12 GB of dedicated GDDR6 memory for the RTX 3060. The hardware table applies the planner's Q4 memory estimate to this capacity; it does not establish checkpoint/runtime compatibility, context fit or speed. NVIDIA's RTX 3060 family page also lists 8 GB variants, so confirm the exact card SKU before using this profile.

Photo: ASUS partner-board photo of a GeForce RTX 3060 12 GB · other partner boards vary. · Manufacturer source

Memory bandwidth is not measured model speed; verify runtime and checkpoint compatibility. Hardware specification reviewed: 2026-10-09.

How estimates work
Candidates
Reviewed pages use published parameter counts; automatic text candidates use official repository tensor counts (packed quantized weights and unsupported architectures excluded).
Planning
Q4 weights + 20% allowance + reserve: 25% of unified or system memory (at least 2 GB, at most 8 GB; 2 GB here) or 2 GB per dedicated GPU. Adjustable in “Adjust the estimate”. No KV cache is calculated from context.
Fit tiers
Runs well: the estimate fits the fast memory (VRAM, or unified memory within the macOS GPU share). Runs slowly: within the machine's memory but outside the fast part; a graphics card offloads layers to PC RAM (the listed RAM, or 32 GB assumed). Doesn't run: above both. Tiers are planning estimates, never a speed.
Vision
Vision-language models use the Q4 planning estimate from the parameter count (published by the model page, or read from Hugging Face). Classic vision models (classify, detect, segment) use a 16-bit estimate. No vision benchmark is tracked, so rows carry no score.
Media and embeddings
Media rows use documented execution paths (exact hardware or platform) and publisher minimums; a published requirement above this memory moves a model to the next tier. Without either, a 16-bit estimate from the published parameter count (× 1.2 + 2 GB), labelled as such. Embeddings use 8-bit weights + 20% + 2 GB.

Local AI basics: parameters, Q4 and memory · Full model catalog · Data and methodology

Alternatives

Other machines on the same memory ladder (VRAM); each page lists the models that fit it.