Skip to content
Local AI Wizard
Your machine
Theme

DGX H100 · 8 × 80 GB

NVIDIA · DGX H100

Workstations & servers640 GB aggregate GPU memory

207 text models run well · 3,350 GB/s bandwidth

Models on this hardware

Start here

Text

Within 640 GB: 207 run well · 47 reviewed

Text · LiveBench · overall

0–100 · higher is better

  • Color = publisher

Source: LiveBench 2026-06-25 · collected 2026-09-26 · Open models without a score are not drawn (no score ≠ 0).

Rows run well first, then tight fits; within each, reviewed models, then Hub downloads over 30 days, not quality. Sort a score column to rank by that source alone. Models are planned in this system's combined GPU memory: each one fits or it does not.
Text models that run well within 640 GB
Smaug MiniAbacus AI · 27BQ4 · published params · source76.9Not on the LMArena text leaderboard of 2026-10-22~37 GBQ4 · published paramssource
GLM-5.3Z.ai · 744BQ4 · published params · source76.1Arena lists only GLM-5.3 (Max), a named effort setting, so no rating is matched to this entry~574 GBQ4 · published paramssource
Qwen3.8 27BAlibaba · 27B · LMArena 1,437Q4 · published params · source75.31,437 ± 5~37 GBQ4 · published paramssource
DeepSeek V4 Flash · 0731DeepSeek · 304BQ4 · published params · source74.2Not on the LMArena text leaderboard of 2026-10-22~244 GBQ4 · published paramssource
GLM-5.2Z.ai · 744BQ4 · published params · source73.2Arena lists only GLM-5.2 (Max), a named effort setting, so no rating is matched to this entry~574 GBQ4 · published paramssource
GLM-5.3 FlashZ.ai · 320B · LMArena 1,476Q4 · published params · source71.61,476 ± 5~256 GBQ4 · published paramssource
Nemotron 3 Ultra 550B A55BNVIDIA · 550B · LMArena 1,426Q4 · published params · source67.41,426 ± 7~429 GBQ4 · published paramssource
MiniMax M3MiniMax · 428B · LMArena 1,440Q4 · published params · source67.31,440 ± 4~337 GBQ4 · published paramssource
8 of 207

LMArena ratings in this table: LMArena 2026-10-22 · collected 2026-10-25 · dataset · CC BY 4.0

Every reviewed text model here 47

Every one that runs here, well or slowly, scored or not; 27 of 47 have a LiveBench or LMArena result. Run well first, then tight fits, then slow; within each, by Hub downloads over 30 days, read 2026-10-25, not by quality.

Reviewed text models that run on DGX H100 · 8 × 80 GB
Qwen3 14BAlibaba · 14.8BNot in LiveBench 2026-06-25Not on the LMArena text leaderboard of 2026-10-22~28 GB9.0Apache 2.010.4M
Gemma 4 31BGoogle · 30.7BNot in LiveBench 2026-06-251,452~40 GB18.3Apache 2.08.4M
Gemma 4 26B A4BGoogle · 25.2BNot in LiveBench 2026-06-251,437~35 GB16.9Apache 2.08.2M
Qwen3 8BAlibaba · 8.2BNot in LiveBench 2026-06-25Not on the LMArena text leaderboard of 2026-10-22~23 GB5.0Apache 2.08.1M
Qwen3.8 27BAlibaba · 27B75.31,437~37 GB16.5Apache 2.07.8M
GLM-5.3 FlashZ.ai · 320B71.61,476~256 GB156.8MIT6.4M
Qwen3.5 9BAlibaba · 9BNot in LiveBench 2026-06-25Not on the LMArena text leaderboard of 2026-10-22~23 GB5.7Apache 2.06.1M
gpt-oss-20bOpenAI · 21BNot in LiveBench 2026-06-251,317~32 GB11.6Apache 2.05.8M
8 of 47

Browse Text models

Adjust the estimate: quantization, context, reserveloads the live catalog

Vision

Within 640 GB: 157 run well · 12 reviewed

Vision · LMArena

rating · higher is better

  • Color = publisher

Source: LMArena 2026-10-19 · collected 2026-10-25 · dataset · CC BY 4.0 · Models without a rating are left out, not scored as zero.

Rows run well first, then tight fits; within each, reviewed models, then Hub downloads over 30 days, not quality. Sort a score column to rank by that source alone. Models are planned in this system's combined GPU memory: each one fits or it does not.
Vision models that run well within 640 GB
GLM-5.3 FlashZ.ai · 320B · Image to text · LMArena 1,271Q4 · published params · source1,271 ± 6~256 GBQ4 · published paramssource
Gemma 4 31BGoogle · 30.7B · Image to text · LMArena 1,262Q4 · published params · source1,262 ± 6~40 GBQ4 · published paramssource
Qwen3.8 27BAlibaba · 27B · Image to text · LMArena 1,242Q4 · published params · source1,242 ± 8~37 GBQ4 · published paramssource
Gemma 4 26B A4BGoogle · 25.2B · Image to text · LMArena 1,242Q4 · published params · source1,242 ± 6~35 GBQ4 · published paramssource
MiniMax M3MiniMax · 428B · Image to text · LMArena 1,233Q4 · published params · source1,233 ± 6~337 GBQ4 · published paramssource
Qwen3-VL-235B-A22B-InstructQwen · 235.67B · Image to text · LMArena 1,215Q4 · Hugging Face count · source1,215 ± 7~193 GBQ4 · Hugging Face countsource
gemma-3-27b-itgoogle · 27.43B · Image to text · LMArena 1,160Q4 · Hugging Face count · source1,160 ± 8~37 GBQ4 · Hugging Face countsource
GLM-4.5Vzai-org · 107.71B · Image to text · LMArena 1,153Q4 · Hugging Face count · source1,153 ± 12~97 GBQ4 · Hugging Face countsource
8 of 157

LMArena ratings in this table: LMArena 2026-10-19 · collected 2026-10-25 · dataset · CC BY 4.0

12 more vision models in the catalog have no usable memory figure or parameter count for this profile. Browse Vision models

Image

82 within 640 GB

Image · Prompt alignment · GenEval

0–1 · higher is better

Source: GenEval · 2023 · reviewed 2026-10-05 · Published quality in the original benchmark preset; not measured on this hardware.

Image models within 640 GB
Stable Diffusion XL 1.0Stability AI · 2.6B · Text to imagePlatform path documented · stable-diffusion.cpp · source0.55Not on the LMArena text-to-image leaderboard of 2026-10-07Documented path without a memory figurePlatform path documentedstable-diffusion.cpp · source
Stable Diffusion 1.5Stability AI · 0.98B · Text to imagePlatform path documented · stable-diffusion.cpp · source0.43Not on the LMArena text-to-image leaderboard of 2026-10-07Documented path without a memory figurePlatform path documentedstable-diffusion.cpp · source
Cosmos3-Super-Text2ImageNVIDIA · 64B · Text to image · LMArena 1,152Publisher minimum · CUDA · sourceNo comparable published result1,152 ± 6~640 GBPublisher minimumCUDA · source
Cosmos3-Super-Text2Image (agentic)NVIDIA · 64B · Text to image · LMArena 1,165Publisher minimum · CUDA · sourceNo comparable published result1,165 ± 7~640 GBPublisher minimumCUDA · source
Cosmos3-Super-Text2Image-4StepNVIDIA · 64B · Text to imagePublisher minimum · CUDA · sourceNo comparable published resultArena rated Cosmos3-Super-Text2Image; this entry is the 4-step distilled checkpoint~640 GBPublisher minimumCUDA · source
HunyuanImage 3.0 Instruct (Fal)Tencent · 80B · Text to imagePublisher minimum · CUDA · sourceNo comparable published resultArena rated hunyuan-image-3.0; this entry is the Instruct checkpoint~640 GBPublisher minimumCUDA · source
HunyuanImage 3.0 (Fal)Tencent · 80B · Text to image · LMArena 1,150Publisher minimum · CUDA · sourceNo comparable published result1,150 ± 3~240 GBPublisher minimumCUDA · source
Emu3.5-ImageBAAI · 34.1B · Text + image to image16-bit est. · params · sourceNo comparable published resultNot reviewed for LMArena~84 GB16-bit est. · paramssource
8 of 82

LMArena ratings in this table: LMArena 2026-10-07 · collected 2026-10-25 · dataset · CC BY 4.0

17 more image models in the catalog have no usable memory figure or parameter count for this profile. Browse Image models

Video

75 within 640 GB
Video models within 640 GB
Cosmos3-Super-Image2Videonvidia · 64.6B · Image to video16-bit est. · params · source~158 GB16-bit est. · paramssource
Cosmos3-Super-Image2Video-4Stepnvidia · 64B · Image to video16-bit est. · params · source~156 GB16-bit est. · paramssource
MiniMax H3MiniMax · 33B · Text to video16-bit est. · params · source~82 GB16-bit est. · paramssource
Cosmos-H-Surgicalnvidia · 15.2B · Image to videoConditional path · CUDA · source~80 GBConditional pathCUDA · source
Wan-Dancer-14BWan-AI · Image to videoConditional path · CUDA · source~80 GBConditional pathCUDA · source
Wan2.2 T2V-A14BWan-AI · 27B · Text to videoConditional path · CUDA · source~80 GBConditional pathCUDA · source
Wan2.2-I2V-A14BWan-AI · 27B · Image to videoConditional path · CUDA · source~80 GBConditional pathCUDA · source
Wan2.2-I2V-A14B-DiffusersWan-AI · 14.3B · Image to videoConditional path · CUDA · source~80 GBConditional pathCUDA · source
8 of 75

How this list works Documented execution paths on this hardware first, then publisher minimums (CUDA or Apple MPS).

20 more video models in the catalog have no usable memory figure or parameter count for this profile. Browse Video models

Audio

134 within 640 GB

Audio · Word error rate · Open ASR

% · lower is better

Source: Word error rate · Open ASR · reviewed 2026-10-20 · Published quality in the original benchmark preset; not measured on this hardware.

Audio models within 640 GB
Whisper large-v3-turboOpenAI · 0.81B · Speech to textPlatform path documented · whisper.cpp / CUDA · source7.83Documented path without a memory figurePlatform path documentedwhisper.cpp / CUDA · source
Whisper large-v3OpenAI · 1.55B · Speech to textPlatform path documented · whisper.cpp / CUDA · source7.44Documented path without a memory figurePlatform path documentedwhisper.cpp / CUDA · source
Step-Audio-R1.1stepfun-ai · 33.5B · Speech to text16-bit est. · params · sourceNo comparable published result~83 GB16-bit est. · paramssource
Voxtral-Small-24B-2507mistralai · 24.3B · Speech to text16-bit est. · params · sourceNo comparable published result~61 GB16-bit est. · paramssource
Kimi-Audio-7Bmoonshotai · 9.8B · Text to speech16-bit est. · params · sourceNo comparable published result~26 GB16-bit est. · paramssource
Kimi-Audio-7B-Instructmoonshotai · 9.8B · Text to speech16-bit est. · params · sourceNo comparable published result~26 GB16-bit est. · paramssource
AuKtencent · 1.5B · Text to speechPublisher minimum · CUDA · sourceNo comparable published result~25 GBPublisher minimumCUDA · source
AuK-Flashtencent · Text to speechPublisher minimum · CUDA · sourceNo comparable published result~25 GBPublisher minimumCUDA · source
8 of 134

33 more audio models in the catalog have no usable memory figure or parameter count for this profile. Browse Audio models

Embeddings

91 run well within 640 GB

Embeddings · MTEB (Multilingual, v2) · mean(task)

0–100 · higher is better

  • Color = publisher

Source: MTEB results (curated rows, sourced per model)

Embeddings models that run well within 640 GB
Qwen3-Embedding 8BAlibaba (Qwen)8-bit · published params · source70.6~12 GB8-bit · published paramssource
Qwen3-Embedding 4BAlibaba (Qwen)8-bit · published params · source69.5~7 GB8-bit · published paramssource
Harrier OSS v1 (0.6B)Microsoft8-bit · published params · source69.0~3 GB8-bit · published paramssource
Qwen3-Embedding 0.6BAlibaba (Qwen)8-bit · published params · source64.3~3 GB8-bit · published paramssource
Multilingual E5 Large Instructintfloat8-bit · published params · source63.2~3 GB8-bit · published paramssource
EmbeddingGemma 2Google8-bit · published params · source61.4~3 GB8-bit · published paramssource
EmbeddingGemma (300M)Google8-bit · published params · source61.1~3 GB8-bit · published paramssource
BGE-M3BAAI8-bit · published params · source59.6~3 GB8-bit · published paramssource
8 of 91

Browse Embeddings models

Runs well within 640 GB of aggregate GPU memory · planning estimates, not measured speed bundled catalog copy · 1,777 repositories · checked

Measurements

Published tokens per second, exactly as measured; never estimated.

Published measurements on this hardware
DeepSeek V4 Flash FP8sgl-project/DeepSeek-V4-Flash-FP8 (FP8 repa…NVIDIA HGX H100 · 8× H100 80 GB (TP=8) · Lambda Cloud39 per user (32 concurrent)FP8 (sgl-project/DeepSeek-V4-Flash-FP8)
8192 / 1024 · 32 concurrent requests; per…
SGLang with EAGLE speculative decoding (docker lmsysorg/sglang:deepseek-v4-hopper)
docker lmsysorg/sglang:deepseek-v4-hopp…
Lambda inference models · DeepSeek-V4-Flash (vendor benchmark)
2026-10-09

Specifications

19 sourced fields; the source is named once per group, and a field from another source links to it. Official specifications

Aggregate GPU memory640 GB8 GPUs · summed, not one address space · source
Memory bandwidth3,350 GB/sManufacturer spec · not model speed · source

Marker: the best LiveBench-scored text model that runs well here. No other text model would run well with the next memory size. Q4 planning estimate, not a measured fit.

Selected configuration

Latest check · Manufacturer source

Memory capacity
640 GB
Memory type
8 × 80 GB HBM3
Chip / processor
8 × NVIDIA H100
Host CPUs
2 × Intel Xeon 8480C
CPU cores (two CPUs combined)
112
Installed storage (system specification)
OS: 2 × 1.92 TB NVMe SSD (RAID 1); cache: 8 × 3.84 TB NVMe SSD (RAID 0)
Memory bandwidth
3,350 GB/s
CUDA cores
16,896 per GPU
GPU power per GPU (manufacturer specification)
700 W

Family specification

Latest check · Manufacturer source

Storage options (family)
OS: 2 × 1.92 TB NVMe M.2 SSD in RAID 1
GPU interconnect
NVLink GPU-to-GPU: 900 GB/s
Network connections (system specification)
4 × OSFP / 8 × ConnectX-7 400 Gbps · 2 × dual-port ConnectX-7 · BMC 1 GbE RJ45 · management 100 GbE + 10 GbE
Maximum system input power (specification)
10,200 W
CUDA compute capability
9.0
Dimensions (cm)
35.6 (H) × 48.23 (W) × 89.71 (D), max
Weight
130.45 kg
Operating system
DGX OS (Ubuntu server distribution)
Announcement date
2022-03-22
Introduced
2022

Eight 80 GB GPUs provide 640 GB in aggregate, not one 640 GB address space. Sharding, interconnect and runtime configuration are required for models exceeding one GPU.

Photo: Official manufacturer image. · Manufacturer source

Memory bandwidth is not measured model speed; verify runtime and checkpoint compatibility. Hardware specification reviewed: 2026-10-13.

How estimates work
Candidates
Reviewed pages use published parameter counts; automatic text candidates use official repository tensor counts (packed quantized weights and unsupported architectures excluded).
Planning
Q4 weights + 20% allowance + reserve: 25% of unified or system memory (at least 2 GB, at most 8 GB; 16 GB here) or 2 GB per dedicated GPU. Adjustable in “Adjust the estimate”. No KV cache is calculated from context.
Fit tiers
Runs well: the estimate fits the fast memory (VRAM, or unified memory within the macOS GPU share). Runs slowly: within the machine's memory but outside the fast part; a graphics card offloads layers to PC RAM (the listed RAM, or 32 GB assumed). Doesn't run: above both. Tiers are planning estimates, never a speed.
Vision
Vision-language models use the Q4 planning estimate from the parameter count (published by the model page, or read from Hugging Face). Classic vision models (classify, detect, segment) use a 16-bit estimate. No vision benchmark is tracked, so rows carry no score.
Media and embeddings
Media rows use documented execution paths (exact hardware or platform) and publisher minimums; a published requirement above this memory moves a model to the next tier. Without either, a 16-bit estimate from the published parameter count (× 1.2 + 2 GB), labelled as such. Embeddings use 8-bit weights + 20% + 2 GB.

Local AI basics: parameters, Q4 and memory · Full model catalog · Data and methodology