Skip to content
Local AI Wizard
Your machine
Theme

What is an embedding model, and which one runs locally?

The small models behind search and RAG, how they are scored and how little memory they need.

Models and families · updated

Short answer

A small model that turns text into numbers, so a program can find passages with a similar meaning. Almost any computer runs one.

  • Used for: Search and RAG: answering questions from your own documents.
  • Memory: Qwen3-Embedding 0.6B needs about 3 GB by the site's 8-bit estimate.
  • Choosing: Prefer a multilingual model if your texts are not in English.

Estimate: 8-bit weights, 20% allowance, 2 GB reserve. Formulas

What it does

A chat model writes text. An embedding model writes nothing: it reads a sentence or a document and returns a fixed-length vector, a list of numbers. Texts with similar meanings get vectors that point the same way, so a program can compare a question with thousands of stored passages and return the closest ones. That is how semantic search works, and it is the retrieval half of RAG, where the passages found are handed to a chat model to answer from.

Because the model only reads, it can be small. Most curated embeddings on this site have under one billion parameters; even the largest, Qwen3-Embedding 8B, needs about 12 GB by the site's estimate. The vector size (dimensions) sets how much storage each passage takes in your index.

Scores and sizes

MTEB (Multilingual, v2), mean over tasks, independent runs only (publisher-reported scores are left out so the scale is the same). Memory is the 8-bit planning estimate.

Curated embedding models by MTEB (Multilingual, v2) score, with parameters, dimensions, context and memory estimate.
Model · parameters · dimensions · contextMTEB score (0–100)Memory (GB)
Qwen3-Embedding 8B
Alibaba (Qwen) · 7.57B · 4,096d · 32,768 tokens
70.5812
Qwen3-Embedding 4B
Alibaba (Qwen) · 4.02B · 2,560d · 32,768 tokens
69.457
Harrier OSS v1 (0.6B)
Microsoft · 0.6B · 1,024d · 32,768 tokens
69.013
Qwen3-Embedding 0.6B
Alibaba (Qwen) · 0.6B · 1,024d · 32,768 tokens
64.343
Multilingual E5 Large Instruct
intfloat · 0.56B · 1,024d · 512 tokens
63.223
EmbeddingGemma (300M)
Google · 0.3B · 768d · 2,048 tokens
61.143
BGE-M3
BAAI · 0.57B · 1,024d · 8,192 tokens
59.563
Nomic Embed Text v2 (MoE)
Nomic AI · 0.48B · 768d · 512 tokens
57.643
Multilingual E5 Base
intfloat · 0.28B · 768d · 512 tokens
57.033
Snowflake Arctic Embed L v2.0
Snowflake · 0.57B · 1,024d · 8,192 tokens
57.033

Scores: (MTEB results dataset, read Oct 25, 2026)

How to choose

Your language
Prefer a multilingual modelEnglish-only models score far lower on the multilingual benchmark; check each page's language and PT-BR evidence.
Long documents
Look at the contextA short context means splitting documents into more, smaller passages.
Index size
Fewer dimensions store lessSome models can shorten their vectors (Matryoshka); the page says when the publisher supports it.
Same model, always
Index and query with one modelVectors from two different models cannot be compared; changing models means re-indexing.

Side by side: BGE-M3 vs Qwen3-Embedding 0.6B for local search · EmbeddingGemma vs multilingual-e5-large for embeddings