What is an embedding model, and which one runs locally?
The small models behind search and RAG, how they are scored and how little memory they need.
Short answer
A small model that turns text into numbers, so a program can find passages with a similar meaning. Almost any computer runs one.
- Used for: Search and RAG: answering questions from your own documents.
- Memory: Qwen3-Embedding 0.6B needs about 3 GB by the site's 8-bit estimate.
- Choosing: Prefer a multilingual model if your texts are not in English.
Estimate: 8-bit weights, 20% allowance, 2 GB reserve. Formulas
What it does
A chat model writes text. An embedding model writes nothing: it reads a sentence or a document and returns a fixed-length vector, a list of numbers. Texts with similar meanings get vectors that point the same way, so a program can compare a question with thousands of stored passages and return the closest ones. That is how semantic search works, and it is the retrieval half of RAG, where the passages found are handed to a chat model to answer from.
Because the model only reads, it can be small. Most curated embeddings on this site have under one billion parameters; even the largest, Qwen3-Embedding 8B, needs about 12 GB by the site's estimate. The vector size (dimensions) sets how much storage each passage takes in your index.
Scores and sizes
MTEB (Multilingual, v2), mean over tasks, independent runs only (publisher-reported scores are left out so the scale is the same). Memory is the 8-bit planning estimate.
| Model · parameters · dimensions · context | MTEB score (0–100) | Memory (GB) |
|---|---|---|
| Qwen3-Embedding 8B Alibaba (Qwen) · 7.57B · 4,096d · 32,768 tokens | 70.58 | 12 |
| Qwen3-Embedding 4B Alibaba (Qwen) · 4.02B · 2,560d · 32,768 tokens | 69.45 | 7 |
| Harrier OSS v1 (0.6B) Microsoft · 0.6B · 1,024d · 32,768 tokens | 69.01 | 3 |
| Qwen3-Embedding 0.6B Alibaba (Qwen) · 0.6B · 1,024d · 32,768 tokens | 64.34 | 3 |
| Multilingual E5 Large Instruct intfloat · 0.56B · 1,024d · 512 tokens | 63.22 | 3 |
| EmbeddingGemma (300M) Google · 0.3B · 768d · 2,048 tokens | 61.14 | 3 |
| BGE-M3 BAAI · 0.57B · 1,024d · 8,192 tokens | 59.56 | 3 |
| Nomic Embed Text v2 (MoE) Nomic AI · 0.48B · 768d · 512 tokens | 57.64 | 3 |
| Multilingual E5 Base intfloat · 0.28B · 768d · 512 tokens | 57.03 | 3 |
| Snowflake Arctic Embed L v2.0 Snowflake · 0.57B · 1,024d · 8,192 tokens | 57.03 | 3 |
Scores: (MTEB results dataset, read Oct 25, 2026)
How to choose
- Your language
- Prefer a multilingual modelEnglish-only models score far lower on the multilingual benchmark; check each page's language and PT-BR evidence.
- Long documents
- Look at the contextA short context means splitting documents into more, smaller passages.
- Index size
- Fewer dimensions store lessSome models can shorten their vectors (Matryoshka); the page says when the publisher supports it.
- Same model, always
- Index and query with one modelVectors from two different models cannot be compared; changing models means re-indexing.
Side by side: BGE-M3 vs Qwen3-Embedding 0.6B for local search · EmbeddingGemma vs multilingual-e5-large for embeddings