Skip to content
Local AI Wizard
Your machine
Theme

BGE-M3 vs Qwen3-Embedding 0.6B for local search

Two small multilingual embedding models for local search and RAG

Facts from the reviewed catalog

Facts from the reviewed catalog
SpecificationBGE-M3Qwen3-Embedding 0.6B
TaskEmbeddings & searchEmbeddings & search
Parameters568M596M
LicenseMITApache 2.0
Commercial useAllowedlicense textAllowedlicense text
Quality · MTEB (Multilingual, v2)59.561,024d · independent run · source64.341,024d · independent run · source
Memory8-bit embedding estimate (~3 GB)8-bit embedding estimate (~3 GB)
Checkpointofficial cardofficial card

In MTEB (Multilingual, v2), Qwen3-Embedding 0.6B scores 64.34 and BGE-M3 59.56, 4.8 points apart on the same benchmark version. Independent runs from the public MTEB results dataset, same benchmark version for both models. Scale: mean over tasks, 0–100, higher is better.

Memory on three machines

8-bit embedding estimates: weights plus a 20% allowance and a flat 2 GB reserve, so the figure is the same on every machine. Planning estimates, not measured runs.

BGE-M3 vs Qwen3-Embedding 0.6B for local search: memory on three machines
MachineBGE-M3Qwen3-Embedding 0.6B
RTX 4060 · 8 GB VRAM~3 GBFast memory holds it~3 GBFast memory holds it
MacBook Air 13-inch M4 · 16 GB unified~3 GBFast memory holds it~3 GBFast memory holds it
Mac mini M4 · 16 GB unified~3 GBFast memory holds it~3 GBFast memory holds it

BGE-M3: 8-bit embedding estimate (~3 GB). Qwen3-Embedding 0.6B: 8-bit embedding estimate (~3 GB). Formulas

Everything that runs on each machine: RTX 4060 · 8 GB VRAM · MacBook Air 13-inch M4 · 16 GB unified · Mac mini M4 · 16 GB unified

Which to choose

Both have about 0.6 billion parameters and 1,024-dimension vectors, so both run on almost any machine. Qwen3-Embedding 0.6B scores higher on MTEB (Multilingual, v2) in an independent run and reads up to 32,768 tokens; BGE-M3 reads 8,192. BGE-M3's advantage is that one model returns dense, sparse and multi-vector (ColBERT) outputs, useful for hybrid search. BGE-M3 is MIT and Qwen3-Embedding Apache 2.0; both allow commercial use.

Files and run commands

The default file of the most downloaded collected source, or the commands each card documents. Not run by this site.

BGE-M3

Documented ways to run it

Copied as printed in the linked source (read 2026-10-18); not run by this site.

Setup stated first by the source · FlagEmbedding
pip install -U FlagEmbedding
FlagEmbedding · model card · Generate Embedding for text
from FlagEmbedding import BGEM3FlagModel

model = BGEM3FlagModel('BAAI/bge-m3',  
                       use_fp16=True) # Setting use_fp16 to True speeds up computation with a slight performance degradation

sentences_1 = ["What is BGE M3?", "Defination of BM25"]
sentences_2 = ["BGE M3 is an embedding model supporting dense retrieval, lexical matching and multi-vector interaction.", 
               "BM25 is a bag-of-words retrieval function that ranks a set of documents based on the query terms appearing in each document"]

embeddings_1 = model.encode(sentences_1, 
                            batch_size=12, 
                            max_length=8192, # If you don't need such a long length, you can set a smaller value to speed up the encoding process.
                            )['dense_vecs']
embeddings_2 = model.encode(sentences_2)['dense_vecs']
similarity = embeddings_1 @ embeddings_2.T
print(similarity)
# [[0.6265, 0.3477], [0.3499, 0.678 ]]
Setup stated first by the source · FlagEmbedding
pip install -U FlagEmbedding
FlagEmbedding · model card · Generate Embedding for text
from FlagEmbedding import BGEM3FlagModel

model = BGEM3FlagModel('BAAI/bge-m3',  use_fp16=True) # Setting use_fp16 to True speeds up computation with a slight performance degradation

sentences_1 = ["What is BGE M3?", "Defination of BM25"]
sentences_2 = ["BGE M3 is an embedding model supporting dense retrieval, lexical matching and multi-vector interaction.", 
               "BM25 is a bag-of-words retrieval function that ranks a set of documents based on the query terms appearing in each document"]

output_1 = model.encode(sentences_1, return_dense=True, return_sparse=True, return_colbert_vecs=False)
output_2 = model.encode(sentences_2, return_dense=True, return_sparse=True, return_colbert_vecs=False)

# you can see the weight for each token:
print(model.convert_id_to_token(output_1['lexical_weights']))
# [{'What': 0.08356, 'is': 0.0814, 'B': 0.1296, 'GE': 0.252, 'M': 0.1702, '3': 0.2695, '?': 0.04092}, 
#  {'De': 0.05005, 'fin': 0.1368, 'ation': 0.04498, 'of': 0.0633, 'BM': 0.2515, '25': 0.3335}]


# compute the scores via lexical mathcing
lexical_scores = model.compute_lexical_matching_score(output_1['lexical_weights'][0], output_2['lexical_weights'][0])
print(lexical_scores)
# 0.19554901123046875

print(model.compute_lexical_matching_score(output_1['lexical_weights'][0], output_1['lexical_weights'][1]))
# 0.0

Qwen3-Embedding 0.6B

Documented ways to run it

Copied as printed in the linked source (read 2026-10-18); not run by this site.

Sentence Transformers · model card · Sentence Transformers Usage
# Requires transformers>=4.51.0
# Requires sentence-transformers>=2.7.0

from sentence_transformers import SentenceTransformer

# Load the model
model = SentenceTransformer("Qwen/Qwen3-Embedding-0.6B")

# We recommend enabling flash_attention_2 for better acceleration and memory saving,
# together with setting `padding_side` to "left":
# model = SentenceTransformer(
#     "Qwen/Qwen3-Embedding-0.6B",
#     model_kwargs={"attn_implementation": "flash_attention_2", "device_map": "auto"},
#     tokenizer_kwargs={"padding_side": "left"},
# )

# The queries and documents to embed
queries = [
    "What is the capital of China?",
    "Explain gravity",
]
documents = [
    "The capital of China is Beijing.",
    "Gravity is a force that attracts two bodies towards each other. It gives weight to physical objects and is responsible for the movement of planets around the sun.",
]

# Encode the queries and documents. Note that queries benefit from using a prompt
# Here we use the prompt called "query" stored under `model.prompts`, but you can
# also pass your own prompt via the `prompt` argument
query_embeddings = model.encode(queries, prompt_name="query")
document_embeddings = model.encode(documents)

# Compute the (cosine) similarity between the query and document embeddings
similarity = model.similarity(query_embeddings, document_embeddings)
print(similarity)
# tensor([[0.7646, 0.1414],
#         [0.1355, 0.6000]])
Transformers · model card · Transformers Usage
# Requires transformers>=4.51.0

import torch
import torch.nn.functional as F

from torch import Tensor
from transformers import AutoTokenizer, AutoModel


def last_token_pool(last_hidden_states: Tensor,
                 attention_mask: Tensor) -> Tensor:
    left_padding = (attention_mask[:, -1].sum() == attention_mask.shape[0])
    if left_padding:
        return last_hidden_states[:, -1]
    else:
        sequence_lengths = attention_mask.sum(dim=1) - 1
        batch_size = last_hidden_states.shape[0]
        return last_hidden_states[torch.arange(batch_size, device=last_hidden_states.device), sequence_lengths]


def get_detailed_instruct(task_description: str, query: str) -> str:
    return f'Instruct: {task_description}\nQuery:{query}'

# Each query must come with a one-sentence instruction that describes the task
task = 'Given a web search query, retrieve relevant passages that answer the query'

queries = [
    get_detailed_instruct(task, 'What is the capital of China?'),
    get_detailed_instruct(task, 'Explain gravity')
]
# No need to add instruction for retrieval documents
documents = [
    "The capital of China is Beijing.",
    "Gravity is a force that attracts two bodies towards each other. It gives weight to physical objects and is responsible for the movement of planets around the sun."
]
input_texts = queries + documents

tokenizer = AutoTokenizer.from_pretrained('Qwen/Qwen3-Embedding-0.6B', padding_side='left')
model = AutoModel.from_pretrained('Qwen/Qwen3-Embedding-0.6B')

# We recommend enabling flash_attention_2 for better acceleration and memory saving.
# model = AutoModel.from_pretrained('Qwen/Qwen3-Embedding-0.6B', attn_implementation="flash_attention_2", torch_dtype=torch.float16).cuda()

max_length = 8192

# Tokenize the input texts
batch_dict = tokenizer(
    input_texts,
    padding=True,
    truncation=True,
    max_length=max_length,
    return_tensors="pt",
)
batch_dict.to(model.device)
outputs = model(**batch_dict)
embeddings = last_token_pool(outputs.last_hidden_state, batch_dict['attention_mask'])

# normalize embeddings
embeddings = F.normalize(embeddings, p=2, dim=1)
scores = (embeddings[:2] @ embeddings[2:].T)
print(scores.tolist())
# [[0.7645568251609802, 0.14142508804798126], [0.13549736142158508, 0.5999549627304077]]
vLLM · model card · vLLM Usage
# Requires vllm>=0.8.5
import torch
import vllm
from vllm import LLM

def get_detailed_instruct(task_description: str, query: str) -> str:
    return f'Instruct: {task_description}\nQuery:{query}'

# Each query must come with a one-sentence instruction that describes the task
task = 'Given a web search query, retrieve relevant passages that answer the query'

queries = [
    get_detailed_instruct(task, 'What is the capital of China?'),
    get_detailed_instruct(task, 'Explain gravity')
]
# No need to add instruction for retrieval documents
documents = [
    "The capital of China is Beijing.",
    "Gravity is a force that attracts two bodies towards each other. It gives weight to physical objects and is responsible for the movement of planets around the sun."
]
input_texts = queries + documents

model = LLM(model="Qwen/Qwen3-Embedding-0.6B", task="embed")

outputs = model.embed(input_texts)
embeddings = torch.tensor([o.outputs.embedding for o in outputs])
scores = (embeddings[:2] @ embeddings[2:].T)
print(scores.tolist())
# [[0.7620252966880798, 0.14078938961029053], [0.1358368694782257, 0.6013815999031067]]

Numbers each publisher reports

Each card's own results with its own settings. Publishers choose different tests and settings, so these values are never compared across publishers here.

BGE-M3 · reported by BAAI

The publisher's card states no benchmark result for this checkpoint (read 2026-10-25). not published on the card · read 2026-10-25

Qwen3-Embedding 0.6B · reported by Alibaba (Qwen)

28 results · Alibaba (Qwen)'s model card · read 2026-10-25
Mean (Task)
64.33MTEB (Multilingual) · MTEB
Mean (Type)
56.00MTEB (Multilingual) · MTEB
Bitxt Mining
72.22MTEB (Multilingual) · MTEB
Class.
66.83MTEB (Multilingual) · MTEB
Clust.
52.33MTEB (Multilingual) · MTEB
Inst. Retri.
5.09MTEB (Multilingual) · MTEB
Multi. Class.
24.59MTEB (Multilingual) · MTEB
Pair. Class.
80.83MTEB (Multilingual) · MTEB
Every reported result28
Rerank
61.41MTEB (Multilingual) · MTEB
Retri.
64.64MTEB (Multilingual) · MTEB
STS
76.17MTEB (Multilingual) · MTEB
Mean (Task)
70.70MTEB (Eng v2) · MTEB
Mean (Type)
64.88MTEB (Eng v2) · MTEB
Class.
85.76MTEB (Eng v2) · MTEB
Clust.
54.05MTEB (Eng v2) · MTEB
Pair Class.
84.37MTEB (Eng v2) · MTEB
Rerank.
48.18MTEB (Eng v2) · MTEB
Retri.
61.83MTEB (Eng v2) · MTEB
STS
86.57MTEB (Eng v2) · MTEB
Summ.
33.43MTEB (Eng v2) · MTEB
Mean (Task)
66.33C-MTEB (MTEB Chinese) · MTEB
Mean (Type)
67.45C-MTEB (MTEB Chinese) · MTEB
Class.
71.40C-MTEB (MTEB Chinese) · MTEB
Clust.
68.74C-MTEB (MTEB Chinese) · MTEB
Pair Class.
76.42C-MTEB (MTEB Chinese) · MTEB
Rerank.
62.58C-MTEB (MTEB Chinese) · MTEB
Retr.
71.03C-MTEB (MTEB Chinese) · MTEB
STS
54.52C-MTEB (MTEB Chinese) · MTEB

Values as printed on the card, with the publisher's own settings (card section: Evaluation › MTEB (Multilingual); Evaluation › MTEB (Eng v2)); column “Qwen3-Embedding-0.6B”. Not comparable across publishers and not used in any ranking. Model card

Limits and dates

The published quality benchmark does not test a local quantized build. Memory estimates are not speed measurements or guaranteed hardware fits. Repository size, when available, is not VRAM. Formulas

Page updated · Catalog reviewed · MTEB (Multilingual, v2)