Pular para o conteúdo
Local AI Wizard
Sua máquina
Tema

EmbeddingGemma vs multilingual-e5-large para embeddings

Um modelo de embeddings multilíngue minúsculo e um médio

Fatos do catálogo revisado

Fatos do catálogo revisado
EspecificaçãoEmbeddingGemma (300M)Multilingual E5 Large Instruct
TarefaEmbeddings e buscaEmbeddings e busca
Parâmetros303M560M
LicençaTermos do publicador: Gemma Terms of UseMIT
Uso comercialPermitidotexto da licençaPermitidotexto da licença
Qualidade · MTEB (Multilingual, v2)61,14768d · execução independente · fonte63,221.024d · execução independente · fonte
Memóriaestimativa de embeddings de 8 bits (~3 GB)estimativa de embeddings de 8 bits (~3 GB)
Checkpointficha oficialficha oficial

No MTEB (Multilingual, v2), o Multilingual E5 Large Instruct tem 63,22 e o EmbeddingGemma (300M) 61,14, 2,1 pontos de diferença na mesma versão do benchmark. Execuções independentes do conjunto público de resultados do MTEB, mesma versão do benchmark para os dois modelos. Escala: média das tarefas, 0 a 100, maior é melhor.

Memória em três máquinas

Estimativas de embeddings de 8 bits: pesos mais 20% de folga e reserva fixa de 2 GB, então o valor é o mesmo em todas as máquinas. Estimativas de planejamento, não execuções medidas.

EmbeddingGemma vs multilingual-e5-large para embeddings: memória em três máquinas
MáquinaEmbeddingGemma (300M)Multilingual E5 Large Instruct
RTX 4060 · 8 GB de VRAM~3 GBCabe na memória rápida~3 GBCabe na memória rápida
MacBook Air 13 polegadas M4 · 16 GB unificados~3 GBCabe na memória rápida~3 GBCabe na memória rápida
Mac mini M4 · 16 GB unificados~3 GBCabe na memória rápida~3 GBCabe na memória rápida

EmbeddingGemma (300M): estimativa de embeddings de 8 bits (~3 GB). Multilingual E5 Large Instruct: estimativa de embeddings de 8 bits (~3 GB). Fórmulas

Tudo o que roda em cada máquina: RTX 4060 · 8 GB de VRAM · MacBook Air 13 polegadas M4 · 16 GB unificados · Mac mini M4 · 16 GB unificados

Qual escolher

O EmbeddingGemma tem 0,3 bilhão de parâmetros e vetores de 768 dimensões; o Multilingual E5 Large Instruct tem 0,56 bilhão e 1.024 dimensões. Em execuções independentes do MTEB (Multilingual, v2) o E5 Large Instruct tem nota maior, e o EmbeddingGemma tem cerca da metade do tamanho, o que pesa em celulares e máquinas pequenas. O EmbeddingGemma lê até 2.048 tokens e o E5 até 512, então trechos longos precisam ser mais divididos no E5. O E5 é MIT; o EmbeddingGemma usa os Termos de Uso do Gemma, que permitem uso comercial.

Arquivos e comandos para rodar

O arquivo padrão da fonte coletada mais baixada, ou os comandos que cada ficha documenta. Não executados por este site.

EmbeddingGemma (300M)

Formas documentadas de rodar

Não foi possível ler a ficha do modelo (ausente ou restrita)

Multilingual E5 Large Instruct

Formas documentadas de rodar

Copiado como aparece na fonte vinculada (lida em 2026-10-18); não executado por este site.

Sentence Transformers · ficha do modelo · Sentence Transformers
from sentence_transformers import SentenceTransformer

def get_detailed_instruct(task_description: str, query: str) -> str:
    return f'Instruct: {task_description}\nQuery: {query}'

# Each query must come with a one-sentence instruction that describes the task
task = 'Given a web search query, retrieve relevant passages that answer the query'
queries = [
    get_detailed_instruct(task, 'how much protein should a female eat'),
    get_detailed_instruct(task, '南瓜的家常做法')
]
# No need to add instruction for retrieval documents
documents = [
    "As a general guideline, the CDC's average requirement of protein for women ages 19 to 70 is 46 grams per day. But, as you can see from this chart, you'll need to increase that if you're expecting or training for a marathon. Check out the chart below to see how much protein you should be eating each day.",
    "1.清炒南瓜丝 原料:嫩南瓜半个 调料:葱、盐、白糖、鸡精 做法: 1、南瓜用刀薄薄的削去表面一层皮,用勺子刮去瓤 2、擦成细丝(没有擦菜板就用刀慢慢切成细丝) 3、锅烧热放油,入葱花煸出香味 4、入南瓜丝快速翻炒一分钟左右,放盐、一点白糖和鸡精调味出锅 2.香葱炒南瓜 原料:南瓜1只 调料:香葱、蒜末、橄榄油、盐 做法: 1、将南瓜去皮,切成片 2、油锅8成热后,将蒜末放入爆香 3、爆香后,将南瓜片放入,翻炒 4、在翻炒的同时,可以不时地往锅里加水,但不要太多 5、放入盐,炒匀 6、南瓜差不多软和绵了之后,就可以关火 7、撒入香葱,即可出锅"
]
input_texts = queries + documents

model = SentenceTransformer('intfloat/multilingual-e5-large-instruct')

embeddings = model.encode(input_texts, convert_to_tensor=True, normalize_embeddings=True)
scores = (embeddings[:2] @ embeddings[2:].T) * 100
print(scores.tolist())
# [[91.92853546142578, 67.5802993774414], [70.38143157958984, 92.13307189941406]]
Transformers · ficha do modelo · Transformers
import torch.nn.functional as F

from torch import Tensor
from transformers import AutoTokenizer, AutoModel


def average_pool(last_hidden_states: Tensor,
                 attention_mask: Tensor) -> Tensor:
    last_hidden = last_hidden_states.masked_fill(~attention_mask[..., None].bool(), 0.0)
    return last_hidden.sum(dim=1) / attention_mask.sum(dim=1)[..., None]

def get_detailed_instruct(task_description: str, query: str) -> str:
    return f'Instruct: {task_description}\nQuery: {query}'

# Each query must come with a one-sentence instruction that describes the task
task = 'Given a web search query, retrieve relevant passages that answer the query'
queries = [
    get_detailed_instruct(task, 'how much protein should a female eat'),
    get_detailed_instruct(task, '南瓜的家常做法')
]
# No need to add instruction for retrieval documents
documents = [
    "As a general guideline, the CDC's average requirement of protein for women ages 19 to 70 is 46 grams per day. But, as you can see from this chart, you'll need to increase that if you're expecting or training for a marathon. Check out the chart below to see how much protein you should be eating each day.",
    "1.清炒南瓜丝 原料:嫩南瓜半个 调料:葱、盐、白糖、鸡精 做法: 1、南瓜用刀薄薄的削去表面一层皮,用勺子刮去瓤 2、擦成细丝(没有擦菜板就用刀慢慢切成细丝) 3、锅烧热放油,入葱花煸出香味 4、入南瓜丝快速翻炒一分钟左右,放盐、一点白糖和鸡精调味出锅 2.香葱炒南瓜 原料:南瓜1只 调料:香葱、蒜末、橄榄油、盐 做法: 1、将南瓜去皮,切成片 2、油锅8成热后,将蒜末放入爆香 3、爆香后,将南瓜片放入,翻炒 4、在翻炒的同时,可以不时地往锅里加水,但不要太多 5、放入盐,炒匀 6、南瓜差不多软和绵了之后,就可以关火 7、撒入香葱,即可出锅"
]
input_texts = queries + documents

tokenizer = AutoTokenizer.from_pretrained('intfloat/multilingual-e5-large-instruct')
model = AutoModel.from_pretrained('intfloat/multilingual-e5-large-instruct')

# Tokenize the input texts
batch_dict = tokenizer(input_texts, max_length=512, padding=True, truncation=True, return_tensors='pt')

outputs = model(**batch_dict)
embeddings = average_pool(outputs.last_hidden_state, batch_dict['attention_mask'])

# normalize embeddings
embeddings = F.normalize(embeddings, p=2, dim=1)
scores = (embeddings[:2] @ embeddings[2:].T) * 100
print(scores.tolist())
# => [[91.92852783203125, 67.580322265625], [70.3814468383789, 92.1330795288086]]

Números que cada publicador informa

Os resultados de cada ficha, com a configuração de cada uma. Publicadores escolhem testes e configurações diferentes, então estes valores nunca são comparados entre publicadores aqui.

EmbeddingGemma (300M) · informado por Google

6 resultados · ficha de Google · lida em 2026-10-25
Mean (Task)
61,15MTEB (Multilingual, v2)
Mean (TaskType)
54,31MTEB (Multilingual, v2)
Mean (Task)
69,67MTEB (English, v2)
Mean (TaskType)
65,11MTEB (English, v2)
Mean (Task)
68,76MTEB (Code, v1)
Mean (TaskType)
68,76MTEB (Code, v1)

Valores como impressos na ficha, com a configuração do próprio publicador (seção da ficha: Evaluation › Benchmark Results › Full Precision Checkpoint); coluna “768d”. Não comparáveis entre publicadores e fora de qualquer ranking. Ficha do modelo

Multilingual E5 Large Instruct · informado por intfloat

293 resultados · ficha de intfloat · lida em 2026-10-25
MTEB AmazonCounterfactualClassification (en) (en; test; accuracy)
76,239Classification
MTEB AmazonCounterfactualClassification (de) (de; test; accuracy)
66,713Classification
MTEB AmazonCounterfactualClassification (en-ext) (en-ext; test; accuracy)
73,853Classification
MTEB AmazonCounterfactualClassification (ja) (ja; test; accuracy)
76,049Classification
MTEB AmazonPolarityClassification (test; accuracy)
96,287Classification
MTEB AmazonReviewsClassification (en) (en; test; accuracy)
56,716Classification
MTEB AmazonReviewsClassification (de) (de; test; accuracy)
53,000Classification
MTEB AmazonReviewsClassification (es) (es; test; accuracy)
48,806Classification
Todos os resultados informados293
MTEB AmazonReviewsClassification (fr) (fr; test; accuracy)
48,508Classification
MTEB AmazonReviewsClassification (ja) (ja; test; accuracy)
47,710Classification
MTEB AmazonReviewsClassification (zh) (zh; test; accuracy)
44,662Classification
MTEB ArguAna (test; ndcg_at_10)
58,384Retrieval
MTEB ArxivClusteringP2P (test; v_measure)
46,404Clustering
MTEB ArxivClusteringS2S (test; v_measure)
40,486Clustering
MTEB AskUbuntuDupQuestions (test; map)
63,889Reranking
MTEB BIOSSES (test; cos_sim_spearman)
86,962STS
MTEB BUCC (de-en) (de-en; test; f1)
99,607BitextMining
MTEB BUCC (fr-en) (fr-en; test; f1)
99,145BitextMining
MTEB BUCC (ru-en) (ru-en; test; f1)
97,866BitextMining
MTEB BUCC (zh-en) (zh-en; test; f1)
99,386BitextMining
MTEB Banking77Classification (test; accuracy)
85,734Classification
MTEB BiorxivClusteringP2P (test; v_measure)
40,935Clustering
MTEB BiorxivClusteringS2S (test; v_measure)
36,276Clustering
MTEB CQADupstackRetrieval (test; ndcg_at_10)
42,707Retrieval
MTEB ClimateFEVER (test; ndcg_at_10)
29,858Retrieval
MTEB DBPedia (test; ndcg_at_10)
38,36Retrieval
MTEB EmotionClassification (test; accuracy)
51,51Classification
MTEB FEVER (test; ndcg_at_10)
77,988Retrieval
MTEB FiQA2018 (test; ndcg_at_10)
47,712Retrieval
MTEB HotpotQA (test; ndcg_at_10)
69,317Retrieval
MTEB ImdbClassification (test; accuracy)
94,601Classification
MTEB MSMARCO (dev; ndcg_at_10)
40,43Retrieval
MTEB MTOPDomainClassification (en) (en; test; accuracy)
93,926Classification
MTEB MTOPDomainClassification (de) (de; test; accuracy)
92,677Classification
MTEB MTOPDomainClassification (es) (es; test; accuracy)
92,268Classification
MTEB MTOPDomainClassification (fr) (fr; test; accuracy)
89,966Classification
MTEB MTOPDomainClassification (hi) (hi; test; accuracy)
90,025Classification
MTEB MTOPDomainClassification (th) (th; test; accuracy)
87,555Classification
MTEB MTOPIntentClassification (en) (en; test; accuracy)
82,462Classification
MTEB MTOPIntentClassification (de) (de; test; accuracy)
79,372Classification
MTEB MTOPIntentClassification (es) (es; test; accuracy)
80,340Classification
MTEB MTOPIntentClassification (fr) (fr; test; accuracy)
76,721Classification
MTEB MTOPIntentClassification (hi) (hi; test; accuracy)
77,870Classification
MTEB MTOPIntentClassification (th) (th; test; accuracy)
77,736Classification
MTEB MassiveIntentClassification (af) (af; test; accuracy)
67,663Classification
MTEB MassiveIntentClassification (am) (am; test; accuracy)
60,713Classification
MTEB MassiveIntentClassification (ar) (ar; test; accuracy)
63,611Classification
MTEB MassiveIntentClassification (az) (az; test; accuracy)
69,001Classification
MTEB MassiveIntentClassification (bn) (bn; test; accuracy)
68,541Classification
MTEB MassiveIntentClassification (cy) (cy; test; accuracy)
63,033Classification
MTEB MassiveIntentClassification (da) (da; test; accuracy)
71,809Classification
MTEB MassiveIntentClassification (de) (de; test; accuracy)
72,471Classification
MTEB MassiveIntentClassification (el) (el; test; accuracy)
71,237Classification
MTEB MassiveIntentClassification (en) (en; test; accuracy)
77,058Classification
MTEB MassiveIntentClassification (es) (es; test; accuracy)
72,821Classification
MTEB MassiveIntentClassification (fa) (fa; test; accuracy)
74,254Classification
MTEB MassiveIntentClassification (fi) (fi; test; accuracy)
72,286Classification
MTEB MassiveIntentClassification (fr) (fr; test; accuracy)
73,315Classification
MTEB MassiveIntentClassification (he) (he; test; accuracy)
70,215Classification
MTEB MassiveIntentClassification (hi) (hi; test; accuracy)
71,584Classification
MTEB MassiveIntentClassification (hu) (hu; test; accuracy)
71,923Classification
MTEB MassiveIntentClassification (hy) (hy; test; accuracy)
68,073Classification
MTEB MassiveIntentClassification (id) (id; test; accuracy)
72,623Classification
MTEB MassiveIntentClassification (is) (is; test; accuracy)
65,770Classification
MTEB MassiveIntentClassification (it) (it; test; accuracy)
73,453Classification
MTEB MassiveIntentClassification (ja) (ja; test; accuracy)
74,691Classification
MTEB MassiveIntentClassification (jv) (jv; test; accuracy)
63,036Classification
MTEB MassiveIntentClassification (ka) (ka; test; accuracy)
58,914Classification
MTEB MassiveIntentClassification (km) (km; test; accuracy)
54,428Classification
MTEB MassiveIntentClassification (kn) (kn; test; accuracy)
66,335Classification
MTEB MassiveIntentClassification (ko) (ko; test; accuracy)
70,585Classification
MTEB MassiveIntentClassification (lv) (lv; test; accuracy)
69,112Classification
MTEB MassiveIntentClassification (ml) (ml; test; accuracy)
69,701Classification
MTEB MassiveIntentClassification (mn) (mn; test; accuracy)
66,439Classification
MTEB MassiveIntentClassification (ms) (ms; test; accuracy)
70,800Classification
MTEB MassiveIntentClassification (my) (my; test; accuracy)
64,788Classification
MTEB MassiveIntentClassification (nb) (nb; test; accuracy)
71,658Classification
MTEB MassiveIntentClassification (nl) (nl; test; accuracy)
74,425Classification
MTEB MassiveIntentClassification (pl) (pl; test; accuracy)
73,648Classification
MTEB MassiveIntentClassification (pt) (pt; test; accuracy)
73,628Classification
MTEB MassiveIntentClassification (ro) (ro; test; accuracy)
71,886Classification
MTEB MassiveIntentClassification (ru) (ru; test; accuracy)
74,159Classification
MTEB MassiveIntentClassification (sl) (sl; test; accuracy)
69,963Classification
MTEB MassiveIntentClassification (sq) (sq; test; accuracy)
69,496Classification
MTEB MassiveIntentClassification (sv) (sv; test; accuracy)
74,681Classification
MTEB MassiveIntentClassification (sw) (sw; test; accuracy)
63,006Classification
MTEB MassiveIntentClassification (ta) (ta; test; accuracy)
66,910Classification
MTEB MassiveIntentClassification (te) (te; test; accuracy)
67,623Classification
MTEB MassiveIntentClassification (th) (th; test; accuracy)
69,509Classification
MTEB MassiveIntentClassification (tl) (tl; test; accuracy)
69,307Classification
MTEB MassiveIntentClassification (tr) (tr; test; accuracy)
72,243Classification
MTEB MassiveIntentClassification (ur) (ur; test; accuracy)
67,498Classification
MTEB MassiveIntentClassification (vi) (vi; test; accuracy)
71,288Classification
MTEB MassiveIntentClassification (zh-CN) (zh-CN; test; accuracy)
73,315Classification
MTEB MassiveIntentClassification (zh-TW) (zh-TW; test; accuracy)
69,378Classification
MTEB MassiveScenarioClassification (af) (af; test; accuracy)
73,336Classification
MTEB MassiveScenarioClassification (am) (am; test; accuracy)
65,844Classification
MTEB MassiveScenarioClassification (ar) (ar; test; accuracy)
69,765Classification
MTEB MassiveScenarioClassification (az) (az; test; accuracy)
72,024Classification
MTEB MassiveScenarioClassification (bn) (bn; test; accuracy)
72,757Classification
MTEB MassiveScenarioClassification (cy) (cy; test; accuracy)
68,023Classification
MTEB MassiveScenarioClassification (da) (da; test; accuracy)
77,609Classification
MTEB MassiveScenarioClassification (de) (de; test; accuracy)
77,683Classification
MTEB MassiveScenarioClassification (el) (el; test; accuracy)
76,133Classification
MTEB MassiveScenarioClassification (en) (en; test; accuracy)
80,474Classification
MTEB MassiveScenarioClassification (es) (es; test; accuracy)
76,967Classification
MTEB MassiveScenarioClassification (fa) (fa; test; accuracy)
78,097Classification
MTEB MassiveScenarioClassification (fi) (fi; test; accuracy)
75,208Classification
MTEB MassiveScenarioClassification (fr) (fr; test; accuracy)
77,075Classification
MTEB MassiveScenarioClassification (he) (he; test; accuracy)
73,534Classification
MTEB MassiveScenarioClassification (hi) (hi; test; accuracy)
75,753Classification
MTEB MassiveScenarioClassification (hu) (hu; test; accuracy)
77,095Classification
MTEB MassiveScenarioClassification (hy) (hy; test; accuracy)
71,076Classification
MTEB MassiveScenarioClassification (id) (id; test; accuracy)
77,105Classification
MTEB MassiveScenarioClassification (is) (is; test; accuracy)
71,258Classification
MTEB MassiveScenarioClassification (it) (it; test; accuracy)
77,081Classification
MTEB MassiveScenarioClassification (ja) (ja; test; accuracy)
79,354Classification
MTEB MassiveScenarioClassification (jv) (jv; test; accuracy)
68,423Classification
MTEB MassiveScenarioClassification (ka) (ka; test; accuracy)
66,157Classification
MTEB MassiveScenarioClassification (km) (km; test; accuracy)
60,114Classification
MTEB MassiveScenarioClassification (kn) (kn; test; accuracy)
71,247Classification
MTEB MassiveScenarioClassification (ko) (ko; test; accuracy)
76,463Classification
MTEB MassiveScenarioClassification (lv) (lv; test; accuracy)
73,248Classification
MTEB MassiveScenarioClassification (ml) (ml; test; accuracy)
74,116Classification
MTEB MassiveScenarioClassification (mn) (mn; test; accuracy)
70,017Classification
MTEB MassiveScenarioClassification (ms) (ms; test; accuracy)
74,412Classification
MTEB MassiveScenarioClassification (my) (my; test; accuracy)
68,396Classification
MTEB MassiveScenarioClassification (nb) (nb; test; accuracy)
77,209Classification
MTEB MassiveScenarioClassification (nl) (nl; test; accuracy)
78,524Classification
MTEB MassiveScenarioClassification (pl) (pl; test; accuracy)
76,691Classification
MTEB MassiveScenarioClassification (pt) (pt; test; accuracy)
77,122Classification
MTEB MassiveScenarioClassification (ro) (ro; test; accuracy)
75,259Classification
MTEB MassiveScenarioClassification (ru) (ru; test; accuracy)
77,707Classification
MTEB MassiveScenarioClassification (sl) (sl; test; accuracy)
74,835Classification
MTEB MassiveScenarioClassification (sq) (sq; test; accuracy)
74,718Classification
MTEB MassiveScenarioClassification (sv) (sv; test; accuracy)
79,065Classification
MTEB MassiveScenarioClassification (sw) (sw; test; accuracy)
67,919Classification
MTEB MassiveScenarioClassification (ta) (ta; test; accuracy)
70,931Classification
MTEB MassiveScenarioClassification (te) (te; test; accuracy)
72,408Classification
MTEB MassiveScenarioClassification (th) (th; test; accuracy)
75,182Classification
MTEB MassiveScenarioClassification (tl) (tl; test; accuracy)
72,865Classification
MTEB MassiveScenarioClassification (tr) (tr; test; accuracy)
76,469Classification
MTEB MassiveScenarioClassification (ur) (ur; test; accuracy)
71,893Classification
MTEB MassiveScenarioClassification (vi) (vi; test; accuracy)
74,751Classification
MTEB MassiveScenarioClassification (zh-CN) (zh-CN; test; accuracy)
78,695Classification
MTEB MassiveScenarioClassification (zh-TW) (zh-TW; test; accuracy)
75,457Classification
MTEB MedrxivClusteringP2P (test; v_measure)
36,927Clustering
MTEB MedrxivClusteringS2S (test; v_measure)
35,535Clustering
MTEB MindSmallReranking (test; map)
33,085Reranking
MTEB NFCorpus (test; ndcg_at_10)
35,534Retrieval
MTEB NQ (test; ndcg_at_10)
57,754Retrieval
MTEB QuoraRetrieval (test; ndcg_at_10)
89,149Retrieval
MTEB RedditClustering (test; v_measure)
56,598Clustering
MTEB RedditClusteringP2P (test; v_measure)
64,274Clustering
MTEB SCIDOCS (test; ndcg_at_10)
18,723Retrieval
MTEB SICK-R (test; cos_sim_spearman)
81,732STS
MTEB STS12 (test; cos_sim_spearman)
82,571STS
MTEB STS13 (test; cos_sim_spearman)
87,150STS
MTEB STS14 (test; cos_sim_spearman)
84,971STS
MTEB STS15 (test; cos_sim_spearman)
91,051STS
MTEB STS16 (test; cos_sim_spearman)
87,314STS
MTEB STS17 (en-en) (en-en; test; cos_sim_spearman)
90,028STS
MTEB STS22 (en) (en; test; cos_sim_spearman)
67,630STS
MTEB STSBenchmark (test; cos_sim_spearman)
88,379STS
MTEB SciDocsRR (test; map)
85,872Reranking
MTEB SciFact (test; ndcg_at_10)
71,852Retrieval
MTEB SprintDuplicateQuestions (test; cos_sim_ap)
91,177PairClassification
MTEB SprintDuplicateQuestions (test; max_ap)
91,224PairClassification
MTEB StackExchangeClustering (test; v_measure)
66,851Clustering
MTEB StackExchangeClusteringP2P (test; v_measure)
42,461Clustering
MTEB StackOverflowDupQuestions (test; map)
51,445Reranking
MTEB SummEval (test; cos_sim_spearman)
30,385Summarization
MTEB TRECCOVID (test; ndcg_at_10)
82Retrieval
MTEB Tatoeba (sqi-eng) (sqi-eng; test; f1)
96,75BitextMining
MTEB Tatoeba (fry-eng) (fry-eng; test; f1)
82,466BitextMining
MTEB Tatoeba (kur-eng) (kur-eng; test; f1)
77,333BitextMining
MTEB Tatoeba (tur-eng) (tur-eng; test; f1)
98,267BitextMining
MTEB Tatoeba (deu-eng) (deu-eng; test; f1)
99,333BitextMining
MTEB Tatoeba (nld-eng) (nld-eng; test; f1)
97,2BitextMining
MTEB Tatoeba (ron-eng) (ron-eng; test; f1)
97,183BitextMining
MTEB Tatoeba (ang-eng) (ang-eng; test; f1)
72,811BitextMining
MTEB Tatoeba (ido-eng) (ido-eng; test; f1)
91,917BitextMining
MTEB Tatoeba (jav-eng) (jav-eng; test; f1)
85,276BitextMining
MTEB Tatoeba (isl-eng) (isl-eng; test; f1)
95BitextMining
MTEB Tatoeba (slv-eng) (slv-eng; test; f1)
91,908BitextMining
MTEB Tatoeba (cym-eng) (cym-eng; test; f1)
89,971BitextMining
MTEB Tatoeba (kaz-eng) (kaz-eng; test; f1)
87,722BitextMining
MTEB Tatoeba (est-eng) (est-eng; test; f1)
88,03BitextMining
MTEB Tatoeba (heb-eng) (heb-eng; test; f1)
91,457BitextMining
MTEB Tatoeba (gla-eng) (gla-eng; test; f1)
78,326BitextMining
MTEB Tatoeba (mar-eng) (mar-eng; test; f1)
92,583BitextMining
MTEB Tatoeba (lat-eng) (lat-eng; test; f1)
74,505BitextMining
MTEB Tatoeba (bel-eng) (bel-eng; test; f1)
95,55BitextMining
MTEB Tatoeba (pms-eng) (pms-eng; test; f1)
77,985BitextMining
MTEB Tatoeba (gle-eng) (gle-eng; test; f1)
84,992BitextMining
MTEB Tatoeba (pes-eng) (pes-eng; test; f1)
94,567BitextMining
MTEB Tatoeba (nob-eng) (nob-eng; test; f1)
98,2BitextMining
MTEB Tatoeba (bul-eng) (bul-eng; test; f1)
94,383BitextMining
MTEB Tatoeba (cbk-eng) (cbk-eng; test; f1)
84,104BitextMining
MTEB Tatoeba (hun-eng) (hun-eng; test; f1)
94,333BitextMining
MTEB Tatoeba (uig-eng) (uig-eng; test; f1)
86,820BitextMining
MTEB Tatoeba (rus-eng) (rus-eng; test; f1)
93,567BitextMining
MTEB Tatoeba (spa-eng) (spa-eng; test; f1)
98,6BitextMining
MTEB Tatoeba (hye-eng) (hye-eng; test; f1)
93,518BitextMining
MTEB Tatoeba (tel-eng) (tel-eng; test; f1)
96,083BitextMining
MTEB Tatoeba (afr-eng) (afr-eng; test; f1)
95,400BitextMining
MTEB Tatoeba (mon-eng) (mon-eng; test; f1)
95,492BitextMining
MTEB Tatoeba (arz-eng) (arz-eng; test; f1)
81,859BitextMining
MTEB Tatoeba (hrv-eng) (hrv-eng; test; f1)
96,75BitextMining
MTEB Tatoeba (nov-eng) (nov-eng; test; f1)
82,996BitextMining
MTEB Tatoeba (gsw-eng) (gsw-eng; test; f1)
61,897BitextMining
MTEB Tatoeba (nds-eng) (nds-eng; test; f1)
86,32BitextMining
MTEB Tatoeba (ukr-eng) (ukr-eng; test; f1)
94,783BitextMining
MTEB Tatoeba (uzb-eng) (uzb-eng; test; f1)
80,545BitextMining
MTEB Tatoeba (lit-eng) (lit-eng; test; f1)
92,013BitextMining
MTEB Tatoeba (ina-eng) (ina-eng; test; f1)
96,267BitextMining
MTEB Tatoeba (lfn-eng) (lfn-eng; test; f1)
80,678BitextMining
MTEB Tatoeba (zsm-eng) (zsm-eng; test; f1)
96,483BitextMining
MTEB Tatoeba (ita-eng) (ita-eng; test; f1)
94,667BitextMining
MTEB Tatoeba (cmn-eng) (cmn-eng; test; f1)
96,367BitextMining
MTEB Tatoeba (lvs-eng) (lvs-eng; test; f1)
92,807BitextMining
MTEB Tatoeba (glg-eng) (glg-eng; test; f1)
96,223BitextMining
MTEB Tatoeba (ceb-eng) (ceb-eng; test; f1)
70,782BitextMining
MTEB Tatoeba (bre-eng) (bre-eng; test; f1)
32,938BitextMining
MTEB Tatoeba (ben-eng) (ben-eng; test; f1)
89,133BitextMining
MTEB Tatoeba (swg-eng) (swg-eng; test; f1)
77,679BitextMining
MTEB Tatoeba (arq-eng) (arq-eng; test; f1)
63,070BitextMining
MTEB Tatoeba (kab-eng) (kab-eng; test; f1)
57,848BitextMining
MTEB Tatoeba (fra-eng) (fra-eng; test; f1)
95,017BitextMining
MTEB Tatoeba (por-eng) (por-eng; test; f1)
94,907BitextMining
MTEB Tatoeba (tat-eng) (tat-eng; test; f1)
84,613BitextMining
MTEB Tatoeba (oci-eng) (oci-eng; test; f1)
71,907BitextMining
MTEB Tatoeba (pol-eng) (pol-eng; test; f1)
97,267BitextMining
MTEB Tatoeba (war-eng) (war-eng; test; f1)
74,396BitextMining
MTEB Tatoeba (aze-eng) (aze-eng; test; f1)
93,790BitextMining
MTEB Tatoeba (vie-eng) (vie-eng; test; f1)
97,1BitextMining
MTEB Tatoeba (nno-eng) (nno-eng; test; f1)
94,257BitextMining
MTEB Tatoeba (cha-eng) (cha-eng; test; f1)
51,462BitextMining
MTEB Tatoeba (mhr-eng) (mhr-eng; test; f1)
16,918BitextMining
MTEB Tatoeba (dan-eng) (dan-eng; test; f1)
95,3BitextMining
MTEB Tatoeba (ell-eng) (ell-eng; test; f1)
95,117BitextMining
MTEB Tatoeba (amh-eng) (amh-eng; test; f1)
87,143BitextMining
MTEB Tatoeba (pam-eng) (pam-eng; test; f1)
19,631BitextMining
MTEB Tatoeba (hsb-eng) (hsb-eng; test; f1)
79,452BitextMining
MTEB Tatoeba (srp-eng) (srp-eng; test; f1)
94,533BitextMining
MTEB Tatoeba (epo-eng) (epo-eng; test; f1)
98,483BitextMining
MTEB Tatoeba (kzj-eng) (kzj-eng; test; f1)
14,979BitextMining
MTEB Tatoeba (awa-eng) (awa-eng; test; f1)
91,991BitextMining
MTEB Tatoeba (fao-eng) (fao-eng; test; f1)
86,768BitextMining
MTEB Tatoeba (mal-eng) (mal-eng; test; f1)
98,933BitextMining
MTEB Tatoeba (ile-eng) (ile-eng; test; f1)
89,780BitextMining
MTEB Tatoeba (bos-eng) (bos-eng; test; f1)
95,857BitextMining
MTEB Tatoeba (cor-eng) (cor-eng; test; f1)
11,821BitextMining
MTEB Tatoeba (cat-eng) (cat-eng; test; f1)
94,933BitextMining
MTEB Tatoeba (eus-eng) (eus-eng; test; f1)
84,723BitextMining
MTEB Tatoeba (yue-eng) (yue-eng; test; f1)
93,473BitextMining
MTEB Tatoeba (swe-eng) (swe-eng; test; f1)
95,717BitextMining
MTEB Tatoeba (dtp-eng) (dtp-eng; test; f1)
14,511BitextMining
MTEB Tatoeba (kat-eng) (kat-eng; test; f1)
92,466BitextMining
MTEB Tatoeba (jpn-eng) (jpn-eng; test; f1)
96,117BitextMining
MTEB Tatoeba (csb-eng) (csb-eng; test; f1)
66,589BitextMining
MTEB Tatoeba (xho-eng) (xho-eng; test; f1)
88,263BitextMining
MTEB Tatoeba (orv-eng) (orv-eng; test; f1)
58,288BitextMining
MTEB Tatoeba (ind-eng) (ind-eng; test; f1)
94,417BitextMining
MTEB Tatoeba (tuk-eng) (tuk-eng; test; f1)
49,640BitextMining
MTEB Tatoeba (max-eng) (max-eng; test; f1)
73,442BitextMining
MTEB Tatoeba (swh-eng) (swh-eng; test; f1)
80,915BitextMining
MTEB Tatoeba (hin-eng) (hin-eng; test; f1)
97,6BitextMining
MTEB Tatoeba (dsb-eng) (dsb-eng; test; f1)
72,004BitextMining
MTEB Tatoeba (ber-eng) (ber-eng; test; f1)
55,577BitextMining
MTEB Tatoeba (tam-eng) (tam-eng; test; f1)
90,647BitextMining
MTEB Tatoeba (slk-eng) (slk-eng; test; f1)
95,133BitextMining
MTEB Tatoeba (tgl-eng) (tgl-eng; test; f1)
95,850BitextMining
MTEB Tatoeba (ast-eng) (ast-eng; test; f1)
89,764BitextMining
MTEB Tatoeba (mkd-eng) (mkd-eng; test; f1)
92,49BitextMining
MTEB Tatoeba (khm-eng) (khm-eng; test; f1)
73,274BitextMining
MTEB Tatoeba (ces-eng) (ces-eng; test; f1)
96,567BitextMining
MTEB Tatoeba (tzl-eng) (tzl-eng; test; f1)
61,092BitextMining
MTEB Tatoeba (urd-eng) (urd-eng; test; f1)
93,412BitextMining
MTEB Tatoeba (ara-eng) (ara-eng; test; f1)
91,100BitextMining
MTEB Tatoeba (kor-eng) (kor-eng; test; f1)
91,973BitextMining
MTEB Tatoeba (yid-eng) (yid-eng; test; f1)
90,380BitextMining
MTEB Tatoeba (fin-eng) (fin-eng; test; f1)
96,917BitextMining
MTEB Tatoeba (tha-eng) (tha-eng; test; f1)
96,715BitextMining
MTEB Tatoeba (wuu-eng) (wuu-eng; test; f1)
92,340BitextMining
MTEB Touche2020 (test; ndcg_at_10)
27,249Retrieval
MTEB ToxicConversationsClassification (test; accuracy)
71,059Classification
MTEB TweetSentimentExtractionClassification (test; accuracy)
64,618Classification
MTEB TwentyNewsgroupsClustering (test; v_measure)
51,329Clustering
MTEB TwitterSemEval2015 (test; cos_sim_ap)
80,267PairClassification
MTEB TwitterSemEval2015 (test; max_ap)
80,267PairClassification
MTEB TwitterURLCorpus (test; cos_sim_ap)
87,117PairClassification
MTEB TwitterURLCorpus (test; max_ap)
87,117PairClassification

Valores como impressos na ficha, com a configuração do próprio publicador. Não comparáveis entre publicadores e fora de qualquer ranking. Ficha do modelo

Limites e datas

O benchmark de qualidade publicado não testa uma versão local quantizada. Estimativas de memória não medem velocidade nem garantem compatibilidade. Tamanho de repositório, quando disponível, não é VRAM. Fórmulas

Página atualizada em · Catálogo revisado em · MTEB (Multilingual, v2)