Model matrix /

embeddinggemma-300m

Checkpoint embeddinggemma-300m/hf-57c266a. Reference: fp32-google/sentence-transformers-torch-cuda-st6.1.0-torch2.14.1-cu126. Scores are the mean MTEB main score ×100 over the 15 quick-tier tasks; Δ is paired with the reference, task by task.

Quality at 768 dimensions

Variant \ HardwareNVIDIA P100 · 16 GBApple M4 Pro · 24 GB
fp32-googlesentence-transformers68.81reference15,211 prefill tok/ssentence-transformers-torch-cuda st6.1.0-torch2.14.1+cu12668.81+0.00 [+0.00, +0.00]losslesscos 1.00007,488 prefill tok/ssentence-transformers-torch-mps st6.1.0-torch2.14.1
bf16-googlesentence-transformersnot run68.80−0.01 [−0.11, +0.07]losslesscos 1.00009,266 prefill tok/ssentence-transformers-torch-mps st6.1.0-torch2.14.1

Fidelity compares each run's vectors with the reference's on the same 66,939 texts: lossless (cosine ≥ 0.999 and top-10 neighbor overlap ≥ 0.98), faithful, degraded (cosine < 0.98 or overlap < 0.80), broken.

Output dimension (Matryoshka truncation)

each line is one run: its own vectors truncated, then re-normalized
6364656667686970128256512768output dimension (vector length kept after truncation, log scale)mean MTEB score ×100 (15 tasks)fp32-google · sentence-transformers · p100:p100: 63.53 at 128fp32-google · sentence-transformers · p100:p100: 67.34 at 256fp32-google · sentence-transformers · p100:p100: 68.25 at 512fp32-google · sentence-transformers · p100:p100: 68.81 at 768fp32-google · sentence-transformers · apple-m:m4-pro: 63.53 at 128fp32-google · sentence-transformers · apple-m:m4-pro: 67.34 at 256fp32-google · sentence-transformers · apple-m:m4-pro: 68.25 at 512fp32-google · sentence-transformers · apple-m:m4-pro: 68.81 at 768bf16-google · sentence-transformers · apple-m:m4-pro: 63.50 at 128bf16-google · sentence-transformers · apple-m:m4-pro: 67.29 at 256bf16-google · sentence-transformers · apple-m:m4-pro: 68.17 at 512bf16-google · sentence-transformers · apple-m:m4-pro: 68.80 at 768
fp32-google · sentence-transformers · p100:p100fp32-google · sentence-transformers · apple-m:m4-probf16-google · sentence-transformers · apple-m:m4-pro

Per task at 768 dimensions

MTEB main score ×100; Δ vs reference below
Taskfp32-google · sentence-transformers · p100:p100fp32-google · sentence-transformers · apple-m:m4-probf16-google · sentence-transformers · apple-m:m4-pro
NanoArguAnaRetrieval61.4661.46
+0.00		61.52
+0.06
NanoClimateFeverRetrieval33.4133.41
+0.00		33.40
−0.01
NanoDBPediaRetrieval66.6366.63
+0.00		66.84
+0.22
NanoFEVERRetrieval93.5993.59
+0.00		93.59
+0.00
NanoFiQA2018Retrieval57.0457.04
+0.00		57.26
+0.22
NanoHotpotQARetrieval83.2283.22
+0.00		83.40
+0.18
NanoMSMARCORetrieval63.3563.35
+0.00		62.81
−0.54
NanoNFCorpusRetrieval39.2639.26
+0.00		39.05
−0.21
NanoNQRetrieval77.4477.44
+0.00		77.41
−0.03
NanoQuoraRetrieval91.7291.72
+0.00		91.78
+0.06
NanoSCIDOCSRetrieval40.2840.28
+0.00		40.28
−0.00
NanoSciFactRetrieval86.1386.13
+0.00		86.03
−0.10
NanoTouche2020Retrieval59.1459.14
+0.00		59.12
−0.03
STSBenchmark88.1888.18
+0.00		88.18
+0.01
Banking77Classification91.3191.31
+0.00		91.30
−0.01
Mean68.81
reference		68.81
+0.00 [+0.00, +0.00]		68.80
−0.01 [−0.11, +0.07]

Speed

per hardware class, fidelity-gated

Apple M4 Pro · 24 GB

SubjectPrefill tokens/s (engine)Prefill tokens/s (wall)Texts/sBatch p50Batch p95Fidelity
bf16-google·sentence-transformers-torch-mps-st6.1.0-torch2.14.19,2669,22153.4337 ms1.76 slossless
fp32-google·sentence-transformers-torch-mps-st6.1.0-torch2.14.17,4887,45843.2379 ms2.26 slossless

NVIDIA P100 · 16 GB

SubjectPrefill tokens/s (engine)Prefill tokens/s (wall)Texts/sBatch p50Batch p95Fidelity
fp32-google·sentence-transformers-torch-cuda-st6.1.0-torch2.14.1-cu12615,21112,08670.0299 ms1.18 sreference

Embedding is prefill only: input tokens through the model, no decode. Engine time is the server's own forward passes; wall time adds HTTP and base64 on the client. Batches of 32 texts, longest first, one request at a time. A degraded subject is listed but not ranked for speed: being fast is not a win if the vectors moved.

Subjects