embeddings / lfm2.5-embedding-350m/hf-f35ae2c/fp32-liquidai/sentence-transformers-torch-cuda-st5.1.1-tf4.56.2-torch2.14.1-cu126 /
2026-10-08-quick-card-prompts-efe68a
Reference run. Every other run of this checkpoint is measured against these vectors.
By output dimension
| Dims | Score (95% interval over tasks) | Δ vs reference | Fidelity |
|---|---|---|---|
| 1024 | reference | reference |
Per task
MTEB main score ×100
| Task | 1024 |
|---|---|
| NanoArguAnaRetrieval | 50.43 |
| NanoClimateFeverRetrieval | 45.07 |
| NanoDBPediaRetrieval | 64.70 |
| NanoFEVERRetrieval | 95.31 |
| NanoFiQA2018Retrieval | 55.72 |
| NanoHotpotQARetrieval | 85.23 |
| NanoMSMARCORetrieval | 64.29 |
| NanoNFCorpusRetrieval | 36.79 |
| NanoNQRetrieval | 72.38 |
| NanoQuoraRetrieval | 89.57 |
| NanoSCIDOCSRetrieval | 39.20 |
| NanoSciFactRetrieval | 74.99 |
| NanoTouche2020Retrieval | 54.14 |
| STSBenchmark | 80.12 |
| Banking77Classification | 82.89 |
Throughput
- Prefill tokens/s (engine)
- 6,602 (embedding has no decode)
- Prefill tokens/s (wall)
- 5,604 (includes HTTP and base64)
- Texts
- 66,939 texts · 9,883,483 tokens · 38.0 texts/s
- Engine
- backend=sentence-transformers · device=cuda · dtype=float32 · torch=2.14.1+cu126 · cuda_arch_list=sm_50,sm_60,sm_70,sm_75,sm_80,sm_86,sm_90 · device_name=Tesla P100-PCIE-16GB · sentence_transformers=5.1.1 · transformers=4.56.2 · max_seq_length=512 · dimension=1024 · trust_remote_code=true · unpadded=true · triton_overrides=disabled torch's native Triton op overrides (sm_60 is below Triton's sm_70 minimum) · token_budget_per_forward=16384 · python=3.12.15 · platform=Linux-7.0.0-34-generic-x86_64-with-glibc2.36 · machine=x86_64 · numpy=2.5.3
Run
- Suite
- embeddings · quick tier · profile card-prompts · mode native
- Items
- 2,092 succeeded of 2,092 attempted (66,939 planned)
- When
- 2026-10-08 20:12:55 UTC · 31 min
- Hardware
- NVIDIA P100 · 16 GB
- Harness
- bench @ e72ea88 · scorer mteb 2.22.5
- Suite hash
- c8280a4454c525bf91218ad69c4c09d220fcedd2cb41dfce6134c902d40b49cd
- Recipe
- lfm2.5-embedding-350m/p100/st · profile st-fp32 · inference-engines @ e6ceae9
- Latency
- p50 639 ms · p95 1.87 s · p99 2.09 s (per request)
- Usage
- 9,883,483 input tokens · 0 output tokens · $0.00
- dimensions below 768 are the same vectors truncated and re-normalized (Matryoshka); fidelity compares each dimension with the reference at the same dimension
Reproduce
npm run bench -- run --recipe lfm2.5-embedding-350m/p100/st \
--recipe-profile st-fp32 \
--on <host with a p100:p100> \
--suite embeddings --tier quick --profile card-promptsFiles
exactly as committed under results/