A score is never just the model's.
It belongs to the model, the exact version you ran, and the engine that served it, on stated hardware. We vary those three and hold the harness constant, so when two numbers differ you can see which one moved.
Measured by one harness, held constant: bench sends every request, records every byte and scores offline. How it works →
- Runs
- 51
- Subjects
- 33
- Suites
- 3
- Machines
- 3
- Latest run
- 2026-10-09
Top subjects
One row per subject (a model at one version on one engine), one column per category, one table per kind of model. Sort by any column; the Index combines the categories a subject ran.
System One decisions: typed Choice, Score and Noul answers with full probability distributions, from decision models or LLMs through the gateway. The Index compares only subjects in this table.
Showing 10 of 17 subjects. Category scores are each suite's headline on a 0–100 scale and compare only within a column. — means not measured, never zero. Faded columns have no ranked suite yet. Latency is per request; * marks a batch.
Latest matrix: gev-26b-decide
One checkpoint: each row is a variant (a quant served by an engine), each column a machine. Each cell shows the quality change against the reference, how far its vectors moved, and speed.
| Variant \ Hardware | NVIDIA P100 · 16 GB |
|---|---|
| gguf-iq4_xs-mradermacherllamacpp | 71.2%no referenceBrier 0.130 · CI 69.1%–73.1%0.7 decisions/sllamacpp-cuda b11515 |