d1-3b
Checkpoint d1-3b/hf-051bcc4. Reference: fp32-liquidai/decide-server-d1-torch-cuda-tf5.19.0-torch2.14.1-cu126. Scored on typed-decisions, core tier: 400 cases × 5 questions = 2,000 decisions per run. Δ is accuracy minus the reference's, in points.
Accuracy by variant and machine
| Variant \ Hardware | NVIDIA P100 · 16 GB | Apple M4 Pro · 24 GB |
|---|---|---|
| fp32-liquidaidecide-server-d1 | 64.8%referenceBrier 0.143 · CI 62.6%–66.8%11.3 decisions/sdecide-server-d1-torch-cuda tf5.19.0-torch2.14.1+cu126 | 64.8%+0.0 ptsBrier 0.143 · CI 62.6%–66.8%7.9 decisions/sdecide-server-d1-torch-mps tf5.19.0-torch2.14.1 |
| bf16-liquidaidecide-server-d1 | not run | 64.9%+0.2 ptsBrier 0.143 · CI 62.8%–67.0%10.3 decisions/sdecide-server-d1-torch-mps tf5.19.0-torch2.14.1 |
| fp16-liquidaidecide-server-d1 | 64.8%+0.0 ptsBrier 0.143 · CI 62.6%–66.8%10.3 decisions/sdecide-server-d1-torch-cuda tf5.19.0-torch2.14.1+cu126 | not run |
Accuracy counts every dispatched decision. The interval (CI) is Wilson 95%; differences smaller than the intervals' overlap are not resolved by 2,000 decisions.
All runs
| Variant · machine | Accuracy | KL ↓ | Brier ↓ | ECE ↓ | choice | noul | score | Decisions/s | Input tok/s* | p50 / case | Conformance |
|---|---|---|---|---|---|---|---|---|---|---|---|
| fp32-liquidai · NVIDIA P100 · 16 GBreference | 0.28 | 0.143 | 0.041 | 60.5% | 79.8% | 56.6% | 11.3 | 1,307 | 895 ms | 49/49 | |
| bf16-liquidai · Apple M4 Pro · 24 GB | 0.28 | 0.143 | 0.036 | 60.3% | 80.2% | 56.9% | 10.3 | 1,196 | 987 ms | 49/49 | |
| fp16-liquidai · NVIDIA P100 · 16 GB | 0.28 | 0.143 | 0.042 | 60.5% | 79.8% | 56.6% | 10.3 | 1,191 | 996 ms | 49/49 | |
| fp32-liquidai · Apple M4 Pro · 24 GB | 0.28 | 0.143 | 0.041 | 60.5% | 79.8% | 56.6% | 7.9 | 919 | 1.21 s | 49/49 |
Decisions/s is over the run's wall-clock time, client to server, so a remote machine includes the network. Engine-only figures are on each run's page. *Input tok/s counts tokens with each model's own tokenizer and prompt layout (d1 counts the state once per question), so it compares one model across machines, not models with each other. p50 is per case (all five questions); runs kept two requests in flight against servers that answer one at a time, so it includes waiting for the other.