Leaderboards

Every subject, one table per kind of model: the full version of the home page's top 10. Sort by any column. Each table's Index compares only the subjects in it. Below each table are the per-suite boards, with intervals, every tier and speed.

System One decisions: typed Choice, Score and Noul answers with full probability distributions, from decision models or LLMs through the gateway. The Index compares only subjects in this table.

Sort by
#SubjectScenesClassicSealed
1clefgguf-q3_k_m-bartowski · llamacpp-cuda-b1151598.32/271.389.0———0.9NVIDIA P100 · 16 GB4490 ms—
2jev-latestopaque · typesafeprovider-opaque97.92/273.885.2———59.6provider-hosted154 ms$0.0076
3clef-flashmlx-q8-mlx-community · clef-mlx-mlx0.32.3-mlxvlm0.7.497.72/270.588.9———0.5Apple M4 Pro · 24 GB1619 ms—
4clef-flashbnb-nf4-iammrduncan · decide-server-clef-torch-cuda-tf5.19.0-torch2.14.1-cu12697.32/270.288.6———2.5NVIDIA P100 · 16 GB1645 ms—
5gev-26b-decidegguf-iq4_xs-mradermacher · llamacpp-cuda-b1151597.12/271.287.0———0.7NVIDIA P100 · 16 GB2464 ms—
6jev-27b-vlgguf-q3_k_m-iammrduncan · llamacpp-cuda-b11370-jev94.32/269.783.9———0.4NVIDIA P100 · 16 GB6506 ms—
7d1-3bbf16-liquidai · decide-server-d1-torch-mps-tf5.19.0-torch2.14.192.12/264.985.7———10.3Apple M4 Pro · 24 GB353 ms—
8d1-3bfp16-liquidai · decide-server-d1-torch-cuda-tf5.19.0-torch2.14.1-cu12692.02/264.885.7———10.3NVIDIA P100 · 16 GB341 ms—
9d1-3bfp32-liquidai · decide-server-d1-torch-cuda-tf5.19.0-torch2.14.1-cu12692.02/264.885.7———11.3NVIDIA P100 · 16 GB397 ms—
10d1-3bfp32-liquidai · decide-server-d1-torch-mps-tf5.19.0-torch2.14.192.02/264.885.7———7.9Apple M4 Pro · 24 GB439 ms—
11d1-omni-600mfp32-liquidai · decide-server-d1-torch-cuda-tf5.19.0-torch2.14.1-cu12685.32/257.482.7———28.5NVIDIA P100 · 16 GB116 ms—
12d1-omni-600mfp32-liquidai · decide-server-d1-torch-mps-tf5.19.0-torch2.14.185.32/257.482.7———20.7Apple M4 Pro · 24 GB98 ms—
13d1-omni-600mfp16-liquidai · decide-server-d1-torch-mps-tf5.19.0-torch2.14.185.32/257.482.7———24.5Apple M4 Pro · 24 GB145 ms—
14d1-omni-600mfp16-liquidai · decide-server-d1-torch-cuda-tf5.19.0-torch2.14.1-cu12685.32/257.482.7———29.7NVIDIA P100 · 16 GB73 ms—
15layaauto-amp-convai · decide-server-laya-torch-cuda-tf5.19.0-torch2.14.1-cu12662.92/236.068.5———22.7NVIDIA P100 · 16 GB103 ms—
16layafp32-convai · decide-server-laya-torch-cuda-tf5.19.0-torch2.14.1-cu12662.82/235.968.5———24.5NVIDIA P100 · 16 GB108 ms—
17layafp32-convai · decide-server-laya-torch-mps-tf5.19.0-torch2.14.162.82/235.968.5———19.9Apple M4 Pro · 24 GB112 ms—

Showing 17 of 17 subjects. Category scores are each suite's headline on a 0–100 scale and compare only within a column. — means not measured, never zero. Faded columns have no ranked suite yet. Latency is per request; * marks a batch.

Per-suite tables in Decision, with intervals, per-tier boards and speed:decision-conformance2typed-decisions2