Decision Models · demos
When the decision is hard, s1-pro gets it right more often.
We ran s1-pro and Jev 1.14.0 on the same public JevBench items, with the same request format on the same server. Below: the published scores, the item-by-item comparison (including the 12 items where Jev wins), and a way to ask s1-pro yourself.
s1-pro vs Jev 1.14.0. Published JevBench v1.6.27, 17 Oct 2026, 1,500 items (1,200 sealed).
267 vs 258 of 297 items right. s1-fast: 80.5 %. Our run, 17 Oct 2026.
How good the probabilities are, not only the top pick.
Out of 297 items, the models disagree on correctness for only 33. See the 12 where Jev is right.
Where the gap is
Chance-corrected competence per category (0 = chance, 100 = perfect), open and sealed items pooled. Compare the two models within a category, not categories with each other. Categories where Jev leads are shown too.
Why a few points matter in a workflow
A workflow often chains decisions: route, classify, approve, escalate. One wrong step spoils the result. How likely is it that all of them are right?
Illustration: assumes independent decisions at today's public-set accuracy (0.899 vs 0.869), so the chance that all k are right is accuracyk. Real workflows are not independent.
Item explorer
Recorded answers are from 17 Oct 2026. "Ask s1-pro live" sends the item again; live answers can differ slightly from the recorded run.