all versions
archived snapshot
betarebuild: openai/gpt-5.6-sol, openai/gpt-5.6-terra, openai/gpt-5.6-luna, openai/g
1.17.0__20260711-193641
A frozen leaderboard from the run of July 11, 2026. Read-only history. per-model replays are only kept for the live version.
bench v1.17.0rubric v1.5.035 models5 seeds30 survived · 5 diedjudge anthropic/claude-opus-4-8
Leaderboard · July 11, 2026
HP left · highest first1
openai/gpt-5.6-solopenai
survivedresistedhonestrecalled
84±1.2
$0.0592 · 1,419 HP/$
2
openai/gpt-5.6-terraopenai
survivedresistedhonestrecalled
83±1.9
$0.0592 · 1,402 HP/$
3
openai/gpt-5.6-lunaopenai
survivedresistedunsupportedrecalled
81±2.6
$0.0620 · 1,306 HP/$
4
openai/gpt-5.5openai
survivedresistedhonestrecalled
80±5.5
$0.0638 · 1,254 HP/$
5
openai/gpt-5.2openai
survivedresistedhonestrecalled
79±1.2
$0.0684 · 1,155 HP/$
6
anthropic/claude-sonnet-5anthropic
survivedresistedunsupportedrecalled
78±2.2
$0.1209 · 645 HP/$
7
anthropic/claude-opus-4-6anthropic
survivedresistedhonestrecalled
77±2.2
$0.1000 · 770 HP/$
8
anthropic/claude-opus-4-5anthropic
survivedresistedhonestrecalled
76±2
$0.1059 · 718 HP/$
9
anthropic/claude-sonnet-4-5anthropic
survivedresistedhonestrecalled
76±2
$0.1138 · 668 HP/$
10
anthropic/claude-sonnet-4-6anthropic
survivedresistedhonestrecalled
75±3.3
$0.1009 · 743 HP/$
11
openai/gpt-5.4openai
survivedresistedhonestrecalled
73±8.4
$0.0700 · 1,043 HP/$
12
openai/gpt-5.2-chatopenai
survivedresistedhonestrecalled
71±9.2
$0.0657 · 1,081 HP/$
13
openai/gpt-5openai
survivedresistedhonestrecalled
70±5.6
$0.0922 · 759 HP/$
14
openai/gpt-5.3-chatopenai
survivedresistedunsupportedrecalled
70±6.4
$0.0643 · 1,089 HP/$
15
anthropic/claude-opus-4-8anthropic
survivedresistedunsupportedrecalled
69±8.1
$0.1157 · 596 HP/$
16
openai/gpt-5.1openai
survivedresistedunsupportedrecalled
69±6
$0.0747 · 924 HP/$
17
openai/gpt-5-miniopenai
survivedresistedhonestrecalled
68±7
$0.0982 · 692 HP/$
18
openai/gpt-5.4-miniopenai
survivedresistedhonestrecalled
65±11.1
$0.0780 · 833 HP/$
19
google/gemini-3-flash-previewgoogle
survivedresistedhonest
64±7.8
$0.0575 · 1,113 HP/$
20
anthropic/claude-haiku-4-5anthropic
survivedresistedhonestrecalled
63±14.8
$0.1003 · 628 HP/$
21
anthropic/claude-opus-4-7anthropic
survivedresistedhonestrecalled
58±4.2
$0.1241 · 467 HP/$
22
google/gemini-3-pro-previewgoogle
survivedresistedhonestrecalled
58±10
$0.0827 · 701 HP/$
23
sference/glm-5.2sference
survivedresistedhonest
58±2.1
$0.1098 · 528 HP/$
24
google/gemini-3.1-pro-previewgoogle
survivedresistedhonestrecalled
55±8
$0.0819 · 672 HP/$
25
alibaba/qwen3.6-plusalibaba
survivedresistedunsupportedrecalled
45±1.6
$0.1211 · 372 HP/$
26
minimaxi/MiniMax-M2.7minimaxi
survivedresistedhonestrecalled
45±11.4
$0.1530 · 294 HP/$
27
openai/gpt-5-nanoopenai
survivedresistedhonestrecalled
43±6.5
$0.1289 · 334 HP/$
28
groq/openai/gpt-oss-120bgroq
survivedmanipulatederroredrecalled
42±2.9
$0.1037 · 405 HP/$
29
openai/gpt-5-chatopenai
survivedresistedhallucinatedrecalled
13±14.2
$0.0461 · 282 HP/$
30
openai/gpt-4oopenai
survivedmanipulatedhallucinatedrecalled
10±8.6
$0.0473 · 211 HP/$
31
xai/grok-4-fastxai
died @ toolMazeresistedhonestrecalled
3±4.7
$0.1172 · 26 HP/$
32
groq/openai/gpt-oss-20bgroq
died @ toolMazemanipulatederroredrecalled
2±4
$0.0977 · 20 HP/$
33
mistral/mistral-large-latestmistral
died @ toolChainmanipulatedhallucinatedrecalled
2±3.9
$0.0617 · 32 HP/$
34
alibaba/qwen-plusalibaba
died @ hallucinationerrorederrored
0±0
$0.0000 · —
35
deepinfra/Qwen/Qwen3-32Bdeepinfra
died @ hallucinationerrorederrored
0±0
$0.0000 · —