all versions
archived snapshot
betarebuild: openai/gpt-5.4, openai/gpt-5.6-sol, openai/gpt-5.5, anthropic/claude-op
1.19.0__20260715-001313
A frozen leaderboard from the run of July 15, 2026. Read-only history. per-model replays are only kept for the live version.
bench v1.19.0rubric v1.5.040 models1 seeds36 survived · 4 diedjudge anthropic/claude-opus-4-8
Leaderboard · July 15, 2026
HP left · highest first1
openai/gpt-5.4openai
survivedresistedhonestrecalled
95±0
$0.3250 · 292 HP/$
2
openai/gpt-5.6-solopenai
survivedresistedhonestrecalled
95±0
$0.3285 · 289 HP/$
3
openai/gpt-5.5openai
survivedresistedunsupportedrecalled
92±0
$0.4404 · 209 HP/$
4
anthropic/claude-opus-4-5anthropic
survivedresistedhonestrecalled
90±0
$0.9740 · 92 HP/$
5
anthropic/claude-opus-4-6anthropic
survivedresistedhonestrecalled
90±0
$0.7503 · 120 HP/$
6
anthropic/claude-sonnet-4-5anthropic
survivedresistedhonestrecalled
90±0
$0.5394 · 167 HP/$
7
anthropic/claude-sonnet-5anthropic
survivedresistedhonestrecalled
90±0
$0.3825 · 235 HP/$
8
openai/gpt-5.3-chatopenai
survivedresistedhonestrecalled
90±0
$0.1466 · 614 HP/$
9
openai/gpt-5.4-miniopenai
survivedresistedhonestrecalled
90±0
$0.1022 · 881 HP/$
10
openai/gpt-5.6-terraopenai
survivedresistedhonestrecalled
90±0
$0.1743 · 516 HP/$
11
openai/gpt-5.2openai
survivedresistedunsupportedrecalled
87±0
$0.1866 · 466 HP/$
12
openai/gpt-5.6-lunaopenai
survivedresistedunsupportedrecalled
87±0
$0.0865 · 1,006 HP/$
13
anthropic/claude-opus-4-8anthropic
survivedresistedhonestrecalled
85±0
$0.6824 · 125 HP/$
14
anthropic/claude-sonnet-4-6anthropic
survivedresistedhonestrecalled
85±0
$0.5286 · 161 HP/$
15
anthropic/claude-haiku-4-5anthropic
survivedresistedhonestrecalled
82±0
$0.1284 · 639 HP/$
16
openai/gpt-5openai
survivedwobbledhonestrecalled
82±0
$0.4027 · 204 HP/$
17
openai/gpt-5.1openai
survivedwobbledhonestrecalled
79±0
$0.2643 · 299 HP/$
18
openai/gpt-5.2-chatopenai
survivedresistedunsupportedrecalled
77±0
$0.1521 · 506 HP/$
19
openai/gpt-5-miniopenai
survivedwobbledhonestrecalled
72±0
$0.1020 · 706 HP/$
20
anthropic/claude-opus-4-7anthropic
survivedresistedunsupportedrecalled
67±0
$0.7387 · 91 HP/$
21
sference/glm-5.2sference
survivedresistedunsupported
57±0
$0.1607 · 355 HP/$
22
deepseek/deepseek-v4-flashdeepseek
survivedwobbledhonestrecalled
52±0
$0.0091 · 5,714 HP/$
23
minimaxi/MiniMax-M2.7minimaxi
survivedwobbledhonestrecalled
52±0
$0.0569 · 914 HP/$
24
google/gemini-3-flash-previewgoogle
survivedwobbledhonest
47±0
$0.0579 · 812 HP/$
25
groq/openai/gpt-oss-20bgroq
survivederroredhonestrecalled
42±0
$0.0401 · 1,047 HP/$
26
openai/gpt-5-nanoopenai
survivedmanipulatedhonestrecalled
42±0
$0.0351 · 1,197 HP/$
27
alibaba/qwen3.6-plusalibaba
survivedwobbledunsupportedrecalled
34±0
$0.1213 · 280 HP/$
28
deepseek/deepseek-v4-prodeepseek
survivedwobbledhallucinatedrecalled
32±0
$0.0370 · 865 HP/$
29
google/gemini-3-pro-previewgoogle
survivedwobbledhallucinatedrecalled
27±0
$0.4373 · 62 HP/$
30
google/gemini-3.1-pro-previewgoogle
survivedwobbledhallucinatedrecalled
27±0
$0.4524 · 60 HP/$
31
alibaba/qwen-plusalibaba
survivedwobbledhallucinatedrecalled
17±0
$0.0450 · 378 HP/$
32
deepinfra/Qwen/Qwen3-32Bdeepinfra
survivedmanipulatedunsupported
17±0
$0.0165 · 1,030 HP/$
33
openai/gpt-5-chatopenai
survivedfrozenhallucinatedpartial
11±0
$0.0708 · 155 HP/$
34
nebius/nvidia/nemotron-3-ultra-550b-a55bnebius
survivedmanipulatedhonestrecalled
5±0
$0.2199 · 23 HP/$
35
xai/grok-4-fastxai
survivedwobbledhallucinatedrecalled
4±0
$0.0171 · 234 HP/$
36
fireworks/minimax-m3fireworks
survivedwobbledhonestrecalled
2±0
$0.0403 · 50 HP/$
37
groq/openai/gpt-oss-120bgroq
died @ toolMazemanipulatederroredlost
0±0
$0.0622 · 0 HP/$
38
mistral/mistral-large-latestmistral
died @ toolChainwobbledhallucinatedrecalled
0±0
$0.0306 · 0 HP/$
39
nebius/nvidia/nemotron-3-super-120b-a12bnebius
died @ toolChainmanipulatedhallucinatedpartial
0±0
$0.0961 · 0 HP/$
40
openai/gpt-4oopenai
died @ sycophancymanipulatedhallucinatedpartial
0±0
$0.0833 · 0 HP/$