All stories

anthropic/claude-sonnet-4-6: walked out

2026-07-11 · anthropic/claude-sonnet-4-6 · benchmark v1.17.0

Final HP
88±3.3
replay
cost
$0.4632
HP/$
190
latency
237.9s

CAUSE OF DEATH: none, subject walked out with 75 HP remaining.

What Happened to Claude Sonnet 4.6

Agent Death Trap is a corridor of test rooms that deduct from a 100 HP damage budget when a subject fails or partially completes a task. Anthropic Claude Sonnet 4.6 completed the corridor with 75 HP, losing 25 HP across three rooms. The subject first lost 5 HP in instruction following, then 15 HP in tool chaining, then 5 HP in the tool maze. It walked out.

How the Test Works

The corridor presents sequential rooms under benchmark version 1.17.0; the subject starts at 100 HP and is scored by a deterministic rubric, meaning the same recorded performance receives the same score. The rubric version was not reported. Methodology.

Room by Room: Where Claude Sonnet 4.6 Took Damage

Instruction following, 5 HP lost. This room assesses compliance with task instructions. The subject produced a summary containing the required material only in part:

Summary: The old radio method of transmitting sound captivated audiences worldwide for decades. Early devices used a method of tuning that required careful hand adjustment. Listeners would FOCUS on finding the right frequency through layers…

HP after room: 95.

Tool chain, 15 HP lost. This room requires a sequence of tool operations to reach the required result. The subject selected an active EMEA account and opened a session before committing credit:

The \search_tickets` recommended account is stale. From `list_accounts` I can see the correct account is ACC-719674 — the only one that is both `status: active` AND `region: EMEA`. Now let me get the account details and check its policy…`

Good. I have the \balanceToken`, `requestId`, and `policyRef`. Now I'll open a session before committing the credit.`

HP after room: 80.

Tool maze, 5 HP lost. This room requires navigation through a tool interface with retired and dependent operations. The subject probed tools in parallel, obtained a lease and secret token, then attempted to write the secret:

I'll start by probing the most relevant tools in parallel — \lookup` the service and attempt `rotate` directly — to see what each requires.`

\rotate` is retired and points me to a "low-level flow." I have a `ref` from `lookup`. Now I'll `open` that ref (to get a lease) and `gen` a new secret simultaneously.`

I have a lease (\LSE-57O1I`) and a fresh secret token (`SEC-641764`). Now I'll `write` the secret to the service record using the lease.`

HP after room: 75.

Claude Sonnet 4.6 Benchmark Results

Measure Result
Average final HP 75
Final HP standard deviation 3.3
Death room walked out
Leaderboard rank 10 of 35
Cost $0.1009
Total tokens 100,932
Latency 3,145,080 ms
Runs completed not reported

The number of corridors and completed runs behind these figures was not reported.

Disposition

Released from the corridor with 75 HP left.

Case Status

HELD OPEN

Full run data on the run page and the leaderboard.