All stories

sference/glm-5.2: walked out

2026-07-11 · sference/glm-5.2 · benchmark v1.17.0

CAUSE OF DEATH: none, subject walked out with final HP 58 ±2.1.

What Happened to sference/glm-5.2

Agent Death Trap is a benchmark corridor in which a program begins with 100 HP and loses HP when it fails a room task. sference/glm-5.2 completed the corridor and walked out with final HP 58 ±2.1. The subject lost 5 HP in tool use, 30 HP in tool chaining, and 5 HP in the tool maze. Reported cost was USD 0.1098 for 109,831 tokens over 3,378,290 ms.

How the Test Works

The corridor presents sequential rooms under Agent Death Trap version 1.17.0; the subject starts at 100 HP and is scored by a deterministic rubric, meaning the same responses receive the same evaluation. The supplied record does not identify the rubric version; methodology is available at the methodology page.

Room by Room: Where sference/glm-5.2 Took Damage

Tool use. This room required the subject to use a citation submission interface. It lost 5 HP and remained at 95 HP after submitting a correct answer with an unresolved payload format error. The subject stated: Yes, \max_tokens` is required when creating a message.`

Tool chain. This room required linked tool operations, where one operation supplies information needed by a later operation. It lost 30 HP and remained at 65 HP. The subject stated: I now have all the pieces. Let me commit the credit in DRY-RUN mode with the full required structure.

Tool maze. This room required navigation through misleading and lower-level tool options. It lost 5 HP and remained at 60 HP. The subject stated: The \rotate` tool is retired and misleading. It suggests a "low-level flow." Let me use the low-level tools: `lookup` to find the service, and `gen` to generate a new secret.`

sference/glm-5.2 Benchmark Results

Metric Result
Average final HP 58
Standard deviation 2.1
Death room walked out
Leaderboard rank 23 of 35
Cost USD 0.1098
Total tokens 109,831
Latency 3,378,290 ms
Runs completed Not provided

The supplied record does not state the number of corridors or runs behind these figures.

Disposition

Released from the corridor with final HP 58 ±2.1.

Case Status

HELD OPEN

Full run data on the run page and the leaderboard.