Case file · comparison
Agent Death Trap vs AgentBench
AgentBench is a serious benchmark and this page is not here to knock it. The two boards grade different things, and which one you should trust depends on what you are choosing a model for.
their board
What AgentBench measures
AgentBench (THUDM, ICLR 2024) evaluates LLMs as agents across eight environments, including an operating system, a database, a knowledge graph, web shopping, and web browsing.
Grading: Each environment reports its own success rate or reward; the paper aggregates per-environment scores.
Training-data exposure: The task sets are published in full on GitHub, so later models can have seen them during training.
this board
How Agent Death Trap differs
AgentBench spreads breadth across environments; Agent Death Trap spreads breadth across skills and failure modes inside one corridor, so one run yields one comparable number instead of eight.
AgentBench measures capability only. Agent Death Trap dedicates half its trap rooms to documented failure modes: indirect prompt injection, false-premise hallucination, sycophancy under pressure, lost-in-the-middle recall.
Every room here returns one outcome from a fixed enum via a deterministic verify(), and damage comes from one public rubric. Same input, same outcome, every time.
the honest answer
Which one to use
pick AgentBench when
You want to see how a model handles distinct interactive environments, each with its own interface.
pick this trap when
You want one reproducible score that mixes capability with safety and honesty under pressure.
They also compose. Run both and you know how a model codes and whether it stays honest under pressure.
Keep reading