Observed arrival · 2026-11-08
Chaos Bench
A benchmark for testing AI agents on real database and API tasks while faults are injected, then checking whether the agent’s report matches the resulting data.
○Open
⊠Login
$Paid
†Ads
✦Pretty
●Pro
◎Niche
◉Human
⚑Risk
ƒJS
- For
- AI-agent benchmark builders and reliability testers
- Worth noticing
- Tasks range from roughly five to 150 tool calls; scripted cheaters are included to check whether the grader catches false success.
Field notes
The described workflow pairs an agent’s claimed outcome with a hidden-answer check against the database state, rather than letting the agent grade itself. Tasks vary in length from roughly five to 150 tool calls, and scripted agents that do nothing or take blanket shortcuts are included to test whether the grader flags false success. The page labels results an early, small sample and says fault-injection runs have not yet begun.
Observed signals
Read the marks
Editorial observations of this landing page, not a rating.
○OpenPublic substance visible
✦PrettyNotable craft visible
●ProPolished or operationally mature
◎NicheUnusually specific use
One card from the complete issue