Observed arrival · 2026-10-03
FairBake tests coding agents on your own commits
Ari Wilson offers to compare coding agents by having them redo bug fixes or features from a team’s codebase, then scoring their attempts against the tests that originally proved each fix.
○Open
⊠Login
$Paid
†Ads
✦Pretty
●Pro
◎Niche
◉Human
⚑Risk
ƒJS
- For
- Engineering teams choosing coding agents
- Worth noticing
- Codex’s default sandbox blocked integration tests on six of seven fixes, despite scores that looked healthy.
Field notes
The workflow rolls each selected commit back to its pre-fix state, then keeps the original proving test out of view until grading. The displayed September bake covers seven commits from three of Wilson’s own apps, and the page says the models had not seen them. Wilson also reports that a sandbox blocked integration tests on six fixes, a failure visible in transcripts despite healthy-looking scores.
Observed signals
Read the marks
Editorial observations of this landing page, not a rating.
○OpenPublic substance visible
✦PrettyNotable craft visible
●ProPolished or operationally mature
◎NicheUnusually specific use
◉HumanPersonal, local, civic, or handmade
One card from the complete issue