Observed arrival · 2026-10-28

PROBE, the proactive bug-discovery benchmark

probebench.com Visit website

Editorial interest 84/100 Selection signal · not a rating of the site

An open benchmark compares coding agents’ whole-repository bug audits with 2,457 bugs maintainers fixed across 20 repositories.

Landing page captured for the 2026-10-28 issue.

For: Researchers and developers evaluating coding-agent bug audits
Worth noticing: The reference set counts bugs maintainers fixed during the 18 months after agent audits; unmatched reports may still be valid.

Field notes

The page groups 20 repositories across Python (10), TypeScript (6), and JavaScript (4), with reference-bug counts shown for projects including sqlglot (512), prefect (429), and Svelte (248). Its leaderboard reports F1, recall, precision, finding totals, and per-run API cost; users can sort columns, filter by language, inspect rows, and download a CSV. It also notes that unmatched findings may still be valid.

Observed signals

Read the marks

Editorial observations of this landing page, not a rating.

○OpenPublic substance visible

✦PrettyNotable craft visible

●ProPolished or operationally mature

◎NicheUnusually specific use

One card from the complete issue

Already Warned

254,543 arrived 1,000 judged 981 catalogued Enter the complete issue