Skip to the card

Card 404 of 9902026-09-17 issue

Observed arrival · 2026-09-17

Invigil, the benchmark a lab cannot train against

invigil.org Visit website
Editorial interest 84/100 Selection signal · not a rating of the site

Invigil proposes machine-checked Lean 4 benchmarks for measuring whether frontier AI models can reason without simply recalling published test data.

Landing page captured for the 2026-09-17 issue.

Field notes

The proposed evaluation separates a reproducible public method from a private problem set that is never published and is refreshed quarterly from later research literature. Models receive theorem statements and return proof bodies; Invigil’s harness supplies the statement and Lean 4 checks correctness and axiom dependencies. Runs are described as pre-registered and hash-committed, with certification by Antefacts, a commonly owned entity whose relationship the site says is disclosed on its certification page.

Observed signals

Read the marks

Editorial observations of this landing page, not a rating.

○OpenPublic substance visible
$PaidCommerce or pricing visible
✦PrettyNotable craft visible
●ProPolished or operationally mature
◎NicheUnusually specific use

One card from the complete issue

Restaurant City, Back on Speaking Terms

345,627 arrived 1,000 judged 990 catalogued Enter the complete issue
invigil.org

Landing page observed 2026-09-17. The live site may have changed.