Skip to the card

Card 404 of 9902026-09-17 issue

Observed arrival · 2026-09-17

Invigil, the benchmark a lab cannot train against

invigil.org Observed source
Editorial interest 84/100 Selection signal · not a rating of the site

Invigil proposes machine-checked Lean 4 benchmarks for measuring whether frontier AI models can reason without simply recalling published test data.

Landing page captured for the 2026-09-17 issue.

Field notes

The proposed evaluation separates a reproducible public method from a private problem set that is never published and is refreshed quarterly from later research literature. Models receive theorem statements and return proof bodies; Invigil’s harness supplies the statement and Lean 4 checks correctness and axiom dependencies. Runs are described as pre-registered and hash-committed, with certification by Antefacts, a commonly owned entity whose relationship the site says is disclosed on its certification page.

Observed signals

Read the marks

Editorial observations of this landing page, not a rating.

OpenPublic substance visible
$PaidCommerce or pricing visible
PrettyNotable craft visible
ProPolished or operationally mature
NicheUnusually specific use

One card from the complete issue

Restaurant City, Back on Speaking Terms

345,627 arrived 1,000 judged 990 catalogued Enter the complete issue
invigil.org

Landing page observed 2026-09-17. The live site may have changed.