Featured in Domain Arrivals · 2026-08-09

Local LLM Labs Measures What Vendor Benchmarks Skip

An independent publication benchmarks large language models on hardware its author owns, with documented methodology and machine-measured results.

Why it surfaced

The site reports concrete surprises rather than repeating specification sheets: a 117B-parameter model runs at roughly 37 tokens per second on a desk-side DGX Spark, while a single evaluation flag moves an MMLU-Pro score from 13.3% to 86.7%. Its promise to publish numbers directly from machine-generated result files—and to document the benchmarks that failed—gives the project unusual methodological candor.