Observed arrival · 2026-08-26
CodeReef, the bug-reproduction Action
A GitHub Action that runs labeled bug reports in CI, captures the command and output, and returns a verdict on whether the reported failure was reproduced.
Field notes
The workflow begins with a tracker label rather than a codebase installation, then records a command, Docker-sandbox output, and an explicit outcome state in a comment. The public benchmark covers 19 reports from nine open-source repositories, with 9 right failures reproduced and 10 runs capturing a different real failure. A displayed camelcase case is pinned to commit 24d711f and identifies Node and Ava as its test environment.
Observed signals
Read the marks
Editorial observations of this landing page, not a rating.
One card from the complete issue