Observed arrival · 2026-10-05
Windway Data’s replayable agent-evaluation fixtures
Windway Data presents tools and downloadable datasets for inspecting agent navigation, failure recovery, and deterministic replay.
- For
- Agent-systems evaluators and dataset builders
- Worth noticing
- The page reports zero public agent runs and says its 1,800 local trials tested control policies, not LLMs or customer agents.
Field notes
The page distinguishes a deterministic grid teaching fixture from its production corpus: the example blocks an attempted move, then checks whether a detour reaches the task predicate under the environment rules. Public downloads include two MIT-licensed 100-item samples—navigation fixtures and two-agent joint-planning fixtures—with schemas, verification information, and SHA-256 checksums; the canonical failure/recovery corpus is not public.
Observed signals
Read the marks
Editorial observations of this landing page, not a rating.
One card from the complete issue