Observed arrival · 2026-08-22
Promptry, the “Sentry for prompts”
A local-first observability and testing tool for LLM pipelines that versions prompts, records model-call costs and latency, and runs regression suites in CI.
Why it surfaced
Promptry swaps into an OpenAI import and records each call in a local SQLite file, then lets developers diff prompt versions, trace multi-step runs, and detect statistical drift. It makes a clear case for keeping prompt evaluation close to the codebase: MIT-licensed, account-free, and without Docker or SaaS.
A local-first observability and testing tool for LLM pipelines that versions prompts, records model-call costs and latency, and runs regression suites in CI.
Observed signals
Read the marks
Editorial observations of this landing page, not a rating.
One card from the complete issue