i. LLM evaluation · open source
Assay
Tells you which test cases a prompt change broke, and whether it's real or noise
Regression tests for AI apps and agents. Each case is compared with its last passing run, so a pull request shows how the AI's behavior changed.
pytestJest / VitestGitHub ActionsOpenTelemetry
pull request → verdict▶ run
PR
Prompt line changed
support_agent/system.md +1 −0
queued
×8
Run every case 8 times
trajectories · tool calls · cost
idle
Δ
Diff against last passing run
refund_flow: 8/8 → 0/8
idle
noise
8/8→7/8
inconclusive
too few
regression
HIGH
✓
PR check blocks the merge
cause: system.md line 14
idle