agent-regression-evals
This plugin provides three deterministic scripts to detect regressions in agent behavior: a trajectory checker grades tool-call paths with multiple match modes and forbidden-tool enforcement; a flake detector classifies cases as stable or flaky and reports pass@k and pass^k with statistical confidence intervals; and a regression gate performs paired statistical testing (t-based CI, McNemar test) to block merges on significant drops, critical-case regressions, dropped cases, or invalid quarantine entries.
Install
Source: https://github.com/alirezarezvani/claude-skills/tree/HEAD/engineering/agent-regression-evals
What it's made of
2 commands · 1 agent · 1 skill
- Commands
- 2
- Agents
- 1
- Skills
- 1
- MCP servers
- 0
- Hooks
- 0
What it needs & plugs into
- API keys
- none
- Paid services
- none detected
- External tools
- none
- Talks to
- nothing external detected
Analyzed . Facts extracted from the plugin's files. Prose generated by claude-haiku-4-5-20251001.
Is this your plugin?
Claim it to keep the card accurate and enter the weekly contest. Requires signing in as the GitHub owner (alirezarezvani).