claude

agent-regression-evals

Prove an agent, prompt, model or tool change did not make the agent worse. Three deterministic stdlib scripts: a trajectory checker that grades tool-call paths (strict / unordered / subset / superset match modes, argument matching, forbidden tools that block in every mode, call ceilings); a flake detector that classifies each case as stable or flaky from repeated trials and reports pass@k and pass^k with Wilson intervals; and a paired regression gate that averages trials per case, tests per-case differences with a t-based CI and an exact one-sided McNemar test, and blocks the merge on a significant drop, a critical-case regression, a dropped case, or a quarantine entry without owner and expiry. Ships cs-eval-gatekeeper agent, /cs-agent-evals and /cs-eval-gate commands, 3 references, and a CI job example.

29,725 on GitHubby Alireza Rezvaniv2.14.5MITupdated yesterdaydevelopment
/

Plugin agent-regression-evals from marketplace claude-code-skills in alirezarezvani/claude-skills. Commands read from the repo's manifest — today.

What it installs

Skills (1)

  • agent-regression-evals

    Use before shipping any agent, prompt, model or tool change, or when a pass-rate drop needs a verdict. Runs repeated trials, gates candidate vs baseline with paired statistics in regression_gate.py, checks tool-call trajectories and finds flaky cases.

Slash commands (2)

  • /agent-regression-evals:cs-agent-evals

    Full regression-eval pass on an agent change - trajectories, flaky cases, and the paired baseline-vs-candidate gate.

  • /agent-regression-evals:cs-eval-gate

    Just the merge gate - paired statistical comparison of a candidate eval run against baseline, with a PASS / WARN / BLOCK exit code.

Subagents (1)

  • cs-eval-gatekeeper

    Use when someone wants to know whether an agent, prompt, model or tool change made their agent worse, is setting up a regression eval suite or CI eval gate, or is arguing about whether a pass-rate drop is real. Runs repeated-trial evals through three deterministic scripts and returns a PASS / WARN /

View source on GitHub alirezarezvani.medium.com

More from alirezarezvani/claude-skills

All 113 plugins →

45 advanced engineering skills: agent designer, agent workflow designer, RAG architect, database designer + schema designer + SQL assistant, migration architect, observability designer, dependency auditor, changelog generator, API design reviewer, API test suite builder, CI/CD pipeline builder, MCP server builder, skill security auditor, skill tester, performance profiler, focused-fix, browser-automation, full-page-screenshot, git-worktree-manager, monorepo-navigator, codebase-onboarding, interview-system-designer, runbook-generator, spec-driven-workflow, secrets-vault-manager, env-secrets-manager, pr-review-expert, self-eval, tc-tracker, feature-flags-architect, kubernetes-operator, chaos-engineering, ship-gate, slo-architect (error-budget + multi-window burn-rate alerts per Google SRE Workbook), tech-debt-tracker, agent-regression-evals, coding-agent-rollout, commit-craft, agent-run-forensics and i18n-localization. Agent skill and plugin for Claude Code, Codex, Gemini CLI, Cursor, OpenClaw.

48 skills · hooks

29,725

47 marketing skills across 8 pods: Content, SEO & AEO, CRO, Channels, Growth, Intelligence, Sales enablement, and X/Twitter growth. 62 Python tools, 89 reference docs.

47 skills · hooks

29,725

32 engineering skills: architecture, frontend, backend, fullstack, QA, DevOps, security, AI/ML, data engineering, Playwright (9 sub-skills), self-improving agent, Stripe integration, TDD guide, tech stack evaluator, Google Workspace CLI, a11y audit (WCAG 2.2), Azure cloud architect, GCP cloud architect, security pen testing, Snowflake development, adversarial-reviewer, ai-security, cloud-security, incident-response, red-team, threat-detection. v2.8.1 audits senior-fullstack / senior-frontend / senior-backend against karpathy-coder + Matt Pocock — each ships a 7-question forcing-question library, 4 customization profiles (JSON), deterministic decision engine, composition map into POWERFUL specialists, plus cs-fullstack-engineer / cs-frontend-engineer / cs-backend-engineer orchestrator agents (context: fork) + /cs-fullstack-review, /cs-frontend-review, /cs-backend-review, /cs-engineer-grill slash commands.

35 skills · hooks

29,725

Founder-mode executive team plugin: 13 cs-* C-suite agents (CFO, CMO, CRO, CPO, COO, CHRO, CISO, Chief of Staff, General Counsel, Chief Data Officer, Chief AI Officer, Chief Customer Officer, VP of Engineering) with distinct cognitive voices, plus 21 /c-level-agents:* slash commands — forcing-question office hours (CFO/CMO/CPO/CRO/CTO/CISO/GC/CDO/CAIO/CCO/VPE reviews), strategic sprint pipeline (brief → boardroom → decide → execute → post-mortem), and meta routing (/c-level-agents:founder-mode auto-router, /cs-onboard, /c-level-agents:cross-eval multi-model consensus, /c-level-agents:freeze cooldown lock). Wraps the 33 c-level skills with cognitive gearing, persona voice, and artifact-driven handoffs. The business-domain answer to YC Garry Tan's gstack.

22 skills · 13 agents

29,725

33 C-level advisory skills (install the separate companion c-level-agents plugin for the persona layer): virtual board of directors (CEO, CTO, COO, CPO, CMO, CFO, CRO, CISO, CHRO) plus General Counsel, CDO, CAIO, CCO, and VP of Engineering (DORA delivery throughput analyzer, engineering hiring funnel calculator with conversion + pipeline gap, eng team structure designer with squad/tribe + manager-trigger), executive mentor, founder coach, orchestration (Chief of Staff, board meetings, decision logger), strategic capabilities (board deck builder, scenario war room, competitive intel, M&A playbook), culture frameworks. The 13 cs-* persona agents + 21 /c-level-agents:* slash commands (founder-mode router, office-hours intake, multi-role boardroom, strategic sprint pipeline, cross-model consensus, cooldown freeze) ship in the companion c-level-agents plugin.

34 skills · hooks

29,725

14 regulatory affairs & quality management skills for HealthTech/MedTech: ISO 13485 QMS, MDR 2017/745, FDA 510(k)/PMA, GDPR/DSGVO, ISO 27001 ISMS, CAPA management, risk management, clinical evaluation, SOC 2 compliance.

21 skills

29,725

Similar plugins

superpowers

claudecodex

Core skills library for Claude Code: TDD, debugging, collaboration patterns, and proven techniques

15 skills · hooks

obra/superpowers 316,836

context7

claudecodex

Up-to-date documentation lookup. Pull version-specific documentation and code examples directly from source repositories into your coding agent's context.

1 MCP server

upstash/context7 64,709

Build powerful, stateful CLI interfaces for any GUI application using the cli-anything harness methodology.

5 commands

HKUDS/CLI-Anything 58,665

Automates browser interactions for web testing, form filling, screenshots, and data extraction

12 skills

vercel-labs/agent-browser 50,373

Use Alibaba Open Code Review from local Codex.

2 skills

alibaba/open-code-review 50,068

slidev

claude

Claude Code skill for creating and presenting Slidev slidedecks

1 skill

slidevjs/slidev 49,452

Maintain agent-regression-evals?

This page updates itself from your repo. Add the badge to your README, or get it Featured to pin it above the organic results with a followed link to your site.

claude & codex plugins113 plugins