review-bench

Plugin review-bench from marketplace claude-craft in whichguy/claude-craft. Commands read from the repo's manifest — 14 days ago.

Skills (8)

• ablate-review-plan

  Ablation test harness for the review-plan skill. Runs the structured question-based control (SKILL.md) and the directive-based ablated variant (SKILL-v-ablation-na.md, per-directive N/A semantics) against the same fixture set with k=3 repetition, then uses the review-plan-ablation-judge to compare o

• compare-questions

  A/B test two planning questions by applying each to one or more plans, judging which question produces a better plan revision. Priority chain: quality > input tokens (question size) > time. AUTOMATICALLY INVOKE when user mentions: - "compare questions", "which question is better", "A/B test question

• derive-questions

  Iteratively researches real software project failures and wins, extracts key planning questions via 5-whys analysis, validates them against synthetic test plans, judges their effectiveness via a parallel judge agent, refines them, and persists to a growing questions library. Builds a curated, langua

• improve-prompt

  Research-backed prompt improvement workflow. Analyzes with Q1-Q13 structural diagnostics, researches domain + prompt engineering best practices, generates fixed+dynamic evaluation questions, validates improvement plan (quality gate), runs E parallel experiment variants, scope-preservation gate (12-q

• optimize-questions

  Systematically optimize review-plan evaluator questions for token efficiency while maintaining quality. For each target question: generates 3 token-optimized variants (STRUCTURAL, SEMANTIC, RADICAL) in parallel, selects the best one (optionally synthesizing strengths from others), A/B tests it again

• optimize-system-prompt

  Optimize or refine the Sheets Chat ClaudeConversation system prompt. Analyze, compress, refine, A/B test, and deploy system prompt changes with automated quality gates. **AUTOMATICALLY INVOKE** when: - User mentions "system prompt", "optimize prompt", "compress prompt" - User wants to change ClaudeC

• question-bench

  Benchmark review-plan question effectiveness via experiment-based ablation. Applies different question subsets to a plan (or directory of plans) in parallel experiments, evaluates each against fixed plan-quality questions (Q-PQ1..Q-PQ8), and compares quality spreads to identify which questions drive

• validate-questions

  End-to-end validation of review-plan question effectiveness. Runs a multi-stage pipeline against the bench bank of realistic plans to determine which questions create material value, which overlap, and which should be removed. Produces a Question Effectiveness Report with per-question KEEP/MERGE/CON

Subagents (4)

• compare-questions-judge

  Pairwise LLM judge for compare-questions skill. Evaluates two revised plans (produced by applying two different planning questions to the same original plan) on 5 research-informed criteria from a staff engineer's perspective, derives winner by majority vote, self-checks trial validity (recusing if

• prompt-comparator

  Compare prompts across quality, token efficiency, and output effectiveness. **AUTOMATICALLY INVOKE** when user mentions: - "compare prompts", "which prompt is better", "prompt efficiency" - "A/B test prompts", "evaluate prompts", "test these prompts" - Multiple prompt variations to choose between **

• question-bench-judge

  Position-blind comparative plan quality judge for question-bench skill. Compares two implementation plan versions on 8 fixed evaluation dimensions (Q-PQ1..Q-PQ8). Returns structured JSON only — no prose. Spawned by question-bench skill for each experiment comparison. Receives two plan versions as X/

• review-plan-ablation-judge

  Logical-equivalence judge for the review-plan ablation experiment. Compares two plan reviews — one from the structured question-based control, one from the directive-based ablated variant — on 5 criteria that measure whether the ablation silently drops, adds, or misweights issues relative to the con

Similar plugins