Generic agentic evaluation for skills and agents. Provides end-to-end skills to analyze, test, score, review, and iteratively improve agent skills, plus compare models/configurations and run Design-of-Experiments (ANOVA) sweeps. MLflow support for experiment tracking, tracing, and reporting. Schema-driven evaluation via eval.yaml with support for inline, LLM-based, and external judges.
Plugin agent-eval-harness from marketplace opendatahub-skills in opendatahub-io/skills-registry. Commands read from the repo's manifest — 8 days ago.
This plugin runs code on your machine — its hooks fire automatically on events like tool use. The exact commands are listed below; read them before installing.
eval-analyzeGenerate eval.yaml for the agent eval harness. Two modes - (1) Skill-based - examines SKILL.md, sub-skills, scripts, test cases to verify implementation quality, OR (2) Prompt-based - tests agent capabilities using custom analysis prompts (documentation effectiveness, pattern understanding, API usag
eval-anovaRun Design-of-Experiments (DoE) evaluations with ANOVA over a matrix of agent configurations — comparing models, thinking-effort levels, prompts, or other factors across shared test cases, with repeated-measures / mixed-effects statistics that account for case difficulty plus a cost/quality Pareto v
eval-checkEvaluate the full harness configuration as a system. Scans all skills, commands, CLAUDE.md, and hooks for redundancy, overlap, type misclassification, and structural issues. Produces an informational report with restructuring suggestions. Use when the user wants to check their overall setup health,
eval-compareCompare evaluation results across multiple models or runs. Takes a directory of eval run artifacts (summary.yaml, run_result.json, HTML reports) and produces a tabbed HTML comparison report with model cards, quality/cost tables, per-case breakdowns, and embedded original reports. Use when the user w
eval-datasetGenerate evaluation test cases for an eval.yaml. Sources cases per generation.strategy - skill analysis (default), synthetic LLM generation from generation prompts (documentation and agent-capability evals), or MLflow production traces. Bootstraps a starter dataset or augments an existing one to imp
eval-mlflowMLflow integration for evaluation — sync datasets, log run results, push/pull feedback between the harness and MLflow traces. Use when the user wants to log eval results to MLflow, sync test cases to MLflow datasets, connect judge scores to traces, pull MLflow annotations for eval-optimize, or view
eval-optimizeAutomated skill improvement loop. Runs eval, identifies judge failures, reads traces and rationale, edits the SKILL.md to fix issues, re-runs to verify, and checks for regressions. Use when the user wants to automatically improve a skill based on eval results, fix failing judges, make the skill bett
eval-reviewInteractive review of evaluation results. Presents judge scores and skill outputs for human feedback, then proposes SKILL.md improvements based on what the user identifies. Use when the user wants to review eval results, look at results, check scores, see what went wrong, give qualitative feedback o
eval-runExecute an evaluation against test cases (skill or prompt mode), score with judges, and report results. Requires eval.yaml (generated by /eval-analyze). Use when the user wants to test a skill, run eval, benchmark, compare models, detect regressions, check skill quality, or verify changes didn't bre
eval-setupOptional environment configurator for the agent-eval-harness. Configures MLflow tracking, verifies API keys, and troubleshoots dependency issues. Detects available skills and agentic documentation (CLAUDE.md, AGENTS.md, ai-docs/) to suggest appropriate evaluation modes. Not required for basic usage
python3 "${CLAUDE_PLUGIN_ROOT}/scripts/ensure_deps.py" "${CLAUDE_PLUGIN_DATA}"
[DEPRECATED] Backwards-compatibility umbrella for the ODH AI Helpers plugins. Re-exports the skills that existed before the split into the individual odh-* plugins under their original odh-ai-helpers:* names, so existing agents and workflows keep working. Install the odh-* plugins you need, then uninstall this one; it will be removed after the migration window.
72 skills
Documentation review, writing, and workflow tools for AsciiDoc and Markdown documentation. Includes an orchestrated multi-step pipeline, standalone review skills, codebase analysis for onboarding, and JIRA/PR integration.
52 skills · 16 agents
Everything you need for PatternFly development -- React components, design guidance, migration, and MCP docs. Installs all PatternFly sub-plugins in one step.
55 skills
Consensus-based security review for RHOAI strategy documents (STRATs). An orchestrator spawns three independent reviewers to identify security risks, then synthesizes findings with confidence tagging based on cross-reviewer agreement. Covers 39 catalog patterns across auth, data protection, cryptographic compliance, network security, supply chain, and infrastructure.
29 skills
React component development -- coding standards, testing, and structure
16 skills · 3 agents
Claude Code plugin with specialized skills for DevOps and cloud-native development workflows. Provides skills for GitLab CI/CD analysis, AWS CloudWatch Logs troubleshooting, Slack integration, GitLab branch management, Jira utilities, Konflux ITS analysis, cloud infrastructure provisioning via mapt, and Google Workspace integration.
18 skills
Matt Pocock's agent skills for real engineering (grilling, spec/ticket flows, TDD, code review, domain modelling and more).
39 skills
Core skills library for Claude Code: TDD, debugging, collaboration patterns, and proven techniques
15 skills · hooks
Official Next.js skills: adopt and optimize Cache Components, adopt Partial Prefetching, and verify runtime behavior against a running dev server.
6 skills
Persistent memory for Claude Code: the next session starts where the last one left off.
29 skills · 1 MCP server · hooks
tasteskill 2.5.1: professional company websites at the figma.com and x.ai level, with one section per screen, real photographs, real product UI, effects kept for tech and gaming pages, and a real-browser checker and finish gate.
1 skill
Open-source AI job search that runs locally in your AI coding CLI: scan job portals, evaluate listings into a structured A-H report with a 1-5 score, tailor your CV, track applications
5 skills · hooks
This page updates itself from your repo. Add the badge to your README, or get it Featured to pin it above the organic results with a followed link to your site.