claude

mizan-eval

Run Mizan evaluations from an agent: score a single response or compare two responses against a metric template via the mizan CLI, reasoning over the -o json result. Use when you need to evaluate an asset/response against guidance and explain the verdict.

3 on GitHubApache-2.0updated 2 days ago
/

Plugin mizan-eval from marketplace mizan in ghchinoy/mizan. Commands read from the repo's manifest — 2 days ago.

What it installs

Skills (2)

  • run-eval-set

    Run a curated multi-concern Mizan eval-set (an EvalSet manifest) against a shared asset/response, interpret the weighted scorecard — per-member verdicts, the aggregate, the overall PASS/FAIL, and the gate — and honor the gate exit code for CI, driving the `mizan` CLI over `-o json`. Use when a user

  • run-eval

    Run a single (pointwise) or pairwise Mizan evaluation of a response/asset against a metric template and explain the verdict, driving the `mizan` CLI over `-o json` and reasoning over the parsed result. Use when a user asks to evaluate, score, grade, or A/B-compare a text response or media asset agai

View source on GitHub ghchinoy.github.io

More from ghchinoy/mizan

All 5 plugins →

mizan-authoring

claude

Author metric-template packs for Mizan from an agent: scaffold a pack, create boul/choice/score/rubric/pointwise/pairwise/custom_schema/heuristic templates via the mizan CLI, validate the pack creds-free (the CI PR gate), and run the export->PR->import collaborator loop; and generate a rubric from a brand book (suggest-first, agent-side synthesis) then freeze it into the registry. Use when you need to author, package, validate, or share a Mizan metric template or template pack, or to generate evaluation rubrics from a brand book or guidance document.

3 skills

3

mizan-mcp

claude

Expose the Mizan eval engine to agents and MCP clients as callable tools via the `mizan mcp` server. Registers a local stdio MCP server (`mizan mcp stdio`) advertising four tools — mizan_list_metrics, mizan_get_metric (read-only), mizan_eval_run, and mizan_eval_pairwise — plus an agent-in-the-loop eval skill. Use when an agent should discover a judge and run an evaluation over MCP rather than shelling out to the CLI.

1 skill · 1 MCP server

3

mizan-results

claude

Turn stored Mizan eval results into a self-contained, standalone HTML report from an agent: query results via the mizan CLI over -o json, aggregate client-side, and render a single offline report.html. Use when you need to review, summarize, or visualize past evaluation results as an HTML report.

2 skills

3

mizan-setup

claude

Configure Mizan for first use from an agent: inspect the resolved configuration via the mizan CLI over -o json, persist project/location/default-model/results-store settings with `config set`, verify the CLI build with `version`, and check (never store) Application Default Credentials. Use when you need to set up or verify a Mizan environment before running live evaluations.

1 skill

3

Maintain mizan-eval?

This page updates itself from your repo. Add the badge to your README, or get it Featured to pin it above the organic results with a followed link to your site.

claude & codex plugins5 plugins