scorecard

Plugin scorecard from marketplace scorecard in scorecard-ai/scorecard-skills. Commands read from the repo's manifest — 1 month ago.

Skills (7)

• scorecard-agent-brief

  Write the Scorecard agent brief. Reads an AI agent's code and writes a short brief (what it does, inputs, outputs, tools, users, what "good" and "bad" look like) to scorecard/agent-brief.md. The Scorecard metrics, test set, and eval skills read this brief, so it must use this skill's exact section h

• scorecard-evals

  Run a Scorecard evaluation of an AI agent. Runs it on a Scorecard test set, scores it with Scorecard metrics, and prints the run URL. This skill has the exact Scorecard SDK calls and record shapes, so load it before you write an eval script instead of guessing the API. Also covers linking traces to

• scorecard-metrics

  Create Scorecard metrics (evaluators, graders, LLM-as-judge) for an AI agent. Turns the agent brief into 3 to 5 metrics (AI-judged, human, or heuristic; boolean, 1-5, or 0-1), writes them as a re-runnable script in scorecard/, and creates them in the Scorecard project. This skill has the exact Score

• scorecard-onboard

  Onboard an AI agent or LLM app to Scorecard end to end, for requests like "set up Scorecard", "onboard my agent to Scorecard", "evaluate my agent with Scorecard", or "add Scorecard". Runs setup, agent brief, tracing, metrics, test set, and a first evaluation in order, using the other Scorecard skill

• scorecard-setup

  Connect a repo to Scorecard. Gets and checks the Scorecard API key, picks or creates a Scorecard project, and writes SCORECARD_API_KEY and SCORECARD_PROJECT_ID to the env file. Also connects the Scorecard MCP server. This skill has the exact Scorecard API endpoints and env var names, so load it inst

• scorecard-testsets

  Turn a CSV, TSV, JSON, JSONL, or spreadsheet file into Scorecard test cases (scorecard/testcases.jsonl), or write new test cases for an agent. Scorecard test cases are flat rows keyed by the test set's schema field names, plus a field mapping. Generic shapes such as {"input", "expected_output"} or a

• scorecard-tracing

  Add Scorecard tracing to an AI agent or LLM app, for requests like "add Scorecard tracing" or "trace, instrument, monitor, or log this agent in Scorecard". Scorecard tracing uses specific packages, wrapper functions, and endpoints that are easy to guess wrong (for example, the Vercel AI SDK uses wra

MCP servers (1)

• scorecard

  connects to: https://mcp.scorecard.io/mcp

Similar plugins