truesight

Plugin truesight from marketplace goodeye-labs-truesight in Goodeye-Labs/truesight-mcp-skills. Commands read from the repo's manifest — 1 month ago.

Skills (9)

• bootstrap-template-evaluation

  Fastest route to a deployed live evaluation using a pre-built Truesight template. Use when the user wants a quick start without building judgment configs from scratch.

• build-review-interface

  Build a custom web interface for trace annotation and review. Use when users need a bespoke review surface for their workflow.

• create-evaluation

  Scope what quality should be measured, convert it into one or more actionable binary evaluations, deploy those evaluations through Truesight MCP, and generate a companion skill that applies them correctly. Use when a user wants to create new evals, quality checks, guardrails, or pass/fail criteria f

• error-analysis

  Systematically identify and categorize failure modes in evaluated traces using Truesight datasets and error-analysis tools. Use when quality issues are unclear, after major pipeline changes, or when incidents indicate drift.

• eval-audit

  Audit an existing evaluation workflow and produce severity-ranked findings with concrete next actions. Use when inheriting an eval setup, diagnosing quality regressions, or checking LLM evaluation process maturity.

• evaluate-trace

  Evaluate one or more traces against an existing Truesight live evaluation. Use when a deployed live evaluation already exists and the user wants run outputs with optional handoff to review and promotion.

• generate-synthetic-data

  Generate synthetic test data for LLM evaluations using dimension-based tuple expansion. Use when the user needs synthetic traces, test cases, eval datasets, or when create-evaluation needs synthetic fallback data.

• review-and-promote-traces

  Judge flagged trace outputs and promote judged items back to datasets. Use when an evaluation run requires human judgment or when review queue items need to be judged for promotion into the dataset.

• truesight-workflows

  Orchestrator for Truesight MCP skills. Use this when the user needs help choosing the right Truesight workflow or when intent is ambiguous across LLM evaluate, error analysis, review, templates, or evaluation creation.

MCP servers (1)

• truesight

  connects to: https://api.truesight.goodeyelabs.com/mcp/

Similar plugins