claude

qa-llm-evaluation

LLM and prompt evaluation with Promptfoo, DeepEval and Ragas, Giskard vulnerability scans, Langfuse tracing, and golden-dataset regression suites. Audits eval suites for methodology errors. Agents (prompt-eval-reviewer, llm-red-team-planner) review an eval suite and plan adversarial probe campaigns across the OWASP LLM Top 10.

13 on GitHubby testlandv1.4.6MITupdated yesterdayai-ml
/

Plugin qa-llm-evaluation from marketplace testland-qa in testland/qa. Commands read from the repo's manifest — today.

What it installs

Skills (7)

  • deepeval-evaluation

    Use for any question or task about testing an LLM or RAG app with pytest, including hallucination checks in CI and answers that invent facts missing from the retrieved documents: DeepEval LLMTestCase, built-in metrics such as G-Eval, answer relevancy, faithfulness, contextual recall and precision, h

  • giskard-llm

    Use for any question or task about red-teaming or adversarial testing of an LLM app or chatbot, including prompt injection, system-prompt or data leaks, and harmful output before release. Runs giskard.scan on any model wrapped as giskard.Model to probe hallucination, harmful content, prompt injectio

  • langfuse-tracing

    Use for any question or task about tracing, monitoring or debugging LLM calls in production, including having no record of what the model saw for a bad answer, or turning production traces into eval datasets. Adds Langfuse with @observe or startActiveObservation, records metadata, cost and latency,

  • llm-eval-anti-patterns

    Use for any question about mistakes in LLM evals or auditing an eval suite that gates a release, model swap or prompt change, including evals that pass while users say the model got worse. Checks eight methodology errors (too few cases, exact match on open text, no baseline, unpinned models, no adve

  • llm-regression-suite-author

    Use for any question or task about catching LLM quality regressions, including checking a new model version or provider against the old one and prompt tweaks that break other cases. Builds a versioned golden dataset, deterministic runs, exact, semantic, judge and rubric checks, and a pass-rate CI ga

  • promptfoo-evaluation

    Use for any question or task about Promptfoo or adding quick regression checks for prompts or RAG pipelines in CI, including turning written requirements into assertions: promptfooconfig.yaml providers, prompts and tests, deterministic, semantic, model-graded, latency and cost assertions, promptfoo

  • ragas-evaluation

    Use for any question or task about RAG or agent evaluation metrics, including which metrics to use, scoring an answer against its contexts, and telling retrieval problems from generation problems after a change such as chunk size: Ragas faithfulness, response relevancy, context precision and recall,

Subagents (2)

  • llm-red-team-planner

    Use proactively whenever an LLM app needs a red-team campaign beyond default scanner presets. Enumerates attack classes (jailbreaks, indirect prompt injection, data exfiltration, OWASP LLM Top 10), maps each to a Giskard detector or promptfoo red-team plugin and writes the scan scripts and configs.

  • prompt-eval-reviewer

    Use proactively whenever an LLM eval suite (Promptfoo, OpenAI Evals, DeepEval, Ragas, Giskard or custom) is added or changed. Flags too few cases, one provider only, missing model-graded or semantic checks, no baseline diff, no cost or latency cap, unpinned models and no adversarial cases.

View source on GitHub testland.io

More from testland/qa

All 78 plugins →

qa-security-scanning

claude

Security scanning across SAST, DAST, SCA, secrets and SBOM: Semgrep, CodeQL, SonarQube, ZAP, Nuclei, Snyk, OSV-Scanner, gitleaks, Syft and Trivy. Also covers language-native SAST linters, Dependabot and Renovate, package-manager audits, reachability and CVE exploitability triage, secret rotation, and VEX. A gate agent (security-finding-triager) normalizes, dedupes and waiver-checks multi-scanner output, including IaC findings from qa-iac, into one BLOCK / PASS verdict per domain. Every scanner skill includes a false-positive triage section.

17 skills · 1 agent

13

qa-process

claude

Test process and strategy: risk-based testing, test strategies, Definition of Done, smoke gates, test pyramid balance, effort estimates and post-mortems. Also covers E2E suite budgets, test ideas from stories and live features, framework choice, and attack-surface checklists. Agents (release-readiness-checker, quality-coach, release-engineer) run multi-gate go / no-go checks, audit work against the Definition of Done, and walk a release runbook with human gates.

12 skills · 3 agents

13

qa-test-data

claude

Test data engineering: Faker and factories, WireMock and MSW API mocks, seed datasets, golden files, and boundary, pairwise and negative test inputs. Includes FactoryBot, Mimesis and Bogus, a Mountebank reference, synthetic PII, and a malicious payload bank. Agents (golden-file-manager, test-data-setup-agent) maintain snapshot baselines and set up a feature's full test-data layer.

12 skills · 2 agents

13

qa-test-reporting

claude

Test reporting and coverage analytics: JUnit XML, LCOV and Cobertura parsing, Allure reports, coverage diffs, and test run summaries for pull requests. Also covers JaCoCo and coverage.py analysis, risk-ranked coverage targets, TestRail / Xray / Zephyr Scale result sync, and Currents analytics. Agents (daily-test-suite-aggregator, release-quality-report-agent) roll up daily results across suites and assemble a release-quality go / no-go report. Jest and Vitest coverage lives in qa-unit-tests-js.

10 skills · 2 agents

13

qa-test-review

claude

Test code review for test files only: conventions, isolation, object-model patterns, framework architecture audits, and test suite health checks. Includes a framework blueprint for new suites and a test design scorecard. Agents (test-code-critic, e2e-selector-quality-critic, framework-architecture-auditor, test-suite-health-auditor) review structure, assertions and mocking, selector fragility, framework architecture, and suite health. Production code is out of scope.

7 skills · 4 agents

13

qa-web-e2e

claude

Web E2E testing with Playwright, Cypress, Selenium and WebdriverIO, plus cloud grids, a self-hosted Selenium Grid 4, and cross-browser test matrices. Covers BrowserStack, Sauce Labs and LambdaTest, traffic-based T1/T2/T3 browser tiers, a framework decision guide, and Playwright's agent tooling (playwright-cli, Playwright MCP, planner/generator/healer Test Agents). Agents (playwright-codegen-reviewer, automation-harness-bootstrapper) refactor recorded Playwright or Cypress Studio specs and scaffold a full E2E harness for a repo with no suite.

9 skills · 2 agents

13

Similar plugins

agent-browser

claude

Automates browser interactions for web testing, form filling, screenshots, and data extraction

12 skills

vercel-labs/agent-browser 50,373

backend-development

claudecodex

Backend API design, GraphQL architecture, workflow orchestration with Temporal, and test-driven backend development

9 skills · 1 command · 8 agents

wshobson/agents 42,461

gpt-image-2-style-library

claude

Choose GPT-Image2 visual styles, industrial prompt templates, categories, and scene tags.

1 skill

freestylefly/awesome-gpt-image-2 35,755

agentmemory

claudecodex

Persistent memory for AI coding agents -- captures tool usage, compresses via LLM, injects context into future sessions

17 skills · 1 MCP server · hooks

rohitg00/agentmemory 30,326

promptfoo

claudecodex

Connect providers and targets, write eval suites, and set up or run red team scans with promptfoo

4 skills

promptfoo/promptfoo 27,115

ego

claudecodex

Automates browser interactions for web testing, form filling, screenshots, and data extraction

1 skill

citrolabs/ego-lite 22,708

Maintain qa-llm-evaluation?

This page updates itself from your repo. Add the badge to your README, or get it Featured to pin it above the organic results with a followed link to your site.

claude & codex plugins78 plugins