claude

llm-evals

LLM testing and evaluation framework with promptfoo, DeepEval, golden datasets, and Supabase-backed eval tracking

10 on GitHubby AI Dev Marketplacev1.0.0updated 1 month agodevelopment
/

Plugin llm-evals from marketplace ai-dev-marketplace in vanman2024/ai-dev-marketplace. Commands read from the repo's manifest — 1 month ago.

What it installs

Skills (3)

  • deepeval-testing

    DeepEval pytest-style LLM testing patterns with built-in metrics, custom evaluators, and CI integration. Use when creating LLM tests, evaluating RAG quality, or measuring faithfulness/relevance.

  • eval-tracking

    Supabase-backed evaluation tracking with runs, cases, and scores tables. Use when storing eval results, building dashboards, or tracking regression over time.

  • promptfoo-config

    promptfoo configuration patterns for prompt regression testing, multi-provider comparison, and assertion-based validation. Use when setting up prompt testing, comparing LLM providers, or creating eval pipelines.

Slash commands (2)

  • /llm-evals:add

    Add a specific eval feature to an existing project. Features include promptfoo, deepeval, golden-dataset, supabase-tracking.

  • /llm-evals:build

    Build complete LLM evaluation system with promptfoo, DeepEval, golden datasets, and Supabase tracking

Subagents (4)

  • dataset-curator

    Manages golden datasets for LLM evaluation - test case creation, categorization, and version control

  • deepeval-specialist

    Specializes in DeepEval pytest-style LLM testing with built-in metrics and custom evaluations

  • eval-orchestrator

    Orchestrates LLM evaluation workflows - coordinates promptfoo, DeepEval, datasets, and tracking

  • promptfoo-specialist

    Specializes in promptfoo configuration, prompt regression testing, and multi-provider comparison

View source on GitHub

More from vanman2024/ai-dev-marketplace

All 25 plugins →

clerk

claude

Comprehensive Clerk authentication plugin with OAuth (19 providers), organizations, billing, MFA, Next.js integration, and Supabase sync for building secure multi-tenant applications

10 skills · 2 commands · 15 agents

10

Machine learning training and inference pipeline using cloud GPUs (Modal, Lambda Labs, RunPod) with HuggingFace ecosystem - no local GPU required

8 skills · 2 commands · 16 agents

10

supabase

claude

Comprehensive Supabase integration for AI applications with database, auth, storage, realtime, and vector search capabilities

8 skills · 2 commands · 15 agents

10

celery

claude

Production-ready Celery distributed task queue with worker management, beat scheduling, monitoring (Flower), and framework integrations (Django, Flask, FastAPI)

10 skills · 2 commands · 12 agents

10

Modular Vercel AI SDK development plugin with feature bundles and specialized agents. Build AI applications incrementally or all-at-once. Supports TypeScript, JavaScript, and Python.

5 skills · 2 commands · 14 agents

10

AI-powered website builder with Astro, MDX, content-image-generation MCP, Supabase CMS, and component integration for marketing sites, blogs, and documentation

6 skills · 2 commands · 10 agents

10

Similar plugins

superpowers

claudecodex

Core skills library for Claude Code: TDD, debugging, collaboration patterns, and proven techniques

15 skills · hooks

obra/superpowers 316,836

career-ops

claudecodex

Open-source AI job search that runs locally in your AI coding CLI: scan job portals, evaluate listings into a structured A-H report with a 1-5 score, tailor your CV, track applications

5 skills · hooks

career-ops-hq/career-ops 82,253

context7

claudecodex

Up-to-date documentation lookup. Pull version-specific documentation and code examples directly from source repositories into your coding agent's context.

1 MCP server

upstash/context7 64,709

Build powerful, stateful CLI interfaces for any GUI application using the cli-anything harness methodology.

5 commands

HKUDS/CLI-Anything 58,665

5 skills + 35 modes + Material Passport pipeline. Includes v3.6.7 cross-model audit gate and v3.6.8 generator-evaluator contract.

5 skills · 16 commands · 4 agents · hooks

Imbad0202/academic-research-skills 56,698

Automates browser interactions for web testing, form filling, screenshots, and data extraction

12 skills

vercel-labs/agent-browser 50,373

Maintain llm-evals?

This page updates itself from your repo. Add the badge to your README, or get it Featured to pin it above the organic results with a followed link to your site.

claude & codex plugins25 plugins