agentic-bundle-aas-ai-product-evaluation-ops

Plugin agentic-bundle-aas-ai-product-evaluation-ops from marketplace agentic-awesome-skills in sickn33/agentic-awesome-skills. Commands read from the repo's manifest — today.

Skills (13)

• ab-test-setup

  Use when designing an A/B or split test: define the hypothesis, control and variants, estimate sample size, verify tracking, and predeclare metrics and stopping rules.

• advanced-evaluation

  Design LLM-as-judge rubrics, direct scores and pairwise comparisons. Use for judge models, evals, position bias and calibration against human labels.

• agent-evaluation-reporting

  Use when summarizing agent evaluations where autonomous, assisted, failed, timed-out, or invalid outcomes must remain distinct and comparable.

• agent-evaluation

  Evaluate agent behavior with versioned cases and explicit verifiers. Use when comparing agent or prompt changes, reproducing failures, or running agent regression tests.

• ai-wrapper-product

  Expert in building products that wrap AI APIs (OpenAI, Anthropic, etc. ) into focused tools people will pay for. Not just "ChatGPT but different" - products that solve specific problems with AI.

• analytics-product

  Analytics de produto — PostHog, Mixpanel, eventos, funnels, cohorts, retencao, north star metric, OKRs e dashboards de produto.

• context-window-management

  Strategies for managing LLM context windows including summarization, trimming, routing, and avoiding context rot

• hugging-face-evaluation

  Add and manage evaluation results in Hugging Face model cards. Supports extracting eval tables from README content, importing scores from Artificial Analysis API, and running custom model evaluations with vLLM/lighteval. Works with the model-index metadata format.

• kpi-dashboard-design

  Comprehensive patterns for designing effective Key Performance Indicator (KPI) dashboards that drive business decisions.

• langfuse

  Expert in Langfuse - the open-source LLM observability platform. Covers tracing, prompt management, evaluation, datasets, and integration with LangChain, LlamaIndex, and OpenAI. Essential for debugging, monitoring, and improving LLM applications in production.

• llm-app-patterns

  Architecture and integration sketches for LLM applications, with explicit retrieval, tool, privacy and verification boundaries.

• product-manager

  Guide product strategy, discovery, prioritization and metrics; upstream templates and formula collections are external prerequisites.

• skill-security-audit

  Audit an Agent Skill, MCP server, connector, or desktop extension before installation by tracing code, dependencies, permissions, credentials, data flow, and irreversible actions.

Similar plugins