mlflow
Skills for tracing, evaluating, and improving AI agents with MLflow. Supports the full agent improvement loop: instrument → trace → evaluate → iterate → validate.
Not in a marketplace yet, so it loads for one session with --plugin-dir.
Plugin mlflow in mlflow/skills. Commands read from the repo's manifest — 9 days ago.
This plugin runs code on your machine — its hooks fire automatically on events like tool use. The exact commands are listed below; read them before installing.
What it installs
Skills (10)
mlflow-agent-improvementFix or change an AI agent's behavior using MLflow traces and regression tests. Use for wrong answers, missing tool calls, new business rules, or context and token-cost optimization. Diagnose first, prove a regression fails, implement the smallest fix, and verify it passes without breaking existing b
mlflow-agentMaster dispatcher for all MLflow workflows. Use this skill when the user wants to do anything with MLflow — tracing, evaluating, debugging, or improving an agent. Routes to the right MLflow sub-skill automatically. Triggers on: "use mlflow", "help with mlflow", "mlflow agent", "add mlflow to my proj
mlflow-docsSearches and retrieves MLflow documentation from the official docs site. Use when the user asks about MLflow features, APIs, integrations (LangGraph, LangChain, OpenAI, etc.), tracing, tracking, or requests to look up MLflow documentation. Triggers on "how do I use MLflow with X", "find MLflow docs
mlflow-evaluationDesign MLflow scorers and run GenAI evaluation on an agent, dataset, precomputed outputs, or traces. Use for quality criteria, judges, MemAlign, datasets, evaluation runs, and results analysis.
mlflow-onboardingOnboards users to MLflow by determining their use case (GenAI agents/apps or traditional ML/deep learning) and guiding them through relevant quickstart tutorials and initial integration. If an experiment ID is available, it should be supplied as input to help determine the use case. Use when the use
mlflow-sagemakerConnect to SageMaker Managed MLflow (mlflow-app or mlflow-tracking-server ARN) as an MLflow backend, then hand off to the other MLflow skills. Triggers on a SageMaker MLflow ARN (arn:aws:sagemaker:...:mlflow-app/... or arn:aws:sagemaker:...:mlflow-tracking-server/...) or "SageMaker Managed MLflow".
mlflow-trace-analysisAnalyze MLflow traces, agent failures, and multi-turn chat sessions. Explain errors, unexpected output, quality issues, latency, or cross-turn propagation without editing application code.
mlflow-trace-instrumentationAdd or repair MLflow tracing in Python or TypeScript applications, including tracing required for evaluation. Covers framework autologging, custom spans, backend setup, and persisted trace verification.
mlflow-trace-metricsQuery aggregate MLflow trace, span and assessment metrics for latency, tokens, counts and quality trends. Use for operational summaries rather than individual-trace diagnosis.
mlflow-trace-retrievalRetrieve MLflow traces by ID or search by run, session, user, status, time, tags, feedback, or span fields. Use for trace lookup and filtering; retrieval alone does not require diagnosis.
Hooks (1)
- UserPromptSubmit
${CLAUDE_PLUGIN_ROOT}/hooks/mlflow-suggest-hook.py
Maintain mlflow?
This page updates itself from your repo. Add the badge to your README, or get it Featured to pin it above the organic results with a followed link to your site.