langfuse

Plugin langfuse from marketplace zweihander in Alex-Kopylov/zweihander. Commands read from the repo's manifest — 25 days ago.

Skills (28)

• langfuse

  General-purpose Langfuse integration for observability, evaluations, datasets, dataset items, experiment runs, remote experiment webhooks, dashboard widgets, dashboards, metrics, traces, scores, models, and filter discovery. Use for any Langfuse task including discovering project data, querying metr

• analyze-experiment-results

  Use when the user wants to analyze experiment results, inspect scores from a dataset run, check pass/fail rates, review per-item outputs, or deep-dive into experiment performance. Trigger phrases: "analyze results", "experiment scores", "how did the experiment perform", "show results", "inspect run"

• compare-experiments

  Use when the user wants to compare two or more experiment runs, detect regressions, see score deltas between runs, or evaluate model performance differences. Trigger phrases include "compare runs", "compare experiments", "diff runs", "regression check", "which run is better", "model comparison", "A/

• configure-remote-experiment

  This skill should be used when the user wants to configure a Langfuse dataset for remote experiment triggering from the UI, set up a webhook URL, update the default experiment payload, or enable the Custom Experiment feature. Trigger phrases include "configure remote experiment", "set webhook URL",

• create-dataset

  This skill should be used when the user wants to create a new Langfuse dataset, set up a dataset for benchmarking, or create a dataset with input/output schema validation. Trigger phrases include "create dataset", "new dataset", "set up dataset", "add dataset".

• create-evaluator

  Use when the user wants to create a Langfuse LLM-as-a-Judge evaluator. Trigger phrases: "create evaluator", "add evaluator", "new evaluation", "set up evaluation criteria", "create judge". Handles prompt composition, schema validation, ID generation, SQL insertion into eval_templates and job_configu

• create-widget

  This skill should be used when the user wants to create a new Langfuse dashboard widget. Trigger phrases include "create widget", "add widget", "new visualization", "create chart", "add chart to dashboard". It handles ID generation, schema validation, SQL insertion, and post-creation verification of

• delete-evaluator

  Use when the user wants to delete or remove an LLM-as-a-Judge evaluator from Langfuse. Trigger phrases: "delete evaluator", "remove evaluator", "drop evaluator", "clean up evaluator". Handles safety checks, confirmation, transaction-wrapped deletion of job_configurations and eval_templates, and post

• delete-widget

  Use when the user wants to delete/remove a Langfuse dashboard widget. Trigger phrases: "delete widget", "remove widget", "drop widget". Performs referential integrity checks against dashboards before deletion and cleans up dashboard references.

• design-dataset-schema

  This skill should be used when the user needs help designing the input or expected output schema for Langfuse dataset items. This includes deciding what fields to include, how to structure item inputs for specific pipelines, and creating JSON Schema validation rules. Trigger phrases include "design

• discover-datasets

  This skill should be used when the user asks to discover datasets, list datasets, find what datasets exist, browse dataset items, inspect dataset contents, check dataset schemas, or explore dataset metadata in their Langfuse project. It enumerates all datasets with their items, runs, and schema conf

• discover-filter-options

  Use this skill when the user needs to build or configure trace filters for LLM-as-a-Judge evaluators. It discovers all filterable dimensions in the project (trace names, tags, environments, observation names, levels), helps construct the filter JSON array, validates the filter, and optionally runs a

• discover-models

  This skill should be used when the user asks to discover models, list models, find what models are available, check model pricing, or see which models are actually used in their Langfuse project. It lists both the registered model definitions (with pricing) and the models observed in actual trace da

• discover-scores

  Use when the user asks to discover, list, enumerate, or inspect Langfuse score names, data types, or sources. Enumerates unique score names, data types, and sources via the Langfuse REST API with a direct-database fallback.

• discover-traces

  This skill should be used when the user asks to discover traces, list traces, find what traces exist, inspect trace names, explore trace tags, or check trace environments in their Langfuse project. It enumerates unique trace names, tags, environments, and user IDs via the Langfuse REST API with a di

• inspect-evaluator

  This skill should be used when the user asks to inspect an evaluator, view evaluator details, show evaluator configuration, check an evaluator's prompt, or see the full setup of a specific LLM-as-a-Judge evaluator. It shows the complete template, version history, and associated job configurations.

• layout-widgets

  Use when calculating Langfuse dashboard widget grid positions. Trigger phrases: "layout widgets", "widget placement", "grid position", "arrange widgets", "widget positioning". Computes x/y coordinates and sizes for the 12-column dashboard grid.

• list-dataset-runs

  Use when the user wants to list dataset runs, browse experiment runs, see what experiments have been executed, check run status, or inspect run metadata. Trigger phrases include "list runs", "show runs", "experiment runs", "dataset runs", "what experiments have been run", "browse runs".

• list-evaluators

  This skill should be used when the user asks to list evaluators, see all evaluators, check what evaluators exist, or get an overview of LLM-as-a-Judge configurations in their Langfuse project. It queries eval_templates and job_configurations to produce a comprehensive summary.

• list-widgets

  This skill should be used when the user wants to list existing dashboard widgets in a Langfuse project, show widgets, see what widgets exist, get a widget inventory, or understand which widgets are deployed on which dashboards. Trigger phrases include "list widgets", "show widgets", "existing widget

• manage-dashboard

  This skill should be used when the user wants to perform CRUD operations on Langfuse dashboards. Trigger phrases include "create dashboard", "list dashboards", "manage dashboards", "delete dashboard", "add widget to dashboard", "remove widget from dashboard". It handles listing, creating, updating m

• manage-dataset-items

  This skill should be used when the user wants to create, update, archive, delete, list, or bulk-populate dataset items in Langfuse. Trigger phrases include "add items", "create item", "populate dataset", "update item", "archive item", "delete item", "bulk import", "add test cases".

• query-metrics

  This skill should be used when the user wants to query metrics from Langfuse, run a metrics query, aggregate data, hit the v1 metrics API, or retrieve analytics data from Langfuse. Trigger phrases include "query metrics", "run metrics query", "metrics API", "v1 metrics", and "aggregate data".

• suggest-widgets

  This skill should be used when the user wants recommendations for meaningful Langfuse dashboard visualizations based on available data. Trigger phrases include "suggest widgets", "recommend visualizations", "what should I visualize", "widget suggestions", "dashboard recommendations". It discovers av

• toggle-evaluator-status

  This skill should be used when the user wants to activate, deactivate, pause, resume, enable, or disable an LLM-as-a-Judge evaluator in Langfuse. Trigger phrases include "activate evaluator", "deactivate evaluator", "pause evaluator", "enable evaluator", "disable all evaluators". It handles safety c

• trigger-experiment

  This skill should be used when the user wants to trigger an experiment, run an experiment, start a dataset run, execute a benchmark, or send a webhook to start an experiment. Trigger phrases include "trigger experiment", "run experiment", "start experiment", "execute benchmark", "run dataset", "POST

• update-evaluator

  Use when updating an existing Langfuse LLM-as-a-Judge evaluator. Trigger phrases include "update evaluator", "modify evaluator", "change evaluator prompt", "update evaluation filters", and "change evaluator model". Handles template-level changes (new version) and job-config-level changes (in-place u

• update-widget

  Use when the user wants to update or modify an existing Langfuse dashboard widget configuration. Triggers: "update widget", "modify widget", "change widget", "edit widget configuration". It reads the current widget state, accepts changes conversationally, validates the new configuration, and applies

Subagents (5)

• langfuse-data-explorer

  Use this agent when the user wants to explore or discover what data exists in their Langfuse project. This includes listing scores, traces, models, tags, prompts, datasets, or querying metrics. This agent is read-only and never modifies data. <example> Context: User wants to understand what data is

• langfuse-dataset-expert

  Use this agent when the user wants to create, browse, populate, or manage Langfuse datasets and dataset items. This includes creating datasets, adding items, designing item schemas, archiving items, or exploring what datasets exist. This agent handles the data preparation layer for experiments. <exa

• langfuse-eval-manager

  Use this agent when the user wants to create, update, delete, list, inspect, or manage LLM-as-a-Judge evaluators in Langfuse. This agent operates directly on the Langfuse PostgreSQL database to manage eval_templates and job_configurations. <example> Context: User wants to see what evaluators exist.

• langfuse-experiment-manager

  Use this agent when the user wants to trigger experiments, browse dataset runs, analyze experiment results, compare runs, or configure remote experiment webhooks in Langfuse. This agent handles the experiment execution and analysis layer. <example> Context: User wants to trigger an experiment agains

• langfuse-widget-manager

  Use this agent when the user wants to create, update, delete, or manage Langfuse dashboard widgets and dashboards. This agent writes directly to the Langfuse PostgreSQL database. <example> Context: User wants to create a new visualization. user: "Create a chart showing average score over time" assis

Similar plugins