Experimentation

Plugin experimentation from marketplace agile-agentic-analytics in weisberg/agile_agentic_analytics. Commands read from the repo's manifest — yesterday.

Skills (12)

• advanced-experiment-analysis

  Analyze complex experiments whose design breaks ordinary two-proportion or t-test analysis: sequential testing, Bayesian decision rules, CUPED/CUPAC, ratio metrics, correlated or clustered observations, CATE, uplift modeling, and bandit-versus-controlled tradeoffs. Trigger when the analysis method i

• compliance-trust-review

  Review experiments for financial-services compliance, conduct risk, model risk, fairness, disclosures, advice-versus-marketing boundaries, dark patterns, trust erosion, auditability, and consumer perception. Trigger before customer-facing tests in regulated or high-trust contexts. For regulatory scr

• early-signal-monitoring

  Monitor early experiment health and temporal signal maturity: first-48-hours reads, sample-ratio- mismatch (SRM), exposure validation, operational guardrails, novelty and primacy effects, proxy signals, and early-versus-late evidence. Trigger when interpreting opening-day results. For the assignment

• email-incrementality

  Design or evaluate email experiments for true incremental lift: post-Apple-MPP measurement, holdout and control-group design, send-frequency and fatigue effects, proxy metrics, delayed outcomes, and financial-services email constraints. Trigger when the question is whether email caused the outcome —

• executive-evidence-brief

  Convert experiment evidence into a calibrated, board-ready executive decision memo — uncertainty, assumptions, and limitations intact — when an experiment result will justify launch, budget, compliance, or leadership action. Trigger for board-ready decision memos and leadership readouts built from e

• experiment-decision-review

  Review existing experiment evidence and recommend ship, kill, iterate, extend, retest, or escalate using decision thresholds beyond statistical significance — business materiality, risk, guardrails, and temporal validity. Trigger when a result exists and the call is contested: significant but small,

• experiment-operating-model

  Design the standing experimentation operating model — center of excellence, review board, decision rights, maturity curve, first-year roadmap, governance process, and earned-autonomy model. Trigger when the problem is repeatable organizational experimentation capability, not one test. For designing

• measurement-integration

  Reconcile experiment evidence with other measurement systems — MMM, attribution, geo-lift, global holdouts, and proxy calibration — when systems disagree and a budget or spend decision depends on the resolution. Trigger when experiment and non-experiment measurement conflict and need calibrating. Fo

• null-results-registry

  Capture null, flat, negative, and inconclusive experiment results into a durable experiment- learnings registry so failed and ambiguous tests compound into institutional memory instead of being silently rerun — repository schemas, tagging, learning summaries, and meta-analysis. Trigger after a flat,

• personalization-governance

  Decide whether personalization, segmentation, CATE, or uplift evidence is strong and safe enough to deploy in high-trust or regulated environments — the governance and deployment call, not the modeling. Trigger when a team wants to target, suppress, personalize, or automate based on subgroup experim

• power-duration-planning

  Plan experiment feasibility and program-level duration strategy in regulated or low-velocity settings: is a test feasible given scarce traffic, what MDE is realistic, is a default 30-day habit justified, and what low-velocity alternatives (validated proxies, sequential monitoring, non- experimental

• safe-experiment-design

  Design safe, governed experiments for regulated or high-trust settings: hypotheses, success metrics, guardrails, pre-registration, compliance-constrained and pre-registered plans, risk tiers, kill switches, and audit trails for a regulated journey. Trigger when a test touches investor-facing content

Subagents (9)

• ab-testing-expert

  End-to-end A/B and A/B/n design consultant operating inside regulated, high-trust experimentation. Use when the task is to design or shape a controlled test as a whole — hypothesis, OEC and guardrails, randomization unit, sample-size and MDE framing, decision rules, and a ship/kill/iterate recommend

• email-measurement-specialist

  Email experimentation and incrementality specialist for the post-Apple-MPP world: holdout and control design, send-frequency and fatigue tests, suppression tests, proxy metrics, delayed conversion, and financial-services email constraints. Use when the question is whether email *caused* an outcome a

• executive-brief-editor

  Executive communication specialist for experiment readouts: calibrated, uncertainty-aware decision memos, board-ready narratives, and senior-stakeholder summaries built from analyzed evidence. Use when a result must be communicated to leadership to justify launch, budget, compliance, or a strategic

• experiment-librarian

  Experiment repository and learning-system specialist for capturing null, negative, invalid, and inconclusive results so they compound into institutional memory. Use to design repository schemas, classify and tag results, extract durable lessons, prevent reruns of disproven hypotheses, and scope meta

• experimentation-statistician

  Statistical analysis and method-review specialist. Use to select, apply, or critique the statistical method behind an experiment — power and MDE math, confidence intervals, Bayesian posteriors, sequential testing, CUPED/CUPAC, ratio metrics, clustered data, CATE, uplift, and multiplicity — and to fl

• measurement-architect

  Measurement-architecture specialist for reconciling experiment evidence with MMM, multi-touch attribution, geo-lift, global holdouts, proxy calibration, and budget-decision governance. Use when measurement systems disagree, when an experiment result must calibrate a model or a spend decision, or whe

• operating-model-advisor

  Operating-model advisor for building repeatable experimentation capability: centers of excellence, governance bodies and review boards, decision rights, maturity curves, earned autonomy, training, tooling, and first-year roadmaps. Use when the problem is the *program* — how the organization runs exp

• regulated-experiment-auditor

  Independent, adversarial auditor of experiment design, implementation, analysis, and decision quality in regulated or high-trust settings. Use before launch (design audit), in-flight (SRM and telemetry health), at readout (validity and Twyman's-law checks), and pre-decision (does the recommendation

• regulated-risk-reviewer

  Compliance, fairness, conduct-risk, model-risk, disclosure, and trust reviewer for experiments in financial services and other high-trust environments. Use before a customer-facing test launches, when personalization or targeting could create disparate impact, or when an experiment touches advice, d

Similar plugins