profile-and-optimize

Plugin profile-and-optimize from marketplace profile-and-optimize-plugins in cfregly/gpu-perf-tune. Commands read from the repo's manifest — 1 month ago.

Skills (32)

• analyze-zymtrace-workload

  Investigate a GPU or CPU workload through the zymtrace MCP. The MCP does most of the analysis. This skill enforces the cross-view discipline -- always pull the matching opposite-side flamegraph (CPU for GPU workloads, GPU for CPU workloads) with the same filter. Most bottlenecks hide on the side the

• evidence-bundle-init

  Scaffold a new evidence bundle directory ready for reproducibility-grade evidence capture: SOURCE.md (operator + cluster + git SHA + UTC timestamp), summary.md (verdict skeleton), commands/ (for the four-file .cmd/.stdout/.stderr/.exit tuple capture per shell command). Workload-agnostic. Works for a

• inference-aa-workload

  Reproduce the Artificial Analysis (AA) language-model performance workload shapes against an OpenAI-compatible chat endpoint using NVIDIA AIPerf. Drives the three AA text shapes (1k input / >=1k answer, 10k / >=1.5k, 100k / >=2k) with temperature 0, top_p 1, and the vLLM-style min_tokens + ignore_eo

• inference-capacity-sizing

  SLA-first GPU capacity sizing for a serving deployment: given a tokens-per-minute (TPM) target AND the interactivity SLA (output tokens/s/user), compute the pods and GPUs needed from a model's measured tok/s/user-vs-concurrency curve. Sizing MUST start from the tok/s/user SLA, not the TPM number alo

• inference-dcgm-correlate

  Correlate DCGM Prometheus byte-traffic counters with an inference-perf-bench sweep window to compute byte-grounded workload-level Speed-of-Light. The third tier of the SoL rigor hierarchy (after zymtrace sample-share and ncu per-kernel arithmetic intensity). Reads the sweep window from `inference_pe

• inference-decode-step-budget

  Measure the low-concurrency (c=1..c=8) decode hot-path of a live vLLM pod FAST and CORRECTLY: where each token's time actually goes (GPU-busy vs host-idle gap vs comm), whether the workload is kernel-bound / host-bound / comm-bound, and what the addressable lever is. Uses vLLM's native /start_profil

• inference-fleet-leaderboard

  Render cross-model fleet leaderboards from the local perf-report campaigns in ONE command (`perftunereport fleet_leaderboard`): a latency tier (aa-1k/10k/100k tok/s/user + TTFT + cost at c=1/c=10), a throughput tier (peak tok/s/GPU per (model,quant,TP) with latency + $/1M-tok), and a decision capsto

• inference-graph-diff

  Diff the compiled FX / Inductor graphs across two vLLM versions or two helm configs to see exactly which fused kernels / passes / partitions changed. Uses `torch._dynamo.explain` + `torch.compile`'s graph-dump hooks. Useful when you anticipate a model-graph-level change (e.g. swapping cudagraph_mode

• inference-kernel-ncu-profile

  Capture per-kernel CUDA hardware-counter data (occupancy, achieved warps active, regs/thread, smem/block, DRAM throughput, arithmetic intensity, branch divergence %, warp-stall reason %) from a live vLLM inference pod via an `ncu` debug sidecar (no production image rebuild). Outputs `.ncu-rep` + per

• inference-kernel-profile

  Capture per-kernel CUDA profile data from a live vLLM inference pod via an nsys debug sidecar (no production image rebuild). Outputs `.nsys-rep` + summary CSV + top-kernels table that joins with the zymtrace per-kernel breakdown and the inference-perf-bench bundle's `inference_perfbench_v1.json`. Tr

• inference-kernel-whitebox-debug

  White-box debug a custom CUDA/CUTLASS kernel producing a WRONG numeric result (over-amplification, NaN, coherence) after black-box bisection is EXHAUSTED: every external lever varied (grid, pipeline depth, layout, dtype), the result deterministic + lever-independent, yet the structure reads correct.

• inference-known-good-config

  Capture + enforce per-model KNOWN-GOOD serving configs: the REQUIRED serve flags (boot-blockers / crash-at-high-c / deploy-correctness workarounds) plus the champion + the bug each flag avoids, in one queryable registry (perf-tune-report/configs/known-good-configs.yaml) so a hard-won workaround (e.g

• inference-model-eval

  Drive lm-eval-harness quality evals (GPQA, MMLU-Pro) inside model pods plus optional ExternalEval (externally operated). Use to validate model quality before promoting to staging or prod, after vLLM / quantization / KV-cache changes, or to compare against published HuggingFace / paper baselines. Pai

• inference-model-optimize

  Coordinate an evidence-backed inference model optimization run from a user-supplied deployment or campaign configuration. Establishes readiness and a production-shaped baseline, profiles measured bottlenecks, runs a bounded tune sweep, checks quality, selects a champion, and renders a report. Use fo

• inference-perf-baseline-bridge

  Bridge between the inference-perf-bench / ai-bench output bundle and the profile-and-optimize perf-baseline registry. Knows the canonical inference perf metric shape (TTFT p50/p95/p99, ITL p50, throughput, tok/s/user, request latency p50, prefix cache hit rate delta, GPU cache utilization peak), rec

• inference-perf-bench

  Canonical inference perf-bench skill (formal name. The colloquial alias is `ai-bench` - identical behaviour). Drives NVIDIA AIPerf + the replay-playback dataset against an in-cluster vLLM endpoint to measure TTFT, ITL, throughput, tok/s/user, request latency, and prefix cache hit rate. Iterative 9-p

• inference-perf-synthesize

  Fuse the four profilers -- nsys (timeline + kernel durations), ncu (roofline / occupancy / arithmetic intensity), zymtrace (CPU+GPU flamegraph), and DCGM (byte-grounded %SoL) -- into ONE ranked, data-backed recommendation ledger for a vLLM deployment. Every recommendation (including concrete vLLM so

• inference-perf-tune-report

  Build a multi-page benchmark report PDF (scatter panels + per-concurrency heatmap tables) from `vllm bench sweep serve_workload` and/or AIPerf outputs, in the GLM-5.1 reference layout (5x2 scatter facet by max_num_batched_tokens + 3x2 heatmap tables for C={8,16,32} x {tok/s/GPU, TTFT}. First-class f

• inference-performance-hints

  Applies Jeff Dean and Sanjay Ghemawat's Performance Hints to GPU inference. Builds a back-of-the-envelope cost ledger, identifies the hot-path class, bounds each optimization with profile share, and routes the highest-value hypothesis to the repo's benchmark, profiling, tuning, quantization, or capa

• inference-quantize-calibrate

  Produce quantized inference weights from a BF16/FP8 base checkpoint via a post-training-quantization (PTQ) pipeline -- instead of only ever pulling NVFP4 weights pre-quantized. A calibration prep Job then a quantize Job: NVIDIA ModelOpt -> NVFP4 (default), or llm-compressor -> FP8 (alt). Writes weig

• inference-spec-decode-service

  Coordinate a measured speculative decoding workflow for a served model. Profile representative traffic, build a matched corpus, run an operator-provided draft trainer, compare the draft with the standing config, and promote only a measured win. Use for profile-matched speculative decoding work that

• inference-spec-decode-train

  Train + validate a speculative-decoding draft head (EAGLE3 or DFlash) for an ARBITRARY target LLM, generalizing the GLM-5.1-only SpecForge recipe so any new model gets one. Parameterizes what that offline recipe hard-codes: target path, aux-hidden-states layers (derived [1, L/2-1, L-4] from config.j

• inference-spec-decode-tune

  Run a bounded comparison of speculative decoding draft training settings. Use measured serving acceptance and latency as the decision metrics, with training accuracy only as a triage signal. Use when one draft recipe already trains correctly and the operator can provide reproducible train and evalua

• inference-tune-sweep

  Search a bounded inference serving configuration matrix, compare every cell with a named baseline, and select a throughput or latency champion. Uses the checked-in campaign, evidence, baseline, aggregation, and report tools. Use when asked to tune batching, concurrency, KV cache, CUDA graph, or rela

• inference-value-ledger

  Render the leadership value-prop ledger for the inference effort: the deployable wins vs the FlashInfer-TRTLLM + best-tuned-vLLM 0.21/0.22 baseline, grouped DONE / IN-PROGRESS / NOT-DONE / CLOSED-NEGATIVE, each row data-backed by a perf-lake campaign (live sol_rigor + verdict tier), plus the ranked

• inference-workload-profile

  Profile live inference traffic into a token/shape distribution artifact that drives profile-matched speculative-decoding draft training -- an analog of Fireworks FireOptimizer's "profile-driven customization" (the documented source of its higher draft hit-rate). Reads an OpenAI-style access JSONL, e

• k8s-troubleshooting

  Expert Kubernetes troubleshooting assistant for diagnosing and resolving issues across the full stack - pods, control plane, nodes, networking, storage, and underlay infrastructure - in GPU cloud environments. Triggers on any report of a broken, degraded, or mysterious Kubernetes issue: pod crashes,

• mirage-graph-coverage

  Read-only coverage auditor for the mirage / MPK persistent-megakernel's GENERATED task graph: cross-references DECLARED tensors (all_tensors[...] in kernel_N.cu) against tensors CONSUMED by tasks (inputs/outputs base_ptr in task_graph_N.json) and flags any tensor declared but never consumed as an in

• perf-baseline-diff

  Diff a current performance measurement against a registered baseline from the perf-baselines registry. Works for any measurement type (NCCL BW, MFU, step-time, latency, throughput, per-kernel times, structured heatmaps, nsys-rep profiles). Honors the operator-supplied tolerance, reports per-dimensio

• perf-baseline-record

  Capture any performance measurement (NCCL bandwidth, MFU, step-time, latency, throughput, kernel-time, anything operator-defined) into a versioned baseline registry under experiments/artifacts/perf-baselines/. Workload-agnostic. Records full provenance (operator, cluster, cohort, git SHA, capture co

• prometheus-anchored-query

  Reusable wrapper for the knowledge-base-first PromQL pattern. Operator names the metric / question / time range. Skill calls query_observability_knowledge_base first to confirm labels + cardinality, derives the safe narrow PromQL, runs it via the Prometheus MCP, and saves the raw payload to a proven

• zymtrace-anchored-query

  Reusable wrapper for the knowledge-base-first SQL pattern, adapted to the zymtrace ClickHouse profiling backend. Operator names the metric / question / time range. Skill anchors the `zymtrace_profiling.events` schema first (DESCRIBE + label-value + cardinality probes), derives the safe narrow SQL, r

MCP servers (1)

• profile_and_optimize

  runs: ${CLAUDE_PLUGIN_ROOT}/server/.venv/bin/python -m profile_and_optimize_mcp serve

Hooks (1)

• PreToolUse

  ${CLAUDE_PLUGIN_ROOT}/hooks/claude-hook-adapter.sh ${CLAUDE_PLUGIN_ROOT}/hooks/provenance-commit-gate.sh

Similar plugins