llm-d

Clears all KV / prefix cache tiers (GPU, CPU, and FS offload) on every vLLM pod in an llm-d deployment for a clean state, including both roles of a disaggregated prefill/decode setup. Use it to flush the cache, reset vLLM state, or start fresh before a test run. Prefers the /reset_prefix_cache API and falls back to pod restart. Acts on an already deployed stack; pairs with run-llm-d-benchmark for clean-cache benchmark runs.

8 skills

Compare the benchmark performance of two llm-d stack configurations end-to-end. For each configuration, deploys the stack, runs a benchmark, tears down, then generates a side-by-side results comparison. If one configuration was already benchmarked previously, only the new one is deployed and benchmarked, and its results are compared against the existing ones. Use it to A/B test llm-d configurations, compare against a baseline with no llm-d Inference Scheduler, compare models or serving strategies, evaluate the effect of a configuration change, or compare a new run against a previous one. Drives the deploy, benchmark, and teardown skills directly, so installing it also installs deploy-llm-d, run-llm-d-benchmark, and teardown-llm-d.

8 skills

Configure and deploy Workload Variant Autoscaler (WVA) for llm-d inference deployments. Guides users through namespace selection, WVA repo location, configuration (with presets or custom values), deployment via Makefile + kubectl apply, and verification. Produces a reusable deployment script.

8 skills

Create or prepare GKE infrastructure (cluster, GPU node pools, RDMA/RoCE networking, DRA and DRANET drivers, Gateway API prerequisites) so that an llm-d stack can be deployed on it. Use this skill when users want to provision a new GKE cluster for llm-d, or make an existing GKE cluster llm-d-ready, especially for prefill/decode disaggregation or wide expert parallelism which require RDMA. Stops at a cluster that is ready for llm-d: install deploy-llm-d (config already decided) or llm-d-autoconfig (config to be recommended) to deploy onto it.

8 skills

Configure and deploy an llm-d stack on an existing Kubernetes or OpenShift cluster by driving the machine-readable guide.yaml of a Well-Lit Path guide (llm-d v1.0 and later). Use this skill when users want to deploy, verify, or customize llm-d from a guide they have already chosen.

8 skills

Install every llm-d skill at once. Ships no skills of its own, it only pulls in the eight individual plugins, so it can be combined with them without duplicating anything.

8 skills

Configure an EPP EndpointPickerConfig for an llm-d deployment based on workload + SLA inputs, then optionally deploy it to the user's cluster. Activate when the user asks about configuring llm-d, setting up the inference scheduler, tuning EPP plugins, choosing what config to use for a model or workload, or deploying llm-d. Owns config generation and its own deploy path; install the single-purpose plugins (deploy-llm-d, run-llm-d-benchmark, teardown-llm-d, configure-wva-autoscaling-llm-d) for standalone operations on a config you already have.

8 skills

Benchmark an already-deployed llm-d stack using the llmdbenchmark CLI, either following a named guide (e.g. optimized-baseline, pd-disaggregation) or using a custom workload profile against any inference endpoint. Use it to measure performance (TTFT, throughput, ITL), validate a deployment under load, or run inference-perf/guidellm/vllm-benchmark. Expects an endpoint to already exist; install deploy-llm-d if you also need to stand one up.

8 skills

Tears down, removes, cleans up, or undeploys a deployed llm-d stack (llm-d v1.0 and later) on Kubernetes or OpenShift: removes the model servers (Deployments or DisaggregatedSets), uninstalls the llm-d router Helm release, and prefers the guide's own cleanup steps from guide.yaml. Handles partial cleanup (for example model servers only, keep the router). PVCs, the HuggingFace Secret, the Gateway, and the namespace are removed only on explicit request.

8 skills