modelopt

Plugin modelopt from marketplace modelopt in NVIDIA/Model-Optimizer. Commands read from the repo's manifest — today.

Skills (31)

• accessing-mlflow

  Compatibility name for modelopt-accessing-mlflow. Browse ModelOpt evaluation runs and artifacts in MLflow.

• benchmark-model-kernels

  Compatibility name for modelopt-benchmark-model-kernels. Inspect model shapes and benchmark GEMM or fused-MoE kernels.

• common

  Shared ModelOpt support files. Use only when another ModelOpt skill directs you here.

• compare-results

  Compatibility name for modelopt-compare-results. Compare baseline and quantized evaluation results.

• day0-release

  Compatibility name for modelopt-day0-release. Run the full ModelOpt day-0 quantization and validation workflow.

• debug

  Compatibility name for modelopt-debug. Run commands through the remote Docker debugger relay.

• deployment

  Compatibility name for modelopt-deployment. Serve model checkpoints with vLLM, SGLang, or TRT-LLM.

• evaluation

  Compatibility name for modelopt-evaluation. Evaluate model accuracy with NeMo Evaluator Launcher.

• launching-evals

  Compatibility name for modelopt-launching-evals. Run, monitor, and debug NeMo Evaluator Launcher jobs.

• modelopt-accessing-mlflow

  Query and browse evaluation results stored in MLflow. Use when the user wants to look up runs by invocation ID, compare metrics across models, fetch artifacts (configs, logs, results), or set up the MLflow MCP server. ALWAYS triggers on mentions of MLflow, experiment results, run comparison, invocat

• modelopt-benchmark-model-kernels

  Inspect Hugging Face decoder layers on meta tensors and plan or run per-rank BF16, FP8, and NVFP4 GEMM or fused-MoE microbenchmarks with the bundled scripts and a local FlashInfer checkout. Use when choosing a model, GPU, TP, EP, or M/token-concurrency sweep; deriving common fused QKV and gate/up sh

• modelopt-compare-results

  Establish baseline-vs-candidate evaluation plans, delegate missing evaluations, compare validated results, and decide quantization feasibility. Use when the user asks to compare baseline vs quantized runs, explain an accuracy drop/regression, verify whether a quantized checkpoint is acceptable, or c

• modelopt-day0-release

  Deterministic end-to-end driver for day-0 quantized-checkpoint releases — chains PTQ → evaluation → comparison with enforced gates between stages (the evaluation stage deploys the checkpoint itself), and returns a publish decision (ACCEPT / REGRESSION / INDETERMINATE / ANOMALOUS / INFEASIBLE). Use w

• modelopt-debug

  Run commands inside a remote Docker container via the file-based command relay (tools/debugger). Use when the user says "run in Docker", "run on GPU", "debug remotely", "run test in container", "check nvidia-smi", "run pytest in Docker", or needs to execute any command inside a Docker container that

• modelopt-deployment

  Serve a quantized or unquantized LLM checkpoint as an OpenAI-compatible API endpoint using vLLM, SGLang, or TRT-LLM. Use when user says "deploy model", "serve model", "start vLLM server", "launch SGLang", "TRT-LLM deploy", "AutoDeploy", "benchmark throughput", "serve checkpoint", or needs an inferen

• modelopt-evaluation

  Evaluates accuracy of quantized or unquantized LLMs using NeMo Evaluator Launcher (NEL). Triggers on "evaluate model", "benchmark accuracy", "run MMLU", "evaluate quantized model", "run nel". Handles deployment, config generation, and evaluation execution. Always load when validating completed runs

• modelopt-launching-evals

  Run, monitor, analyze, and debug LLM evaluations via nemo-evaluator-launcher. Covers running evaluations, checking status and live progress, debugging failed runs, exporting artifacts and logs, and analyzing results. ALWAYS triggers on mentions of running evaluations, checking progress, debugging fa

• modelopt-monitor

  Monitor submitted jobs (PTQ, evaluation, deployment) on SLURM clusters. Use when the user asks "check job status", "is my job done", "monitor my evaluation", "what's the status of the PTQ", "check on job" followed by a SLURM job ID, or after any skill submits a long-running job. Also triggers on "ne

• modelopt-ptq

  Use when the user asks to "quantize a model", "run PTQ", "post-training quantization", "NVFP4 quantization", "FP8 quantization", "INT8 quantization", "INT4 AWQ", "quantize LLM", "quantize MoE", "quantize VLM", or needs to produce a quantized HuggingFace checkpoint from a pretrained model using Model

• modelopt-qad

  Run explicitly requested ModelOpt Quantization-Aware Distillation (QAD) through Megatron Bridge (recommended) or Hugging Face Transformers to recover a measured BF16-to-PTQ accuracy gap. Use only when the user explicitly asks for QAD, including its topology, data preparation, launch, resume, checkpo

• modelopt-quant-recipe-search

  Use when the user asks to find, search for, or optimize the best quantization recipe for a model, including direct requests like "find the best quantization recipe and generate a PTQ checkpoint." Guides the multi-candidate loop: choose compute-vs-memory success metrics, select ModelOpt recipe baseli

• modelopt-release-cherry-pick

  Audit merged bug-fix PRs and release NVBugs for missing cherry-pick labels, then cherry-pick labeled PRs into a release branch and open a PR. Use when asked to "cherry-pick PRs for release/X.Y.Z", "pick PRs to release branch", "verify cherry-pick labels", or "cherry-pick labeled PRs".

• modelopt-run-agentx

  Run the AgentX agentic serving benchmark with the SemiAnalysis harness. Use for AgentX setup, dataset downloads, concurrency sweeps, or serving latency and throughput comparisons.

• modelopt-speculative-decoding

  Train, debug, and validate a speculative decoding draft model (EAGLE3, DFlash, DSpark, Domino) through the ModelOpt launcher pipeline. Use when the user wants to add a new model to a draft-training pipeline, asks why a pipeline run failed, wants experiment logs reviewed, or wants to check whether a

• monitor

  Compatibility name for modelopt-monitor. Monitor submitted ModelOpt jobs on SLURM.

• ptq

  Compatibility name for modelopt-ptq. Quantize a model checkpoint with ModelOpt PTQ.

• qad

  Compatibility name for modelopt-qad. Run explicitly requested ModelOpt quantization-aware distillation.

• quant-recipe-search

  Compatibility name for modelopt-quant-recipe-search. Search for a ModelOpt quantization recipe.

• release-cherry-pick

  Compatibility name for modelopt-release-cherry-pick. Audit and cherry-pick fixes for a ModelOpt release.

• run-agentx

  Compatibility name for modelopt-run-agentx. Run AgentX serving benchmarks with the SemiAnalysis harness.

• speculative-decoding

  Compatibility name for modelopt-speculative-decoding. Train, debug, and validate speculative decoding draft models.

Similar plugins