tao-skills

Plugin tao-skills from marketplace tao-skill-bank in NVIDIA-TAO/tao-skill-bank. Commands read from the repo's manifest — 2 days ago.

Skills (82)

• tao-analyze-changenet-rca

  Performs deep Root Cause Analysis (RCA) on NVIDIA TAO Visual ChangeNet classification experiments with

• tao-finetune-huggingface-model

  Fine-tune any HuggingFace CV / VLM / LLM model on local NVIDIA GPUs inside an NGC PyTorch container when no dedicated TAO model skill matches. Use when the user wants to fine-tune a HuggingFace model (full or LoRA), train a vision / VLM / LLM model end-to-end, generate a reproducible HF training pip

• tao-port-huggingface-model

  Integrate a HuggingFace Computer Vision model into the NVIDIA TAO Toolkit ecosystem (tao-core config, tao-pytorch trainer, tao-deploy TensorRT pipeline). Use when the user asks to "integrate a HuggingFace model into TAO", "add an HF model to TAO Toolkit", "wire a HuggingFace ViT/DETR/ SegFormer into

• tao-run-automl-deft-pipeline

  Run the canonical NVIDIA AOI three-phase training pipeline — Phase 1 AutoML baseline (HPO), Phase 2 DEFT loop (RCA → SDG → mining → plain-train retrain), Phase 3 AutoML refinement on the DEFT-augmented dataset. Use when the user asks to "run the AOI workflow", "fine-tune my PCB AOI model end-to-end"

• tao-run-automl

  Run container-backed AutoML / hyperparameter optimization (HPO) for NVIDIA TAO networks using AutoMLRunner. Handles algorithm

• tao-run-deft-aoi-cosmos3

  Run the disk-backed DEFT AOI improvement loop for NVIDIA Cosmos Reason 3 / Cosmos3 models, using Nano by default and Edge or Super when explicitly requested: evaluate the base model on Proxy and frozen Benchmark splits, mine real image pairs from Proxy gaps, generate AnomalyGen synthetic NG pairs, a

• tao-run-deft-aoi

  Run the full DEFT AOI improvement loop for NVIDIA TAO VisualChangeNet / ChangeNet PCB inspection models: baseline evaluate, RCA, Cosmos AnomalyGen / AMP synthetic defects, k-NN mining, retraining, and deployment gating against a customer-defined primary metric and optional constraints. Use only when

• tao-run-deft-cr-its-mining

  Run the mining-based DEFT improvement workflow for ITS Cosmos-Reason binary video questions, focused on the non-reasoning classification/evaluation path. Use when the user asks for a DEFT CR ITS mining workflow, traffic-camera Cosmos Reason improvement loop, collision-identification workflow with da

• tao-run-deft-object-detection

  Run the full DEFT smart-data-augmentation loop for NVIDIA TAO Grounding DINO object detection: zero-shot baseline inference, KPI analysis, per-class gap analysis, SigLIP embedding of weak images, unique-neighbor mining against a source pool, ODVG dataset staging, and retraining — repeated for a fixe

• tao-run-deft-od-aoi

  Run a binary industrial-inspection DEFT loop with TAO RT-DETR: measure on frozen KPI/test roles, mine gap-similar real and clean images with SigLIP, accumulate admitted COCO data, retrain from one base checkpoint, and select by KPI AP50. Use for iterative AOI defect detection, not generic multiclass

• tao-run-deft-pas

  Run iterative improvement for NVIDIA TAO CLIP / SigLIP image-text retrieval on attribute-labelled data. Use when a request combines retrieval evaluation, weak-attribute or caption-pair mining, repeated retraining, and a stopping condition based on a retrieval KPI, validation plateau, or iteration bu

• tao-run-dinov3-ssl-deft

  Run iterative DINOv3 SSL DEFT data selection and retraining from an unlabeled pool, using GRIT or multi-task weakness and fixed-encoder retrieval. Use for active-learning-style DINOv3 mining, not plain pretraining/fine-tuning (tao-train-dinov3) or AOI/PAS DEFT.

• tao-run-inference-service

  Start, query, and stop a network-specific TAO inference microservice ({network_arch}-inference-microservice) by delegating container execution to the appropriate platform skill. Handles container image resolution, job-payload JSON construction, and the service registry. Use when the user wants to ru

• tao-train-single-step

  Standard single-step train/eval/export workflow for any TAO model. Use when training a TAO model on a dataset

• tao-validate-recipe-transfer

  Port a published computer vision paper's official code and training recipe onto a customer's own dataset, or diagnose why such a transfer produced bad numbers. Use this whenever someone wants to reproduce a CV paper, run a paper's repo on their own images, fine-tune a published detection/segmentatio

• tao-artifacts

  The contract home for TAO's SDK-free execution pipeline — authoritative JSON Schemas for the four typed artifacts (spec-bundle, job-record, results_dir layout, best_rec) plus the fixed job-status vocabulary and the nested-not-dotted spec rule. Use when authoring or validating a spec-bundle before su

• tao-launch-workflow

  The mandatory pre-launch gate and four-verb execution contract for every TAO workflow or action. Invoke BEFORE launching anything side-effecting — AutoML, train, evaluate, inference, export, TensorRT engine generation, or DEFT/application workflows — on any execution platform. Covers platform select

• tao-list-capabilities

  Answer what the TAO Skill Bank plugin can do by generating the response from packaged application, data, model, AutoML, and platform manifests. Use when the user asks "what can TAO Skill Bank do", "list TAO models", "which TAO workflows are available", or "what supports AutoML". author: NVIDIA Corpo

• tao-setup

  One-time session setup and orchestration map for the TAO skill bank. Run this first when the TAO skills were installed individually (e.g. from a public skills catalog) so the session gets the cross-skill discovery flow, credential checks, and host preflight that the bundled plugin hook would otherwi

• tao-token-efficient-execution

  Token-efficient execution kit for repeated TAO agentic workflows. Compiles a skill into stage cards once (with a strong model), then executes them forever in fresh, small headless sessions coordinated by a bash driver — measured 74-91% token reduction and 4-5x lower peak context vs one long conversa

• paidf-cosmos-predict

  Prepare and run PAIDF Cosmos Predict video generation for DEFT media samples. author: NVIDIA Corporation version: "0.1.0

• tao-analyze-detection-kpi

  Run TAO Data Services KPI analysis for object detection, comparing inference annotations against ground truth to compute per-class TP/FP/FN/TN, precision, recall, accuracy, and AP at a fixed IoU. Use when an object detection workflow needs per-class mAP reported after inference, or when the user ask

• tao-analyze-gaps-od-map

  Run TAO Data Services object-detection gap analysis from ground-truth and inference annotations. Use when an object detection workflow needs to identify weak images by comparing model predictions against ground truth using per-class recall, precision, and AP50 thresholds. Use when the user asks to "

• tao-analyze-gaps-visual-changenet

  Performs gap analysis on NVIDIA TAO VCN Classify (Visual Component Net) experiments by invoking the pinned TAO data-services container directly via `docker run … gap_analysis vcn_aoi …` — picks the optimal decision threshold, ranks per-sample weakness, and emits a top-K weakest parquet expanded per-

• tao-analyze-gaps-vlm-bcq

  Extract false-positive and false-negative gaps from VLM binary-classification-question (BCQ, yes/no) predictions.

• tao-convert-dataset-format

  Run `tao-daft convert` to convert NVIDIA TAO DAFT datasets between supported formats. Do not use for non-DAFT data.

• tao-finetune-anomalygennext

  Validate a user AnomalyGenNext dataset and recipe, freeze its validation set, and prepare a canonical texture fine-tuning recipe. Use when AnomalyGenNext task weights do not yet exist. Do not use for ordinary defect generation. author: NVIDIA Corporation version: "0.1.0

• tao-generate-anomalies

  Full PAIDF AnomalyGen pipeline — fine-tune on a new anomaly dataset, generate synthetic anomaly images (SDG), evaluate quality (nn_score), and search per-sample (guidance, crop_ratio) parameters. Three modes: full (Phase 0→7: finetune then generate), finetune_only (Phase 0→1: train only), inference_

• tao-generate-image-embeddings

  Run TAO Data Services image embedding to turn a parquet of image filepaths into an embedding parquet using CLIP, SigLIP, or a TAO checkpoint. Use when a workflow needs embeddings before nearest-neighbor or unique-neighbor mining, or when the user asks to "embed images", "compute image embeddings", o

• tao-generate-image-grounding

  Two-step image grounding pipeline: extracts referring expressions from (image, caption) pairs and grounds them

• tao-generate-od-defects

  Run AnomalyGenNext 1.1 inference from a native testcase or a prepared generation plan, then publish validated fine-grained and binary COCO labels. Use for synthetic object-detection defect generation, not model training. author: NVIDIA Corporation version: "0.1.0

• tao-generate-referring-expressions

  Four-step image referring-expression pipeline: turns images plus KITTI bounding-box labels into region

• tao-generate-video-reasoning-annotations

  Multi-step video annotation pipeline that turns raw videos into Chain-of-Thought training data — multi-level captions, structured descriptions, and QA pairs (MCQ, binary, open-ended) with reasoning traces, via VLM/LLM distillation. Use when the user wants to "create video training data", "generate v

• tao-mine-aoi-images

  Runs the DEFT embed-then-mine workflow for VCN AOI iterations — embeds the gap-analysis target parquet, embeds a source pool, and mines nearest-neighbour source images for downstream augmentation. Use as the immediate next step after `tao-route-visual-changenet-samples` when expanding a real-image a

• tao-mine-nearest-neighbors

  Run TAO Data Services TMM nearest-neighbor mining from embedding parquet files. Use when a workflow needs to mine source samples closest to target samples. author: NVIDIA Corporation version: "0.1.0

• tao-mine-od-images

  Run TAO Data Services TMM unique-neighbor matching mining from embedding parquet files for object detection workflows. Use when an object detection workflow needs to mine a bijectively-assigned set of unique source images closest to target samples. Use global allocation when mining without class con

• tao-prepare-anomalygennext-inputs

  Prepare normalized object-detection false negatives, source masks, and clean-image embedding inputs for AnomalyGenNext. Use when turning box-level FN gaps into a frozen, pair-preserving preparation plan. Do not use for training or generation. author: NVIDIA Corporation version: "0.1.0

• tao-route-visual-changenet-samples

  Routes the weakest VCN samples (output of `tao-analyze-gaps-visual-changenet`) into per-augmentation-module

• tao-validate-dataset-format

  Run `tao-daft validate` to check NVIDIA TAO DAFT datasets for structure, schema, and cross-reference errors. Do

• tao-finetune-clip

  CLIP vision-language model for image-text retrieval, zero-shot classification, embedding extraction, ONNX

• tao-finetune-cosmos-embed

  Cosmos-Embed1 video-text embedding for text-to-video retrieval, video-to-video search, semantic deduplication, and LoRA fine-tuning. Use when the user asks to "fine-tune Cosmos-Embed1", "LoRA fine-tune Cosmos-Embed1 on PhysicalAI-Event-Videos", "train Cosmos-Embed1 on vad_r1_chunks manifests", "scor

• tao-finetune-cosmos-reason

  Shared Cosmos3 frontend that explicitly routes Cosmos Framework and Cosmos-RL, validates runtime model/video-dataset/SLURM inputs, consumes an SQSH or packaged backend image, optionally plans explicit clean source builds, prepares checkpoints, validates the first update in-process, and returns token

• tao-finetune-kumo-anomaly

  Kumo-Anomaly — diffusion-based anomaly detection and fine-tuning for multivariate time series. Use when the user asks to "fine-tune Kumo-Anomaly", "run Kumo-Anomaly inference", "detect anomalies with diffusion", "time series anomaly detection", "finetune Kumo-Anomaly", "use perform_anomaly_analysis_

• tao-finetune-kumo-forecast

  Kumo-Forecast — transformer-based multivariate time series forecasting with DARR (context-enhanced kNN retrieval), interpretability, and fine-tuning. Use when the user asks to "forecast with Kumo-Forecast", "run forecasting inference", "use perform_forecasting", "DARR mode", "context-enhanced foreca

• tao-finetune-video-clip

  InternVideo2-CLIP L14 (TAO video_clip) for video-text retrieval, zero-shot classification, embedding extraction, LoRA fine-tuning, ONNX export, and TensorRT deployment. Use when the user asks to "fine-tune IV2CLIP", "run video_clip train/evaluate/inference/export", "build a Video-CLIP TensorRT engin

• tao-train-action-recognition

  Action recognition from video sequences. Supports RGB, optical flow, and joint (multi-stream) input types for

• tao-train-bevfusion

  BEVFusion for multi-sensor 3D object detection. Fuses LiDAR point clouds and camera images in bird's-eye-view

• tao-train-centerpose

  CenterPose for keypoint / pose estimation. Detects object centers and regresses keypoint locations for 6-DoF

• tao-train-codetr

  Co-DETR (CoDINO) for object detection. A DETR-family detector with collaborative hybrid

• tao-train-deformable-detr

  Deformable DETR for 2D object detection. Uses deformable attention for efficient multi-scale feature processing,

• tao-train-depth-anything-v2

  Monocular depth estimation using Metric Depth Anything v2 or Relative Depth Anything architectures. Predicts

• tao-train-dino

  DINO (DETR with Improved DeNoising Anchor Boxes) for 2D object detection. Transformer-based detector with

• tao-train-dinov3

  DINOv3 continual self-supervised pre-training. Domain-adapts public DINOv3 ViT backbones

• tao-train-fast-foundation-stereo

  Real-time stereo depth estimation using FastFoundationStereo (FFS), the distilled bp2 commercial variant of

• tao-train-foundation-stereo

  Stereo depth estimation using FoundationStereo. Predicts disparity maps from stereo image pairs for 3D

• tao-train-grounding-dino

  Grounding DINO for open-set object detection. Combines DINO-style detection with a BERT text encoder for

• tao-train-image-classification

  PyTorch-based TAO image classification. Supports a wide range of backbones (FAN, EfficientNet, ResNet, etc.)

• tao-train-mask-auto-encoder

  Masked Auto-Encoder (MAE) for self-supervised pretraining and fine-tuning. Masks random patches and reconstructs

• tao-train-mask-auto-label

  MAL (Mask Auto-Label) for weakly-supervised segmentation. Produces segmentation masks from minimal annotations

• tao-train-mask-grounding-dino

  Mask Grounding DINO for grounded instance segmentation. Extends Grounding DINO with a mask-prediction head for

• tao-train-mask2former

  Mask2Former for universal image segmentation (panoptic, instance, and semantic). Transformer-based with

• tao-train-metric-learning-recognition

  Metric-learning recognition (ml-recog) for fine-grained visual recognition. Learns embeddings for

• tao-train-nvdinov2

  NVDINOv2 for self-supervised visual representation learning. Trains vision transformers via self-distillation

• tao-train-nvpanoptix3d

  NVPanoptix3D for panoptic 3D scene reconstruction from posed RGB images. Produces 3D panoptic segmentation

• tao-train-ocdnet

  OCDNet for scene text detection. Detects arbitrary-oriented text regions in natural images using a

• tao-train-ocrnet

  OCRNet for scene text recognition. Recognizes text content from cropped text-region images and supports CTC

• tao-train-oneformer

  OneFormer for universal image segmentation. Unifies panoptic, instance, and semantic segmentation with a

• tao-train-optical-inspection

  Optical Inspection for defect detection using Siamese networks. Compares image pairs to detect manufacturing

• tao-train-pointpillars

  PointPillars for 3D object detection from LiDAR point clouds. Encodes point clouds into a pseudo-image via a

• tao-train-pose-classification

  Pose classification using ST-GCN (Spatial Temporal Graph Convolutional Network). Classifies skeleton sequences

• tao-train-reid

  Person re-identification (ReID). Learns discriminative embeddings to match the same person across different

• tao-train-rtdetr

  RT-DETR (Real-Time DEtection TRansformer) for 2D object detection. Designed for real-time inference with

• tao-train-segformer

  SegFormer for semantic segmentation. Lightweight transformer-based architecture with hierarchical feature

• tao-train-sparse4d

  Prepare calibrated multi-camera data and train, evaluate, run inference, export, or quantize TAO Sparse4D 3D detection and tracking models. Includes 2D-to-3D geometric distillation with released LTT weights. Use for Sparse4D adaptation and temporal multi-camera perception workflows.

• tao-train-visual-changenet

  Visual ChangeNet for binary image classification and segmentation in AOI defect detection. Use when training,

• tao-data-io

  The data-mover for TAO jobs — decides the storage tier (A pre-positioned mount with zero fetch / B volume-from-S3 / C ephemeral in-compute fetch), stages inputs (bulk + annotation-selective + archive extract + HF/NGC PTM), maps credentials to env, routes outputs 3-way with upload-excludes, and runs

• tao-run-on-brev

  Run a TAO training/evaluation/inference container on an NVIDIA Brev GPU instance. Instance provisioning (create/search/stop/delete/login) is delegated to the official brev-cli agent skill or the Brev MCP server; this skill covers only the TAO-specific part — running the container over `brev exec` vi

• tao-run-on-docker

  The Docker execution platform for TAO jobs — a local daemon or a remote GPU box via DOCKER_HOST=ssh://user@host. Implements the four-verb consumer contract (submit/status/logs/cancel) over the docker CLI, wired to the job-record, tao-data-io staging, and the redact lint, on top of the underlying doc

• tao-run-on-kubernetes

  Kubernetes execution platform — submits TAO container jobs as k8s Jobs with NVIDIA GPU scheduling; single-pod

• tao-run-on-slurm

  Remote SLURM GPU cluster execution over SSH with sbatch/srun, Pyxis/Enroot containers, and Lustre-backed

• tao-run-on-virtualenv
• tao-setup-nvidia-gpu-host

Hooks (1)

• SessionStart

  ${CLAUDE_PLUGIN_ROOT}/hooks/session_start.sh