Huawei Ascend NPU server model quantization tool skills, using msmodelslim for quantization compression, the tool is specifically designed for LLM quantization acceleration on Ascend NPU cards. It supports INT4/INT8/INT16 quantization and provides a complete quantization workflow, including environment checking, dynamic scheme query, container deployment, quantization execution, etc., and supports quantization tuning, adapter adaptation, etc. Interactive experience with controllable processes.
Guides migration of traditional models from CPU/GPU to Ascend NPU. Invoke when user needs to adapt PyTorch code for Ascend devices, set up NPU environment, or verify migration correctness.
Run model inference on FiftyOne datasets using Zoo models or custom models. Use when applying ML models for detection, classification, segmentation, or embeddings on existing datasets.
Inference KV-cache and transport suite — LMCache multiprocess (MP) standalone-server mode (DaemonSet + Deployment K8s pattern, ZMQ connector, L1 + L2 NIXL/POSIX/GDS/HF3FS/fs/s3/mooncake adapters) and NVIDIA NIXL transfer library (UCX/GDS/Mooncake/libfabric/HF3FS/S3 plugins, agent API, telemetry) used by Dynamo/vLLM/SGLang. Pairs with the vllm-caching skill in the `vllm` suite and sglang-hicache in the `sglang` suite.