gpu-vllm-mi300x-31b-q4w4a16

Manage AWS Inferentia2 EC2 and Neuron vLLM inference.

2 skills · 1 MCP server

Manage Google Cloud TPU capacity and Gemma 4 serving from Rust on Compute Engine TPU VMs: flex-start provisioning, capacity probing, quotas, cost estimates, and SRE diagnostics via the gce-jaxrust-v6e1-2b MCP server. The engine is Rust — rlx's JAX-shaped IR lowered to HLO and executed through libtpu's PJRT plugin, no Python in the serving path. Forked from tpu-jax-v6e1-2b 2026-08-28; nothing provisioned or measured on this path yet.

1 skill · 1 MCP server

Compute Engine DevOps for the PyTorch/XLA Gemma 4 E2B rig: flex-start ct6e-standard-1t provisioning, capacity probing, quotas, cost estimates, and SRE diagnostics via the gce-pytorch-v6e1-2b MCP server. Serves google/gemma-4-E2B-it on one TPU v6e chip through public torch_xla (device "xla"), with the xprof + tensorboard profiling kit installed at boot. Forked from tpu-pytorch-v6e1-2b 2026-08-29 and migrated off the Cloud TPU API onto gcloud compute instances.

1 skill · 1 MCP server

EC2 G4dn DevOps for the pure-JAX Gemma 4 E2B rig: instance lifecycle, G-family quotas, x86_64 GPU DLAMI resolution, JAX-on-Turing verification, and SRE diagnostics via the gpu-jax-g4dn-2b MCP server. Serves google/gemma-4-E2B-it on an NVIDIA T4 (Turing, SM 7.5, 15360 MiB measured). Forked from gpu-jax-g5g-2b 2026-08-28 as the A/B control that separates Turing from Graviton2, and first served 2026-08-29: decode 13.1 tok/s against the T4G's 13.10, weight bytes identical to the byte, and the xprof profile reproducing 54.4% dtype conversion / 32.8% fp32 GEMV / 0.0% TensorCore — so the sibling's dtype tax is a chip finding, not a host one.

1 skill · 1 MCP server

EC2 G5g DevOps for the pure-JAX Gemma 4 E2B rig: instance lifecycle, G-family quotas, arm64 GPU DLAMI resolution, JAX-on-Turing verification, and SRE diagnostics via the gpu-jax-g5g-2b MCP server. Serves google/gemma-4-E2B-it on an NVIDIA T4G (Turing, SM 7.5) behind an AWS Graviton2 host — no build, no CUDA toolkit, no Triton patch, because jax[cuda12] publishes aarch64 wheels that already carry sm_75.

1 skill · 1 MCP server

EC2 G6 DevOps for the pure-JAX Gemma 4 E2B rig: instance lifecycle, G-family quotas, x86_64 GPU DLAMI resolution, JAX-on-Ada verification, and SRE diagnostics via the gpu-jax-g6-2b MCP server. Serves google/gemma-4-E2B-it on an NVIDIA L4 (Ada, SM 8.9, 23034 MiB). Forked from gpu-jax-g5g-2b 2026-08-28 to move the same engine off Turing onto a native bfloat16 datapath: measured the same day at 48.3-48.5 tok/s against the T4G's 12.80 at an identical config, with the 54% dtype-conversion tax at 0.0% and decode at the bandwidth roofline.

1 skill · 1 MCP server

AMD's verified Agent Skills in one plugin: route image/audio through local AI on Ryzen AI, serve LLMs on AMD Instinct GPUs with vLLM, optimize inference throughput with Hyperloom, and analyze GPU kernel and PyTorch trace performance.

21 skills

Production LLM inference including vLLM, TensorRT-LLM, llama.cpp, and SGLang. Use when deploying models for production inference.

98 skills

LLM serving benchmarks, SGLang model Day-0 support, profiling, code review, prod incident triage, PR-history dossiers, and SGLang explainer videos.

11 skills

Claude Code plugin for sparkrun - launch, manage, and stop LLM inference workloads on NVIDIA DGX Spark systems. Provides slash commands and skills for recipe browsing, cluster management, and multi-node orchestration.

3 skills · 8 commands

AMD's verified Agent Skills in one plugin: route image/audio through local AI on Ryzen AI, serve LLMs on AMD Instinct GPUs with vLLM, optimize inference throughput with Hyperloom, and analyze GPU kernel and PyTorch trace performance.

1 skill

Xilinx full-toolchain assistant covering Vivado, Vitis HLS, Vitis Unified IDE, and PetaLinux -- from HLS algorithm to boot image

1 skill