gemma4-dev

Manage AWS Inferentia2 EC2 and Neuron vLLM inference.

2 skills · 1 MCP server

Manage Google Cloud TPU capacity and Gemma 4 serving from Rust on Compute Engine TPU VMs: flex-start provisioning, capacity probing, quotas, cost estimates, and SRE diagnostics via the gce-jaxrust-v6e1-2b MCP server. The engine is Rust — rlx's JAX-shaped IR lowered to HLO and executed through libtpu's PJRT plugin, no Python in the serving path. Forked from tpu-jax-v6e1-2b 2026-08-28; nothing provisioned or measured on this path yet.

1 skill · 1 MCP server

Compute Engine DevOps for the PyTorch/XLA Gemma 4 E2B rig: flex-start ct6e-standard-1t provisioning, capacity probing, quotas, cost estimates, and SRE diagnostics via the gce-pytorch-v6e1-2b MCP server. Serves google/gemma-4-E2B-it on one TPU v6e chip through public torch_xla (device "xla"), with the xprof + tensorboard profiling kit installed at boot. Forked from tpu-pytorch-v6e1-2b 2026-08-29 and migrated off the Cloud TPU API onto gcloud compute instances.

1 skill · 1 MCP server

EC2 G4dn DevOps for the pure-JAX Gemma 4 E2B rig: instance lifecycle, G-family quotas, x86_64 GPU DLAMI resolution, JAX-on-Turing verification, and SRE diagnostics via the gpu-jax-g4dn-2b MCP server. Serves google/gemma-4-E2B-it on an NVIDIA T4 (Turing, SM 7.5, 15360 MiB measured). Forked from gpu-jax-g5g-2b 2026-08-28 as the A/B control that separates Turing from Graviton2, and first served 2026-08-29: decode 13.1 tok/s against the T4G's 13.10, weight bytes identical to the byte, and the xprof profile reproducing 54.4% dtype conversion / 32.8% fp32 GEMV / 0.0% TensorCore — so the sibling's dtype tax is a chip finding, not a host one.

1 skill · 1 MCP server

EC2 G5g DevOps for the pure-JAX Gemma 4 E2B rig: instance lifecycle, G-family quotas, arm64 GPU DLAMI resolution, JAX-on-Turing verification, and SRE diagnostics via the gpu-jax-g5g-2b MCP server. Serves google/gemma-4-E2B-it on an NVIDIA T4G (Turing, SM 7.5) behind an AWS Graviton2 host — no build, no CUDA toolkit, no Triton patch, because jax[cuda12] publishes aarch64 wheels that already carry sm_75.

1 skill · 1 MCP server

EC2 G6 DevOps for the pure-JAX Gemma 4 E2B rig: instance lifecycle, G-family quotas, x86_64 GPU DLAMI resolution, JAX-on-Ada verification, and SRE diagnostics via the gpu-jax-g6-2b MCP server. Serves google/gemma-4-E2B-it on an NVIDIA L4 (Ada, SM 8.9, 23034 MiB). Forked from gpu-jax-g5g-2b 2026-08-28 to move the same engine off Turing onto a native bfloat16 datapath: measured the same day at 48.3-48.5 tok/s against the T4G's 12.80 at an identical config, with the 54% dtype-conversion tax at 0.0% and decode at the bandwidth roofline.

1 skill · 1 MCP server

EC2 G5g DevOps for the Rust-driven-XLA Gemma 4 E2B rig: instance lifecycle, G-family quotas, arm64 GPU DLAMI resolution, Turing verification, and SRE diagnostics via the gpu-jaxrust-g5g-2b MCP server. Forked from gpu-jax-g5g-2b 2026-08-28 to move the engine off CPython onto Rust. The control plane is complete and measured nothing yet — the Rust engine is not written, and the rig currently runs the inherited Python JAX engine as its fallback and parity oracle.

1 skill · 1 MCP server

EC2 G5g DevOps for the llama.cpp Gemma 4 E2B rig serving the QAT Q4_0 GGUF: instance lifecycle, G-family quotas, arm64 GPU DLAMI resolution, from-source llama.cpp build pinned to CMAKE_CUDA_ARCHITECTURES=75, and SRE diagnostics via the gpu-llamacpp-g5g-2b-q4_0 MCP server. Serves google/gemma-4-E2B-it-qat-q4_0-gguf on an NVIDIA T4G (Turing, SM 7.5) behind a Graviton2 host. Scaffolded 2026-09-02 and HAS SERVED NOTHING. The only rig here serving 4-bit weights on a GPU: vLLM 0.26.0 has no gguf module on any platform, no GGUF reader exists in JAX, and transformers dequantizes to fp32 while silently dropping 35 layer_scalar tensors. It compiles because llama.cpp publishes no prebuilt Linux aarch64 CUDA binary — the asymmetry against its gpu-ollama-g5g-2b-q4_0 sibling, which ships one with native sm_75 SASS.

1 skill · 1 MCP server

EC2 G5g DevOps for the Ollama Gemma 4 E2B rig serving the QAT Q4_0 weights: instance lifecycle, G-family quotas, arm64 GPU DLAMI resolution, Ollama bundle install with the CUDA variant pinned to cuda_v12, and SRE diagnostics via the gpu-ollama-g5g-2b-q4_0 MCP server. Serves gemma4:e2b-it-qat on an NVIDIA T4G (Turing, SM 7.5) behind a Graviton2 host. Created 2026-09-02 and HAS SERVED NOTHING. Shares an engine with gpu-llamacpp-g5g-2b-q4_0 — Ollama links libllama.so — so slot 2 names the front end, not the decoder; the pair exists for the four differences a benchmark can still see. No build step: Ollama ships an aarch64 CUDA bundle with native sm_75 SASS where llama.cpp publishes none. Its failure mode is a CPU-resident model, which serves correctly and slowly and logs nothing, so the rig asserts on size_vram from /api/ps.

1 skill · 1 MCP server

EC2 G4dn DevOps for the PyTorch/transformers Gemma 4 E2B rig: instance lifecycle, G-family quotas, x86_64 PyTorch DLAMI resolution, torch-on-Turing verification, and SRE diagnostics via the gpu-pytorch-g4dn-2b MCP server. Serves google/gemma-4-E2B-it on an NVIDIA T4 (Turing, SM 7.5, 15360 MiB, 14.07 GiB usable). Forked from gpu-pytorch-g5g-2b 2026-08-29 as the fourth corner of the {jax, pytorch} x {g5g, g4dn} square. HAS SERVED NOTHING. Torch comes from the AMI rather than pip — the bootstrap installs into the DLAMI's own venv — so the image is a PyTorch DLAMI (2.13 / Ubuntu 26.04) and not the base driver image the JAX rigs use. Now that gpu-jax-g4dn-2b has closed the host axis, this is the runtime A/B on identical hardware: is the 86.9% dtype-plus-fp32-GEMV tax Turing, or XLA?

1 skill · 1 MCP server

Manage AWS EC2 G5g (Graviton2 + NVIDIA T4G) instances and pure-JAX Gemma 4 E2B serving.

1 skill · 1 MCP server

EC2 G6 DevOps for the stock-PyTorch Gemma 4 E2B rig: instance lifecycle, G-family quotas, x86_64 PyTorch DLAMI resolution, torch-on-Ada verification, and SRE diagnostics via the gpu-pytorch-g6-2b MCP server. Serves google/gemma-4-E2B-it on an NVIDIA L4 (Ada, SM 8.9, 23034 MiB) through HF transformers -- no custom port, no XLA, no build. Retargeted from gpu-pytorch-g5g-2b 2026-08-29 and measured the same day at 20.93 tok/s, a clean 1.92x over the T4G at byte-identical weights. Sits at 71% of the bandwidth roofline where the JAX rig on the same chip reaches 99%: the gap is eager-mode kernel-launch overhead, and batching is untried.

1 skill · 1 MCP server

EC2 G6 DevOps for Gemma 4 E2B with Google's QAT 4-bit weights (xbill9/gemma-4-E2B-it-qat-q4_0-w4a16-ct-text) on an NVIDIA L4, served by the hand-built PyTorch int4 engine of local-pytorch-1650ti-2b-q4w4a16 (Triton int4 GEMV, CUDA-graph decode). Instance lifecycle, PyTorch DLAMI resolution, pinned runtime install, payload deploy over SSM and SRE diagnostics via the gpu-pytorch-g6-2b-q4w4a16 MCP server; provisioning forked from gpu-pytorch-g6-2b. Created 2026-10-26; not yet measured.

1 skill · 1 MCP server

Manage AWS EC2 G4dn (x86_64 + NVIDIA T4, Turing SM 7.5) instances and vLLM inference for Gemma 4 E2B.

1 skill · 1 MCP server

Manage AWS EC2 G4dn (x86_64 + NVIDIA T4, Turing SM 7.5) instances and vLLM inference for Gemma 4 E2B.

1 skill · 1 MCP server

EC2 G5g DevOps for the vLLM Gemma 4 E2B rig: instance lifecycle, G-family quotas, arm64 AMI resolution, SM 7.5 build orchestration, and SRE diagnostics via the gpu-vllm-g5g-2b MCP server. Serves google/gemma-4-E2B-it on an NVIDIA T4G (Turing) behind an AWS Graviton2 host — the one rig here whose GPU has no prebuilt aarch64 CUDA kernels.

1 skill · 1 MCP server

EC2 G6 DevOps for vLLM 0.30.0 serving google/gemma-4-12B-it-qat-w4a16-ct (Gemma 4 12B) on one NVIDIA L4 (Ada, SM 8.9): instance lifecycle, G-family quotas, x86_64 GPU DLAMI resolution and SRE diagnostics via the gpu-vllm-g6-12b-w4a16 MCP server. The raw-EC2 counterpart of a sagemaker-gemma SageMaker run, with matching vLLM version and settings. Forked from gpu-vllm-g6-2b 2026-09-29. Has served nothing.

1 skill · 1 MCP server

EC2 G6 DevOps for vLLM 0.30.0 serving xbill9/gemma-4-26B-A4B-it-qat-q4_0-w4a16-ct (Gemma 4 26B A4B) on one NVIDIA L4 (Ada, SM 8.9): instance lifecycle, G-family quotas, x86_64 GPU DLAMI resolution and SRE diagnostics via the gpu-vllm-g6-26b-q4w4a16 MCP server. The raw-EC2 counterpart of a sagemaker-gemma SageMaker run, with matching vLLM version and settings. Forked from gpu-vllm-g6-2b 2026-09-29. Has served nothing.

1 skill · 1 MCP server

EC2 G6 DevOps for the vLLM Gemma 4 E2B rig: instance lifecycle, G-family quotas, x86_64 GPU DLAMI resolution, SM 8.9 verification, and SRE diagnostics via the gpu-vllm-g6-2b MCP server. Serves google/gemma-4-E2B-it on an NVIDIA L4 (Ada, SM 8.9). Forked from gpu-vllm-g5g-2b 2026-08-28, and the fork REMOVES that rig's reason for existing: G6 is x86_64 and SM 8.9, both carried by the published amd64 image, so there is no from-source build, no CUDA toolkit, no Rust and no Triton patch. Exists as the runtime control for gpu-jax-g6-2b on identical silicon. Served 2026-08-30; settings aligned with the sagemaker-gemma runs 2026-09-29.

1 skill · 1 MCP server

EC2 G6 DevOps for vLLM 0.30.0 serving google/gemma-4-E2B-it-qat-w4a16-ct (Gemma 4 E2B) on one NVIDIA L4 (Ada, SM 8.9): instance lifecycle, G-family quotas, x86_64 GPU DLAMI resolution and SRE diagnostics via the gpu-vllm-g6-2b-w4a16 MCP server. The raw-EC2 counterpart of a sagemaker-gemma SageMaker run, with matching vLLM version and settings. Forked from gpu-vllm-g6-2b 2026-09-29. Has served nothing.

1 skill · 1 MCP server

EC2 G6 DevOps for vLLM 0.30.0 serving google/gemma-4-31B-it-qat-w4a16-ct (Gemma 4 31B) on one NVIDIA L4 (Ada, SM 8.9): instance lifecycle, G-family quotas, x86_64 GPU DLAMI resolution and SRE diagnostics via the gpu-vllm-g6-31b-w4a16 MCP server. The raw-EC2 counterpart of a sagemaker-gemma SageMaker run, with matching vLLM version and settings. Forked from gpu-vllm-g6-2b 2026-09-29. Has served nothing.

1 skill · 1 MCP server

EC2 G6 DevOps for vLLM 0.30.0 serving google/gemma-4-E4B-it (Gemma 4 E4B) on one NVIDIA L4 (Ada, SM 8.9): instance lifecycle, G-family quotas, x86_64 GPU DLAMI resolution and SRE diagnostics via the gpu-vllm-g6-4b MCP server. The raw-EC2 counterpart of a sagemaker-gemma SageMaker run, with matching vLLM version and settings. Forked from gpu-vllm-g6-2b 2026-09-29. Has served nothing.

1 skill · 1 MCP server

EC2 G6 DevOps for vLLM 0.30.0 serving google/gemma-4-E4B-it-qat-w4a16-ct (Gemma 4 E4B) on one NVIDIA L4 (Ada, SM 8.9): instance lifecycle, G-family quotas, x86_64 GPU DLAMI resolution and SRE diagnostics via the gpu-vllm-g6-4b-w4a16 MCP server. The raw-EC2 counterpart of a sagemaker-gemma SageMaker run, with matching vLLM version and settings. Forked from gpu-vllm-g6-2b 2026-09-29. Has served nothing.

1 skill · 1 MCP server

AMD MI300X DevOps for the vLLM Gemma 4 E2B rig: DigitalOcean droplet lifecycle (no create/destroy, by design), rocm-smi diagnostics that parse output rather than trust an exit code, image vetting that catches the newest vendor build being unable to load Gemma 4 at all, serving deployment with the gemma4 reasoning and tool-call parsers, and text/thinking/tool-calling/vision verification via the gpu-vllm-mi300x-2b MCP server. Serves google/gemma-4-E2B-it on one Instinct MI300X (gfx942, 191.69 GiB) reached through AMD Developer Cloud. The first AMD rig in this monorepo; added 2026-09-16.

1 skill · 1 MCP server

AMD MI300X DevOps for one arm of the E2B data-type sweep: xbill9/gemma-4-E2B-it-qat-q4_0-fp8-text (fp8, text only) on one Instinct MI300X via a DigitalOcean droplet. Forked from gpu-vllm-mi300x-2b-q4w4a16 2026-10-07; sweep plan in gpu-vllm-mi300x-2b/DTYPE-SWEEP.md. Has served nothing.

1 skill · 1 MCP server

AMD MI300X DevOps for one arm of the E2B data-type sweep: xbill9/gemma-4-E2B-it-qat-q4_0-fp8-text-emb4 (fp8emb4, text only) on one Instinct MI300X via a DigitalOcean droplet. Forked from gpu-vllm-mi300x-2b-q4w4a16 2026-10-07; sweep plan in gpu-vllm-mi300x-2b/DTYPE-SWEEP.md. Has served nothing.

1 skill · 1 MCP server

AMD MI300X DevOps for the 4-bit E2B rig: the QAT Q4_0 W4A16 repack (xbill9/gemma-4-E2B-it-qat-q4_0-w4a16-ct-text, text only) on one Instinct MI300X via a DigitalOcean droplet. Image vetting also requires TritonW4A16LinearKernel in the image's ROCm kernel list. A/B twin of gpu-vllm-mi300x-2b. Added 2026-10-02.

1 skill · 1 MCP server

AMD MI300X DevOps for one arm of the E2B data-type sweep: xbill9/gemma-4-E2B-it-qat-q4_0-w4a16-ct-text-emb4 (q4w4a16emb4, text only) on one Instinct MI300X via a DigitalOcean droplet. Forked from gpu-vllm-mi300x-2b-q4w4a16 2026-10-07; sweep plan in gpu-vllm-mi300x-2b/DTYPE-SWEEP.md. Has served nothing.

1 skill · 1 MCP server

AMD MI300X DevOps for one arm of the E2B data-type sweep: xbill9/gemma-4-E2B-it-qat-q4_0-w4a16-ct-text-ple4 (q4w4a16ple4, text only) on one Instinct MI300X via a DigitalOcean droplet. Forked from gpu-vllm-mi300x-2b-q4w4a16 2026-10-07; sweep plan in gpu-vllm-mi300x-2b/DTYPE-SWEEP.md. Has served nothing.

1 skill · 1 MCP server

AMD MI300X DevOps for one arm of the E2B data-type sweep: xbill9/gemma-4-E2B-it-qat-w8a8-int8 (w8a8, text only) on one Instinct MI300X via a DigitalOcean droplet. Forked from gpu-vllm-mi300x-2b-q4w4a16 2026-10-07; sweep plan in gpu-vllm-mi300x-2b/DTYPE-SWEEP.md. Has served nothing.

1 skill · 1 MCP server

AMD MI300X DevOps for one arm of the E2B data-type sweep: xbill9/gemma-4-E2B-it-qat-w8a8-ct-text-emb4 (w8a8emb4, text only) on one Instinct MI300X via a DigitalOcean droplet. Forked from gpu-vllm-mi300x-2b-q4w4a16 2026-10-07; sweep plan in gpu-vllm-mi300x-2b/DTYPE-SWEEP.md. Has served nothing.

1 skill · 1 MCP server

AMD MI300X DevOps for bf16 Gemma 4 31B (google/gemma-4-31B-it, served text-only) on one Instinct MI300X via a DigitalOcean droplet. The bf16 control for gpu-vllm-mi300x-31b-q4w4a16. Added 2026-10-02.

1 skill · 1 MCP server

AMD MI300X DevOps for the 4-bit 31B rig: the QAT Q4_0 W4A16 repack (xbill9/gemma-4-31B-it-qat-q4_0-w4a16-ct-text, text only) on one Instinct MI300X via a DigitalOcean droplet. Holds the preregistration of the 4-bit-on-MI300X study. A/B twin of gpu-vllm-mi300x-31b. Added 2026-10-02.

1 skill · 1 MCP server

Serve Gemma 4 E2B under pure JAX on the local CPU — no accelerator, no cloud, no provisioning.

1 skill · 1 MCP server

Manage AWS Inferentia2 EC2 and Neuron vLLM inference.

1 skill · 1 MCP server

TPU DevOps for the pure-JAX Gemma 4 rig: provisioning, queued resources, quotas, cost estimates, benchmarking, and SRE diagnostics via the tpu-jax-v5e1-2b MCP server. Serves google/gemma-4-E2B-it-qat-w4a16-ct on v5e-1 with no Docker and no HF token.

1 skill · 1 MCP server

Manage Google Cloud TPU capacity and Gemma 4 JAX serving on Compute Engine TPU VMs: flex-start provisioning, capacity probing, quotas, cost estimates, benchmarking, and SRE diagnostics via the tpu-jax-v6e1-2b MCP server.

1 skill · 1 MCP server

Manage AWS Inferentia2 EC2 and Neuron vLLM inference.

1 skill · 1 MCP server

Manage AWS Inferentia2 EC2 and Neuron vLLM inference.

1 skill · 1 MCP server

Manage AWS Inferentia2 EC2 and Neuron vLLM inference.

1 skill · 1 MCP server

Manage AWS Inferentia2 EC2 and Neuron vLLM inference.

1 skill · 1 MCP server

Manage AWS Trainium (trn1) EC2 instances serving prebuilt Gemma-4 Neuron images.

1 skill · 1 MCP server

Manage AWS Trainium (trn1) EC2 instances for the Gemma-4 E2B fp8 checkpoint.

1 skill · 1 MCP server

Manage AWS Trainium (trn1) EC2 instances for the Gemma-4 E2B q4w4a16 checkpoint.

1 skill · 1 MCP server

Manage AWS Trainium (trn1) EC2 instances for the Gemma-4 E2B q4w4a16emb4 checkpoint.

1 skill · 1 MCP server

Manage AWS Trainium (trn1) EC2 instances for the Gemma-4 E2B w8a8 checkpoint.

1 skill · 1 MCP server

TPU DevOps for the PyTorch/torch_xla Gemma 4 12B rig: provisioning, queued resources, quotas, cost estimates, benchmarking, and SRE diagnostics via the tpu-pytorch-v5e1-12b MCP server. Static-shape decode server on v5e-1.

1 skill · 1 MCP server

TPU DevOps for the PyTorch/torch_xla Gemma 4 2B rig: provisioning, queued resources, quotas, cost estimates, benchmarking, and SRE diagnostics via the tpu-pytorch-v5e1-2b MCP server. Static-shape decode server on v5e-1.

1 skill · 1 MCP server

Manage Google Cloud TPU capacity and Gemma 4 vLLM serving on TPU VMs: provisioning, queued resources, quotas, cost estimates, benchmarking, and SRE diagnostics via the tpu-pytorch-v6e1-2b MCP server.

1 skill · 1 MCP server

Manage Google Cloud TPU capacity and Gemma 4 vLLM serving on TPU VMs: provisioning, queued resources, quotas, cost estimates, benchmarking, and SRE diagnostics via the tpu-pytorch-v6e1-2b MCP server.

1 skill · 1 MCP server

Manage AWS Trainium (trn1) EC2 instances running Gemma-4 through the vLLM-Neuron DLC.

1 skill · 1 MCP server

Manage Google Cloud TPU capacity and Gemma 4 vLLM serving on TPU VMs: provisioning, queued resources, quotas, cost estimates, benchmarking, and SRE diagnostics via the tpu-vllm-v6e8-2b MCP server.

1 skill · 1 MCP server

Compute Engine DevOps for the vLLM Gemma 4 E2B rig on one already-attached NVIDIA Tesla T4 (Turing, SM 7.5, 15360 MiB): host capacity arithmetic across three filesystems, GPU architecture verification against the target interpreter, the Triton shared-memory clamp Turing needs, server lifecycle and SRE diagnostics via the gpu-vllm-t4-2b MCP server. Serves the bf16 reference google/gemma-4-E2B-it (4-bit builds moved to gpu-vllm-t4-2b-w4a16 2026-09-29), for the A/B against gpu-vllm-g4dn-2b — same TU104 T4 at the same clocks, one rig provisioning its hardware and the other simply having it. The first gpu rig here with no provisioning half: no launch, no image resolution, no teardown. Measured 2026-09-18 — QAT decodes 1.79x bf16 per stream.

1 MCP server

Manage vLLM serving of Gemma 4 E2B as 4-bit weights end to end (xbill9/gemma-4-E2B-it-qat-q4_0-w4a16-ct-text-emb4: QAT W4A16 with int4 embeddings, text only) on one NVIDIA Tesla T4 (Turing, SM 7.5) already attached to a Compute Engine VM, via the gpu-vllm-t4-2b-w4a16 MCP server: host capacity across three filesystems, GPU architecture verification, the Triton shared-memory clamp Turing needs, server lifecycle and SRE diagnostics. Carries the repack tools that built the checkpoint. Forked from gpu-vllm-t4-2b 2026-09-29; slot 5 is the only difference. Measured: 2.86 GiB loaded, 1,099,362 KV tokens, 109.7 tok/s at c=1.

1 MCP server

Manage vLLM serving of Gemma 4 E4B as 4-bit weights end to end (xbill9/gemma-4-E4B-it-qat-q4_0-w4a16-ct-text-emb4: QAT W4A16 with int4 embeddings, text only) on one NVIDIA Tesla T4 (Turing, SM 7.5) already attached to a Compute Engine VM, via the gpu-vllm-t4-4b-q4w4a16emb4 MCP server: host capacity across three filesystems, GPU architecture verification, the Triton shared-memory clamp Turing needs, server lifecycle and SRE diagnostics. Forked from gpu-vllm-t4-2b-w4a16 2026-09-30; slot 4 is the only difference. Nothing measured yet.

1 MCP server

Manage vLLM serving of EmbeddingGemma 2 (google/embeddinggemma-2, Gemma 4-derived multimodal embedding model, 768-d with Matryoshka 512/256/128) on one NVIDIA Tesla T4 (Turing, SM 7.5) already attached to a Compute Engine VM, via the gpu-vllm-t4-embed2 MCP server: host capacity across three filesystems, stack verification against the rig's own PYTHONUSERBASE, the encoder-attention Triton clamp float32 needs on Turing, server lifecycle, and embedding tools that add the model's trained task prefixes (embed_text, rank_documents, verify_embeddings). Pooling runner on /v1/embeddings at float32, text + image. Measured 2026-10-07: 1-cos <= 7e-9 against sentence-transformers.

1 MCP server