ai-infra

Plugin ai-infra from marketplace oh-my-aidlcops in aws-samples/sample-oh-my-aidlcops. Commands read from the repo's manifest — 28 days ago.

Skills (6)

• agentic-eks-bootstrap

  Bootstrap an AWS EKS cluster optimized for Agentic AI workloads — Karpenter v1.2+ GPU node pools, EKS Auto Mode, Kubernetes 1.32+ with DRA 1.35 GA, VPC CNI, GPU Operator, and baseline observability. Use when starting a new EKS cluster that will host vLLM, Inference Gateway, Langfuse, or Kagent.

• ai-gateway-guardrails

  Enforce Input/Output Guardrails at the LLM Gateway layer — PII redaction, Prompt Injection defense, Jailbreak detection, Toxicity filter, and Tool Allow-list. Integrates Bedrock Guardrails, NeMo Guardrails, Llama Guard 3, and regex/regex-ML policies on Bifrost/LiteLLM with Langfuse audit trail.

• gpu-resource-management

  Design GPU orchestration on EKS using Karpenter v1.2+ NodePools, KEDA scale-to-zero, and DRA 1.35 GA for multi-instance GPU (MIG) partitioning. Right-size NodePool for p5/g6e/trn2 instance mix, spot/on-demand split, consolidation, and topology-aware scheduling.

• inference-gateway-routing

  Configure kgateway v2.0+ as L1 and Bifrost v1.x or LiteLLM v1.60+ as L2 for a 2-Tier Inference Gateway on EKS. Apply Cascade Routing (Haiku→Sonnet→Opus fallback), Semantic Router (intent-based model pick), and HTTPRoute with OTel trace propagation to Langfuse.

• langfuse-observability

  Self-host Langfuse v3.x on EKS (Web, API, Worker) with PostgreSQL, ClickHouse, Redis, and S3 backends. Wire an OpenTelemetry Collector pipeline that receives traces from vLLM, Bifrost/LiteLLM, and LangChain agents so token cost, latency, and quality scores are tracked end-to-end.

• vllm-serving-setup

  Design, deploy, and tune vLLM v0.18.2 inference serving on EKS with PagedAttention v2, Multi-LoRA, FP8 KV Cache, Chunked Prefill, and Continuous Batching. Produces Helm values.yaml, PodMonitor, HPA, and kubectl validation steps for production agentic workloads.

Slash commands (2)

• /ai-infra:oma-platform-bootstrap

  Agentic AI 플랫폼을 EKS 위에 5-체크포인트(Gather Context → Pre-flight → Plan → Execute → Validate) 로 부트스트랩합니다. 클러스터 생성, GPU NodePool, vLLM, Inference Gateway, Langfuse, Guardrails 를 순서대로 배포합니다.

• /ai-infra:oma-platform-review

  기존 Agentic AI 플랫폼 배포를 리뷰하여 GPU 사이징, 관측성 커버리지, Guardrails, 비용 이상, 보안 취약점을 점검합니다. 진단 리포트와 개선 제안을 `.omao/state/platform-review-<date>.md` 에 저장합니다.

Subagents (4)

• inference-gateway-operator

  kgateway v2.0+ 기반 Inference Gateway 를 운영합니다. 2-Tier Gateway 아키텍처(kgateway L1 + Bifrost/LiteLLM L2) 를 설치하고, HTTPRoute, GatewayClass, Cascade Routing, Semantic Router 정책을 구성하며, OpenAI 호환 엔드포인트와 모델별 라우팅을 설정합니다.

• langfuse-observer

  Langfuse v3 self-hosted 관측성 스택을 설치하고 운영합니다. PostgreSQL, ClickHouse, Redis, MinIO/S3 backend 를 EKS 에 배포하고, OpenTelemetry Collector 로 vLLM/Gateway/Agent 트레이스를 수집하며, 비용 추적·프롬프트 버저닝·평가 파이프라인을 구성합니다.

• platform-architect

  Agentic AI 플랫폼의 상위 설계 판단을 담당합니다. SageMaker Unified Studio, Bedrock AgentCore, EKS 기반 오픈 아키텍처 중 하나를 선택하고, 6개 핵심 레이어(모델 레지스트리·서빙·게이트웨이·관측성·데이터·정책)를 라이트사이징하며, 비용·지연·통제권 트레이드오프를 평가합니다.

• vllm-deployer

  vLLM 모델 서빙을 EKS 위에 배포하고 튜닝합니다. PagedAttention v2, Continuous Batching, Chunked Prefill, Multi-LoRA, FP8 KV Cache 등 v0.18.2 핵심 기능을 설정하고, GPU 메모리·TP/PP 병렬화·Throughput 목표에 맞춰 Helm values 와 Kubernetes Deployment 를 생성합니다.

MCP servers (11)

• aws-documentation

  runs: uvx awslabs.aws-documentation-mcp-server==1.1.21

• eks

  runs: uvx awslabs.eks-mcp-server==0.1.28

• bedrock-agentcore

  runs: uvx awslabs.amazon-bedrock-agentcore-mcp-server==0.0.17

• bedrock-kb-retrieval

  runs: uvx awslabs.bedrock-kb-retrieval-mcp-server==1.0.20

• sagemaker-ai

  runs: uvx awslabs.sagemaker-ai-mcp-server==1.0.8

• cloudwatch

  runs: uvx awslabs.cloudwatch-mcp-server==0.0.25

• prometheus

  runs: uvx awslabs.prometheus-mcp-server==0.2.15

• aws-iac

  runs: uvx awslabs.aws-iac-mcp-server==1.0.17

• aws-pricing

  runs: uvx awslabs.aws-pricing-mcp-server==1.0.28

• well-architected-security

  runs: uvx awslabs.well-architected-security-mcp-server==0.1.7

• aws-knowledge

  runs: uvx awslabs.aws-knowledge-mcp-server==0.1.0

Hooks (2)

• PreToolUse

  python3 "${CLAUDE_PLUGIN_ROOT}/hooks/enforce.py"

• SessionStart

  bash "${CLAUDE_PLUGIN_ROOT}/hooks/session-start-ontology.sh"

Similar plugins