foundry-ai

Plugin foundry-ai from marketplace foundry in fedcal/foundry. Commands read from the repo's manifest — 1 month ago.

Skills (4)

• build-eval-suite

  Build a regression eval suite for an LLM or agent feature — failure taxonomy from real traces, a frozen stratified dataset, deterministic checks before judged ones, binary rubrics, a judge calibrated against human labels before it may gate, k-run CI execution and results reported with uncertainty. U

• build-rag-pipeline

  Stand up or repair a retrieval pipeline the measurable way — audit the corpus, build a labelled query set before the index exists, baseline with lexical search, then add embeddings, hybrid fusion and reranking one variable at a time with recall@k recorded for each. Use when starting a RAG feature, w

• design-agent-tools

  Design or repair the tool surface an LLM agent acts through — one responsibility per tool, schema-constrained arguments, structured errors the model can recover from, capped and paginated results, an explicit side-effect class per tool with runtime enforcement, idempotency keys on irreversible effec

• harden-prompt

  Take a production prompt from a string literal to a versioned, tested artefact — extract it to prompts/<name>/v<N>.md, remove contradictory instructions, pin the output schema with boundary validation and a bounded repair loop, budget the context with a truncation policy that cannot drop instruction

Subagents (4)

• agent-architect

  Designs LLM agent and multi-agent systems — decides first whether an agent is warranted at all versus deterministic code, then chooses topology (single loop, router, supervisor/worker, pipeline, blackboard), designs the state object and checkpointing, specifies tool contracts and side-effect classes

• llm-evaluator

  Measurement discipline for LLM and agent systems — builds frozen reference datasets from real traces, writes binary rubrics, tiers checks from deterministic assertions up to LLM-as-judge, calibrates any judge against human labels before it is allowed to gate, wires non-flaky regression suites into C

• prompt-engineer

  Treats prompts as source code — structure and instruction ordering, few-shot example selection, schema-constrained structured output with boundary validation and bounded repair, context budgeting and cache-friendly prefix layout, explicit abstention and failure behaviour, injection resistance for un

• rag-engineer

  Retrieval-augmented generation as a search problem — corpus audit, chunking strategy, embedding model selection and reindex policy, hybrid lexical+dense retrieval with rank fusion, metadata filtering and tenant isolation, cross-encoder reranking, and retrieval metrics (recall@k, MRR, nDCG) measured

Similar plugins