Model Lab Skills

Plugin model-lab-skills from marketplace socket in gaelic-ghost/socket. Commands read from the repo's manifest — 1 month ago.

Skills (13)

• ablate-refusal-representations

  Reproduce and evaluate refusal-direction ablation on controlled checkpoints. Use when estimating refusal representations, applying activation or weight orthogonalization, measuring behavior change, or comparing ablated artifacts.

• benchmark-model-runtime

  Benchmark model runtimes across latency, throughput, memory, energy, load time, size, and stability. Use when comparing Core AI, Core ML, MLX, ExecuTorch, PyTorch, quantization, devices, or packaging.

• choose-apple-model-runtime

  Compare and select Core AI, Core ML, MLX, MLX Swift, MLX LM, ExecuTorch Apple delegates, or Foundation Models. Use when an Apple model workflow needs a runtime choice and implementation handoff.

• choose-model-lab-workflow

  Route language-model training, data, evaluation, checkpoint, representation, steering, ablation, jailbreak, tool-calling, Apple-runtime, and benchmark requests. Use when the primary workflow or Socket owner is unclear.

• compare-model-checkpoints

  Compare base, adapter, merged, quantized, converted, or intervention-derived checkpoints. Use when selecting an artifact, investigating regression, verifying packaging, or balancing quality, safety, size, and runtime.

• design-model-experiment

  Design a reproducible model experiment with hypotheses, controls, baselines, metrics, budgets, provenance, artifacts, and stop conditions. Use before model work whose results must support a decision.

• evaluate-jailbreak-resilience

  Evaluate authorized jailbreak and prompt-injection resilience with reproducible attacks, budgets, graders, and side-effect boundaries. Use for controlled checkpoints, local harnesses, or explicitly authorized systems.

• evaluate-language-model

  Design repeatable language-model capability, quality, behavior, regression, or safety evaluations. Use when creating cases and graders, comparing a treatment with a baseline, or deciding whether a checkpoint passes.

• evaluate-tool-calling-model

  Evaluate tool selection, arguments, schemas, execution, observation use, recovery, and side effects. Use when comparing tool-capable checkpoints, prompts, parsers, agent loops, function schemas, or harnesses.

• fine-tune-language-model

  Plan supervised fine-tuning, LoRA, QLoRA, or full-parameter training with reproducible evaluation gates. Use when adapting a checkpoint or choosing precision, adapters, optimization, checkpointing, or resume behavior.

• prepare-language-model-dataset

  Prepare language-model training, preference, red-team, or evaluation datasets. Use when collecting, filtering, deduplicating, templating, splitting, or auditing examples for provenance, leakage, consent, or sensitive data.

• research-model-representations

  Design causal research into model activations, features, attention, residual streams, probes, and circuits. Use when locating behavior, testing a direction or feature, comparing layers, or reproducing interpretability research.

• steer-language-model-behavior

  Implement and evaluate activation steering, representation engineering, logit changes, or weight-space interventions. Use when changing model behavior without ordinary fine-tuning or sweeping layer, strength, and persistence choices.

Similar plugins