research

Plugin research from marketplace greenblacked-ai in greenblacked/AI. Commands read from the repo's manifest — 2 days ago.

Skills (19)

• array-chunking-layout

  Choose file format, chunk shape and compression for large array or gridded data (HDF5, NetCDF, Zarr). Writes the top three access patterns first (spatial slice per time step, time series per pixel, random windows), picks one chunk shape serving them all at a few megabytes per chunk, benchmarks candi

• batch-effect-audit

  Check whether technical batches rather than biology or treatment drive a result. Starts from the design: tabulates condition against batch (plate, lane, day, operator, lot, instrument), because a condition wholly in one batch cannot be separated by any correction and needs a randomised rerun. Otherw

• checkpoint-restart-design

  Make a long compute job survive walltime limits, preemption and node failures. A checkpoint is valid only if resuming gives the same trajectory: step counter, model or state, optimiser, RNG states and data-loader position, written atomically (temp file, fsync, rename) keeping two. Interval from the

• convergence-study

  Show that a numerical result does not depend on the grid, time step or tolerance, and that the model is validated on data it was not calibrated on. Fixes a quantity of interest first, runs at least three refinement levels at a constant ratio (ideally 2) changing one parameter at a time, computes the

• fair-method-comparison

  Compare a new method, model or algorithm with baselines so the claim survives scrutiny, starting by reproducing the baseline to its published number at exact versions, logging deviations. Then equal tuning budget for baseline and proposal, tuning on validation only with the test set touched once, at

• floating-point-error-hunt

  Track down a numerical precision bug by finding the first operation where a value goes wrong. Runs a shadow computation in higher precision (mpmath, Decimal, longdouble, or float64 against float32) and diffs intermediates, not just the output; judges error in ULPs or relatively against the problem's

• hpc-resource-sizing

  Choose cores, memory, GPUs and walltime for a cluster job to cut queue wait and waste. Runs a short representative pilot (one task, a fraction of the data) and reads seff (where installed) and sacct for peak MaxRSS, CPU efficiency and elapsed time, then requests a stated margin over measured peak me

• monte-carlo-run-length

  Decide how many samples or chains a stochastic computation needs and whether it has converged. Sets a target confidence-interval half-width and runs until it is met instead of choosing N by habit; uses the standard error for independent runs and batch means or effective sample size for correlated ou

• nan-origin-trace

  Find where a NaN or Inf first appears in a simulation or training run. Restart from the last good checkpoint with floating-point traps on (np.seterr all raise, torch.autograd.set_detect_anomaly, FPE trapping in Fortran and C), add finite checks per stage, bisect steps then layers. Check in order: ba

• nondeterminism-bisect

  Find why the same code, data and seed give different numbers from run to run. Measures first by running the identical job ten times and recording the spread of the headline metric, then removes sources in a fixed order: unseeded or per-worker RNG, hash and set iteration order, thread count and paral

• nonindependence-audit

  Check whether the observations being analysed are really independent, then correct the analysis. Names the unit that was randomised or sampled independently (animal, patient, plate, site, run), counts it as the effective n, recomputes the test on one aggregate per unit as a sanity floor, then refits

• out-of-core-dataset-handling

  Work with tabular data too large for memory on one machine. Measures first (row count, bytes, dtypes, expected in-memory size), converts once to columnar Parquet with sensible row groups sorted by the main filter key, reads only needed columns with filters pushed down, and picks the engine by operat

• paper-statistics-review

  Review the statistics of a paper, preprint, thesis chapter or internal result before relying on it, in a fixed order: estimand and primary outcome and whether prespecified; independent unit versus unit analysed; tests actually run, including forks in the garden; recomputation from the text (p from s

• parallel-filesystem-hygiene

  Stop jobs from hammering a shared cluster filesystem. Counts inodes as well as bytes (lfs quota, lfs df -i), never writes one file per task per timestep to shared scratch but writes to node-local storage ($TMPDIR) and packs to one archive or HDF5 or Parquet file per task group copied out once, stage

• parallel-scaling-study

  Measure how a code scales and choose a production core count. Decides strong scaling (fixed problem) or weak scaling (fixed work per core) up front, runs 1, 2, 4, 8, 16 and more cores with three repeats on one node type, times the compute phase apart from I/O and startup, computes speedup and effici

• reproducible-analysis-environment

  Make an analysis rerunnable by others later, with every derived file traceable to raw data, and prove it on a clean machine. One command (make, snakemake or just) from raw data to every figure and table, run clean; a hash-locked lockfile plus what lockfiles miss (OS, compiler, BLAS build, CUDA and d

• slurm-job-failure-triage

  Diagnose a failed or killed Slurm batch job from the scheduler's record before the log: sacct -j with state, exit codes, time, memory and nodes, plus seff where installed. Classify by State (OUT_OF_MEMORY or a Slurm job's exit 137 judged on the step's MaxRSS, not the batch script's; TIMEOUT against

• sweep-campaign-management

  Run thousands of independent simulations or sweep tasks so finished work survives failures and only the missing part is rerun. Builds a manifest (one row per task: id, parameters, status) before anything is submitted, makes every task idempotent (write to a temp name, rename, drop a done-marker, ski

• unit-mismatch-audit

  Audit code and data for unit and dimension errors. Inventories every quantity crossing a boundary (column, argument, config value, constant) with its unit and frame, then enforces units there with a units library (pint, astropy.units, Unitful) or name suffixes (_m, _s, _kPa) and one conversion modul

Subagents (1)

• slurm-log-reader

  Read a failed Slurm job's sacct rows and its stderr and stdout across an array or MPI run, and return the State class, the step row that decided it and the one decisive log line, so the accounting dump and the per-task logs never enter the caller's context. Use when a batch job, array or MPI run die

Similar plugins