as-of-historyAnswer what was true, or what we believed, on a past date, and keep the history to answer it: classify the question as state, validity or arrival, pick a periodic snapshot, type-2 dimension or append-only change log to match, add recorded time beside valid time for anything restated or backdated, jo
causal-inference-reviewEstimate the effect of something that was not randomised — a launch, a price change, a policy, a campaign — and say honestly whether the data supports a causal claim: write the estimand and a confounder diagram first, let the assignment mechanism pick the method (difference-in-differences with an ev
consumer-discoveryFind who really reads a warehouse table, view or column before you rename, drop or retire it, and retire it safely: pull 90 days or more of warehouse access history and BI query logs, resolve views, service accounts and BI extracts to named people and jobs, add orchestrator lineage and a code search
data-quality-checksDecide which data quality checks a warehouse table gets, what each threshold is and what happens when one fails. Mine the last ten data incidents and write one check per root cause before anything generic, then cover freshness, volume, key uniqueness, null rate and enum drift on the columns consumer
entity-resolutionMatch records for the same customer, company, person or product across systems that share no key, so that a merge can be measured and undone. Normalise fields first, block on cheap candidate keys instead of comparing all pairs, apply deterministic match and hard non-match rules before any fuzzy scor
experiment-designPlan an A/B test before launch so its result can be trusted: write one primary metric and its guardrails down first, randomise on the same unit you analyse, size the test from baseline variance and a minimum detectable effect worth shipping (delta method for ratios), round the duration up to whole w
experiment-readoutRead out a finished A/B test and turn it into a ship, hold or rerun decision: run the sample ratio gate first and stop if it fails, estimate the effect with a confidence interval on the pre-registered primary metric only, check guardrails against their margins, label every segment cut exploratory wi
feature-leakage-auditDetect target leakage before trusting a model's offline score: for every feature record when its value becomes known against the moment of prediction, rebuild joins point-in-time, split by time or by entity group instead of random rows, fit encoders, scalers, imputers and feature selection inside ea
forecast-backtestingBuild or judge a time-series forecast — demand, revenue, traffic, load — with a backtest that mirrors how it will be used: rolling-origin evaluation at the real decision horizon using only data known at each origin, never a random split; a seasonal-naive baseline it must beat or the work stops; a sc
join-fanout-huntFind why a total is inflated or a row count grew after a join, and fix it at the grain instead of hiding it: write the grain of every input in one sentence, compare COUNT(*) with COUNT(DISTINCT key) after each join while bisecting the query join by join, classify the cause as many-to-many, a dimensi
late-data-maturityDecide when a daily or hourly figure is final enough to publish, from measured arrival lag: record event time and load time on every row, build a maturity curve of the share of each day's eventual rows and value present 1h, 6h, 24h and 7d after the day closes, publish where it passes an agreed compl
metric-movement-decompositionExplain why a ratio metric moved — conversion, average order value, churn — with arithmetic instead of a story: rule out data breaks first (volume, null rate on the segment column, a tracking release, an incomplete latest day), check the move is outside normal variation, split the change into rate e
metric-reconciliationSettle why two dashboards, teams or systems report different values for the same metric, and end with one definition instead of an explanation: freeze one metric, one day and one segment, reproduce each figure from raw data, diff the two row sets by key with a full outer join, and assign the gap to
model-drift-monitoringMonitor a deployed classical model and decide when to retrain: check in order input schema and null rates, feature distributions (PSI or KS statistic against the training window, per-feature thresholds from history), prediction distribution, then delayed-label performance on matured cohorts; measure
model-error-analysisImprove a classifier or regressor by studying where it is wrong instead of tuning hyperparameters: choose the metric from what each error costs (PR-AUC and precision at a recall floor when positives are rare), pull the hundred worst errors and relabel them by hand to tally label noise against missin
notebook-to-productionTurn an exploratory Jupyter notebook into a scheduled, tested job without changing its answer: prove it runs top to bottom headless, record golden output on a pinned sample as a characterisation test, extract functions behind that test, take dates and paths as parameters, secrets from the environmen
scan-cost-forensicsFind which queries, users, dashboards and jobs drive a data warehouse bill on Snowflake, BigQuery, Redshift or ClickHouse: group query history by normalised-SQL fingerprint and rank by total bytes or credits across every run rather than by the slowest single run, attribute each fingerprint to a dash
serving-skew-replayFind why a model scores well offline and badly live by proving or ruling out train/serve skew: log the exact served feature vector with a request id and timestamp, replay the same entities through the training feature pipeline as of that timestamp, diff column by column, ranked by scaled mean absolu
spreadsheet-retirementReplace a business-critical spreadsheet or manual report with a pipeline without losing the trust of the people who rely on it. Inventory the hidden logic first by tracing every input, hard-coded cell, manual override, macro, external link and monthly ritual; turn each override into an explicit, own
srm-diagnosisFind why an experiment's traffic split differs from its design (sample ratio mismatch) and decide what to do with the result: confirm with a chi-square test at p below 0.001 on randomisation units, locate the stage where the arms diverge by counting at assignment, exposure, first event and the analy
switchback-test-designDesign and analyse an experiment where units interact, so a user-level split is biased — marketplaces, dispatch, matching, pricing, shared inventory, ops queues: randomise time windows or regions instead of users, make each window longer than the effect's carryover after measuring it, balance arms w
upstream-change-defenseProtect warehouse pipelines from source teams changing a field's schema or meaning: classify each surprise as structural (column dropped, renamed, retyped) or semantic (same column, new meaning: a new status code, pence to pounds, a time zone switch, re-keyed ids), land raw untouched, diff the schem