consumer-lag-triageDiagnose a Kafka or similar partitioned-log consumer group that is lagging, rebalancing or stuck. Take two per-partition snapshots a minute apart using kafka-consumer-groups.sh --describe, compare offset and lag deltas, then read the pattern: lag growing everywhere while offsets advance (throughput
distributed-lock-auditReview every lock, lease and leader election in a service. For each, decide whether it exists for efficiency (a duplicate only wastes work) or correctness (a duplicate corrupts data); a TTL lock alone never protects correctness because a holder can pause past its lease and still write, so the resour
event-schema-evolutionChange an event, message, protobuf or Avro schema without breaking consumers or replay. Choose the registry compatibility mode from deploy order and retention (BACKWARD means upgrade consumers first, FORWARD producers first, FULL when order is uncontrolled, TRANSITIVE when history spans versions), a
graphql-reviewReview or design a GraphQL schema and its resolvers when you are authorised to examine the service: read the schema from its SDL or an introspection result, find the N+1 resolver behind a slow query and fix it with a batched loader, bound what a client can ask for with depth and complexity limits an
grpc-runtime-debuggingDiagnose a gRPC call that fails, stalls or is slow at run time. Read the status code for what the failure class actually is — DEADLINE_EXCEEDED is a deadline, UNAVAILABLE is often a connection, RESOURCE_EXHAUSTED is flow control — then check that the deadline propagates, that keepalive and connectio
hot-partition-huntDiagnose throttling or lag on a sharded store while aggregate utilisation looks fine. Get the per-partition or per-key distribution first, compute max over mean and the top-1-percent traffic share, then classify the cause (single hot entity, monotonic key, low-cardinality key, whale tenant) because
overload-protectionKeep a service producing useful work at twice its capacity instead of collapsing. Find the first resource to saturate, cap in-flight work at the entrance with Little's law (throughput times latency, not a default pool of 200 threads), bound queues by waiting time and drop requests older than the cal
rate-limiter-designDesign the limiter itself, not only its headers. Start from the failure being prevented (abuse, cost, fairness, downstream protection), key on the authenticated tenant plus an endpoint cost class, pick the algorithm (a fixed window admits up to 2x across a boundary; use a sliding-window counter or G
saga-designDesign or review a multi-step business process that spans services, databases or third parties without a distributed transaction. Persist the saga's own state in one row before step 1, classify each step as compensable, pivot or retriable-after-pivot and order them that way, make every compensation
scheduled-job-reliabilityMake cron-style jobs run when intended, once, and visibly. Give every job a contract (idempotent or not, maximum runtime, overlap policy, missed-run policy), pass the logical window in as a parameter instead of reading now(), stop overlap with a lease row or a Kubernetes CronJob set to concurrencyPo
search-index-syncKeep an Elasticsearch or OpenSearch index (self-managed or hosted) consistent with the database. The database is the source of truth and the index is derived from a change stream (outbox or CDC), never a dual write from request code; updates carry the row version so late retries cannot overwrite new
timeout-and-retry-budgetMake a multi-hop synchronous call chain fail fast and stop amplifying its own outages. Draw the chain with timeout, attempts and who retries; retry at exactly one layer because attempts multiply (three layers of three is 27x on the deepest dependency); descend timeouts via a propagated absolute dead
transaction-race-huntFind and fix check-then-act races in code that talks to a relational database. Name the anomaly first (lost update, write skew, check-then-insert phantom, double claim of work), establish the isolation level in force, then take the weakest fix that closes it: a constraint, one atomic conditional sta
webhook-integrationReceive third-party webhooks reliably, or send your own to customers. Receiving: verify the HMAC over the raw request bytes before parsing, compare in constant time, bound replay with a timestamp tolerance where the provider signs a timestamp, store the raw event and return 2xx fast, dedupe on an id