Tail latency and production systems

Shows why fan-out makes a rare slow server a common slow request, and names the techniques that tolerate variance.

Defines the stall band that out-of-order hardware cannot hide and a context switch cannot amortise.

Shows that a summary without a max discards the samples that define the tail, and closed-loop load never records them.

The original definition of the recording error, with arithmetic for how far a reported percentile sits from the truth.

Keeps the whole distribution at fixed relative precision in constant time, so the far percentiles and max survive.

The reference wakeup-latency measurement for Linux, whose README states that an unloaded run proves nothing.

Counts the noise a spinning thread suffers and attributes each event to NMI, IRQ, softirq, thread or hardware.

Derives the queueing-ideal tail and attributes the excess to scheduling, interrupt placement, power saving and NUMA.

Measures with code the page-fault, TLB-shootdown and writeback stalls that memory mapping hides from the caller.

Defines the host-side polling after a vCPU halt that trades idle host CPU for guest wakeup time, unseen by the guest.

Shows that open and closed load models disagree on response time and scheduling gains, with rules for choosing one.

Issues requests on a fixed schedule and times each from when it was due, so server stalls reach the percentiles.

Shows the tail, not throughput, caps a latency-critical server's utilisation, and how far co-located work lowers it.

Pairs latency-critical services with an open-loop harness that records sojourn against service time per request.

Measures the key, value and inter-arrival distributions of live key-value traffic, the shape load generators imitate.

Measures with code the floor for handing a cache line between cores, which every queue and lock is built on.

States the design rule that removes write contention outright, using a contended increment's cost as the argument.

Adds cached indices to a single-producer single-consumer ring and shows with counters the coherence traffic removed.

Applies the single writer rule and cache-line padding to a ring buffer, with the queue comparison that motivated it.

Carries the single writer and batching rules through a whole transport, the reference beyond one in-process queue.