Hey everyone, welcome to the forty-fourth issue of The Main Thread.
Most engineers carry exactly one probability distribution in their head: the bell curve. It’s the one from school, it’s in every stats explainer, and it’s the one people reach for by reflex when they need to reason about anything random. The problem is that the bell curve is the wrong model for most of the data we actually deal with. The latency isn’t normal. Request arrivals aren’t normal. Cache hit rates, file sizes, retry counts, popularity of the keys, none of them are normal. Assuming they are is how we end up reporting a meaningless “average latency”, provisioning capacity for a load that never looks like the forecast, and being blindsided by a tail event that our model said was impossible.
The good news is that we don’t need all of probability theory; just 7 distributions. They cover the overwhelming majority of randomness in systems engineering and machine learning, and once we recognize them, a lot of behavior that looked like noise resolves into something we can predict and design around. Each one corresponds to a physical story: a way events happen in the world, and learning to match the story to the distribution is most of the skill.
This is that cheat sheet. For each distribution: the story it tells, when it shows up in real systems, the formula and a line of code, and most importantly, the gotcha, the place where reaching for it naively will burn us. At the end, a recognition guide and a one-page table to bookmark. Everything here lives in Ross’s A First Course in Probability. The contribution of this article is to map onto the things we ship.
1. Normal: The Default That’s Usually Wrong
Models: the sum or average of many small, independent, additive effects.
The normal distribution, or the bell curve, N(μ, σ²) earns its ubiquity from one of the deepest results in probability: the Central Limit Theorem (CLT). The intuition is worth internalizing because it tells us exactly when to expect bell curve and when not to. Take many independent random quantities, each with finite variance, and add them up. It doesn’t matter what each individual quantity’s distribution looks like - uniform, skewed, bimodal, whatever. Their sum converges to a normal distribution as we add more of them. The idiosyncrasies of each piece cancels out, and what’s left is a bell.
This is the whole reason why normals are everywhere: any time an outcome is the accumulation of many small independent contributions, we get a bell. Measurement noise (many tiny perturbations summed). The average of a large sample. Aggregate metrics over many users. The position of a particle taking many random steps.
Where it legitimately shows up:
Simple means and aggregate metrics. The mean of a large batch is normal even when the individual values aren't. This is what lets us put confidence intervals on A/B test averages.
Measurement and sensor noise. This is modeled as Gaussian because it is a sum of many small error sources.
ML internals. Weight initialization, Gaussian noise in diffusion models, the assumption behind least-squares regression and many priors.
The two facts to keep handy: the 68-95-99.7 rule (that fraction of mass within 1, 2, 3 standard deviations), and that the standard error of a mean shrinks like 1/√n;to halve our uncertainty, we need four times the data.
from scipy import stats
# Standard error of the mean shrinks as 1/sqrt(n),
# diminishing returns
sem = data.std(ddof=1) / len(data) ** 0.5The gotcha: most engineering data is not normal. Latency, file sizes, wealth, request rates, popularity - all skewed, all heavy-tailed, none symmetric. The CLT requires the effects to be additive and finite-variance; the moment effects are multiplicative (next section) or the variance is effectively infinite (power laws, section 7), the bell curve is the wrong model. When someone reports a single “average” for latency, they have assumed a symmetric distribution for data that is anything but - and the average they computed is a number that describes no real request. We should reach for the normal only when the generating story is genuinely “many small things added up”. Otherwise, keep reading.
2. Log-Normal: Why Latency is Never a Bell Curve
Models: the product of many small independent effects; equivalently, anything whose logarithm is normal.
This distribution is relatively less-known among engineers; even though it governs the metric we stare at most: response time. A variable is log-normal if log(X) is normally distributed. The story is the multiplicative twin of the CLT: when an outcome is the product of many independent factors rather than their sum, the logs add (because log(a·b) = log(a) + log(b)), the CLT applies to those logs, and we get a log-space normal in the original space.
Real latency is multiplicative: a request’s total time is queueing × processing × network × serialization × downstream calls, each a factor that can stretch the others. That’s why latency distributions are right-skewed with long tail - a floor near zero, a hump at the typical value, and a tail of slow requests stretching far to the right. The mean sits well above the median, dragged up by the tail.
Where it shows up:
Latency and response times. This is the canonical case. This is why we report p50/p95/p99 and never a bare average (the latency issue earlier in this newsletter leaned hard on this; here’s the distribution underneath it).
File sizes, object sizes, message sizes in storage and networking.
Time-to-failure of components, and many "duration" quantities in general.
from scipy import stats
# Fit latency to a log-normal; note how far the mean
# sits above the median
shape, loc, scale = stats.lognorm.fit(latencies, floc=0)
median = scale
mean = scale * (2.718281828 ** (shape ** 2 / 2))# mean > medianThe gotchas: people fit (or implicitly assume) a normal to latency and then every downstream conclusion is wrong. They report the mean, which the tail inflates into a number no user experiences. They set alerts at “mean + 3σ” and either never fire or fire consistently, because σ is huge and the distribution isn’t symmetric. The fix is cultural as much as mathematical: for anything that looks like a duration or size, assume log-normal until proven otherwise, summarize it with percentiles, and look at it on a log axis where it straightens into something legible.
3. Poisson: Counting Events in a Window
Models: the number of independent events that occur in a fixed interval, given a constant average rate.
Where the normal and log-normal are about the magnitude, the Poisson is about counts. If events happen independently, at some constant average rate λ per unit time (or space), and no two happen at exactly the same instant, then the number we see in a fixed window follows a Poisson distribution: P(k) = λᵏ·e⁻λ / k!. Its signature property is that mean and variance are both equal to λ; a fact that we will use to test whether something is actually Poisson.
Where it shows up:
Request arrivals per second to a service - the foundational assumption of queueing theory and M/M/1 model.
Rare failure or errors per day, packet arrivals, disk faults, mutations.
Event counts generally: cache misses in a window, log lines per minute, page faults.
from scipy import stats
lam = counts.mean()
# Poisson's tell: variance ≈ mean.
# If variance >> mean, it's NOT Poisson.
dispersion = counts.var() / counts.mean() # ~1.0 for PoissonThe gotcha - overdispersion: real traffic is bursty, and bursty means variance far exceeds the mean, which Poisson forbids. Arrivals cluster (a retry storm, a viral moment, a cron job firing thousands of jobs at once), violating the independence assumption. When we measure variance/mean and it’s 5 or 50 instead of ~1, our data is overdispersed and Poisson will badly underestimate how often we get a punishing burst. The honest model there is often negative binomial (Poisson with a varying rate). Capacity planning that assumes Poisson arrivals provisions for the average and gets destroyed by the burst; which is exactly the failure mode the load-balancer and sharding issues kept running into.
4. Exponential: Time Between Events
Models: the waiting time until the next event in the Poisson process.
Poisson counts events; the exponential measures the gap between them. If events arrive as a Poisson process at rate λ, the time we wait for the next one is exponentially distributed. f(t) = λ·e⁻λᵗ, with mean 1/λ. The two distributions are the same phenomenon viewed two ways - counts vs. inter-arrival times.
The exponential has one defining, deeply weird property: it is memoryless. The probability we wait another 10 seconds is the same whether we have already waited 0 seconds or 100. The distribution has no memory of how long we have been waiting. This is mathematically elegant, as we will see, frequently false in the real world - which is exactly why it’s worth knowing.
Where it shows up:
Inter-arrival times of requests, and service times in the M/M/1 queueing model (which assumes exponential service).
Time-to-failure under a constant hazard rate: the “random failures” middle of the bathtub curve.
TTL, timeout, and backoff design, where we are reasoning about waiting times.
from scipy import stats
# lambda; mean wait = 1/lambda
rate = 1.0 / inter_arrival_times.mean()
# Memoryless check: P(T > s+t | T > s) should not depend on sThe gotchas: memorylessness is usually a lie for the things engineers care about. A request that has already been running for 10s is more likely to be slow, not equally likely - real service times are heavy-tailed (log-normal!), not exponential, because slow requests are slow for persistent reasons (a bad shard, a lock, a GC pause). Assuming exponential service times makes our queueing math tractable and our tail predictions optimistic. Use the exponential for quick back-of-envelope queueing intuition; distrust it the moment the tail matters, and reach for log-normal.
5. Binomial: Counting Success Out of N
Models: the number of successes in n independent yes/no trials, each with the same success probability p.
This is the distribution of A/B testing and reliability. We run n independent trials, each succeeding with probability p, and the count of successes is binomial: P(k) = C(n,k)·pᵏ·(1−p)ⁿ⁻ᵏ with mean np and variance np(1-p). Every time we are counting how many out of a fixed number did the thing, this is our model.
Where it shows up:
A/B tests and feature flag rollouts: conversion out of visitors, the basis for every significance test on a rate.
Error rates: failures out of n requests; the binomial gives us the confidence interval around an observed rate.
k-of-n redundancy and availability: the probability that at least k or n replicas are up.
from scipy import stats
# 95% confidence interval on a conversion rate
# from a binomial proportion
successes, n = 84, 1000
ci_low, ci_high = stats.binomtest(successes, n).proportion_ci(0.95)The gotchas: the binomial demands independent trials with constant p, and both fail quietly in practice. Users in A/B test aren’t independent (they share a viral source, a time-of-day effect, a buggy region), which makes our true variance larger than np(1-p) and our p-values overconfident: we declare a winner that isn’t. And p often isn’t constant (it drifts over the experiment). Before trusting a binomial confidence interval, we must ask whether our trials are really independent and really identically distributed. Two useful approximations to keep handy: when n is large and p moderate, the binomial is well-approximated by a normal (CLT again); when n is large and p tiny, by a Poisson (rare events).
6. Geometric: How Many Tries Until It Works
Models: the number of independent attempts until the first success.
This is the discrete cousin of exponential. If we keep trying something that succeeds with probability p each time, independently, and the number of attempts until our first success is geometric: P(k) = (1−p)ᵏ⁻¹·p, with mean 1/p. If something works one time in twenty (p = 0.05), we should expect to try it twenty times. Like the exponential, it’s memoryless: past failures don’t improve our odds on the next attempt.
Where it shows up:
Retry counts: how many attempts until a flaky call succeeds. Expected retries = 1/p, which is the number that should drive the retry budgets and timeouts.
First-hit scenarios: probe until a hash slot is free, attempts until a lock is acquired, draws until a sample passes rejection sampling.
Time-to-first-event: in any “keep trying” loop.
# Expected attempts until first success,
# and the tail of the retry count
p = 0.05
expected_attempts = 1 / p # 20
p_more_than_50_tries = (1 - p) ** 50 # ~7.7%The gotchas: the independence assumption is exactly wrong for the retries we care about most. If a call failed because the downstream service is down, the next immediate retry fails for the same reason - the attempts are correlated, not independent, so the clean 1/p expectation doesn’t hold and naive retrying just hammers a struggling system. This is the probabilistic justification for exponential backoff with jitter (which the saga and latency issue both invoked): backoff de-correlates retries in time so the constant-p assumption becomes closer to true, and jitter de-correlates them across clients so they don’t synchronize into a thundering herd. The geometric tells us how many tries we will need; backoff is how we make its assumptions hold.
7. Power Law: The Few That Dominate the Many
Models: quantities where a small number of values account for most of the mass: heavy-tailed, scale-free popularity.
The most important distribution in large systems, and the one furthest from the bell curve. A power law (Pareto, or Zipf in its discrete/ranked form) has P(X = x) ∝ x⁻ᵃ: a heavy tail that decays polynomially, far more slowly than the exponential cliff of a normal. It’s the math of “80% of the effect from 20% of the causes”, except in real systems it’s often more like 99/1. There’s no characteristic scale: the distribution looks the same whether we zoom into the top 10 or the top 10,000.
Where it shows up, and this list is why the distribution matters so much:
Popularity and access frequency. A few keys, videos, products, or pages get the overwhelming majority of requests. This is why caching works at all: a small hot set serves most traffic. Zipf-distributed access is the assumption behind cache hit-rate math.
The hot-shard / celebrity problem. The same heavy tail that makes caching works makes sharding hard; a few keys carry so much load they overwhelm their shard (the exact failure mode the sharding issue dwelt on). Power laws are why “high cardinality“ isn’t enough and we have to worry about traffic skew.
Network structure: node degrees in social and dependency graphs, the basis of network effects.
File sizes, fault sizes, fan-out: and many “rich get richer” processes.
import numpy as np
# Power-law tell: roughly a straight line on a
# log-log rank-frequency plot
ranks = np.arange(1, len(freqs) + 1)
# ≈ -alpha
slope = np.polyfit(np.log(ranks), np.log(np.sort(freqs)[::-1]), 1)[0]The gotchas: power laws break our Gaussian instincts entirely:
The “average” can be meaningless or undefined. If the tail is heavy enough (α ≤ 2), the variance is infinite and the sample mean never stabilizes - it keeps jumping every time a bigger outlier shows up. Reporting an average file size or an average degree can be nonsense.
Rare events aren’t rare enough to ignore. The max grows with the sample size; the biggest event we will ever see is still ahead of you. Provisioning for the observed maximum guarantees that we will be exceeded.
Sampling underestimates the tail. Small samples systematically miss the giant rare values, so our estimate of “how bad can it get” is always too optimistic.
When the data is power-law, we need to throw out the toolkit from steps 1-2. The mean lies, the standard deviation lies, and “3-sigma events” happen on Tuesdays. Reason in percentiles, ranks, and tail ratios instead, and design for the hot few explicitly - cache them, replicate them, split them - because they are our system’s behavior.
How to Recognize Which Distribution Our Data Follows
We will rarely be told which distribution we have. We have to read it off the data. A practical procedure, fastest checks first:
1. Plot a histogram before anything else
Half of identification is just looking. Symmetric and bell-shaped → normal. Right-skewed with a long tail → log-normal or power law. A spike near zero decaying smoothly → exponential. Discrete counts → Poisson, binomial, or geometric.
2. Ask what the data is: the generating story usually settles it
Counting events in a fixed window? → Poisson
Time (or gap) between events? → Exponential
Successes out of a fixed number of trials? → Binomial
Attempts until the first success? → Geometric
A sum/average of many small things? → Normal
A product of many factors / a positive right-skewed magnitude (latency, size)? → Log-normal
A few values dominate everything, heavy tail? → Power law
3. Use the cheap numerical tells
Variance vs mean. Equal → Poisson. Variance ≫ mean on counts → overdispersed, not Poisson (negative binomial).
Mean vs median. Roughly equal → symmetric (normal). Mean noticeably above median → right-skewed (log-normal / power law).
4. Let log-scales do the work; each distribution straightens a different axis
Exponential → straight line on a log-y (semi-log) plot.
Log-normal → bell-shaped after you take the log of the values.
Power law → straight line on a log-log plot (rank vs frequency).
5. Confirm with a Q–Q plot
Use this against the candidate distribution; points on the diagonal mean a good fit, and the way they deviate (usually in the tail) tells you how it's failing.
The meta-rule: don't default to normal. The single most common modeling error in engineering is assuming a bell curve for skewed, heavy-tailed, or count data, computing a mean, and trusting it. When in doubt, plot it on a log axis and compare the mean to the median; those two five-second checks catch the majority of mistakes in this entire issue.
Cheat Sheet
Distribution | Models | Key params | Mean | Where it shows up | The tell |
|---|---|---|---|---|---|
Normal | Sum of many small additive effects | μ, σ | μ | Sample means, aggregate metrics, sensor noise, ML init | Symmetric bell; mean ≈ median |
Log-normal | Product of many factors; positive & skewed | μ, σ (in log) |
| Latency, file sizes, time-to-failure | Right-skew; bell after log; mean ≫ median |
Poisson | Count of independent events in a window | λ | λ | Request arrivals, rare failures, event counts | Discrete; variance ≈ mean |
Exponential | Time between Poisson events | λ | 1/λ | Inter-arrivals, service times, TTL/backoff | Spike at 0, decay; straight on log-y; memoryless |
Binomial | Successes in n fixed trials | n, p | np | A/B tests, error rates, k-of-n redundancy | Discrete; bounded by n |
Geometric | Attempts until first success | p | 1/p | Retry counts, first-hit, probing | Discrete; decreasing; memoryless |
Power law | A few values dominate; heavy tail | α (xₘᵢₙ) | often unstable | Popularity, cache/hot-shard, node degree | Straight on log-log; mean unstable |
Quick decision path:
count of events? Poisson.
time between them? Exponential.
successes in n trials? Binomial.
tries until success? Geometric.
sum of many things? Normal.
product of many things / a duration or size? Log-normal.
a few hot items running everything? Power law.
The Takeaway
Probability isn't an academic subject we took once; it's the implicit model behind every metric we report and every capacity decision we make. The reason it's worth carrying these seven in the head is that each one is a story about how randomness is generated, and matching the story to the data is what turns a confusing scatter of numbers into something we can reason about, forecast, and design for.
The throughline is that the normal distribution is the exception in engineering, not the rule. Latency is log-normal, arrivals are Poisson-ish-but-burstier, popularity is power-law, retries are geometric. The bell curve and its tidy "mean ± standard deviation" describe almost none of it, and the habit of forcing everything into that shape is the source of a remarkable number of production surprises; the average that describes no real request, the capacity plan that the burst destroyed, the "impossible" tail event, the hot key that took down a shard. Every one of those is a distribution mismatch.
So the practical discipline is small and high-leverage: before we compute a mean, we should look at a histogram. Before we trust an average, we should compare it to the median. Before we assume a bell curve, we should ask what story generated the data; sum or product, count or gap, successes or attempts, uniform or dominated-by-the-few. The seven distributions here cover the vast majority of what we will meet, and the recognition guide turns "what even is this data" into a few quick checks.
Bookmark the table. The next time a metric behaves in a way that surprises you, the odds are good you have been modeling it with the wrong distribution, and the right one is on this page.
Which distribution has bitten you in production? I am collecting examples: the "average" that hid a brutal tail, the Poisson assumption that missed the burst, the power-law hot key nobody saw coming. The mismatches are remarkably consistent across systems, which is what makes them learnable.
Hit reply. I read everything.
Namaste!
If this clicked, forward it to the engineer on your team who reports averages for latency, gently. And if you want more of this kind of cross-domain thinking, subscribe to The Main Thread: practical engineering from unusual angles, one essay per week.
