Sampling: Recording Less Without Knowing Less

If you only record 10% of your traffic, how do you know the bug you’re hunting landed in the 10% you kept? That question is the entire subject of sampling in monitoring, and the answer is more reassuring than intuition suggests — if you sample the right way. Sampled telemetry with a smart keep-rule catches more real problems per dollar than full recording of a system you can’t afford to keep watching.

Let’s build the intuition first, then do the arithmetic, because the arithmetic is where people stop being scared of the idea.

Sampling in monitoring: head vs tail

You don’t drink the whole pot to know if the soup needs salt. You stir, and you taste a spoonful. Stirring matters — a spoonful off the top before mixing tells you about the top — but given a stirred pot, one spoonful is genuinely representative. That’s the whole statistical claim behind sampling, and it’s the one analogy this article gets, so let’s cash it out precisely.

Head sampling decides whether to record at the start of a session, trace, or request — before anything has happened. Roll a die when the session begins; on a 1, record everything about it; otherwise, record nothing. It’s cheap (no buffering, the decision is one random number) and unbiased (every session had equal odds). Its weakness is that it’s blind: it decides before knowing whether the session will turn out to be interesting. The one session where checkout exploded had the same 10% odds as the ten thousand where nothing happened.

Tail sampling decides at the end, after seeing what happened. Buffer the session (or trace) as it runs; when it completes, apply rules: contained an error? Keep. Took longer than 5 seconds? Keep. Boring? Roll the die. Tail sampling keeps exactly the interesting stuff, at the cost of buffering everything temporarily — which in distributed tracing means real infrastructure, since every span of a trace has to be held somewhere until the trace finishes and can be judged as a whole.

Head sampling asks “should I watch this?” before the movie starts. Tail sampling watches the movie and then decides if it was worth keeping. Everything in between — and most production configurations are in between — is some hybrid.

Error-biased sampling: the strategy that matters

For session replay and error monitoring specifically, the hybrid that matters is simple to state:

Keep 100% of sessions with errors. Sample the clean ones.

This works because the value of a recording is wildly skewed. A session where a user browsed three pages and left tells you almost nothing you didn’t know. A session that ends in an exception is the exact artifact you’ll want at 2am. Uniform sampling treats them identically; error-biased sampling spends your storage budget where the information is.

Mechanically, replay SDKs do this with a buffered mode: record everything into a rolling in-memory buffer, and only persist when an error fires (plus a small random rate of clean sessions, so you keep a baseline of normal behavior — you need it, both as a comparison set and because plenty of real problems never throw). LogReplay and most rrweb-descended tools support some version of this; if yours only offers a flat percentage, that’s a meaningful gap, not a nuance.

The result is the trade the title promised: you record less without knowing less, because “what you know” was never proportional to gigabytes stored. It was proportional to interesting-sessions captured.

“Did we miss it?” — the actual odds

Now the arithmetic. The fear with sampling is the invisible bug: something real happening out there that your telemetry never caught. Let’s put numbers on it, no stats background required.

Suppose you head-sample clean sessions at 10%, and some bug affects sessions you’d classify as clean (no exception thrown — say, a broken layout). Each affected session has a 90% chance of being missed individually. But bugs don’t happen once. The chance you miss every occurrence is 0.9 multiplied by itself once per affected session:

Read that middle of the table again, because it’s the whole insight: sampling doesn’t hide problems, it delays them. A bug hitting 50 users has a 99.5% chance of appearing in your 10% sample. What sampling costs you is the first few occurrences — you might learn about the bug on user 20 instead of user 1 — and the rare one-off, the bug that affected exactly one person ever. For the one-off you’d need 100% recording, and honestly, one unreproduced weird session rarely changes what you ship anyway.

The flip side deserves stating too: sampling wrecks counting small things. If 3 sampled sessions hit an error at 10% sampling, the true count is “about 30, plus or minus a lot.” Sampled data is good at detecting and diagnosing, mediocre at precisely measuring rare events. Alert on symptoms measured from unsampled signals (error counts, server metrics) where you can, and use sampled replays for the diagnosis.

The cost math

Concrete example. Say a recorded session averages 2 MB stored, and you see 100,000 sessions a month with a 2% error rate:

The third row is the argument in miniature: for roughly the cost of the flat-10% plan, error-biased sampling captures every errored session — the flat plan threw away 90% of the recordings you’d actually open. Recording cost isn’t only storage, either; it’s client CPU and upload bandwidth on your users’ devices, which is its own topic — see how much session replay costs the page.

What I’d actually configure

For a small SaaS product, my defaults, in order of conviction:

1. 100% of sessions with errors. Non-negotiable. This is the entire point of having replay.
2. 100% of clean sessions too, if volume allows it. Under ~50k sessions/month, storage is cheap enough that sampling is solving a problem you don’t have yet.
3. When volume forces it, sample the clean sessions, never the errored ones. Start at 25%, watch your storage bill, adjust.
4. Pin high-value flows to 100% if your tool supports rules — checkout, onboarding, anything where one lost recording costs more than a month of storage.

The mistake to avoid is the flat global percentage chosen to hit a price point. It’s the one strategy in the table that manages to cost money and lose the sessions you wanted. Stir the pot, then taste — don’t pour out 90% of the soup at random and hope the salt was evenly distributed.