Scalability
Backoff
Waiting progressively longer between retries — typically exponential (1s, 2s, 4s and so on) with random jitter — so clients do not hammer a struggling service in lockstep and trigger a retry storm. The jitter spreads the load out over time. It is the essential companion to retries, usually capped at a maximum delay and attempt count.
Purpose
Backoff spaces retries out — typically exponentially (1s, 2s, 4s, 8s…) — so a failing service gets breathing room instead of a beating. Jitter adds randomness so thousands of clients that failed together do not all return in the same instant, which would just recreate the spike on schedule.
When to Use It
Everywhere retries exist: HTTP clients, queue consumers, database reconnects, SDK calls. It is also polite citizenship toward third-party APIs — respecting rate limits instead of hammering them — and most cloud SDKs implement exponential backoff with jitter by default.
Trade-offs
Longer waits raise tail latency for the caller, so interactive paths need tighter caps than background jobs. Without jitter, exponential backoff still synchronises clients into waves — 'full jitter' (random delay up to the exponential cap) is the standard fix, at the cost of less predictable timing.
Implementation
Delay = random(0, min(cap, base × 2^attempt)) — full jitter with a sensible cap (say 30s) and a bounded attempt count. Honour Retry-After when the server sends one, reset the counter after sustained success, and put the whole policy in one shared client wrapper so every call site behaves the same.