Retry & Backoff Strategies

Exponential backoff

@Retry(name = "orderService")
public Order fetchOrder(String id) { return orderClient.get(id); }

resilience4j:
  retry:
    instances:
      orderService:
        max-attempts: 4
        wait-duration: 200ms
        exponential-backoff-multiplier: 2  # 200ms, 400ms, 800ms between attempts

Doubling the wait time between successive retry attempts, rather than retrying immediately or at a fixed interval, gives a struggling dependency progressively more breathing room — an immediate-retry storm from many callers hitting a momentarily-slow service is exactly what makes a brief blip turn into a full outage (every failed request immediately retries, doubling load right when the service is least able to handle it).

Jitter: preventing synchronized retry storms

long baseDelay = (long) (initialDelayMs * Math.pow(2, attempt));
long jitteredDelay = baseDelay + ThreadLocalRandom.current().nextLong(0, baseDelay / 2); // add randomness

Even with exponential backoff, if every client experiencing the same failure retries on the exact same schedule, their retries arrive synchronized — a thundering herd hitting the recovering service at the same moments, again and again. Jitter — adding a randomized offset to each client's backoff delay — desynchronizes retries across clients, spreading the same total retry volume out over time instead of concentrating it in synchronized bursts. This is a small code change with an outsized real-world impact at scale.

Idempotency as a prerequisite for retrying writes

Retrying a read is always safe — re-fetching the same data has no side effect. Retrying a write is only safe if the operation is idempotent (calling it twice has the same effect as calling it once) — retrying a non-idempotent "charge $50" request risks a duplicate charge if the first attempt actually succeeded but the response was lost before the caller saw it. This is exactly why Payment — Idempotency Implementation's idempotency-key pattern isn't optional infrastructure for a payment system with retries enabled — it's the prerequisite that makes retrying writes safe at all, not a nice-to-have.

Retry budgets

resilience4j:
  retry:
    instances:
      orderService:
        max-attempts: 4        # cap total attempts
        # combined with a timeout budget ensures retries can't compound into unbounded latency

Unbounded retrying — keep trying until it eventually succeeds — can turn a single slow request into an effectively-hung one, and at scale, can multiply load on an already-struggling dependency far beyond what a capped retry count would. A retry budget caps both the total number of attempts AND, ideally, total elapsed retry time, so a caller fails cleanly (returning control to whatever's waiting on it) rather than retrying indefinitely against a dependency that simply isn't coming back soon.

Retryable vs non-retryable errors

Same distinction as Circuit Breaker Pattern's "which failures should trip the breaker": retryable — timeouts, 503 Service Unavailable (transient, likely to succeed on a later attempt). Non-retryable — 400 Bad Request, validation errors (the request itself is wrong; retrying an invalid request produces the exact same invalid result every time, wasting the retry budget and adding latency with zero chance of success).

Combining retry with circuit breaker

@CircuitBreaker(name = "orderService")
@Retry(name = "orderService") // Resilience4j applies decorators in a defined order — CircuitBreaker wraps OUTSIDE Retry
public Order fetchOrder(String id) { return orderClient.get(id); }

The correct composition: retry within a closed breaker (attempting the configured number of retries while the breaker still considers the dependency healthy), and stop retrying the moment the breaker opens — continuing to retry against an already-open breaker defeats the breaker's entire purpose (reducing load on a struggling dependency) by generating exactly the traffic it exists to suppress. Getting the decorator ordering right (breaker checked before/around retry attempts, not retry attempts happening independently of breaker state) is a real, easy-to-get-wrong configuration detail worth naming explicitly.

Follow-up questions this topic invites — and their answers

Q: Why does jitter add randomness rather than just using a slightly different fixed multiplier per client? A: A fixed-but-different multiplier per client would still produce a deterministic, repeatable pattern across that client's own retries — genuine per-attempt randomness (not just per-client variation) is what prevents even a single client's own retry sequence from ever aligning with another's, which matters when thousands of clients are all affected by the same outage simultaneously.

Q: Is GET always safe to retry without an idempotency concern? A: Generally yes for a properly RESTful GET (no side effects by definition) — but worth noting some real APIs violate this convention (a GET that increments a view counter, for instance), which is why 'idempotent by HTTP method convention' and 'idempotent in actual implementation' aren't guaranteed to be the same thing for every API you integrate with.

Q: How would you decide the right max-attempts value for a given integration? A: Balance the dependency's typical recovery time (how long transient blips usually last) against the caller's own latency budget (how long the calling flow can tolerate waiting) — a payment flow might tolerate very few retries before failing fast to the user, while a background batch job might reasonably retry many more times over a longer window.

Q: What's a concrete failure mode of retrying a non-idempotent operation without an idempotency key? A: A client sends a payment request, the server processes it successfully but the response is lost on the way back (network blip) — the client, seeing no response, retries, and without an idempotency key, the server has no way to recognize this as the same logical request, resulting in a genuine duplicate charge — exactly the scenario Payment — Idempotency Implementation's key-based deduplication exists to prevent.