Bulkhead & Rate Limiting

Bulkhead isolation: the ship-compartment analogy, applied to thread pools

A ship's bulkheads are physical compartments that keep one hull breach from flooding the entire vessel — the software pattern borrows the name directly: separate thread pools per downstream dependency, so a slow or hanging call to Service A can't exhaust the threads that requests to Service B also need.

@Bulkhead(name = "paymentService", type = Bulkhead.Type.THREADPOOL)
public CompletableFuture<ChargeResult> chargeCard(ChargeRequest request) {
    return CompletableFuture.supplyAsync(() -> paymentClient.charge(request));
}

Without this isolation, a single shared thread pool serving all downstream calls means a hanging paymentService integration can consume every available thread waiting on it, starving completely unrelated calls to a healthy inventoryService — the classic "one slow dependency takes down the whole application" failure mode, exactly the kind of cascading failure Why Microservices Fail's chatty-chain scenario warns about.

Semaphore-based vs thread-pool-based bulkhead

@Bulkhead(name = "recommendationService", type = Bulkhead.Type.SEMAPHORE)
public List<Product> getRecommendations(String userId) { return recommendationClient.fetch(userId); }

Thread-pool-based: a genuinely separate, dedicated thread pool per protected call — real isolation (a slow call literally cannot consume threads outside its own pool), at the cost of real overhead (each pool reserves its own threads, and there's a context-switch cost moving work onto a separate pool). Semaphore-based: limits concurrent calls using a permit count (see Concurrent Utilities & Coordination's Semaphore) on the same thread the caller is already running on — much lighter weight (no dedicated pool, no thread hand-off), but weaker isolation, since a hung call still occupies whatever thread it started on rather than being confined to a separate pool. Semaphore-based is Resilience4j's default and generally the right choice for most calls; thread-pool-based earns its extra overhead specifically for genuinely high-risk, hang-prone dependencies where true isolation matters more than the efficiency cost.

The three rate-limiting algorithms

• Token bucket: a bucket refills at a steady rate up to a capacity; each request consumes a token. Allows bursts up to the bucket size, then throttles to the steady refill rate — the implementation covered fully in In-Memory Rate Limiter.
• Sliding window: tracks request timestamps within a trailing time window, more accurately reflecting "requests in the last N seconds" than a fixed window — more accurate, at the cost of more memory/computation (tracking individual timestamps or sub-window counts, as opposed to token bucket's single running counter).
• Fixed window: simplest — a counter reset at fixed intervals (e.g. every minute). The known flaw: it allows up to 2x the intended limit in a short burst spanning a window boundary (all requests at the very end of one window, plus all requests at the very start of the next, both technically within their own window's limit but concentrated in a much shorter actual time span).

Where to enforce limits

Client-side throttling: the calling service self-limits its own outbound request rate — cooperative, easily bypassed by a misbehaving or buggy client, useful mainly as a courtesy/backpressure signal rather than a hard control. Gateway-level: centralized enforcement before any backend service sees the request — the standard primary control point, covered in API Gateway and the API Rate Limiting Gateway system design case's distributed-enforcement discussion. Per-service: allows different limits for different, specifically expensive endpoints — often layered underneath a gateway-level limit as defense in depth, not a replacement for it.

Follow-up questions this topic invites — and their answers

Q: Could you combine bulkhead and circuit breaker on the same call, and in what order? A: Yes, and Resilience4j supports composing them — bulkhead typically wraps outside circuit breaker (limiting concurrency first, then applying the breaker's fail-fast logic to whatever calls the bulkhead admits), so the two concerns (concurrency isolation, failure-rate-based fast-failing) compose independently rather than interfering with each other.

Q: Why would fixed window's 2x-burst flaw matter less for some use cases than others? A: For a generous, coarse-grained limit (a daily API quota, say) the boundary-burst effect is a rounding error relative to the overall limit; for a tight, latency-sensitive limit protecting a fragile downstream resource, that same 2x burst at a boundary could genuinely overwhelm it — the algorithm choice should match how much headroom the protected resource actually has for a brief overshoot.

Q: How does thread-pool bulkhead interact with virtual threads (see Virtual Threads)? A: Virtual threads change the cost calculus significantly — since virtual threads are cheap to create (unlike platform threads), a dedicated 'thread pool' per dependency becomes much less expensive to maintain, potentially making thread-pool-based bulkhead's isolation benefit available more broadly without its traditional overhead cost being as prohibitive.

Q: What's a concrete symptom that would indicate a missing bulkhead, as opposed to a missing circuit breaker? A: A slow (not fully failing) dependency causing seemingly UNRELATED endpoints to also degrade or time out is the bulkhead-specific symptom — a circuit breaker protects against a dependency's OWN calls failing repeatedly, but doesn't prevent that dependency's slowness from starving a shared thread pool that other, healthy code paths also depend on, which is exactly the gap bulkhead isolation closes.