API Gateway, Circuit Breaker & Retry Patterns — Interview Questions

How to use this lesson

These three patterns form the edge and resilience toolkit. Interviewers probe:

• gateway anti-patterns (business logic in the gateway, a single point of failure);
• the exact circuit-breaker state machine;
• when retries hurt: non-idempotent operations and retry storms.

Answer with Spring Cloud Gateway and Resilience4j specifics.

Q1. What is the API Gateway pattern for?

Short answer: A gateway is a single entry point for all client requests to a microservices system. It:

• routes each request to the right service;
• handles cross-cutting concerns centrally: authentication and token validation, rate limiting, TLS termination, CORS, logging and metrics, request size limits;
• can aggregate responses from several services.

Example: a product page needs product details, reviews and recommendations. The gateway (or a BFF) calls the three services in parallel, and returns one response, so the mobile client makes one round trip instead of three.

Learn it in depth → API Gateway

Q2. What is the Circuit Breaker pattern for?

Short answer: It stops a service from repeatedly calling a dependency that's failing or too slow:

• After a failure threshold, it fails fast, for a cool-down period, often with a fallback.
• That prevents cascading failures (thread and connection exhaustion in the caller), and gives the dependency time to recover.

Example: the payment gateway is down, so the breaker opens, and checkout queues payments for later, instead of hanging every request.

Learn it in depth → Circuit Breaker Pattern

Q3. What is the Retry pattern for?

Short answer: It automatically re-attempts a failed operation when the failure is likely transient: a network blip, a timeout, a 503 during a deployment, a lost connection, or a rate-limit response with a Retry-After. Example: the Inventory service briefly fails to respond while checking stock, so the call is retried twice with backoff before being treated as failed.

Learn it in depth → Retry & Backoff Strategies

Q4. What is the API Gateway pattern, and why is it important in microservices?

Short answer: Without a gateway, clients must know every service's address, API and authentication scheme. Every service must re-implement the edge concerns, and internal refactoring breaks clients. The gateway:

• decouples clients from the internal topology (services can be split, merged or moved);
• centralises edge policy (one place for authentication, quotas and TLS);
• reduces client round trips (aggregation);
• gives one observability point for all external traffic.

Key points to cover:

• Implementations: Spring Cloud Gateway, Kong, NGINX or Envoy-based gateways, and cloud offerings (AWS API Gateway, Azure APIM, Apigee).
• The Backend-for-Frontend (BFF) variant gives each client type its own gateway, tailored to its needs.

spring:
  cloud:
    gateway:
      routes:
        - id: orders
          uri: lb://order-service                       # discovery + load balancing
          predicates: [ "Path=/api/orders/**" ]
          filters:
            - StripPrefix=1
            - name: RequestRateLimiter                   # Redis token bucket per key
              args: { redis-rate-limiter.replenishRate: 50, redis-rate-limiter.burstCapacity: 100, key-resolver: "#{@userKeyResolver}" }
            - name: CircuitBreaker
              args: { name: orders, fallbackUri: "forward:/fallback/orders" }
            - TokenRelay=                                # forward the user's OAuth2 token downstream

Q5. How does an API Gateway improve security and performance?

Short answer:

• Security:
  • Terminates TLS.
  • Validates tokens (JWT or OAuth2) and rejects anonymous traffic before it reaches services.
  • Enforces rate limits and quotas (brute force, scraping, DoS), plus request size and schema validation.
  • Maintains IP allow- or deny-lists, and integrates with a WAF.
  • Hides internal endpoints and topology.
  • Strips or normalises dangerous headers, and adds security headers.
• Performance:
  • Response caching for cacheable GETs.
  • Aggregation and parallel fan-out, which means fewer client round trips.
  • Compression.
  • HTTP/2 or HTTP/3 to clients, with connection pooling to the backends.
  • Load balancing across instances, and offloading repeated work (auth) from every service.

Common trap: the gateway handles the coarse checks. Services must still authorise each request (object-level checks, zero trust), because internal traffic can bypass the gateway.

Q6. What's the difference between using an API Gateway and letting clients call microservices directly?

Short answer:

Direct calls are acceptable for internal service-to-service traffic (usually through a mesh), or very small systems.

Q7. What are the common challenges of implementing an API Gateway?

Short answer:

• A single point of failure or bottleneck: run several instances across zones, autoscale, and keep it stateless. Rate-limit state belongs in Redis.
• Added latency: keep the filters lightweight, use non-blocking gateways (Spring Cloud Gateway on Netty, or Envoy), and pool the connections.
• "God gateway" coupling: business logic or orchestration creeping in means every team changes the gateway, and it becomes a deployment bottleneck. Keep it to edge concerns only. Put aggregation in BFFs owned by the client teams.
• Configuration sprawl and ownership: manage routes as code, with review. Consider per-team route ownership.
• Consistency with service-side security.
• Observability: trace propagation starts here.
• Versioning and deprecation of public APIs.
• Timeouts: the gateway's timeout must exceed the backend's, including its retries, or you get phantom failures.

Q8. What is the Circuit Breaker pattern, and how does it improve resilience?

Short answer: The breaker wraps calls to a dependency, and tracks their outcomes (failures and slow calls) in a sliding window. When the failure rate crosses a threshold, it opens, rejecting calls immediately (CallNotPermittedException), which triggers a fallback. It improves resilience because:

1. The caller fails in milliseconds instead of waiting for timeouts, so threads and connections stay free for other work.
2. The failing dependency gets breathing room, instead of a flood of requests.
3. Failures don't cascade up the call chain.
4. It provides a clear signal (the breaker state) for monitoring and alerting.

Q9. Explain the circuit breaker's states: closed, open and half-open.

Short answer:

• CLOSED (normal): calls pass through, and results are recorded. If the failure rate or slow-call rate in the window reaches the threshold (after a minimum number of calls), it moves to OPEN.
• OPEN: all calls are rejected immediately (the fallback runs) for waitDurationInOpenState. Then it moves to HALF_OPEN, automatically or on the next call.
• HALF_OPEN (probing): a limited number of trial calls are allowed:
  • If they succeed (below the threshold), it goes back to CLOSED, and the metrics reset.
  • If they fail, it goes back to OPEN for another wait.

Key points to cover:

• Resilience4j also has the special states DISABLED (always allow), FORCED_OPEN (always reject) and METRICS_ONLY, useful for operations and incidents.
• The source's wording "before fully reopening or closing" means: a half-open breaker returns to open on failure, or closes on success.

CircuitBreaker cb = CircuitBreaker.of("inventory", CircuitBreakerConfig.custom()
        .slidingWindowSize(20).minimumNumberOfCalls(10)
        .failureRateThreshold(50).slowCallRateThreshold(80).slowCallDurationThreshold(Duration.ofSeconds(1))
        .waitDurationInOpenState(Duration.ofSeconds(15)).permittedNumberOfCallsInHalfOpenState(3)
        .build());
cb.getEventPublisher().onStateTransition(e -> log.warn("inventory breaker {}", e.getStateTransition()));
Supplier<Stock> guarded = CircuitBreaker.decorateSupplier(cb, () -> inventoryClient.stock(sku));

Q10. How does the Circuit Breaker pattern differ from the Retry pattern?

Short answer: They handle different kinds of failure:

• Retry is optimistic: it assumes the failure is momentary, and tries again soon, to turn a transient error into a success for this request.
• The circuit breaker is pessimistic: it assumes the failure is persistent, and stops trying for a while, to protect the system (caller and callee) across all requests.

Retry on its own against a hard-down service multiplies the load: every request becomes 3–4 requests, which is a retry storm. The breaker on its own gives up on the first blip. Combined, they give quick recovery from blips, and fast failure during outages.

Q11. When would you use a circuit breaker in a microservices architecture?

Short answer: On every remote call where a slow or failing dependency could hurt the caller:

• External third-party APIs (payment, SMS, shipping carriers, maps), which you don't control.
• Internal services in synchronous chains on the critical path.
• Shared infrastructure calls (a search cluster, a legacy system) that can degrade.
• Calls with a meaningful fallback (cached data, a default, a queue for later, a degraded UI).

It's not useful for local in-memory calls, and it's less relevant for asynchronous message consumption, where you pause consumers or use backoff instead. Business errors (validation failures, card declined) must not trip the breaker.

Q12. What is the Retry pattern, and when should you use it?

Short answer: Use it for transient failures on idempotent operations:

• Retry: connection resets, timeouts (only if the operation is idempotent, or carries an idempotency key), HTTP 502/503/504, 429 with Retry-After, deadlocks or serialisation failures in the database, leader elections in a broker.
• Don't retry:
  • 4xx client errors (400, 401, 403, 404, 422), which will fail again;
  • non-idempotent POSTs without an idempotency key (you'll double-charge);
  • when the caller's own deadline has almost expired.

RetryConfig config = RetryConfig.custom()
        .maxAttempts(3)                                                        // 1 call + 2 retries
        .intervalFunction(IntervalFunction.ofExponentialRandomBackoff(Duration.ofMillis(200), 2.0, 0.5))   // backoff + jitter
        .retryOnException(ex -> ex instanceof IOException || ex instanceof HttpServerErrorException.ServiceUnavailable)
        .ignoreExceptions(HttpClientErrorException.class)
        .build();

Q13. How does the Retry pattern improve fault tolerance?

Short answer: Many distributed failures are brief and self-healing: packet loss, a pod restarting during a rolling deployment, a leader failover, a momentary pool exhaustion. Retrying with a short backoff masks these blips from users and upstream services. That means:

• fewer error responses and fewer manual interventions;
• higher effective availability;
• smoother deployments.

Key points to cover:

• It only helps if retries are bounded, delayed and targeted. Otherwise it adds latency, and makes overloads worse.
• Add retries at one layer only (the client library, the application, or the mesh). Retries at several layers multiply: 3 × 3 × 3 = 27 attempts.

Q14. What's the relationship between the Retry and Circuit Breaker patterns?

Short answer: They're complementary. The usual composition is Retry outside, Circuit Breaker inside (Resilience4j's default aspect order):

• Retry handles an individual transient failure.
• Every attempt is recorded by the breaker. When failures persist, the breaker opens, and later retries fail fast with CallNotPermittedException. Configure the retry to not retry that exception, so it stops immediately.
• The time limiter bounds each attempt, and the overall deadline bounds the whole retry sequence.

Supplier<Stock> call = () -> inventoryClient.stock(sku);
Supplier<Stock> resilient = Decorators.ofSupplier(call)
        .withCircuitBreaker(circuitBreaker)          // inner: records every attempt
        .withRetry(retry)                            // outer: re-invokes on transient failures
        .withFallback(List.of(CallNotPermittedException.class, IOException.class), ex -> Stock.unknown(sku))
        .decorate();

Q15. What strategies limit retries, and avoid overwhelming downstream services?

Short answer:

• Bounded attempts: 2–3 at most, and an overall deadline or timeout budget propagated downstream.
• Exponential backoff: 200 ms, 400 ms, 800 ms…, with a maximum delay.
• Jitter: randomise the delays (full or decorrelated jitter), so thousands of clients don't retry in sync (the thundering herd).
• Retry budgets: allow retries only while they stay under a percentage of total traffic (for example 10%). Envoy, gRPC and Finagle support this.
• A circuit breaker stops retries during sustained outages.
• Honour Retry-After and the server's back-pressure signals (429, 503).
• Retry at one layer only.
• Retry only idempotent operations, or use idempotency keys.
• For asynchronous work: move to a delayed retry queue or topic (@RetryableTopic) with increasing delays, and finally a DLQ, instead of blocking.

Follow-up questions this topic invites — and their answers

Q: Where should aggregation live: the gateway or a BFF? A: In a BFF owned by the client team (web BFF, mobile BFF), or a dedicated composition service. The shared gateway should stay thin (routing plus edge policies), so it doesn't become a coupled bottleneck.

Q: How do you rate-limit fairly at the gateway? A: Key the limits by the authenticated user, API key or tenant, not only by IP. Use a distributed token bucket (Redis) for consistency across gateway instances, return 429 with Retry-After, and have separate tiers for different plans.

Q: What happens to in-flight retries when the circuit opens? A: The next attempt gets CallNotPermittedException immediately. Configure the retry to ignore that exception, so the whole operation fails fast (or falls back), instead of sleeping through backoff delays pointlessly.

Q: How do you choose timeout values? A: From the dependency's observed latency (for example, a bit above its p99), and the caller's own SLO budget, minus the time needed for retries. Downstream timeouts must be shorter than upstream ones, so that failures propagate cleanly.