Tracing, Discovery, Load Balancing, Service Mesh & Resilience — Interview Questions

How to use this lesson

These questions ask "what have you actually used?", so name concrete tools, and today's versions:

• Spring Cloud Sleuth is retired. Boot 3 uses Micrometer Tracing.
• On Kubernetes, discovery and load balancing are built in: Services and DNS.

Then show you understand the mechanics underneath: trace context propagation, the circuit-breaker states, and client-side vs server-side discovery.

Q1. How do you implement tracing in a microservices architecture?

Short answer: Use distributed tracing:

• Each request gets a trace ID, and each unit of work a span ID.
• They're propagated across service calls in W3C traceparent headers (and in message headers for Kafka).
• Spans are exported to a tracing backend, which shows the whole call tree with timings.

In Spring Boot 3:

• Add micrometer-tracing-bridge-otel (or -brave) plus an exporter: OTLP to an OpenTelemetry Collector → Jaeger/Tempo, or directly to Zipkin.
• Boot instruments RestClient/WebClient/RestTemplate (when built from the auto-configured builders), MVC/WebFlux, @Scheduled, Kafka (with observation enabled), JDBC (through datasource-micrometer), and more.
• Trace IDs appear in the MDC, so logs are correlated.

management:
  tracing:
    sampling:
      probability: 0.1            # 10% in production; 1.0 in development. Or tail-based sampling in the collector
  otlp:
    tracing:
      endpoint: http://otel-collector:4318/v1/traces
logging:
  pattern:
    correlation: "[${spring.application.name:},%X{traceId:-},%X{spanId:-}] "

Common trap: answering with Spring Cloud Sleuth. It doesn't support Boot 3; its functionality moved to Micrometer Tracing.

Key points to cover:

• Add custom spans for business steps with the Observation API (@Observed).
• Use tail-based sampling in the collector, to keep all error and slow traces.

Learn it in depth → Distributed Tracing

Q2. How can you track the flow of requests across multiple microservices?

Short answer: Combine the three pillars, linked together by the trace ID:

1. Traces (Jaeger, Zipkin, Tempo): the request's path, per-hop latency, and which service errored.
2. Correlated logs: every log line carries traceId, so you can jump from a trace to its logs in Kibana or Loki.
3. Metrics with exemplars: a latency spike in Grafana links to example traces.

The ID is created at the edge (the gateway), or accepted from the client. It's propagated automatically by the instrumented clients, and by message headers for asynchronous hops, so a Kafka consumer's span continues the producer's trace.

Key points to cover:

• Return the trace ID in error responses, or a response header, so support can find the trace from a customer's report.

Q3. Which service discovery tools have you implemented? What configuration is needed in the application or the Kubernetes cluster?

Short answer: Name the two you've used, with their setup:

• Netflix Eureka (Spring Cloud Netflix):
  • Run a Eureka server (@EnableEurekaServer, clustered for HA).
  • Services add spring-cloud-starter-netflix-eureka-client, set spring.application.name, and set eureka.client.service-url.defaultZone.
  • Instances register themselves and send heartbeats. Clients use @LoadBalanced builders.
  • On Kubernetes, you usually don't need it.
• HashiCorp Consul:
  • Add spring-cloud-starter-consul-discovery, and set spring.cloud.consul.host/port.
  • Health checks through Actuator. It also provides key/value configuration.
  • On Kubernetes, install it with Helm (Consul servers + clients or dataplane), optionally with its service mesh.
• Kubernetes-native (the most common today):
  • A Service gives a stable DNS name (http://inventory.shop.svc.cluster.local), and kube-proxy balances across the ready pods.
  • The application config is just the URL. Pods need readiness probes, so only healthy ones receive traffic.
  • Spring Cloud Kubernetes (DiscoveryClient backed by the Kubernetes API) is optional. It needs RBAC permissions to list Services and Endpoints.

# Eureka client
spring:
  application:
    name: order-service
eureka:
  client:
    service-url:
      defaultZone: http://eureka-1:8761/eureka,http://eureka-2:8761/eureka
  instance:
    prefer-ip-address: true

Learn it in depth → Service Discovery

Q4. How would you implement service discovery? What's the difference between client-side and server-side discovery?

Short answer:

• Client-side discovery:
  • The client queries the registry (Eureka or Consul), gets the instance list, and picks one itself, with a client-side load balancer (Spring Cloud LoadBalancer).
  • Pros: no extra hop; smart, client-specific balancing (zone affinity, weighted).
  • Cons: every client needs discovery logic, which is language-specific, and it's coupled to the registry.
• Server-side discovery:
  • The client calls a stable address (a load balancer, a Kubernetes Service, or a gateway). That component consults the registry and routes the request.
  • Pros: clients stay simple, and it's language-agnostic.
  • Cons: an extra hop, and the load balancer must be highly available. Kubernetes Services, AWS ALB and a service mesh are examples.

Key points to cover:

• Registration can be self-registration (the service registers itself: the Eureka client) or third-party (the platform registers it: Kubernetes endpoints, or a Consul sync).
• Health checks remove dead instances. Client caches make discovery resilient to short registry outages.

Q5. Which load balancer do you use, and how do you configure it with Kubernetes or the cloud?

Short answer: There are layers of load balancing:

1. Inside the cluster, a Kubernetes Service (ClusterIP) balances across the ready pods. That's L4, through kube-proxy or eBPF.
2. North-south traffic: a cloud load balancer (AWS ALB/NLB, GCP LB, Azure) in front of an ingress layer: the NGINX Ingress Controller or Traefik, or, increasingly, the Gateway API implementations (Envoy Gateway, Istio, cloud gateway controllers).

Steps for NGINX Ingress:

1. Install the controller with Helm. Its Service is type: LoadBalancer, which provisions the cloud load balancer and a public IP.
2. Create Ingress resources with host and path rules pointing to Services.
3. Configure TLS (cert-manager with Let's Encrypt, or cloud certificates).
4. Tune timeouts, body size, rate limits and sticky sessions through annotations, if needed.
5. Point DNS at the load balancer.
6. Make sure pods have readiness probes and graceful shutdown, so rollouts don't drop connections.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: shop
  annotations:
    nginx.ingress.kubernetes.io/proxy-read-timeout: "30"
spec:
  ingressClassName: nginx
  tls: [{ hosts: [api.shop.example], secretName: shop-tls }]
  rules:
    - host: api.shop.example
      http:
        paths:
          - { path: /orders, pathType: Prefix, backend: { service: { name: order-service, port: { number: 80 } } } }

Key points to cover:

• On a managed cloud, you might use the cloud's native ingress instead: the AWS Load Balancer Controller creating an ALB per Ingress or Gateway, or GKE Gateway.
• The Kubernetes project now steers new setups towards the Gateway API. The community ingress-nginx controller has been retired, so plan for the migration.

Learn it in depth → Services & Ingress

Q6. What is a service mesh?

Short answer: A dedicated infrastructure layer for service-to-service communication. Proxies (Envoy sidecars, or Istio's ambient mode with node-level proxies; Linkerd uses its own micro-proxy) intercept all the traffic, and a control plane configures them. It gives you, without application code changes:

• Security: automatic mTLS with rotating workload identities, and authorisation policies (which service may call which).
• Traffic management: retries, timeouts, circuit breaking, canary and weighted routing, fault injection, mirroring.
• Observability: uniform golden-signal metrics, traces and access logs for every call.

The costs:

• Operational complexity.
• Resource overhead and extra latency (from sidecars).
• A learning curve.
• Debugging through proxies.

It's worth it with many services and polyglot stacks, or strict zero-trust requirements.

Common trap: duplicating retries in both the application (Resilience4j) and the mesh. The retries multiply, and can overload a struggling service. Decide which layer owns which policy.

Learn it in depth → Design Service Mesh Basics

Q7. What do you use for application resilience?

Short answer: Resilience4j (through Spring Cloud CircuitBreaker, or its Spring Boot starter), combining:

• Timeouts (TimeLimiter, plus the HTTP client's connect and read timeouts): the first line of defence. Never wait forever.
• Retries with exponential backoff and jitter, only for transient errors, and idempotent operations.
• A circuit breaker: stop calling a failing dependency, and fail fast.
• Bulkheads (semaphore or thread-pool): limit the concurrent calls per dependency, so one slow service can't exhaust all threads.
• Rate limiters, for protecting yourself or honouring a provider's quotas.
• Fallbacks: cached data, defaults, "pending" states, or degraded features.

At the platform level:

• Kubernetes restarts and reschedules failed pods; readiness removes unhealthy ones.
• Multiple replicas across zones.
• Queues buffer load.
• Chaos testing validates all of it.

Learn it in depth → Circuit Breaker Pattern

Q8. Describe a scenario where a microservice might fail. How would you implement a circuit breaker to keep the system resilient?

Short answer: Scenario: during a sale, the external payment provider slows down to 20 s responses. The Order service's request threads pile up waiting, its connection pool drains, and checkout, and then the whole site, stops responding. That's a cascading failure.

Solution:

1. Time out payment calls (for example at 2 s).
2. Wrap the calls in a circuit breaker:
   • CLOSED: calls pass through; the failure and slow-call rates are measured over a sliding window.
   • Above the threshold (for example 50% failures, or 80% slow calls), it goes OPEN: calls fail immediately, with no waiting.
   • After a wait (for example 30 s), HALF_OPEN lets a few trial calls through. Success closes the breaker; failure reopens it.
3. Fall back: accept the order as PAYMENT_PENDING, queue the payment for asynchronous retry, and tell the user "we'll confirm shortly". Or offer another payment method.
4. Bulkhead the payment calls, so other features keep their threads.
5. Alert on circuit-breaker state changes, and on the fallback rate.

@Service
class PaymentAdapter {
    private final PaymentClient client;
    PaymentAdapter(PaymentClient client) { this.client = client; }

    @CircuitBreaker(name = "payments", fallbackMethod = "deferPayment")
    @Bulkhead(name = "payments")
    @TimeLimiter(name = "payments")                  // requires an async return type
    public CompletableFuture<PaymentResult> authorize(PaymentRequest req) {
        return CompletableFuture.supplyAsync(() -> client.authorize(req));
    }

    private CompletableFuture<PaymentResult> deferPayment(PaymentRequest req, Throwable cause) {
        pendingPayments.enqueue(req);                // retried later by a worker; the order stays PAYMENT_PENDING
        return CompletableFuture.completedFuture(PaymentResult.pending());
    }
}

resilience4j:
  circuitbreaker:
    instances:
      payments:
        sliding-window-type: COUNT_BASED
        sliding-window-size: 50
        failure-rate-threshold: 50
        slow-call-rate-threshold: 80
        slow-call-duration-threshold: 2s
        wait-duration-in-open-state: 30s
        permitted-number-of-calls-in-half-open-state: 5
        record-exceptions: [java.io.IOException, java.util.concurrent.TimeoutException]
        ignore-exceptions: [com.shop.payments.CardDeclinedException]   # business errors must not trip the breaker

Key points to cover:

• An open breaker protects the struggling dependency too: it gets time to recover, instead of retry storms.
• Business errors (card declined) must not count as failures.

Follow-up questions this topic invites — and their answers

Q: What's the order of the Resilience4j decorators, and why does it matter? A: The default aspect order (outermost first) is: Retry → CircuitBreaker → RateLimiter → TimeLimiter → Bulkhead → the call. So retries see the breaker's fast failures, and every retry attempt counts against the breaker. Configure the aspect order if you need different semantics.

Q: How do you propagate trace context through Kafka? A: Spring Kafka with observation enabled (spring.kafka.template.observation-enabled=true, spring.kafka.listener.observation-enabled=true) writes traceparent into the record headers, and continues it in the consumer, so the consumer's span links to the producer's trace.

Q: Eureka vs Kubernetes Services: when would you still use Eureka? A: When services run outside Kubernetes (VMs, mixed estates), or across several clusters and environments without a mesh, or when you need client-side, zone-aware balancing logic. On one Kubernetes platform, native Services plus DNS are simpler.

Q: What's the difference between liveness and readiness in resilience terms? A: Readiness failing removes the pod from the load balancer (temporary: dependencies warming up, overload). Liveness failing restarts the pod (it's stuck). Don't include external dependencies in liveness, or a database outage will restart every pod in a loop.