Deployment, Scaling & Security for Microservices — Interview Questions

How to use this lesson

Operational questions separate people who have run microservices from people who have only built them. Show that you know:

• what the platform does vs what the application must do;
• that scaling decisions start with measuring the bottleneck;
• that security is zero trust: authenticate every hop.

Q1. You're designing a microservices architecture, and service failures must not take down the whole system. What strategies would you use?

Short answer: Isolate, degrade and recover:

• Contain failures at call sites: timeouts on every remote call, circuit breakers, bulkheads, retries with backoff (for idempotent calls only), and fallbacks (cached or default data, feature degradation).
• Decouple in time: use asynchronous messaging for non-critical paths, so a down consumer just builds up a backlog instead of failing requests.
• Redundancy: multiple replicas across availability zones, with no single points of failure (a highly available database, broker and gateway).
• Self-healing: Kubernetes restarts crashed pods (liveness), stops routing to unready ones (readiness), and reschedules pods off failed nodes. PodDisruptionBudgets protect availability during maintenance.
• Protect against overload: rate limiting at the gateway, load shedding, autoscaling, and bounded queues.
• Limit the blast radius: separate data stores; cell-based or per-tenant partitioning for large systems; canary releases with automatic rollback.
• Verify it: chaos engineering (kill pods, inject latency), load tests, and SLO-based alerting.

Q2. What's the difference between Docker and Kubernetes?

Short answer:

• Docker is a container toolchain:
  • It builds images (Dockerfile or BuildKit), and runs containers on one host (Docker Engine, on containerd).
  • It distributes images through registries.
  • Docker Compose runs multi-container applications on a single machine.
• Kubernetes is a container orchestrator for a cluster of machines:
  • scheduling;
  • desired-state reconciliation (Deployments, ReplicaSets);
  • self-healing;
  • service discovery and load balancing (Services);
  • rolling updates and rollbacks;
  • autoscaling (HPA/VPA, cluster autoscaler);
  • configuration and secrets;
  • storage orchestration.

Key points to cover:

• Kubernetes runs OCI images, built by Docker or by other tools. It removed the "dockershim", and uses containerd or CRI-O directly, but Docker-built images work unchanged.

Learn it in depth → Core Objects

Q3. When deploying a microservices application, how would you decide between Docker, Kubernetes, or both?

Short answer: You almost always build with Docker (or buildpacks or Jib). The question is what runs the containers:

• Docker (Compose) alone: local development, CI test environments, or a small, single-host deployment with a few services and low availability needs.
• Kubernetes: many services; HA across nodes and zones; autoscaling; rolling or canary deployments; a platform team; or a multi-team organisation.
  • A managed service (EKS, GKE, AKS) reduces the burden.
• Alternatives to consider:
  • Serverless containers (Cloud Run, AWS App Runner or Fargate/ECS) for small teams that want orchestration benefits without running Kubernetes.
  • A PaaS.

Decision factors: the number of services, scaling and availability requirements, team skills and platform capacity, cost, and portability needs.

Q4. Describe how you dockerised a Spring Boot application: the steps, challenges and benefits.

Short answer:

• Steps:
  1. A multi-stage Dockerfile: build with Maven or Gradle in a JDK stage, then copy the extracted layered JAR into a slim JRE runtime stage. Or use spring-boot:build-image (buildpacks) or Jib, with no Dockerfile at all.
  2. Run as a non-root user. Pin the base image versions.
  3. Externalise configuration through environment variables and secrets; keep one image for all environments.
  4. Log to stdout; expose the health endpoints; enable graceful shutdown.
  5. Build in CI, scan the image, tag it with the Git SHA, and push it to a registry.
• Challenges:
  • JVM memory in containers: use MaxRAMPercentage, and set container limits. OOMKilled pods come from non-heap memory (metaspace, threads, direct buffers).
  • Image size and build time: solved with layering and caching.
  • Startup time vs probes: startup probes, CDS/AOT caches, or native images.
  • Environment-specific configuration baked into images: removed.
  • Timezone and locale, and CA certificates for internal TLS.
  • File-system writes: read-only root file systems, with /tmp volumes.
• Benefits:
  • Identical runtime everywhere, with no "works on my machine".
  • Fast, repeatable deployments and rollbacks (immutable images).
  • Horizontal scaling.
  • Isolation.
  • The foundation for orchestration and GitOps.

Learn it in depth → Docker Fundamentals

Q5. What security practices do you follow when developing microservices?

Short answer:

• Identity and access:
  • A central identity provider (OAuth2/OIDC); every service validates JWTs as a resource server.
  • Least-privilege scopes and roles.
  • Object-level authorisation in each service.
• Encryption in transit: TLS at the edge, and mTLS between services (often through a mesh).
• Secrets: a vault or cloud secret manager, rotated. Never in images, Git or logs.
• Least privilege for workloads:
  • Separate database credentials per service.
  • Kubernetes RBAC, service accounts and NetworkPolicies.
  • Non-root, read-only containers, and Pod Security Standards.
• Supply chain:
  • Dependency scanning (OWASP Dependency-Check, Snyk, Dependabot), image scanning, and SBOMs.
  • Signed images (cosign), and prompt patching.
• Application security: input validation, parameterised queries, output encoding, and safe error messages against the OWASP Top 10.
• Edge protection: gateway authentication, rate limiting, a WAF, CORS.
• Audit and detection: security logging, monitoring, anomaly alerts, and incident runbooks.

Q6. What are the security challenges in microservices, and how would you secure service-to-service communication?

Short answer:

• The challenges:
  • A larger attack surface: many endpoints and network paths.
  • Identity propagation across hops.
  • Secrets sprawl.
  • Inconsistent enforcement across teams and languages.
  • Lateral movement once one service is compromised.
  • Auditing a distributed request.
• The strategies for service-to-service calls:
  1. mTLS everywhere, ideally through a service mesh: automatic certificates, rotation, and SPIFFE workload identities.
  2. Service authentication with OAuth2 client credentials: tokens with audience and scopes per target service. Or token exchange to carry the end user's identity downstream.
  3. Authorisation per service: mesh AuthorizationPolicy rules ("only order-service may call payment-service"), plus in-application checks.
  4. Network segmentation: NetworkPolicies that deny by default. Internal services are not exposed publicly.
  5. Gateway security for external traffic: token validation, rate limiting, request validation.
  6. Short-lived credentials and workload identity (IRSA, GKE Workload Identity), instead of static keys.

Learn it in depth → Design an Authentication System at Scale

Q7. How would you scale one or many microservices? Is application.yml enough, or do you have to configure the cloud environment too?

Short answer: Scaling is mostly a platform concern. application.yml alone is not enough.

Scaling techniques:

• Horizontal scaling: more replicas behind a load balancer. The service must be stateless.
• Autoscaling: a Kubernetes HPA on CPU, memory or custom metrics (request rate, Kafka lag through KEDA). The cluster autoscaler or Karpenter adds nodes.
• Data tier: read replicas, caching (Redis), partitioning or sharding, connection pooling (PgBouncer).
• Asynchronous processing with queues, to smooth peaks.
• A service mesh or gateway for traffic management.

In the application (application.yml):

• Thread and connection pool sizes (Hikari maximum-pool-size, Tomcat threads, or virtual threads).
• Timeouts and resilience settings.
• Graceful shutdown.
• Actuator probes and metrics.
• Externalised configuration, and discovery settings.

In the platform or cloud:

• Replica counts and HPA policies (min, max, targets, scale-down stabilisation).
• Resource requests and limits.
• Readiness, liveness and startup probes.
• PodDisruptionBudgets.
• Anti-affinity and topology spread across zones.
• Load balancer and ingress configuration.
• Node pools.
• Database capacity limits.
• Environment variables and secrets for configuration.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata: { name: order-service }
spec:
  scaleTargetRef: { apiVersion: apps/v1, kind: Deployment, name: order-service }
  minReplicas: 3
  maxReplicas: 20
  metrics:
    - type: Resource
      resource: { name: cpu, target: { type: Utilization, averageUtilization: 65 } }
  behavior:
    scaleDown: { stabilizationWindowSeconds: 300 }

Learn it in depth → Probes & Autoscaling

Q8. One of your microservices is under high load. How would you approach scaling it, and what would you consider?

Short answer:

1. Diagnose the resource:
   • Is it CPU-bound (serialisation, computation)?
   • Memory or GC-bound?
   • I/O-bound: waiting on a database or downstream service?
   • Is the load legitimate, or a retry storm or bot?
2. Match the fix to the cause:
   • CPU: scale out horizontally, and optimise hot code paths.
   • Waiting on I/O: more replicas won't help if the dependency is the bottleneck. Scaling can even make it worse by adding database connections. Fix the queries and indexes, add caching, batch the calls, or scale the dependency.
   • Memory: fix leaks, or tune the heap and container limits, then scale.
3. Scale safely:
   • Check the downstream capacity (database connections = replicas × pool size).
   • Check partition counts for Kafka consumers: you can't have more active consumers than partitions.
   • Check whether the service is stateless.
4. Consider cost vs benefit: vertical scaling for quick relief, horizontal for elasticity, and autoscaling policies for recurring patterns.
5. Protect with rate limiting and load shedding while scaling catches up.
6. Verify with metrics (latency percentiles, saturation), and do a capacity plan for next time.

Follow-up questions this topic invites — and their answers

Q: Why might adding replicas not reduce latency? A: The bottleneck is elsewhere: the database, a lock, a downstream service, or a single Kafka partition (a hot key). Or the pods are CPU-throttled by their limits, or new pods aren't warmed up (JIT, caches). Measure before you scale.

Q: Requests vs limits in Kubernetes: what do you set for a Java service? A: Set memory requests = limits (to avoid OOM surprises and eviction), sized for heap plus non-heap memory. Set a CPU request that reflects normal use. Many teams avoid CPU limits or set them generously, because CFS throttling hurts JVM latency (GC and JIT threads).

Q: How does KEDA help scale event consumers? A: It scales Deployments on external metrics, like Kafka consumer lag, queue depth, or cron schedules, including down to zero. It's more responsive for consumers than CPU-based scaling.

Q: How do you keep secrets out of container images? A: Inject them at runtime (Kubernetes Secrets, preferably synced from a vault through the External Secrets Operator or the CSI driver), use workload identity for cloud APIs, scan images for leaked secrets in CI, and never COPY .env files into the image.