Deployment, High Availability, Scaling & Caching — Interview Questions

How to use this lesson

These are scenario questions. Structure your answer:

1. Find the bottleneck first.
2. Apply the quick wins.
3. Make the structural changes.
4. Verify.

Interviewers penalise "just split it into microservices" as the answer to a load problem.

Q1. How would you ensure high availability for a Spring Boot e-commerce application at peak times?

Short answer:

• Redundancy everywhere:
  • ≥ 2 instances per service, spread across availability zones, behind a load balancer.
  • A database with a standby or replicas and automatic failover.
  • A clustered Redis and broker.
• Elasticity:
  • Stateless application instances: sessions in Redis, or JWTs.
  • Autoscaling (Kubernetes HPA on CPU, latency or custom metrics).
  • Pre-scale before known peaks (sale events), because autoscaling lags.
• Protect the core:
  • Caching (CDN for static and catalogue content, Redis for hot data).
  • Read replicas for browsing.
  • Queues to absorb order spikes (asynchronous fulfilment).
  • Rate limiting.
  • A virtual waiting room for flash sales.
• Resilience:
  • Timeouts, retries with backoff, and circuit breakers and bulkheads around dependencies (payments, recommendations).
  • Graceful degradation: hide recommendations, but keep checkout working.
• Safe operations:
  • Readiness and liveness probes, and graceful shutdown.
  • Rolling or blue-green or canary deployments, with a code freeze during peaks.
  • Load testing at 2–3× the expected peak, and chaos testing.
  • Dashboards and alerts on the SLOs.

Split into services only where scaling or ownership needs differ (for example, catalogue vs checkout). HA doesn't require microservices.

Learn it in depth → Horizontal vs Vertical Scaling

Q2. Your application suddenly needs to handle twice the load it was designed for. What would you do immediately?

Short answer:

1. Look before you act: find the saturated resource (CPU, the database connection pool, a slow dependency, GC or memory, threads) from metrics and traces.
2. Scale out the stateless tier: more instances behind the load balancer. It only helps if the bottleneck isn't downstream.
3. Protect the database:
   • Size the connection pools so that instances × pool ≤ database capacity. Scaling the application can overload the database.
   • Add caching for hot reads.
   • Route reads to replicas.
   • Fix the top slow queries and indexes.
4. Shed or defer load:
   • Rate-limit abusive or low-priority clients.
   • Move non-critical work to queues.
   • Turn off expensive features with feature flags.
5. Tune the runtime: JVM heap and GC, Tomcat thread and connection limits, timeouts, virtual threads for I/O-bound work.
6. Afterwards: capacity planning and load tests, autoscaling policies, and architectural fixes (read models, sharding) for sustained growth.

Learn it in depth → Load Balancing

Q3. What are the ways to deploy a Spring Boot application?

Short answer:

1. An executable fat JAR with an embedded server: java -jar app.jar, run as a systemd service, or on a PaaS (Render, Heroku-style, Elastic Beanstalk, Azure App Service, Cloud Run).
2. A WAR deployed to an external servlet container (Tomcat, Jetty, or an application server).
3. Containers: a Dockerfile, or Cloud Native Buildpacks (./mvnw spring-boot:build-image), run on Kubernetes, ECS or Docker Compose.
4. A GraalVM native image (native profile): a millisecond startup and a low memory footprint, good for serverless and scale-to-zero, at the cost of longer builds and reflection configuration.
5. Serverless functions (Spring Cloud Function on AWS Lambda or Azure Functions).

Key points to cover:

• Deployment strategies on top of these: rolling, blue-green, canary.
• CRaC (checkpoint/restore) and class data sharing or AOT caches speed up JVM startup without going native.

Q4. How does Spring Boot simplify deployment compared with traditional Spring applications?

Short answer: Traditional Spring meant building a WAR, then installing, configuring and tuning an external server per environment, often with server-specific datasources and JNDI. Boot produces one self-contained executable JAR: embedded server, dependencies and configuration defaults included. It adds:

• externalised configuration per environment (the same artifact everywhere);
• Actuator health checks for orchestrators;
• graceful shutdown;
• layered JARs and buildpacks for efficient container images.

The result is "build once, run anywhere with java -jar", which fits CI/CD and containers.

Q5. What's the difference between embedded and external application-server deployment?

Short answer:

• Embedded: the server (Tomcat, Jetty, Undertow or Netty) is a library inside your application. The application starts the server: one application per process, and the server version is pinned in your build.
• External: you build a WAR. The server starts your application (through SpringBootServletInitializer), possibly alongside other applications, and the server version, configuration and lifecycle are managed separately by operations.

Q6. What are the pros and cons of an embedded server?

Short answer:

• Pros:
  • Simple, self-contained deployment.
  • The same runtime from a developer's laptop to production.
  • The server version is versioned with the code (upgrades go through CI).
  • Configured through properties (server.*) and Java.
  • Ideal for containers and microservices.
  • Faster startup and easier horizontal scaling.
• Cons:
  • Each application carries its own server: more memory than several applications sharing one server, though that matters less with containers.
  • Server patching means rebuilding and redeploying every application. That's a security process question: keep dependencies updated, and automate it.
  • Organisations with centrally managed application servers (shared JNDI resources, a server-level security realm, or licensing) may require WARs.
  • Some server-specific features need programmatic customisation (WebServerFactoryCustomizer).

Q7. You need to move an application from embedded Tomcat to an external Tomcat. What steps would you follow?

Short answer:

1. Packaging: <packaging>war</packaging>, or the Gradle war plugin.
2. Mark the embedded Tomcat as provided, so it's not bundled but is still available at compile time and for java -jar in development:

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-tomcat</artifactId>
  <scope>provided</scope>
</dependency>

3. Bootstrap through the container:

@SpringBootApplication
public class ShopApplication extends SpringBootServletInitializer {
    @Override
    protected SpringApplicationBuilder configure(SpringApplicationBuilder builder) {
        return builder.sources(ShopApplication.class);
    }
    public static void main(String[] args) { SpringApplication.run(ShopApplication.class, args); }   // still runnable locally
}

4. Check compatibility:
   • The external Tomcat must match the Servlet/Jakarta version. Boot 3 needs Tomcat 10.1+ (Jakarta EE 10). Tomcat 9 (javax) won't work.
   • Use the same Java version.
5. Move the configuration:
   • server.* properties (port, context path, SSL, compression) no longer apply. Configure them in Tomcat's server.xml and context.xml. The context path comes from the WAR name, or ROOT.war.
   • Supply the externalised configuration through environment variables, -D flags in setenv.sh, or JNDI.
6. Deploy to webapps/, or through the manager or CI. Verify health, logging (the server's logging vs the application's) and graceful shutdown behaviour.

Common trap: the source says to "remove" the embedded Tomcat dependency. Mark it provided instead. Removing it breaks compilation against servlet APIs, and local runs.

Q8. Your application is read-heavy and needs efficient caching. Which caching solutions would you consider?

Short answer: Layer the caches:

1. HTTP and CDN caching for public, cacheable responses (Cache-Control, ETag): catalogue pages, images.
2. A local in-process cache: Caffeine. Nanosecond access, size- and time-bounded. Good for reference data (categories, configuration, exchange rates), where small staleness per instance is acceptable.
3. A distributed cache: Redis (or Hazelcast). Shared across instances, survives application restarts, supports TTLs and eviction policies. Good for product details, sessions and computed results.
4. A two-level cache (Caffeine in front of Redis), for the hottest keys.

All of these are used through Spring's cache abstraction (@EnableCaching, @Cacheable, @CachePut, @CacheEvict), which makes the provider a configuration choice.

@Cacheable(cacheNames = "product", key = "#id", unless = "#result == null")
public ProductDto get(UUID id) { ... }

@CacheEvict(cacheNames = "product", key = "#cmd.id()")
@Transactional
public void update(UpdateProductCommand cmd) { ... }       // or evict after commit, via an event

Key points to cover:

• Cache-aside is the default pattern. Decide the TTLs and the invalidation strategy (evict on write, events over pub/sub for local caches).
• Protect against stampedes (sync = true, request coalescing, jittered TTLs), and penetration (cache negative results briefly).
• Measure the hit ratio (Micrometer cache metrics).

Learn it in depth → Caching Strategies

Q9. What are the disadvantages of Spring Boot's default caching?

Short answer: Without a cache library on the classpath, Boot uses ConcurrentMapCacheManager: a plain ConcurrentHashMap per cache. It has:

• no TTL or expiry;
• no size limit or eviction, so it grows until an OutOfMemoryError;
• no statistics;
• no sharing between instances: each node has its own copy, so after an update, different instances serve different data;
• everything is lost on restart (a cold cache on every deployment).

It's fine for demos and tests. In production, use Caffeine (local, with bounds and TTLs) or Redis (distributed).

Key points to cover:

• Caching annotations don't work on self-invocation, like any proxy-based feature.
• Cached mutable objects can be modified by callers. Cache immutable DTOs.

Q10. You're moving the application to Docker. What changes would you make to the deployment process?

Short answer:

1. Build the image reproducibly:
   • A multi-stage Dockerfile, or buildpacks (spring-boot:build-image).
   • Layered JARs, so dependencies are cached in their own layer and code changes rebuild only a thin layer.
   • A small base image (a JRE, distroless or Alpine), running as a non-root user.
2. Make the application container-friendly:
   • Configuration through environment variables and secrets, with no environment-specific images.
   • Log to stdout.
   • Container-aware memory (-XX:MaxRAMPercentage=75).
   • Graceful shutdown (server.shutdown=graceful), with SIGTERM handled.
   • Health endpoints for liveness and readiness.
3. CI/CD:
   • Build, scan the image (Trivy or Grype), and push to a registry with immutable tags (the Git SHA).
   • Deploy through Compose (for local development and tests), or Kubernetes manifests or Helm, with rolling updates.
   • Testcontainers for integration tests against real dependencies.
4. Operate:
   • Resource requests and limits, autoscaling, and centralised logs and metrics.
   • No state in the container: use volumes or external stores.

FROM eclipse-temurin:21-jdk AS build
WORKDIR /app
COPY . .
RUN ./mvnw -q package -DskipTests && java -Djarmode=tools -jar target/app.jar extract --layers --launcher --destination extracted

FROM eclipse-temurin:21-jre
RUN useradd -r -u 1001 app
WORKDIR /app
COPY --from=build /app/extracted/dependencies/ ./
COPY --from=build /app/extracted/spring-boot-loader/ ./
COPY --from=build /app/extracted/snapshot-dependencies/ ./
COPY --from=build /app/extracted/application/ ./
USER app
ENV JAVA_TOOL_OPTIONS="-XX:MaxRAMPercentage=75"
EXPOSE 8080
ENTRYPOINT ["java", "org.springframework.boot.loader.launch.JarLauncher"]

Learn it in depth → Multi-Stage Builds

Follow-up questions this topic invites — and their answers

Q: What is graceful shutdown, and why does it matter for deployments? A: With server.shutdown=graceful, the server stops accepting new requests, and in-flight ones finish within spring.lifecycle.timeout-per-shutdown-phase. Combined with readiness going down first (plus a short pre-stop delay in Kubernetes), rolling deployments drop no requests.

Q: How do you size database connection pools when autoscaling? A: The total connections equal the maximum replicas × the pool size. They must stay under the database's limit, with headroom. Use modest pools (HikariCP: roughly cores × 2 on the database side, divided across instances), or a proxy (PgBouncer, RDS Proxy), and alert on pool wait time.

Q: Blue-green vs canary? A: Blue-green switches all traffic between two full environments: an instant rollback, but double the capacity. Canary shifts a small percentage of traffic to the new version, and watches the metrics before increasing it: safer for risky changes, but it needs traffic splitting and good observability.

Q: Why are sticky sessions a problem for scaling? A: They tie users to instances, which gives uneven load, lost sessions when an instance dies, and harder deployments. Externalise the sessions (Spring Session with Redis), or use stateless tokens.