Probes & Autoscaling

Three distinct probes, three distinct consequences

livenessProbe:
  httpGet: { path: /actuator/health/liveness, port: 8080 }
  periodSeconds: 10
readinessProbe:
  httpGet: { path: /actuator/health/readiness, port: 8080 }
  periodSeconds: 5
startupProbe:
  httpGet: { path: /actuator/health/liveness, port: 8080 }
  failureThreshold: 30
  periodSeconds: 2

This is Kubernetes' concrete implementation of the liveness/readiness distinction from the Health Checks lesson — but Kubernetes adds a THIRD probe type with its own distinct purpose:

• Liveness probe: failing it triggers a container RESTART. Kubernetes concludes the process is stuck/broken and the only fix is killing and restarting it.
• Readiness probe: failing it removes the pod from the SERVICE's endpoint list (no traffic routed to it) WITHOUT restarting anything — exactly matching Health Checks' readiness semantics.
• Startup probe: exists specifically to protect SLOW-STARTING containers — while the startup probe hasn't yet succeeded, the liveness and readiness probes are NOT executed at all, preventing a legitimately-slow-to-initialize application (a large in-memory cache warmup, a slow framework startup) from being killed by liveness checks before it's even had a chance to start.

Why the startup probe matters concretely

Without a startup probe, a liveness probe configured with a normal steady-state timeout (e.g. failing after 10 seconds of no response) would repeatedly kill a container that simply takes 45 seconds to fully initialize — an endless restart loop where the container is killed just before it would have become ready, every single time. The startup probe's higher failureThreshold (tolerating many more failed checks before giving up) specifically accommodates this slow-start window, handing off to the normal liveness/readiness probes only once startup genuinely completes.

Horizontal Pod Autoscaler (HPA)

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
spec:
  scaleTargetRef: { kind: Deployment, name: order-service }
  minReplicas: 3
  maxReplicas: 20
  metrics:
    - type: Resource
      resource: { name: cpu, target: { type: Utilization, averageUtilization: 70 } }

The HPA automatically adjusts a Deployment's replica count based on an observed metric — CPU/memory utilization by default, or a custom metric (e.g. request queue depth, directly connecting to Back-of-Envelope Estimation's read:write-ratio-driven scaling reasoning) for more workload-specific autoscaling. This is the Kubernetes-native implementation of the auto-scaling concept introduced in HLD Fundamentals Refresher's scaling deep-dive — adding/removing replicas based on live demand rather than provisioning statically for peak load year-round.

Resource requests vs limits — a critical, often-confused distinction

resources:
  requests: { cpu: "250m", memory: "256Mi" }   # SCHEDULING guarantee
  limits:   { cpu: "500m", memory: "512Mi" }   # HARD CAP

• Requests: what the pod is GUARANTEED to get, and what the SCHEDULER uses to decide which node has room for this pod — the scheduler will never place a pod on a node that can't satisfy its requests.
• Limits: the HARD CAP the pod is never allowed to exceed. Exceeding a memory limit gets the container OOMKilled (the exact scenario diagnosed via kubectl describe pod in Core Objects); exceeding a CPU limit gets the container THROTTLED (slowed down), not killed — CPU and memory limits behave meaningfully differently when exceeded.

Setting requests too low relative to actual usage risks the scheduler over-packing a node (many pods each 'requesting' little but collectively using much more, leading to real resource contention); setting limits too low risks the application being OOMKilled or throttled under entirely normal load. Getting both right requires actually observing real resource usage (Metrics & Monitoring) rather than guessing — and, as Core Objects covered, a pod with NO requests/limits set at all gets the WORST QoS class and is evicted first under any node pressure, making "just don't set them" the worst available option, not a safe default.

Follow-up questions this topic invites — and their answers

Q: Can a startup probe and a liveness probe check the same endpoint? A: Yes, commonly they do (as shown above, both hitting /liveness) — what differs is the TOLERANCE configuration (failureThreshold, periodSeconds), not necessarily the endpoint itself; the startup probe is fundamentally a more PATIENT version of the same check, active only during the startup window.

Q: Does HPA scaling happen instantly when load increases? A: No — HPA polls metrics periodically (not continuously) and, once it decides to scale, the new pods still need to actually start and pass their readiness probe before serving traffic, meaning there's a real, non-trivial lag between a load spike and additional capacity actually coming online — this is exactly why over-provisioning some baseline headroom (minReplicas set above the bare minimum) is common practice for latency-sensitive services, rather than relying on HPA alone to react instantly.

Q: Why does CPU throttling instead of killing make sense, while memory OOMKill does not use the same approach? A: CPU is a COMPRESSIBLE resource (a process can simply run slower without failing outright), while memory is INCOMPRESSIBLE (a process literally cannot function with less memory than it needs at a given moment) — this fundamental difference is why Kubernetes throttles CPU overages gracefully but has no equivalent graceful option for memory overages beyond killing the process.

Q: How would a service known to have unpredictable, bursty load size its HPA differently from one with steady load? A: A bursty service typically wants a lower averageUtilization target (scaling out earlier, before utilization gets dangerously high) and a wider gap between minReplicas and maxReplicas to absorb spikes, while a steady-load service can run closer to its target utilization with less headroom — directly connecting back to Back-of-Envelope Estimation's peak-vs-average traffic reasoning, now expressed as concrete HPA configuration.