JIT Compilation & Runtime Optimisations — Interview Questions

How to use this lesson

JIT questions test whether you understand why Java is fast after warm-up, and slow before it, and how to reason about micro-benchmarks and latency spikes. Keep the model simple: interpret → profile → compile speculatively → deoptimise when wrong.

Q1. What is the JIT compiler, and when does it kick in?

Short answer: The Just-In-Time compiler turns frequently executed bytecode into optimised native machine code while the program runs. HotSpot starts by interpreting everything. It counts method invocations and loop back-edges. When a method crosses a threshold, it's queued for background compilation, and later calls jump to the compiled code. Long-running loops are compiled mid-execution, through on-stack replacement (OSR).

Key points to cover:

• Only the hot fraction of code (typically a few percent) is ever compiled. That's the "HotSpot" idea: spend compile time where it pays back.

Q2. What's the difference between the C1 and C2 JIT compilers?

Short answer:

• C1 (the client compiler): compiles quickly, with light optimisations (some inlining, and simple register allocation). It can insert profiling code that gathers type and branch statistics.

• C2 (the server compiler): a slow, aggressive, profile-guided optimiser:

  • deep inlining;
  • escape analysis;
  • loop unrolling and vectorisation;
  • range-check elimination;
  • speculative devirtualisation.

  It produces the fastest code.

• (Graal is an alternative JIT written in Java, available in GraalVM and as Oracle GraalVM's JIT.)

Q3. What is tiered compilation, and how does it balance startup against peak performance?

Short answer: Tiered compilation (the default since Java 8) combines both compilers in levels:

• Level 0: interpreter (with profiling).
• Levels 1–3: C1-compiled code:
  • level 1 is C1 without profiling, for trivial methods;
  • level 2 is C1 with limited profiling;
  • level 3 is C1 with full profiling.
• Level 4: C2, fully optimised using the collected profile.

A hot method usually goes 0 → 3 → 4. C1 gives fast startup, and C2 gives peak throughput, using the profiles gathered at level 3.

Tuning: -XX:TieredStopAtLevel=1 gives the fastest startup and lowest compile overhead (good for short CLI tools and dev loops). -XX:-TieredCompilation goes straight to C2 after interpreting.

Q4. What is a "hot spot", and how is it detected?

Short answer: A method or loop that runs often enough to be worth compiling. HotSpot keeps per-method counters: invocation count and back-edge (loop iteration) count. When they exceed tier thresholds (scaled by compiler queue load), the method is submitted for compilation at the next tier. A loop whose back-edge counter overflows triggers OSR compilation, which replaces the running interpreted frame with a compiled one.

Q5. What role does profiling play in JIT optimisations?

Short answer: The interpreter and C1 (at level 3) record, per bytecode:

• receiver types at call sites;
• branch taken/not-taken counts;
• null-seen flags;
• type-check outcomes.

C2 uses this profile to speculate:

• inline the observed receiver type behind a cheap type guard;
• prune never-taken branches, compiling them as "uncommon traps";
• remove null checks.

These speculations are why Java JIT code can beat static compilers on real workloads. They're also why unrepresentative warm-up (in benchmarks, or with rare code paths) produces misleading results.

Q6. What is method inlining, and why does it matter so much?

Short answer: Inlining replaces a call with the callee's body. It removes call overhead, but, more importantly, it exposes the combined code to other optimisations: escape analysis, constant folding, dead-code elimination, lock elision, loop optimisations. It's called the "mother of all optimisations".

The key limits (HotSpot defaults):

• Small methods (≤ 35 bytes of bytecode, MaxInlineSize) are inlined readily.
• Hot methods up to 325 bytes (FreqInlineSize) can be inlined.
• There's also an inlining depth limit.
• Megamorphic call sites can't be inlined.

Key points to cover:

• Small, focused methods help: getters and small helpers disappear after inlining.
• Diagnose with -XX:+UnlockDiagnosticVMOptions -XX:+PrintInlining, or JITWatch.

Q7. What is polymorphic inline caching (PIC)?

Short answer: An optimisation for virtual calls: cache the receiver type(s) seen at a call site, so dispatch avoids a full lookup. HotSpot works per call site, based on the type profile:

• Monomorphic (one type seen): inline that implementation behind a type guard. The call is effectively free.
• Bimorphic (two types): inline both, with a type check.
• Megamorphic (three or more types): fall back to a vtable/itable dispatch, with no inlining. That's much slower in hot loops.

Key points to cover:

• In HotSpot, "polymorphic inline caching" in the Smalltalk or JavaScript sense (caching many types) mostly stops at bimorphic. Beyond that, the call site is megamorphic.
• Practical implication: a hot interface call that sees many implementations (a Function in a generic stream pipeline, or strategy maps) can become megamorphic. Splitting call sites, or specialising the code, can recover performance. Measure with JMH before and after.

Q8. How do loop unrolling, vectorisation and escape analysis work?

Short answer:

• Loop unrolling: C2 duplicates the loop body several times per iteration. That cuts branch and counter overhead, and gives the CPU more independent instructions. It's combined with range-check elimination: array bounds checks are hoisted out of the loop.
• Vectorisation (SuperWord): C2 turns suitable loops over arrays (simple arithmetic on int[], float[] and so on) into SIMD instructions (SSE/AVX/NEON), processing 4–16 elements per instruction. For explicit control, use the Vector API (jdk.incubator.vector, still incubating in recent releases).
• Escape analysis: for objects that don't escape the compiled method (after inlining), C2 can scalar-replace them (fields become locals or registers, so there's no heap allocation) and elide locks on them. HotSpot doesn't literally allocate on the stack. Scalar replacement is its equivalent.

long sum(int[] a) {               // a classic unrolled and vectorised loop after C2 compilation
    long s = 0;
    for (int i = 0; i < a.length; i++) s += a[i];
    return s;
}

Common trap: saying "escape analysis allocates objects on the stack". In HotSpot, it removes the allocation entirely through scalar replacement, when the object doesn't escape and is fully inlined.

Q9. Can JIT-compiled code be deoptimised? Why would the JVM do that?

Short answer: Yes. Compiled code is based on speculative assumptions. When one breaks, the JVM deoptimises: it transfers the running frames back to the interpreter at a safe point, marks the compiled method "not entrant", and later recompiles with the updated profile. Triggers:

• an uncommon trap is hit: a branch the profile said was never taken, or a new receiver type at an inlined call site;
• class hierarchy changes: a class is loaded that overrides a method assumed to be monomorphic (CHA, class hierarchy analysis);
• a null appears where none was seen;
• redefinition by agents or debuggers.

Symptoms: "made not entrant" in -XX:+PrintCompilation, and latency hiccups when rare paths first execute in production.

Q10. What is the code cache, and what happens when it fills up?

Short answer: The code cache is native memory that stores JIT-compiled code (plus adapters and stubs). Since Java 9 it's segmented: non-method code, profiled (C1) code, and non-profiled (C2) code. The default size is 240 MB with tiered compilation (-XX:ReservedCodeCacheSize). When it's full:

• you'll see the warning "CodeCache is full. Compiler has been disabled";
• no new methods are compiled, so newly hot code stays interpreted, which is a sudden, severe slowdown;
• code cache flushing (UseCodeCacheFlushing, on by default) tries to evict cold code.

Fix: increase ReservedCodeCacheSize for very large applications (heavy frameworks, lots of generated code), and monitor it through JMX, NMT or JFR's CodeCacheFull events.

Q11. How does the JVM optimise synchronized blocks at runtime? What happened to biased locking?

Short answer:

• Lock elision: escape analysis proves that the lock object is thread-local, so the lock is removed. For example, StringBuffer used inside one method.
• Lock coarsening: adjacent synchronized blocks on the same object (often in a loop, after inlining) are merged into one lock and unlock, reducing overhead.
• Lightweight (thin) locking: an uncontended lock uses a CAS on the object header (or, since Java 23, a per-thread lock stack), with no OS monitor. Only on contention is the lock inflated to a full monitor (ObjectMonitor) with a waiter queue, after adaptive spinning.
• Biased locking (DXB-599, DXB-619): it used to let the first thread "own" a lock, so re-locking cost almost nothing. When another thread appeared, the JVM had to revoke the bias at a safepoint (a stop-the-world operation), and bulk revocation added pauses. Modern concurrent code rarely benefits from it, and it complicated the JVM. So it was disabled by default in Java 15 (JEP 374), and removed in Java 18.

Key points to cover:

• Since Java 24 (JEP 491), virtual threads no longer pin their carrier thread when blocking inside synchronized, which removes the main reason to replace synchronized with ReentrantLock for Loom.

Q12. What are safepoints?

Short answer: A safepoint is a state in which every Java thread is stopped at a known point, with its stack and object references precisely described (oop maps). That makes it safe for the VM to do global operations:

• some GC phases (root scanning, evacuation, depending on the collector);
• deoptimisation;
• class redefinition;
• thread dumps;
• (historically) biased-lock revocation.

Compiled code polls for safepoint requests at method returns and loop back-edges.

Key points to cover:

• Time-to-safepoint (TTSP) problems: a long counted loop without a poll can delay everyone else. That was mitigated by loop strip mining (Java 10+).
• Thread-local handshakes (Java 10, JEP 312) let many operations stop individual threads, instead of the whole VM.
• Diagnose with -Xlog:safepoint (the time to reach the safepoint, and the time inside it).
• Safepoint bias also makes naive sampling profilers misleading. Prefer JFR or async-profiler.

Q13. What are the disadvantages of JIT compilation, and when would you avoid relying on it?

Short answer:

• Warm-up: the application is slow until the hot code is compiled. It's bad for short-lived processes, autoscaling bursts, and serverless cold starts.
• CPU and memory for compiler threads, and the code cache. Noticeable in small containers.
• Unpredictability: deoptimisations and recompilation cause latency jitter, and performance depends on the workload's profile.
• Harder benchmarking: you need JMH, with warm-up iterations.

What to do instead (rather than "disabling the JIT"):

• -Xint (interpreter only) is almost never right: it's 10–50× slower. Use it only to rule out a suspected JIT bug.
• -XX:TieredStopAtLevel=1 for short-lived tools.
• CDS/AppCDS and AOT caches (see the next question), for faster startup.
• GraalVM native images: AOT-compiled, starting in milliseconds with small memory, but lower peak throughput without PGO, and build-time constraints on reflection.
• CRaC (checkpoint/restore of a warmed-up JVM) on supported distributions.
• Warm up before taking traffic: synthetic requests, and a readiness probe that waits for warm-up.

Q14. How does Class Data Sharing (CDS) work?

Short answer: CDS stores pre-parsed, pre-verified class metadata in an archive file, which is memory-mapped at startup. That skips parsing and verification, and lets several JVMs share the same read-only pages.

• A default CDS archive for JDK classes has shipped since Java 12.
• AppCDS (open since Java 10) adds application classes. Create it with -XX:ArchiveClassesAtExit=app.jsa (a dynamic archive, Java 13+), and use it with -XX:SharedArchiveFile=app.jsa.
• Project Leyden's AOT cache (Java 24, JEP 483) goes further: it also loads and links classes ahead of time. Java 25 adds method profiles (JEP 515) and simpler creation (JEP 514, -XX:AOTCacheOutput). Typical startup gains are 30–40%+.
• Spring Boot 3.3+ has built-in support for training runs that produce CDS and AOT caches.

Q15. What is the principle of locality, and how does it apply to Java?

Short answer: CPUs are fast only when data is in cache:

• Temporal locality: recently used data is likely to be used again.
• Spatial locality: data near recently used data is likely to be used next, because whole 64-byte cache lines are loaded at once.

In Java:

• Arrays of primitives (int[], long[]) are contiguous, and very cache-friendly.
• ArrayList<Integer> or linked structures are arrays of pointers to scattered objects, which means cache misses and pointer chasing. LinkedList iteration is dramatically slower than ArrayList, even though both are O(n).
• Object layout matters: hot fields together; avoid false sharing (unrelated hot fields of different threads on the same cache line, fixed with @Contended or padding).
• Access patterns: iterate 2-D arrays row by row (a[i][j], with the inner loop over j).
• Project Valhalla's value classes aim to flatten objects into arrays, for better locality.

Follow-up questions this topic invites — and their answers

Q: Why do micro-benchmarks written with System.nanoTime() lie? A: Because of JIT warm-up, OSR compilation of the benchmark loop, dead-code elimination (unused results get optimised away), constant folding, and GC noise. Use JMH: warm-up iterations, forks, blackholes to consume results, and statistical reporting.

Q: What does -XX:+PrintCompilation show? A: One line per compilation event: timestamp, compile ID, attributes (% for OSR, s for synchronized, ! for exception handlers, n for native), tier level, method name and size, and events like "made not entrant" (deoptimised) or "made zombie".

Q: What is the Vector API? A: jdk.incubator.vector lets you write explicit SIMD code (for example FloatVector.fromArray(SPECIES, a, i).mul(...)) that compiles to the platform's vector instructions, more reliably than hoping for auto-vectorisation. It's still an incubator module, waiting on Valhalla.

Q: Does final on methods help the JIT inline them? A: Barely, today. Class hierarchy analysis and type profiles already let C2 inline effectively-monomorphic virtual methods, and deoptimise if that assumption changes. Use final for design reasons, not for speed.