BackendBench
Coffee

Unravel โ€บ Java concepts โ€บ JVM and performance

Java language

Measuring performance and JMH

Timing code with a start and end clock is easy and often wrong: the JIT compiler needs time to warm up and may delete work whose result is unused.

๐Ÿ”Ž Unravel this example

At a glance

Mental model

Timing a runner who has not warmed up, on a track that changes under their feet.

What happens, in order
  1. 1long t0 = System.nanoTime(); work(); long ms = (System.nanoTime() - t0) / 1_000_000; is fine to find out where a request spends its time in a log line, but not to compare two implementations
  2. 2JMH: annotate a method with @Benchmark, state in a @State class, and use @Warmup(iterations = 5), @Measurement(iterations = 5), @Fork(2)Return the result or give it to a Blackhole so the JIT cannot discard the work.
  3. 3Reports give average time, error and unitsCompare means with their error bars; a 3 percent difference inside a 10 percent error is noise.
  4. 4For real systems measure with a profiler (JFR, async-profiler) under realistic load, because microbenchmarks do not include GC, contention or I/O
Remember
Watch out
A single timed run without warm-upIgnoring the result of the measured codeComparing results from different machines or loadsOptimising without a profile
Example (highlighted lines explain it)
import org.openjdk.jmh.annotations.*;import java.util.concurrent.TimeUnit;@State(Scope.Thread)@BenchmarkMode(Mode.AverageTime)@OutputTimeUnit(TimeUnit.NANOSECONDS)@Warmup(iterations = 3)@Measurement(iterations = 5)@Fork(1)public class SumBench {  int[] data = new int[1000];  @Benchmark  public long loop() {    long s = 0;    for (int v : data) s += v;    return s;  }}
  1. line 3 uses State, the core of Measuring performance and JMH
  2. line 4 uses Benchmark, the core of Measuring performance and JMH
  3. line 5 uses OutputTimeUnit, the core of Measuring performance and JMH
  4. line 6 uses Warmup, the core of Measuring performance and JMH
  5. line 7 uses Measurement, the core of Measuring performance and JMH
  6. line 8 uses Fork, the core of Measuring performance and JMH
Read the full explanation (about 2 min)

Timing code with a start and end clock is easy and often wrong: the JIT compiler needs time to warm up and may delete work whose result is unused. JMH (Java Microbenchmark Harness) handles warm-up, forks and statistics.

A mental model

Timing a runner who has not warmed up, on a track that changes under their feet. You need several warm-up laps, many measured runs, and a fresh runner each session before the average means anything.

How it works

long t0 = System.nanoTime(); work(); long ms = (System.nanoTime() - t0) / 1_000_000; is fine to find out where a request spends its time in a log line, but not to compare two implementations.

JMH: annotate a method with @Benchmark, state in a @State class, and use @Warmup(iterations = 5), @Measurement(iterations = 5), @Fork(2). Return the result or give it to a Blackhole so the JIT cannot discard the work.

Reports give average time, error and units. Compare means with their error bars; a 3 percent difference inside a 10 percent error is noise.

For real systems measure with a profiler (JFR, async-profiler) under realistic load, because microbenchmarks do not include GC, contention or I/O.

Going deeper

Typical traps: constant folding (inputs known at compile time), dead-code elimination (unused result), loop unrolling that is not representative, a cold JIT, running on a laptop with power saving, and measuring one thread when production has many.

Optimise after measuring where the time is; an algorithmic improvement (see Big O) usually beats micro-tuning. Do not guess.

System.currentTimeMillis() can jump with clock changes; use nanoTime for durations and Instant/Clock for dates.

Under the hood

HotSpot starts interpreting bytecode, compiles hot methods with C1 and then C2, and may deoptimise later. Its profile-guided optimisations (inlining, escape analysis) depend on the code that actually runs, which is why benchmark code must look like real code.

Common mistakes

  • A single timed run without warm-up.
  • Ignoring the result of the measured code.
  • Comparing results from different machines or loads.
  • Optimising without a profile.

How Unravel recognises it

When you paste code, Unravel looks for these signs of measuring performance and jmh. Spotting them yourself is a good habit when you read unfamiliar code:

Part of a learning path

Related concepts

Frequently asked questions

What is Measuring performance and JMH?

Timing code with a start and end clock is easy and often wrong: the JIT compiler needs time to warm up and may delete work whose result is unused. JMH (Java Microbenchmark Harness) handles warm-up, forks and statistics.

What are the most common mistakes with Measuring performance and JMH?

A single timed run without warm-up. Ignoring the result of the measured code. Comparing results from different machines or loads. Optimising without a profile.

How can I see Measuring performance and JMH in real code?

Paste your own Java into Unravel and it will point out where measuring performance and jmh appears, explain how it works at the detail level you choose, and draw the flow of the code around it.