Unravel โบ Java concepts โบ Messaging and Kafka
Rebalancing
A rebalance is Kafka reshuffling which consumer owns which partition, triggered by a member joining, leaving, crashing or being too slow.
Learn these first
At a glance
A rebalance is a team reshuffling lanes at the checkout when someone goes on break.
- 1A rebalance starts when a consumer joins or leaves the group, a consumer misses its heartbeats or stops polling, or the topic gains partitions
- 2Before partitions are taken away, onPartitionsRevoked runsThis is the last chance to commit offsets and flush state for those partitions. After the new plan is known, onPartitionsAssigned runs, where you can seek to a stored position.
- 3During an eager rebalance everyone stops processing until the new assignment is agreed ("stop the world")The cooperative-sticky assignor moves only the partitions that must move and lets the rest keep running.
- Frequent rebalances usually mean slow processing (exceeding max.poll.interval.ms), unstable pods, or session timeouts that are too tight
- onPartitionsLost is different from revoked: the partitions were already taken (for example after a session timeout), so committing for them will fail
- The group leader computes the assignment and the coordinator distributes it
consumer.subscribe(List.of("orders"), new ConsumerRebalanceListener() { @Override public void onPartitionsRevoked(Collection<TopicPartition> partitions) { // last chance to save progress for partitions we are about to lose consumer.commitSync(); } @Override public void onPartitionsAssigned(Collection<TopicPartition> partitions) { log.info("Now reading {}", partitions); }});- line 1 uses ConsumerRebalanceListener, the core of Rebalancing
- line 5 last chance to save progress for partitions we are about to lose
Read the full explanation (about 2 min)
A rebalance is Kafka reshuffling which consumer owns which partition, triggered by a member joining, leaving, crashing or being too slow. Your code gets callbacks so it can save progress before losing a partition.
A mental model
A rebalance is a team reshuffling lanes at the checkout when someone goes on break. For a moment nobody serves some lanes, and the cashier giving up a lane must hand over their notes first.
How it works
A rebalance starts when a consumer joins or leaves the group, a consumer misses its heartbeats or stops polling, or the topic gains partitions.
Before partitions are taken away, onPartitionsRevoked runs. This is the last chance to commit offsets and flush state for those partitions. After the new plan is known, onPartitionsAssigned runs, where you can seek to a stored position.
During an eager rebalance everyone stops processing until the new assignment is agreed ("stop the world"). The cooperative-sticky assignor moves only the partitions that must move and lets the rest keep running.
Going deeper
Frequent rebalances usually mean slow processing (exceeding max.poll.interval.ms), unstable pods, or session timeouts that are too tight. Symptoms: lag spikes, duplicate processing and logs full of "Revoking previously assigned partitions".
onPartitionsLost is different from revoked: the partitions were already taken (for example after a session timeout), so committing for them will fail. Do cleanup, not commits.
Manual assign bypasses groups entirely: no rebalancing, no group coordination. It suits tools that must read specific partitions, not scalable services.
Under the hood
The group leader computes the assignment and the coordinator distributes it. With the cooperative protocol, the rebalance runs in two rounds: first partitions that must move are revoked, then they are assigned to their new owner, while untouched partitions never stop.
Common mistakes
- Not committing in
onPartitionsRevoked, causing duplicates after every rebalance. - Long processing inside poll that triggers endless rebalances.
- Doing blocking work inside a rebalance callback.
How Unravel recognises it
When you paste code, Unravel looks for these signs of rebalancing. Spotting them yourself is a good habit when you read unfamiliar code:
- Types:
ConsumerRebalanceListenerCooperativeStickyAssignorStickyAssignorRangeAssignor - Method calls:
assign()pause()resume() - Method names you declare:
onPartitionsRevoked()onPartitionsAssigned()onPartitionsLost() - Configuration keys:
partition.assignment.strategy
Part of a learning path
- Kafka for Java developers ยท previous: Offsets and commits ยท next: Spring Kafka @KafkaListener
Related concepts
Frequently asked questions
What is Rebalancing?
A rebalance is Kafka reshuffling which consumer owns which partition, triggered by a member joining, leaving, crashing or being too slow. Your code gets callbacks so it can save progress before losing a partition.
What are the most common mistakes with Rebalancing?
Not committing in onPartitionsRevoked, causing duplicates after every rebalance. Long processing inside poll that triggers endless rebalances. Doing blocking work inside a rebalance callback.
How can I see Rebalancing in real code?
Paste your own Java into Unravel and it will point out where rebalancing appears, explain how it works at the detail level you choose, and draw the flow of the code around it.