Spot the Bug #121

Committing the work before doing it

A medium Java puzzle in the messaging and kafka mistakes family. One line below contains a real bug. Try to spot it first, then read why it fails and how to fix it.

mediumMessaging and Kafka mistakesbuggy line: 8

1import org.apache.kafka.clients.consumer.*;

2import java.time.Duration;

4public class Worker {

5 void loop(KafkaConsumer<String, String> consumer) {

6 while (true) {

7 ConsumerRecords<String, String> records = consumer.poll(Duration.ofSeconds(1));

8 consumer.commitSync();

9 for (ConsumerRecord<String, String> r : records) {

10 process(r.value());

11 }

12 }

13 }

15 void process(String value) { }

16}

Highlighted: line 8. Play this one as a puzzle without the answer.

What is wrong

The offsets are committed before the records are processed. If the process crashes midway, the committed offsets are already past the unprocessed records and they are never delivered again, so messages are lost. This is at-most-once delivery by accident.

The fix

for (ConsumerRecord<String, String> r : records) { process(r.value()); } consumer.commitSync();

How to avoid it next time

Commit after the work is done and make the work idempotent. Choose the delivery guarantee deliberately.

Want the background? Read Offsets and commits in the Java concepts library.

About messaging and kafka mistakes

Message handling combines retries, offsets and transactions, and the order of those steps decides whether a failure loses data or processes it twice.

In code review: Trace what happens if the process dies after each line. Commit an offset only after the work is done, and make handlers idempotent.

More puzzles in this family

#122: The consumer kicked out of its group (hard)
#123: A producer that never learns of failure (medium)
#124: A bad record that blocks the topic (hard)
#125: The payment applied twice after a rebalance (hard)
#126: Orders for one customer in random order (medium)

🐞 Play today's puzzle All 132 bugs ← #120 #122 →

Frequently asked questions

What is the bug in "Committing the work before doing it"?

How do I fix it?

Replace it with: for (ConsumerRecord r : records) { process(r.value()); } consumer.commitSync();. Commit after the work is done and make the work idempotent. Choose the delivery guarantee deliberately.

How do I catch messaging and kafka mistakes in code review?

Trace what happens if the process dies after each line. Commit an offset only after the work is done, and make handlers idempotent.

Is there a new puzzle every day?

Yes. The daily Spot the Bug puzzle picks one of these snippets for each UTC day, with the difficulty set by the weekday. Your streak is stored only in your browser.

More tools you might need

🎯Daily Cron QuizFive fresh questions every day 🔀Kafka rebalance simulatorRange, sticky and cooperative assignment, animated 🐧systemd OnCalendar ⇄ cronTimer schedules both ways, checked ⏲️systemd timer generatorTimer and service unit files