Spring Kafka @KafkaListener

Spring Kafka runs the poll loop for you. Annotate a method with @KafkaListener and Spring creates the consumer, polls in the background and calls your method once per record (or batch).

A mental model

Instead of writing the reader's loop yourself (open book, read a page, move the bookmark), you hire an assistant who does that and just hands you each page to deal with.

How it works

@KafkaListener(topics = "orders", groupId = "billing") tells Spring which topic and group. At startup it builds a listener container: one or more consumer threads running the poll loop for you.

For every record the container calls your method with the payload (and optionally the ConsumerRecord, Acknowledgment or headers). Your method contains only business logic.

By default offsets are committed by the container after your method returns normally for the whole batch (AckMode.BATCH). If your method throws, the error handler decides: retry, skip, or send to a dead-letter topic.

concurrency starts several consumer threads in the container, each a member of the group, up to the partition count.

Going deeper

The default error handler retries a failing record up to 10 times with no delay and then logs and skips it, which silently loses data unless you configure a DeadLetterPublishingRecoverer. Check this before trusting a listener in production.

Messages from one partition are handled one at a time, in order. Starting your own threads inside the method breaks that ordering and also breaks offset safety, because the container commits as soon as the method returns, not when your async work ends.

Because a record can be delivered more than once (rebalance, retry), the handler should be idempotent: handling the same order event twice must not charge a customer twice.

Under the hood

The container wraps a KafkaConsumer in a KafkaMessageListenerContainer running its own thread. A bean post-processor (KafkaListenerAnnotationBeanPostProcessor) scans beans for the annotation at startup and registers an endpoint for each annotated method.

Common mistakes

• Assuming a thrown exception retries forever (it is skipped after the default retries).
• Spawning threads from the listener so offsets are committed too early.
• Not making the handler idempotent.
• Using the same groupId in two apps.