Skip to content

fix(kafka): rewind every partition of a batch on nack - #3282

Closed
bercedev wants to merge 1 commit into
ag2ai:mainfrom
bercedev:fix/batch-nack-every-partition
Closed

bercedev wants to merge 1 commit into
ag2ai:mainfrom
bercedev:fix/batch-nack-every-partition

Conversation

@bercedev

@bercedev bercedev commented Oct 22, 2026 •

Copy link
Copy Markdown

Description

KafkaAckableMessage.nack() seeked only the partition of raw_message[0]. A batch can span partitions, as in the issue's [(0, 1), (0, 2), (1, 0), (1, 1), (2, 0), (2, 1)]. With AckPolicy.NACK_ON_ERROR, only partition 0 was rewound, and the next successful batch's commit() committed past the records of partitions 1 and 2 that were never processed.

nack() now collects the lowest offset the batch holds in each partition and seeks every one of them. A single message works as before: one seek to its own offset. The confluent KafkaMessage.nack() had the same raw_message[0] pattern, so it gets the same change: one seek(topic, partition, offset) per partition in the batch. Records without a topic, partition or offset are still skipped.

Fixes #3280

Type of change

  • Bug fix (a non-breaking change that resolves an issue)

Checklist

  • My code adheres to the style guidelines of this project: ruff check, ruff format --check (0.14.14) and mypy (1.19.1) are clean on the changed files. I didn't run the full just lint.
  • I have conducted a self-review of my own code
  • I have added tests to validate the effectiveness of my fix or the functionality of my new feature
  • Both new and existing unit tests pass successfully on my local environment by running just test-coverage: not run in full, see below.

New tests in tests/brokers/kafka/test_message_nack.py and tests/brokers/confluent/test_message_nack.py build the message with a stub consumer that records its seek() calls:

  • a batch over three partitions, with the records out of order within a partition, seeks partition 0 to 1, partition 1 to 0 and partition 2 to 4;
  • a single message seeks its own partition and offset.

Without the change, the two batch tests fail (one seek, for partition 0 only), and the single-message tests pass. I had no Kafka broker, so I didn't rerun the issue's real-broker scenario. In my container, the test_test_client.py suites for both backends gave 167 passed and 10 failed. The first failure is KafkaConnectionError: Unable to bootstrap from localhost:9092 (a test that wants a real broker). I didn't compare the other nine against main.

nack() on a batch seeked only the partition of the first record, so the other partitions' records were not redelivered and a later successful batch committed past them. Both the aiokafka and the confluent message now seek each partition in the batch to the first offset the batch holds there. Fixes ag2ai#3280.
@github-actions github-actions Bot added Confluent Issues related to `faststream.confluent` module AioKafka Issues related to `faststream.kafka` module labels Oct 22, 2026
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@IvanKirpichnikov

Copy link
Copy Markdown
Collaborator

Hi. I have to close your Pull Request because someone else submitted #3281 before you, and I merged it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AioKafka Issues related to `faststream.kafka` module Confluent Issues related to `faststream.confluent` module

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: Kafka batch nack() rewinds only the first partition, allowing unprocessed messages from other partitions to be committed

3 participants