Skip to content

feat(testing): Kafka handlers and publishers assert on key and partition - #3113

Merged
IvanKirpichnikov merged 10 commits into
mainfrom
feat/557-kafka-fields
Oct 7, 2026
Merged

IvanKirpichnikov merged 10 commits into
mainfrom
feat/557-kafka-fields

Conversation

@Lancetnik

Copy link
Copy Markdown
Member

Stacked on #3111. Ticket 02 of the Call assertions plan.

A Kafka test wanted the record key by name. The only route was a context path into the raw message, context={"message.raw_message.key": b"..."}, which nothing completes in an editor, and which reads differently under aiokafka and confluent: an attribute on one, a method on the other.

Handlers and publishers of both Kafka packages now take key and partition in all three Call assertions, under the names their publish() uses:

await handle.assert_called_once_with(
    Data(name="John", user_id=1),
    key=b"user-1",
    partition=0,
)

Both compare exactly, as the client delivered them: key as bytes, nothing run through a serializer. A batch refuses them the way it refuses headers.

What changes

Endpoint reads the wrapper class it builds from a class attribute, _call_wrapper_class, and ensure_call_wrapper takes that class while still returning an already wrapped handler untouched. So the first decorator on a shared handler decides its class, and a Kafka field asked of a message another broker delivered raises SetupError naming Kafka.

Both Kafka LogicSubscriber and LogicPublisher set KafkaHandlerCallWrapper there and repeat their __call__ overloads with the return type narrowed to it, so mypy sees @broker.subscriber(...) return the Kafka wrapper. The class is internal and not exported.

KafkaCallAssertions in _internal/kafka declares the three methods with the two arguments and hands them to the recorder as BrokerFields: the values asked for and a reader (name, message) -> value. Each package supplies its reader from a new call_wrapper module: attributes on an aiokafka record, method calls on a confluent message. The spec placed the reader in testing.py, but that module imports the broker, which imports the subscriber, which needs the wrapper class at class-definition time. That is an import cycle, so the reader sits one module lower.

The in-memory confluent message is now a confluent_kafka.Message subclass, so the reader's type check holds under the in-memory broker as it does against a real client. Its headers() and error() signatures widened to match the base.

ADR-0010 has the reasoning for a class rather than **fields.

Tests and docs

  • tests/brokers/kafka/test_test_client.py and its confluent twin: the three methods with key and partition on a handler and on a publisher, the mismatch text naming key, the batch refusal, and a handler shared with a Rabbit broker refusing a Kafka field for the Rabbit message.
  • tests/mypy/kafka.py and tests/mypy/confluent.py assert_type a decorated handler and a published handler as KafkaHandlerCallWrapper, and call the three methods with the two arguments on a publisher.
  • The AIOKafka and Confluent advanced_testing.py snippets gain test_kafka_fields, the docs runner calls it, and the "Validates Message Fields" section says each broker's handlers and publishers take that broker's own fields. The context example no longer reaches the key by path, since the key is a named argument now.

Tickets 03–06 (Rabbit, NATS, Redis, MQTT) branch from here.

🤖 Generated with Claude Code

@github-actions github-actions Bot added documentation Improvements or additions to documentation Confluent Issues related to `faststream.confluent` module AioKafka Issues related to `faststream.kafka` module labels Sep 9, 2026
@Lancetnik
Lancetnik added this pull request to stack #3112 September 9, 2026 08:47
@Lancetnik
Lancetnik marked this pull request as draft September 9, 2026 08:48
@Lancetnik
Lancetnik marked this pull request as ready for review September 9, 2026 09:00
Comment thread tests/brokers/kafka/test_test_client.py Outdated
Comment thread tests/brokers/kafka/test_test_client.py Outdated
Base automatically changed from feat/557-stacked to main September 14, 2026 21:35
Lancetnik and others added 6 commits September 15, 2026 00:35
…tition`

`Endpoint` reads the wrapper class it builds from `_call_wrapper_class`, and
`ensure_call_wrapper` takes that class while still returning an already
wrapped handler untouched, so the first decorator on a shared handler decides.
Both Kafka packages set `KafkaHandlerCallWrapper` on their `LogicSubscriber`
and `LogicPublisher`, and repeat the `__call__` overloads with the return type
narrowed to it.

`KafkaCallAssertions` in `_internal/kafka` declares the three Call assertions
with `key=` and `partition=` and forwards them as Broker fields. `ExpectedCall`
carries the fields with a reader `(name, message) -> value` that each package
supplies from its `call_wrapper` module: attributes on an aiokafka record,
method calls on a confluent message, a `SetupError` naming Kafka for a raw
message of another broker. A batch refuses the two fields as it refuses
`headers`. The in-memory confluent message is now a `confluent_kafka.Message`
subclass, so the reader takes it for the client's own.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The AIOKafka and Confluent tabs gain a `test_kafka_fields` snippet, the
context example no longer reaches for the key by path, and the testing page
says each broker's handlers and publishers take that broker's own fields.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…eir reader

`ExpectedCall.broker_fields` is one `BrokerFields`: the values asked for, already
stripped of the ones left EMPTY, and the reader that answers them. The recorder
no longer filters EMPTY twice or guards a reader that may be missing, and the
Kafka mixin's helper is named for what it returns.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The confluent reader's `isinstance` guard had no test going red on its own.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
pyright spells an `async def`'s return as `CoroutineType[...]` where mypy says
`Coroutine[...]`, so an `assert_type` over an async handler cannot satisfy both
checkers; a sync handler reads the same in both.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…der`

Both Kafka packages built the same reader by hand: an isinstance guard, the same
refusal text, and one line that differs (an attribute on aiokafka, a method on
confluent). `field_reader(record, read)` in `_internal/kafka` builds it once, so
each `call_wrapper` module opens with its class and binds the reader inline. The
mixin declares the reader as a `ClassVar` and looks it up on the class, replacing
a staticmethod stub that only raised; a `LogicPublisher` takes the reader from its
wrapper class instead of restating the import.

The refusal names the message class a shared handler got (`RabbitMessage`), not
the raw client type. `ExpectedCall.message_fields()` introspects the dataclass
again instead of listing its fields by hand. The shared-handler tests carry
`require_aiopika`, as the other cross-broker tests do.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Comment thread tests/brokers/confluent/test_test_client.py
Comment thread tests/brokers/confluent/test_test_client.py
The subscriber that forwarded the message added nothing: the publisher's
mock records the call either way, and the test still goes red when the
reader returns the wrong field.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Lancetnik and others added 3 commits September 23, 2026 23:25
Conflicts, each resolved by keeping both sides: `Endpoint` and the Kafka and
confluent `LogicPublisher` take main's `__slots__` (#3220) next to this branch's
`_call_wrapper_class` and `_read_field` class attributes, and
`tests/mypy/confluent.py` keeps both imports and both checks.
- The Kafka and confluent `__call__` overrides take `dependencies` as a
  `Sequence`, as `SubscriberUsecase.__call__` does since #3192.
- The tests type their handlers (#3176 put `tests/brokers` under strict mypy).
  The shared-handler test subscribes RabbitMQ by a call rather than a second
  decorator, so the name keeps the Kafka wrapper's type; the RabbitMQ import stays
  inside the test (`# noqa: PLC0415`) because the test is `@require_aiopika`.
- `MockConfluentMessage` answers `len()` as the client's message does: its
  `Message` base read a C-level value the mock never sets, so every mock message
  was falsy and `assert self.msg` in `tests/brokers/confluent/test_requests.py`
  failed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@IvanKirpichnikov
IvanKirpichnikov added this pull request to the merge queue Oct 7, 2026
Merged via the queue into main with commit 0ba23f2 Oct 7, 2026
32 checks passed
@IvanKirpichnikov
IvanKirpichnikov deleted the feat/557-kafka-fields branch October 7, 2026 21:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AioKafka Issues related to `faststream.kafka` module Confluent Issues related to `faststream.confluent` module documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants