[Redis 8.10] Add FT.AGGREGATE REDUCE COLLECT support with complex-value FieldValue by uglide · Pull Request #3878 · redis/lettuce

uglide · 2026-07-31T07:48:41Z

Adds the experimental REDUCE COLLECT reducer for FT.AGGREGATE, which gathers per-row field projections within each GROUPBY group. To carry its nested column values, FieldValue is extended from a scalar-only wrapper into a tagged union (Kind: SCALAR, ARRAY, MAP, NULL), and SearchReplyParser decodes nested reply structures recursively instead of failing on the ByteBuffer cast.

Key Decisions & Assumptions

Stacked on fix: rework search API generics to replace unnecessary type params with String #3716; the FieldValue extension assumes that reply model and should be rebased if it changes.
Parsing keeps raw protocol shapes (a RESP2 collected entry stays ARRAY, RESP3 MAP); normalization happens at read time via a lenient asMap() that interprets a flat even-length array as key/value pairs. Callers read collected columns uniformly as asList() → per-entry asMap() on both protocols. Any even-length scalar array (for example a TOLIST column) is accepted by this view; documented as caller responsibility.
COLLECT stays in AggregateArgs (the aggregateutils package serves FT.HYBRID only).

Behavioral / Conceptual Changes

Nested reply values (COLLECT, TOLIST) now parse into FieldValue arrays/maps; previously the parser failed on them. Scalar columns decode byte-identically to before.
Accessing a FieldValue through the wrong kind (for example asString() on an array) throws IllegalStateException; an odd-length pair array is rejected rather than silently truncated.
The builder validates COLLECT usage: FIELDS * and explicit fields are mutually exclusive, and a reducer without fields fails at encode time.

Testing

New FieldValueUnitTests cover kind discrimination, the pair-normalizing asMap(), immutability, and structural equality. SearchReplyCollectParserUnitTests verify RESP2/RESP3 collected-column parsing produces equal normalized entries, null preservation inside entries, and TOLIST-style arrays. Command-builder tests assert the exact REDUCE COLLECT wire format; an integration test runs a full COLLECT aggregation, self-skipping on servers without search-enable-unstable-features.

Notes

COLLECT is gated behind search-enable-unstable-features; both the server feature and this API are marked @Experimental.
HybridReplyParser still decodes field values as scalars only; FT.HYBRID cannot emit COLLECT today, so this is a known limitation, not a regression.

Note

^{Cursor Bugbot is generating a summary for commit 03a3031. Configure here.}

…th String # Conflicts: # .gitignore

…th String

…pdate all affected files. Add additional unit and integration tests for the WITHSCORE param.

… a String type.

… field - it stays generic K.

…ot neede and fix all affected files.

…o be returned by search.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 03a3031e49

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-07-31T08:02:03Z

     * @return {@code this} {@link SugAddArgs}.
     */
-    public SugAddArgs<K, V> payload(V payload) {
+    public SugAddArgs payload(String payload) {


Preserve binary suggestion payloads

When a connection uses ByteArrayCodec or another non-UTF-8 value codec, changing the payload from V to String and encoding it with args.add(...) makes arbitrary binary FT.SUGADD payloads impossible to send; the corresponding Suggestion.getPayload() change also forces replies through UTF-8 and can corrupt previously round-trippable bytes. Keep payloads codec-backed or expose a binary-safe alternative.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-07-31T08:02:03Z

+        } catch (RedisCommandExecutionException e) {
+            assumeTrue(false, "FT.AGGREGATE REDUCE COLLECT not supported by this Redis Search build: " + e.getMessage());


Let unexpected COLLECT failures fail the test

On a Redis build that supports COLLECT, any server-side rejection—including malformed argument counts, invalid syntax, or a regression in this builder—is converted into an aborted assumption, so the new integration test reports success without exercising the feature. Restrict the skip to the specific unsupported-feature response and rethrow other command errors.

AGENTS.md reference: AGENTS.md:L42-L45

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-07-31T08:02:03Z

-        for (Map<String, String> result : reply.getResults()) {
+        for (HybridReply.HybridResult<String> hybridResult : reply.getResults()) {
+            Map<String, FieldValue> result = hybridResult.getFields();
            if ("apple".equals(result.get("brand"))) {


Compare the brand's scalar value

After result changed to Map<String, FieldValue>, result.get("brand") is a FieldValue, so neither String comparison can ever match and all AVG/MIN/MAX assertions are skipped even when the returned reducer values are wrong. Compare result.get("brand").asString() and assert that the expected groups were found.

AGENTS.md reference: AGENTS.md:L42-L45

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-07-31T08:02:03Z

+ *
+ * @author Viktoriya Kutsarova
+ */
+class AggregateArgsTest {


Rename the new unit tests for Surefire

AggregateArgsTest and the newly added HybridReplyTest are pure unit tests, but pom.xml excludes **/*Test from Surefire and routes that suffix to Failsafe, whose integration phase is disabled by the normal mvn test workflow. Rename both classes to end in UnitTests so these assertions run in the unit suite.

AGENTS.md reference: AGENTS.md:L131-L134

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-07-31T08:02:03Z

+     * @param value the raw field value exactly as returned by the server. Must not be {@code null}.
+     * @return a {@link Kind#SCALAR} {@link FieldValue} view over the given bytes
+     */
+    public static FieldValue of(byte[] value) {


Add @SInCE to every new FieldValue API

FieldValue.of(byte[]) is a newly introduced public method but lacks the required @since 7.7 tag; the same omission occurs on other new public members such as asBytes(), asString(), and isNull(). Add the release tag to each new public element.

AGENTS.md reference: AGENTS.md:L145-L146

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-07-31T08:02:03Z

        CommandArgs<K, V> args = new CommandArgs<>(codec).add(index).add(fieldName);

-        return createCommand(FT_TAGVALS, new ValueListOutput<>(codec), args);
+        return createCommand(FT_TAGVALS, new StringListOutput<>(codec), args);


Decode FT.TAGVALS entries through the value codec

When a Tag field was written through a non-identity value codec, FT.TAGVALS returns the encoded field bytes; replacing ValueListOutput with StringListOutput interprets those bytes directly as UTF-8 instead of applying codec.decodeValue, so serialized or binary tag values no longer round-trip and may be corrupted. Keep the codec-backed List<V> result or expose a raw binary-safe representation.

Useful? React with 👍 / 👎.

viktoriya-kutsarova and others added 26 commits July 22, 2026 16:24

fix: rework search API generics to replace unnecessary type params wi…

359cbec

…th String # Conflicts: # .gitignore

fix: rework search API generics to replace unnecessary type params wi…

742d05b

…th String

Add verification tests

ab79f50

Expand codec verification tests and revert K for field names and INKEYS.

33343a2

Add more verification tests

67ba954

Add more codec tests ana parametrize them.

6f1ea6d

Fix prefix issue.

b24c456

Fix params issue.

6508a72

Add more tests for search on HASH using codec.

61e9130

Remove generics from FieldArgs and update all affected files.

1d077ab

Remove generics from SugGetArgs and update all affected files.

d453d1b

Remove generics from ftSugadd, ftSugdel, ftSugget, their params and u…

7204b58

…pdate all affected files. Add additional unit and integration tests for the WITHSCORE param.

Fix RESP2 inability to properly parse scores because they are sent as…

aac1ff1

… a String type.

Remove generics from ftTagvals command and all affected files.

dcc86ef

Rework most of the generic fields in CreateArgs apart from the prefix…

c8e9189

… field - it stays generic K.

Rework SearchArgs to remove K and V and all affected files.

3dc3689

Rework AggregateArg to remove generics and all affected files.

8c1f3fe

Rework HybridArgs to remove generics and fix all affected files.

38cad1c

Rework SearchReply and HybridReply to remove K and V generics where n…

668dc49

…ot neede and fix all affected files.

Remove phantom-V leftover from search module files.

ecaa00a

Fix issues with tests.

335530c

Remove generic from withPrefix in CreateArgs and fix all affected files.

05ed7d3

Update redis-search docs.

52b2946

Introduce FieldValue to allow for mixed types (vectors and strings) t…

41e9aca

…o be returned by search.

Decode FT.HYBRID document keys with the connection codec.

e00e813

Add support for COLLECT reducer

03a3031

uglide requested a review from viktoriya-kutsarova July 31, 2026 07:48

uglide changed the title ~~[Redis 8.10]Add support ft aggregate collect reducer~~ [Redis 8.10] Add FT.AGGREGATE REDUCE COLLECT support with complex-value FieldValue Jul 31, 2026

chatgpt-codex-connector Bot reviewed Jul 31, 2026

View reviewed changes

uglide changed the base branch from main to rework-codecs-for-redis-search-module August 19, 2026 11:50

Navigation Menu

Uh oh!

[Redis 8.10] Add FT.AGGREGATE REDUCE COLLECT support with complex-value FieldValue - #3878

[Redis 8.10] Add FT.AGGREGATE REDUCE COLLECT support with complex-value FieldValue#3878
uglide wants to merge 26 commits into
rework-codecs-for-redis-search-modulefrom
feat/ft-aggregate-collect-reducer-v2

uglide commented Jul 31, 2026 •

edited

Loading

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

chatgpt-codex-connector Bot Jul 31, 2026

Uh oh!

chatgpt-codex-connector Bot Jul 31, 2026

Uh oh!

chatgpt-codex-connector Bot Jul 31, 2026

Uh oh!

chatgpt-codex-connector Bot Jul 31, 2026

Uh oh!

chatgpt-codex-connector Bot Jul 31, 2026

Uh oh!

chatgpt-codex-connector Bot Jul 31, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		} catch (RedisCommandExecutionException e) {
		assumeTrue(false, "FT.AGGREGATE REDUCE COLLECT not supported by this Redis Search build: " + e.getMessage());

Uh oh!

Conversation

uglide commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Key Decisions & Assumptions

Behavioral / Conceptual Changes

Testing

Notes

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Jul 31, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Jul 31, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Jul 31, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Jul 31, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Jul 31, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Jul 31, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

uglide commented Jul 31, 2026 •

edited

Loading