Repository navigation
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add an opt-in keyspace reader for deployments where repeatedly sending large XREAD key lists saturates Redis. The existing blocking strategy remains the default. This PR is stacked on #140 so the independent RedisCache correction can be reviewed first.
The new mode uses a dedicated subscriber, acknowledged exact keyspace subscriptions, bounded dirty-key reads, and periodic reconciliation from existing cursors. Notifications coalesce wakeups while retained stream entries still pass through the existing ordered callback, epoch and recovery machinery.
Missing notification flags, denied subscriptions and unavailable/stalled subscribers select a logged blocking fallback with retry. Configuration inspection is optional and read-only; the adapter never changes Redis configuration. Public options select the mode and reconciliation interval, and readerBackendStatus() reports requested/effective mode and fallback reason.
Validation: all 131 tests pass in blocking and keyspace modes against private Linux Redis 7.4.11 and 8.2.0 fixtures. This includes 16 focused notification tests and regressions for partial subscription failures, stalled subscribers, and snapshot-only adapters opening unexpected notification connections. All six CI jobs pass: both reader modes in ordinary builds, address/undefined sanitizers and thread sanitizers. Independent reviews covered scheduler lifecycle, cursor/epoch behavior and subscriber ownership/recovery.
Includes a standalone isolated benchmark for the seven-adapter, many-key workload from #139. The 24-case Redis 7.4.11 matrix covers 769/1,025 keys per adapter, readers 1/4/16, and sparse/burst traffic at a requested 5,400 updates/s. A four-case follow-up compared a matched rate in both execution orders after the concurrent image build finished.
Matched-rate follow-up: seven adapters, 769 keys each, one reader per adapter, sparse traffic at approximately 2,000 accepted updates/s in both modes:
This is approximately 90% less Redis CPU and 98% less request traffic. Across all 28 cases, all 4,650,951 accepted writes were observed, with zero omissions, duplicates, order violations, malformed payloads or write errors, and the requested backend was active at every measurement sample.
The overloaded sparse blocking cases in the full matrix could not generate the requested 5,400/s; their CPU values are capacity observations, not equal-throughput savings. These are private Unix-socket tests with retained synthetic history, not production UDP-loss or trimming measurements. The tested source is
fec3b3fc5426e814125fcebecd5fc2bae1f4e377; the transferred source archive SHA-256 is443d03c10950ad309f97c3f536165685831a56cacf8a2b504c8bd50d6830dc4f.Related: #139, #140, fermi-ad/general-memory-mapped-data-mover#88, fermi-ad/redis-pvxs-ioc#173 and fermi-ad/general-redis-acnet-frontend#216. IOC/GRAFE pin the tested adapter commit. No production settings or services are changed.