feat: support resilient reader config for indexer and add retry policy - #1364
Open
huangzhen1997 wants to merge 6 commits into
Open
feat: support resilient reader config for indexer and add retry policy#1364huangzhen1997 wants to merge 6 commits into
huangzhen1997 wants to merge 6 commits into
Conversation
|
👋 huangzhen1997, thanks for creating this pull request! To help reviewers, please consider creating future PRs as drafts first. This allows you to self-review and make any final changes before notifying the team. Once you're ready, you can mark it as "Ready for review" to request feedback. Thanks! |
huangzhen1997
marked this pull request as draft
August 19, 2026 21:13
Contributor
There was a problem hiding this comment.
Pull request overview
This PR makes the indexer’s resilient reader policies configurable via TOML, so rate limiting / bulkhead / circuit breaker / timeout parameters are no longer hardcoded and can be tuned to mitigate downstream rate limiting.
Changes:
- Adds
[Resilience]configuration to the indexer config model and example TOML. - Introduces a
readers.NewResilienceConfig(config.ResilienceConfig)adapter that applies defaults for zero-valued config fields. - Threads the resilience configuration through reader construction (REST + Aggregator) and adds tests for the new adapter.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| indexer/pkg/readers/rest_reader.go | Adds resilience config to RestReaderConfig and passes it into NewResilientReader. |
| indexer/pkg/readers/rest_reader_test.go | Updates REST reader tests to provide a resilience config. |
| indexer/pkg/readers/resilient_reader.go | Adds adapter to convert indexer config resilience settings into reader resilience config with defaults. |
| indexer/pkg/readers/resilient_reader_test.go | Adds unit tests for NewResilienceConfig defaulting/overrides behavior. |
| indexer/pkg/readers/aggregator_reader.go | Extends constructor signature to accept resilience config and wires aggregator-specific handlers. |
| indexer/pkg/config/config.go | Adds Resilience and ResilienceConfig to the indexer configuration model. |
| indexer/config.example.toml | Documents the new [Resilience] TOML block and its default values. |
| indexer/cmd/replay/main.go | Passes configured resilience settings into reader creation in replay mode. |
| indexer/cmd/main.go | Passes configured resilience settings into reader creation in normal indexer mode. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
huangzhen1997
changed the title
KodeyThomas
previously approved these changes
Aug 25, 2026
… jh/resilient-reader-config
huangzhen1997
marked this pull request as ready for review
September 4, 2026 22:07
huangzhen1997
force-pushed
the
jh/resilient-reader-config
branch
from
September 5, 2026 00:43
d71ce3f to
0a85110
Compare
|
Code coverage report:
|
huangzhen1997
requested review from
KodeyThomas
and
a balanced review from Copilot
September 5, 2026 01:00
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
A documented setting has no runtime effect, retry delays lack validation, and the retry test does not exercise retries.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
- Files reviewed: 10/10 changed files
- Comments generated: 3
- Review effort level: Balanced
Comment on lines
+77
to
+79
| // CircuitBreakerTimeout is the timeout for circuit breaker operations. | ||
| // 0 uses the default (1s). | ||
| CircuitBreakerTimeout common.Duration `toml:"CircuitBreakerTimeout"` |
Comment on lines
+86
to
+91
| // RetryDelay is the initial delay between retries, using exponential backoff. | ||
| // 0 uses the default (1s). | ||
| RetryDelay common.Duration `toml:"RetryDelay"` | ||
| // RetryMaxDelay is the maximum delay between retries. | ||
| // 0 uses the default (10s). | ||
| RetryMaxDelay common.Duration `toml:"RetryMaxDelay"` |
Comment on lines
+120
to
+123
| start := time.Now() | ||
| resp2, err := rr.ReadCCVData(ctx) | ||
| elapsed := time.Since(start) | ||
| require.NoError(t, err) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Encountered rate limiting issue value of the resilient reader, and it's never configurable, it always using this hardcoded default value.