Skip to content

A75: Implement the new LB policy topology for the non-aggregate cluster - #9234

Open
mswierq wants to merge 13 commits into
grpc:masterfrom
mswierq:a75/non-aggregate-lb-policy-tree-refactoring
Open

A75: Implement the new LB policy topology for the non-aggregate cluster#9234
mswierq wants to merge 13 commits into
grpc:masterfrom
mswierq:a75/non-aggregate-lb-policy-tree-refactoring

Conversation

@mswierq

@mswierq mswierq commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Implements A75 for the non-aggregate cluster.

This change implements the new LB policy topology for the non-aggregate clusters only, the aggregate cluster uses the legacy approach. Refactoring of the new topology for the aggregate cluster case will be delivered in a follow-up PR. This change does not break non-aggregate to aggregate (and vice versa) switch.

RELEASE NOTES:

  • xds: refactored LB policy topology for the non-aggregate clusters according to A75

@codecov

codecov Bot commented Jul 15, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 91.66667% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.25%. Comparing base (44332db) to head (f2fc6b5).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
internal/xds/balancer/cdsbalancer/cdsbalancer.go 87.87% 3 Missing and 1 partial ⚠️
internal/xds/balancer/cdsbalancer/configbuilder.go 93.65% 2 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #9234      +/-   ##
==========================================
+ Coverage   83.20%   83.25%   +0.04%     
==========================================
  Files         423      423              
  Lines       35321    35208     -113     
==========================================
- Hits        29389    29311      -78     
+ Misses       4425     4402      -23     
+ Partials     1507     1495      -12     
Files with missing lines Coverage Δ
internal/xds/balancer/cdsbalancer/cdsbalancer.go 69.66% <87.87%> (+7.54%) ⬆️
internal/xds/balancer/cdsbalancer/configbuilder.go 92.15% <93.65%> (+0.26%) ⬆️

... and 26 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@easwars
easwars requested a review from Pranjali-2501 July 15, 2026 18:13
@easwars easwars added Type: Feature New features or improvements in behavior Area: xDS Includes everything xDS related, including LB policies used with xDS. labels Jul 15, 2026
@easwars easwars added this to the 1.84 Release milestone Jul 15, 2026
@easwars

easwars commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the CDS balancer to support the gRFC A75 topology for single (non-aggregate) clusters, routing them through an outlier detection policy instead of a priority policy. The changes include updating the configuration builder to construct the new balancer tree (outlier_detection -> cluster_impl -> priority -> wrr_locality -> round_robin) and updating corresponding tests. The review feedback highlights several critical areas in the new code paths where missing configuration data (such as nil cluster configs, endpoint configs, or empty priorities) could lead to nil pointer dereferences or out-of-bounds panics, and suggests adding defensive checks to ensure robustness.

Comment thread internal/xds/balancer/cdsbalancer/cdsbalancer.go
Comment thread internal/xds/balancer/cdsbalancer/cdsbalancer.go
Comment thread internal/xds/balancer/cdsbalancer/configbuilder.go Outdated
Comment thread internal/xds/balancer/cdsbalancer/configbuilder.go
Comment thread internal/xds/balancer/cdsbalancer/configbuilder.go
Comment thread internal/xds/balancer/cdsbalancer/cdsbalancer.go Outdated
Comment thread internal/xds/balancer/cdsbalancer/cdsbalancer.go Outdated
Comment thread internal/xds/balancer/cdsbalancer/cdsbalancer.go Outdated
Comment on lines +51 to +54
parser, ok := builder.(balancer.ConfigParser)
if !ok {
return nil, nil, fmt.Errorf("xds: balancer builder for %v does not implement ConfigParser", name)
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since you have added the logic to retrieve the childConfigParser here, we can remove it from `Build()``.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this has been removed as a result of using gracefulswitch.Balancer

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

setting childConfigParser in Build has been removed

Comment on lines +46 to +47
newChildBalancer = func(name string, cc balancer.ClientConn, opts balancer.BuildOptions) (balancer.Balancer, balancer.ConfigParser, error) {
builder := balancer.Get(name)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use gracefulswitch.Balancer here and make cdsBalancer.childLB of type *gracefulswitch.Balancer?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

newChildBalancer function and its output have been removed as gracefulswitch.Balancer and gracefulswitch.ParseConfig implementation already covers the same functionality

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to start a discussion on this before making any change and want to hear your thoughts on this.

Now based on my offline discussions with @easwars, We don't need to use gracefulSwitch here because it is very rare that the type of cluster get changed.

Could you please revert back the changes you made to use gracefulSwitch.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, I have reverted it

Comment on lines +315 to +317
if len(b.priorities) == 0 {
return fmt.Errorf("no priorities configured for non-aggregate cluster %q", clusterName)
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need this check here?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can directly pass b.priorities instead of passing its first element in buildSingleClusterConfigJSON.

I also think handleClusterUpdate() will never set b.priorities to nil.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reverted + getting the first element is inside ConfigJSON function

Comment thread internal/xds/balancer/cdsbalancer/configbuilder.go Outdated
Comment thread internal/xds/balancer/cdsbalancer/configbuilder.go Outdated
Comment thread internal/xds/balancer/cdsbalancer/configbuilder.go
Comment thread internal/xds/balancer/cdsbalancer/cdsbalancer.go
Comment thread internal/xds/balancer/cdsbalancer/configbuilder.go Outdated
@mswierq
mswierq requested a review from Pranjali-2501 August 4, 2026 11:13
@easwars easwars assigned Pranjali-2501 and unassigned mswierq Aug 11, 2026
// management server sends two updates for the same leaf cluster resource. The
// test verifies that the load balancing configuration pushed to the priority LB
// policy contains the expected discovery mechanism corresponding to the leaf
// test verifies that the load balancing configuration pushed to the top-level LB

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wrap the comment in 80 column.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Comment on lines +259 to +262
// the priority LB policy contains the discovery mechanisms for both child
// clusters. The test then updates the root cluster resource requested by the
// cds LB policy to a leaf cluster of type EDS and verifies the load balancing
// configuration pushed to the priority LB policy contains a single discovery
// mechanism.
// configuration pushed to the outlier detection LB policy contains the leaf cluster config.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, please wrap the comment in 80 column. Here and elsewhere.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Comment on lines +260 to +264
if isAggregate {
childCfgBytes, endpoints, err = buildAggregateClusterConfigJSON(b.priorities, &b.xdsLBPolicy)
} else {
childCfgBytes, endpoints, err = buildLeafClusterConfigJSON(b.priorities, &b.xdsLBPolicy)
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the variable topLBName can be set here only instead of setting it in another if-else block.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

after reverting gracefulswitch this actually has to stay that way, because we need to check first if the top policy type has changed due to update (e.g. update from non-aggregate to aggregate cluster)

Comment on lines +155 to +157
select {
case <-exitIdleCh:
default:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need this here?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be called more than once in test cases where balancers are being reconfigured, e.g. transition from non-aggregate to aggregate cluster (TestAggregatedClusterSuccess_SwitchBetweenLeafAndAggregate).

However, those channels are not used in the mentioned aggregate test case. We may consider having these exitIdleCh and closeCh channels configurable and remove this default option. WDYT?

Comment on lines +163 to +164
select {
case <-closeCh:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, Close() will not going to be called more that once for a balancer.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Look at my previous comment

Comment on lines 135 to 136
func (s) TestBuildAggregateClusterConfigJSON(t *testing.T) {
testLRSServerConfig, err := bootstrap.ServerConfigForTesting(bootstrap.ServerConfigTestingOptions{

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You have added a similar test for leaf cluster TestBuildLeafClusterConfigJSON. I think we can merge both in a table-driven test.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

merged into a table-driven test case

return retEndpoint
}

func (s) TestBuildLeafClusterConfigJSON(t *testing.T) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a docstring about what the test is doing. Here and elsewhere.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

@mswierq
mswierq requested a review from Pranjali-2501 August 12, 2026 14:54
@easwars easwars assigned Pranjali-2501 and unassigned mswierq Aug 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area: xDS Includes everything xDS related, including LB policies used with xDS. Type: Feature New features or improvements in behavior

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants