Skip to content

Add REST accessor for decoded WAGED assignments - #222

Open
LZD-PratyushBhatt wants to merge 1 commit into
devfrom
lzd/waged-assignment-rest
Open

Add REST accessor for decoded WAGED assignments#222
LZD-PratyushBhatt wants to merge 1 commit into
devfrom
lzd/waged-assignment-rest

Conversation

@LZD-PratyushBhatt

Copy link
Copy Markdown
Collaborator

The controller persists the WAGED baseline and best possible assignments through ZkBucketDataAccessor, which GZIPs the serialized record and splits it across numbered bucket ZNodes. Reading those ZNodes over the existing /zookeeper API therefore returns opaque compressed chunks, and the propertyStore API cannot reach them at all because ASSIGNMENT_METADATA lives at the cluster root rather than under PROPERTYSTORE. The only other assignment API, /partitionAssignment, recomputes a what-if placement and never reads what was actually persisted.

Add WagedAssignmentAccessor, which reassembles the buckets, decompresses, and deserializes each resource assignment server side:

GET /clusters/{clusterId}/wagedAssignment/bestPossible
GET /clusters/{clusterId}/wagedAssignment/baseline

Supported query params:
format=IdealStateFormat (default) | CurrentStateFormat
resources, instances, partitions - comma separated allowlists, since
these payloads are large on real clusters
includeMetadata (default true) - the persisted write version, its ZK
mtime, and the bucket metadata, so callers can tell how fresh the
decoded assignment is

A missing assignment returns 404 rather than an empty body, so callers can distinguish "WAGED never persisted here" from "assignment is empty".

The path layout and the per-resource decode are exposed from AssignmentMetadataStore instead of being duplicated in helix-rest, so the reader cannot drift from the writer.

Issues

  • My PR addresses the following Helix issues and references them in the PR description:

(#200 - Link your issue number here: You can write "Fixes #XXX". Please use the proper keyword so that the issue gets closed automatically. See https://docs.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue
Any of the following keywords can be used: close, closes, closed, fix, fixes, fixed, resolve, resolves, resolved)

Description

  • Here are some details about my PR, including screenshots of any UI changes:

(Write a concise description including what, why, how)

Tests

  • The following tests are written for this issue:

(List the names of added unit/integration tests)

  • The following is the result of the "mvn test" command on the appropriate module:

(If CI test fails due to known issue, please specify the issue and test PR locally. Then copy & paste the result of "mvn test" to here.)

Changes that Break Backward Compatibility (Optional)

  • My PR contains changes that break backward compatibility or previous assumptions for certain methods or API. They include:

(Consider including all behavior changes for public methods or API. Also include these changes in merge description so that other developers are aware of these changes. This allows them to make relevant code changes in feature branches accounting for the new method/API behavior.)

Documentation (Optional)

  • In case of new functionality, my PR adds documentation in the following wiki page:

(Link the GitHub wiki you added)

Commits

  • My commits all reference appropriate Apache Helix GitHub issues in their subject lines. In addition, my commits follow the guidelines from "How to write a good git commit message":
    1. Subject is separated from body by a blank line
    2. Subject is limited to 50 characters (not including Jira issue reference)
    3. Subject does not end with a period
    4. Subject uses the imperative mood ("add", not "adding")
    5. Body wraps at 72 characters
    6. Body explains "what" and "why", not "how"

Code Quality

  • My diff has been formatted using helix-style.xml
    (helix-style-intellij.xml if IntelliJ IDE is used)

The controller persists the WAGED baseline and best possible assignments
through ZkBucketDataAccessor, which GZIPs the serialized record and splits
it across numbered bucket ZNodes. Reading those ZNodes over the existing
/zookeeper API therefore returns opaque compressed chunks, and the
propertyStore API cannot reach them at all because ASSIGNMENT_METADATA
lives at the cluster root rather than under PROPERTYSTORE. The only other
assignment API, /partitionAssignment, recomputes a what-if placement and
never reads what was actually persisted.

Add WagedAssignmentAccessor, which reassembles the buckets, decompresses,
and deserializes each resource assignment server side:

  GET /clusters/{clusterId}/wagedAssignment/bestPossible
  GET /clusters/{clusterId}/wagedAssignment/baseline

Supported query params:
  format=IdealStateFormat (default) | CurrentStateFormat
  resources, instances, partitions - comma separated allowlists, since
    these payloads are large on real clusters
  includeMetadata (default true) - the persisted write version, its ZK
    mtime, and the bucket metadata, so callers can tell how fresh the
    decoded assignment is

A missing assignment returns 404 rather than an empty body, so callers can
distinguish "WAGED never persisted here" from "assignment is empty".

The path layout and the per-resource decode are exposed from
AssignmentMetadataStore instead of being duplicated in helix-rest, so the
reader cannot drift from the writer.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@LZD-PratyushBhatt
LZD-PratyushBhatt force-pushed the lzd/waged-assignment-rest branch from d646388 to 1172997 Compare August 9, 2026 01:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant