Skip to content

Merge engine capabilities across heterogeneous workers #5467

Description

@amumu96

Context

Supervisor engine discovery queries workers for LLM, embedding, and rerank capabilities, but currently returns the first non-empty worker result instead of merging results across workers.

Impact

In a heterogeneous cluster, such as GPU workers mixed with NPU workers that expose MindIE, the advertised engine set depends on worker ordering. An engine available on a later worker may be omitted even though that worker can launch it.

Expected behavior

  • Merge compatible engine capability results from all relevant workers.
  • Preserve deterministic ordering and deduplicate engine names.
  • Define behavior for unavailable or partially failing workers.
  • Add mixed-capability worker tests.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions