Skip to content

[P1] Establish backend identification for post-training and application frameworks #193

Description

@Flink-ddd

Context

As part of the P1: End-to-End Loop and Joint Validation goal, we are shipping a credible end-to-end path that integrates vLLM rollout with RL-Kernel's logprob/loss kernels.

To seamlessly support post-training distributed scheduling and application frameworks like veRL and slime and vime, we need to establish a clear, non-intrusive backend identification mechanism. This ensures the upper-level frameworks can dynamically route computations to our optimized kernels without hardcoding.

Description

Introduce a configuration flag (e.g., use_rl_kernel_backend=True) within the integration layers of veRL and slime. When enabled, this flag will instruct the framework's actor/learner to dispatch rollout, logprob, and loss computations to the RL-Kernel backend instead of the default paths.

Proposed Implementation

  • Flag Design: Define and expose use_rl_kernel_backend (or similar backend identifier) in the configuration schemas for both veRL and slime and vime.
  • Routing & Dispatch: Wire the initialization logic so that when the flag is True, the framework swaps in RL-Kernel custom ops.
  • Fallback Mechanism: Ensure that if the flag is False or the backend is unavailable, it gracefully falls back to the default implementation without breaking existing workflows.

Acceptance Criteria

  • Backend identification flag is successfully integrated into both veRL and slime and vime.
  • The integration is non-intrusive; default framework behaviors remain completely unaffected when the flag is off.
  • With use_rl_kernel_backend=True, an end-to-end loop runs without errors, and rollout-vs-training consistency is successfully validated on real workloads.
  • A user guide in a doc file

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureplatform: cudaSpecific optimizations or bugs in NVIDIA graphics cards (such as FlashInfer, TMA optimizations)platform: rocmSpecific tasks specific to AMD graphics cards (such as CK, bpreshuffle/FA)priority: highSevere congestion issues require the highest priority for resolution.type: designIssues requiring in-depth discussion of architecture design

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions