Migration Guide from MPIJob to TrainJob

[andreyvelich]

At the previous Training WG calls we discussed that it would be nice to update our migration guide for MPI Operator users. It would be great if we could consolidate our efforts to better support MPI on Kubernetes.

That might helpful for users who run MPIJobs for ML use-cases today, but also for folks who want to use it with Kubeflow SDK and Pythonic interface.

In the migration guide, we should tell that currently Trainer supports OpenMPI, but community is working towards advanced HPC features to support other MPI backends or AI/HPC schedulers:

• MPICH backend support: feat(operator): extend MPI plugin with Intel and MPICH backends #3717
• Flux support
• Slurm Bridge integrations: feat: Add slurm-bridge scheduling documentation #3951

Ref discussion: https://youtu.be/SQGjGTvjObo?t=1351

cc @vsoch @tenzen-y @kannon92 @killianmuldoon

[andreyvelich]

/area mpi
/kind docs
/good-first-issue

[google-oss-prow]

@andreyvelich:
This request has been marked as suitable for new contributors.

Please ensure the request meets the requirements listed here.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-good-first-issue command.

Details

In response to this:

/area mpi
/kind docs
/good-first-issue

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[uzairhameed]

hey @andreyvelich I would like to work on this, can you assign me?

[Sumitha007]

Hi @andreyvelich, I’d like to work on this issue. If it’s still available, could you please assign it to me?

[Sumitha007]

Hi @andreylevich, I had already started working on this issue and have opened a PR for the migration guide. I noticed that the issue has now been assigned to @hari-599. Could you please confirm whether I should continue with my PR or let @hari-599 take over?