REP: Ray Batch Actor Scheduling - #70
Conversation
Signed-off-by: Andrew Sy Kim <andrewsy@google.com>
b8b6d90 to
2d7daf6
Compare
Yicheng-Lu-llll
left a comment
Yicheng-Lu-llll
left a comment
There was a problem hiding this comment.
Thanks! left some nits.
| To achieve this, we extend the actor creation workflow as follows: | ||
|
|
||
| * **ray.batch() API**: The `ray.batch()` context manager signals to the Core Worker (e.g. Cython CoreWorkerProcess.GetCoreWorker().EnterActorBatch()) to enter batching mode for the current thread. Upon exit, it calls ExitActorBatch(). | ||
| * **Core Worker Actor Buffering**: Inside CoreWorker::CreateActor, if "batch mode" is enabled, the TaskSpecification for actors are pushed into a thread local buffer. |
There was a problem hiding this comment.
Brainstorming here: what happens if actors need dependencies? Some actors' dependencies get resolved quickly, while others resolve very slowly (waiting for an upstream task to produce some object).
This dependency requirement might be unusual for the RL case, but if it does happen, then for the actors still waiting on dependency resolution, we can only register them but cannot submit. For the ones whose dependencies are already resolved, we can submit.
I'm wondering how we should handle this. Would we end up submitting one by one? It seems like we can only batch the Register phase but cannot control the Submission Execution, so my sense is that we should keep the Submission Execution as is.
There was a problem hiding this comment.
or opportunistically batching
| * When ExitActorBatch() is called, it flushes the buffer by invoking an RPC to the GCS server asynchronously to register all the actors in the buffer in one atomic operation (see RegisterActorBatchRequest below). | ||
| * **GCS RPC Extension (RegisterActorBatch)**: A new RPC RegisterActorBatchRequest will be added to gcs_service.proto. | ||
| * GcsActorManager implements HandleRegisterActorBatch, which iterates through the batched tasks and invokes RegisterActor for each task locally in the GCS. | ||
| * Once all tasks are successfully registered in the backend storage, a single RPC reply is returned to the Core Worker. |
There was a problem hiding this comment.
If I remember correctly, for named actors, we don't return the actor handle unless the register response comes back. But for normal actors, we don't wait.
My personal thinking is that the reason register phase exists, and we need to wait synchronously is we need to handle this case: if an actor is still waiting on dependency resolution and we do not register that actor first, then GCS does not know about this actor. But the actor handle may have already been given to others, and if the owner dies, those who hold the actor handles will wait forever.
So it makes sense to me that we should synchronously wait for all register responses. But we need to understand why we don't wait for normal actors. Is this for performance reasons? Will we cause a regression here?
| * **GCS RPC Extension (RegisterActorBatch)**: A new RPC RegisterActorBatchRequest will be added to gcs_service.proto. | ||
| * GcsActorManager implements HandleRegisterActorBatch, which iterates through the batched tasks and invokes RegisterActor for each task locally in the GCS. | ||
| * Once all tasks are successfully registered in the backend storage, a single RPC reply is returned to the Core Worker. | ||
| * **Submission Execution**: After the GCS responds to the RegisterActorBatch RPC, the Core Worker loops over the batch and calls actor_task_submitter_->SubmitActorCreationTask(task_spec) to push the tasks to the placement group / scheduling queue. |
There was a problem hiding this comment.
Could we also clarify the behavior when some of the actors get scheduled but the rest fail due to insufficient resources? Though I think the behavior would be the same.
|
|
||
| ### General Motivation | ||
|
|
||
| Ray currently processes actor creation requests one by one. For workloads that spawn hundreds or thousands of actors (e.g., large-scale RL), this sequential creation can become a bottleneck because each actor registration requires a separate RPC call to the GCS server, and the scheduling of actors may be suboptimal when evaluated on a per-actor basis. |
There was a problem hiding this comment.
it's also worth mentioning that, particularly for named actors, we need to wait for the register RPC to come back before we can start the next one. Since the current call has to return first, this serialization ends up slowing everything down. Normal actors can do this concurrently. I would expect Batch really helps here.
2 participants
No description provided.