All pull requests

Pull requests

• Integrate TrajectoryStore across TrajectoryCollectEngine, RolloutManager, and RolloutWorker to log turn steps and trajectory metadata during episode collection:

  #2773·copybara-service[bot] opened Oct 9, 2026Bot·

  ·
  

• fix(deployment): add preStop hook to colocated-python-sidecar and wait for trainer pod deletion

  #2772·YixuanWang-99 opened Oct 8, 2026Collaborator·

  ·
  3

  

• feat(weight_sync): support opt-in parallel H2H transfer and deferred H2D

  #2771·knightwh opened Oct 8, 2026Collaborator·6 tasks·

  ·
  4

  

• Persist MoE router replay assignments in the trajectory store as each turn's raw

  #2770·copybara-service[bot] opened Oct 8, 2026Bot·

  ·
  

• feat(rollout-ft): rollout worker fault tolerance end-to-end (combined #2744-#2749)

  #2767·lc5211 opened Oct 8, 2026Collaborator·

  ·
  8

  

• FileTrajectoryStore.get_trajectories and get_trajectories_metadata accessed one file at a time, so on remote filesystems such as GCS every existence check, read, and directory listing added its full latency to the call.

  #2766·copybara-service[bot] opened Oct 8, 2026Bot·

  ·
  

• Pick up tokamax splash segment pruning via tpu-inference pin

  #2765·igorts-git opened Oct 8, 2026Collaborator·

  ·
  

• Capturing an agent step costs the rollout thread time proportional to its token count, in three places. converter.create_agent_step converted token IDs and logprobs one NumPy scalar at a time. Pydantic validated each IntArray and FloatArray field by first converting the array to a list, which it discarded once the array matched np.ndarray exactly. WriteTask and InMemoryTrajectoryStore deep-copy every step and metadata with model_copy(deep=True), which copies token and logprob lists one element at a time.

  #2764·copybara-service[bot] opened Oct 8, 2026Bot·

  ·
  

• [Tunix] Remove metadata_cls and metadata_type from TrajectoryStore.

  #2763·copybara-service[bot] opened Oct 8, 2026Bot·

  ·
  

• Remove the MLPerf RCP logging (mllog) integration from Tunix

  #2762·copybara-service[bot] opened Oct 8, 2026Bot·

  ·
  

• Enable SqlTrajectoryStore for MLPerf DeepSWE and GSM8K rollouts

  #2760·joshua-goog opened Oct 8, 2026·

  ·
  4

  

• Bounds AsyncWriter's internal FIFO queue (DEFAULT_MAX_QUEUE_SIZE = 10,000) so rollout workers never block or OOM during storage latency spikes:

  #2759·copybara-service[bot] opened Oct 8, 2026Bot·

  ·
  1

  

• cli: honor optimizer_config.max_grad_norm in create_optimizer

  #2758·yangziao56 opened Oct 8, 2026·6 tasks done·

  ·
  4

  

• Add the two inspection subcommands of trajectory-explore, both built on the shared commands/stats.py derivation so they report identical numbers:

  #2757·copybara-service[bot] opened Oct 8, 2026Bot·

  ·
  

• FileTrajectoryStore.get_trajectories and get_trajectories_metadata accessed one file at a time, so on remote filesystems such as GCS every existence check, read, and directory listing added its full latency to the call.

  #2756·copybara-service[bot] opened Oct 8, 2026Bot·

  ·
  1

  

• Capturing an agent step costs the rollout thread time proportional to its token count, in three places. converter.create_agent_step converted token IDs and logprobs one NumPy scalar at a time. Pydantic validated each IntArray and FloatArray field by first converting the array to a list, which it discarded once the array matched np.ndarray exactly. WriteTask and InMemoryTrajectoryStore deep-copy every step and metadata with model_copy(deep=True), which copies token and logprob lists one element at a time.

  #2754·copybara-service[bot] opened Oct 8, 2026Bot·

  ·
  1

  

• Partitions the PostgreSQL steps table by HASH (run_id) across 16 fixed modulus buckets (steps_p00 .. steps_p15) to bound per-partition row count and B-tree index size while enabling single-partition pruning on WHERE run_id = ... queries without unbounded per-run partition growth:

  #2753·copybara-service[bot] opened Oct 8, 2026Bot·

  ·
  1

  

• Moves database schema initialization (initialize_schema) and its concurrency locks (_SCHEMA_INIT_LOCK and _POSTGRES_SCHEMA_INIT_LOCK_ID) from sql_store.py into db_engine.py so SqlTrajectoryStore only manages DML and DQL operations.

  #2752·copybara-service[bot] opened Oct 8, 2026Bot·

  ·
  1

  

• feat: add Gemma4-31B text mapping for SGL-JAX

  #2751·Yanko-7 opened Oct 8, 2026·4 of 6 tasks·

  ·
  8

  

• feat(rollout-ft): [6/6] fault-tolerance observability metrics (cl/994400180)

  #2749·lc5211 opened Oct 8, 2026Collaborator·

  ·
  1

  

• feat(rollout-ft): [5/6] zero-worker retention wait, unknown transfer state recovery, and [rollout-ft] logging (cl/992984415)

  #2748·lc5211 opened Oct 8, 2026Collaborator·

  ·
  2

  

• feat(rollout-ft): [4/6] task_timeout_s silent hang detection, strict request_id matching, and retry budgets (cl/992722943)

  #2747·lc5211 opened Oct 8, 2026Collaborator·

  ·
  2

  

• feat(rollout-ft): [3/6] WeightSyncCoordinator pre-check eviction retry and PENDING_WEIGHT_SYNC promotion (cl/994006794)

  #2746·lc5211 opened Oct 8, 2026Collaborator·

  ·
  2

  

• feat(rollout-ft): [2/6] policy-plane RolloutFaultToleranceConfig and 3-state WorkerRegistry (cl/992595043)

  #2745·lc5211 opened Oct 8, 2026Collaborator·

  ·
  2

  

• feat(rollout-ft): [1/6] data-plane fault detection and dynamic actor pool primitives (cl/994441932)

  #2744·lc5211 opened Oct 8, 2026Collaborator·

  ·
  2