fix(colocate): derive num_gpus_per_node from actor_num_gpus_per_node

[aoshen02]

Problem

In colocate mode the rollout engines share the actor's physical nodes, so the GPUs-per-physical-node always equals actor_num_gpus_per_node. But --num-gpus-per-node defaults to 8 (an 8-GPU/node assumption). On hardware with a different per-node GPU count (e.g. 4× GB200/node), that default is wrong for multi-node colocate runs.

_allocate_rollout_engine_addr_and_ports_normal does:

num_engines_per_node = max(1, args.num_gpus_per_node // _gpus_per_engine)
...
node_index = local_rank // num_engines_per_node

With num_gpus_per_node=8 but only 4 GPUs/node and rollout_num_gpus_per_engine=2, num_engines_per_node becomes 4 instead of 2, so every engine maps to node_index=0 and is handed the head node's IP. Worker-node engines then fail to bind:

OSError: [Errno 99] Cannot assign requested address
  -> server exited unexpectedly

The help text already documents the manual workaround ("If you are going to use less than 8 gpus per node under colocate mode, you should set this number"), but it's silently easy to forget and the resulting failure is opaque.

Fix

Since the value is fully determined in colocate mode, derive it automatically from actor_num_gpus_per_node:

if args.num_gpus_per_node != args.actor_num_gpus_per_node:
    logger.info("colocate: overriding num_gpus_per_node ... -> actor_num_gpus_per_node ...")
    args.num_gpus_per_node = args.actor_num_gpus_per_node

Scope / safety

• Single-node 8-GPU runs: unaffected — actor_num_gpus_per_node == 8 == default, so the override is a no-op.
• Non-colocate (disaggregated) runs: untouched (the override is inside the if args.colocate: block), preserving the ability to configure a rollout cluster with a different per-node GPU count than the actor.
• Consuming/allocation logic is unchanged.

Verification

Reproduced and fixed on a 2-node, 4-GPU/node Blackwell colocate run:

• Before: all rollout engines report the head node's IP → worker engines Errno 99.
• After: log shows colocate: overriding num_gpus_per_node 8 -> 4; engines split correctly across head (engines 0,1) and worker (engines 2,3); bind succeeds.

[ishankaul1]

FYI -- we hit this exact bug while validating a SkyPilot launch tutorial for the docs (#2303).

Repro

2 nodes × 4×H100 on Kubernetes; equivalent to the quick-start Qwen3-4B recipe in colocate mode with --rollout-num-gpus-per-engine 2.

With the default --num-gpus-per-node 8, every engine maps to node_index = 0, and the second node's engines hang for 600s connecting to a TCPStore allocated on the first node's IP. We saw the connect-timeout variant (DistNetworkError: The client socket has timed out ... while trying to connect) rather than Errno 99, so worth knowing both signatures trace here.

Workaround

Setting --num-gpus-per-node 4 fixed it immediately in the same environment.

---

+1 for deriving it from actor_num_gpus_per_node in colocate mode; the failure is opaque enough that the help-text workaround is easy to miss.