From b7082e7e06320e87b03d4f6946cef3c10c9a3b96 Mon Sep 17 00:00:00 2001 From: Ishan Date: Wed, 19 Aug 2026 10:58:22 -0700 Subject: [PATCH 1/3] docs: add SkyPilot launch tutorial under platform_support Adds docs/en/platform_support/skypilot_tutorial.md covering multi-node launch (provision nodes, start Ray, submit train.py from one YAML) and a disaggregated trainer + SGLang-engines setup, with a link to the end-to-end agentic RL example in the SkyPilot repo. Registers the page in the docs toctree and cross-references it from the quick start's multi-node section. Co-Authored-By: Claude Fable 5 --- docs/en/get_started/quick_start.md | 2 + docs/en/index.rst | 3 +- docs/en/platform_support/skypilot_tutorial.md | 116 ++++++++++++++++++ 3 files changed, 120 insertions(+), 1 deletion(-) create mode 100644 docs/en/platform_support/skypilot_tutorial.md diff --git a/docs/en/get_started/quick_start.md b/docs/en/get_started/quick_start.md index d0bd3fb780..81786d0ed6 100644 --- a/docs/en/get_started/quick_start.md +++ b/docs/en/get_started/quick_start.md @@ -584,6 +584,8 @@ export NCCL_SOCKET_IFNAME=$(ip -o -4 addr show | awk '$4 ~ /^10\\./ {print $2}') export NVSHMEM_BOOTSTRAP_UID_SOCK_IFNAME=$(ip -o -4 addr show | awk '$4 ~ /^10\./ {print $2}') ``` +For launching the same multi-node setup on Kubernetes or cloud instances, see the [SkyPilot tutorial](../platform_support/skypilot_tutorial.md). + slime has been deeply optimized for distributed training of large-scale Mixture of Experts (MoE) models. We provide some end-to-end training cases for reference: - [Example: 8xH100 Training GLM-4.7-Flash](../examples/glm4.7-30B-A3B.md) diff --git a/docs/en/index.rst b/docs/en/index.rst index 83230af1fa..d7a1c5df0e 100644 --- a/docs/en/index.rst +++ b/docs/en/index.rst @@ -110,9 +110,10 @@ Start by Use Case .. toctree:: :maxdepth: 1 - :caption: Hardware Platforms + :caption: Platforms platform_support/amd_tutorial.md + platform_support/skypilot_tutorial.md .. toctree:: :maxdepth: 1 diff --git a/docs/en/platform_support/skypilot_tutorial.md b/docs/en/platform_support/skypilot_tutorial.md new file mode 100644 index 0000000000..18afbf39c5 --- /dev/null +++ b/docs/en/platform_support/skypilot_tutorial.md @@ -0,0 +1,116 @@ +# SkyPilot + +[SkyPilot](https://github.com/skypilot-org/skypilot) is an open-source framework for running workloads on Kubernetes or any cloud. This tutorial shows how to launch multi-node slime training with SkyPilot: node provisioning, Ray cluster startup, and job submission are described in a single YAML, replacing the per-node `ray start` steps from the [Quick Start](../get_started/quick_start.md). + +It covers two setups: + +- **Multi-node training on one cluster** — the standard setup from the Quick Start's multi-node section. +- **Disaggregated training and inference** — the trainer and SGLang engines run as separate, gang-scheduled jobs that scale independently. + +This page is maintained by the SkyPilot maintainers. + +## Prerequisites + +Install SkyPilot with the extras for your infrastructure and confirm it can reach it: + +```bash +pip install "skypilot[kubernetes]" # or [aws], [gcp], ... — see SkyPilot docs +sky check +``` + +The examples below use the `slimerl/slime:latest` Docker image from the Quick Start, so no additional environment setup is needed inside the nodes. + +## Multi-Node Training on One Cluster + +The Quick Start starts a Ray cluster by running `ray start` on every node, then submits training with `ray job submit` from node 0. The following SkyPilot task performs the same steps: it provisions `num_nodes` nodes with GPUs, starts the Ray head and workers, and submits the job. Environment variables like `SKYPILOT_NODE_RANK` and `SKYPILOT_NODE_IPS` are injected by SkyPilot on every node. + +```yaml +# slime-multinode.yaml +resources: + infra: kubernetes # or aws / gcp / any infra configured in `sky check` + accelerators: H100:8 + image_id: docker:slimerl/slime:latest + +num_nodes: 2 + +workdir: . # ship your training scripts to every node + +run: | + MASTER_ADDR=$(echo "$SKYPILOT_NODE_IPS" | head -n1) + if [ "$SKYPILOT_NODE_RANK" == "0" ]; then + ray start --head --node-ip-address ${MASTER_ADDR} \ + --num-gpus 8 --disable-usage-stats + + # Wait until every node has joined the Ray cluster. + while [ "$(ray list nodes --format json | python3 -c 'import json,sys; print(len(json.load(sys.stdin)))')" -lt "$SKYPILOT_NUM_NODES" ]; do + sleep 5 + done + + ray job submit --address="http://127.0.0.1:8265" \ + --runtime-env-json='{ + "env_vars": { + "PYTHONPATH": "/root/Megatron-LM/" + } + }' \ + -- python3 train.py \ + --... # Megatron/SGLang/slime arguments, as in the Quick Start + else + sleep 10 + ray start --address=${MASTER_ADDR}:6379 --num-gpus 8 + fi +``` + +Launch it with: + +```bash +sky launch -c slime-train slime-multinode.yaml +``` + +SkyPilot provisions the nodes (creating them if needed), runs `run` on each node, and streams the logs. `sky down slime-train` tears the cluster down. The same YAML can be launched as a managed job with `sky jobs launch`, which adds automatic recovery from node failures. + +## Disaggregated Training and Inference + +slime supports connecting the trainer to SGLang engines launched by an external system (`--rollout-external-engine-addrs`, see [External Rollout Engines](../advanced/external-rollout-engines.md)). With a SkyPilot **Job Group**, the trainer and each engine are separate jobs in one YAML that are gang-scheduled together and reach each other by stable hostname (`.`), so the fleet of engines can be sized independently of the trainer: + +```yaml +# One Job Group: 1 trainer + 1 SGLang engine (add more engine jobs to scale). +--- +name: slime-rl +execution: parallel +primary_tasks: [trainer] +--- +name: sglang +resources: + infra: kubernetes + accelerators: H100:1 + image_id: docker:slimerl/slime:latest +volumes: + /shared/policy: slime-policy # shared RWX volume for disk weight sync +run: | + # Serve SGLang; the trainer reaches this engine at sglang.. + ... +--- +name: trainer +resources: + infra: kubernetes + accelerators: H100:4 + image_id: docker:slimerl/slime:latest +volumes: + /shared/policy: slime-policy +envs: + SGLANG_MEMBERS: "sglang" # engine job names; add sglang-2, sglang-3, ... +run: | + # Start Ray + slime with --rollout-external-engine-addrs pointed at the engines, + # publishing weights per step via the shared volume (delta or full) or NCCL. + ... +``` + +After each optimizer step the trainer publishes updated weights and the engines reload them — over the shared volume with `--update-weight-transport disk` (optionally `--update-weight-mode delta` to ship only changed bytes), or over NCCL ([Delta Weight Sync](../advanced/delta-weight-sync.md)). + +## End-to-End Example: Agentic Coding RL + +A complete, runnable version of the disaggregated setup lives in the SkyPilot repository: + +**[slime on SkyPilot Job Groups](https://github.com/skypilot-org/skypilot/tree/master/llm/slime)** — trains a coding agent (Qwen3-14B) on SWE-smith with slime: a Megatron trainer job plus 1–3 SGLang engine jobs in one Job Group, agent rollouts executing untrusted code in sandboxed pods, and disk-based delta weight sync between the jobs. The example includes launch YAMLs, all setup/run scripts, and benchmark results for scaling the inference fleet (1 → 3 engines cuts async step time from about 1200 s to about 660 s on the example workload). + +Issues with the SkyPilot setups on this page can be reported to the [SkyPilot repository](https://github.com/skypilot-org/skypilot/issues). From 119b189511fb7b05e9f4fd13b74bb848ec8a4d51 Mon Sep 17 00:00:00 2001 From: Ishan Date: Wed, 19 Aug 2026 17:01:36 -0700 Subject: [PATCH 2/3] docs(skypilot): make tutorial YAMLs complete and smoke-validated Both YAMLs now run end-to-end as written (validated on a 2x4xH100 Kubernetes cluster): full Qwen3-4B GRPO recipe args, engine/trainer setup, Ray bring-up that coexists with SkyPilot's node runtime (non-default agent/metrics ports, job-agent readiness probe, --block workers), --num-gpus-per-node passed for colocate engine mapping on <8-GPU nodes, and inter_connection: true on the Job Group. Co-Authored-By: Claude Fable 5 --- docs/en/platform_support/skypilot_tutorial.md | 271 +++++++++++++++--- 1 file changed, 232 insertions(+), 39 deletions(-) diff --git a/docs/en/platform_support/skypilot_tutorial.md b/docs/en/platform_support/skypilot_tutorial.md index 18afbf39c5..ab3c9dded4 100644 --- a/docs/en/platform_support/skypilot_tutorial.md +++ b/docs/en/platform_support/skypilot_tutorial.md @@ -2,7 +2,7 @@ [SkyPilot](https://github.com/skypilot-org/skypilot) is an open-source framework for running workloads on Kubernetes or any cloud. This tutorial shows how to launch multi-node slime training with SkyPilot: node provisioning, Ray cluster startup, and job submission are described in a single YAML, replacing the per-node `ray start` steps from the [Quick Start](../get_started/quick_start.md). -It covers two setups: +It covers two setups, both running the Quick Start's Qwen3-4B GRPO recipe (`scripts/run-qwen3-4B.sh`) on the DAPO-math dataset: - **Multi-node training on one cluster** — the standard setup from the Quick Start's multi-node section. - **Disaggregated training and inference** — the trainer and SGLang engines run as separate, gang-scheduled jobs that scale independently. @@ -22,62 +22,161 @@ The examples below use the `slimerl/slime:latest` Docker image from the Quick St ## Multi-Node Training on One Cluster -The Quick Start starts a Ray cluster by running `ray start` on every node, then submits training with `ray job submit` from node 0. The following SkyPilot task performs the same steps: it provisions `num_nodes` nodes with GPUs, starts the Ray head and workers, and submits the job. Environment variables like `SKYPILOT_NODE_RANK` and `SKYPILOT_NODE_IPS` are injected by SkyPilot on every node. +The Quick Start starts a Ray cluster by running `ray start` on every node, then submits training with `ray job submit` from node 0. The following SkyPilot task performs the same steps: it provisions `num_nodes` nodes with GPUs, downloads and converts the model on each node, starts the Ray head and workers, and submits the job. Environment variables like `SKYPILOT_NODE_RANK` and `SKYPILOT_NODE_IPS` are injected by SkyPilot on every node. ```yaml # slime-multinode.yaml resources: infra: kubernetes # or aws / gcp / any infra configured in `sky check` - accelerators: H100:8 + accelerators: H100:4 image_id: docker:slimerl/slime:latest num_nodes: 2 -workdir: . # ship your training scripts to every node +setup: | + pip install -q -U "huggingface_hub[cli]" + [ -d /root/Qwen3-4B ] || hf download Qwen/Qwen3-4B --local-dir /root/Qwen3-4B + [ -d /root/dapo-math-17k ] || hf download --repo-type dataset zhuzilin/dapo-math-17k --local-dir /root/dapo-math-17k + [ -d /root/aime-2024 ] || hf download --repo-type dataset zhuzilin/aime-2024 --local-dir /root/aime-2024 + # Convert the HF checkpoint to Megatron torch_dist format (each node needs a local copy). + if [ ! -d /root/Qwen3-4B_torch_dist ]; then + cd /root/slime + source scripts/models/qwen3-4B.sh + PYTHONPATH=/root/Megatron-LM python tools/convert_hf_to_torch_dist.py \ + ${MODEL_ARGS[@]} --hf-checkpoint /root/Qwen3-4B --save /root/Qwen3-4B_torch_dist + fi run: | MASTER_ADDR=$(echo "$SKYPILOT_NODE_IPS" | head -n1) - if [ "$SKYPILOT_NODE_RANK" == "0" ]; then - ray start --head --node-ip-address ${MASTER_ADDR} \ - --num-gpus 8 --disable-usage-stats - - # Wait until every node has joined the Ray cluster. - while [ "$(ray list nodes --format json | python3 -c 'import json,sys; print(len(json.load(sys.stdin)))')" -lt "$SKYPILOT_NUM_NODES" ]; do - sleep 5 - done - - ray job submit --address="http://127.0.0.1:8265" \ - --runtime-env-json='{ - "env_vars": { - "PYTHONPATH": "/root/Megatron-LM/" - } - }' \ - -- python3 train.py \ - --... # Megatron/SGLang/slime arguments, as in the Quick Start - else + if [ "$SKYPILOT_NODE_RANK" != "0" ]; then + # Worker nodes join the Ray cluster. --block keeps the worker's Ray daemons in the + # foreground for the whole run (an exiting run command would get them reaped) and + # returns once the head shuts down at the end of training. sleep 10 - ray start --address=${MASTER_ADDR}:6379 --num-gpus 8 + ray start --address=${MASTER_ADDR}:6379 --num-gpus ${SKYPILOT_NUM_GPUS_PER_NODE} --disable-usage-stats \ + --dashboard-agent-listen-port 52366 --metrics-export-port 8091 --block + exit 0 fi + + # Start Ray from /root/slime: job entrypoints run in the head's working directory. + cd /root/slime + source scripts/models/qwen3-4B.sh + + # Non-default agent/metrics ports: SkyPilot's runtime on the node runs its own Ray. + ray start --head --node-ip-address ${MASTER_ADDR} \ + --num-gpus ${SKYPILOT_NUM_GPUS_PER_NODE} --disable-usage-stats \ + --dashboard-host=0.0.0.0 --dashboard-port=8265 \ + --dashboard-agent-listen-port 52366 --metrics-export-port 8091 + + # Wait until every node has joined the Ray cluster. + until python3 -c "import ray, sys; ray.init(address='${MASTER_ADDR}:6379', logging_level='error'); sys.exit(0 if len([n for n in ray.nodes() if n['Alive']]) >= ${SKYPILOT_NUM_NODES} else 1)"; do sleep 5; done + + # Wait for Ray's job agent to be ready to accept submissions. + until ray job submit --address="http://127.0.0.1:8265" --no-wait -- true >/dev/null 2>&1; do + echo "waiting for the Ray job agent..."; sleep 5 + done + ray job submit --address="http://127.0.0.1:8265" \ + --runtime-env-json='{"env_vars": {"PYTHONPATH": "/root/Megatron-LM/", "CUDA_DEVICE_MAX_CONNECTIONS": "1"}}' \ + -- python3 /root/slime/train.py \ + --actor-num-nodes ${SKYPILOT_NUM_NODES} \ + --actor-num-gpus-per-node ${SKYPILOT_NUM_GPUS_PER_NODE} \ + --num-gpus-per-node ${SKYPILOT_NUM_GPUS_PER_NODE} \ + --colocate \ + ${MODEL_ARGS[@]} \ + --hf-checkpoint /root/Qwen3-4B \ + --ref-load /root/Qwen3-4B_torch_dist \ + --load /root/Qwen3-4B_slime/ \ + --save /root/Qwen3-4B_slime/ \ + --save-interval 20 \ + --prompt-data /root/dapo-math-17k/dapo-math-17k.jsonl \ + --input-key prompt \ + --label-key label \ + --apply-chat-template \ + --rollout-shuffle \ + --rm-type deepscaler \ + --num-rollout 3000 \ + --rollout-batch-size 32 \ + --n-samples-per-prompt 8 \ + --rollout-max-response-len 8192 \ + --rollout-temperature 1 \ + --global-batch-size 256 \ + --balance-data \ + --eval-interval 20 \ + --eval-prompt-data aime /root/aime-2024/aime-2024.jsonl \ + --n-samples-per-eval-prompt 16 \ + --eval-max-response-len 16384 \ + --eval-top-p 1 \ + --advantage-estimator grpo \ + --use-kl-loss \ + --kl-loss-coef 0.00 \ + --kl-loss-type low_var_kl \ + --entropy-coef 0.00 \ + --eps-clip 0.2 \ + --eps-clip-high 0.28 \ + --optimizer adam \ + --lr 1e-6 \ + --lr-decay-style constant \ + --weight-decay 0.1 \ + --adam-beta1 0.9 \ + --adam-beta2 0.98 \ + --tensor-model-parallel-size 2 \ + --sequence-parallel \ + --pipeline-model-parallel-size 1 \ + --context-parallel-size 1 \ + --expert-model-parallel-size 1 \ + --expert-tensor-parallel-size 1 \ + --recompute-granularity full \ + --recompute-method uniform \ + --recompute-num-layers 1 \ + --use-dynamic-batch-size \ + --max-tokens-per-gpu 9216 \ + --rollout-num-gpus-per-engine 2 \ + --sglang-mem-fraction-static 0.7 \ + --attention-dropout 0.0 \ + --hidden-dropout 0.0 \ + --accumulate-allreduce-grads-in-fp32 \ + --attention-softmax-in-fp32 \ + --attention-backend flash ``` -Launch it with: +The training arguments are the Quick Start's Qwen3-4B recipe; the only adjustments are the topology flags (`--actor-num-nodes`, `--actor-num-gpus-per-node`, and `--num-gpus-per-node`), whose values come from the SkyPilot-injected environment. `--num-gpus-per-node` matters on nodes with fewer than 8 GPUs: slime's colocated engine mapping assumes 8 per node unless told otherwise. Launch it with: ```bash sky launch -c slime-train slime-multinode.yaml ``` -SkyPilot provisions the nodes (creating them if needed), runs `run` on each node, and streams the logs. `sky down slime-train` tears the cluster down. The same YAML can be launched as a managed job with `sky jobs launch`, which adds automatic recovery from node failures. +SkyPilot provisions the nodes (creating them if needed), runs `setup` and `run` on each node, and streams the logs. `sky down slime-train` tears the cluster down. The task assumes a fresh cluster: to re-run training, recreate the cluster (`sky down slime-train && sky launch -c slime-train ...`) rather than re-launching onto one whose Ray daemons are still running. The same YAML can be launched as a managed job with `sky jobs launch`, which adds automatic recovery from node failures. ## Disaggregated Training and Inference -slime supports connecting the trainer to SGLang engines launched by an external system (`--rollout-external-engine-addrs`, see [External Rollout Engines](../advanced/external-rollout-engines.md)). With a SkyPilot **Job Group**, the trainer and each engine are separate jobs in one YAML that are gang-scheduled together and reach each other by stable hostname (`.`), so the fleet of engines can be sized independently of the trainer: +slime supports connecting the trainer to SGLang engines launched by an external system (`--rollout-external-engine-addrs`, see [External Rollout Engines](../advanced/external-rollout-engines.md)). With a SkyPilot **Job Group**, the trainer and each engine are separate jobs in one YAML that are gang-scheduled together and reach each other by stable hostname (`-0.`), so the fleet of engines can be sized independently of the trainer. + +The trainer publishes updated weights after each optimizer step and the engines reload them from a shared `ReadWriteMany` volume (`--update-weight-transport disk`). Create the volume once: ```yaml -# One Job Group: 1 trainer + 1 SGLang engine (add more engine jobs to scale). +# policy-volume.yaml +name: slime-policy +type: k8s-pvc +size: 100Gi +infra: kubernetes +config: + access_mode: ReadWriteMany +``` + +```bash +sky volumes apply policy-volume.yaml +``` + +Then launch the Job Group: + +```yaml +# slime-jobgroup.yaml --- name: slime-rl execution: parallel -primary_tasks: [trainer] +primary_tasks: [trainer] # the group succeeds/fails with the trainer +inter_connection: true # place all jobs on one cluster so they can reach each other +termination_delay: 60s --- name: sglang resources: @@ -85,31 +184,125 @@ resources: accelerators: H100:1 image_id: docker:slimerl/slime:latest volumes: - /shared/policy: slime-policy # shared RWX volume for disk weight sync + /shared/policy: slime-policy +setup: | + pip install -q -U "huggingface_hub[cli]" + [ -d /root/Qwen3-4B ] || hf download Qwen/Qwen3-4B --local-dir /root/Qwen3-4B run: | - # Serve SGLang; the trainer reaches this engine at sglang.. - ... + # One SGLang server; the trainer reaches it at sglang-0.:30000. + python -m sglang.launch_server --model-path /root/Qwen3-4B --tp 1 \ + --host 0.0.0.0 --port 30000 --mem-fraction-static 0.7 --- name: trainer resources: infra: kubernetes - accelerators: H100:4 + accelerators: H100:2 image_id: docker:slimerl/slime:latest volumes: /shared/policy: slime-policy -envs: - SGLANG_MEMBERS: "sglang" # engine job names; add sglang-2, sglang-3, ... +setup: | + pip install -q -U "huggingface_hub[cli]" + [ -d /root/Qwen3-4B ] || hf download Qwen/Qwen3-4B --local-dir /root/Qwen3-4B + [ -d /root/dapo-math-17k ] || hf download --repo-type dataset zhuzilin/dapo-math-17k --local-dir /root/dapo-math-17k + [ -d /root/aime-2024 ] || hf download --repo-type dataset zhuzilin/aime-2024 --local-dir /root/aime-2024 + # Convert the HF checkpoint to Megatron torch_dist format. + if [ ! -d /root/Qwen3-4B_torch_dist ]; then + cd /root/slime + source scripts/models/qwen3-4B.sh + PYTHONPATH=/root/Megatron-LM python tools/convert_hf_to_torch_dist.py \ + ${MODEL_ARGS[@]} --hf-checkpoint /root/Qwen3-4B --save /root/Qwen3-4B_torch_dist + fi run: | - # Start Ray + slime with --rollout-external-engine-addrs pointed at the engines, - # publishing weights per step via the shared volume (delta or full) or NCCL. - ... + # Wait for the engine job to serve (jobs in a group provision independently). + ENGINE_ADDR="sglang-0.${SKYPILOT_JOBGROUP_NAME}:30000" + until curl -sf "http://${ENGINE_ADDR}/health" >/dev/null; do + echo "waiting for engine ${ENGINE_ADDR}..."; sleep 10 + done + echo "engine healthy: ${ENGINE_ADDR}" + + # Start Ray from /root/slime: job entrypoints run in the head's working directory. + cd /root/slime + source scripts/models/qwen3-4B.sh + + # Non-default agent/metrics ports: SkyPilot's runtime on the node runs its own Ray. + ray start --head --node-ip-address 127.0.0.1 --num-gpus 2 --disable-usage-stats \ + --dashboard-host=0.0.0.0 --dashboard-port=8265 \ + --dashboard-agent-listen-port 52366 --metrics-export-port 8091 + + # Wait for Ray's job agent to be ready to accept submissions. + until ray job submit --address="http://127.0.0.1:8265" --no-wait -- true >/dev/null 2>&1; do + echo "waiting for the Ray job agent..."; sleep 5 + done + ray job submit --address="http://127.0.0.1:8265" \ + --runtime-env-json='{"env_vars": {"PYTHONPATH": "/root/Megatron-LM/", "CUDA_DEVICE_MAX_CONNECTIONS": "1"}}' \ + -- python3 /root/slime/train.py \ + --actor-num-nodes 1 \ + --actor-num-gpus-per-node 2 \ + --rollout-external-engine-addrs ${ENGINE_ADDR} \ + --update-weight-mode full \ + --update-weight-transport disk \ + --update-weight-disk-dir /shared/policy \ + ${MODEL_ARGS[@]} \ + --hf-checkpoint /root/Qwen3-4B \ + --ref-load /root/Qwen3-4B_torch_dist \ + --load /root/Qwen3-4B_slime/ \ + --save /root/Qwen3-4B_slime/ \ + --save-interval 20 \ + --prompt-data /root/dapo-math-17k/dapo-math-17k.jsonl \ + --input-key prompt \ + --label-key label \ + --apply-chat-template \ + --rollout-shuffle \ + --rm-type deepscaler \ + --num-rollout 3000 \ + --rollout-batch-size 32 \ + --n-samples-per-prompt 8 \ + --rollout-max-response-len 8192 \ + --rollout-temperature 1 \ + --global-batch-size 256 \ + --balance-data \ + --advantage-estimator grpo \ + --use-kl-loss \ + --kl-loss-coef 0.00 \ + --kl-loss-type low_var_kl \ + --entropy-coef 0.00 \ + --eps-clip 0.2 \ + --eps-clip-high 0.28 \ + --optimizer adam \ + --lr 1e-6 \ + --lr-decay-style constant \ + --weight-decay 0.1 \ + --adam-beta1 0.9 \ + --adam-beta2 0.98 \ + --tensor-model-parallel-size 2 \ + --sequence-parallel \ + --pipeline-model-parallel-size 1 \ + --context-parallel-size 1 \ + --expert-model-parallel-size 1 \ + --expert-tensor-parallel-size 1 \ + --recompute-granularity full \ + --recompute-method uniform \ + --recompute-num-layers 1 \ + --use-dynamic-batch-size \ + --max-tokens-per-gpu 9216 \ + --rollout-num-gpus-per-engine 1 \ + --sglang-mem-fraction-static 0.7 \ + --attention-dropout 0.0 \ + --hidden-dropout 0.0 \ + --accumulate-allreduce-grads-in-fp32 \ + --attention-softmax-in-fp32 \ + --attention-backend flash +``` + +```bash +sky jobs launch -n slime-rl slime-jobgroup.yaml ``` -After each optimizer step the trainer publishes updated weights and the engines reload them — over the shared volume with `--update-weight-transport disk` (optionally `--update-weight-mode delta` to ship only changed bytes), or over NCCL ([Delta Weight Sync](../advanced/delta-weight-sync.md)). +To scale the inference fleet, add more engine jobs (`sglang-2`, `sglang-3`, ...) to the YAML and append their addresses to `--rollout-external-engine-addrs`. For large models, `--update-weight-mode delta` ships only the changed bytes ([Delta Weight Sync](../advanced/delta-weight-sync.md)); NCCL transport (`--update-weight-transport nccl`) avoids the shared volume entirely. ## End-to-End Example: Agentic Coding RL -A complete, runnable version of the disaggregated setup lives in the SkyPilot repository: +A complete agentic RL version of the disaggregated setup lives in the SkyPilot repository: **[slime on SkyPilot Job Groups](https://github.com/skypilot-org/skypilot/tree/master/llm/slime)** — trains a coding agent (Qwen3-14B) on SWE-smith with slime: a Megatron trainer job plus 1–3 SGLang engine jobs in one Job Group, agent rollouts executing untrusted code in sandboxed pods, and disk-based delta weight sync between the jobs. The example includes launch YAMLs, all setup/run scripts, and benchmark results for scaling the inference fleet (1 → 3 engines cuts async step time from about 1200 s to about 660 s on the example workload). From fa1614ebee54279c2f7848e5064881e51be89965 Mon Sep 17 00:00:00 2001 From: Ishan Date: Fri, 21 Aug 2026 11:59:54 -0700 Subject: [PATCH 3/3] docs(skypilot): fold long YAMLs into
, note one-command launch Review feedback: collapse the two full launch YAMLs behind
/ dropdowns so the page reads as prose with expandable configs, and tighten the quick-start cross-link wording. Co-Authored-By: Claude Fable 5 --- docs/en/get_started/quick_start.md | 2 +- docs/en/platform_support/skypilot_tutorial.md | 10 ++++++++++ 2 files changed, 11 insertions(+), 1 deletion(-) diff --git a/docs/en/get_started/quick_start.md b/docs/en/get_started/quick_start.md index 81786d0ed6..0eb848ac93 100644 --- a/docs/en/get_started/quick_start.md +++ b/docs/en/get_started/quick_start.md @@ -584,7 +584,7 @@ export NCCL_SOCKET_IFNAME=$(ip -o -4 addr show | awk '$4 ~ /^10\\./ {print $2}') export NVSHMEM_BOOTSTRAP_UID_SOCK_IFNAME=$(ip -o -4 addr show | awk '$4 ~ /^10\./ {print $2}') ``` -For launching the same multi-node setup on Kubernetes or cloud instances, see the [SkyPilot tutorial](../platform_support/skypilot_tutorial.md). +For launching the same multi-node setup on Kubernetes or cloud instances with one command, see the [SkyPilot tutorial](../platform_support/skypilot_tutorial.md). slime has been deeply optimized for distributed training of large-scale Mixture of Experts (MoE) models. We provide some end-to-end training cases for reference: diff --git a/docs/en/platform_support/skypilot_tutorial.md b/docs/en/platform_support/skypilot_tutorial.md index ab3c9dded4..746d88b666 100644 --- a/docs/en/platform_support/skypilot_tutorial.md +++ b/docs/en/platform_support/skypilot_tutorial.md @@ -24,6 +24,9 @@ The examples below use the `slimerl/slime:latest` Docker image from the Quick St The Quick Start starts a Ray cluster by running `ray start` on every node, then submits training with `ray job submit` from node 0. The following SkyPilot task performs the same steps: it provisions `num_nodes` nodes with GPUs, downloads and converts the model on each node, starts the Ray head and workers, and submits the job. Environment variables like `SKYPILOT_NODE_RANK` and `SKYPILOT_NODE_IPS` are injected by SkyPilot on every node. +
+slime-multinode.yaml + ```yaml # slime-multinode.yaml resources: @@ -139,6 +142,8 @@ run: | --attention-backend flash ``` +
+ The training arguments are the Quick Start's Qwen3-4B recipe; the only adjustments are the topology flags (`--actor-num-nodes`, `--actor-num-gpus-per-node`, and `--num-gpus-per-node`), whose values come from the SkyPilot-injected environment. `--num-gpus-per-node` matters on nodes with fewer than 8 GPUs: slime's colocated engine mapping assumes 8 per node unless told otherwise. Launch it with: ```bash @@ -169,6 +174,9 @@ sky volumes apply policy-volume.yaml Then launch the Job Group: +
+slime-jobgroup.yaml + ```yaml # slime-jobgroup.yaml --- @@ -294,6 +302,8 @@ run: | --attention-backend flash ``` +
+ ```bash sky jobs launch -n slime-rl slime-jobgroup.yaml ```