exp(weight-transfer): add sparse filesystem updates

[samsja]

Summary

Experimental sparse filesystem policy publication for RL runs where most serving-precision weights remain unchanged.

• captures rank-local BF16 changes after optimizer steps without gathering the full model
• publishes sorted global flat indices with absolute BF16 replacement values
• supports DTensor shard offsets and selects one owner for replicated placements
• reconstructs an HF-format CPU state on vLLM workers, then uses vLLM's normal layerwise reload path
• validates an explicit base_step -> target_step sparse chain
• integrates with the shared weight transport sender/receiver handshake

Configure it with:

[weight_broadcast]
type = "sparse_filesystem"

Scope

This is a proof of concept. It currently requires trainer state-dict names and shapes to already match HF checkpoint coordinates. It rejects LoRA, checkpoint resume, and Partial DTensors. It also requires the inference base checkpoint to match the trainer's initial sparse baseline.

The inference worker retains a full BF16 HF state cache in CPU memory. Sparse publication reduces filesystem/network bytes, but vLLM still performs normal layerwise processing across the logical model.

A remaining correctness hardening item is commit-aware baseline promotion: local writer baselines currently advance when rank artifacts are written, before global publication acknowledgement.

Results

A Qwen3-0.6B two-GPU PrimeRL/vLLM smoke run completed through startup and multiple chained sparse reloads.

Observed update artifacts:

Update	Changed BF16 values	Sparse artifact
v3	17,729,393	169.25 MiB
v4	15,076,716	143.95 MiB

The dense model weights were about 2,273.78 MiB, making these artifacts about 13.4x and 15.8x smaller. The sparse format is not compressed yet.

Validation

• uv run ruff check ...
• uv run ruff format --check ...
• uv run pytest -q tests/unit/train/test_sparse_update.py tests/unit/test_configs.py — 125 passed
• uv run pytest -q tests/unit/train -m 'not gpu' — 77 passed, 85 deselected
• four-GPU 2x2 [Replicate(), Shard(1)] DTensor reconstruction smoke test
• two-GPU Qwen3-0.6B PrimeRL/vLLM chained sparse reload smoke test