feat(platform): record, check and publish the compute-partition shape

[rpoornac]

This PR has been rewritten since its first review, so the diff no longer resembles what was reviewed. Same hardware and the same measured wins, but the mode is now a fixed session property established outside the optimizer rather than a lever searched inside it. No sudo, no amd-smi set, no mutation of any kind in the optimizer. The previous revision is preserved at ff451d840 if it is useful for comparison.

It has since been updated for the second review round; the finding-by-finding resolutions are in the comments below, and the description that follows reflects the current behaviour.

--max-latency-ms is no longer part of this PR. It was orthogonal — the constraint applies to any throughput-for-latency trade, not just partitioning — so it moved to #1297 to be reviewed on its own.

Why the design changed

The first revision put amd-smi set compute-partition inside the optimization loop. Three objections from review, all fair:

1. The repo's precedent for platform state is probe and refuse, not set. preflight.py reads NPS and cpufreq and warns; it does not change them.
2. A privileged, card-wide mutation that evicts every process holding a GPU context is not something to drive from a loop that also runs agent-authored code.
3. The mode cannot change mid-session anyway without invalidating every number measured before it, so searching it in-loop bought little.

What survives is the part that was actually load-bearing: validation. A launch-time refusal saves a three-hour session, and the fits_in_partition / read_hbm_gib arithmetic that produces it costs milliseconds.

What this PR does

Three things, all read-only:

Records the shape. The observed mode joins NPS in the platform fingerprint. Without it, the same configuration measured on the same card in SPX and in CPX is indistinguishable in the history — two different experiments filed under one name. The session report names it on partitioned runs, stating that the numbers are not comparable with a whole-card run. Where the framework's benchmark cannot place work per partition, the report says so outright: the figure is one device's, and which device is not knowable from here. Where it can, the report says the aggregate depends on a fan-out this process cannot verify.

Checks it at launch. Fail-closed, on the things it can be wrong about:

• --compute-partition-mode SPX|DPX|QPX|CPX asserts the mode the card is already in. If the card is in another mode, the session is refused. If the card cannot be read at all, that is also a refusal — the flag exists precisely to catch an external set that did not take, so an unverifiable assertion is not a satisfied one. The help text says "assertion, not request" in as many words.
• The per-stream HBM footprint is checked against one partition's memory, with --streams-per-partition (default 2) multiplied in. The footprint is the checkpoint's weight bytes, read byte-exact from the safetensors index. That is a lower bound and is used as one: each stream holds its own copy of the weights, so a "does not fit" verdict from it is a proof, while a "fits" verdict is no evidence — exactly the asymmetry a refusal needs, since it only ever acts on the former. When the checkpoint cannot be sized, the session runs and says so.
  There is a second source in the code, a measured peak_gib_per_stream, and it would be the tighter one — but nothing in this repository writes it, so every refusal today is made on the weights bound. The reader is kept so a harness that starts reporting it is honoured without a change here, and it is labelled as having no in-tree producer rather than presented as a fallback that gets exercised.
• That refusal applies only where streams will actually share a partition: a scriptable framework, or an operator who named the flags and has thereby asserted the shape. A serving session that merely starts on a card someone else left split records the mode and is refused nothing — without a fan-out nothing places a second stream, and nothing pins the benchmark to a partition at all, since whole cards enumerate first. Multiplying by streams there would be arithmetic about a shape the session was never going to run in.
• A --nodes >= 2 session records no shape. The card this process can read is not the card the benchmark runs on, so a declared mode there cannot be checked and is a usage error rather than a silently unchecked assertion.

Publishes it. The env block is split by reader. Mode, partition count and CU per partition describe the card and are published for any single-node session on a readable one, because platform_fingerprint() reads them back from there — it runs on the crash path, where spawning amd-smi is not acceptable. Streams per partition and total streams are instructions to a benchmark that places work on each partition, so they are published only when one will. The entrypoint gets the shape, not a device list, on purpose: HIP enumerates whole cards before partitions, so an index list computed at launch would be wrong in the one case that matters and wrong invisibly. The process holding the GPU context is the one positioned to check a device's CU count and refuse what does not match.

The consumer is out of tree

Worth stating outright, since the diff does not show it: nothing in this repository reads the variables this PR publishes. The program that places work on each partition is the benchmark entrypoint, and it lives outside the repo — assets/benchmark_scripts/ is not in the tree, as you noted. The env block is a contract offered to a consumer that lives elsewhere.

partition_device_predicate() is in the same position: defined, documented and tested, but within this PR called only by its own test. It is the reference implementation of the rule the external script is most likely to get wrong — select partition devices by matching CU count, never by index, because HIP enumerates whole cards first, so under DPX on one card of eight device 0 is a full 256-CU GPU while the partitions are devices 7 and 8. Keeping it here puts that rule next to the documentation describing it instead of leaving it to be re-derived downstream. Its first real caller now exists: the sweep driver in #1299 uses it to select partition devices, so the rule and its consumer land together across the two PRs.

Two limits that follow, both worth being explicit about:

• Until the benchmark entrypoint fans out, an optimize session on a split card measures one partition, not the aggregate. The recorded shape is still correct and still worth having — it is what stops a CPX number being filed as though it were SPX — but the throughput is not yet the aggregate figure the mode is chosen for. The report now states which of the two cases applies rather than leaving it to be inferred.
• Which is why there is no mode sweep in this PR. A sweep across SPX/DPX/QPX/CPX without a fan-out would report CPX as catastrophically slow: it would look like a partitioning comparison while actually measuring a fraction of the card. Feat/rpoornac/partition mode sweep #1299 is that sweep, stacked on this branch, and it carries its own fan-out — it places work on every partition a mode creates and sums the throughput, which is what makes the comparison mean anything. It is a scripts/ operator tool for the same reason the mutation left this PR: it sets the mode, and that belongs outside the optimizer.

Review findings from the first round

Finding	Resolution
sudo / mutation inside the optimizer	Removed entirely. set_partition_mode, partitioned(), the drain/retry loop, the restore path, and the sudo env are all deleted. The card must be in its mode before optimize starts: the shape is checked and recorded at launch, so a mode applied later — by the benchmark entrypoint, for instance — is too late to be either.
--max-latency-ms honoured only by ExploreExecutor while the help text claimed otherwise	Split out to #1297, where it is enforced at _lift_to_current_best so it covers every promotion path.
--framework short-circuit (if framework and not ... silently passes when None)	Replaced by _partition_fanout_supported(), which returns three explicit cases: supported, known-unsupported, and not-yet-resolved. All three are surfaced to the operator, and both call sites now pass a resolved framework.
CU table conflated mi308x with mi300x despite the docstring claiming ISA-not-board	CU per partition is now read from the device. The table is a fallback only, and PartitionLayout.probed records which source was used, so a derived count is never presented as a measurement — and is warned about, since partition devices are selected by matching that count exactly.
Inverted log severity on read_hbm_gib failure	Now WARNING, and says what the consequence is: feasibility will not be checked.
peak_gib_per_stream dropped after the first KEEP	Moot by construction. The footprint is resolved once at launch, not carried through promotions.
Duplicate RUNTIME_MODE_ENV / PARTITION_MODE_ENV	Collapsed to one set of PARTITION_* constants, now in common/gpu_partition.py where the fingerprint can reach them without importing an executor.
Triple amd-smi reads	One observe_partition() call per launch.
Orchestration prompt not told about the new lever	Dissolved: there is no lever, and no partition_* variants appear unprompted. Left the prompt untouched deliberately — platform topology (NPS included) is a recording concern in this repo and reaches no agent prompt.
Schema / capability / timeline churn	Reverted. The diff touches no schema.
Docs not synced: env vars undocumented, no CHANGELOG entry	docs/reference/environment-variables.md gains a "Compute partitioning (AMD)" section covering both the operator inputs and the published runtime hand-off the entrypoint reads; CHANGELOG.md gains an entry under [Unreleased]. The _SUDO variable it asked about no longer exists.

Bugs found while writing the tests

• _export_partition_shape never passed the model path to the footprint resolver, so the headline feasibility check could only ever warn — the refusal was unreachable. Now wired on both the fresh-launch and resume paths, with a test that asserts the resolver actually receives it.
• --streams-per-partition 0 was read as "not passed" (0 or DEFAULT is DEFAULT) and silently became 2, leaving the >= 1 guard unreachable for the value most likely to be a mistake. Same trap in the resume restore path. Both now test against None.
• The export ran unconditionally at the top of _run_optimize and again in the resume branch, so a resume validated twice — the first time against an unresolved model, where it could only warn. There is now exactly one call per path, placed after the framework, GPU type and post-quantization model path are all resolved.

Resume

The shape is part of the measurement contract, so it restores on the same path as the other operator-supplied values, and is then re-checked against the live card rather than trusted. A card can be repartitioned while a session is stopped; resuming into a different topology would compare candidates measured under one shape against a baseline from another.

One case is a recording rather than a refusal: an archive with no recorded mode — the card was unreadable at the first launch — has no assertion to re-check, so a resume onto a partitioned card observes the new shape and files it rather than stopping. The fingerprint then differs between the two halves of the session, which is the honest outcome, but nothing refuses.

Tests

140 new tests in two new files (test_gpu_partition.py, 50; test_partition_shape.py, 90), plus two in test_cli_bootstrap.py. They cover the amd-smi payload shapes actually in the wild, the MI355X case that motivated this (20.7 GiB × 2 streams does not fit a 36 GiB CPX partition), the refusal semantics, the fan-out gate, the multi-node behaviour, resume restore, the CLI exit codes, and the report section's provenance line — which had no coverage at all before, and is where the second round's false "derived from the board table" bug was hiding.

Nothing here needs privilege, and a host without amd-smi behaves exactly as before: unreadable card plus no declaration is the ordinary case and is not an error.

[github-actions]

CI E2E report — ❌ Timeout

item	value
result	❌ Timeout
model	Qwen/Qwen3-0.6B (dense)
resources	1× GPU, TP=1
PR branch	feat/rpoornac/compute-partition-lever
commit	3044f8168537e761480e13291dfdfbe4b46efadc
session_id	3f41b17b-83e7-4d82-be50-df86e4d9d7d2
queue → dispatch	24m 1s
run time	114m 18s
total	138m 19s
reason	Timed out — the run never reached a terminal state in time (task stuck, or the GPU stayed queued too long).
detail	not terminal after 13200s

details

[ZhengGong-amd]

The measurement work is real and the docstrings carry hard-won operational
knowledge. My concern is not how the lever is built but whether the optimizer
should own the mutation — and I think the repo already answered that.

The repo has a policy for this class of knob, and it is the opposite

Compute partitioning is a host-global, privileged, throughput-affecting knob.
There are already two of those — the cpufreq governor and NPS (memory
partitioning) — and both are probed, recorded in platform_fingerprint(),
warned about, and never set. preflight.py:1399: "Deliberately sysfs-only and
WARN-only."

scripts/platform_audit.py states both halves of the rationale:

host tuning ... cancels out of the delta. It does not cancel out of what the
session exports. ... The cost ... is paid by recording the setting: a
system-to-system delta stays explicable because the report says which mode each
run was in.

[Higher-risk knobs are] a different risk class ... so it lives in a separate
tool and its own review rather than being smuggled in behind a --bmc-host flag.

That is why platform_audit_bmc.py is a separate tool. HYPERLOOM_PARTITION_SUDO=1
plus a NOPASSWD sudoers entry for amd-smi is the same risk class behind the same
kind of flag. For what it's worth, sudo and amd-smi set appear nowhere else in
the tree; every other rocm-smi/amd-smi call site is read-only, and
actions/recover.md says --gpureset is "tenant-affecting, never issued."

Suggestion

• Keep the observation. platform_fingerprint() records NPS but has no field
  for the partition mode, so an SPX run and a CPX run are indistinguishable in
  final.json and in anything filed to the recipe KB. That gap is real today and
  closes in ~30 lines of read-only probe.
• Make the mode a session-level launch shape, singular, fixed for the session,
  alongside --tp / --gpu-type. "Which mode wins" is then a comparison across
  sessions — exactly how this PR's own Measured table was produced.
• Move the mutation to the boundary (a scripts/ tool, or the operator before
  launch).

Why this is better and not merely smaller:

• optimization_stack is append-only (writeback.py:2726), and partition variants
  are prepended — so the topology is frozen before batch size, compile and
  attention backend are tuned, which are the knobs it interacts with most.
• Fixed per session, the mode cancels out of the delta. Searched, the reported gain
  mixes "better flags" with "different hardware shape."
• Restore-failure leaving a shared card split, _refuse_split_card_for_unpartitioned_run
  (which exists to catch that), the drain/retry loop, set-then-read-back, mid-session
  tenant eviction, and the resume restore/apply/persist chain exist only because the
  mutation happens mid-loop.

Note also that assets/benchmark_scripts/ does not exist in-tree — the per-partition
fan-out has to live in the external scripts regardless, so having one of them call
amd-smi set is not extra burden.

Please split --max-latency-ms

It is a general KEEP constraint, coupled to partitioning only by motivation. Reviewed
alone the gap is immediate: latency_keep_block is called once, in ExploreExecutor.
_lift_current_best is the shared promotion path for kernel/framework/specialist/
integrate winners and never consults it, and the stack-rebench round KEEPs on
recomputed throughput without re-gating. The help says "Refuse any candidate."

Bugs worth fixing either way

• --framework unset skips the launch check entirely. It defaults to None and
  is not resolved before _export_partition_lever, so if framework and not is_scriptable(framework) short-circuits on "". The session runs sglang, the grid
  logs one warning and returns [], and both the report section and the capability row
  omit themselves because the framework isn't scriptable — the operator sees nothing.
  This is the case the PR description says is refused at launch. is_scriptable
  already resolves "" to the default, so dropping framework and fixes it; the
  existing test passes "sglang" explicitly.
• peak_gib_per_stream is dropped after the first KEEP — set on the baseline branch
  (writeback.py:3058), but _lift_current_best (:2743) rebuilds current_best
  without it, so pruning silently degrades to the weight-bytes bound from round two on.
• read_hbm_gib failures are log.debug, so a skipped feasibility check is quieter
  than an actual drop (log.warning) — inverted relative to consequence.
• layout_for treats AMD_GPU_DISPATCH_IDENTITIES as a per-board CU table, but that
  table's docstring says it identifies the ISA, not the board, and gives mi308x the
  mi300x count. partition_device_predicate matches CU counts exactly, so a wrong entry
  surfaces as "the mode did not take effect."

Smaller

• _export_partition_lever runs three amd-smi partition -a subprocesses for the same
  data (supported_modes → unsupported_modes → partition_count_conflicts each
  re-read); asked == [3, 3, 3] in the test locks it in.
• RUNTIME_MODE_ENV and PARTITION_MODE_ENV are one string under two exported names;
  PartitionProfile.memory_modes is never read; three readers of the same env disagree
  on precedence while read_session_lever's docstring claims to be the only one.
• Not synced: HYPERLOOM_PARTITION_SUDO/_GPU missing from
  docs/reference/environment-variables.md, no CHANGELOG entry, and the orchestration
  prompt's session context mentions neither the new hard KEEP constraint nor the
  partition_* variants appearing in a grid the model did not propose.

[rpoornac]

Thanks for this review — it was detailed and nearly every finding held up when I
checked it. I've rewritten the branch rather than patched it, because the central
objection was right and it changes the architecture.

On the sudo objection. I checked the precedent you pointed at.
preflight.py reads NPS and cpufreq and warns; it does not set them. So the
repo's convention for platform state is probe-and-refuse, and this branch broke
it. Worse, the mutation is card-wide: it evicts every process holding a GPU
context and renumbers devices, which is not something to drive from a loop that
also runs agent-authored code. And since the mode cannot change mid-session
without invalidating every number measured before it, searching it in-loop
bought very little.

All of it is gone: set_partition_mode, the partitioned() context manager, the
drain/retry loop, the restore path, and the sudo env. The amd-smi set moves to
the benchmark entrypoint, which already has to fan work out across partitions and
has a legitimate reason to be there.

What I kept. The validation, which is where the value actually was. A
launch-time refusal saves a three-hour session and the arithmetic costs
milliseconds, so fits_in_partition and read_hbm_gib are still doing work —
just once at launch instead of per candidate. The mode becomes an assertion
(--compute-partition-mode refuses if the card is in a different mode, or if the
card cannot be read, since an unverifiable claim is not a satisfied one) rather
than a request.

The rest of the findings. The --framework short-circuit is fixed properly:
the truthiness guard is replaced by a function returning three explicit cases,
because the interesting one was the third (--framework defaults to None, so
the guard read as "checked and fine" while meaning "not checked"). CU per
partition is now read from the device, with the board table demoted to a fallback
that is recorded and warned about — you were right that the docstring's
ISA-not-board claim did not survive mi308x. The read_hbm_gib severity is
WARNING and now names the consequence. The duplicate env constants are
collapsed. The triple amd-smi reads are one call. The schema/capability/timeline
churn is reverted.

Two findings dissolved rather than got fixed: peak_gib_per_stream dropping
after the first KEEP no longer matters, since the footprint is resolved once at
launch; and the orchestration-prompt gap for partition_* variants is moot
because there is no lever and no such variants. I deliberately left the prompt
untouched — platform topology, NPS included, is a recording concern in this repo
and reaches no agent prompt, so adding a rule about a field the agent cannot see
would be noise.

Your --max-latency-ms finding was correct and I've split it into #1297.
The flag existed here but only ExploreExecutor honoured it while the help text
overclaimed. It is orthogonal to partitioning — the constraint applies to any
throughput-for-latency trade — so it is now enforced at _lift_to_current_best,
the single choke point that writes current_best, and it covers all seven
promotion paths.

Two bugs I found while rewriting, worth flagging since neither was in the review:
the feasibility check never received the model path, so the refusal this branch
was built around was unreachable in practice; and --streams-per-partition 0 was
silently read as "omitted" and became the default of 2.

I've force-pushed the rewrite onto this branch rather than opening a new PR, so
the diff here is entirely different from what you reviewed — the previous
revision is at ff451d840, linked from the force-push entry in the timeline
above. The description is updated to match. The latency half is #1297.

[rpoornac]

/retest

[ZhengGong-amd]

The read-only direction is right, and the objections from the first round are
genuinely closed: no sudo, no amd-smi set, no in-loop mutation, no greedy
topology search. What follows is what still needs fixing.

Must fix

1. A serving session is refused on an already-partitioned card, with no flags passed.

_export_partition_shape (cli/__init__.py:1903) is called unconditionally,
validate_session_shape is not framework-aware, and streams defaults to 2.
So a plain sglang run on a card that happens to be in CPX exits 2 on a
fan-out assumption that path never uses. Reproduced:

--- SERVING session, NO partition flags, card already in CPX ---
>>> SystemExit code = 2
ERROR: this workload does not fit GPU 0's CPX partitions: 2 x 20.7 GiB = 41.4 GiB
needed per partition, 36.0 GiB available (8 x 32 CU).

The _partition_fanout_supported warning does not even print, since it is gated
behind if mode or streams_named (:1764). Apply the multi-stream fit check
only where fan-out is supported, or only when the operator named the flags.
Unpartitioned cards are unaffected — validate_session_shape returns early on
not layout.partitioned.

2. On a fresh launch the report says the CU count was "derived from the board table" when it was probed from the device.

:1903 discards the return value, so bootstrap.py:361 re-derives the shape
from env via published_shape(), which is a lossy subset of
session_shape_summary(). Reproduced:

layout.probed = True
full summary (RESUME):   {..., 'gib_per_partition': 36.0, 'cu_probed': True}
seeded from env (FRESH): {'mode':'CPX','partitions':8,'cu_per_partition':32,'streams_per_partition':2}
lost on fresh launch: ['cu_probed', 'gib_per_partition']

Report output, same session, two paths:

• fresh: CU per partition : 32 (derived from the board table) ← false
• resume: CU per partition : 32 (from the device) + HBM per partition : 36.0 GiB

Since report.py:799 tests shape.get("cu_probed") for truthiness, an absent
key reads as "derived". Fix by capturing the return value at :1903 and passing
it to _seed_shared_state. Worth prioritising: a false provenance line is the
one failure this PR exists to prevent.

Should fix

3. peak_gib_per_stream has no producer. Searching outside the new module
and its tests returns nothing; the files that wrote it in the previous revision
(benchmark_result.py, writeback.py) are not in this diff. The "measured"
branch in per_stream_footprint_gib is unreachable in production, so the check
always falls back to the weights bound. That bound is sound, but the docstring's
"two sources, tightest first" and the PR description's "comes from a prior
measured peak when one exists" are not true today. Either wire the producer or
say the weights bound is the only source.

4. Validation runs before quantization and before GPU-type resolution.
_run_quantization_prelude is at :2332 and args.gpu_type = gpu_type at
:2386, both after :1903. So a --quantize session is sized against the
source checkpoint, and when the CU probe fails gpu_type is still None, so the
board-table fallback raises, observe_partition swallows it, and a partitioned
card's shape is silently dropped. Move the fresh-launch call to after GPU/model
resolution and before _seed_shared_state (:2479).

(For the record: MODEL_PATH is not affected by ordering — it is not in the
dotenv allowlist, so it can only arrive via the shell or --model, both
available at :1903.)

5. "Let the benchmark entrypoint set it" is not achievable. Both
gpu_partition.py:9-10 and the --compute-partition-mode help say the
entrypoint may establish the mode, but validation and publication happen at
launch while the scriptable entrypoint starts at bypass_scriptable.py:197.
Passing the assertion exits before the script can run; omitting it records the
pre-entrypoint mode. Please require the mode to be set before optimize starts,
and drop the entrypoint option from the docs.

Minor

6. Multi-node silently records the controller's shape. The --nodes warning
is gated behind if (mode or streams_named) and nodes >= 2 (:1753), but
validate_session_shape runs regardless. On a controller that has GPUs, a
no-flag multi-node session publishes that machine's topology as the session's.
Either warn unconditionally when nodes >= 2, or skip observation entirely
there.

7. compute_partition is not in CORE_STATE_FIELDS. _validate_state_transition
(gate.py:1085) is a denylist, so an update_state can rewrite the recorded
topology and the report will print it. model_path, model_name and
degraded_mode are locked for the same reason.

8. Dead API. expected_mode() and streams_per_partition() have no
production callers, only tests. Also, runtime_env is written to os.environ
for serving frameworks too, which contradicts "only scriptable frameworks place
work per partition."

Not blocking

Resume can cross shapes when the operator passes a mode that differs from the
archive and the card is actually in it, keeping the old baseline and stack. The
more reachable variant is an archive with no recorded mode (card unreadable at
first launch) resuming onto a partitioned card with no assertion to check. Worth
a follow-up rather than a change here.

[rpoornac]

Both must-fix items were real and I reproduced both before touching anything.
All eight are addressed; the details where I did something other than what you
suggested are called out below.

Must fix

1. A serving session refused on an already-partitioned card. Reproduced
exactly as you have it. The refusal was arithmetic about a shape the session was
never going to run in.

My first fix was to run the check at one stream instead of two, and that is
wrong for a reason worth recording: without a fan-out, nothing pins the
benchmark to a partition at all. Whole cards enumerate before partitions, so
on a node where one card of eight is split, device 0 is a whole 256-CU card
with all of its HBM — this PR's own central claim. A single-stream check would
have refused a session that was about to get the whole card. So the footprint
check is now skipped entirely when nothing will place streams on partitions,
and the mode is recorded regardless:

--- SERVING session, NO partition flags, card already in CPX ---
Compute partitioning : CPX (8 x 32 CU, 36 GiB)
>>> ran. recorded shape = {'mode': 'CPX', ..., 'fanout_expected': False}

It stays a refusal where the premise holds — a scriptable framework, or an
operator who named the flags and has thereby asserted the shape:

--- SERVING session, operator NAMED --streams-per-partition 2 ---   >>> exit 2
--- SCRIPTABLE session, no flags                                --- >>> exit 2

The gate lives in the CLI rather than in validate_session_shape, which takes
a fanout_expected flag instead of learning what a framework is.

2. False "derived from the board table" on a fresh launch. Fixed as you
suggested: _export_partition_shape's return value is captured and passed to
_seed_shared_state, which now takes it as an argument and falls back to
published_shape() only when a caller has no verdict.

I also fixed the rendering, because the seed is not the only reader of the lossy
subset: platform_probe.platform_fingerprint() reads published_shape() too,
by design — it runs on the crash path where spawning amd-smi is not
acceptable. So an absent cu_probed is a state that still exists and had to
stop meaning "board table". It now prints no provenance at all rather than a
claim:

verdict carried through (fresh launch now):   32 (from the device)
re-read from the env (provenance unknown):    32
genuinely table-derived:                      32 (derived from the board table)

There were no tests on _format_compute_partition_section at all, which is
where this hid. There are now.

Should fix

3. peak_gib_per_stream has no producer. Confirmed — the producers were in
benchmark_result.py and writeback.py in the previous revision and are not in
this diff. I took the second option and said so rather than wiring it: adding a
producer means re-opening two files this PR otherwise does not touch, and the
weights bound is sound on its own. The docstring now says the weights bound is
what every refusal is actually made on, and warns against reading the two
bullets as a fallback chain that gets exercised; the CHANGELOG and the env-var
doc no longer claim a measured peak. The reader is kept so a harness that starts
reporting the field is honoured without a change there, but it is labelled as
having no in-tree producer.

4. Validation ran before quantization and GPU-type resolution. Fixed by
moving the fresh-launch call to your suggested position — after args.gpu_type
is final and after _run_quantization_prelude rewrites args.model, still
before _seed_shared_state.

This also removed a second bug: the call was unconditional at the top of
_run_optimize, so a resume validated twice — once against an unresolved
model where it could only warn, then again properly at the resume branch's own
call. There is now exactly one call per path.

One consequence to flag: the refusal now lands after _preflight, so a
mode-mismatch typo costs preflight's runtime before it is caught. I judged
correct-checkpoint sizing worth more than the seconds, since the failure it
still prevents is a three-hour session, but say so if you would rather have a
cheap probe-only mode check kept early.

5. "Let the benchmark entrypoint set it" is not achievable. Agreed and
dropped, from gpu_partition.py, _partition_shape.py, the
--compute-partition-mode help and the CHANGELOG. All four now say the mode
must be set before optimize starts, and the help says why: the shape is
checked and recorded at launch, so a mode applied later is too late to be
either.

Minor

6. Multi-node recorded the controller's shape. Took the stronger of your two
options. --nodes >= 2 now warns unconditionally and records nothing — a shape
read from the wrong node is precisely the mislabelling this PR exists to
prevent, so publishing it "with a warning" was the wrong trade. Since nothing is
observed there, a declared mode cannot be checked, and by the same rule that
governs an unreadable card it is now a refusal rather than a silently unchecked
assertion.

7. compute_partition not in CORE_STATE_FIELDS. Added, next to
model_path for the reason you gave. Note for anyone repeating this: there is a
byte-identical copy in agents/robustness/role/envelope.py that
test_core_state_fields_synced_with_robustness_envelope enforces, so it takes
two edits.

8. Dead API. expected_mode() and streams_per_partition() are deleted,
and so are HYPERLOOM_EXPECTED_PARTITION_MODE and
HYPERLOOM_STREAMS_PER_PARTITION — those two functions were the only readers of
those two variables, so the exports were dead at both ends. Both were introduced
by this PR, so nothing outside it can be relying on them. They are gone from the
env-var doc too.

On runtime_env for serving frameworks: gating the whole block was my first fix
and it was wrong, because platform_fingerprint() reads the mode back out of
that env — suppressing it would have deleted the provenance record this PR
exists to add, for exactly the sessions finding 1 is about. The block is now
split by reader instead. Mode, count and CU describe the card and are always
published. Streams-per-partition and total-streams are instructions to a
benchmark that fans out, and are published only when one will:

serving session:    {'mode': 'CPX', 'partitions': 8, 'cu_per_partition': 32}
scriptable session: {..., 'streams_per_partition': 2, 'total_streams': 16}

The report says plainly which case it is, and no longer prints a
"16 concurrent streams total" line above a paragraph explaining that no work was
placed per partition.

Not blocking

Agreed on resume crossing shapes, and agreed it is a follow-up. The variant you
identify as more reachable — an archive with no recorded mode resuming onto a
partitioned card with no assertion to check — is the one I would fix first.

Tests

test_partition_shape.py goes from 69 to 90, adding the no-fan-out gate, the
publication split, the multi-node refusal, the CORE_STATE_FIELDS lock, and the
first coverage of the report section. Two more in test_cli_bootstrap.py cover
the seed taking the verdict and falling back without one. Full run: 12365
passed, with 18 failures that are identical on the base commit (missing LLM SDKs
in my sandbox, in test_critic_agent_backend / test_external_multi_node /
test_proposal_scorer / test_robustness_agent_e2e).

[rpoornac]

/retest

[ZhengGong-amd]

All eight findings from the last round are verified fixed, including the two I
reproduced. Structure is clean, the docstrings that were stale are now accurate,
and prose density (48% / 44%) is in line with platform_probe.py (44%) and
bypass_scriptable.py (43%). What follows is minor.

Worth fixing

1. streams or DEFAULT contradicts the comment one module over.
cli/__init__.py:1737 explains why falsiness is wrong here:

0 or DEFAULT is DEFAULT, which would quietly honour an invalid request as
the default instead of refusing it

but _partition_shape.py:208 is int(streams or DEFAULT_STREAMS_PER_PARTITION).
So validate_session_shape(streams=0) silently reports Streams per partition: 2
while the same value through the CLI exits 2. It is a public entry point; please
make it refuse, or at least not use the pattern it argues against.

2. partition_gpu_id() swallows a bad GPU id. except ValueError: return 0
(_partition_shape.py:92) means HYPERLOOM_PARTITION_GPU=abc silently reads
card 0 and files its topology as the session's — the exact mislabelling this
module exists to prevent. read_device_cu and read_device_gib already warn on
comparable failures; this should too.

Cleanup

3. UNPARTITIONED_MODE is dead. Defined at gpu_partition.py:80, exported at
:553, referenced nowhere in the tree including tests.

4. __all__ exports the internal call graph. Eight of nineteen entries have
no consumer outside gpu_partition.py: MODE_PARTITION_COUNTS,
UNPARTITIONED_MODE, layout_for, partition_device_predicate,
read_device_cu, read_device_gib, read_partition_mode,
read_partition_modes. Four of those are called only by observe_partition,
whose own docstring calls itself "the single entry point a caller needs" — so
they read as private. partition_device_predicate is fine to keep as the
reference implementation, but its docstring should say it has no in-tree caller,
or the next reader will go looking for one.

5. The unknown-capacity branch in fits_in_partition is unreachable.
validate_session_shape:273 already returns on gib_per_partition is None
before calling it. The two guards also disagree (is None vs falsy), so a
0.0 capacity would take opposite paths.

6. session_shape_summary(None, ...) is a dead branch with a second schema.
_export_partition_shape returns {} when verdict.layout is None and never
calls it; the branch omits cu_probed, gib_per_partition and
fanout_expected, so it is not the same shape as the live one.

[rpoornac]

All six are fixed. One of them turned up a seventh while I was in the file, noted
at the end.

Worth fixing

1. streams or DEFAULT. Fixed, and fixed the way the comment one module over
argues for. validate_session_shape now takes streams: int | None = None:
None means the caller named nothing and takes the default, and anything below
one is refused rather than replaced by it. So streams=0 and
--streams-per-partition 0 now reach the same verdict instead of one exiting 2
while the other reported Streams per partition: 2.

Two details worth stating. The refusal happens before the card is read, because
this is a usage error about the request rather than a fact about the hardware, so
it needs no probe to decide — there is a test that fails if observe_partition
is reached. And a non-numeric value refuses too instead of raising TypeError
out of a launch-time helper.

2. partition_gpu_id() swallowing a bad id. Now warns, naming the variable,
the value and the consequence:

HYPERLOOM_PARTITION_GPU='abc' is not a usable GPU id; reading compute
partitioning from GPU 0 instead, whose topology may not be this session's.

It still returns 0 rather than raising, since a malformed environment variable
should not take down a launch that would otherwise run, but it no longer does so
in silence. The negative case is covered by the same branch: -1 used to be
clamped to 0 just as quietly, and now warns as well.

Cleanup

3. UNPARTITIONED_MODE. Deleted, definition and export both, with a test
that asserts it stays gone rather than trusting the next reader to notice.

4. __all__. The four probe helpers behind observe_partition —
read_partition_mode, read_partition_modes, read_device_cu,
read_device_gib — are out, with a comment above __all__ saying why: they are
the call graph of the function that calls itself "the single entry point a caller
needs", and advertising the steps contradicts that. They stay importable, since
the tests exercise each amd-smi payload shape directly.

I kept MODE_PARTITION_COUNTS and layout_for, which is the one place I have
not followed the finding, because both do have a consumer — the sweep driver in
#1299 imports them by name, along with fits_in_partition, parse_mode and
partition_device_predicate. Removing them here to add them back one PR later
seemed worse than leaving them, but say the word and they come out; __all__
does not affect the explicit imports either way.

partition_device_predicate now opens with "No caller in this repository"
and says not to go looking for one, which was the substance of the request.

5. The unknown-capacity branch. The two guards now share one predicate,
PartitionLayout.capacity_known, so they cannot disagree. That was the real bug
under the finding: a 0.0 capacity passed the validator's is None test and
then hit fits_in_partition's falsiness test, so the arithmetic was skipped
while the warning explaining why was never printed — a session ran unchecked and
said nothing. Both sides now treat a non-positive capacity as unknown, and a
test pins it.

I did keep the guard inside fits_in_partition, with a docstring saying so.
Unreachable from the in-tree caller is accurate, but it is a public predicate,
and one that silently multiplies by None for a caller who has not pre-checked
is worse than one redundant test. The validator is the side that tests first
because it is the only side that can warn; a bool return cannot.

6. session_shape_summary(None, ...). Branch gone, parameter now required.
The docstring records what was wrong with it, since the shape of the mistake is
worth keeping: four keys against the live path's seven, with cu_probed absent
rather than false — and an absent provenance key reading as a positive claim is
exactly the bug you reproduced last round.

One more, same family

Chasing finding 6 I found the claim you retired last round still alive in two
places I had missed, both in cli/__init__.py: the shared_state argument
docstring on _export_partition_shape ("Tighter than the weight-bytes bound, so
preferred when present") and the resume call-site comment ("A resume can size the
workload against what the last session actually measured, which rules out a mode
the weights alone fit"). Neither is true while nothing writes
peak_gib_per_stream. Both now say a resume sizes against the same weight-bytes
bound as a fresh launch.

Tests and docs

test_gpu_partition.py goes 50 → 63 and test_partition_shape.py 90 → 105,
adding the streams refusal and its ordering, the GPU-id warning and its silence
on good input, the zero-capacity agreement between the two guards, the required
layout, the trimmed __all__, and the deleted constant. 199 pass across the
three partition files. Full run: 12529 passed, with the same 18 failures the base
commit has (missing LLM SDKs and no live cluster in my sandbox, in
test_critic_agent_backend / test_external_multi_node / test_proposal_scorer
/ test_robustness_agent_e2e).

Docs followed the two behaviour changes rather than only the code: the
--streams-per-partition help text and the environment-variables reference both
state that a value below one is refused, the HYPERLOOM_PARTITION_GPU row states
that a bad value falls back with a warning, and the CHANGELOG entry says the same
about 0.

This is 0986b1460, pushed as a separate commit on top of ff6ff66f0 rather
than a rewrite, so the round-3 delta is reviewable on its own.