fix: Use the image loader for dicom-seg to allow compressed images

[wayfarer3130]

Context

The previous segmentation loader claimed labelmap + RLE support but neither actually worked, and it loaded SEG by retrieving the entire DICOM object. This PR makes SEG load through the image loader (per-frame imageIds, so standard back ends' SEG frame retrieve is used and compressed frames are supported), and fixes/extends storage to write labelmap or bitmap with correct RLE.

Faster loading from back ends that support frame retrieve, and significantly smaller storage since RLE compresses segmentation images extremely well.

OHIF_REF: feat/load-seg-images (OHIF/Viewers#5806)

Changes & Results

SEG load via image loader (adapters)

• createFromDicomSegImageId / createLabelmapsFromSegImageIds build labelmaps from per-frame imageIds + a decode callback (defaults to the image loader), replacing the whole-object decode path. Legacy buffer path retained.
• decodeSegPixelDataFromFrameIds fetches/decodes frames with bounded concurrency (default 16, configurable via concurrency) so per-frame requests overlap; order preserved, fail-fast unchanged.
• New encodePixelData plus perFrameFunctionalGroups normalization; broad Segmentation_4X / generateToolState / generateSegmentation updates.

RLE encoder / decoder fix

• RLE now correctly supports 1 / 8 / 16-bit and emits real runs (previously a count of 1 per pixel), making RLE-stored segmentations small and correct.

Full-instance Part 10 prefetch (single uniform frame registry)

• prefetchPart10Instance registers a fetched Part 10 instance into the @cornerstonejs/metadata NATURALIZED / COMPRESSED_FRAME_DATA framework (parsed by the dcmjs async reader).
• WADO-RS loadImage now consults that registry first (wadors/loadImageFromRegistry.ts) and falls through to the normal /frames/N request when nothing is registered — so WADO-RS and WADO-URI share one registry and one decode path.

Metadata / transfer syntaxes

• TransferSyntaxes enum and isVideoTransferSyntax updates.

Stack-labelmap rendering fix

• viewportReferencesSegmentationImages guards syncStackLabelmapActors so a labelmap is only mounted on viewports that actually display its source images. It resolves the labelmap's referenced source imageIds from each labelmap image's referencedImageId in cache (not the labelmap's own imageIds), which fixes the previously failing stack-segmentation render unit tests while still excluding unrelated series that merely share a frame of reference.

Tests & docs

• New jest tests: encodePixelData, fillSegmentationFrames, perFrameFunctionalGroups, resolveFrameImageIds, segBufferMetadataRoundTrip; migration notes added.

Testing

• Load compressed and uncompressed multiframe SEGs via the image-loader path and confirm correct rendering.
• Store as labelmap and as bitmap, in RLE and uncompressed; round-trip and confirm pixel-accurate restore and reduced size for RLE.
• Render a SEG on a stack viewport (single + multiple segmentations, and with brush editing) — these unit tests pass.
• Confirm a labelmap is not mounted on an unrelated series that only shares a frame of reference.

[wayfarer3130]

Here are a list of things Claude considered must fixes...

• maxBytesPerChunk default lost → main-thread regression at labelmapImagesFromBuffer.ts:213. Previously defaulted to 199_000_000. Now destructured without a default, so chunkPixelData falls back to Number.POSITIVE_INFINITY and chunking never happens. Large SEGs will block the main thread / blow memory expectations on small devices. Restore the 199000000 default (or wire a sensible per-OS default).
  Not relevant any longer - compressed images and uncompressed images now get a single chunk, so unless you are segmenting images larger than sqrt(199_000_000), you won't get chunks that size.

• Breaking API change with no deprecation at generateToolState.ts:50. createFromDICOMSegBuffer's 2nd arg changed from arrayBuffer to segImageId. Same name, same position, completely different contract — downstream OHIF/integrators will silently pass an ArrayBuffer and get metadataProvider.get('instance', ). Either rename the function (createFromDICOMSegImage) or detect the legacy shape and warn. At minimum, call this out in the PR body and changelog
  Made notes that this has changed and renamed it

• getReferencedSourceImageSequenceItem regex is too narrow at generateSegmentation.ts:175. /[?&]frame=(\d+)/ won't match the wadors form (wadors:.../frames/N) or imageId:multiframe?N shapes used elsewhere in this repo. Multi-frame source references will silently lose ReferencedFrameNumber. Reuse an existing helper (e.g. whatever derives frame index from imageId in the loader) instead of a one-off regex.

Used the new metadata module to get this

• Uint16Array(uint8Array) value-copy vs. byte-reinterpret at generateSegmentation.ts:141. For 16-bit input where pixelData is a Uint8Array, new Uint16Array(pixelData) copies values (length stays the same), not reinterpret bytes. Likely dead path for SEG today (SEGs are 1-bit or 8-bit), but it's a latent landmine if a 16-bit bitmap SEG ever flows through.

This was correct as a value copy

[coderabbitai]

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

packages/adapters/src/adapters/Cornerstone3D/Segmentation/labelmapImagesFromBuffer.ts (1)

1013-1017: ⚠️ Potential issue | 🟡 Minor

Inconsistent ImageOrientationPatient validation between bitmap and labelmap paths creates potential for silent data corruption.

processChunk (line 837) throws an error when ImageOrientationPatient is missing, but processLabelmapChunk (line 1013) silently falls back to validOrientations[0] (the reference image's orientation). If a frame's actual orientation differs from this fallback, pixel data will be aligned incorrectly and written to wrong positions in the labelmap.

Although alignPixelDataWithSourceData validates the orientation after the fallback is applied, it checks against the reference series' 8 possible orientations. If the missing frame's true orientation coincidentally matches one of them (but isn't the correct one), the mismatch goes undetected.

For consistency and safety, apply the same fail-fast validation as the bitmap path:

🛠️ Suggested fix

         const ImageOrientationPatientI =
           sharedPlaneOrientation ||
           PerFrameFunctionalGroups?.PlaneOrientationSequence
-            ?.ImageOrientationPatient ||
-          validOrientations[0];
+            ?.ImageOrientationPatient;
+
+        if (!ImageOrientationPatientI) {
+          throw new Error(
+            `Labelmap SEG frame ${i + 1} is missing ImageOrientationPatient in per-frame and shared functional groups.`
+          );
+        }

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@packages/adapters/src/adapters/Cornerstone3D/Segmentation/labelmapImagesFromBuffer.ts`
around lines 1013 - 1017, The ImageOrientationPatientI variable assignment in
processLabelmapChunk silently falls back to validOrientations[0] when
ImageOrientationPatient is missing, while processChunk throws an error for the
same condition. This inconsistency can cause silent data corruption if the
frame's actual orientation differs from the fallback. Replace the fallback logic
for ImageOrientationPatientI with explicit error validation that matches the
approach used in processChunk, ensuring that a missing ImageOrientationPatient
throws an error instead of silently using the reference image's orientation.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In
`@packages/adapters/src/adapters/Cornerstone3D/Segmentation/labelmapImagesFromBuffer.ts`:
- Around line 1013-1017: The ImageOrientationPatientI variable assignment in
processLabelmapChunk silently falls back to validOrientations[0] when
ImageOrientationPatient is missing, while processChunk throws an error for the
same condition. This inconsistency can cause silent data corruption if the
frame's actual orientation differs from the fallback. Replace the fallback logic
for ImageOrientationPatientI with explicit error validation that matches the
approach used in processChunk, ensuring that a missing ImageOrientationPatient
throws an error instead of silently using the reference image's orientation.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: f3353ddd-f799-4262-bd94-320351a78895

📥 Commits

Reviewing files that changed from the base of the PR and between 59a6f07 and aeb1074.

⛔ Files ignored due to path filters (1)

• pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml

📒 Files selected for processing (10)

• packages/adapters/examples/segmentationVolume/utils.ts
• packages/adapters/package.json
• packages/adapters/src/adapters/Cornerstone3D/Segmentation/generateSegmentation.ts
• packages/adapters/src/adapters/Cornerstone3D/Segmentation/generateToolState.ts
• packages/adapters/src/adapters/Cornerstone3D/Segmentation/index.ts
• packages/adapters/src/adapters/Cornerstone3D/Segmentation/labelmapImagesFromBuffer.ts
• packages/ai/package.json
• packages/docs/docs/migration-guides/5x/1-migration-notes.md
• packages/metadata/package.json
• packages/utils/package.json

✅ Files skipped from review due to trivial changes (1)

• packages/docs/docs/migration-guides/5x/1-migration-notes.md

🚧 Files skipped from review as they are similar to previous changes (5)

• packages/adapters/package.json
• packages/ai/package.json
• packages/metadata/package.json
• packages/utils/package.json
• packages/adapters/src/adapters/Cornerstone3D/Segmentation/generateSegmentation.ts

[coderabbitai]

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

packages/dicomImageLoader/src/shared/decoders/decodeRLE.ts (1)

191-199: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Remove unused variable pixelsPerPlane.

The variable pixelsPerPlane is declared on line 192 but never used. The unpackOneBitPlanar function accesses imageFrame.rows and imageFrame.columns directly.

🧹 Proposed fix

   if (isOneBit) {
-    const pixelsPerPlane = imageFrame.rows * imageFrame.columns;
-
     imageFrame.pixelData = unpackOneBitPlanar(
       imageFrame.pixelData as Uint8Array,
       imageFrame,
       frameSize
     );
   }

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/dicomImageLoader/src/shared/decoders/decodeRLE.ts` around lines 191
- 199, Remove the unused variable `pixelsPerPlane` that is declared in the
isOneBit conditional block within the decodeRLE function. This variable is
assigned the value of imageFrame.rows multiplied by imageFrame.columns but is
never referenced afterwards. The unpackOneBitPlanar function already accesses
imageFrame.rows and imageFrame.columns directly, so the intermediate variable is
not needed. Simply delete the line declaring pixelsPerPlane.

🧹 Nitpick comments (1)

packages/adapters/test/encodePixelData.jest.js (1)

97-158: 💤 Low value

Consider adding a 16-bit bitmap frame test.

The getBitmapFramesFromDataset implementation (per context snippet 3) supports 16-bit layouts with Uint16Array, but there's no test coverage for that path. Adding a test would ensure the 16-bit branch works correctly when needed.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/adapters/test/encodePixelData.jest.js` around lines 97 - 158, The
test suite for getBitmapFramesFromDataset covers 1-bit and 8-bit layouts but
lacks coverage for 16-bit bitmap frames. Add a new test case similar to the
existing ones that validates getBitmapFramesFromDataset correctly processes a
16-bit layout with BitsAllocated set to 16 and PixelData as a Uint16Array. The
test should verify that the function properly extracts frames and returns the
correct bitsAllocated value of 16 while maintaining the expected frame data
structure.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@packages/dicomImageLoader/src/shared/decoders/decodeRLE.ts`:
- Around line 191-199: Remove the unused variable `pixelsPerPlane` that is
declared in the isOneBit conditional block within the decodeRLE function. This
variable is assigned the value of imageFrame.rows multiplied by
imageFrame.columns but is never referenced afterwards. The unpackOneBitPlanar
function already accesses imageFrame.rows and imageFrame.columns directly, so
the intermediate variable is not needed. Simply delete the line declaring
pixelsPerPlane.

---

Nitpick comments:
In `@packages/adapters/test/encodePixelData.jest.js`:
- Around line 97-158: The test suite for getBitmapFramesFromDataset covers 1-bit
and 8-bit layouts but lacks coverage for 16-bit bitmap frames. Add a new test
case similar to the existing ones that validates getBitmapFramesFromDataset
correctly processes a 16-bit layout with BitsAllocated set to 16 and PixelData
as a Uint16Array. The test should verify that the function properly extracts
frames and returns the correct bitsAllocated value of 16 while maintaining the
expected frame data structure.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: ad88ca0c-e6dd-43c7-8bb7-276a0840d0f4

📥 Commits

Reviewing files that changed from the base of the PR and between aeb1074 and 41beed3.

📒 Files selected for processing (8)

• packages/adapters/examples/segmentationVolume/utils.ts
• packages/adapters/src/adapters/Cornerstone/Segmentation_4X.js
• packages/adapters/src/adapters/Cornerstone3D/Segmentation/generateSegmentation.ts
• packages/adapters/src/adapters/Cornerstone3D/Segmentation/perFrameFunctionalGroups.js
• packages/adapters/src/adapters/Cornerstone3D/encodePixelData.ts
• packages/adapters/test/encodePixelData.jest.js
• packages/adapters/test/perFrameFunctionalGroups.jest.js
• packages/dicomImageLoader/src/shared/decoders/decodeRLE.ts

🚧 Files skipped from review as they are similar to previous changes (3)

• packages/adapters/src/adapters/Cornerstone3D/Segmentation/generateSegmentation.ts
• packages/adapters/src/adapters/Cornerstone3D/encodePixelData.ts
• packages/adapters/src/adapters/Cornerstone/Segmentation_4X.js

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/adapters/test/fillSegmentationFrames.jest.js`:
- Around line 39-42: The makeMetadata.get function currently returns
SOPInstanceUID data regardless of the _module parameter, allowing tests to pass
even if fillSegmentation queries the wrong module. Add a condition that
validates the _module parameter matches the expected module name (likely
'dicomweb' or similar based on context) before returning the instance data; only
return the object with SOPInstanceUID when both the imageId exists in
sopByImageId AND the _module matches the expected value, otherwise return an
empty object to keep the test meaningful and prevent false positives.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 234374a9-3379-4696-a4ff-f7c96c95205c

📥 Commits

Reviewing files that changed from the base of the PR and between 41beed3 and 0bcb165.

📒 Files selected for processing (7)

• packages/adapters/src/adapters/Cornerstone/Segmentation_4X.js
• packages/adapters/src/adapters/Cornerstone3D/Segmentation/generateSegmentation.ts
• packages/adapters/src/adapters/Cornerstone3D/encodePixelData.ts
• packages/adapters/test/fillSegmentationFrames.jest.js
• packages/metadata/src/enums/TransferSyntaxes.ts
• packages/metadata/src/enums/index.ts
• packages/metadata/src/utilities/isVideoTransferSyntax.ts

✅ Files skipped from review due to trivial changes (1)

• packages/metadata/src/enums/TransferSyntaxes.ts

🚧 Files skipped from review as they are similar to previous changes (3)

• packages/adapters/src/adapters/Cornerstone3D/Segmentation/generateSegmentation.ts
• packages/adapters/src/adapters/Cornerstone/Segmentation_4X.js
• packages/adapters/src/adapters/Cornerstone3D/encodePixelData.ts

[sedghi]

Looks great, but i would like to try it first with OHIF before merging

[wayfarer3130]

@sedghi - as requested, I added the previous API back in as a compatibility layer, and also updated the descriptions on the frameImageIds parameter.

[salimkanoun]

Hi there if you have a demo link with this change I would like to try it too. I have some images in which loading segmentation make OHIF hangs a lot, I think there are so refresh on display or calculation made that freeze the viewer each time you interact with the segmentation

[sedghi]

putting this as request changes until we figure out the regresssions