Skip to content

FIX: long-term reference list reordering in non-Baseline CABAC streams#3954

Open
niw wants to merge 1 commit into
cisco:masterfrom
niw:fix_long_term_references
Open

FIX: long-term reference list reordering in non-Baseline CABAC streams#3954
niw wants to merge 1 commit into
cisco:masterfrom
niw:fix_long_term_references

Conversation

@niw

@niw niw commented Jun 17, 2026

Copy link
Copy Markdown

Problem

When a slice carries ref_pic_list_reordering commands for long-term references (reordering_of_pic_nums_idc == 2), WelsReorderRefList2() matched the requested long_term_pic_num against each picture's uiLongTermPicNum field.

That field is unreliable because long-term references are created via MMCO 3 (short-to-long) and MMCO 6 (mark current as long), both of which only carry long_term_frame_idx in the bitstream and never a long_term_pic_num. As a result AddLongTermToList() stores uiLongTermPicNum as 0 for every such picture.

With multiple long-term references present, they all compare equal (0), so the reorder failed to select the requested entry (matching either the wrong one or none at all), leaving the wrong picture at the head of the reference list and corrupting inter prediction.

It only surfaces when multiple long-term references and a long-term reorder command occur together. In practice this is hit when decoding AirPlay Mirroring streams from iOS and macOS, whose High-profile CABAC streams use multiple long-term references with reordering, producing visibly corrupted video.

Note that WelsReorderRefList2() is only used for non-Baseline CABAC streams (uiProfileIdc != 66 && bEntropyCodingModeFlag).

The problem was introduced disabled (#if 0) by 1b8caef, and later enabled by fa6d099, which flipped it to #if 1 and exposed it.

Solution

Match by iLongTermFrameIdx instead of uiLongTermPicNum, in both the selection loop and the subsequent compaction loop.

iLongTermFrameIdx is the identifier that is actually maintained on the picture (set in AddLongTermToList) and, per the spec (8.2.4.1), for frame coding LongTermPicNum == LongTermFrameIdx, so the value parsed as long_term_pic_num matches it directly.

This aligns the reorder path with how the rest of the code already identifies long-term pictures, e.g. WelsDelLongFromList() used by MMCO 2, which likewise matches against iLongTermFrameIdx.

Testing

Added a focused unit test (DecUT_RefListReorder.cpp) that reproduces the case via three long-term references sharing uiLongTermPicNum == 0 with distinct iLongTermFrameIdx, plus an example High-profile CABAC regression stream (res/ltr_reorder_high_cabac.264) wired into the decoder API test.

Considerations

Keeping the match on uiLongTermPicNum but populating it correctly, i.e. deriving LongTermPicNum from LongTermFrameIdx per slice (8.2.4.1) or setting it at marking time, would be an alternative option. However, LongTermPicNum is a per-slice derived quantity, not a persistent property of the picture, so storing it is spec-awkward. For frame coding, it just equals LongTermFrameIdx, making the field redundant.
It would also touch the marking path (AddLongTermToList, MarkAsLongTerm) for no functional gain.

**Problem**

When a slice carries `ref_pic_list_reordering` commands for long-term
references (`reordering_of_pic_nums_idc == 2`), `WelsReorderRefList2()`
matched the requested `long_term_pic_num` against each picture's
`uiLongTermPicNum` field.

That field is unreliable because long-term references are created via
MMCO 3 (short-to-long) and MMCO 6 (mark current as long),
both of which only carry `long_term_frame_idx` in the bitstream and
never a `long_term_pic_num`. As a result `AddLongTermToList()` stores
`uiLongTermPicNum` as `0` for every such picture.

With multiple long-term references present, they all compare equal
(`0`), so the reorder failed to select the requested entry (matching
either the wrong one or none at all), leaving the wrong picture at the
head of the reference list and corrupting inter prediction.

It only surfaces when multiple long-term references and a long-term
reorder command occur together. In practice this is hit when decoding
AirPlay Mirroring streams from iOS and macOS, whose High-profile CABAC
streams use multiple long-term references with reordering, producing
visibly corrupted video.

Note that `WelsReorderRefList2()` is only used for non-Baseline CABAC
streams (`uiProfileIdc != 66 && bEntropyCodingModeFlag`).

The problem was introduced disabled (`#if 0`) by 1b8caef, and later
enabled by fa6d099, which flipped it to `#if 1` and exposed it.

**Solution**

Match by `iLongTermFrameIdx` instead of `uiLongTermPicNum`, in both the
selection loop and the subsequent compaction loop.

`iLongTermFrameIdx` is the identifier that is actually maintained
on the picture (set in `AddLongTermToList`) and, per the spec (8.2.4.1),
for frame coding `LongTermPicNum == LongTermFrameIdx`, so the value
parsed as `long_term_pic_num` matches it directly.

This aligns the reorder path with how the rest of the code already
identifies long-term pictures, e.g. `WelsDelLongFromList()`
used by MMCO 2, which likewise matches against `iLongTermFrameIdx`.

**Testing**

Added a focused unit test (`DecUT_RefListReorder.cpp`) that reproduces
the case via three long-term references sharing `uiLongTermPicNum == 0`
with distinct `iLongTermFrameIdx`, plus an example High-profile CABAC
regression stream (`res/ltr_reorder_high_cabac.264`) wired into the
decoder API test.

**Considerations**

Keeping the match on `uiLongTermPicNum` but populating it correctly,
i.e. deriving `LongTermPicNum` from `LongTermFrameIdx` per slice
(8.2.4.1) or setting it at marking time, would be an alternative option.
However, `LongTermPicNum` is a per-slice derived quantity,
not a persistent property of the picture, so storing it is spec-awkward.
For frame coding, it just equals `LongTermFrameIdx`, making the field
redundant.
It would also touch the marking path (`AddLongTermToList`,
`MarkAsLongTerm`) for no functional gain.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant