safari-hevc-qa

Reproduce real Safari HEVC decode on a GitHub-hosted macOS runner, using a prerecorded H.265 stream — so a bug that only shows in Safari's VideoToolbox decoder can be exercised from CI, without a Mac on anyone's desk and without the camera being reachable.

This is the Safari counterpart to OpenIPC/chrome-hevc-qa, which does the same for Chrome + VA-API in Docker on Linux. Chrome on Linux cannot use VideoToolbox, so it cannot answer "does this stream play in Safari?" — a macOS runner can.

Why it exists

OpenIPC/majestic-webui#335: on the Live page with Main + MSE, the H.265 video flashes every ~2 s (video → black → video) in Safari. Chrome plays the same camera cleanly, so the fault is in Safari's decode of the stream, and there is no Safari on Linux to test it. This harness closes that gap.

What's in the box

web/mse-hevc.html   the page: a <video> fed by MediaSource
web/player.js       replays the recording into a SourceBuffer, measures health
web/stream.bin      a recorded fMP4 HEVC stream (init segment + one fragment/frame)
web/stream.json     manifest: codec string, per-fragment offset/length/dts/arrival/key
run.py              drives real Safari via safaridriver, prints the verdict
.github/workflows/safari-hevc.yml   runs it on macos-14 and macos-15
run-share.py        opens a camera share link in Safari, reports whether the camera admits it
.github/workflows/safari-share.yml  runs it on macos-15 (the link is the SHARE_LINK secret)

The recording

web/stream.bin was captured from the /ws/video MSE endpoint of a lab camera — H.265 Main, hvc1.1.6.L153.B0, 2592×1520, 20 fps, all-keyframe GOP of 1 s, ~22 s. At the time the camera had no lens, so the picture is a flat sensor field and the recording is safe to publish. That camera has since been fitted with a lens, so a fresh recording from it is not: look at a snapshot before recording and publishing from any camera. It is the byte stream the WebUI would hand to SourceBuffer.appendBuffer, replayed in order and (by default) at the cadence the camera delivered it.

Each fragment is preceded by a 32-byte prft (producer reference time) box, as the camera's live stream now carries before every moof; the boxes were laid into the existing recording offline, byte for byte as the camera writes them, because they do not depend on the picture. MSE is required to accept and ignore a top-level box it does not know, and this recording is what checks that Safari's does.

web/stream-1080.bin is a second recording, the shape that majestic-webui#335 was reported on — hvc1.1.6.L123.B0, 1920×1080, 30 fps, one keyframe per ~30 frames with P-frames between, ~16 s — captured from a lab gk7205v300 + imx335 whose picture is an out-of-focus featureless field. Select it with ?stream=stream-1080. The all-keyframe recording above decodes one independent frame at a time and so cannot exercise a fault that only shows inside a GOP; this one can, which is what #335 and its cold-start residual (majestic-webui#460) turned out to be.

What it found — Safari's hardware HEVC/MSE decode is cadence-fragile, and the safe grouping of fragments into appendBuffer calls is decoder-specific. On the Apple-silicon runners here, replaying stream-1080:

append grouping (params)	macos-14	macos-15
one fragment each (chunk=1)	clean	stalls ~1 s
five at a time (chunk=5)	clean	clean
a whole GOP (gop=1)	fails	fails

The WebUI coalesces five (APPEND_BATCH=5), which is the value that clears both runners here; but on an Intel Mac's Safari the same five-fragment grouping is what faults (MEDIA_ERR_DECODE, #335's residual), and per-frame is the one that clears it. There is no single grouping that is safe on every decoder, so this harness guards the ends — a regression toward a whole-GOP append fails on every runner, and per-frame catches the macos-15 sensitivity — rather than proving one universal value. Reproducing the Intel-only residual needs an Intel Mac, which these hosted runners are not.

To re-record or record a different configuration, point tools/record.py (the capture script) at any majestic camera's /ws/video?stream=0.

The timed-metadata track probe

A second question this harness answers, on the same runners and with the same page: does MediaSource tolerate an init segment that declares a meta handler track it cannot decode?

majestic wants to write detection boxes into its recordings as an ISO/IEC 14496-12 timed metadata track, and the WebUI's recordings player appends those very fragments to a MediaSource. "Browsers ignore tracks they do not support" is the assumption that whole design rests on, and it is exactly the kind of assumption that holds in one engine and not the next — which is why this repository exists at all.

tools/inject-meta-track.py adds the track to an existing recording offline, byte for byte as the camera would write it, the same way the prft boxes in web/stream.bin were laid in: the bytes do not depend on the picture, so no new footage has to be recorded or published to ask the question.

python3 tools/inject-meta-track.py web/stream.bin web/stream-meta.bin
# then open mse-hevc.html?stream=stream-meta

It writes a meta/nmhd track whose sample entry is mett with mime_format: application/json, a matching trex, and one traf per moof carrying a single sample appended to that fragment's existing mdat. Track order is video first, metadata last, in both the moov and every moof.

Every run is a pair. The control is the same recording without the track, replayed through the same page on the same runner, and the test only has to match its own control. Without that, the runner-specific HEVC quirks documented above would read as a metadata-track failure and the fix would be applied to the wrong thing. A control that does not play cleanly makes the run inconclusive, not negative.

web/h264.bin is a synthetic ffmpeg testsrc clip — no camera footage in it at all — carried so the container question can be asked without the codec question riding along. It decodes in software in every browser.

What it caught, on its first run. The injected track carried a real duration in its tkhd and mdhd. Both are wrong in a fragmented file — every other track in these recordings says 0, because the duration is not known until the fragments are — and the first is wrong twice over: tkhd.duration is in the movie timescale (mvhd, 1000 in both these recordings) while mdhd.duration is in the media timescale (10240 and 1000000). Media ticks in the tkhd declared a 61-second metadata track on a 6-second movie.

Chrome played it anyway, decoding every frame with no error. Safari on macos-14 refused the whole stream with MEDIA_ERR_DECODE at t=0, and Safari was right. That is the entire argument for this repository existing, and for asking the question before the muxer was written rather than after.

Measured after the fix. Chrome 137 on Linux, control and test through the same page: forty fragments appended, two seconds buffered, 40 frames decoded and 0 dropped in both — identical in every field.

And on the macOS runners:

runner	h264 (ffmpeg)	h265 (ffmpeg)	stream (majestic H.265)
macos-14	pass	pass	pass
macos-15	fail	pass	pass

macos-15 is Safari 26.6.1, and the failure is MEDIA_ERR_DECODE at t=0 with nothing played of a six-second stream that plays clean without the track.

h265 is why that table can be read at all. The first run had only h264 and stream, which differ in codec and in provenance at once, so "Safari dislikes this on H.264" and "Safari dislikes this in an ffmpeg-written file" both fitted. h265 comes from the same ffmpeg invocation as h264 with only -c:v libx265 changed, holding provenance fixed — and it passes. It is the codec. Safari 26.6 refuses a timed-metadata track alongside H.264 and accepts the identical track alongside H.265.

That matters because majestic records H.264 by default on most cameras.

Taking the track back out

So the WebUI's recordings player removes it on the way in, and this harness checks that two ways.

tools/roundtrip.mjs asks the stronger question: strip(inject(recording)) must be the recording, byte for byte. Two implementations written independently in two languages have to agree on real files — 661 fragments across the three fixtures, all identical, and the init identical but for next_track_ID, which is left high on purpose (it only has to exceed every id in use, and majestic writes the same number whether or not the track is there).

?strip=1 runs the player's own remover before appending, so the workaround is exercised in the browser that refuses the file without it. web/mp4meta.js is a copy of majestic-webui's www/a/mp4meta.js; what runs here proves the idea, and the round-trip proves the copy still behaves like the original.

What it measures

The page plays the stream through MSE and watches, once per frame:

• video.error — Safari faulting the decode (code=3 is MEDIA_ERR_DECODE);
• black frames — the picture drawn to a canvas and its mean luma read, so a flash to black is counted even when no error fires;
• stalls, currentTime progress, and how much of the stream actually played.

On a decode error it rebuilds the MediaSource from the next keyframe, the way the WebUI's player does — which is what turns a periodic decode fault into the reported flash rather than a dead player. noreinit=1 disables that for a bare "does it decode at all" test.

window.__result carries the verdict; run.py reads it and exits 0 for clean playback, 1 if the fault reproduced (decode error, or ≥3 black events), 2 on a harness failure. The full per-frame luma trace is uploaded as result.json.

Run it

In CI: push, or use the Run workflow button (workflow_dispatch). The params input is appended to the page query string — e.g. burst=1 to append as fast as possible, noreinit=1 for a bare decode test, ms=30000 to run longer.

On a local Mac:

sudo safaridriver --enable
python3 -m pip install 'selenium>=4.20'
( cd web && python3 -m http.server 8000 & )
python3 run.py http://localhost:8000/mse-hevc.html

Plumbing check on Linux (Chrome + VA-API, not Safari): the page itself runs anywhere HEVC MSE is supported; on Linux you can confirm the replay works with OpenIPC/chrome-hevc-qa. Chrome playing it clean only proves the harness is sound — it says nothing about Safari, which is the whole point of the macOS job.

What it found (first runs, Safari 26.6)

Running the committed recording — H.265 Main, 2592×1520, ~2.2 Mbit/s, all-keyframe (GOP 1 s) — on GitHub's macOS runners:

runner	macOS / Safari	result
macos-15	15.7.9 / 26.6.1	HEVC MSE supported, but the SourceBuffer freezes after ~2.8 s with no video.error — currentTime stops, updateend stops — and rebuilding it (as the WebUI does) stalls again within a second. Six freeze/rebuild cycles in the run: the reported flash.
macos-14	14.8.9 / 26.6	HEVC MSE not supported at all: canPlayType empty, MediaSource.isTypeSupported false, addSourceBuffer throws NotSupportedError.

For contrast, Chrome + VA-API (via OpenIPC/chrome-hevc-qa) plays the same recording end to end — 421/421 fragments, 22 s, zero freezes, zero decode errors. So the fault is Safari's MSE HEVC path, not the camera's stream (which is conformant H.265 Main) and not the harness.

Root cause and fix (isolated with this harness)

The trigger is append frequency, not any stream property: the WebUI appended one fMP4 fragment per appendBuffer (~20–30/s, one per frame), and Safari's SourceBuffer wedges under that. Coalescing a handful of fragments into one append fixes it — on macos-15 the modes compare directly:

?params=	Safari macOS 15
chunk=1 (per-frame, the old behaviour)	stalls at ~2.8 s, 6 flash cycles
chunk=5	plays the full 22 s, 0 stalls
gop=1 (coalesce a whole GOP)	plays the full 22 s, 0 stalls

The WebUI fix is OpenIPC/majestic-webui#411: for HEVC, batch ~5 fragments per appendBuffer (H.264 left per-frame for its low-latency path). Use chunk= / gop= here to re-confirm or to size the batch for a new Safari.

Before/after with the real WebUI player

webui.html runs the actual preview.js (window.MajesticVideo) against a stubbed WebSocket replaying the recording, so the shipped code — its queue, its coalescing, its reconnect logic — is what Safari runs. Same harness, same stream, same macOS-15 Safari 26.6.1, only the player build differs:

?page=webui.html&player=	result
preview-unfixed.js (master, per-frame)	plays, then stalls at 3.6 s
preview.js (the #411 fix, default)	full 22 s, 0 rebuilds, clean

Because a reproduced fault exits non-zero, the macOS jobs are red while the bug is present and will go green if a future Safari plays the stream through — i.e. this doubles as a regression watch.

Reading the result

• reproduced: true with errorCodes: [3, …] — Safari's decoder rejected the stream; the camera's H.265 Main / MSE output is what #335 is about.
• reproduced: false, played through — this stream decodes cleanly on this Safari version. That narrows the trigger (a different resolution, frame rate, or a regression since the recording) rather than clearing it.

Live-page startup reproduction (#335 residual)

web/live/livepage.html runs the real Live page — p/player.cgi's markup and live.cgi's actual scripts (preview-page.js, the transport chain, the two-element swap, preview-zoom/adapt/stats/video-check/health) — over a stubbed camera (web/live/live-stub.js: main.js helpers, a captured config.json, empty sources, MSE transport, the recorded /ws/video replay), so preview-page.js runs its true startup on real Safari. run-live.py reloads it N times (the residual is intermittent) and counts a black frame after the first paint.

Result on Safari 26.6.1 (macos-15), 16 reloads: 16/16 painted (~350 ms), zero flickers. With the zoom-ab A/B (resize ruled out) and the sampler's own continuous drawImage (readback ruled out), this shows the residual flicker is not in any code path the recorded replay exercises. It correlates with the live stream's real-time timing and/or the reporter's device, which a deterministic recorded replay on a CI runner cannot reproduce.

The residual was a layout reflow, not decode (#335, iOS/4:3)

Driving the real Live page (above) with ?stream=stream-43&stage=390x664&resizeAt=1200 — a 4:3 clip in a portrait stage, the stage grown at ~1.25 s the way iOS Safari's URL-bar collapse grows the viewport — reflows the picture 885×664 → 1000×750 on 3/4 Safari loads, with zero black frames. The same with the 16:9-ish stream clip reflows 0/4 (width-driven, so a height change doesn't move it). So the residual "flicker" is a layout reflow, aspect-specific, and Live-only. Cause: #page-live { height: 100dvh } tracked the dynamic viewport; the fix (OpenIPC/majestic-webui#417) uses the large viewport so the URL-bar collapse doesn't grow the stage. The no-resize baseline is clean 4/4 — the fix's behaviour. (The iOS URL bar itself can't be driven on a desktop-Safari runner, so the mechanism is shown by inducing the stage grow the dynamic viewport would have caused.)