Repository navigation
pixelpipe: clearing the darkroom caches gives both pipes new keys, so they recompute - #1639
Merged
Merged
Conversation
… they recompute Run > Clear darkroom pipeline caches recomputed nothing. dt_dev_pixelpipe_reset_all() flushes the cache and rebuilds both darkroom pipes, but the flush spares every line somebody holds, the frames on screen first, so as not to free pixels under their holder. Both pipes resynced to the same keys, and the exact hit taken before a run found the frame on screen. Recomputing a held key would mean unpublishing its line while keeping it alive, and the cache cannot: the table holds one line per key, and raster masks, the raw detail mask and a run's final output are released by hash. dt_dev_pixelpipe_t gets a cache_generation instead, folded into the seed every key of the pipe derives from in _default_pipe_hash(), and dt_dev_pixelpipe_reset_all() bumps it on both darkroom pipes to the same value, so the full pipe still reuses what the preview computes. Generation 0 leaves every key as it was. The spared lines stay valid for their holders under their own keys: the screen keeps the old frame until the new one is published, and nothing looks them up again. Measured in the GUI under Xephyr on a NEF with its XMP, the menu item activated by an LD_PRELOAD shim, with -d dev -d perf -d pipecache. Before, both pipes ended on the exact hit in 7 ms and 1 ms without running a module. After, the preview reran its 15 modules from basebuffer and the full pipe ran gamma, while the old frame stayed on screen through 7 exposes. A click 300 ms into that recompute abandoned it in highlights and restarted it, and a third at rest freed what the first had spared (1922 → 864 MiB). No reference count went below zero. ctest passes 32/32 and the static gates pass. doc/pipeline-cache.md records the measurement and the mechanism in a new section. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Jiyone
force-pushed
the
clear-darkroom-caches
branch
from
November 14, 2026 11:59
38043a8 to
916d3bf
Compare
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Closes #1638.
Run > Clear darkroom pipeline cachesrecomputed nothing: it freed memory, and the darkroom kept the frames it already had.Cause.
dt_dev_pixelpipe_reset_all()flushes the cache, then rebuilds both darkroom pipes. Since73ec46a31bthe flush spares every line somebody holds, so that it never frees pixels under their holder.dt_dev_pixelpipe_cache_ref_host_entry_by_hash()inpixelpipe_hb.c) finds the frame on screen, and no module runs.Reproduce on master: start with
-d dev -d perf, open an image in the darkroom and clickRun > Clear darkroom pipeline caches. NoStarted … pipeline recomputeline follows, and noprocessed `…'line.Change. Recomputing a held key would mean unpublishing its line while keeping it alive, and the cache cannot. The table holds one line per key, and raster masks, the raw detail mask and a run's final output are released by hash, so a second line under the same key would take their release.
auto_destroydoes not help either:dt_dev_pixelpipe_cache_get()anddt_dev_pixelpipe_cache_get_writable()still serve a flagged line as an exact hit while it is held. So the keys move instead.dt_dev_pixelpipe_tgets acache_generation, folded into the seed every key of the pipe derives from (_default_pipe_hash()): the base buffer's, the global and mask hashes,upstream_hash. Generation 0 leaves every key as it was.dt_dev_pixelpipe_reset_all()bumps it before the rebuild, on both darkroom pipes and to the same value, so the full pipe still reuses what the preview computes.The lines the flush spared stay valid for their holders under their own keys: the screen keeps the old frame until the new one is published, and nothing looks them up again.
Tested in the GUI under Xephyr (1600×1000), on a NEF with its XMP, with
-d dev -d perf -d pipecache. AnLD_PRELOADshim activated the menu item (gtk_menu_item_activate()on the item found by its label).basebuffer, and the full pipe rangammaon the preview's new lines. The old frame stayed on screen through 7 exposes until the new one replaced it.highlights, the usual kill-switch abort, and restarted it. A third click, at rest, freed the lines the first had spared, released since (1922 → 864 MiB), and recomputed everything again.ctestpasses 32/32, and so do the static gates (layering, module boundaries, return types, list order, include guards and cycles, statelessness, unused and conditional includes). The touched files build without a warning. The build measured was6a9005ef9cplus this change, in a working tree that also carried unrelated uncommitted colour-picker work, which touches no cache, hash or pipe code.What it leaves.
The finding and the mechanism are in
doc/pipeline-cache.md, in the new section "Clearing the darkroom caches moves every key to a new generation".🤖 Generated with Claude Code