This document describes how the pipeline cache stores module outputs, how GUI code fetches those buffers without blocking the interface, and the asynchronous retry protocol used when a buffer is not available yet. It complements:
reorganisation.md, which gives the high-level cache taxonomy (database / image / mipmap / pipeline) and the global locking model;resizing-scaling.md, which describes the ROI passes (modify_roi_out()/modify_roi_in()) and the difference betweenpiece->buf_*andpiece->roi_*.
The code lives in src/caches/pixelpipe_cache.c (the cache itself — it was
src/develop/pixelpipe_cache.c until the caches module was extracted, c889e94dc6),
src/develop/dev_pixelpipe.c
(the GUI fetch wrapper and the cache-wait manager), src/develop/pixelpipe_hb.c (the recompute
that publishes image cachelines), src/develop/pixelpipe_raster_masks.c (raster-mask side-band
retrieval), and src/gui/color_picker_proxy.c (the module color-picker's own input_wait /
output_wait consumer).
Each enabled pipeline node (dt_dev_pixelpipe_iop_t, also called a piece) produces one output
buffer. That buffer is stored in the global pipeline cache, keyed by the piece's
dt_dev_pixelpipe_iop_t.global_hash. The hash is a checksum over everything the output depends on:
module parameters, blend/mask parameters, the input/output ROI, buffer descriptors, GUI states
(mask preview, cache bypass), and — for modules that opt in via runtime_data_hash() — the
committed runtime data blob piece->data. The exact chain is documented in reorganisation.md
(section Pipeline cache).
Two consequences matter for everything below:
- The hash is the contract. A consumer that knows a piece's
global_hashcan fetch its output from any thread. It does not need to know which pipeline run produced it, nor wait for a full pipeline to finish. - A piece's input is the previous enabled piece's output. To read the buffer feeding a
module, fetch the output of
dt_dev_pixelpipe_get_prev_enabled_piece(pipe, piece), not the module's own cacheline.
A piece keeps the metadata of the cacheline it wrote last (piece->cache_entry). When its hash
changes, dt_dev_pixelpipe_cache_get_writable() moves that same cacheline to the new hash instead
of allocating another one (rekey reuse): same arena slot, same OpenCL buffers, no allocation per
frame. Two consequences follow, each with its rule.
The host buffer outlives the pixels it held. An OpenCL output that is not read back to RAM
leaves the previous hash's pixels in the rekeyed cacheline's host buffer. The rekey marks it
(dt_pixel_cache_entry_t.host_stale), and until the host buffer is written for the new hash
dt_pixel_cache_entry_get_data() returns NULL, exactly as for a device-only cacheline. Readers
need nothing else: a CPU consumer copies the device payload back, a GUI reader reports a miss.
The producer clears the mark with dt_dev_pixelpipe_cache_flag_host_written() when it ran on the
CPU, tiled, or read its output back; every device-to-host copy clears it too. Only the producer
reaches the raw buffer, through dt_pixel_cache_entry_get_buffer(), since it still backs the
output's pinned OpenCL image.
Reusing in place forgets the previous state. That is right for a slider drag or a pan, whose
previous states do not come back, and wrong for a module toggle, an undo, a jump in history or a
keyboard edit (a slider step with the arrow keys), after which going back is the likely next step. Those raise DT_DEV_PIPE_SWITCHED, which sets
pipe->keep_outputs for the run that renders the new state: it writes every output into a new
cacheline, so the outputs of the state just left are still there, and switching back is a chain of
exact hits. The flag drops when that run completes. Only the source of a change knows it is a
switch; the pipe's change status does not (SYNCH also carries slider commits on masked modules).
A module may also publish raster masks for downstream modules or multi-page export. These masks
are not stored in dt_dev_pixelpipe_iop_t and are not embedded in the module's RGBA output
cacheline. They are independent single-channel float cachelines in the same global pipeline cache.
dt_dev_pixelpipe_raster_mask_hash(piece, mask_id) derives their key from:
- the provider's
piece->global_mask_hash, which already covers its upstream image state, blend parameters and ROI; - a raster-mask namespace tag, preventing aliasing with image outputs;
- the provider-local mask identifier.
CPU and OpenCL blend paths publish the final provider mask under this key. A consumer calls
dt_dev_get_raster_mask(), which retains and read-locks the canonical cacheline, copies it into a
caller-owned working buffer, then applies the distort_mask() callbacks of enabled modules between
the provider and the consumer. The canonical cached mask remains immutable.
The pipeline keeps references to the raster-mask hashes required by its current graph in
dt_dev_pixelpipe_t.raster_mask_hashes. On a new render it first retains the new set, then releases
the previous set. This ordering prevents an unchanged mask from becoming briefly evictable between
provider publication and downstream consumption or export. The references go with the graph:
dt_dev_pixelpipe_cleanup_nodes() releases them, and the detail mask's (rawdetail_mask_hash)
with them -- see "A pipe's side-band masks go with its nodes" below.
An image cache hit can therefore reuse its associated raster mask without recomputing the provider
or the modules before it. If the side-band mask was nevertheless evicted while the provider image
survived, an interactive consumer requests one bounded DT_DEV_PIPE_REENTRY pass. Immediately
before that retry, the pipe invalidates image cachelines from the provider through the end of the
graph, keeps upstream cachelines, and reruns without rebuilding synchronized nodes. A second miss
stops instead of scheduling a loop. Export enumerates the provider's declared mask IDs and fetches
the same dedicated cachelines; if one is unavailable, the format backend handles it as a missing
export mask instead of starting an interactive retry.
The cache has one short-lived manager mutex (held only while adding/removing/looking up cachelines) and one read/write lock per cacheline. Reads are concurrent; writes are exclusive and wait for all readers. A consumer that copies a cacheline must hold a reference and a read lock for the duration of the copy:
dt_dev_pixelpipe_cache_ref_count_entry(TRUE, entry); // pin: prevent eviction
dt_dev_pixelpipe_cache_rdlock_entry (TRUE, entry); // block writers while we read
... memcpy out of the cacheline ...
dt_dev_pixelpipe_cache_rdlock_entry (FALSE, entry);
dt_dev_pixelpipe_cache_ref_count_entry(FALSE, entry); // the entry may not be named after this
When a producer releases the write lock of a cacheline, dt_dev_pixelpipe_cache_wrlock_entry()
raises DT_SIGNAL_CACHELINE_READY with that cacheline's hash. This is the wake-up that the retry
protocol below is built on.
| Caller | Function | On a miss |
|---|---|---|
| Backend / pipeline | dt_dev_pixelpipe_cache_peek() |
returns FALSE, does nothing |
| GUI | dt_dev_pixelpipe_cache_peek_gui() |
queues a cache-wait and asks the pipe to publish the buffer |
dt_dev_pixelpipe_cache_peek() is a pure probe. It is correct for pipeline code, which can simply
recompute what it needs, and for opportunistic GUI reads that have a fallback. It is the wrong
tool for GUI code that requires a specific intermediate buffer, because intermediate cachelines
are evicted as soon as nothing references them: the probe can keep missing forever even though the
final image is on screen. That failure mode is what produced the ashift "Data pending – Please
repeat" bug (issue #710): structure detection probed ashift's input with a raw cache_peek(),
missed because the input cacheline was not retained, and never recovered.
dt_dev_pixelpipe_cache_peek_gui() is the race-free GUI counterpart and the subject of the rest of
this document.
Backend code that must keep a cacheline past the lookup uses
dt_dev_pixelpipe_cache_ref_entry_by_hash(), or dt_dev_pixelpipe_cache_ref_host_entry_by_hash()
when it needs host pixels and must not wait on a line still being written. Raster-mask retrieval,
the raw-detail mask, the drawn-mask group's cached-prefix resume (develop/masks/group.c) and the
publication of the backbuffer follow this contract: the lookup itself takes the reference, under the
cache mutex; it is then kept as the long-lived one, or released once the pixels are copied (under a
read lock) or once the long-lived reference is taken.
A peek retains nothing, so nothing may be released after one. An entry nobody holds sits at
refcount 0, and any thread's eviction can free it between dt_dev_pixelpipe_cache_peek() returning
and the caller's next step, read lock included. Releasing a reference that was never taken is worse
than a leak: it drives the count below the number of real holders, the LRU then sees a held entry as
free and frees it under them, and every long-lived reference is released by pointer
(dt_dev_pixelpipe_cache_unref_entry()), so the holder's own release writes into freed memory --
wherever the allocator has put something else by then. The crash surfaces in that other object, far
from the release that caused it.
Nor may one be referenced after one. The eviction can fall between the lookup and the
reference, which then counts up freed memory; a backbuffer keepalive taken that way is released by
pointer at the next publication. dt_dev_pixelpipe_cache_get_entry() is the same kind of lookup:
it serves only the producer-to-consumer handoff inside one run, where the reference is already
held. When dt_dev_pixelpipe_cache_get_writable() finds the hash already published, it returns
that entry referenced for the same reason.
Nor may one be named after its release. A release returns with the cache mutex released, and
from then on any thread's eviction can free an entry nobody holds. A holder that wants a line gone
flags it with dt_dev_pixelpipe_cache_flag_auto_destroy() while it still holds it, then releases
it: releasing the last reference of a flagged line removes it, within the same hold of the mutex.
That is how the intermediates of a pipe that keeps no cache go, as well as a module output that
failed, a side-band line a module created and could not fill, and a no-cache pipe's last frame at
cleanup. Others still holding the line keep it until the last of them releases it. A line still
locked when released stays for the LRU, so a producer flags before it releases its write lock and
drops its reference last. There is no removal by pointer: it could only succeed on a line nobody
holds, which is one its caller has no right to name.
A reserved reference is released on every way out. process_rec() returns its output with one
reference reserved for its receiver: the next module, or dt_dev_pixelpipe_process() for the final
output. The receiver releases it whether it goes on or not, so a module aborting before it
processes releases it, and so does a run shut down or failed on OpenCL after it completed. A
reference left behind pins its line, which can then never be evicted.
The GUI fetch can ask a pipe to (re)publish one specific buffer without rendering the whole image.
Two request kinds exist (dt_dev_pixelpipe_cache_request_t):
DT_DEV_PIXELPIPE_CACHE_REQUEST_BACKBUF— the pipe's final output;DT_DEV_PIXELPIPE_CACHE_REQUEST_MODULE— one named module's output in the middle of the graph.
dt_dev_pixelpipe_cache_peek_gui() sets the request with dt_dev_pixelpipe_set_cache_request() and
flags the pipe changed with dt_dev_pixelpipe_or_changed(pipe, DT_DEV_PIPE_CACHE_REQUEST). On its
next run, dt_dev_pixelpipe_process() reads the request, resolves the target piece with
_get_requested_piece_node(), and runs only up to that piece (requested_pos). The piece's
output is published under its global_hash, which releases its write lock and raises
DT_SIGNAL_CACHELINE_READY.
This is why fetching a module input passes the previous piece to peek_gui(): the request then
targets the previous module, and the partial recompute stops exactly where the wanted buffer is
produced.
The retry bookkeeping is centralised in a process-wide singleton, _cache_wait_manager
(src/develop/dev_pixelpipe.c). It owns a pending list of dt_dev_pixelpipe_cache_wait_record_t,
diagnostic counters (queued_requests, served_requests, cancelled_requests, immediate_hits,
misses), the global DT_SIGNAL_CACHELINE_READY subscription, and the darkroom busy-cursor state.
Each consumer owns a small, persistent handle, dt_dev_pixelpipe_cache_wait_t:
typedef struct dt_dev_pixelpipe_cache_wait_t
{
struct dt_dev_pixelpipe_t *pipe; // pipe that must publish the buffer
const struct dt_iop_module_t *module; // target piece's module (NULL for backbuf)
uint64_t hash; // the cacheline hash being awaited
dt_dev_pixelpipe_cache_ready_callback_t restart; // resume callback
gpointer user_data; // passed back to restart()
const char *owner_tag; // debug label
gpointer owner_object;
uint64_t request_id;
gboolean connected; // TRUE while queued
} dt_dev_pixelpipe_cache_wait_t;The handle is caller-owned and must outlive the request (store it in the consumer's GUI data, not on the stack). It is the identity the manager uses to deduplicate repeated requests from the same consumer.
- Unsupported target (realtime pipe,
no_cache, or the target piece is inbypass_cachemode) → returnFALSEimmediately, no wait. Note: bypass only blocks fetching a bypassed piece's own output; upstream targets (a module's input) stay fetchable. - Hit — the cacheline exists: cancel any stale wait for this handle
(
dt_dev_pixelpipe_cache_wait_cleanup()), hand back the buffer and entry, returnTRUE. - Miss — register/refresh the wait:
- if the handle already targets the same
(pipe, module, hash, restart), do not re-emit the cache request (request_cacheline = FALSE); the previous request is still in flight. This is what stops an unsatisfiable target from being re-requested on every expose; - otherwise clean up the old target, fill the handle, append a record to the pending list,
connect the manager to
DT_SIGNAL_CACHELINE_READY(once, lazily), and raise the busy cursor; - emit the cache request (section 3) and return
FALSE.
- if the handle already targets the same
_dt_dev_pixelpipe_cache_wait_ready_callback() runs on every DT_SIGNAL_CACHELINE_READY. Under the
manager lock it removes every pending record whose hash matches the published hash, disconnects
the signal and clears the busy cursor when the queue drains, then releases the lock before
invoking the restart callbacks. Running callbacks outside the lock is deliberate: a restart
handler typically queues a redraw or issues a brand-new cache request, so holding the lock would
serialise unrelated GUI wake-ups and risk re-entrant deadlock.
A served handle is reset to an inert state (connected = FALSE, fields cleared) before its
restart() runs. The restart should simply retry the original operation: call peek_gui() again
(now usually a hit) and proceed. If it misses again — the cacheline was evicted between the signal
and the GUI getting scheduled — the retry transparently re-registers the wait, so the protocol is
self-healing.
DT_SIGNAL_CACHELINE_READY and DT_SIGNAL_HISTORY_RESYNC are declared asynchronous in
src/control/signal.c, so even when raised from a pipeline worker thread their handlers are
marshalled to the GUI main thread via g_main_context_invoke(). Restart callbacks therefore run on
the GUI thread and may touch GTK widgets, set parameters, and flag pipes changed exactly like any
other GUI callback.
To read a module's input buffer from the GUI without blocking:
// 1. persistent handle in the consumer's GUI data
dt_dev_pixelpipe_cache_wait_t my_wait; // zero-initialised; never on the stack
// 2. fetch
const dt_dev_pixelpipe_iop_t *piece = dt_dev_distort_get_iop_pipe(dev->preview_pipe, module);
const dt_dev_pixelpipe_iop_t *prev = dt_dev_pixelpipe_get_prev_enabled_piece(dev->preview_pipe, piece);
void *buf = NULL;
dt_pixel_cache_entry_t *entry = NULL;
dt_dev_pixelpipe_cache_wait_set_owner(&my_wait, "my-consumer", module); // debug label
if(dt_dev_pixelpipe_cache_peek_gui(dev->preview_pipe, prev, &buf, &entry,
&my_wait, _my_restart_cb, user_data))
{
dt_dev_pixelpipe_cache_ref_count_entry(darktable.pixelpipe_cache, TRUE, entry);
dt_dev_pixelpipe_cache_rdlock_entry (darktable.pixelpipe_cache, TRUE, entry);
/* ... copy out of buf ... */
dt_dev_pixelpipe_cache_rdlock_entry (darktable.pixelpipe_cache, FALSE, entry);
dt_dev_pixelpipe_cache_ref_count_entry(darktable.pixelpipe_cache, FALSE, entry);
}
// else: a wait is queued; _my_restart_cb() will be called when the buffer is ready.
// 3. teardown / reset / image change
dt_dev_pixelpipe_cache_wait_cleanup(&my_wait, "my-consumer-cleanup");The restart callback retries the same operation:
static void _my_restart_cb(gpointer user_data) { /* re-run the fetch + the work it gated */ }Reference consumers:
- the histogram (
src/libs/histogram.c) keeps severaldt_dev_pixelpipe_cache_wait_thandles (scope_wait,picker_wait,module_wait) and is the canonical example; - the color picker state lives in
dt_develop_t.color_pickerwith its owninput_wait/output_wait.
This API fits consumers that sample a buffer which the pipe naturally retains (a module output that also serves as a histogram source or backbuffer). It is the wrong tool when the wanted buffer is a transient intermediate that only exists while a specific module runs — see ashift in section 6.
The perspective/horizon module (src/iop/ashift.c) exercises every sharp edge of this
infrastructure; its regressions are worth recording. Its needs differ from the histogram's: it wants
its own input while it is the focused, actively-edited module, and that buffer is a transient
intermediate. The lessons below explain why it ends up not using the cache-wait API.
- A module cache request never runs the requesting module's
process().peek_gui()resolves a module target to the previous enabled piece and the pipe runs only up to it (section 3). So a module that fills a GUI buffer inside its ownprocess()(ashift capturesg->bufthere) cannot obtain it throughpeek_gui(): the partial render stops one module short. Worse, the published intermediate is unreferenced and can be evicted before the GUI restart reads it; the restart then re-requests, the partial render republishes-and-evicts, and the preview pipe aborts at initialscale and restarts forever (an intermittent hang, depending on cache pressure). This is why rawcache_peek()/peek_gui()on ashift's input was removed. - Prefer the module's own
process()capture for its own input. During edit ashift is in cache-bypass mode, so itsprocess()runs on every preview render and copies its input intog->buffor free. The robust pattern is therefore: capture inprocess(); when a GUI action needs the buffer and it is not ready yet, queue the job and resume it fromDT_SIGNAL_DEVELOP_PREVIEW_PIPE_FINISHED(raised afterprocess()ran, unlikeDT_SIGNAL_HISTORY_RESYNCwhich fires before the render). No cache poking, no partial render, no spin. Reservepeek_gui()for buffers the pipe naturally retains (histogram/picker sources). - The
process()-time "is this the full preview image?" guard must compare dimensions the same way the worker computes them — and must be orientation-aware. ashift capturesg->buf(and refreshesg->isflipped) only whendt_dev_pixelpipe_has_preview_output(dev, pipe, roi_out)is TRUE; that guard compares the module'sroi_outtodev->roi.preview_{width,height}. Two independent defects each broke it on a subset of images — which is why "#710 acting crazy" reproduced on some pictures but not others, and why resetting the module to neutral did not help:- Round vs. truncate (primary, parameter-independent).
dt_dev_get_thumbnail_size()setpreview_width = (int)(natural_scale*processed_width)— a C truncation — while_update_darkroom_roi(), which determines the ROI the worker actually requests (hence the backbuffer size, hence ashift's back-propagatedroi_out), usesroundf(). They disagree by 1px whenever the fractional part is ≥ 0.5. Whether a given image lands there is luck of its dimensions and fit-scale, and is independent of the module parameters, so a reset cannot fix it. Fix: roundpreview_{width,height}indt_dev_get_thumbnail_size()too, so both sides agree. The match is then exact — ashift'sroi_outis back-propagated from the requested ROI through theflipswap, not taken from its own preview-scalemodify_roi_out()floorf, so both sides ultimately derive fromprocessed_{width,height}. - Portrait swap. ashift runs before the
flip(orientation) module, so on a portrait image its ownroi_outis landscape whiledev->roi.preview_{width,height}is the post-flip portrait size. Fix:has_preview_output()also accepts the swapped dims (width==preview_height && height==preview_width), which un-breaks every pre-flip consumer (demosaic, highlights, denoise, atrous, nlmeans, lens), not just ashift. The guard also carries a ±2px tolerance now: it is purely defensive, since the genuine discriminators arex==0 && y==0 && scale≈natural_scale(a zoomed/panned ROI has a non-zero origin and a scale strictly abovenatural_scale, so loosening the size test cannot misclassify it).
- Round vs. truncate (primary, parameter-independent).
- A clipping module must render the full uncropped image while editing — by neutralizing its crop
in
commit_params(), the way the crop module does. When a cache-bypass module is focused, the darkroom view already expects the uncropped output (_darkroom_gui_module_requests_uncropped_full_image()keys ondt_iop_get_cache_bypass()). The matching pipeline side is to commit a neutral crop whileg->editing(crop.ccommit_params():cx=cy=0, cw=ch=1; ashift does the same withcl/cr/ct/cb). Thenmodify_roi_out()produces the full output,modify_roi_in()requests the full input, andprocess()captures the whole image intog->buf— all the GUI size caches agree on the full size because they are derived from the same neutralized pipeline data. The crop is reapplied from params on commit/cancel. Because the module opts intoruntime_data_hash(), the neutralized edit state hashes distinctly, so the cache never confuses the edit buffer with the committed one. - Do not widen
roi_inalone while leavingroi_outcropped. It seems clever (the displayed output stays cropped, only the captured buffer grows), but it makes the pipe produce a cropped output while the darkroom view wants the uncropped full image (cache-bypass is on), and the two never reconcile — the preview keeps aborting at initialscale and restarting (the intermittent hang). Neutralize the crop incommit_params()so output, input, view and size caches all describe the same full frame; never split them. - Geometry must not come from a crop-dependent ROI.
piece->roi_inis the minimal input region a module needs for its (possibly cropped) output, so it shrinks with the crop. Anything that reasons about the whole image — auto-crop fitting, aspect ratios — must usepiece->buf_in(the full input at the pipe's scale, crop-independent), neverroi_in. Feedingroi_inback into a crop computation made ashift's auto-crop converge only after several manual toggles.buf_inis a pure geometry query; auto-crop does not read pixels at all, so it should never wait on a buffer. Seeresizing-scaling.mdforbuf_*vsroi_*. - Do not gate buffer-independent GUI work on the pixel buffer. Only pixel-reading work
(auto-detection) needs
g->buf. Manual line/perspective drawing and auto-crop only need geometry, yet the old code routed all of them through the same "fetch the buffer or bail" guard. With the module already carrying parameters the buffer was momentarily unavailable and manual drawing silently stopped accepting input until the module was reset. Run geometry-only jobs immediately. - GUI overlays must invalidate on the geometry they actually track. The control-line overlay caches screen coordinates keyed on the preview-pipe hash, which the worker publishes asynchronously. While editing, the geometry that the overlay transforms through (the virtual pipe) and the displayed size are updated synchronously when the crop/params change, so the cache lagged a frame and "did not adjust" to a new crop mode. ashift now also invalidates that cache when it changes the crop and after each UI-pipe render, so the overlay follows the synchronous geometry.
- Always clean up a cache-wait handle if you do use one. Histogram/picker consumers must call
dt_dev_pixelpipe_cache_wait_cleanup()in their teardown/reset paths: a served wait calls back into the consumer, so a freed consumer would be a use-after-free. (ashift no longer keeps a handle.) - A "won't-settle" recompute loop is driven by a continuous dirty signal, not by the editing
module.
dt_dev_darkroom_pipeline()re-runs a pipe only while eitherpipe_hash != dev_hash(cleared the momentdt_dev_pixelpipe_change()sets the pipe's history hash to the dev's) orpipe->shutdownis raised duringprocess()(then neitherrunsnorreentriesadvances andneeds_updatestays TRUE, so the same pass repeats)._change_pipe()raisesshutdownon every zoom/ROI change, andconfigure()→dt_dev_configure()→dt_dev_pixelpipe_update_zoom_{main,preview}()fires it on each GTK configure-event. So a loop that keeps spinning after the mouse is released is a continuousconfigurestream — a GTK layout/allocation feedback (e.g. in the resizable-panel handle code), amplified while editing because cache-bypass makes every recompute uncached. It is not fixable inside the editing IOP; trace the configure path. Nothing in ashift's settled edit state re-marks the preview pipe.
GUI thread pipeline worker cache-wait manager
---------- --------------- ------------------
peek_gui(prev_piece) ── miss ──▶ set_cache_request(MODULE,prev)
│ or_changed(CACHE_REQUEST)
└── register wait ──────────────────────────────────────▶ pending += {hash}
connect CACHELINE_READY
busy cursor ON
process(): run up to prev_piece
publish prev->global_hash
wrlock_entry(FALSE)
raise CACHELINE_READY(hash) ────▶ match hash in pending
pop record, busy cursor OFF
(marshalled to GUI thread) ◀──── restart(user_data)
restart(): peek_gui() ── hit ──▶ copy buffer, do the work
Note: ashift (section 6) does not follow this lifecycle anymore; it captures its input in its own
process() and resumes from PREVIEW_PIPE_FINISHED. The diagram describes the histogram/picker path.
Two field reports converge here: the color picker eye-dropper produces no value on
some modules (filmicrgb, colorcalibration) while working on others (exposure, tone
EQ) — issue #955 — and the scopes/histogram stay blank — issue #957. Both are the
same defect: a cache-wait whose awaited hash the pipeline never publishes, so
DT_SIGNAL_CACHELINE_READY never matches the pending record and the request "is
never processed and never finishes." (Neither reproduces on the maintainer's
machine, which is why the diagnosis lives in instrumentation — see below.)
The retry protocol is correct only if the hash the GUI awaits equals the hash the worker publishes. Those two hashes are computed at different times, on different threads, by different code:
- GUI (
peek_gui) readspiece->global_hashfrom the currently synchronized node graph — call itH_gui. It registers a pending record keyed onH_guiand emits aCACHE_REQUEST. - Worker (
dt_dev_pixelpipe_process) callsdt_pixelpipe_get_global_hash(pipe), which recomputes every piece'sglobal_hashfrom the pipe's current state (module params, blend/mask params, ROI, buffer descriptors, GUI states — mask preview, cache bypass, color-picker request — andruntime_data_hash()blobs), runs up to the requested piece, and publishes under that recomputed hashH_pipe.
If any hash input changed between the GUI capture and the worker recompute,
H_pipe ≠ H_gui. The worker raises CACHELINE_READY(H_pipe); the manager scans the
pending list for hash == H_pipe, finds only the record holding H_gui, and serves
nobody. The wait stays queued; the request_cacheline dedup then correctly
suppresses re-emission on every subsequent expose (to avoid a request storm), so the
pipe is never re-asked either. Queued forever, busy cursor stuck, no cacheline.
Enabling the eye-dropper sets module->request_color_pick and flags the pipe
changed — which triggers exactly the recompute that re-derives the hashes. Whether
that perturbs the target module's own global_hash, and whether the picker samples
the module's output (needs the module's own cacheline) or only its input
(needs the previous, already-stable module's cacheline), differs per module. A module
whose picker-request or mask-preview state folds into its own global_hash between
capture and publish is inherently exposed to the race; one that samples only an
upstream cacheline is not. runtime_data_hash() modules (colorbalancergb,
colorcalibration's committed data, and the filmic family's) widen the window further,
because piece->hash folds committed piece->data the GUI thread has not committed
yet at capture time. This is the same family as the #710 "acting crazy on some
pictures" defects (§6): a geometry input to the hash (the round-vs-truncate 1px ROI
disagreement, the portrait flip swap) that the two sides compute differently is
just another way to make H_gui ≠ H_pipe.
The scopes (issue #957) are the same failure one hop downstream: they read the
preview backbuffer via peek_gui(piece == NULL), keyed on the published
backbuf.hash. If the preview pipe is itself wedged in the never-served loop (the
picker keeps re-dirtying it, or the awaited module hash never publishes), the
backbuffer is never refreshed and the scopes stay blank.
A secondary way the same wait never settles, even when H_gui == H_pipe: an
OpenCL device-only publish. If the worker writes the output only to vRAM and keeps it
device-only, peek_gui (preferred_devid = -1) reports a miss on restart,
re-requests, the pipe republishes device-only, and the loop never converges. The
reporters saw the bug with OpenCL both on and off, so this is not the primary cause
here, but it is the same symptom and the same instrumentation catches it (a
served → re-queued → served cycle on one hash, the cacheline present but device-only).
The cache-wait manager is now instrumented as its own supervisor domain
(cache-wait, see supervisor.md). Each queued wait carries an awaits edge to the
cacheline hash it is blocked on, so the invisible hang becomes a one-click diagnosis:
- Help → Event supervisor, enable Record, trigger the eye-dropper on the stuck module.
- In Timeline, find the
cache-waitcreatefor ownercolor-picker-input/color-picker-output; note itsawaitshash. A run ofread(dedup-poll) events with no matchingdelete(served) is the stuck signature. - Click / search the
awaitshash. If there is no cachelinecreateunder it, but there is anodeupdate+ cachelinecreatefor the same module under a different hash, that is the mismatch — compare the two hashes'params/roifacets to find which input diverged. Thecomm -23recipe insupervisor.mdlists every orphaned awaited hash in one shot.
The exact-hash match is a fragile primary signal, not a safe only signal. The
cache-wait manager (dev_pixelpipe.c) now serves a pending waiter on either an
exact hash match or a producing-node match — which fixes every consumer that goes
through the manager at once (color picker input_wait/output_wait, histogram
scope_wait/module_wait/picker_wait, autoset input_wait):
- Every cacheline is stamped at publish with the identity of the node that produced
it —
dt_pixel_cache_entry_t.producer_node_key, adt_supervisor_node_key(pipe_type, op, multi_priority). When the write lock is released,DT_SIGNAL_CACHELINE_READYnow carries both the published hash and that producer key (the key is computed on the worker thread and value-copied through the async signal, so no live object or evictable entry is dereferenced on the GUI thread). - Each waiter records the producer key of the output it wants
(
wait->target_node_key, set inpeek_gui();INVALIDfor a backbuf target, which has no single producing node and therefore keeps to exact-hash). _dt_dev_pixelpipe_cache_wait_ready_callback()serves a waiter whenwait->hash == published_hashorwait->target_node_key == producer_node_key. The drift case is exactly "the awaited hash never published but the target module did": the node match then serves the right consumer, and its restart re-reads the module's current output hash and hits. The exact-hash match is kept as the fast path and because it is a pure value comparison independent of the just-published entry (which may already be evicted by the time the GUI-thread callback runs), so the existing served-then-re-miss self-healing is preserved. A node-key serve is logged / supervised asserved (drift: node-key).
The early-return that swallowed the wake-up. The node match above only helps if a
CACHELINE_READY actually fires. But dt_dev_pixelpipe_process() has a fast path: if
the requested target is already host-cached, it returns immediately (after refreshing
the backbuf reference for a backbuf target) without taking a write lock, so no
CACHELINE_READY is raised. Combined with drift this is the exact "no node update,
nothing" hang seen in the field (issue #955, color picker on filmicrgb /
colorcalibration): the picker predicted H_gui, the module's output is sitting in the
cache under H_pipe, the pipe finds that hit and returns, and the waiter — keyed on
H_gui — is never woken even though its buffer exists. dt_dev_pixelpipe_process()
now raises CACHELINE_READY(requested_hash, producer_node_key) on that early-return
when a GUI cache request was pending (cache_request != NONE), so the manager's
producer-node match serves the waiter; its restart re-reads the module's current hash
and hits. It is gated on the pending request so ordinary cache-hit renders add no
signal traffic. This top-level check, in dt_dev_pixelpipe_process() before
process_rec() even starts, does not treat a device-only cached target as a hit: it
uses host-only devid == -1. That only means process_rec() gets invoked next — its own
handling of an existing device-only entry is the separate mechanism described below.
The color picker additionally re-samples on DT_SIGNAL_DEVELOP_PREVIEW_PIPE_FINISHED
(_iop_color_picker_pipe_finished_callback) as belt-and-suspenders: a module-only
CACHE_REQUEST still queues the backbuffer continuation in develop.c, guaranteeing a
PREVIEW_PIPE_FINISHED. It is gated on the picker still being update_pending, so it
is a no-op once a sample has landed.
Established 2026-11-09 on top of
5c15b7c508, in the darkroom under gdb with-d pipecache.
DT_SIGNAL_CACHELINE_READY is asynchronous, so a restart runs on the GUI thread, but it moves
its consumer forward from a place none of the consumer's own callers sees. A GUI state that
mirrors the consumer must be published by the code that changes it, not refreshed by some of its
callers. Autoset is the case in point: a run moves on from the click, from
DT_SIGNAL_DEVELOP_PREVIEW_PIPE_FINISHED and from a served input_wait, and the manager announces
its running state through dt_autoset_manager_t.running_changed, called where it enters and leaves
the busy cursor. The darkroom button is that listener.
To put the last module of a run through the wait path: select only one module, change something
upstream of it (white balance for highlights), and click autoset before the preview re-renders.
The click finds no input and queues a wait on the module before ([cache-wait] queued … owner= autoset-input target=temperature); the wait is served 1.4 s and 5 s later in two runs on a Z 6
raw, and its restart processes the module and ends the run. A listener refreshed only after the
click and on preview-finished while pipe->autoset holds misses that end: the queue is empty, the
busy cursor gone, preview_pipe->autoset false, and the button stays insensitive for the rest of
the session.
process_rec()'s cache acquisition for a node's own output tries two lookups in order:
exact_output_cache_hit(dt_dev_pixelpipe_cache_ref_entry_by_hash()) requires non-NULL host data — a device-only entry does not satisfy it.- If (1) misses,
dt_dev_pixelpipe_cache_get_writable()is asked for a writable handle on the same hash. ItsEXACT_HITstatus only reflects whether an entry exists for that hash (_non_threadsafe_cache_get_entry()), independent of whether it carries host data or of the caller'scache_ram_outputrequest.
Host-residency requirements (piece->cache_output_on_ram, set by
_seal_opencl_cache_policy() — a color picker or histogram starting to sample a piece,
active_in_gui turning on, etc.) are otherwise only consulted when creating a cache
entry. So the EXACT_HIT branch also compares cache_ram_output against the entry's
current residency: when a host copy is wanted and the entry has none -- or only the stale one a
rekey left behind, which dt_pixel_cache_entry_get_data() reports as none -- it calls
dt_dev_pixelpipe_cache_restore_host_payload(cache, exact_entry, pipe->devid, &data) —
the same helper dt_dev_pixelpipe_cache_peek() uses for its own device-owning callers —
to read the GPU payload back to host in place, without recomputing the module.
pipe->devid is a real, locked OpenCL device id at this point in process_rec() (the
lock is taken before the recursion starts).
This matters whenever a module's cachelines were computed device-only before anything
needed a host copy of them — e.g. focusing a color-picker/histogram-consuming module for
the first time on an image where the module (and its upstream piece) already rendered
once, collapsed, before the picker ever engaged _seal_opencl_cache_policy()'s host
requirement for it.
Field signature (-d dev): [pipeline] module=X writable-exact-hit ... has_host_data=0 cache_ram_output=1 followed immediately by [pipeline] module=X exact-hit was device-only, host materialize succeeded ..., both logged from this branch.
This mechanism is independent of the hash-drift race described earlier in this section and does not apply when OpenCL is disabled, since no device-only entries can exist there.
_sample_picker_from_cache() (color_picker_proxy.c) needs two cachelines: the target
module's input (the previous enabled piece's output) and its own output. It
checks output first, and only falls back to requesting input on its own when output can
never resolve (bypass_cache/no_cache/realtime — the color-equalizer-style "input
only" case, where there is nothing downstream to recurse through for us).
The reason: process_rec() always recurses upstream to obtain a node's input before
producing that node's own output, so requesting the module's output as the
CACHE_REQUEST_MODULE target makes a single recompute produce and cache both
cachelines. Checking input first and returning on a miss without looking at output would
cost two sequential CACHE_REQUEST → PIPE needs update → recompute round-trips
whenever both are cold — the common case right after focusing a module whose input/output
were never sampled on the current image (fresh image load, or a module that stayed
collapsed since darkroom opened) — instead of one.
This ordering also means _sample_picker_from_cache() only reaches its final
wait_output_hash/update_pending cleanup once output is either sampled or structurally
blocked, so that cleanup can stay unconditional: it is never reached with output "missing
but expected to arrive later."
The node-key serve makes the drift recover rather than removing it. A backbuf waiter
still relies on exact hash, and consumers that subscribe to CACHELINE_READY directly
(the histogram's initialscale/colorout backbuf refresh triggers, toneequal,
colorequal) still match their own captured hash and could adopt the same producer-node
match if they prove fragile. The cleaner long-term fix removes the GUI-precomputed hash
entirely: (a) have the worker report the actually-published hash for a given
CACHE_REQUEST and match waiters on request identity; or (b) capture H_gui from
the same synchronized graph state the worker will use, i.e. after the pending
pipe-change is synchronized. Either breaks the strict H_gui == H_pipe dependency at
the source.
- The wait manager is a process-wide singleton with a single pending list. It scales with the small
number of concurrent GUI consumers (pickers, histogram), but there is no per-pipe partitioning;
dt_dev_pixelpipe_cache_wait_dump_pending()is the only introspection. - A target that the pipe can never publish (e.g. an OpenCL allocation that keeps failing) leaves
the wait queued and the busy cursor active until the consumer cancels it. The
request_cachelinededup prevents a request storm but does not time the request out. - ashift renders the full uncropped image (and so captures the full
g->buf) only while it is the focused module in edit mode (commit_params()neutralizes the crop then). Outside that window — e.g. a script or future caller that drives auto-detection without entering edit — detection would again see the crop-dependent sub-region. This is acceptable today because the interactive controls are edit-only. - As noted in
reorganisation.md, the long-term direction is to let modules self-trigger theirprocess()and publish directly to the cache, which would let interactive modules refresh their own GUI buffers without a full preview round-trip at all. - The writable-acquire
EXACT_HITmaterialize (§8, "The writable-acquire exact-hit and host-residency policy") is synchronous on the worker thread (a GPU→host read), so the first render after a policy tightens (a picker/histogram starts consuming a previously device-only-cached piece) pays that transfer cost once; subsequent hits are already host-resident and skip it.dt_dev_pixelpipe_cache_get_writable()has a single call site today (pixelpipe_hb.c), so this is not yet a reusable pattern — a second caller needing the same policy-aware-hit behavior should factor it into the cache layer instead of duplicating it.
The startup budgets (dt_configure_runtime_performance(): headroom / mipmap / pixelpipe split of
the detected RAM, or of host_memory_limit) are only plans. They say nothing about what the
system can actually back at any given moment, with other applications competing for the same
physical pages. Historically three compounding problems turned that gap into OOM kills that users
see as "Ansel crashed without message": the budgets were trusted blindly (a host_memory_limit
above physical RAM was even honored as-is), freed arena runs never returned their physical pages
to the OS (RSS was a permanent high-water mark), and nothing at runtime ever looked at the
system-wide available memory.
The defense has five layers, from planning to last resort:
-
Envelope-aware startup budgets (
darktable.c). The detected total RAM is clamped by the physical RAM (so a misconfiguredhost_memory_limitcan only shrink, never grow it) and by the tightest cgroup-v2memory.maxon our own path (containers, Flatpak, systemd slices — the kernel OOM-enforces that envelope no matter how much RAM the machine has). A pressure floor is derived (conf keymemory_pressure_floor,0 = auto= half the OS headroom, bounded to the envelope): the system-wide available RAM under which we start shedding caches. -
Lazy page release on every arena free (
system/memory_arena.c).dt_cache_arena_free()marks the freed runMADV_FREE: the pages stay mapped and re-dirtying them before reclaim costs nothing (the per-frame temp-buffer churn is unaffected), but the kernel may take them back at will under pressure. Measured on a 24 Mpx export: ~44 % of the process RSS is LazyFree at steady state — memory the kernel can have instantly, which starves the OOM killer of a reason to pick us.dt_cache_arena_trim()is the hard variant (MADV_DONTNEED/VirtualFree(MEM_DECOMMIT)) that hands all free runs back NOW; the 3-minute GC and the idle shedder below call it so an idle Ansel doesn't sit on its high-water mark. -
Allocation-time pressure valve (
_system_memory_pressure_valve(),caches/pixelpipe_cache.c). Every arena allocation funnels through_arena_alloc_with_defrag(), which first checks a rate-limited probe of the system-wide available RAM (dt_get_system_available_mem(): MemAvailable on Linux — min'd with the cgroup slack including its reclaimableinactive_file, where our own MADV_FREE'd pages land —ullAvailPhyson Windows, free+inactive on macOS;0means "no information", which disables the valve, not the allocation). If the allocation would push available RAM under the floor, LRU entries holding a host buffer are evicted until the deficit is covered (a device-only line counts its size in the cache but gives the system no RAM back, so it is skipped and cannot cover the deficit on paper), the arena is hard-trimmed, and the probe is then re-read from the OS rather than credited with the bytes we think we released: how much of an eviction actually reaches the system is not ours to guess (the per-freeMADV_FREEis a no-op on Windows, and elsewhere the kernel decides when those pages stop counting against us), and an over-credit would wave the allocation through on a system that is still just as full. If even shedding everything cannot keep HALF the floor, the allocation is refused: the pipeline fails with a clear message, which beats a silent SIGKILL. The message goes through the cache's alert handler, not its warn handler: the GUI installs it (dt_dev_pixelpipe_cache_set_alert_handler(), fromdt_gui_gtk_init()) and shows it withdt_gui_alert()(gui/alert.c), a window kept above Ansel's main window until its OK button is clicked, because what it says -- the module failed, the image was not updated -- stays true long after a toast is gone. Without a GUI it is printed on stderr, since the warn handler is a toast there, which no view draws. It is rate-limited to one every 10 s for one module on one image (pressure_alerts, keyed by both). Until 2026-10-06 the limit was one every 10 s for all of them, and two raws whose thumbnails were refused at startup got one line in the window: the first refusal silenced the second -- found from the code after the user saw one line for two thumbnails out of RAM; the stdout log has no timestamps to show the gap. The size refused, the module that asked for it, that module's image (its file name between backticks -- it can hold spaces -- and its image id, as textpixelpipe_hb.cwrites and hands over withdt_pixelpipe_cache_set_current_image()beside the module's name: the cache prints it, and knows nothing of images) and the RAM left above the floor -- what the valve compared the size with -- are its item, listed under the message, which stays the same at every refusal.dev_pixelpipe.cnames both aroundmodify_roi_in()too, before any module processes: lens allocates its edge buffer there, and on 2026-10-06 a refusal of it (1795728 bytes, with the floor forced to 3000 MiB) listed "1 MiB: only 0 MiB available", with neither module nor image. Lens does not fail on it: it plans its input without the margin. The sizes have two decimals since then: in whole MiB those 1.71 MiB read "1 MiB", which the valve lets through. Indt_dev_pixelpipe_process_rec()the naming starts beforedt_dev_pixelpipe_cache_get_writable(), not atprocess(): that call allocates the module's output when it stays in RAM, as the darkroom's preview does, and opening an image the same day listed "46,69 MiB: only 0,00 MiB available" first -- rawdenoiseai's preview output, refused there -- then the same size named, refused again inpixelpipe_cpu.c. The cache's other user-facing message, "The pipeline cache is full…" (the cache's own cap,_free_space_to_alloc()and_log_arena_allocation_failure()), takes the same path, under the same "Not enough memory" title but in a window of its own --dt_gui_alert()opens one per title and message, and what was being allocated (the cacheline's name, its module) is the item, not part of the text. (Until 2026-10-06 the window was one per title and both messages carried their values, so every size and every module added a paragraph; changed on top of8073021c02.)_log_arena_allocation_failure()raises it only when the arena itself refused, not the valve (_arena_alloc_with_defrag()says which). Until 2026-10-06 a valve refusal raised it too, so one refusal opened two windows and the second was false. Measured on8073021c02by exporting the 24 Mpx X-H1 raw of the image test bank with the installedansel-cli --conf memory_pressure_floor=1250: lens was refused 370 MiB with 1495 MiB available, while the log readcache=741/4461 MiB-- 370 of those 741 being lens's own entry, counted before its buffer exists -- and nothing was evictable (couldn't remove LRU, 2 items and all are used: lens's input and lens's own entry). Lens is where that export peaks: the pipe stays at full resolution untilinitialscale, right after lens, so lens holds demosaic's 370 MiB output while asking for 370 more. The pre-commit image test met the same refusal for real that day, with 445 MiB available. It is raised only when the allocation is actually refused, and then followed by what happens next (_alert_cache_refused()): the module fails, the pipe stops without publishing, nothing retries in tiles (whether to tile was decided beforeprocess()ran, inpixelpipe_cpu.c), and the darkroom keeps its last complete rendering until the next change runs the pipe again (develop.c). Read from the code on 2026-10-03, not measured._free_space_to_alloc()also reaches its message when nothing is left to evict because the memory is held by working buffers, not cachelines:errorstays 0 and the allocation goes ahead past the budget. That case stays a toast. Neither is rate-limited, as the toast was not. Written 2026-10-03 on top of65e7a3adea; the failure it describes was measured on that base with AI denoise on CPU (a 1182 MiB tile refused with 1255 MiB available, the preview left as it was). A 5-second GUI timer (_memory_pressure_shedder()) covers the case where another application creates the pressure while we sit idle and never allocate; it evicts host-buffer lines only, as the valve does.Two traps live in this valve. The probe costs ~61 µs (16 µs
/proc/meminfo+ 45 µs walking the cgroup tree) — enough that the tiling planners alone would spend ~1.8 ms of a 16 ms realtime frame on it — sodt_get_system_available_mem()caches its answer for 50 ms, and any caller that just changed the situation itself must calldt_invalidate_system_available_mem()first or it reads back its own pre-change snapshot. And the valve's running estimate (last probe, minus our own allocations since) legitimately reaches 0 under sustained pressure, so "does this platform answer at all" is a separatesys_probe_validflag: deriving it from anest == 0sentinel would silently disable the valve at exactly the moment it matters most. -
Kernel-pressure shedding (
caches/pixelpipe_cache_pressure.{c,h}). MemAvailable says how much is left, never what it costs to keep it: a machine reclaiming as fast as it allocates reports memory available while systemd-oomd is already counting down on it. Linux PSI reports that directly, andsystem/memory_pressure.cis where every platform detail of it lives — the cumulative "full" counters for the system and every cgroup above the process, plus a watcher that arms the kernel's own triggers and wakes a thread of its own, so the reaction does not wait for a cache that a thrashing machine has already stopped running. The pressure module turns two reads into the stall share of the 2 s window between them and decides what must go;pixelpipe_cache.cpasses it a sink (what it holds, what evicting and trimming gives back) and takeslockaround every call. Past 10 % stall a quarter of the cache goes (half past 30 %) and the budget allocations evict down to is lowered to what is left, climbing back only over minutes of calm and never past 7/8 of the footprint the stall struck at. SeeCLAUDE.mdfor the measurements each of those numbers came from. -
Pressure-aware tiling (
dt_dev_pixelpipe_cache_get_available(),pixelpipe_cache.c). The room tiled modules size their working set from — the tile planners anddt_tiling_piece_fits_host_memory(), which chooses betweenprocess()and tiling — is what an allocation made now would be granted, by the rules that grant or refuse it:MIN(plan − held, (system available − floor) + evictable host RAM). Evictable is what the LRU may take (_entry_evictable():refcount == 0and the entry's lock free), the same test for both, so the planners are never promised memory eviction cannot give back; of the system's share, only lines with a host buffer count, since evicting a line kept on a device gives the OS nothing. The budget is the plan, not the one lowered under kernel pressure: that one is only what allocations evict down to (_free_space_to_alloc()), never a reason to refuse.Found wrong on 2026-10-19, against
fa2dba3dc3. Until then the room wasdt_get_available_mem()(darktable.c, still there with no caller):MIN(lowered budget − used, (system available − floor) + used / 2). It was wrong both ways, and each way failed raw denoise in a reproduction — 4 copies of a 24 Mpx NEF with raw denoise and drawlayer in their history, thumbnails of a lighttable on a fresh profile,--disable-opencl, insystemd-run --user --scope -p MemoryMax=8G -p MemorySwapMax=0with a 4650 MiB process in the same scope. Kernel pressure lowered the budget to 1495 MiB under 1630 MiB held — the four drawlayer sidecars are 4 × 373 MiB of it — so the planner had 0 MiB, cut 1 px tiles, gave up at 5121 × 3453 of them, and_default_process_tiling_ptp()fell back to an untiledprocess(): a 1939 MiB region the valve refused with 1617 MiB available, 1417 over the floor. Planning by the allocator's rules gave 2 tiles of 3072 × 3453 under the same pressure; with the valve's re-read below, all four images went through. The other way,used / 2credited held lines and the module's own allocations as evictable: without the 4650 MiB process, an instrumented run measured that credit at 628 to 815 MiB (half of 1257 to 1631 MiB used) at each of raw denoise's four decisions.A tiling that gives up — too many tiles, or a tile buffer refused — still says "Module failed" and falls back to the untiled
process(), the form of the module that needs the most memory. It is left so:_default_process_tiling_roi()takes the same path when the geometry cannot be tiled, and the planners cap by the largest free run as it stands, without the runs eviction would merge. With the room measured as above, reaching it means the memory really is missing.
The valve itself decides on what the OS says, never on its running estimate: when the estimate is short of a non-negligible request it re-reads the system even if nothing could be evicted. Between probes (100 ms) the estimate is debited by every allocation and credited by no free, so it misses what other pipes gave back since. In the reproduction above with the room fixed but no re-read, the third image's first tile (4608 px, 1724 MiB) was refused on an estimate of 1625 MiB with nothing left to evict, right after the second image's raw denoise had ended. In the next run, with the re-read, the third image's first tile (1508.6 MiB) met an estimate of 1462 MiB, the fresh read said 2123 MiB, and all four images went through.
Verified by running the same export with and without these layers inside
systemd-run --user --scope -p MemoryMax=1536M -p MemorySwapMax=0: before, SIGKILL by the
kernel OOM killer mid-export (journal: Failed with result 'oom-kill'), no output, no message;
after, the envelope is detected at startup, budgets and floor scale to it, the cache sheds under
pressure and the export completes — bit-identical pixels to the unconstrained run — down to a
1280 MB envelope for a 24 Mpx image. Below that (1 GB) the live working set of a single
pipeline pass (2–3 full-resolution float buffers that must coexist) simply doesn't fit, and no
amount of eviction can compress it: that residual floor can only move with more aggressive
tiling of the always-untiled modules, not with cache policy.
Testing lever: the cgroup probe honors whatever limit systemd-run -p MemoryMax= sets, so
pressure behavior is reproducible without actually starving the machine.
Verified against 42eca0e8fe, 2026-09-29. Each finding carries the commit that established it.
Found 22f623c0be, 2026-06-25.
For final-backbuffer display (dt_dev_pixelpipe_cache_peek_gui with piece == NULL), GUI
consumers (center view, navigation thumbnail, scopes) must key the lookup on the published
pipe->backbuf.hash, not the planned dt_dev_pixelpipe_get_hash(pipe) (pipe->hash).
The pipeline plans the next frame's global hash before publishing pixels, so pipe->hash runs
ahead of pipe->backbuf.hash whenever a recompute is in flight. Realtime drawing makes this the
steady state. Peeking the planned hash misses the perfectly valid published frame → the main-surface
lock fails → darkroom expose falls back to the paused preview pipe → flicker.
In peek_gui, for piece == NULL use dt_dev_backbuf_get_hash(&pipe->backbuf) as the display
lookup hash when valid.
Found 22f623c0be, 2026-06-25.
darktable.pixelpipe_cache is shared across all pipes. dt_dev_pixelpipe_cache_flush_clmem
iterates EVERY entry on a device, not just the calling pipe's own. The bug (issue #817): it
released the cl_mem of a buffer another pipe was mid-recursion on, leaving a husk (no RAM,
no vRAM) keyed in the cache, which then aborted downstream consumers → skull thumbnails.
The correct flush predicate: skip any entry where dt_atomic_get_int(&entry->refcount) > 0 OR
dt_pthread_rwlock_trywrlock fails (never wait on writer locks). Idle entries (refcount 0,
unlocked) get their vRAM reclaimed. The flush must hold cache->lock for the entire iteration
so no consumer can mid-acquire a refcount==0 entry.
If a flushed entry is then empty (no host data + no vRAM on any device), remove it from the
hash table via g_hash_table_iter_remove — do NOT subtract current_memory manually, the
_free_cache_entry GDestroyNotify handles it.
Found against 7230d1b355, 2026-10-08 (issue #1546).
dt_dev_pixelpipe_cache_flush() empties the hash table through _for_each_remove(). That predicate
spared locked entries only, although the header promises that referenced ones stay too. The lock is
not what marks a line as in use. A backbuffer keepalive, a scope's, or toneequal's
thumb_preview_entry holds a reference and no lock, so the flush freed such a line under its
holder. The arena got its pixels back while they were still on screen, and the holder's release by
pointer later decremented freed memory. The predicate now has the same refcount guard as
_non_thread_safe_cache_remove() and _for_each_remove_old().
tests/unittests/test_pipe_cache_flush.c pins it.
That this caused any crash is not established. Two use-after-frees on held entries seen on Windows that day had this shape, but a minidump holds no heap, so what freed them is unknown. The defect itself was found by reading every way an entry leaves the table: this was the only one that ignored the refcount.
A consequence of the guard, read rather than measured: a held line now survives the flush. When
the darkroom reruns at an unchanged hash, ref_host_entry_by_hash() finds the frame on screen, so
the flush does not recompute it. Recomputing it would mean unpublishing a held entry while keeping
it alive, and nothing can do that today. Its holder would release it by pointer, but removal goes
through the table by hash, where by then a new entry may answer to the same hash.
Found f4d963f8ca, 2026-09-27.
dt_dev_pixelpipe_cache_peek() is non-owning. An entry nobody holds sits at refcount 0 and any
thread's eviction can free it between the peek and the caller's next line, so code that reads a
cacheline, keeps it, or releases one afterwards, looks it up with dt_dev_pixelpipe_cache_ref_entry_by_hash()
or dt_dev_pixelpipe_cache_ref_host_entry_by_hash() (lookup and reference under one hold of the cache
lock), read-locks while copying, and releases exactly that reference.
Referencing the entry after the peek does not close the window: the eviction can still land before
the reference, which then counts up freed memory. The backbuffer's keepalive is the case that costs
most: taken that way, it is released by pointer at the next publication. So the pipeline publishes
the backbuffer from a retained lookup and drops that reference once the keepalive is taken: the exact
hit before a run from ref_host_entry_by_hash(), the output after it from ref_entry_by_hash()
followed by dt_dev_pixelpipe_cache_restore_host_payload(entry, pipe->devid, NULL).
dt_dev_pixelpipe_cache_get_entry() is a peek too: it serves the producer-to-consumer handoff inside
one run, where the reference is already held, and nothing else. For the same reason,
dt_dev_pixelpipe_cache_get_writable() hands its exact hit back already referenced.
Wrong claim, found 2026-10-05 against 281fc20a37: that commit published the output after the run
from ref_host_entry_by_hash() too, on the premise that "after the run the final output always keeps
a host copy". It does not on an export pipe: dt_imageio_export_with_flags() drives the pipe without
_seal_opencl_cache_policy(), dt_iop_commit_params() leaves the last node at
cache_output_on_ram = 0, and gamma is disabled outside the GUI pipes, so an OpenCL export ends on
a vRAM-only cacheline. The peek it replaced, called with the still-reserved pipe->devid, had been
downloading it; the host-only lookup refused it, the backbuffer was never published, and the export
failed with only [dt_imageio_export_with_flags] no valid output buffer under -d imageio.
Measured by bisect: OpenCL export fails at 281fc20a37, succeeds at its parent; CPU export succeeds
at both. The exact hit before the run is unaffected: pipe->devid is -1 there, so the peek never
materialized a device-only entry either.
A release nobody took is a use-after-free with a delay. It leaves the count below the number of real
holders; the LRU (refcount > 0 is its only guard) frees a held entry; and since every long-lived
reference is released by POINTER (dt_dev_pixelpipe_cache_unref_entry()), the holder's own release
later decrements freed memory -- memory the allocator has meanwhile handed to something else. The
crash lands in that other object, typically as a pointer that reads back as itself minus one (the
refcount decrement landed on it), found by the next free() or dereference of it, nowhere near the
cache. ASAN only sees it when the eviction happens to fall inside the window; the imbalance itself is
deterministic, so the quick way to find one is to report, with a backtrace, every release that takes
a count below zero in _non_thread_safe_cache_ref_count_entry() and exercise the suspect path.
Found ad85705155, 2026-09-28.
Nothing may name a cache entry after releasing its reference: the release returns with the cache mutex released, and from then on any thread's eviction can free an entry nobody holds. A second call on the same pointer, to remove it or to flag it, is a second hold of the mutex, and the eviction can land between the two.
So the removal happens inside the release. dt_dev_pixelpipe_cache_ref_count_entry(FALSE, ...) on
the last reference of an entry flagged with dt_dev_pixelpipe_cache_flag_auto_destroy() removes it
in the same hold of the mutex, and a holder drops a line by flagging it, then releasing it.
tests/unittests/test_pipe_cache_auto_destroy.c pins the contract. Four things a reviewer would
otherwise change:
- Flag before the write lock goes, release last. The release removes only an entry nobody holds
or locks: one released while still write-locked stays, flagged, for the LRU. Flagged before the
unlock, it is also refused by the retained lookups a waiter makes when
DT_SIGNAL_CACHELINE_READYwakes it. - There is no public removal by pointer. It could only succeed on an entry nobody holds, that is, one its caller has no right to name. The peek holds no reference either, so when it discards a line with neither host nor device payload, it looks the entry up again by hash and removes it within one hold of the mutex.
- A line created and not filled goes too, even when no buffer could be allocated. Left in
place, the next
dt_dev_pixelpipe_cache_get()of that hash finds it, allocates its buffer on demand and returns it as found, i.e. as written: the caller reads uninitialised memory as its mask. - The reference
process_rec()reserves for its receiver is released on every way out. The next module, ordt_dev_pixelpipe_process()for the final output, owns it from the moment the recursion returns: an abort after that point releases it as a completed run does, soKILL_SWITCH_ABORT, which releases nothing, has no place after the recursion. A reference left behind pins its line for good.
Found b1a9a18ebd, 2026-09-11.
dt_iop_compute_module_hash() (develop/imageop.c) keys a module by its op, enabled state,
multi_priority, iop_order, params and blendop hash — and must not fold module->instance.
That field is the family id dt_dev_module_duplicate() matches on, assigned at load from
dev->iop_instance++, a counter dt_dev_init() zeroes once for the darkroom's long-lived dev and
nothing resets after: every darkroom entry reloads the modules into the same dev and numbers them
anew. With it in the key, a darkroom → lighttable → darkroom round trip rekeyed every cacheline of
the image, and the preview recomputed from basebuffer while the whole cache was still there —
measured: the entry it could have resumed from was present, and it asked for keys never seen
before. It stayed harmless as long as dt_dev_load_modules() zeroed the counter before numbering,
so every entry numbered the modules alike; only the increment is left there now. The same key feeds hist->hash, hence img->history_hash and the
history_hash.current_hash column, which changed per session for an unchanged history too.
_hash_raster_masks() folded it as well, into the blendop hash of every module consuming another
module's raster mask. dt_iop_check_modules_equal() still compares it, legitimately: that is an
identity test within one session, not a key.
Export paid for it too. dt_imageio_export_with_flags() builds a dev of its own, which
dt_dev_init() numbers from 0, while the darkroom's dev numbers from wherever its counter stands:
0 on the first darkroom entry of a session (its modules come from the dt_dev_init() of the view's
init()), further on after every leave() / enter(). With the id in the key, an export after
any darkroom entry but the first found none of the darkroom's cachelines and recomputed from
basebuffer.
Re-measured 2026-09-29 against 1fb2281865, with -d pipe -d perf -d verbose, OpenCL on, a
6960x4640 raw: darkroom, export, lighttable, darkroom, export. lens's global hash is
2311758751283094755 in the preview, full and export pipes at every step. The first export runs
nothing up to and including lens and starts at initialscale (highlight reconstruction, 3.3 s in
the darkroom, is not rerun); the re-entry runs no module in either pipe; the second export starts
at initialscale again, an export keeping none of its own outputs (see "An export reads the
cache and keeps nothing of its own" below). What the export does recompute is everything after
initialscale, which the darkroom runs at the display scale (0.16 there) and the export at full
size: 3.8 s of that run, tone equalizer, a masked color balance rgb and dither the largest.
With --disable-opencl, darkroom, lighttable, darkroom, export: the same hashes, the re-entry runs
no module, and the export after it again starts at initialscale. On the CPU (i7-3770K) that
remainder is what an export costs: 19.8 s, of which color balance rgb 3.1 s and its masked
instance 9.0 s, filmic 3.3 s.
Anything else folded into a cache key owes the same test: would two sessions editing the same image, with the same history, produce the same value?
Found 540130f59d, 2026-09-29.
The key of a piece covers its parameters, not the pipe state its processing reads. Dither read
two things there: the precision and channels of the output format, pipe->levels, which its
automatic mode quantizes to, and whether the pipe is an export, which sets its step on the other
pipes from the display scale. Neither reached the key. Measured on two ansel-cli exports of one
raw, a JPEG and a 16-bit TIFF: the same dither global hash, two different output content hashes
(-d pipe -d pipecache -d verbose). A pipe keeping its outputs would have handed one export the
other's pixels.
commit_params() now copies both into piece->data, the processing reads them from there only,
and runtime_data_hash() folds piece->data into the key, as colorout already does for the
export's output profile. Same output content hashes as before, two global hashes. A module that
reads pipe state while processing owes the same: copy it at commit, read the copy, opt into
runtime_data_hash().
Found ea59057346, 2026-10-07.
The clipping indicator (overexposed) and the raw clipping indicator (rawoverexposed) render
from darkroom GUI state that no history item carries: dev->overexposed and
dev->rawoverexposed, i.e. mode, colour scheme and thresholds. Both set
dt_iop_set_cache_bypass(module, TRUE) and read that state live in process()/process_cl().
Moving a setting in the toolbox popover resyncs the main pipe, and nothing in the key moves: the
module's parameters are its defaults, and the bypass flag was already set before the change. Every
piece keeps its global hash.
The bypass makes the outputs from the module down auto-destroy, except the last one, so the frame
on screen is still cached under that unchanged hash. dt_dev_pixelpipe_cache_get_writable()
answers DT_DEV_PIXELPIPE_CACHE_WRITABLE_EXACT_HIT for the last module, process_rec()
short-circuits without running anything, and the previous frame comes back. Retouch's preview
toggles failed the same way (iop-notes.md, "combining the mask/wavelet-scale/suppress preview
toggles", fix 1).
The fix follows dither above. commit_params() seals the settings into piece->data, the
processing reads only that copy, and runtime_data_hash() returns TRUE. rawoverexposed used to
write the per-channel raw thresholds it derives at process time back into piece->data. They now
go to a local, because whatever piece->data holds at the next commit is hashed as if it were a
setting. The bypass stays: it keeps overlay frames out of the cache and was never what refreshed
them.
The stale frame was seen in the darkroom. The fix was confirmed there on 2026-10-07, with and
without OpenCL: every setting of both indicators now refreshes the image. The exact-hit path itself
was read from the source, not traced. Its field signature, for whoever traces it: -d dev printing
writable-exact-hit for the last module of the main pipe right after a setting moved.
Found 1019fcd2e0, 2026-09-24.
piece->global_hash folds the module's own parameters, so it moves on every frame of a drag and
is useless as the key of anything the drag does not change. piece->upstream_hash
(pixelpipe_hb.h) is the other half: the cumulative hash of the PARAMETERS of the enabled
modules above this one, and of nothing else — no ROI, not this piece. It identifies the
transformation chain a drawn shape is back-transformed through, which is what a mask
rasterisation depends on, so a memo keyed on it survives an edit of the parameter being dragged.
Three things it has to get right, and each one is a way to key a memo on a lie:
- It is published by a pass of its own (
_publish_upstream_hashes(),dev_pixelpipe.c), called fromdt_dev_pixelpipe_get_roi_in()as well as fromdt_pixelpipe_get_global_hash(). ROI planning is its first consumer and runs BEFORE the global hash is built, so amodify_roi_in()reading it would otherwise find either the valuedt_iop_commit_params()invalidated or one describing a chain that has since changed. It reads no ROI, so running it twice per plan costs a walk and answers the same. - It is deliberately ROI-free, for the same reason: what a consumer reads during ROI planning still describes the previous plan's ROI. It never describes the previous plan's parameters, since every commit path runs a hash pass before any planning.
- It folds
dt_dev_pixelpipe_activemodule_disables_currentmodule()per upstream piece. That is GUI state, not history, so it is in nopiece->hash— yet it decides whether a module takes part indt_dev_distort_transform_plus(). Focusing crop moves every drawn shape's box without moving any parameter.
Because it carries no ROI and no pipe identity, the same value comes out for the FULL, preview
and export pipes of one image, and anything keyed on it that is genuinely pipe-independent — a
shape's bounding box in sensor coordinates — is computed once for all of them. Anything laid out
in the module's own ROI must fold that ROI into its own key on top; iop/retouch.c does both.
Found 5642f84519, 2026-09-11.
dt_tiling_piece_fits_host_memory() (develop/tiling.c) is not a pure question. To answer "does
this module's working set fit untiled?" it evicts LRU cache lines — any pipe's — until the byte
headroom covers factor × roi × bpp AND 0.9 × the largest contiguous arena run does too. The
contiguity term is what makes it expensive: with a fragmented arena it keeps evicting long after
the bytes are there, measured at ~8 GB shed for a 1.95 GB working set.
pixelpipe_cpu.c therefore asks it only when piece->process_tiling_ready — i.e. when the answer
picks process_tiling() over process(). For a module without IOP_FLAGS_ALLOW_TILING (tone
equalizer, among others) process() runs either way, and the probe used to throw the cache away
for nothing: with darkroom/render_size = 0 the preview pipe runs tone equalizer at full sensor
resolution (see the toneequal section), so every edit emptied the cache down to ~3 GB, the FULL
pipe's intermediates went first (least recently used, since the preview had just re-read its own),
and the FULL pipe recomputed from basebuffer — 8 s of highlight reconstruction per edit — while
the preview resumed from the edited module. The module's real allocations still go through the
cache allocator, which evicts what each of them needs when it is made.
Diagnose this class with -d dev -d perf -d pipecache: the processed `Module' … [pipe] lines
say which modules each pipe actually ran, and a burst of LRU … removed lines right after one of
them names the allocation that emptied the cache.
Found 1fb2281865, 2026-09-29.
A tileable module chooses nothing either when default_process_tiling() would not tile it: when
the module's own factor is under 2.2 with a small overhead, tiles save nothing over its input plus
its output, and both tiling paths fall back to process(). That is most pointwise modules. The
probe is therefore asked only when dt_tiling_piece_can_save_memory() is TRUE too, and that
function and both tiling paths share one test (_tiling_saves_memory(), develop/tiling.c). It
reads the module's OWN factor: the probe is handed the factor aggregated with blending, 3.5 for any
module whose blending is not disabled, colorin included, and blending is never tiled.
Measured on a CPU export of a 6959x4639 raw inside systemd-run -p MemoryMax=5G (a 2560 MiB
cache), with a trace in the probe. Asked for every tileable module, it answered "does not fit" from
demosaic to colorout, each time after evicting all it could (the cache left at 985 MiB: the
module's input and output), and every one of those modules but demosaic then fell back to
process(). Asked only where tiling can save memory, it runs once, for demosaic, which tiles as
before; the run takes the same time (24.3 s against 24.1 s) and the pixels do not move (they
differ between the two by what two runs of one binary differ by: the dither is random).
Still open: the probe's total, factor × roi × bpp, counts the input and output buffers the
factor includes, and the pipe has already allocated both when it asks. In that run it asked for
1724 MiB with 1575 MiB free, 985 MiB of the cache being those two buffers; the 740 MiB it lacked
would have fitted. Discounting them would change which modules tile, so it wants its own
measurement.
"with tiling" on a processed line means process_tiling() was called. It can still fall back
(an allocation failing, too many tiles), and -d tiling says when it does.
Found 18f6f8f2c5, 2026-09-12.
The pixelpipe cache budget (total − memory_os_headroom − memory_mipmap_cache) is a plan made at
startup, and _system_memory_pressure_valve() guards a floor of available RAM (200 MiB). Neither
sees a machine that still reports memory available but spends its time reclaiming it: swap full,
other applications' pages evicted and faulted back in. That stall is what systemd-oomd acts on —
on an Ubuntu 24.04 session, past 50 % "full" stall of user@.service for 20 s it SIGKILLs a
cgroup under it, Ansel being the obvious one. Measured on a 24 GB machine with other
applications holding ~9.5 GB and a full swap: the cache reached 11 GB of its 14.5 GB budget,
pressure hit 73 %, and oomd killed Ansel with MemAvailable nowhere near the floor.
Three modules, and the seam between them is what keeps each one readable.
system/memory_pressure.c is the only file that knows what a kernel counter looks like: it reads
PSI's cumulative "full" total for the whole system and every cgroup above the process, and it
owns the watcher — dt_memory_pressure_watch_start() arms the kernel's own triggers, sleeps a
thread of its own in poll(), and calls back. Everything #ifdef-ed on a platform lives there
and nowhere else. caches/pixelpipe_cache_pressure.c turns two of those reads into the stall
share of the window between them (2 s), the highest over the levels; past 10 % it sheds a quarter
of the cache (half past 30 %) and lowers the ceiling, the budget allocations evict down to. Under
2 % the ceiling climbs back by 1/32 of the plan per window, but only to 7/8 of the mark, the
footprint pressure struck at, which itself rises by 1/1024 of the plan per calm window. It reads
no cache entry: pixelpipe_cache.c passes it a dt_pixelpipe_cache_pressure_sink_t — what the
cache holds, and what evicting down to a target and trimming the arena gives back — and takes
lock around every call, the watcher's callback included. dt_pixelpipe_cache_pressure_react()
runs in _free_space_to_alloc() and in the idle shedder, which ticks every 2 s;
dt_pixelpipe_cache_pressure_triggered() runs on the watcher's thread. The arithmetic of the
ceiling and the mark is inline in caches/pixelpipe_cache_pressure.h, kept pure and separate for
the same reason develop/pipe_cache_policy.h is: tests/unittests/test_pipe_cache_pressure.c is
the only thing that can see a policy which changes no pixel and no hash.
Six things a reviewer would otherwise change:
-
A cache holding less than an eighth of the plan sheds nothing and lowers nothing. The stall is somebody else's then. Measured: four kernel wake-ups in the first 12 s of a run, before the cache held anything, recorded a pressure mark of 0 and pinned the budget at the floor, where it still sat a quarter of an hour later. The same eighth is the floor the ceiling never falls under.
-
The kernel wakes the watcher; the cache does not only poll. A measured window needs the cache to be running something, and past a certain stall nothing of ours runs: a second test died with its last 32 seconds silent — no timer, no allocation, the process frozen at 13.6 GB while oomd counted its 20 seconds, and the valve never sampled the ramp that killed it. PSI triggers (
dt_memory_pressure_watch_start(),poll()forPOLLPRI) are raised by the kernel as soon as a window is stalled past the threshold. Unprivileged triggers need a window that is a multiple of 2 s, and only the levels the process may write to accept one: the whole system, its own cgroup, andapp.slice— the session'suser@.service, which is what oomd actually watches, belongs to root. -
The ceiling does not climb straight back to the plan. Measured on the machine above: with the ceiling restored in ~80 s, the stall came back within half a minute of each recovery, five times in five minutes, once at 49 % — one point under oomd's limit. The footprint that caused it (11.5–12.8 GB there) is what the cache must stay under, until minutes of calm say the rest of the machine has let go.
-
The share comes from the
totalcounters, never from PSI'savg10. That is a 10 s moving average, which keeps reading high for some 20 s after a stall has ended: shedding on it drains the whole cache for pressure that is already gone. -
The ceiling is a target, not a limit. An allocation it cannot make room for still goes ahead; only
max_memoryfails one. A hard ceiling would turn pressure into failed pipelines. -
dt_dev_pixelpipe_cache_get_usage()reports the ceiling, floored at the current usage. Tiling must plan against the lowered budget, and both of its readers computemax − currentunsigned.
Found 22f623c0be, 2026-06-25.
dt_dev_pixelpipe_cache_peek_gui must pass preferred_devid = -1 (CPU caller signal). Passing
a real GPU id causes the GUI thread to enqueue a GPU read without owning the device, racing the
pipeline's OpenCL events → SIGSEGV in clReleaseEvent. Device-only entries then report a miss
to the GUI, which waits for the pipeline to publish a host copy instead.
Found 0c28ca29a5, 2026-09-27.
_seal_opencl_cache_policy() (develop/dev_pixelpipe.c) decides per module whether its output
must be copied from device to host RAM (piece->cache_output_on_ram), through the pure
dt_dev_pipe_cache_policy_decide() (develop/pipe_cache_policy.h, pinned by
tests/unittests/test_pipe_cache_policy.c). The requirement travels ONE hop: a node publishes to
RAM when the node consuming it reads RAM (CPU-only, focused, histogram, autoset), never because
some node further downstream does. Carried transitively, it would copy every output device->host
on every frame: measured on a painting stroke, 152 MB and 68 ms of a 113.8 ms frame, of which
gamma's 11.7 MB alone is read.
What makes one hop safe is the cache, not the policy. A module's output cacheline is reused for
its next output by rekeying it in place (_cache_try_rekey_reuse_locked()), host buffer included,
and an OpenCL output kept on the device leaves the previous hash's pixels in that buffer. The rekey
therefore sets dt_pixel_cache_entry_t.host_stale, and while it is set
dt_pixel_cache_entry_get_data() answers NULL, which every lookup (ref_entry_by_hash,
ref_host_entry_by_hash, peek, the fast-track exact hit in process_rec()) and every consumer
already treats as "the pixels are on the device": a CPU consumer copies the device payload back,
a GUI reader owning no device reports a miss. The flag is cleared where the host buffer is written
for the new hash: by process_rec() through dt_dev_pixelpipe_cache_flag_host_written() when the
module ran on CPU, tiled, or read its output back (cache_ram_output), and by every device->host
copy (the materialize path, dt_dev_pixelpipe_cache_restore_cl_buffer(), _gpu_init_input()).
The producer alone reads the raw buffer, through dt_pixel_cache_entry_get_buffer(): it still
backs the output's pinned OpenCL image. -d pipecache prints stale 0|1 on every entry line;
tests/unittests/test_pipe_cache_host_stale.c pins the contract.
An overlay toggle is the case that exposes it: with rawoverexposed on, colorout's requirement
drops and its rekeyed cacheline keeps pre-toggle host bytes under each new pan/zoom hash; once the
overlay is off, dither (CPU-only) needs colorout's output in RAM, and only the flag stops it
from reading those bytes.
Do not detect this in the seal. Invalidating a node's cacheline when its requirement goes
FALSE->TRUE looks equivalent and is not. dt_iop_commit_params() resets cache_output_on_ram to
0 before every commit, so every recommitted node "transitions" on every resync; and at that point
piece->global_hash holds the module's LOCAL hash (the commit writes it, the cumulative key is
computed later). Such an invalidation would fire for every node of every resync and match no
cacheline at all.
Found 0c28ca29a5, 2026-09-27.
Toggling a module, undo, redo and a jump in the history list are switches the user is likely to make again in reverse. Rekeying in place overwrites the output each module produced for the state being left, and switching back then recomputes everything downstream of the change: 2.0 s per toggle on the CPU for tone equalizer on a 5198x3904 raw, the same ten modules every time, with 1.7 GB of a 23 GB cache in use.
DT_DEV_PIPE_SWITCHED (pixelpipe_hb.h) marks such a change. dt_dev_pixelpipe_change() turns
it into pipe->keep_outputs, process_rec() then writes each output into a new cacheline instead
of rekeying (allow_rekey_reuse), and dt_dev_pixelpipe_process() lowers it once a run completes
without error or shutdown, so only the run rendering the new state keeps the old one. Measured
toggling color balance rgb in the darkroom: the first toggle recomputes its downstream once, every
later one is a chain of exact hits, 1-28 ms on the CPU and 3-24 ms with OpenCL. It is raised
by dt_dev_history_commit_item_now() when add_new_pipe_node (the module's first history entry,
or an entry enabled differently from its last one -- the top item rewritten in place included),
and by dt_dev_history_pixelpipe_update(), which every wholesale history replacement goes through.
It is also raised for an edit made from the keyboard, see the next section.
Two things a reviewer would otherwise simplify:
- The pipe's change status cannot tell a switch from a drag.
SYNCHalso carries every slider commit on a module with a drawn or raster mask, and crop/ashift edits; the darkroom worker raisesTOP_CHANGEDon its own whenever the history hash moved (_resync_pipe_with_history()), toggles included. Keeping outputs on everySYNCHgives each step of a masked-module drag its own cachelines and its own vRAM, which fills a 6 GB card within seconds. Only the source knows. add_new_pipe_nodeis computed from the module's last history entry in every case, whether the top item is rewritten in place, a new item appended or one forced. A toggle it misses reaches the pipe asTOP_CHANGED, and nothing downstream knows it was a switch.
Found on top of 3ce1ec6716, 2026-11-09.
Stepping a slider with the arrow keys, then back, recomputed everything downstream of the module every time. There were two causes, and either one alone was enough.
The step rekeyed the state it left. A slider commit reaches the pipe as TOP_CHANGED (SYNCH
on a masked module), not as a switch. Every module downstream therefore rekeyed its cacheline to
the new state, and stepping back found nothing. The keyboard is the source that knows:
_queue_pending_commit() (develop/dev_history_gui.c) marks a request switched when
gtk_get_current_event() is a key press, and dt_dev_history_commit_item_now() raises
DT_DEV_PIPE_SWITCHED for it, as it does for a toggle. A drag, a click and a scroll are not
marked, and keep rekeying. Neither is a press that follows the previous one by less than the
user's timeout (processing/timeout, 200 ms by default): see the held key below.
A step and its reverse did not land on the same float. dt_bauhaus_slider_set_normalized()
rounds the value to the displayed precision but stores a position in [0, 1].
dt_bauhaus_slider_get() turned that position back into min + pos * (max - min), which for most
values is one ulp off the rounded value. A value the slider had produced itself was a fixed point.
A value from anywhere else (module defaults, presets, a picker) was not: 0.7 EV, exposure's default
for raws, stepped by +0.04 and back, committed 0.700000048 instead of 0.699999988, and so hashed
differently. dt_bauhaus_slider_get() now rounds again. Measured offline with a copy of the
slider's arithmetic, over 15 003 start values in [-2, 3] and steps of 0.04, 0.2 and 0.008: 9 243
first returns differed before the change, 0 after. A second return never differed.
Measured in the darkroom with --disable-opencl and -d perf -d history -d dev, on a 418x628 px
preview. Exposure was stepped by key presses 8 s apart, sent through gtk_main_do_event() to the
focused slider. Before the change, every return to a state already rendered ran the 8 modules from
exposure down in the preview pipe. With -d nocache_reuse (no rekeying), none ran, except the
first return to 0.7 EV loaded from history: it landed on another history hash and ran all 8. After
the change, every press is switched and no return runs any module, including 0.7 EV and two steps
back. The same presses emitted on the widget outside a GTK dispatch (no current event) still rekey,
as a drag does.
A held key is a stream, like a drag. It repeats every 30 to 40 ms, and its states come back no
more than a drag's. With every press switched, each commit of a hold kept its outputs. Measured
with OpenCL on a GTX 1060 6 GB, 845x1268 px render, 600 presses at 30 Hz: vRAM rose by about
80 MiB/s for as long as the key was held, peaking at 3012 MiB against 1447 MiB with rekeying, and
stayed there after the release. A press less than the user's timeout after the previous one is
therefore not switched. In the same hold, 1 commit of 142 is switched (the first press), and vRAM
peaks at 1421 MiB. The test reads the user's timeout (dt_gui_throttle_get_user_timeout_ms()),
not the batching window: dt_gui_throttle_get_timeout_ms() shortens the window to the average
redraw time, which on a fast pipe falls below the repeat period. With the timeout set to 0, every
press is switched, held ones included.
The test compares event times, which are the server's and stay spaced by the repeat period however late the application reads them. A shim that stamps presses when it dispatches them sees a busy main loop as gaps, and marks repeats switched: stamp a simulated hold on its schedule.
Found 1fb2281865, 2026-09-29.
dt_dev_pixelpipe_init_export() sets no_cache, as the thumbnail pipe does. Every output the
export computes is then flagged auto-destroy (_bypass_cache() in process_rec()) and goes once
the next module has consumed it; the final output goes at dt_dev_pixelpipe_cleanup(), after
dt_imageio_export_with_flags() has copied it. Lines other pipes hold are still reused:
_bypass_cache() skips the fast-track lookup, so the recursion walks up to basebuffer, and each
line that exists comes back from dt_dev_pixelpipe_cache_get_writable() as an exact hit, never
flagged (-d dev prints writable-exact-hit for it). The darkroom's full-resolution lines, up to
lens, are reused that way.
Kept, the export's outputs were gigabytes nothing reads again. Measured on a CPU export of a
6959x4639 raw after the darkroom, with -d pipecache -d memory and a 200 ms sampler of
/proc/meminfo, /proc/vmstat and PSI: the cache went from 1.6 to 9.0 GB in 19 s and, right after,
~550 ms of "full" stall in a second made the pressure reaction shed 5 GB, least recently used
first: the darkroom's full-resolution lines. The next export recomputed from basebuffer, 24.4 s
against 19.3 s, and pushed MemFree down to ~200 MiB while 11 GB still counted as available: kswapd
swapped other applications out (up to 17k pages per 0.2 s), the system-wide stall passed the
threshold again (~210 ms in a window, Ansel's own cgroup not stalled at all), and 4.5 GB more were
shed. With no_cache: the cache is at 2.1 GB when the export ends and back to 1.6 GB after it, RSS
3.7 GB at most, MemFree never under 905 MiB, no page swapped, 3 ms of stall, nothing shed, and the
second export hits the darkroom's lines again (19.4 s). Six raws exported by ansel-cli on the
CPU: peak RSS 9.6 GB -> 3.0 GB, cache between images 4.1-6.4 GB -> 0, pressure reactions 2 -> 0,
same run time (94 s, 90 s), pixels identical to each image exported alone.
What goes: an identical re-export no longer finds its final output.
Found 1fb2281865, 2026-09-29. RCD and lens re-measured against 119b6c8cd1, 2026-09-30.
dt_pixelpipe_cache_alloc_align_*() hands out arena memory. Fresh pages read zero; a recycled
allocation holds whatever its last user left. A module reading elements it never wrote is
therefore deterministic only as long as its memory is fresh, and the bug shows up as an image
that exports differently depending on what ran before it -- more often the more the arena
recycles: after a pressure shed, or when a pipe drops its outputs as it goes.
RCD (iop/demosaic/rcd.c) did: full tiles read elements of the per-thread rgb scratch that no
tile writes, and only partial tiles cleared it. Found on a batch export: an image differed from
its isolated export (mean 0.36 on 8 bits, in patches); comparing each module's output across the
two runs (-d pipecache -d verbose, [pixelpipe_stats] ... content_hash), demosaic was the first
to differ, from the same input. Poisoning each scratch buffer at allocation, only rgb moved the
output (9.6 M pixels), besides VH_Dir, whose clearing the poison overwrote. Zeroing rgb once at
allocation, as VH_Dir already was, made the images of a six-raw ansel-cli batch whose demosaic
ran untiled identical to each one exported alone -- reproducible, not right: the zeroes still
reached the output. RCD's tiles kept one row and column too many; with a 10-pixel border, nothing
the scratch holds reaches the output any more, on the CPU and in OpenCL, whose kernels had the same
fault at the image edges. See RCD's own tiles need a 10-pixel border.
Found wrong 2026-09-30: this section called "demosaic run in tiles does not give the pixels it
gives untiled" still open and separate. In the same batch, a pressure shed had lowered the
budget and the last three images ran demosaic tiled (-d tiling: no fallback); those three
differed from their isolated exports (mean 0.33-0.49 on 8 bits), the three untiled ones did not. It
is the same defect: the zeroes sat on RCD's tile seams, and pipe tiling moves the seams. With the
border fixed, tiled and untiled differ by float rounding upstream of dither; the 8-bit output
still differs by one level on many pixels, because Floyd-Steinberg diffusion carries any
difference along the rows.
lens wrote the fourth channel of its output only when a mask is displayed. Elsewhere it was
whatever the arena slot last held: two isolated exports of the same raw published different lens
outputs (alpha -0.031..0.999 in one, -0.020..0.9997 in the other), and poisoning RCD's scratch put
±5.7e7 in lens's alpha through the recycled slot. colorin rewrites alpha, which is why nothing
after it differed. lens now always writes the whole pixel, alpha 0 outside mask display; the same
two poisons leave its output bit-identical. RCD's image border had the same fault: the outer three
pixels' alpha was never written on the CPU (rcd_ppg_border()), and in OpenCL
border_interpolate wrote an uninitialised w.
Found on top of 4306d5f952, 2026-10-01. In OpenCL, the three lens_distort_* kernels sample
alpha along with green and used to write it out whether or not a mask was displayed: their alpha
was the input's, resampled, where the CPU writes 0. Not a read of unwritten memory, but a GPU output
that is not the CPU's. On a NIKON Z 6 raw (6064x4040 out), lens's input alpha is 0 everywhere, so
nothing showed. Overwriting the input's alpha with ±5.7e7 just before the kernel (read back with
dt_opencl_copy_device_to_host() in process_cl(), written with
dt_opencl_write_host_to_device()) put nonzero alpha on 23.9 M (bilinear) and all 24.5 M
(bicubic, mitchell) of the 24.5 M output pixels. The kernels now take the mask-display flag and
write 0 outside it: with the same poison their output is bit-identical to the unpoisoned one, and
with the flag forced on it is bit-identical to the old kernels', poisoned or not. The exported
image is the same in every case, colorin rewriting alpha. Where nothing moves (the copy and
lens_vignette), lens carries its input's alpha, on the CPU and in OpenCL alike.
To look for others: poison with a large FINITE value. NaN is absorbed by fmaxf() and by
comparisons, and -d nan fills only the output buffer and checks only its RGB channels. In OpenCL,
fill each buffer after allocation from a host buffer holding the poison
(dt_opencl_write_buffer_to_device()). To read an OpenCL module's output, its host copy has to be
forced for the measurement: the export pipe is no_cache, so /plugins/<op>/cache does not apply
there (cache_ram_output in pixelpipe_hb.c requires the cache not to be bypassed).
Found 199597e02a, 2026-10-19.
dt_dev_pixelpipe_cleanup_nodes() releases the references the pipe holds on its raster masks
(raster_mask_hashes) and on the detail mask (rawdetail_mask_hash), with the nodes that
published them. Left to the next render, they outlive the image on a pipe that does not render
again: the darkroom's two pipes live for the whole session, and leave() (darkroom and studio
capture) destroys their nodes only. A referenced line is skipped by the LRU, the pressure valve
and the kernel-pressure shedding alike, so no demand for memory could free those masks.
Measured in the GUI with an LD_PRELOAD library that writes dt_dev_pixelpipe_cache_get_entries_stats()
on request (the source of the supervisor's memory view, refcounts included): open in the darkroom
an image whose modules consume raster masks, go back to the lighttable, and no line may remain
referenced. With the release left to the next render, two raster masks of the preview pipe, 3.5 MiB
each, stayed at refs=2 there; a full-resolution one, 68 MiB, was seen held the same way.
tests/unittests/test_pipe_cleanup_nodes_masks.c pins the release.
On the rebuild of a changed topology (DT_DEV_PIPE_REMOVE in dt_dev_pixelpipe_change()), the
masks are released before the following render retains them again. For a raster mask, an eviction
in between costs one recompute of the provider (the DT_DEV_PIPE_REENTRY pass of "Raster masks are
dedicated side-band cachelines"), never a wrong mask: a consumer reads a mask through its own
retained lookup. The detail mask has no such pass; the next section is what covers it.
That statistics call, and the memory view built on it, listed entries only then. The working
buffers of dt_pixelpipe_cache_alloc_align_*() live in external_entries: they counted in
current_memory and in the usage bar, but appeared in no list. Seen that day: 6.0 GiB of usage
against 0.2 GiB of listed entries, the rest being the working buffers of a thumbnail pipe
reconstructing highlights at full resolution. The call returns them now: see "The cache's usage
counts its allocations as well as its lines".
Found ca319461b9, 2026-10-19, from a review of the release above.
The hidden detailmask stage publishes two cachelines: its output, a copy of its input, and the
detail mask, under dt_dev_pixelpipe_rawdetail_mask_hash(). Either can be evicted without the
other, and the output alone cannot rebuild the mask. Its consumers, the detail refinement of the
blend masks (_refine_with_detail_mask() and its OpenCL twin in blend.c), find the mask with
dt_dev_retrieve_rawdetail_mask() and, on a miss, skip the refinement without an error: the
consumer's output is then cached under the hash of a refined one. A raster mask that goes missing
fails its consumer and gets a DT_DEV_PIPE_REENTRY pass; the detail mask had nothing of the kind.
So dt_dev_pixelpipe_process_rec() calls dt_dev_pixelpipe_rawdetail_mask_check_hit() before it
looks up the stage's output. When the mask is there, the pipe takes it at once, as the stage's
process() would. When it is not, the output is invalidated: neither the exact hit nor the
writable acquisition after the recursion finds it, the stage runs, and the mask is published again.
The invalidation removes only a line nobody holds; pipes run one at a time, under
dt_pipeline_threadsafe_mutex(), so no other pipe holds the stage's output then.
The hazard predates the release in cleanup_nodes(): any pipe that did not hold the mask could hit
the stage's output without it -- a thumbnail pipe, or a darkroom pipe back on a history state whose
mask it had released. The release widened it to darkroom re-entry and to the topology rebuild.
Read from the code, not reproduced: the eviction has to take the mask and spare the output.
tests/unittests/test_pipe_cleanup_nodes_masks.c pins the rule.
Found 948fbf7c90, 2026-10-19.
What dt_pixelpipe_cache_alloc_align_*() allocates lives in external_entries, not in entries,
and counts in current_memory all the same: an entry of either table adds its size when it is
created (dt_pixel_cache_new_entry()) and takes it back when it is freed, so the usage is the sizes
of both tables summed. dt_dev_pixelpipe_cache_get_entries_stats() returns both, under one hold of
lock: the allocations flagged allocation, named after the file:line that made them (from
src/ on: the absolute root of __FILE__ would push the line number out of the 64-byte name) and
aged since then (age_us; _pixel_cache_touch() stamps an allocation when it is made, and the one
lookup that would touch it again, dt_dev_pixelpipe_cache_get_entry_by_data(), has no caller). The
supervisor's memory view lists them with the lines in RAM (supervisor.md).
Not every allocation is a working buffer of one run. Some are kept for the session, and seen at
rest the same day: demosaic's two LMMSE gamma tables (iop/demosaic.c:1193-1194, 0.25 MiB each,
made on the first LMMSE run and freed in cleanup_global()), and the histogram's state struct
(libs/histogram.c:3075, made in gui_init(), freed in gui_cleanup()). They are held, so neither
the LRU nor the pressure valve can free them, and they count against the budget for the session.