Let systemd-oomd kill a runaway app instead of losing the session - #6445
Open
dhh wants to merge 1 commit into
Open
Let systemd-oomd kill a runaway app instead of losing the session#6445dhh wants to merge 1 commit into
dhh wants to merge 1 commit into
Conversation
Nothing stood between "memory is tight" and "processes die at random": the kernel OOM killer only fires after an allocation has already failed, so a machine under pressure thrashes in reclaim until something breaks somewhere fatal — seen in practice as Hyprland taking SIGBUS mid-memcpy from a client shm pool that memory pressure was tearing down. Enable systemd-oomd, which keys on PSI stall time rather than free pages, and mark app.slice — and only app.slice — as its kill candidate. Hyprland runs in session.slice, so the compositor is structurally ineligible as a victim: oomd takes the app that caused the pressure and the session survives. Thresholds (50% stall over 20s, Fedora's desktop defaults) live in an oomd.conf.d drop-in; candidacy ships as a vendor drop-in under /usr/lib/systemd/user so existing users get it on package upgrade with no per-user seeding. New installs enable the daemon from enable-services.sh; a migration covers existing ones, restarting an already-running oomd so it doesn't keep stale thresholds until reboot. Deliberately not earlyoom: it triggers only when free RAM and free swap are both under threshold, so Omarchy's large, mostly-idle swapfiles keep it asleep through exactly this failure. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Post-mortem of a real crash: a 62 GB box ran out of RAM, no OOM daemon intervened, and the kernel OOM killer never fired (it acts only after an allocation fails outright, and the machine was thrashing in reclaim instead). A Chromium renderer died on a fatal allocation CHECK, its
wl_shmpool was torn down mid-frame, and Hyprland took SIGBUS copying from the unbacked page — the whole session gone because one app misbehaved.What
etc/systemd/oomd.conf.d/10-omarchy.conf— thresholds: act when half of a 10s window is spent stalled on reclaim, sustained for 20s (Fedora's shipped desktop defaults). systemd-oomd keys on PSI stall time, so it fires while the machine thrashes, not after an allocation fails.default/systemd/user/app.slice.d/10-oomd.conf— kill candidacy onapp.sliceand onlyapp.slice. Hyprland runs insession.slice, so the compositor is structurally ineligible as a victim: oomd takes the browser/terminal that caused the pressure and the session survives to show the notification. Ships as a vendor drop-in under/usr/lib/systemd/user/so existing users get it on package upgrade with no per-user seeding, and~/.config/systemd/user/app.slice.d/stays free as an override layer.install/config/enable-services.sh— new installs enable the daemon (socket comes along viaAlso=).migrations/1785424256.sh— existing installs enable it; if a user had oomd enabled already, it's restarted so it doesn't keep pre-package thresholds until reboot. Then a user-managerdaemon-reloadso app.slice candidacy is reported without a relogin.test/shell.d/systemd-test.sh— the load-bearing assertion is the negative one: the app.slice drop-in must be the onlyManagedOOM*=killin the tree. Setting candidacy onuser@.serviceinstead (what Fedora does) puts the compositor back in the victim pool — the exact crash this prevents.Deliberately not earlyoom: it triggers only when free RAM and free swap are both under threshold. The crashed box had 73 GB of untouched swap, so earlyoom would have slept through the entire failure — Omarchy's large hibernation swapfiles make that its steady state.
Kill granularity
oomd kills a whole cgroup, so the victim is an entire
app-*.scope— the whole browser (session-restorable), or the whole terminal scope including a tmux server inside it. "Lose one app" rather than "lose one tab," but categorically better than losing the compositor.ManagedOOMPreference=avoidexists per-unit if this ever needs tuning; no preferences shipped so the candidate pool can't be whittled down to nothing.Verified
systemctl --user show app.sliceresolves the/usr/libdrop-in, andoomctllistsapp.sliceunder both monitored sections with 50%/20s../test/shell(three new assertions) and./test/clipass. One pre-existing failure (bar icons share slot and baseline geometry) fails identically on a clean tree.0644, no shebang, idempotent).Needs a companion change
omarchy-settingsPKGBUILD inomarchy-pkgsneeds to install the new vendor file — the existing user-unit glob won't pick up a subdirectory:install -Dm644 default/systemd/user/app.slice.d/10-oomd.conf \ "$pkgdir/usr/lib/systemd/user/app.slice.d/10-oomd.conf"(And a check that
etc/systemd/oomd.conf.d/rides along with howeveretc/**is installed.)🤖 Generated with Claude Code