Make pivot_root work in initramfs; deprecate and ignore --no-pivot by kolyshkin · Pull Request #5575 · opencontainers/runc

kolyshkin · 2026-10-25T08:42:16Z

This is a carry/rework of #4434.

1. pivot_root in initramfs

Before Linux 7.0 (which added nullfs underneath the initramfs), pivot_root(2) fails with EINVAL when running from initramfs, as the initramfs mount has no parent. Users were told to use --no-pivot, which is insecure.

The workaround (as in #4434) is to make a recursive bind-mount clone of /, move it on top of /, and chroot into it, after which pivot_root(2) works. Changes compared to #4434:

rebased onto the preopened rootfs fd (pivotRoot now takes *os.File);
the workaround is only used if the current root mount is its own parent (i.e. it is the initramfs), rather than on any EINVAL (which can also mean e.g. shared propagation);
/ is made a slave before the move, so it does not propagate to the host if / is shared;
the original mounts on top of the initramfs are detached. Since the initramfs can never be unmounted, otherwise those (including full procfs and sysfs mounts) stay in the container's mount namespace, hidden underneath the container root, and a container can use them to mount a fresh procfs/sysfs from a user namespace, bypassing masked paths. I have verified that this was possible with the approach from #4434 and the new test checks for it.

2. Deprecating --no-pivot, making it a no-op

Since the main use case for --no-pivot is now covered, and MS_MOVE+chroot is insecure, the MS_MOVE+chroot code (msMoveRoot) is removed, and --no-pivot is now a no-op (with a warning), deprecated and hidden from --help. If pivot_root(2) fails, it is an error. configs.Config.NoPivotRoot and specconv.CreateOpts.NoPivotRoot are kept for compatibility, but are deprecated and ignored. The checks specific to --no-pivot are removed.

See #4435 for the deprecation tracking issue.

3. Tests

New tests/integration/initramfs.bats boots a qemu VM with the host kernel and an initramfs made of busybox image, runc, and a bundle, and checks that:

a container runs (and its root is the container rootfs, not the initramfs);
host mounts are not changed (including the rshared root case);
--no-pivot is ignored;
a container can't mount fresh procfs/sysfs from a user namespace.

Both x86_64 and aarch64 are supported (EFI zboot kernel images are unpacked, as qemu can only load gzip-compressed ones). The tests do not require root, but are skipped if qemu or a readable host kernel image is not available. CI now installs qemu in the test jobs; ubuntu-24.04 (kernel 6.17) exercises the workaround, and ubuntu-26.04 (kernel 7.0) the native path.

Closes: #4434

Fixes: #1961

Closes: #5532

kolyshkin · 2026-10-25T08:51:29Z

@lifubang @cyphar PTAL (I remember there was some issue with #4434 found by @lifubang -- do we have it here)?

AkihiroSuda · 2026-10-26T03:28:49Z

-  refused. Previously, `--no-pivot` resulted in host's `/proc` and `/sys` being
-  unmounted, and `rootfsPropagation` changed the propagation of host mounts.
-  (#5535)
+- runc now uses `pivot_root(2)` when running from initramfs on kernels older


The behavior on >= v7.0 should be described too?

added "(on v7.0+, pivot_root(2) works in initramfs natively)"

(and I checked it on 7.2 -- it works without --no-pivot even before this PR)

Copilot

🟡 Changes recommended

Mount cleanup can retain full procfs/sysfs beneath stacked mounts and abort startup when inherited mounts are locked by a user namespace.

2 open issues

What changed in this PR

This PR adds an initramfs workaround for pivot_root, replacing the insecure --no-pivot path while retaining deprecated CLI and configuration compatibility.

Changes:

Adds root-mount cloning and cleanup for initramfs on older kernels.
Makes --no-pivot ignored and hidden, with a warning and updated documentation.
Adds QEMU-based integration coverage and installs QEMU in CI.

File	Description
`utils_linux.go`	Warns when `--no-pivot` is supplied.
`tests/integration/no_pivot.bats`	Checks the warning and removes obsolete validation coverage.
`tests/integration/initramfs.bats`	Adds VM tests for startup, propagation, and mount isolation.
`tests/integration/helpers.bash`	Adds QEMU setup, kernel unpacking, and execution helpers.
`run.go`	Hides and deprecates `--no-pivot`.
`restore.go`	Hides and deprecates `--no-pivot`.
`man/runc-run.8.md`	Documents the ignored option.
`man/runc-restore.8.md`	Documents the ignored option.
`man/runc-create.8.md`	Documents the ignored option.
`libcontainer/specconv/spec_linux.go`	Stops applying `NoPivotRoot` and removes its propagation restriction.
`libcontainer/rootfs_linux.go`	Replaces MS_MOVE-based root switching with the initramfs pivot workaround.
`libcontainer/configs/validate/validator.go`	Removes obsolete no-pivot validation.
`libcontainer/configs/config.go`	Marks `NoPivotRoot` deprecated and ignored.
`create.go`	Hides and deprecates `--no-pivot`.
`CHANGELOG.md`	Records initramfs support and no-pivot deprecation.
`.github/workflows/test.yml`	Installs architecture-appropriate QEMU packages.

_{Review effort: Balanced}

kolyshkin · 2026-10-26T07:13:58Z

+	// mounted first. Note that paths starting with / are resolved from
+	// our current root (the initramfs), not from the clone on top of it.
+	for _, m := range slices.Backward(mounts) {
+		if m.Parent != rootID || m.ID == rootID || m.Mountpoint == "/" {


Fixed by re-running umount until we get EINVAL (the first EINVAL is still considered fatal, the second means this is no longer a mount point).

kolyshkin · 2026-10-26T07:15:17Z

+		if err := unmount(m.Mountpoint, unix.MNT_DETACH); err != nil {
+			return true, err


Yes, this is a limitation of the new code. It looks like no one is using this combination (runc on initramfs root with user namespace and kernel < 7.0), but if I'm wrong, we may need to carry msMoveMount.

Before Linux 7.0 (which added nullfs underneath the initramfs), pivot_root(2) fails with EINVAL if the current root is the initramfs, because the initramfs mount has no parent. Users were told to use --no-pivot instead, which is far less secure. Work around it by creating a recursive bind-mount clone of / (which does have a parent), moving it on top of /, and chroot(2)-ing into it, after which pivot_root(2) can be retried. The chroot(2) is only temporary; once pivot_root(2) succeeds, the clone is detached as usual. The container rootfs is used as a temporary mount point for the clone, so this works with read-only initramfs images. Variants of this hack are fairly well known (see [1], [2]). The workaround is only used when pivot_root(2) fails with EINVAL and the current root mount is its own parent (i.e. it is the initramfs), rather than for any EINVAL (which can also mean e.g. shared propagation). Before the move, / is made a slave so the move does not propagate to the host if / is shared. Since the initramfs can never be unmounted, any mounts on top of it would stay in the container's mount namespace forever (hidden underneath the container root). Those include full procfs and sysfs mounts, which can be used to bypass procfs/sysfs mount restrictions (i.e. a container could mount a fresh procfs from within a user namespace). To prevent that, detach all the original mounts on top of the initramfs (we only use their clones), including any mounts stacked on top of them. If the container has a user namespace, the original mounts are locked (as the mount namespace is created after the user namespace), and can't be detached, so such a container can't be run from initramfs on kernels older than v7.0. [1]: containers/bubblewrap#592 (comment) [2]: https://aconz2.github.io/2024/07/29/container-from-initramfs.html Signed-off-by: Aleksa Sarai <cyphar@cyphar.com> [kolyshkin: rebase onto preopened rootfs; only use the workaround for initramfs; make / a slave before the move; detach original mounts; reword commit message] Assisted-by: Claude Opus 5.5 Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>

The main use case for --no-pivot was running from initramfs, where pivot_root(2) used to fail. Now pivot_root(2) works there (natively since Linux 7.0, and with a workaround added by the previous commit for older kernels). MS_MOVE+chroot(2) is insecure (even with all the hardening we added over the years), so remove it, and always use pivot_root(2) when a mount namespace is used. If pivot_root(2) fails, it is an error. Keep configs.Config.NoPivotRoot and specconv.CreateOpts.NoPivotRoot for compatibility, but deprecate them, as they are now ignored. Make runc --no-pivot a no-op, printing a warning. Remove the checks which are no longer needed, as they were specific to --no-pivot: - no-pivot with no mount namespace (added by commit 66e3067e); - rootfsPropagation of [r]private with no-pivot. Drop the "runc run --no-pivot [host mount ns] must fail" test, and amend the "runc run --no-pivot must not expose bare /proc" test to check for the warning. Assisted-by: Claude Opus 5.5 Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>

Now that pivot_root(2) works in initramfs, --no-pivot is not needed, and it is ignored. Mark it as deprecated, hide it from runc create, run and restore --help, and update the man pages and the changelog. Signed-off-by: Aleksa Sarai <cyphar@cyphar.com> [kolyshkin: also hide it for runc restore; update man pages and changelog] Assisted-by: Claude Opus 5.5 Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>

Add tests which run runc from initramfs, using a qemu VM with the host kernel, and an initramfs made of busybox image, runc binary (with its libraries), and a container bundle. The tests check that: - a container can be run (and its root is the container rootfs); - host mounts are not changed (including the case of rshared root); - --no-pivot is ignored; - container can't mount fresh procfs or sysfs from a user namespace (including the case of stacked /proc and /sys mounts); - a container with a user namespace works on v7.0+, and fails with a clear error on older kernels. Note that the pivot_root workaround is only used on kernels < v7.0. Both x86_64 and aarch64 are supported. Since qemu can't load EFI zboot kernel images unless they are gzip-compressed (and Ubuntu arm64 kernels since v7.0 are zstd-compressed), unpack those. The qemu and initramfs helpers are added to helpers.bash, so they can be reused by other tests. Install qemu in CI jobs (the tests are skipped if qemu or the host kernel image is not available). Based on the test originally written by Aleksa Sarai. Co-Authored-By: Aleksa Sarai <cyphar@cyphar.com> Assisted-by: Claude Opus 5.5 Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>

kolyshkin force-pushed the kir/carry-4434 branch from 87b5b53 to ae91f44 Compare October 25, 2026 08:42

kolyshkin requested review from cyphar and lifubang October 25, 2026 08:51

kolyshkin mentioned this pull request Oct 26, 2026

rootfs: make pivot_root(2) dance handle initramfs case #4434

Closed

AkihiroSuda reviewed Oct 26, 2026

View reviewed changes

Comment thread tests/integration/initramfs.bats Resolved

AkihiroSuda reviewed Oct 26, 2026

View reviewed changes

Comment thread tests/integration/initramfs.bats Resolved

AkihiroSuda reviewed Oct 26, 2026

View reviewed changes

kolyshkin force-pushed the kir/carry-4434 branch from ae91f44 to 3716f65 Compare October 26, 2026 05:55

AkihiroSuda requested a balanced review from Copilot October 26, 2026 06:05

Copilot started reviewing on behalf of AkihiroSuda October 26, 2026 06:06

View session

Copilot AI reviewed Oct 26, 2026

View reviewed changes

kolyshkin force-pushed the kir/carry-4434 branch from 3716f65 to 0860950 Compare October 26, 2026 06:51

kolyshkin mentioned this pull request Oct 26, 2026

[tracking issue] deprecation of insecure --no-pivot flag #4435

Open

kolyshkin force-pushed the kir/carry-4434 branch from 0860950 to 859d350 Compare October 26, 2026 07:21

cyphar and others added 4 commits October 30, 2026 11:45

kolyshkin force-pushed the kir/carry-4434 branch from 859d350 to 1882783 Compare October 30, 2026 18:46

Navigation Menu

Repository navigation

Make pivot_root work in initramfs; deprecate and ignore --no-pivot - #5575

Make pivot_root work in initramfs; deprecate and ignore --no-pivot#5575
kolyshkin wants to merge 4 commits into
opencontainers:mainfrom
kolyshkin:kir/carry-4434

kolyshkin commented Oct 25, 2026 •

edited

Loading

Uh oh!

kolyshkin commented Oct 25, 2026

Uh oh!

Uh oh!

Uh oh!

AkihiroSuda Oct 26, 2026

Uh oh!

kolyshkin Oct 26, 2026

Uh oh!

kolyshkin Oct 26, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

kolyshkin Oct 26, 2026

Uh oh!

kolyshkin Oct 26, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

		if err := unmount(m.Mountpoint, unix.MNT_DETACH); err != nil {
		return true, err

Conversation

kolyshkin commented Oct 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

1. pivot_root in initramfs

2. Deprecating --no-pivot, making it a no-op

3. Tests

Uh oh!

kolyshkin commented Oct 25, 2026

Uh oh!

Uh oh!

Uh oh!

AkihiroSuda Oct 26, 2026

Choose a reason for hiding this comment

Uh oh!

kolyshkin Oct 26, 2026

Choose a reason for hiding this comment

Uh oh!

kolyshkin Oct 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

kolyshkin Oct 26, 2026

Choose a reason for hiding this comment

Uh oh!

kolyshkin Oct 26, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

kolyshkin commented Oct 25, 2026 •

edited

Loading

kolyshkin Oct 26, 2026 •

edited

Loading