Repository navigation
Run gVisor sandboxes without a pause container - #2773
Jing (milantracy) wants to merge 1 commit into
Conversation
Each actor sandbox had a root container that ran /pause from a pause image. atelet pulled that image and built its bundle, and every SandboxConfig had to name it. runsc can now boot a sandbox with no root container and checkpoint and restore it, so the pause container is not necessary. ateworker-gvisor now creates the _pause sandbox with -no-root-container and writes its spec with no process, rootfs, or mounts. The spec keeps the network namespace, cgroup, resource limits, and durable-dir mount hints. Application containers do not change. Remove pauseImage from the SandboxConfig CRD, from SandboxAssets in the atelet API, from the snapshot manifest, from the sandbox runtime hash, and from asset prewarm. Remove the pause_rootfs restore phase.
|
I verified the change locally The gVisor pin in manifests/ate-install/sandboxconfig-gvisor.yaml still points at the 2026-10-10 nightly, which can't checkpoint/restore a sandbox with no root container. I'll move it to the first nightly with google/gvisor#15551 or later from a different PR before this change is merged |
Benjamin Elder (BenTheElder)
left a comment
Benjamin Elder (BenTheElder)
left a comment
There was a problem hiding this comment.
minor comments, generally +1
you'll probably have to rebase a bit when the nightly is ready, feel free to drop me a ping when the nightly is ready.
| // -no-root-container, so it has no process and no rootfs. The underscore | ||
| // keeps it outside the k8s-short-name an ActorTemplate container | ||
| // name is drawn from, so no actor container can collide with it. | ||
| const PauseContainer = "_pause" |
There was a problem hiding this comment.
Nit: With no pause container left, should we rename _pause and PauseContainer to something like _sandbox? We can still change it pre-1.0, and right now the docs have to explain that the sandbox's runsc output lives under _pause. The pause_create/pause_restore phase names in phaselog.go and phase_report.py, and the "pause leaf" wording in ckptcgroup.go, would follow.
| ### Sandbox Classes | ||
|
|
||
| A `WorkerPool` selects a **sandbox class** (`spec.sandboxClasses[].name`), and each class has a matching `ateworker` herder image. The sandbox binaries themselves are not baked into the worker image — they, and the pause image holding the sandbox's namespaces, come at runtime from a cluster-scoped [`SandboxConfig`](api-guide.md#3-sandboxconfig-the-sandbox-itself) the `ActorTemplate` names in its sandbox config (naming one is currently required; per-class cluster defaults are planned) and are pinned into each snapshot's manifest so restores stay reproducible across runtime upgrades. | ||
| A `WorkerPool` selects a **sandbox class** (`spec.sandboxClasses[].name`), and each class has a matching `ateworker` herder image. The sandbox binaries themselves are not baked into the worker image — they come at runtime from a cluster-scoped [`SandboxConfig`](api-guide.md#3-sandboxconfig-the-sandbox-itself) the `ActorTemplate` names in its sandbox config (naming one is currently required; per-class cluster defaults are planned) and are pinned into each snapshot's manifest so restores stay reproducible across runtime upgrades. |
There was a problem hiding this comment.
Nit: Not new to this PR, but since we're touching this line: "pinned into each snapshot's manifest so restores stay reproducible across runtime upgrades" contradicts api-guide.md, which says a restore runs the current default version. Restore takes the sandbox assets from the request, so we should probably drop that clause. cc Zoe Zhao (@zoez7)
2 participants
Each actor sandbox had a root container that ran /pause from a pause image. atelet pulled that image and built its bundle, and every SandboxConfig had to name it. runsc can now boot a sandbox with no root container and checkpoint and restore it, so the pause container is not necessary.
ateworker-gvisor now creates the _pause sandbox with -no-root-container and writes its spec with no process, rootfs, or mounts. The spec keeps the network namespace, cgroup, resource limits, and durable-dir mount hints. Application containers do not change.
Remove pauseImage from the SandboxConfig CRD, from SandboxAssets in the atelet API, from the snapshot manifest, from the sandbox runtime hash, and from asset prewarm. Remove the pause_rootfs restore phase.
Breaking change
Fixes #967