Scope the default sccache S3 prefix per repository and make it configurable - #58
Open
garysassano wants to merge 2 commits into
Open
Scope the default sccache S3 prefix per repository and make it configurable#58garysassano wants to merge 2 commits into
garysassano wants to merge 2 commits into
Conversation
…urable
With `sccache: s3`, every repository on a stack shared one flat prefix
(`cache/sccache`). Content keys keep compilation results from colliding, so this
was not a correctness problem, but a single stack-wide namespace left no way to
attribute cache growth to a repository, expire one repository's objects, or
change one repository's layout without affecting the others.
The default is now derived from repository and platform identity:
cache/sccache/<repository id>/<runner os>-<runner arch>/v1
The numeric repository id is preferred over the owner/name slug so a rename or
transfer does not discard the cache; the slug is the fallback when the id is
absent, kept as two key components so `foo-bar/baz` and `foo/bar-baz` cannot
collide. RUNNER_OS/RUNNER_ARCH give the platform component, falling back to the
binary's GOOS/GOARCH. The trailing `v1` is a layout version, so a future change
can roll out without reusing objects written under the old scheme. Scope
components are lowercased and restricted to [a-z0-9._-] so an unexpected value
cannot reshape the key.
An explicit prefix is still honoured, and is rejected when it cannot be exported
or used as a key.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
With
sccache: s3, every repository on a stack gets the same flat prefix:sccachecontent keys keep compilation results from colliding, so this is not a correctness problem, but a single stack-wide namespace leaves no way to attribute cache growth and cost to a repository, expire or invalidate one repository's objects, or change one repository's cache layout without affecting the others. This is operational isolation, not a security boundary: repositories sharing a runner IAM role still share bucket access whatever their key prefixes are.What this changes
The default prefix is now derived from repository and platform identity:
GITHUB_REPOSITORY_ID) is preferred over theowner/nameslug so renaming or transferring a repository does not throw away its cache; the slug is the fallback when the id is absent, kept as two key components so thatfoo-bar/bazandfoo/bar-bazcannot collide.RUNNER_OS/RUNNER_ARCHgive the platform component, falling back to the running binary'sGOOS/GOARCH.v1is a layout version, so a future change to the key layout can roll out without reusing objects written under the old one.[a-z0-9._-], so an unexpected value cannot reshape the key.cache/sccacheroot is unchanged: the runner instance profile is granted S3 access oncache/*only (cloudformation/template.yaml), and the stack'sExpireCachelifecycle rule is keyed on the same prefix, so the scoping happens inside the namespace sccache already has.A new
sccache_prefixinput covers what the default cannot: repositories that should deliberately share a namespace.The value replaces the whole prefix rather than extending it; leading and trailing slashes are trimmed, so the example above resolves to
cache/sccache/shared-namespace. A value with no key components (empty, whitespace, only slashes) falls back to the default rather than writing to the bucket root, which is shared with the other RunsOn caches. Failing the step instead would be equally defensible; happy to switch it.Backward compatibility
The scoped default starts one cold cache per repository and platform on upgrade. Objects under the old prefix are not orphaned: the stack's
ExpireCacherule covers all ofcache/and expires entries afterS3CacheExpirationInDays(10 by default), which also bounds how much a cold start actually costs. The previous behaviour is one input away:Changing the default seemed worth that one-time cost, since a compiler cache refills quickly and the scoped layout is the one worth keeping long term. If the preference is to ship this opt-in first, or to hold the default change for a major version,
DefaultKeyPrefix()can return the flat root instead and the input still covers both policies.