Skip to content

Scope the default sccache S3 prefix per repository and make it configurable - #58

Open
garysassano wants to merge 2 commits into
runs-on:mainfrom
garysassano:feat/sccache-prefix-v2
Open

Scope the default sccache S3 prefix per repository and make it configurable#58
garysassano wants to merge 2 commits into
runs-on:mainfrom
garysassano:feat/sccache-prefix-v2

Conversation

@garysassano

Copy link
Copy Markdown

Problem

With sccache: s3, every repository on a stack gets the same flat prefix:

SCCACHE_S3_KEY_PREFIX=cache/sccache

sccache content keys keep compilation results from colliding, so this is not a correctness problem, but a single stack-wide namespace leaves no way to attribute cache growth and cost to a repository, expire or invalidate one repository's objects, or change one repository's cache layout without affecting the others. This is operational isolation, not a security boundary: repositories sharing a runner IAM role still share bucket access whatever their key prefixes are.

What this changes

The default prefix is now derived from repository and platform identity:

cache/sccache/<repository id>/<runner os>-<runner arch>/v1
  • The numeric repository id (GITHUB_REPOSITORY_ID) is preferred over the owner/name slug so renaming or transferring a repository does not throw away its cache; the slug is the fallback when the id is absent, kept as two key components so that foo-bar/baz and foo/bar-baz cannot collide.
  • RUNNER_OS/RUNNER_ARCH give the platform component, falling back to the running binary's GOOS/GOARCH.
  • The trailing v1 is a layout version, so a future change to the key layout can roll out without reusing objects written under the old one.
  • Scope components are lowercased and restricted to [a-z0-9._-], so an unexpected value cannot reshape the key.
  • The cache/sccache root is unchanged: the runner instance profile is granted S3 access on cache/* only (cloudformation/template.yaml), and the stack's ExpireCache lifecycle rule is keyed on the same prefix, so the scoping happens inside the namespace sccache already has.

A new sccache_prefix input covers what the default cannot: repositories that should deliberately share a namespace.

- uses: runs-on/action@v2
  with:
    sccache: s3
    sccache_prefix: /cache/sccache/shared-namespace/

The value replaces the whole prefix rather than extending it; leading and trailing slashes are trimmed, so the example above resolves to cache/sccache/shared-namespace. A value with no key components (empty, whitespace, only slashes) falls back to the default rather than writing to the bucket root, which is shared with the other RunsOn caches. Failing the step instead would be equally defensible; happy to switch it.

Backward compatibility

The scoped default starts one cold cache per repository and platform on upgrade. Objects under the old prefix are not orphaned: the stack's ExpireCache rule covers all of cache/ and expires entries after S3CacheExpirationInDays (10 by default), which also bounds how much a cold start actually costs. The previous behaviour is one input away:

with:
  sccache: s3
  sccache_prefix: cache/sccache

Changing the default seemed worth that one-time cost, since a compiler cache refills quickly and the scoped layout is the one worth keeping long term. If the preference is to ship this opt-in first, or to hold the default change for a major version, DefaultKeyPrefix() can return the flat root instead and the input still covers both policies.

…urable

With `sccache: s3`, every repository on a stack shared one flat prefix
(`cache/sccache`). Content keys keep compilation results from colliding, so this
was not a correctness problem, but a single stack-wide namespace left no way to
attribute cache growth to a repository, expire one repository's objects, or
change one repository's layout without affecting the others.

The default is now derived from repository and platform identity:

    cache/sccache/<repository id>/<runner os>-<runner arch>/v1

The numeric repository id is preferred over the owner/name slug so a rename or
transfer does not discard the cache; the slug is the fallback when the id is
absent, kept as two key components so `foo-bar/baz` and `foo/bar-baz` cannot
collide. RUNNER_OS/RUNNER_ARCH give the platform component, falling back to the
binary's GOOS/GOARCH. The trailing `v1` is a layout version, so a future change
can roll out without reusing objects written under the old scheme. Scope
components are lowercased and restricted to [a-z0-9._-] so an unexpected value
cannot reshape the key.

An explicit prefix is still honoured, and is rejected when it cannot be exported
or used as a key.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant