Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,23 @@ budget limits attempts and adds at most seven seconds of backoff; it does not
impose a new transfer timeout on large repositories. LFS and local upload-pack
operations are outside this retry policy.

If a client `want`s a commit by SHA that the mirror never captured - typically a
GitHub pull-request merge commit, which lives only under the unadvertised
`refs/pull/<n>/merge` and so is what `actions/checkout` fetches on a PR build - the
proxy fetches that SHA from upstream on demand and serves it, instead of failing
with `not our ref`. This relies on the upstream serving arbitrary SHAs (GitHub's
`uploadpack.allowAnySHA1InWant`, which is on); an upstream that refuses simply
leaves the request to fail as it would without the proxy. Ordinary branch/tag
clones are unaffected - they pay only a cheap local object check, never an extra
upstream call. Each fetched SHA is pinned under a reserved ref so the mirror can
keep serving it; `--max-wants` bounds how many such pins a mirror retains, pruning
the oldest so they cannot accumulate without bound (like the mirror-level LRU, but
scoped to a single mirror's pins). If a single request needs more new SHAs than
`--max-wants`, only that many are fetched and the excess wants are left to fail with
`not our ref` (the proxy logs a `capping want-by-sha fetch` warning); such a request
would immediately have its pins pruned anyway, so raise `--max-wants` if you expect
that many unadvertised objects at once.

### git-LFS

LFS objects use a different HTTP API from the git protocol, so they are cached
Expand Down Expand Up @@ -190,6 +207,7 @@ Every flag has an environment-variable equivalent.
| `--max-concurrent-requests` | `GITCACHEPROXY_MAX_CONCURRENT_REQUESTS` | `64` | Max concurrent in-flight requests; excess queue (`0` = unlimited) |
| `--max-decoded-body-mb` | `GITCACHEPROXY_MAX_DECODED_BODY_MB` | `512` | Cap on a decoded upload-pack request body, in MiB (bounds memory / gzip bombs) |
| `--cache-max-mb` | `GITCACHEPROXY_CACHE_MAX_MB` | `0` | Cap on total on-disk mirror cache, in MiB; evicts least-recently-used idle mirrors when exceeded (`0` = unlimited, no eviction) |
| `--max-wants` | `GITCACHEPROXY_MAX_WANTS` | `100` | Cap on want-by-SHA pins retained per mirror; oldest pruned beyond it (`0` = unlimited) |
| `--git-binary` | `GITCACHEPROXY_GIT_BINARY` | `git` | Path to git |
| `--big-file-threshold` | `GITCACHEPROXY_BIG_FILE_THRESHOLD` | `8m` | Stream blobs larger than this to disk during upstream clone/fetch instead of holding them in memory, bounding `index-pack` RSS so one very large repo can't OOM the proxy |

Expand Down
8 changes: 8 additions & 0 deletions src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,14 @@ pub struct Config {
#[arg(long, env = "GITCACHEPROXY_CACHE_MAX_MB", default_value_t = 0)]
pub cache_max_mb: u64,

/// Maximum number of want-by-SHA pins a mirror retains. A client may `want` a
/// commit that lives only under an unadvertised upstream ref (e.g. a GitHub PR
/// merge commit); the proxy fetches it on demand and pins it so it can keep
/// serving it. Beyond this cap the oldest pins are pruned so they cannot
/// accumulate without bound. `0` = unlimited.
#[arg(long, env = "GITCACHEPROXY_MAX_WANTS", default_value_t = 100)]
pub max_wants: usize,

/// Path to the git binary.
#[arg(long, env = "GITCACHEPROXY_GIT_BINARY", default_value = "git")]
pub git_binary: String,
Expand Down
1 change: 1 addition & 0 deletions src/evict.rs
Original file line number Diff line number Diff line change
Expand Up @@ -612,6 +612,7 @@ mod tests {
upstream_auth_header: None,
big_file_threshold: "8m".into(),
fetch_ttl: Duration::from_secs(10),
max_wants: 100,
}
}

Expand Down
Loading
Loading