11# Benchmarks
22
3- Numbers below are order-of-magnitude estimates, regenerated by
4- [ ` scripts/bench.sh ` ] ( ../scripts/bench.sh ) running across CI runners
5- (` ubuntu-latest ` , ` ubuntu-24.04-arm ` , ` macos-latest ` , ` windows-latest ` ).
6- The committed ` benchmark_results.txt ` ships as a placeholder; the
7- real per-platform tables are produced by the ` .github/workflows/bench.yml `
8- manual-trigger workflow once PR C lands.
9-
10- Headline: dinov2.cpp is ** 3x faster than native PyTorch inference on CPU**
11- for the small model and roughly ** 1.5-2x faster** for the rest, with
12- substantially lower memory use. (See "vs PyTorch" below for the
13- historical comparison numbers.)
3+ The table below contains the actual Ubuntu benchmark artifact produced by
4+ [ ` scripts/bench.sh ` ] ( ../scripts/bench.sh ) in GitHub Actions. Multi-platform
5+ measurements are deferred.
6+
7+ The benchmark ran on ` .github/workflows/bench.yml ` run
8+ [ 35629935462] ( https://github.com/espetro/dinov2.cpp/actions/runs/35629935462 )
9+ from ` feat/batched-inference ` at commit
10+ ` 50d5dee10da4db7b7820082da7a3ea0eaab5f346 ` . The workflow inputs were
11+ ` variant=small ` , ` repeats=5 ` , and ` threads=2 ` ; the uploaded artifact was
12+ ` bench-results-small-35629935462/benchmark_results.txt ` .
1413
1514## Forward pass wall time (CPU backend)
1615
1716Forward pass only -- the timed block excludes image load, preprocess,
1817and PCA visualization. Numbers in milliseconds (lower is better).
1918
20- | Platform | Arch | ggml | ViT-S | ViT-B | ViT-L | ViT-g | Peak RSS (ViT-S) |
21- | --------------------| --------| ------| ------:| ------:| ------:| ------:| ------------------|
22- | macos-arm64 (M1) | arm64 | cpu | 72 | 198 | 580 | 1980 | 110 MB |
23- | ubuntu-x64 (Xeon) | x86_64 | cpu | 165 | 410 | 1190 | 4020 | 110 MB |
24- | ubuntu-arm64 | arm64 | cpu | 140 | 360 | 1050 | 3550 | 110 MB |
25- | windows-x64 | x86_64 | cpu | 170 | 420 | 1210 | 4080 | 110 MB |
26-
27- Seed values from the plan file at
28- [ ` .agents/plans/2026-09-18-post-v0.2.0.md ` ] ( ../.agents/plans/2026-09-18-post-v0.2.0.md#3-benchmark-infrastructure--workstream-b-in-progress ) .
29- Treat as placeholders until bench.yml produces real numbers.
30-
31- Quantized GGUFs (q4_0 / q4_1 / q5_0 / q5_1 / q8_0) are pulled from
32- [ ` dinov2-cpp-core/<variant>-gguf ` ] ( https://huggingface.co/dinov2-cpp-core ) ;
33- the ` dinov2-cli ` binary loads them transparently and a per-quant bench
34- table can be produced by pre-staging those GGUFs under
35- ` models/<variant>/model.<q>.gguf ` and running
36- ` scripts/bench.sh --models small --repeats 5 ` with the desired quant
37- selected by which GGUF you pre-place.
19+ | Platform | Arch | ggml | Model | mean (ms) | stddev (ms) | min (ms) | max (ms) | Peak RSS |
20+ | -----------------| --------| ------| --------| ----------:| ------------:| ---------:| ---------:| ---------:|
21+ | ubuntu-latest | x86_64 | cpu | ViT-S | 212.8 | 9.1 | 204 | 225 | 104 MB |
22+
23+ This is the only current platform measurement. Other platform rows remain
24+ deferred, and no quantization-specific benchmark was run.
3825
3926## How to read the tables
4027
@@ -65,40 +52,23 @@ cold CPU cache) is in the loop.
6552 linearly with the patch grid.
6653- ** Warmup:** 1 discarded run (` --bench-warmup 1 ` ).
6754- ** Timed runs:** 5 (` --bench-runs 5 ` ).
68- - ** Threads:** ` min(4, hw_concurrency) ` -- the same default as
69- ` dino_params::n_threads ` .
70- - ** Batch:** 1 (` --batch ` exists for multi-image runs; the tables bench
55+ - ** Threads:** 2 OpenMP threads, supplied by the workflow input.
56+ - ** Batch:** 1 (` --batch ` exists for multi-image runs; this table benches
7157 one image per pass).
72- - ** Backend:** CPU only -- the macos-arm64 row does NOT exercise Metal.
73- - ** Quantization:** f16 baseline. Pre-quantized GGUFs can be benched by
74- pre-staging them under ` models/<variant>/model.<q>.gguf ` (download from
75- ` dinov2-cpp-core/<variant>-gguf ` ); ` dinov2-cli ` loads them transparently.
58+ - ** Backend:** CPU only.
59+ - ** Precision:** f16 GGUF.
7660- ** Build:** ` cmake --build build --target dinov2-cli ` in
7761 ` Release ` configuration (no ` DEBUG ` /` RelWithDebInfo ` ).
7862
79- The current ` benchmark_results.txt ` is regenerated by
80- ` scripts/bench.sh ` writing a markdown table per platform, then
81- ` --aggregate ` 'd into a single file with platform headers .
63+ The committed ` benchmark_results.txt ` is the downloaded artifact from the
64+ Ubuntu workflow run identified above. Additional platform measurements are
65+ deferred .
8266
8367## Hardware
8468
85- Four CI runners, all on the GitHub-hosted matrix:
86-
87- - ** macos-arm64 (M1)** -- Apple M1, 8 cores (4 perf + 4 efficiency), 16 GB.
88- Uses the Accelerate-backed BLAS in ggml's CPU backend. The fastest
89- per-core row in the matrix.
90- - ** ubuntu-x64 (Xeon)** -- ` ubuntu-latest ` , Intel Xeon Platinum 8275CL
91- @ 3.0 GHz, 2 vCPU visible to the runner (the rest are billed but not
92- scheduled). Single-threaded matmul performance is roughly 60% of the
93- M1's perf cores.
94- - ** ubuntu-arm64** -- ` ubuntu-24.04-arm ` , AWS Graviton3 (Neoverse V1),
95- 4 vCPU. The Neon SIMD backend is competitive with AVX2 on Skylake for
96- the matmul shapes ViT uses.
97- - ** windows-x64** -- ` windows-latest ` , Intel Xeon E5-2673 v4 @ 2.3 GHz,
98- 2 vCPU. MSVC-compiled binary; the slowest per-core row.
99-
100- CPU specs above are GitHub-hosted runner specifications as of 2025;
101- they may drift without notice.
69+ The evidence currently covers one GitHub-hosted ` ubuntu-latest ` runner. The
70+ runner's exact hardware may drift without notice; the artifact records the
71+ platform as ` linux-x86_64 ` .
10272
10373## Caveats
10474
@@ -121,24 +91,18 @@ they may drift without notice.
12191- ** No KV cache.** DINOv2 is a vision encoder with no autoregressive
12292 decode step -- there is no KV cache to populate. The "no KV cache"
12393 caveat in some other engines doesn't apply here.
124- - ** Metal/CUDA rows deferred.** The ` macos-arm64 ` row above is the CPU
125- backend, not Metal. Adding Metal is a one-line cmake flag change in
126- ` release.yml ` ; tracked in
127- [ the plan, section 1] ( ../.agents/plans/2026-09-18-post-v0.2.0.md#1-gpu-ci--deferred ) .
128- CUDA needs a larger GitHub-hosted runner (Tesla T4) which is not yet
129- configured for the org.
130- - ** CI runner noise.** The ` stddev ` column on regenerated runs is
131- expected to be 1-3% of the mean; >5% means the runner was loaded.
132- - ** Numbers are placeholders.** The committed ` benchmark_results.txt `
133- is seed data; bench.yml will overwrite it.
94+ - ** Multi-platform coverage is deferred.** This evidence contains only the
95+ Ubuntu CPU run described above. Metal and CUDA measurements are not included.
96+ - ** CI runner noise.** The ` stddev ` column is sensitive to load on the shared
97+ runner; the recorded run measured 9.1 ms of standard deviation.
13498- ** Embeddings flags are single-shot only.** ` --print-embeddings ` and
13599 ` --print-patch-tokens ` exist for output inspection; the bench path
136100 ignores them and reports timings only.
137101
138102## Reproduce locally
139103
140- The committed ` benchmark_results.txt ` will eventually be a CI artifact;
141- in the meantime, you can reproduce locally:
104+ The committed ` benchmark_results.txt ` records the Ubuntu CI artifact;
105+ you can reproduce the measurement locally with :
142106
143107``` bash
144108# Build the dinov2-cli binary first.
0 commit comments