Skip to content

Commit afd036b

Browse files
authored
Merge pull request #12 from espetro/feat/batched-inference
docs: recommend register-token models for feature workflows
2 parents c08cfda + 12f15a5 commit afd036b

5 files changed

Lines changed: 212 additions & 20 deletions

File tree

‎CONTRIBUTING.md‎

Lines changed: 12 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -64,16 +64,18 @@ Format violations fail the lint job.
6464

6565
The README's ready-to-download GGUFs live at `dinov2-cpp-core/<variant>-gguf` on Hugging Face. To convert a variant yourself from the upstream PyTorch checkpoints:
6666

67-
| Model | Source checkpoint (PyTorch) |
68-
|:-----:|:----------------------------|
69-
| small (no registers) | [facebook/dinov2-small-imagenet1k-1-layer](https://huggingface.co/facebook/dinov2-small-imagenet1k-1-layer) |
70-
| base (no registers) | [facebook/dinov2-base-imagenet1k-1-layer](https://huggingface.co/facebook/dinov2-base-imagenet1k-1-layer) |
71-
| large (no registers) | [facebook/dinov2-large-imagenet1k-1-layer](https://huggingface.co/facebook/dinov2-large-imagenet1k-1-layer) |
72-
| giant (no registers) | [facebook/dinov2-giant-imagenet1k-1-layer](https://huggingface.co/facebook/dinov2-giant-imagenet1k-1-layer) |
73-
| small (registers) | [facebook/dinov2-with-registers-small-imagenet1k-1-layer](https://huggingface.co/facebook/dinov2-with-registers-small-imagenet1k-1-layer) |
74-
| base (registers) | [facebook/dinov2-with-registers-base-imagenet1k-1-layer](https://huggingface.co/facebook/dinov2-with-registers-base-imagenet1k-1-layer) |
75-
| large (registers) | [facebook/dinov2-with-registers-large-imagenet1k-1-layer](https://huggingface.co/facebook/dinov2-with-registers-large-imagenet1k-1-layer) |
76-
| giant (registers) | [facebook/dinov2-with-registers-giant-imagenet1k-1-layer](https://huggingface.co/facebook/dinov2-with-registers-giant-imagenet1k-1-layer) |
67+
| Model | Source checkpoint (PyTorch) | Recommendation |
68+
|:-----:|:----------------------------|:---------------|
69+
| small (no registers) | [facebook/dinov2-small-imagenet1k-1-layer](https://huggingface.co/facebook/dinov2-small-imagenet1k-1-layer) | Baseline reproduction or task comparison |
70+
| base (no registers) | [facebook/dinov2-base-imagenet1k-1-layer](https://huggingface.co/facebook/dinov2-base-imagenet1k-1-layer) | Baseline reproduction or task comparison |
71+
| large (no registers) | [facebook/dinov2-large-imagenet1k-1-layer](https://huggingface.co/facebook/dinov2-large-imagenet1k-1-layer) | Baseline reproduction or task comparison |
72+
| giant (no registers) | [facebook/dinov2-giant-imagenet1k-1-layer](https://huggingface.co/facebook/dinov2-giant-imagenet1k-1-layer) | Baseline reproduction or task comparison |
73+
| small (registers) | [facebook/dinov2-with-registers-small-imagenet1k-1-layer](https://huggingface.co/facebook/dinov2-with-registers-small-imagenet1k-1-layer) | **Recommended for patch/dense features** |
74+
| base (registers) | [facebook/dinov2-with-registers-base-imagenet1k-1-layer](https://huggingface.co/facebook/dinov2-with-registers-base-imagenet1k-1-layer) | **Recommended for patch/dense features** |
75+
| large (registers) | [facebook/dinov2-with-registers-large-imagenet1k-1-layer](https://huggingface.co/facebook/dinov2-with-registers-large-imagenet1k-1-layer) | **Recommended for patch/dense features** |
76+
| giant (registers) | [facebook/dinov2-with-registers-giant-imagenet1k-1-layer](https://huggingface.co/facebook/dinov2-with-registers-giant-imagenet1k-1-layer) | **Recommended for patch/dense features** |
77+
78+
For the bounded rationale, see [Vision Transformers Need Registers](https://arxiv.org/abs/2309.16588) and the official [DINOv2 results](https://github.com/facebookresearch/dinov2/blob/main/README.md). Register-token checkpoints are recommended for patch and dense feature workflows, while no-register checkpoints remain useful for exact baseline reproduction and task-specific comparisons.
7779

7880
```bash
7981
python ./scripts/dinov2-to-gguf.py --model_name facebook/dinov2-small-imagenet1k-1-layer

‎README.md‎

Lines changed: 12 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -82,16 +82,18 @@ That's it. Up to **3x faster than PyTorch on CPU** with up to **4x less memory**
8282

8383
Ready-to-download f16 GGUF weights, published by CI to the [`dinov2-cpp-core`](https://huggingface.co/dinov2-cpp-core) Hugging Face profile:
8484

85-
| Model | GGUF download | Size |
86-
|:-----:|:--------------|-----:|
87-
| small (no registers) | [`dinov2-cpp-core/dinov2-small-gguf`](https://huggingface.co/dinov2-cpp-core/dinov2-small-gguf) | ~50 MB |
88-
| base (no registers) | [`dinov2-cpp-core/dinov2-base-gguf`](https://huggingface.co/dinov2-cpp-core/dinov2-base-gguf) | ~180 MB |
89-
| large (no registers) | [`dinov2-cpp-core/dinov2-large-gguf`](https://huggingface.co/dinov2-cpp-core/dinov2-large-gguf) | ~620 MB |
90-
| giant (no registers) | [`dinov2-cpp-core/dinov2-giant-gguf`](https://huggingface.co/dinov2-cpp-core/dinov2-giant-gguf) | ~2.2 GB |
91-
| small (registers) | [`dinov2-cpp-core/dinov2-with-registers-small-gguf`](https://huggingface.co/dinov2-cpp-core/dinov2-with-registers-small-gguf) | ~50 MB |
92-
| base (registers) | [`dinov2-cpp-core/dinov2-with-registers-base-gguf`](https://huggingface.co/dinov2-cpp-core/dinov2-with-registers-base-gguf) | ~180 MB |
93-
| large (registers) | [`dinov2-cpp-core/dinov2-with-registers-large-gguf`](https://huggingface.co/dinov2-cpp-core/dinov2-with-registers-large-gguf) | ~620 MB |
94-
| giant (registers) | [`dinov2-cpp-core/dinov2-with-registers-giant-gguf`](https://huggingface.co/dinov2-cpp-core/dinov2-with-registers-giant-gguf) | ~2.2 GB |
85+
| Model | GGUF download | Size | Recommendation |
86+
|:-----:|:--------------|-----:|:---------------|
87+
| small (no registers) | [`dinov2-cpp-core/dinov2-small-gguf`](https://huggingface.co/dinov2-cpp-core/dinov2-small-gguf) | ~50 MB | Baseline reproduction or task comparison |
88+
| base (no registers) | [`dinov2-cpp-core/dinov2-base-gguf`](https://huggingface.co/dinov2-cpp-core/dinov2-base-gguf) | ~180 MB | Baseline reproduction or task comparison |
89+
| large (no registers) | [`dinov2-cpp-core/dinov2-large-gguf`](https://huggingface.co/dinov2-cpp-core/dinov2-large-gguf) | ~620 MB | Baseline reproduction or task comparison |
90+
| giant (no registers) | [`dinov2-cpp-core/dinov2-giant-gguf`](https://huggingface.co/dinov2-cpp-core/dinov2-giant-gguf) | ~2.2 GB | Baseline reproduction or task comparison |
91+
| small (registers) | [`dinov2-cpp-core/dinov2-with-registers-small-gguf`](https://huggingface.co/dinov2-cpp-core/dinov2-with-registers-small-gguf) | ~50 MB | **Recommended for patch/dense features** |
92+
| base (registers) | [`dinov2-cpp-core/dinov2-with-registers-base-gguf`](https://huggingface.co/dinov2-cpp-core/dinov2-with-registers-base-gguf) | ~180 MB | **Recommended for patch/dense features** |
93+
| large (registers) | [`dinov2-cpp-core/dinov2-with-registers-large-gguf`](https://huggingface.co/dinov2-cpp-core/dinov2-with-registers-large-gguf) | ~620 MB | **Recommended for patch/dense features** |
94+
| giant (registers) | [`dinov2-cpp-core/dinov2-with-registers-giant-gguf`](https://huggingface.co/dinov2-cpp-core/dinov2-with-registers-giant-gguf) | ~2.2 GB | **Recommended for patch/dense features** |
95+
96+
For patch and dense feature workflows, the register-token variants are the recommended starting point. In the settings studied in [Vision Transformers Need Registers](https://arxiv.org/abs/2309.16588), register tokens reduce high-norm patch-token artifacts and produce smoother local feature and attention maps. Use them with `--print-patch-tokens`, PCA, dense features, and object discovery. This is not a universal accuracy claim: the official [DINOv2 results](https://github.com/facebookresearch/dinov2/blob/main/README.md) show classification and retrieval results that depend on the task and model size. Choose the no-register variant for exact baseline reproduction or task-specific classification and retrieval comparisons.
9597

9698
## Documentation
9799

‎docs/benchmarks.md‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -167,6 +167,31 @@ For JSON output, pass `--bench-json` to the binary directly:
167167
--bench --bench-runs 5 --bench-warmup 1 --bench-json
168168
```
169169

170+
## Register and no-register variants
171+
172+
Both variants appear in benchmark tables because they are separate published
173+
checkpoints with different representations and legitimate use cases. Register
174+
tokens are especially relevant to patch and dense feature inspection: the
175+
register-token paper reports fewer high-norm patch-token artifacts and smoother
176+
local feature and attention maps. No-register variants remain necessary for
177+
exact baseline reproduction and fair comparisons with systems or published
178+
results built from those checkpoints.
179+
180+
Runtime rows should stay comparable by model size, precision, backend, input,
181+
and measurement method. Keep register and no-register timings in explicitly
182+
labeled rows or headings rather than combining them. Existing speed and memory
183+
tables are implementation measurements and do not establish downstream
184+
representation quality.
185+
186+
The strongest consistent rationale for registers is dense-feature quality and
187+
local feature-map behavior. Classification and retrieval results in the official
188+
[DINOv2 results](https://github.com/facebookresearch/dinov2/blob/main/README.md)
189+
are mixed by task and model size, so they do not support a universal ranking.
190+
The primary [Vision Transformers Need Registers](https://arxiv.org/abs/2309.16588)
191+
paper provides the motivation and reports feature-map effects and selected
192+
downstream evaluations. Preserve the model variant, task, dataset, evaluation
193+
protocol, and model size when interpreting those results.
194+
170195
## DINOv2 vs PyTorch (historical)
171196

172197
This is the qualitative comparison that motivated the project. Numbers

‎docs/cli.md‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,22 @@ huggingface-cli download dinov2-cpp-core/dinov2-small-gguf --local-dir models
2828
The binary loads any ggml-supported quantization transparently (f16,
2929
q4_0 through q8_0): point `-m` at whichever GGUF you have.
3030

31+
### Choosing a checkpoint
32+
33+
For patch and dense feature work, start with the register-token small model:
34+
35+
```bash
36+
huggingface-cli download dinov2-cpp-core/dinov2-with-registers-small-gguf --local-dir models
37+
```
38+
39+
The same `dinov2-with-registers-{size}-gguf` naming pattern is available for
40+
`base`, `large`, and `giant`. Use the matching no-register repository when you
41+
need an exact baseline checkpoint or a task-specific classification or retrieval
42+
comparison. The register-token recommendation is based on the feature behavior
43+
reported in [Vision Transformers Need Registers](https://arxiv.org/abs/2309.16588)
44+
and should not be read as a universal accuracy ranking. For classification and
45+
retrieval context, see the official [DINOv2 results](https://github.com/facebookresearch/dinov2/blob/main/README.md).
46+
3147
## Flags
3248

3349
Adapted from `dinov2-cli --help` (v0.3.0); see `--help` for the exact
@@ -88,6 +104,14 @@ the PNG in the same run. `--print-patch-tokens` only affects the
88104
embeddings JSON: on its own it selects no output mode and the run fails
89105
the nothing-to-do guard.
90106

107+
For patch-token inspection, PCA maps, dense features, or object discovery,
108+
prefer a `with-registers` checkpoint. In the settings studied in the
109+
register-token paper, registers reduce high-norm patch-token artifacts and
110+
smooth local feature and attention maps. This is a feature-quality and
111+
visual-behavior recommendation, not a universal classification or retrieval ranking.
112+
Use the matching no-register checkpoint for exact baseline reproduction or a
113+
task-specific comparison.
114+
91115
## Batch inference
92116

93117
Multiple inputs are given by repeating `-i` and/or comma-separating

0 commit comments

Comments
 (0)