|
| 1 | +# Launch blog post outline |
| 2 | + |
| 3 | +Working title options: |
| 4 | + |
| 5 | +- "dinov2.cpp: DINOv2 embeddings without the Python stack" |
| 6 | +- "Running DINOv2 like llama.cpp: one binary, GGUF weights, JSON out" |
| 7 | + |
| 8 | +Positioning one-liner (use verbatim or near-verbatim): |
| 9 | + |
| 10 | +> llama.cpp for vision encoders: DINOv2 on ggml, one binary, GGUF weights, |
| 11 | +> embeddings as JSON on stdout. |
| 12 | +
|
| 13 | +The analogy holds mechanically (ggml backend, GGUF format, single-binary |
| 14 | +CLI, prebuilt releases, HF-hosted weights). Do not overextend it into |
| 15 | +performance or maturity claims. |
| 16 | + |
| 17 | +## 1. Hook |
| 18 | + |
| 19 | +Draft: |
| 20 | + |
| 21 | +> DINOv2 is the backbone under a lot of vision pipelines: retrieval, |
| 22 | +> dedup, clustering, dense features. Running it today means PyTorch, |
| 23 | +> transformers, and a few hundred MB of Python environment before you get |
| 24 | +> a single vector out. I wanted `model.gguf` plus one binary that prints |
| 25 | +> an embedding. dinov2.cpp is that: the DINOv2 encoder ported to C++ on |
| 26 | +> ggml, with prebuilt binaries for macOS, Linux, and Windows, and the |
| 27 | +> weights already converted on Hugging Face. |
| 28 | +
|
| 29 | +## 2. What it is |
| 30 | + |
| 31 | +- C++20 port of DINOv2 (ViT-S/B/L/g, patch 14) on ggml. Vendored stb for |
| 32 | + image I/O, no other runtime dependencies (`ARCHITECTURE.md`). |
| 33 | +- One CLI (`dinov2-cli`) plus a C API (`include/dinov2.h`) for embedding. |
| 34 | +- Outputs: ImageNet top-k (`-c`), `cls`/`pooled`/patch embeddings as JSON |
| 35 | + (`--print-embeddings`), PCA feature map PNG (`-o`), bench mode |
| 36 | + (`--bench-json`). |
| 37 | +- GGUF weights for all 8 classifier checkpoints on |
| 38 | + `dinov2-cpp-core` HF, f16, published monthly by CI |
| 39 | + (`docs/hf-publishing.md`). Real sizes: 46.9 MB (small), 178.7 MB (base), |
| 40 | + 616.5 MB (large), 2.29 GB (giant), measured via `x-linked-size` on |
| 41 | + 2026-09-23. |
| 42 | +- Install story: download a ~1 MB release archive (mac 1,156,752 B, |
| 43 | + ubuntu x64 945,186 B, ubuntu arm64 895,818 B, win 1,120,200 B per |
| 44 | + `gh release view v0.4.0`), `hf download` a weight, run. |
| 45 | + |
| 46 | +## 3. The demo (centerpiece): local photo dedup |
| 47 | + |
| 48 | +`examples/dedup/dedup.py` is the proof that embeddings-as-JSON is a real |
| 49 | +interface, not a demo stub. |
| 50 | + |
| 51 | +- Stdlib-only Python (~30 lines of logic): shells out to |
| 52 | + `dinov2-cli --print-embeddings`, takes `pooled`, pairwise cosine, |
| 53 | + single-linkage clustering, writes an HTML review page |
| 54 | + (`examples/dedup/README.md`). |
| 55 | +- Suggested narrative: point it at a real photo dump, show the HTML |
| 56 | + clusters, then show the three lines of code that matter (subprocess, |
| 57 | + cosine, union-find). |
| 58 | +- Talking point: threshold 0.99 catches re-encodes and resizes, 0.95 |
| 59 | + catches same-scene shots. This is embedding-similarity behavior, tunable, |
| 60 | + not magic. |
| 61 | +- Honest limit to name in passing: pairwise comparison is O(n^2) in the |
| 62 | + example script; fine for a few thousand images, real indexes (FAISS, |
| 63 | + sqlite-vec) beyond that. |
| 64 | +- Secondary demo if Pages is enabled: the wasm build (~1 MB |
| 65 | + `dinov2-wasm.wasm`, 1,048,008 B locally) runs the encoder in-browser and |
| 66 | + shows CLS cosine plus a patch heatmap (`docs/wasm.md`). |
| 67 | + |
| 68 | +## 4. Does it match the reference? (parity evidence) |
| 69 | + |
| 70 | +- Nightly CI job (`.github/workflows/parity.yml`, cron `17 4 * * *`) runs |
| 71 | + `scripts/parity_check.py` against the HF reference for both small |
| 72 | + checkpoints; a drift fails the job. |
| 73 | +- Committed runs (`docs/parity/`): regular small CLS cosine 0.999911, |
| 74 | + pooled 0.999928; registers small CLS 0.999984, pooled 0.999984; patch |
| 75 | + mean cosine 0.9996/0.9998. Gates: cls/pooled >= 0.999, patches >= 0.99. |
| 76 | +- Classification matches too: same top-1 label ("tench"), probability diff |
| 77 | + 0.000039 on the recorded run. |
| 78 | +- Frame it as: the numbers are published, the checker is in the repo, the |
| 79 | + gate runs nightly. Not "trust me". |
| 80 | + |
| 81 | +## 5. Performance |
| 82 | + |
| 83 | +- Committed evidence is narrow on purpose: ViT-S f16, CPU, ubuntu-latest, |
| 84 | + 12 threads: 255.0 ms mean, stddev 9.1, 104 MB peak RSS |
| 85 | + (`benchmark_results.txt`, `docs/benchmarks.md`). |
| 86 | +- Historical i9-14900HX table exists (small 62 ms vs PyTorch 181 ms) but is |
| 87 | + labeled non-reproducible context. Mention it as history, not a claim. |
| 88 | +- wasm estimate is explicitly an estimate: ~4-10 s/image for ViT-S at 518 |
| 89 | + px, single-threaded (`docs/wasm.md`). |
| 90 | + |
| 91 | +## 6. Caveats (draft below) |
| 92 | + |
| 93 | +## 7. How it works (optional technical section) |
| 94 | + |
| 95 | +- ggml graph, Catmull-Rom resize, position-embedding interpolation for |
| 96 | + arbitrary grids, `--preprocess bounded|hf|crop518`, `--max-tokens` guard |
| 97 | + against quadratic attention memory on huge inputs. |
| 98 | +- Batch inference via repeated `-i` and `--batch`; per-image outputs |
| 99 | + identical to single runs. |
| 100 | + |
| 101 | +## 8. CTA |
| 102 | + |
| 103 | +- Try it: `hf download dinov2-cpp-core/dinov2-small-gguf --local-dir models` |
| 104 | + plus a release binary; three commands total. |
| 105 | +- File issues on parity gaps, missing formats, quantization results. |
| 106 | +- Star/watch if you want the backbone-only checkpoints and Windows ARM |
| 107 | + binaries when they land. |
| 108 | + |
| 109 | +## Caveats section (suggested draft) |
| 110 | + |
| 111 | +> What this is not, and where the edges are: |
| 112 | +> |
| 113 | +> - The committed benchmark is one row: ViT-S, f16, CPU, Ubuntu. There is |
| 114 | +> no GPU measurement, no quantization comparison, and no base/large/giant |
| 115 | +> row yet. The 3x-vs-PyTorch table in the docs is an older laptop |
| 116 | +> measurement kept as history, not CI evidence. |
| 117 | +> - Parity is recorded for the two small checkpoints on one image and one |
| 118 | +> environment. It is strong evidence, not a proof for every checkpoint, |
| 119 | +> image, and platform; the nightly job exists because drift is possible. |
| 120 | +> - Feature-mode preprocessing is not bit-identical to HF's |
| 121 | +> `AutoImageProcessor`: the CLI uses Catmull-Rom, HF uses antialiased |
| 122 | +> bicubic. Under `--preprocess hf` the measured residual on the test |
| 123 | +> image is CLS cosine ~0.996. Use the published GGUFs with the default |
| 124 | +> `bounded` recipe unless you specifically need token-for-token HF |
| 125 | +> comparison. |
| 126 | +> - `-fa` (flash attention) is documented as faster but less accurate; a |
| 127 | +> K/V padding bias was fixed in the audit pass, but avoid `-fa` when |
| 128 | +> comparing against HF outputs. |
| 129 | +> - Image decode is whatever stb_image supports (JPEG, PNG, BMP, TGA, GIF, |
| 130 | +> PSD, HDR, PNM). No WebP, HEIC, or AVIF. |
| 131 | +> - The C API is explicitly unstable for 0.4.x; the CLI JSONL schema is the |
| 132 | +> stable surface (`docs/stability.md`). The `.d2e` binary format is a |
| 133 | +> preview with no compatibility promise. |
| 134 | +> - Weights are the ImageNet classifier checkpoints only; backbone-only |
| 135 | +> DINOv2 GGUFs are supported by the code but not published yet. |
| 136 | +
|
| 137 | +## Distribution notes |
| 138 | + |
| 139 | +- Cross-post targets: personal blog, Show HN (see show-hn-draft.md), maybe |
| 140 | + r/LocalLLaMA (fits the local-inference audience). |
| 141 | +- The parity and benchmarks pages are the credibility backbone; link them |
| 142 | + early, not just at the end. |
0 commit comments