Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ graph TD
A["dinov2-cli (dinov2-cli.cpp)<br/>CLI shell: argv parsing, image load,<br/>graph build, output formatting"]
B["dinov2.h<br/>public API: structs, signatures,<br/>IMAGENET defaults"]
C["dinov2.cpp<br/>encoder graph:<br/>attn, mlp, swiglu_ffn, build_graph,<br/>dino_predict"]
D["src/image.h + src/image.cpp<br/>stb wrappers: load_image,<br/>dino_preprocess, dino_classify_preprocess"]
D["src/image.h + src/image.cpp<br/>stb wrappers: load_image,<br/>dino_preprocess, dino_classify_preprocess,<br/>preprocess_resize_crop (bounded/hf/crop518)"]
E["ggml/ submodule<br/>tensor library + CPU backend"]

A --> B
Expand Down Expand Up @@ -87,7 +87,7 @@ Abridged table:
| Path | Role | One-line purpose |
|:-----|:-----|:-----------------|
| `dinov2.h`, `dinov2.cpp` | Core | Public API + encoder graph. |
| `src/image.{h,cpp}` | Library | stb-backed image load + `dino_preprocess`. |
| `src/image.{h,cpp}` | Library | stb-backed image load + `dino_preprocess` / feature-mode `--preprocess` recipes (bounded 518 bound, hf, crop518). |
| `dinov2-cli.cpp` | CLI | `main`: arg parsing + bench loop. |
| `tests/test_{image,dinov2}.cpp` | Test | doctest pure-function coverage. |
| `scripts/{bench.sh,dinov2-to-gguf.py,publish-gguf.sh}` | Tool | Bench sweep + PyTorch→GGUF + HF upload. |
Expand Down
29 changes: 29 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,35 @@ Hand-edit only the lines **above** the `<!-- git-cliff: end of header -->`
marker; everything below is regenerated.

<!-- git-cliff: end of header -->
## [Unreleased]

### Changed
- Feature-mode preprocessing now bounds the shortest edge to 518 px by
default (`--preprocess bounded`) and aligns dimensions by true ceil to
patch multiples (a dimension already at a multiple is unchanged).
`--no-resize` restores native resolution, still subject to
`--max-tokens`.
- The embeddings JSON record gains `index` (0-based input order) and
`grid` (`{"h","w"}` patch-grid dims); `patches` stays flat row-major
`h*w*hidden`.
- The D2EMB preview header is version 2, 40 bytes, adding `grid_w` and
`grid_h` fields. The format remains intentionally unstable.
- Version bumped to 0.4.0 (`--version` reports `dinov2-cli 0.4.0`).

### Added
- `--preprocess {bounded,hf,crop518}` feature-mode selector: `hf`
mirrors the HF AutoImageProcessor recipe (256 + 224 crop); `crop518`
gives a fixed 518x518 grid for batching mixed aspect ratios.
- `--max-tokens N` hard cap on patch tokens per image (default
`4*(518/patch)^2`, 0 disables), enforced before graph construction.
- `scripts/check_jsonl_contract.py`: stdlib validator for the JSONL
record contract.

### Fixed
- Graph and model buffer allocation failures now print a diagnostic and
exit 1 instead of aborting (e.g. the ~222 GB Metal request from a
3440x5601 feature-mode input).

## [0.3.0] - 2026-09-18

### Changed
Expand Down
2 changes: 1 addition & 1 deletion CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
cmake_minimum_required(VERSION 3.22.1)
project(dinov2
VERSION 0.3.0
VERSION 0.4.0
DESCRIPTION "Inference DINOv2 in plain C/C++ with ggml"
HOMEPAGE_URL "https://github.com/lavaman131/dinov2.cpp")

Expand Down
10 changes: 9 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,9 +58,17 @@ hf download dinov2-cpp-core/dinov2-small-gguf --local-dir models

Emits `cls`/`pooled` vectors on stdout as one JSON object.

Feature mode bounds the input's shortest edge to 518 px by default
(`--preprocess bounded`) so attention memory stays predictable on large
images; `--preprocess hf` mirrors the HF `AutoImageProcessor` recipe
(256 shortest edge + 224 center crop) for like-for-like comparisons, and
`--preprocess crop518` fixes every input at 518x518 for batching mixed
aspect ratios. `--no-resize` keeps native resolution, and `--max-tokens N`
caps the patch-token count per image (default `4*(518/patch)^2`).

**Preview binary embeddings:** for high-throughput consumers, use
`--embeddings-binary -o output.d2e`. This deliberately unstable preview writes
an explicit 32-byte little-endian header followed by float32 `cls`, `pooled`,
an explicit 40-byte little-endian header followed by float32 `cls`, `pooled`,
and optional row-major patch vectors. It writes no binary bytes to stdout.
With multiple inputs, `-o` is a directory containing indexed `.d2e` files.
The format may change without compatibility guarantees and is not a standard.
Expand Down
51 changes: 41 additions & 10 deletions dinov2-cli.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -103,12 +103,16 @@ static std::string binary_out_path(const std::string &out_path, const std::strin
// cls (both modes), pooled (feature mode), topk (classify mode),
// patches (feature mode + --print-patch-tokens). With multiple inputs the
// caller prints one such line per image (JSONL).
static void print_embeddings_json(const dino_params &params, const dino_model &model, const ImageF &img_f,
static void print_embeddings_json(const dino_params &params, const dino_model &model, const ImageF &img_f, size_t index,
const std::string &image_path, const dino_output &output) {
const int n_patches = (img_f.ny / model.hparams.patch_size) * (img_f.nx / model.hparams.patch_size);
fprintf(stdout, "{\"model\":\"%s\",\"image\":\"%s\",\"n_patches\":%d,\"hidden\":%u",
json_escape(model_label_from_path(params.model)).c_str(), json_escape(image_path).c_str(), n_patches,
model.hparams.hidden_size);
const int grid_w = img_f.nx / model.hparams.patch_size;
const int grid_h = img_f.ny / model.hparams.patch_size;
const int n_patches = grid_h * grid_w;
fprintf(stdout,
"{\"model\":\"%s\",\"index\":%zu,\"image\":\"%s\",\"n_patches\":%d,"
"\"grid\":{\"h\":%d,\"w\":%d},\"hidden\":%u",
json_escape(model_label_from_path(params.model)).c_str(), index, json_escape(image_path).c_str(), n_patches,
grid_h, grid_w, model.hparams.hidden_size);
if (output.cls_token) {
fprintf(stdout, ",\"cls\":[");
print_float_array(*output.cls_token);
Expand Down Expand Up @@ -224,10 +228,30 @@ int main(int argc, char **argv) {
// preprocess every input; classify mode always yields 224x224 crops
std::vector<ImageF> imgs_f;
imgs_f.reserve(imgs.size());
for (const Image &img : imgs) {
ImageF img_f =
params.classify ? dino_classify_preprocess(img, model.hparams) : dino_preprocess(img, model.hparams);
fprintf(stderr, "%s: preprocessed image (%d x %d)\n", __func__, img_f.nx, img_f.ny);
const int64_t token_limit =
params.max_tokens >= 0 ? params.max_tokens : (int64_t)dino_default_max_tokens(model.hparams.patch_size);
for (size_t i = 0; i < imgs.size(); ++i) {
const Image &img = imgs[i];
// hard cap on patch tokens, applied on the prospective preprocessed
// size so oversize inputs fail before the (expensive) resize and
// before any graph is constructed
if (token_limit > 0 && !params.classify) {
const ImgSize out_size = dino_feature_output_size(img, model.hparams, params);
const int64_t n_patches = (int64_t)(out_size.height / (int)model.hparams.patch_size) *
(out_size.width / (int)model.hparams.patch_size);
if (n_patches > token_limit) {
fprintf(stderr,
"error: image '%s' yields %lld patch tokens after preprocessing (limit %lld). "
"Use a smaller input, --preprocess crop518, or raise --max-tokens.\n",
params.fnames_inp[i].c_str(), (long long)n_patches, (long long)token_limit);
free_model();
return 1;
}
}
ImageF img_f = params.classify ? dino_classify_preprocess(img, model.hparams)
: dino_feature_preprocess(img, model.hparams, params);
fprintf(stderr, "%s: preprocessed image '%s' (%d x %d)\n", __func__, params.fnames_inp[i].c_str(), img_f.nx,
img_f.ny);
imgs_f.push_back(std::move(img_f));
}
std::vector<Image>().swap(imgs);
Expand Down Expand Up @@ -268,6 +292,11 @@ int main(int argc, char **argv) {
{
ggml_backend_synchronize(model.backend);
ggml_gallocr_t allocr = ggml_gallocr_new(ggml_backend_get_default_buffer_type(model.backend));
if (!allocr) {
fprintf(stderr, "%s: failed to create graph allocator\n", __func__);
free_model();
return 1;
}

if (params.bench_repeats == 0) {
// Single-shot path: run the inputs through dino_predict in chunks
Expand All @@ -293,7 +322,7 @@ int main(int argc, char **argv) {
const std::string &image_path = params.fnames_inp[idx];

if (params.print_embeddings) {
print_embeddings_json(params, model, imgs_f[idx], image_path, output);
print_embeddings_json(params, model, imgs_f[idx], idx, image_path, output);
}
if (params.embeddings_binary) {
const std::string out_path =
Expand All @@ -302,6 +331,8 @@ int main(int argc, char **argv) {
if (!write_embeddings_binary(out_path, output, model.hparams.hidden_size,
static_cast<uint32_t>((imgs_f[idx].ny / model.hparams.patch_size) *
(imgs_f[idx].nx / model.hparams.patch_size)),
static_cast<uint32_t>(imgs_f[idx].nx / model.hparams.patch_size),
static_cast<uint32_t>(imgs_f[idx].ny / model.hparams.patch_size),
params.print_patch_tokens, params.l2_normalize, error)) {
fprintf(stderr, "%s: failed to write binary embeddings '%s': %s\n", __func__,
out_path.c_str(), error.c_str());
Expand Down
Loading
Loading