Skip to content

fix: bound feature-mode inputs, clean allocation errors, record contract (0.4.0 prep) - #14

Merged
espetro merged 10 commits into
mainfrom
fix/large-image-preprocessing
Sep 21, 2026
Merged

espetro merged 10 commits into
mainfrom
fix/large-image-preprocessing

Conversation

@espetro

@espetro espetro commented Sep 21, 2026

Copy link
Copy Markdown
Owner

What this fixes

Findings from an external beta test of dinov2-cli (main at ce9429b):

  • G6 crash: a 3440x5601 PNG in feature mode built a ~246x401 patch grid (~98k tokens) and requested a ~222 GB Metal buffer, ending in Abort trap 6. Feature preprocessing preserved native resolution with no bound; attention memory is quadratic in tokens.
  • Unchecked allocation failures: ggml_gallocr_alloc_graph / backend buffer alloc return values were unchecked, so OOM aborted instead of erroring.
  • HF parity mismatch: HF AutoImageProcessor yields 256 tokens (224 crop) while the CLI produced 1444, causing cosine deltas ~0.046 and a rank flip. A preprocessing-grid mismatch, not numerical error.
  • Wrapper ergonomics: patches had no grid dimensions in the record; JSONL carried no per-image identity/order contract; D2EMB lacked grid dims.
  • Version: reported dinov2-cli 0.3.0 for the 0.4.0 beta.

Changes

  • Bounded feature preprocessing (default --preprocess bounded): shortest edge > 518 resizes to 518 preserving aspect, then true-ceil alignment to patch multiples in a single bicubic resample. hf mode replicates the HF recipe (shortest edge 256 + center crop 224); crop518 gives a fixed 37x37 grid for batching mixed aspect ratios. --no-resize opts back to native resolution, still capped.
  • --max-tokens N: hard cap on patch tokens (default 4 * ceil(518/patch)^2 = 5476 at patch 14, 0 disables), checked on prospective output dims before the expensive resize and before any graph construction. Exceeding it is a clean exit-1 error.
  • Allocation guards: model buffer alloc, ggml_gallocr_new, and ggml_gallocr_alloc_graph failures now print dims/token count with a --max-tokens/crop518 hint and exit 1, freeing resources.
  • Record contract: every JSON/JSONL record gains index (0-based input order) and grid {h, w}; record order equals input order; patches documented as row-major grid.h * grid.w * hidden.
  • D2EMB v2: 40-byte header adds grid_w/grid_h u32s.
  • Version 0.4.0; spec at docs/superpowers/specs/2026-09-22-large-image-and-wrapper-ergonomics-design.md.

Behavior changes

  • Feature mode now bounds inputs by default (native resolution via --no-resize); patch alignment is true-ceil, so already-aligned dims are unchanged (518 stays 518 -> 37 patches, previously 532/38).
  • D2EMB format bumped to v2 (explicitly an unstable preview format).
  • --no-resize only valid with --preprocess bounded; -c rejects --preprocess/--no-resize.

Verification

  • ctest 3/3 pass on debug, asan, ubsan presets (all with DINOV2_FATAL_WARNINGS).
  • clang-format 18.1.8 --dry-run --Werror clean; git diff --check clean.
  • Parity at this SHA, both checkpoints, 8/8 each (regular: cls 0.999911, pooled 0.999928; registers: cls 0.999984, pooled 0.999983). Tench (612x408) grid is unchanged at 44x30 = 1320 patches, so no parity artifacts needed a re-run; provenance fields updated.
  • HF comparison with --preprocess hf vs facebook/dinov2-small + AutoImageProcessor defaults on tench: 256 tokens both sides, cls cosine 0.996245, pooled cosine 0.996995 (residual = PIL antialiased bicubic vs Catmull-Rom, documented in docs/parity/README.md).
  • Release-build smoke on a generated 3440x5601 PNG: default mode exits 0 with grid h61 x w37 (n_patches 2257); --no-resize exits 1 in ~0.2s with the cap error; crop518 --batch 2 on 3440x5601 + 3440x4200 yields two records at grid 37x37 (index 0, 1).
  • Classifier output on tench is byte-identical to a pre-change build (cls, topk arrays); only the new grid/index keys were added.
  • D2EMB v2 header verified by struct decode (version 2, size 40, grid 44x30, payload length exact).

@espetro
espetro merged commit 7d8850a into main Sep 21, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant