Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
121 changes: 121 additions & 0 deletions .github/workflows/convert-and-publish-gguf.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
name: Convert and publish GGUF weights

on:
workflow_dispatch:
inputs:
variant:
description: "Variant to convert (short name from the matrix, or 'all')"
required: false
default: "all"
type: string
schedule:
# Monthly, 1st of the month at 06:00 UTC
- cron: "0 6 1 * *"

permissions:
contents: read

concurrency:
group: convert-and-publish-gguf
cancel-in-progress: false

env:
UV_VERSION: "0.9.x"

jobs:
validate:
name: Validate conversion (dinov2-small)
runs-on: ubuntu-latest
timeout-minutes: 60
steps:
- uses: actions/checkout@v4

- uses: astral-sh/setup-uv@v5
with:
version: ${{ env.UV_VERSION }}

- name: Install dependencies
run: uv sync

- name: Convert dinov2-small (dry run)
run: uv run python scripts/dinov2-to-gguf.py --model_name facebook/dinov2-small-imagenet1k-1-layer

- name: Verify GGUF is loadable
run: |
uv run python - <<'EOF'
import gguf
reader = gguf.GGUFReader("./ggml-model.gguf")
tensors = [t.name for t in reader.tensors]
assert len(tensors) > 0, "no tensors in GGUF"
print(f"OK: {len(tensors)} tensors")
EOF

convert:
name: Convert ${{ matrix.variant }}
needs: validate
if: ${{ !cancelled() && needs.validate.result == 'success' }}
runs-on: ubuntu-latest
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
include:
- variant: dinov2-small
hf_id: facebook/dinov2-small-imagenet1k-1-layer
- variant: dinov2-base
hf_id: facebook/dinov2-base-imagenet1k-1-layer
- variant: dinov2-large
hf_id: facebook/dinov2-large-imagenet1k-1-layer
- variant: dinov2-giant
hf_id: facebook/dinov2-giant-imagenet1k-1-layer
- variant: dinov2-with-registers-small
hf_id: facebook/dinov2-with-registers-small-imagenet1k-1-layer
- variant: dinov2-with-registers-base
hf_id: facebook/dinov2-with-registers-base-imagenet1k-1-layer
- variant: dinov2-with-registers-large
hf_id: facebook/dinov2-with-registers-large-imagenet1k-1-layer
- variant: dinov2-with-registers-giant
hf_id: facebook/dinov2-with-registers-giant-imagenet1k-1-layer
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
HF_HUB_ENABLE_HF_TRANSFER: "1"
steps:
- name: Skip unselected variants (manual dispatch)
if: ${{ github.event_name == 'workflow_dispatch' && inputs.variant != 'all' && inputs.variant != matrix.variant }}
run: echo "SKIP=true" >> "$GITHUB_ENV"

- uses: actions/checkout@v4
if: env.SKIP != 'true'

- uses: astral-sh/setup-uv@v5
if: env.SKIP != 'true'
with:
version: ${{ env.UV_VERSION }}

- name: Free disk space (giant variants)
if: env.SKIP != 'true' && contains(matrix.variant, 'giant')
uses: jlumbroso/free-disk-space@main
with:
tool-cache: false
android: true
dotnet: true
haskell: true
large-packages: true
docker-images: true
swap-storage: false

- name: Install dependencies
if: env.SKIP != 'true'
run: uv sync

- name: Convert to GGUF
if: env.SKIP != 'true'
run: uv run python scripts/dinov2-to-gguf.py --model_name ${{ matrix.hf_id }}

- name: Ensure target repo exists
if: env.SKIP != 'true'
run: uv run hf repo create dinov2-cpp-core/${{ matrix.variant }}-gguf --repo_type model --exist_ok

- name: Upload GGUF to Hugging Face
if: env.SKIP != 'true'
run: uv run hf upload dinov2-cpp-core/${{ matrix.variant }}-gguf ./ggml-model.gguf model.gguf
80 changes: 80 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
# Contributing

Thanks for considering a contribution. Keep PRs focused and small.

## Dev harness

CMake presets (defined in `CMakePresets.json`) configure the standard workflow:

```bash
cmake --preset debug # or: release, asan, ubsan
cmake --build --preset debug
ctest --preset debug
```

- `debug` / `release`: standard configurations. Both enable `DINOV2_FATAL_WARNINGS` by default, so warnings are errors.
- `asan` / `ubsan`: AddressSanitizer and UndefinedBehaviorSanitizer builds for debugging memory and UB issues.

Build directories are `build-<preset>/`.

## Tests

Unit tests use [doctest](https://github.com/doctest/doctest). Run the suite with:

```bash
cmake --build --preset debug && ctest --preset debug
```

Run the same suite under the `asan` and `ubsan` presets before submitting anything that touches memory handling or the compute path.

## Formatting

CI enforces `clang-format-18`. Before committing:

```bash
clang-format-18 -i <changed files>
```

Format violations fail the lint job.

## Warnings

`DINOV2_FATAL_WARNINGS=ON` (the preset default) promotes warnings to errors. Keep the build clean under this option; do not disable it to mask warnings.

## PR guidelines

- One logical change per PR. Keep refactors separate from features.
- Ensure the full test suite passes under `debug`, `asan` and `ubsan`.
- Update docs (`README.md`, `docs/`) if you change behavior or interfaces.
- Follow conventional commit format (e.g. `feat:`, `fix:`, `docs:`, `ci:`).

## Converting weights yourself

The README's ready-to-download GGUFs live at `dinov2-cpp-core/<variant>-gguf` on Hugging Face. To convert a variant yourself from the upstream PyTorch checkpoints:

| Model | Source checkpoint (PyTorch) |
|:-----:|:----------------------------|
| small (no registers) | [facebook/dinov2-small-imagenet1k-1-layer](https://huggingface.co/facebook/dinov2-small-imagenet1k-1-layer) |
| base (no registers) | [facebook/dinov2-base-imagenet1k-1-layer](https://huggingface.co/facebook/dinov2-base-imagenet1k-1-layer) |
| large (no registers) | [facebook/dinov2-large-imagenet1k-1-layer](https://huggingface.co/facebook/dinov2-large-imagenet1k-1-layer) |
| giant (no registers) | [facebook/dinov2-giant-imagenet1k-1-layer](https://huggingface.co/facebook/dinov2-giant-imagenet1k-1-layer) |
| small (registers) | [facebook/dinov2-with-registers-small-imagenet1k-1-layer](https://huggingface.co/facebook/dinov2-with-registers-small-imagenet1k-1-layer) |
| base (registers) | [facebook/dinov2-with-registers-base-imagenet1k-1-layer](https://huggingface.co/facebook/dinov2-with-registers-base-imagenet1k-1-layer) |
| large (registers) | [facebook/dinov2-with-registers-large-imagenet1k-1-layer](https://huggingface.co/facebook/dinov2-with-registers-large-imagenet1k-1-layer) |
| giant (registers) | [facebook/dinov2-with-registers-giant-imagenet1k-1-layer](https://huggingface.co/facebook/dinov2-with-registers-giant-imagenet1k-1-layer) |

```bash
python ./scripts/dinov2-to-gguf.py --model_name facebook/dinov2-small-imagenet1k-1-layer
```

See [docs/hf-publishing.md](docs/hf-publishing.md) for how CI publishes the mirrors (`HF_TOKEN` setup included).

## Build from source

```bash
git clone --recurse-submodules https://github.com/espetro/dinov2.cpp.git
cd dinov2.cpp
cmake --preset release && cmake --build --preset release
```

Per-device optimizations (AMD hosts, OpenMP), sanitizer presets and benchmark instructions: [docs/build.md](docs/build.md).
Loading
Loading