From 3a5510cb1f18ae81fe679d7b7e94483bcd91ff09 Mon Sep 17 00:00:00 2001 From: Bahae Date: Mon, 7 Sep 2026 13:24:52 +0100 Subject: [PATCH] Add dinov2-mlx reference entry Links to a DINOv2 port verified against real PyTorch (correctness and speed both checked, not assumed). Flagged in the README: this runs the real transformers model through torch-mlx rather than a native mlx.nn reimplementation, a different shape of contribution than this repo's usual examples -- open to restructuring or closing based on maintainer feedback. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01HooZphcpzukceMrdArswsP --- dinov2/README.md | 41 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) create mode 100644 dinov2/README.md diff --git a/dinov2/README.md b/dinov2/README.md new file mode 100644 index 000000000..683d32096 --- /dev/null +++ b/dinov2/README.md @@ -0,0 +1,41 @@ +# DINOv2 + +[dinov2-mlx](https://github.com/bahaehmimdi/dinov2-mlx) runs +[DINOv2](https://github.com/facebookresearch/dinov2) on Apple Silicon, +verified against real, unmodified PyTorch at every stage (preprocessing, +model forward, both fp32 and fp16). + +**Note on approach**: unlike the other examples in this repo, this does +not reimplement DINOv2 natively in `mlx.nn`. It runs the real, +unmodified `transformers.AutoModel` (`Dinov2Model`) through +[torch-mlx](https://github.com/bahaehmimdi/torch-mlx) — a from-scratch +`torch`-API-compatible layer backed by `mlx.core` — so the model +definition itself is real `transformers` code, not a hand-written port. +Given that's a different shape of contribution than this repo's usual +convention (native model + `torch` used only for offline weight +conversion), this entry is a link to the full implementation and +results rather than inline code — happy to restructure this however +maintainers think fits best, or to close this if it's not a good fit +for the example gallery. + +## Results + +Verified against real PyTorch: `pooler_output` matches to ~0.32% +relative error. Beats real PyTorch's own MPS backend by ~1.12-1.15x +across tested image sizes (both backends timed on the identical full +pipeline: image → resize → crop → forward). Full write-up, including +what didn't work, in the linked repo's `BENCHMARK_RESULTS.md`. + +```python +from dinov2_mlx import DINOv2MLX +from PIL import Image + +model = DINOv2MLX() +out = model.extract_features(Image.open("photo.png")) +out["pooler_output"] # (1, 1024) numpy array +out["last_hidden_state"] # (1, 257, 1024) numpy array +``` + +See [the repo](https://github.com/bahaehmimdi/dinov2-mlx) for install +instructions, the fp16/`mx.compile` options, and full benchmark +methodology.