Skip to content

perf(eval): reuse contiguous prepared IDs - #579

Open
JESUSROYETH wants to merge 1 commit into
roboflow:developfrom
JESUSROYETH:perf/eval-reuse-contiguous-ids
Open

perf(eval): reuse contiguous prepared IDs#579
JESUSROYETH wants to merge 1 commit into
roboflow:developfrom
JESUSROYETH:perf/eval-reuse-contiguous-ids

Conversation

@JESUSROYETH

Copy link
Copy Markdown
Contributor

The MOT evaluator already remaps IDs to zero-based integers in _prepare_mot_sequence. The metric functions then rebuild the mapping with np.searchsorted in each frame: CLEAR, HOTA, and Identity.

This patch checks each sorted unique-ID range once. If it has integer dtype, starts at zero and ends at count - 1, the per-frame IDs are already the array indices, so the code can skip straight to them. Otherwise it falls back to the current searchsorted path. This matters for public calls with arbitrary IDs, and also for prepared data where distractor removal can leave holes — three of the seven MOT17 tracker ranges hit that fallback in my tests.

Validation

I compared develop against itself first, then against this patch, using exact dtype/shape/byte checks rather than tolerances.

  • 6,000 seeded inputs / 18,000 metric calls across int16/32/64, uint16/32/64, float32/64, contiguous and non-contiguous values, empty sides, and strided views: identical aggregate digest.
  • 7 MOT17 sequences / 2,652 frames: identical prepared metric dictionary and complete serialised evaluator payload.
  • 25 DanceTrack sequences / 25,508 frames: identical prepared metric dictionary and complete serialised evaluator payload.
  • pytest -m 'not integration': 1,564 passed, 3 skipped, 14 deselected.
  • Focused evaluator tests: 53 passed. Evaluator integrations: 4 passed. Full pre-commit: passed.

Performance

CPU-only on an i9-13900HX, Python 3.12.3 and NumPy 1.26.4. Inputs are loaded before kernel timing, order alternates, and garbage collection runs before each sample.

full evaluation develop median [range] this patch median [range] delta paired wins
MOT17 511.944 ms [506.340, 516.109] 500.403 ms [495.573, 509.707] -2.25% 15/15
DanceTrack 3165.763 ms [3015.867, 3174.534] 3044.532 ms [2912.865, 3054.515] -3.83% 10/10
prepared metric kernel MOT17 delta DanceTrack delta paired wins
CLEAR -3.55% -4.59% 15/15 each
HOTA -6.19% -10.06% 15/15 each
Identity -12.21% -20.61% 15/15 each

I also relabelled the real-shaped MOT17 inputs to non-contiguous IDs to time the fallback. CLEAR, HOTA and Identity moved by +0.24%, +0.30% and -0.33% — the three baseline/candidate ranges overlap, and the observed fallback cost is at most 0.30% of kernel time.

Zero-based preparation arrived in #214. #462 optimised HOTA's generic ID remapping, and #574 later carried that approach into Identity. Both kept the general path correct for arbitrary public inputs. This patch keeps that same behaviour and adds the prepared fast path only when the precondition is present.

@JESUSROYETH
JESUSROYETH requested a review from SkalskiP as a code owner August 22, 2026 13:51
@Borda
Borda requested a balanced review from Copilot August 23, 2026 12:13
@Borda Borda added the enhancement New feature or request label Aug 23, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Optimizes evaluator ID remapping by directly reusing contiguous zero-based integer IDs while preserving searchsorted for arbitrary public inputs.

Changes:

  • Adds contiguous-ID fast paths to CLEAR, HOTA, and Identity metrics.
  • Retains fallback behavior for sparse, nonzero-based, or non-integer IDs.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
src/trackers/eval/clear.py Reuses contiguous ground-truth IDs directly.
src/trackers/eval/hota.py Reuses contiguous IDs across both HOTA passes.
src/trackers/eval/identity.py Reuses contiguous IDs during global association.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants