Fix simple matcher dropping the prediction matched at index 0 (reading-order edit)

[lcarrere]

Problem

match_gt2pred_simple (src/core/matching/match.py) uses pred_idx = "" for an unmatched ground truth and an integer for a matched one, then tests it with truthiness:

'pred_category_type': get_pred_category_type(pred_idx, pred_items) if pred_idx else "",
'pred_position': pred_items[pred_idx]['position'][0] if pred_idx else "",

Index 0 is falsy, so the prediction matched at index 0 is reported with pred_position = "" and pred_category_type = "".

Impact

• The end-to-end evaluation matches tables with this function (end2end_dataset.py, match_gt2pred_simple(gt_table, pred_table_candidates, 'html_table', img_name)), and the table matches are fed to the reading-order computation.
• get_order_paired keeps only items with pred_position != "" on the prediction side, but keeps every ground-truth position. The first matched table is therefore removed from the predicted order and counted as a deletion.
• Result: on every page with at least one table, a correct prediction is penalized in the reading-order edit distance. Text edit, TEDS and the formula metrics are not affected (they read pred/norm_pred, which are set correctly).

match_quick.py already compares explicitly (if entry['pred_idx'] != [""] else "").

Reproduction

Two ground-truth tables, two predictions with identical content in the same order:

from src.core.matching import match_gt2pred_simple
from src.dataset.end2end_dataset import End2EndDataset

A = "<table><tr><td>Year</td><td>Revenue</td></tr><tr><td>2023</td><td>10</td></tr></table>"
B = "<table><tr><td>Region</td><td>Share</td></tr><tr><td>North</td><td>40%</td></tr></table>"
gt = lambda h, o: {"category_type": "table", "html": h, "order": o, "position": [o]}
pred = lambda h, p: {"category_type": "html_table", "content": h, "position": [p]}

m, _ = match_gt2pred_simple([gt(A, 5), gt(B, 9)], [pred(A, 0), pred(B, 1)], "html_table", "page")
print([(x["pred_idx"], repr(x["pred_position"]), x["edit"]) for x in m])
print(End2EndDataset.get_order_paired(None, m, "page"))

On main (f133a71):

[([0], "''", 0.0), ([1], '1', 0.0)]
{'gt': [5, 9], 'pred': [9], 'img_id': 'page', 'edit': 0.5}

With this PR:

[([0], '0', 0.0), ([1], '1', 0.0)]
{'gt': [5, 9], 'pred': [5, 9], 'img_id': 'page', 'edit': 0.0}

A perfect prediction scores a reading-order edit of 0.5 instead of 0.0. With a single table on the page, the table is always dropped.

Fix

Compare with "" explicitly (two lines). No other behavior changes: unmatched items still get "".

🤖 Generated with Claude Code

[lcarrere]

@dt-yy @ouyanglinke the failing CLAAssistant check is not related to this change: the CLA workflow itself cannot run. Its log ends with Could not retrieve repository contents. Status: 404, because .github/workflows/cla.yml sets branch: 'cla' while the repository has no cla branch (signatures/version1/cla.json exists only on main). The same failure appears on #253, #255 and #259.

Creating a cla branch from main (or setting branch: 'main' in cla.yml) lets the bot record signatures again; a recheck comment will then re-run it here. The PR itself is a two-line fix and merges cleanly.