Skip to content

Use URL authorities for radar artifact filters - #729

Closed
rudycelekli wants to merge 1 commit into
ktwu01:mainfrom
rudycelekli:fix/query-radar-artifact-urls
Closed

rudycelekli wants to merge 1 commit into
ktwu01:mainfrom
rudycelekli:fix/query-radar-artifact-urls

Conversation

@rudycelekli

Copy link
Copy Markdown
Contributor

Derive Radar artifact flags from URL authorities

Radar search currently recognizes artifact domains anywhere in URL text, so an unrelated query, userinfo or lookalike hostname can satisfy has_paper, has_repo and has_dataset filters. Parse HTTP(S) URLs and use exact normalized first-party authorities while preserving the existing arXiv/GitHub domain indicators and HF/Kaggle dataset path prefix. These flags report URL indicators; page existence and page type are unverified. Skip non-string artifact entries accepted by the snapshot reader.

Validation:

  • Focused query suite: 60 passed in 2.47s.
  • Original source: 10 failed, 19 passed, 31 deselected in 0.41s. Removing the fix: 10 failed, 19 passed, 31 deselected in 0.37s.
  • Earlier helper: 11 failed, 18 passed, 31 deselected in 0.29s. Boundary mutant: 11 failed, 18 passed, 31 deselected in 0.57s. Controls cover arXiv abs/pdf/html/format/src/legacy ftp paths in main and artifact URL locations.

Full clean-checkout validation

Exact signed head: 8024a917e76404ac03bd432090350976868edeb7. Recursive paper submodule initialized at its repository-pinned commit. All six required commands pass:

  • ruff check .
  • ruff format --check .
  • benchmark-radar normalize-catalog
  • benchmark-radar classify
  • benchmark-radar build-data-release
  • pytest -q: 1494 passed, no skipped tests

The rebuilt corpus retains 3,212 records, five sources, and 67 snapshots. Generated outputs are not committed. Signed commit with DCO sign-off. GitHub checks are separate from these local results.

Signed-off-by: Rudy Celekli <47457359+rudycelekli@users.noreply.github.com>
@rudycelekli

Copy link
Copy Markdown
Contributor Author

Agent-authored PR annotation: 330226 GPT.

ktwu01 commented Oct 7, 2026

Copy link
Copy Markdown
Owner

Closing as not planned. One author now has a cap of 10 open issues + PRs, and this account had 56 open PRs and 11 open issues, mostly paired with each other. We merged the three changes that fix silent data loss in the corpus (#723, #774, and #773 once rebased) and are closing the rest so review stays possible.

This is not a verdict that the change is wrong. If you think this one matters, reopen it later when you are under the cap, one at a time, with a reproduction from a real user or caller rather than a hypothetical edge case.

不是否定这个改动本身。为保证评审能跟上,同一作者最多同时保留 10 个未关闭的 issue/PR;该账号曾有 56 个未关闭 PR 和 11 个 issue。已合并修复语料静默丢失的三个改动,其余先关闭。若认为本条重要,请在低于上限后逐个重新打开,并附真实用户或调用方的复现。


Generated by Claude Code

@ktwu01 ktwu01 closed this Oct 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants