Skip to content

kinship and nell benchmark results for kgfm's - #463

Merged
Demirrr merged 5 commits into
developfrom
more_kgfm_benchmarks
Sep 22, 2026
Merged

Demirrr merged 5 commits into
developfrom
more_kgfm_benchmarks

Conversation

@LckyLke

@LckyLke LckyLke commented Sep 21, 2026

Copy link
Copy Markdown
Collaborator

As requested by @Louis-Mozart

@LckyLke

LckyLke commented Sep 21, 2026

Copy link
Copy Markdown
Collaborator Author

#462

@LckyLke

LckyLke commented Sep 21, 2026

Copy link
Copy Markdown
Collaborator Author

also added Ultra-4g and 50g eval

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Resolve validation/test overlap handling and preserve configurable ULTRA checkpoint injection.

Review effort: Lite
Findings: None

What changed in this PR

Adds KGFM benchmark results for KINSHIP and NELL variants, including ULTRA checkpoint metadata, leakage handling, tests, and documentation.

Changes:

  • Adds checkpoint selection and pretraining metadata.
  • Adds split-overlap validation and regression tests.
  • Documents benchmark results and reproduction guidance.
File Description
tests/​test_kgfm_zero_shot.py Tests leakage handling and checkpoint metadata.
README.md Adds benchmark results and leakage notes.
docs/​kgfm_benchmarks.md Documents protocol, datasets, checkpoints, and reproduction.
benchmarks/​kgfm_zero_shot.py Adds overlap validation and checkpoint metadata.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@LckyLke
LckyLke requested a review from Demirrr September 21, 2026 17:44

@Demirrr Demirrr left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great PR. Thank you for runing experiments and adding the tests!

@Demirrr
Demirrr merged commit 07e372d into develop Sep 22, 2026
8 checks passed
@Demirrr
Demirrr deleted the more_kgfm_benchmarks branch September 22, 2026 09:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants