Skip to content

ProofPFN: pretrain ULTRA on synthetic graphs - #477

Draft
LckyLke wants to merge 123 commits into
kgfm_cqdfrom
proofpfn
Draft

LckyLke wants to merge 123 commits into
kgfm_cqdfrom
proofpfn

Conversation

@LckyLke

@LckyLke LckyLke commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

Stacked on #464: the base is kgfm_cqd, so the diff shows only the ProofPFN changes. This PR targets develop after #464 merges.

ProofPFN pretrains ULTRA only on synthetic graphs that a prior generates. The pretrained model then ranks the test facts of real graphs zero-shot. This PR ports the research code of the thesis into dicee.

Changes

  • dicee/synthetic_kg/: the prior, the seven training arms (S, R, M, N, HT, CU, UL), pretraining and the checkpoint format.
  • dicee_pretrain_synthetic: a command that trains an arm, converts research exports and builds the real source graphs.
  • benchmarks/proofpfn/: the zero-shot benchmark, with adapters for KGPFN, MERIT and AnyBURL. registered/ holds byte-identical copies of the frozen analysis files.
  • ULTRA: two options, ultra_relation_weighting and ultra_pair_channels, also as dicee flags. The defaults keep official ULTRA. GraphKGE can attach a graph and load a checkpoint.

Guide: docs/proofpfn.md.

Tests

Golden tests compare every ported part with the research code. Some need the large goldens, the real source graphs or the reference GPU, and skip without them.

  • Full suite with all goldens on the local GPU (93509bf6): the only failures are the 7 that kgfm_cqd also has.
  • CI-like run (Python 3.11, torch 2.14, pytest 9.2): 1,672 passed, 0 failed.
  • ruff is clean. mypy finds 1,103 errors (baseline 1,131).

Open

  • A comparison with the research code on kiowl, after the fresh-v1 analysis.

Two options of ULTRA's relation graph, read in ULTRA.__init__ and declared in
dicee.config.Namespace next to ultra_dim:

- ultra_relation_weighting: 'binary' (default, official ULTRA) or
  'normalized'. 'normalized' multiplies each relation-graph message by the
  normalized incidence count sqrt(c^2 / (d_a d_b)) of its edge, formed from
  the reduced exact fraction. The weighted relation reasoner runs the same
  plain PyTorch operations in training and inference (gather, relation row,
  weight, index_add, linear, layer norm, ReLU, residual); the binary path and
  the entity reasoner are unchanged, fused kernels included.
- ultra_pair_channels: 'off' (default) or 'on' (requires 'normalized'). 'on'
  appends the same-orientation and reverse-orientation entity-pair edges
  (types 4 and 5) with their weights after ULTRA's types 0 to 3. The relation
  tables get 6 rows, drawn after all of ULTRA's tensors with rows 0 to 3
  copied from ULTRA's, so the other tensors equal official ULTRA's under the
  same seed.

dicee/models/_relation_weights.py holds the weight computations (CPU,
float64); weighted_relation_graph in ultra.py composes them with
build_relation_graph for set_graph and for training-time preparation. The
state dictionary keeps ULTRA's 82 keys. load_pretrained refuses a checkpoint
whose model_config records other relation-graph options.

Tests: hand-checked counts and pair matches, the draw order of the
initialization, unit weights against the binary reasoner, the checkpoint
option guard, and, with PROOFPFN_GOLDEN set, the relation graphs and float64
weights of 24 task graphs prepared by the research worker (golden G2).
…ld order

New package dicee/synthetic_kg, the base of the ProofPFN port:

- streams.py: the seeded SHA-256 streams with the research version strings:
  the family draw of every update ('prior-fix-v1:mix'), the capacity-prior
  stream with the initial-weight seed ('capacity-prior-v1:<seed>:initial:d64'),
  and the stage-2 routing of st_prior and sw_prior. The world generators add
  their streams in a marked section.
- recipe.py: the values of the top of the research configuration chain as a
  frozen RecipeConfig (aa_common.cfg), the stream layout and schedule
  constants, and the parameters of the record families categorical2, attr2,
  rec1 and list1 with their override rule.
- batch.py: WorldBatch (the RecipeBatch fields that an update reads), World,
  check_batch, world_sizes and the content digest of a world.
- digests.py: the tensor and file digest rules of the golden references.
- priors/__init__.py: PriorConfig, FamilySpec, Slot, parse_mix,
  build_order with the reserved stream ranges, order_sha256, and the world()
  dispatcher, whose generators raise NotImplementedError until they are
  ported.

Tests against the research code: all 21 order digests of the arms S, CU, UL,
HT, N, R and M for seeds 42 to 44 and the first 400 slots of the 7 distinct
orders (golden G1), built from the research run options; the shares and
cumulative bounds of every mixture; the recipe and parameters against a dump
of the research chain (new fixture recipe_research.json.gz); with
PROOFPFN_GOLDEN set, the batch check and content digests of 12 research worlds
(golden G2).
arms.py holds the arms S, R, M, N, HT, CU and UL as ArmPreset values: the
prior (the research --mix strings verbatim, the order names REC10A-XLE, RO2
and M2-REC10A-XLE, the family specs; N's record families with the structural
null budget 3), the ULTRA options (normalized with pair channels for S, R,
M, N and HT; normalized without them for CU; binary for UL), the target (hard
for HT, rbw otherwise) and the research order digests of seeds 42 to 44.

initial_model(arm, seed) seeds the CPU generator from the stream
capacity-prior-v1:<seed>:initial:d64 and builds ULTRA with the arm's options,
restoring the global random state afterwards. research_hook and
research_record restate a preset in the research run-record form, for the
checkpoint converter and the tests.

Tests: each preset equals the run options of its research arm (golden G1
records, built independently from the opts hooks), its 24,000-slot orders
give the pinned digests, and the initial weights of every arm and seed give
the golden G3 digests over ULTRA's 82 keys (S, HT, N, R, M with 6-row
relation tables; CU and UL with 4 rows).
…verter

checkpoint.py defines proofpfn-checkpoint-v1, a dictionary that torch.load
reads with weights_only=True: ULTRA's state dictionary, the arm's
model_config (the two relation-graph options), arm, seed, updates, raw or EMA
weights, the order digest and a provenance record. make_checkpoint,
check_checkpoint, save_checkpoint (atomic) and read_checkpoint handle the
format; load_model builds ULTRA from model_config and loads the weights
through ULTRA.load_pretrained, leaving the global random state unchanged.

from_research_export(path, arm, weights) converts a research export-<N>.pt
(read with weights_only=True). It checks the run record against the arm (the
research options, the recipe constants and the pinned order digest), the
model kind and model_config, the keys and shapes, the marker values, that
both adapter output layers are zero, that adapters and markers are equal in
the raw and EMA weights, and, without pair channels, that relation-table rows
4 and 5 are zero. It drops these parts and records the export's name,
SHA-256 and run record. determinism.py holds deterministic_execution() and
the cuBLAS workspace check.

Tests: the 41 research exports of golden G5 (CPU and genuine GPU, all seven
arms) give the golden clean raw and EMA states, and their adapters still
equal the initial values of golden G3; refusals of non-inert or foreign
exports; converted exports of S, CU and UL (seeds 42 to 44) scored on UMLS
by ULTRA with the arm's options give the research models' score and rank
digests on the CPU, and on CUDA in deterministic mode also the per-seed raw
ranks of golden G6. The research score digests were recorded in phase 3a
(umls_research_scores.json).
The repository ignores JSON files below benchmarks/ except listed folders. Add the
manifests and the golden test fixtures of benchmarks/proofpfn to the list, as the
cqa benchmark does for its recipes and fixtures.
…nchmark

benchmarks/proofpfn/manifests.py resolves the graph sets without PyTorch:
- dev121: the 121 audited development graphs of graph-audit-v1 (views main 91,
  contamination 29, apart UMLS), content hashes from the audit inventory, file
  hashes where the standard suite manifest records the folder;
- t1: the 54 zero-shot graphs of p0-suite-v1 with suite ids, file hashes, counts;
- gate: UMLS, NELLInductive_v1 and grail-WN18RR_v1, the graphs of the exact-rank
  goldens (the audit excludes the grail copy, so the gate lists it).
Every graph is hash-checked before it loads. A blind manifest opens only with a
release flag naming it, and only it may name KGs/fresh-v1 or KGs/lockbox3; the
branch holds none. models.json pins the research exports of the families and the
reference checkpoints; HT's pins are pending and refused until the lead fills them.
The build functions derive the manifests from the kiowl inputs; a test rebuilds
them byte for byte when PROOFPFN_GOLDEN is set. Small golden fixtures (G6, G7, G8)
go to tests/golden with their SHA-256 and source.
…ules

benchmarks/proofpfn/data.py hash-checks a graph against its manifest, then loads
it as the research evaluation did: ids are the sorted names over train, valid and
test, the triples of each split keep their file order. It holds the first-1000
sample rule of allgraph-v1 (index = position in test.txt, unsampled facts stay in
the filter), the filter sets, the relation-ordered query groups, the share1
self-loop rule and the GT2 inputs. On the three gate graphs the query order gives
the row identity of the research rows (golden G6) without a model. dicee's KG is
not used: its pandas reader merges entities named NA and null (a test shows it).
benchmarks/proofpfn/systems.py builds the systems of the benchmark on the CPU,
frozen, with the global random state restored:
- ProofPFNFamily: three seeds of one arm from clean checkpoints or research
  exports (converted on load with the 3a converter); a registered key requires the
  research export pins of models.json; the row file records the research export's
  SHA-256 where there is one;
- TRIX and ULTRA-3g from the released checkpoints, query batch 1;
- official Flock at the graph's official walk count and passes (the fo_rule table
  of the 54 suite graphs, else the rule), seed 42, and the passes of one query in
  one microbatch as an explicit option instead of the research run-time subclass.
attach() sets the inference graph on the CPU and then moves the model, as the
research harness did.
…with their checks

benchmarks/proofpfn/registered holds fv1_analysis.py, test_fv1_analysis.py,
fv1_tables.py, test_fv1_tables.py, fv1_drops.py and fv1_devrows.py as copies of
the research snapshot, whose SHA-256 equal the freeze anchors of the registration.
Ruff skips the folder (a nested ruff.toml) and pytest never collects or imports the
frozen test scripts (conftest.py). tests/test_registered.py checks the anchors,
runs the frozen fast checks as subprocesses, and reproduces golden G8 in a
subprocess: the simulated rows of the expected design through fv1_tables.py and
fv1_analysis.py (B = 999 and 50,000), every output byte for byte. The 9-minute
simulations run only with PROOFPFN_SIMULATE=1 (marker simulate).
…h protocol

benchmarks/proofpfn/evaluate.py ranks a graph as fair_eval with ag_seedraw did,
without registry patches or frame hooks: one call per query group and model under
deterministic_execution with two CPU threads (both restored), log_softmax in
float64 per model, the family view as the mean over the seeds, the share1 rule
view, pessimistic ranks through FilteredRanker(tie_policy='pessimistic') (equal
to the research rank on every NaN-free vector, as a test checks), and the GT2
views. The row keys and their order are the research rows' (F, F:gt2, F:raw,
F:raw:gt2, F.seedS:raw, F.seedS, F.seedS:gt2); F.seedS:raw is a plain rank. The
row file keeps the research record that fv1_tables.py reads and adds the views
per direction and a provenance record. Shards of the query groups write part
files in dot folders (which fv1_tables.py skips), resume after the last stored
group, and merge into the same rows as one run. A test feeds a clean row file to
the registered fv1_tables.py.
benchmarks/proofpfn/cells.py writes queries.csv as fresh-v1-runtime/fv1_cells.py
with s5_measure.evaluate_rows and the R1, R2 and PEER predictors of
realsrc-v1/xrel/xr_diag.py: train.txt as the observed graph with 2R relations,
every test fact in both directions in the order of random.Random(0), the same
random draws for the R2 bodies, rank_of with mean ties and the anchor excluded.
POP, VOTE and CF are not ported (no output holds them, and they draw no random
numbers). On the three gate graphs the files equal golden G7 byte for byte.
python -m benchmarks.proofpfn with the commands list, check (hashes, counts and
pins on the CPU, no score), run (row files; --shard, --parts, --save-minutes,
--max-minutes; CUDA out of memory writes an error record in .errors/ and goes on),
cells and fv1-models (the --models file of the registered fv1_tables.py). A family
is --family [KEY=]ARM:TEMPLATE with {seed}; the key defaults to the arm's
registered key. The command sets CUBLAS_WORKSPACE_CONFIG when it is unset, saves a
shard's part on SIGTERM, and restores both afterwards.
… flag in the record

A family reads its files once and a reference checks its file for every graph, in
one loop. The allgraph record of a T1 graph keeps the suite's tail_only_view flag.
…gistered tables

The clean runner ranks the three gate graphs with the G4 exports of S, CU, UL and
HT (converted on load, under the golden's test keys) and TRIX, ULTRA-3g and
official Flock, on the GPU of the golden run in deterministic mode; every key's
digest, the row identity, the key order, the metrics, the examples and the record
must equal the research rows. Clean and research rows under the registered keys
give a byte-identical mrr.csv through the registered fv1_tables.py. On the CPU the
per-seed raw ranks of S, CU and UL on UMLS equal the research models' (phase 3a).
The tests skip without CUDA, on another GPU, without CUBLAS_WORKSPACE_CONFIG or
without PROOFPFN_GOLDEN.
…n G8

The writer of benchmarks/proofpfn/evaluate.py serializes every simulated row file
of the expected design byte for byte as the research harness did, so the
registered chain reads clean rows alike. The test then removes its 0.5 GB of
temporary rows and tables.
…le test

mypy reads Queries.positions as optional; the test now asserts it is set first.
…hmark

README.md: the commands, the family and reference options, the graph sets and the
blind-set rule, the systems and the row file, sharding, how to run the registered
analysis files, and the golden tests with what they need. PROTOCOL.md: each step
of the protocol with the research module it reproduces and the function here, the
row keys, official Flock's setting, the differences of the row record from the
research record, and the cells.
HT's three exports finished training on 29 October 2026 (24,000 updates,
verified by its verified-ht-24000.json files). The research pins are in
K/fresh-v1/models-pcuht.json (SHA-256 64d8e37d...), which the frozen
worker merges with models.json. models.json now holds the same pins for
arm HT (key pcuht) and lists the second research file as a source, so
fv1-models writes the --models file of fv1_tables.py with all families.

Tests: the HT key is registered with its pins; a manifest with pending
pins (built in the test) still refuses the key and leaves it out of the
fv1 models; the pins equal the union of the two research files.
The legacy generator of the prior. priors/legacy.py: sparse rule worlds with exact
Horn truth, the pair-affinity rewiring and the new rule programs of their selected
slots, the two-task batches and the targeted observation masks. priors/factorial.py:
the control worlds of one legacy index in five (world-first and logical-family source
worlds, sparse demonstrations, the factorial treatment, the observation dose, the
second matching and endpoint reuse) and the Horn-world primitives both modules share,
with the research provenance chain that ends in the batches' world_sha256.

world() builds the plain worlds of the stage-1 stream; research_fingerprint() gives
the research RecipeBatch fingerprint of each task. The legacy streams join streams.py
with their research version strings. The generator reads the top of the research
configuration chain (recipe.recipe_config).

Tests: the 11 plain worlds of golden G2 (content digest, research fingerprint,
metadata, template provenance, task order; 5 in the default run, all with -m slow),
the closure, the streams, the source selection and the slot checks.
priors/stage2.py: the stage-2 stream (st_prior.structured): statistical worlds from
latent similarity, structured worlds (attribute block, self relations, many-relation
background on the legacy template) and the stage-2 plain worlds, each in the layout
of the legacy template of its index. query_world holds the observation, query and
negative sampling that the statistical, record and real worlds share.

Tests: the 22 structured and statistical worlds of golden G2 (2 in the default run).
priors/records.py: the latent worlds of pf_prior2f (categorical2), pf_attr2 (the same
draws with a second generator for the fidelities and the value chain), pf_rec1
(tables in latent taxonomies) and pf_list1 (anchor lists from communities), with
their parameters from recipe.record_params and the observation and queries of
stage2.query_world in the template's layout.

Tests: the 44 record worlds of arm S in golden G2 (4 in the default run).
…Ex-M

priors/real.py: the real worlds of arms R and M (rw_prior.real_world): a breadth-first
entity set of a train split, renumbered, in the template's layout. The source folder
is an argument or the environment variable PROOFPFN_REAL_SOURCES; every data.pt is
checked against its pinned SHA-256 (recipe.REAL_SOURCE_SHA256) before it loads. The
size cap of the research (RW_MAXN 20,000) is the parameter max_entities of the real
family (recipe.REAL_PARAMS), which FamilySpec now accepts.

Tests: the 10 real worlds of golden G2 (1 in the default run; both skip without
PROOFPFN_REAL_SOURCES), the pin check and the family parameters.
priors/null.py: the degree-preserving swaps of s5_swap (v2: directed swaps, and
undirected swaps of symmetric relations after v1's directed draws) with their own
streams per seed, index and family. FamilySpec.null_swap_budget applies them to a
record world's truth between the latent world and the observation, as s5_null did,
without patching a generator; the metadata keeps the research summary without its
timing. The option is refused for families other than the record families.

Tests: the 36 nulled worlds of golden G2 (4 in the default run), the invariants of the
swaps, the null streams, and that the group tests cover all 123 golden worlds.
…rker

The world-generator tests skipped their full G2 checks unless pytest got
-m slow or PROOFPFN_SLOW was set. The repository's slow marker means
"runs by default, deselect with -m 'not slow'", and the other ProofPFN
tests follow it. The checks are now plain slow tests (all 123 worlds in
about 45 s). The fixtures README lists the generator tests as users of
the G2 fixture.
…ark's ranks

Port of the research runner anyburl-ref-v1-runtime/anyburl_run.py (view ag)
as benchmarks/proofpfn/adapters/anyburl.py, with the shared adapter pieces in
adapters/__init__.py: the external folder from an option or an environment
variable, pin and commit checks, the rule-view and raw ranks of a score vector
with the benchmark's pessimistic ranker, and the row writer in the layout of
the benchmark's row files.

AnyBURL learns in the KR 2026 setting (30 s, 4 threads) on train.txt, applies
with TOP_K_OUTPUT min(1000, entities) and ranks by its printed confidences;
every Java step runs under a systemd-run memory scope. models.json pins the
jar, the sources archive and, for the next adapters, the KGPFN commit and the
MERIT commit and source files (section external).

The tests port the research checks of hand-made ranking files and compare the
rebuild of a stored ranking file of NELLInductive_v1 with the research
rebuild; with $ANYBURL_HOME, Apply with the stored rule file and one Learn run.
Host side (adapters/kgpfn.py) of the research runners kgpfn_ag.py and
fv1_worker.py: hash-checked loading and the ranked facts of the benchmark, the
context negatives that fill every ranked query (80, 40 or 20), the batch by the
entity count, staged split files with meta.json, the worker run with one retry
at half the batch after CUDA out of memory, a check of the worker's ranks
against the benchmark's ranker on the dumped scores of small graphs, and the
row file with kgpfn (rule view) and kgpfn:raw.

The worker (adapters/kgpfn_worker.py) ports kgpfn_eval.py in its all-graph
mode with the answer-blind context and context seed 0. It runs in KGPFN's own
Python (an interpreter or a container wrapper, --python or $KGPFN_PYTHON) with
the KGPFN folder on PYTHONPATH and imports nothing of the benchmark.

tests/golden/kgpfn_probe_ranks.json holds the ranks of three probe graphs per
GPU type: the unchanged research runner on the local RTX 4070 Ti SUPER and the
kiowl rows (H100 NVL). The probe test runs with KGPFN's environment on a GPU
of the fixture; the other tests need no KGPFN.
…rences

Port of allgraph-v1/ag_merit.py with merit-ref-v1-runtime/merit_ref.py as
adapters/merit.py: dicee's ULTRA with MERIT's relation Transformer and motif
tensor, the noise of each query seeded by its head entity and converted
relation, flex attention on CUDA, a fresh compiler cache per graph with
recompilation as an error. The five systems merit1024 to merit8000 are
references of evaluate.evaluate_graph, so their rows have the benchmark's
views and allgraph.model_sha256 holds the checkpoint pins of models.json.

MERIT's code is passed to the model instead of loaded with stand-in modules:
it imports from MERIT's package, which needs MERIT's requirements, so the
adapter checks the setup and runs the scoring in a child process with the
MERIT folder on its PYTHONPATH (--python or $MERIT_PYTHON).

The tests use a stand-in relation model of the same interface;
tests/golden/merit_check_ranks.json holds the raw ranks of merit-ref-v1/check
(kiowl) for the test that needs MERIT's environment on an H100 NVL.
python -m benchmarks.proofpfn adapter {kgpfn,merit,anyburl} runs an external
baseline on the graphs of a manifest, with the graph selection of the other
commands, --code (or the adapter's environment variable), --work, --models
and the options of each adapter; --check checks the setup, the pins and the
graphs without a run. A graph whose external step fails gets an error record
in .errors/ and exit code 3; a setup or pin problem exits with 2. For MERIT the
command checks the setup and reruns itself in MERIT's environment.

tests/test_adapters.py covers the shared checks and ranks, the row files of
the three adapters, the command-line refusals, and the seam with the
registered fv1_tables.py, which reads the rows under the registered keys
(kgpfn, anyburl, merit<seed>) with their rule views.
README: the adapter command, one section per adapter (code and commit,
checkpoints, runtime, a container wrapper for KGPFN's image, the settings,
how the frozen run used it), the licence note, and the tests that need the
external code. PROTOCOL: the scores and ranks of each adapter next to the
benchmark's protocol. The golden README lists the five adapter fixtures.
…ole graph

GraphKGE.internal_graph computes set_graph's internal graph without a model, attach_graph attaches it (set_graph
and load_graph now go through it), and ultra.relation_graph gives ULTRA's relation graph for its options, so that a
worker process can prepare a graph that the model attaches as set_graph would build it. forward_query_facts scores
every entity in the queried position of query facts without hiding the graph edges of the queried facts, in the
model's current mode.
…ds a file once

read_checkpoint and load_model documented CheckpointError, but raised
UnpicklingError, EOFError or FileNotFoundError, and only
from_research_export wrapped them. read_member loaded each file before
the converter loaded it again, so a --family file that was no checkpoint
ended check, run and gate in a traceback with exit 1.

- _load wraps every load error (not a pickle, empty, a broken archive,
  missing or unreadable) as CheckpointError, for read_checkpoint,
  load_model and from_research_export.
- _convert is the converter of a loaded export.
- read_arm_checkpoint reads a checkpoint or a research export of an arm
  with one load and one hash, and read_member uses it.
- convert catches CheckpointError only.
…it 0

- A device that this machine lacks, a malformed device string, and CUDA
  without a deterministic cuBLAS setting are wrong input of
  dicee_pretrain_synthetic train and of the benchmark commands run and
  gate (determinism.device_problem). pretrain raises RunInputError.
- A --family template allows the field {seed} only.
- A graph selection without graphs and an unknown audit group code are
  refused.
- adapter merit refuses a Python that does not start (AdapterSetupError).
- A MERIT child process that a signal stopped gives exit code 143. The
  docstring and the README name exit code 3 for a child at its time
  limit, which writes no error record.
…d the meta pins

- systems.py imported logging and defined a logger that nothing used.
- evaluate_graph took a threads parameter that no caller passed. The
  protocol fixes THREADS.
- json_sha256 took compact=False for one part-file identity only. The
  identity now uses the JSON digest of the goldens, so part files of
  an earlier run are refused as parts of another run.
- The KGPFN meta.json carried the pins, which the worker no longer
  reads, so prepare loses its setup parameter. The stand-in worker
  of the tests reports the files of FAKE_KGPFN_IMPORTED.
…pers only

batch.content_sha256 and tests/synthetic_worlds.content_tensors held
the same layout twice, and only tests called content_sha256. The layout
and its digest now live in tests/synthetic_worlds.py, with the dtype
rules of the golden. The world goldens pass unchanged.
write_queries kept its own atomic writer with a fixed temporary name.
It now builds the CSV in memory and writes it through the shared writer,
whose temporary name carries the process id. G7 stays byte-identical
with scipy 1.13.0 and 1.17.1.
…teps

- A run with a schedule of two updates (TOTAL_UPDATES patched) ends as
  'complete', exports its last update, and a second call does nothing.
- A stand-in Java step runs in the systemd-run scope of AnyBURL: its
  cgroup is a scope and its usage is measured, and a stop of this
  process kills and reaps it. The test skips without systemd-run --user.

The CoDEx-M rebuild test came with the content pins.
…ol check

rec1_world and list1_world defined the same add() closure over three
lists. A small class _Relations holds the lists, add() and the distinct
facts. _rule_batches repeated the short-pool check of _check_pools and
now calls it. The G2 worlds stay bitwise equal.
…lation graph

incidence_counts built a dense int64 incidence tensor and four int64
matrix products per set_graph, which torch runs without BLAS. Float64
products of the 0/1 matrices are exact, since every partial sum is an
integer of at most num_entities. The counts stay equal (checked against
the int64 products on random graphs up to FB15k-237 size), and the G2
prepared worlds and the UMLS scores on the CPU stay bitwise equal.
pretraining.py imported fcntl at module level. On Windows this broke
not only train but also convert, sources and the benchmark gate, which
import the module. The import moves into _folder_lock, and the guide
says that train runs on Linux and macOS.
After a CUDA out-of-memory record of an earlier run, a later run of the
same graph wrote the row file and kept .errors/<graph>.json. run now
removes the record when the graph is done, as adapter does.
…s.py

test_systems imported built_export from test_synthetic_checkpoint, and
test_cli and test_evaluate imported clean_checkpoints from test_systems.
Both builders now live in the shared fixtures module, so no test module
imports another.
…aligned continuation lines

- systems.family and systems.reference call their ModelPins argument
  models. pins stays the FamilyPins of a family.
- dicee/scripts/run.py takes the choices of the two ULTRA options from
  ultra.RELATION_WEIGHTINGS and ultra.PAIR_CHANNELS.
- Three continuation lines in null.py and factorial.py align with their
  opening parenthesis again.
…s close their files

- atomic_write_bytes left .<name>.<pid>.tmp behind when the write or the
  rename failed. It now removes the temporary file and raises.
- The rule count of AnyBURL's Learn and the KGPFN score dump read their
  files in a with block.
The worker hashes and builds every file of model/ultra/rspmm/source,
while the setup checked the pinned names only. A stray file there passed
the setup and stopped the run after a full GPU run of the first graph.
The setup now refuses a kernel source folder with a file without a pin.
…e child is ready, exact error types

- The AnyBURL command-line test sets the 2 s step limit for the slow step
  only, so a slow start of another step cannot fail the test.
- The stop test of call sends its signal once the child ignores SIGTERM,
  so it covers the SIGKILL path every time.
- The KGPFN output checks name the exact error type: the command line
  stops a run at an AdapterSetupError and goes on after a ValueError.
…--help

- synthetic_kg names what recipe holds, and legacy says that every
  family keeps the RecipeBatch provenance of its template.
- The adapters say which work root is a dot folder, that --work serves
  kgpfn and anyburl only, and that a differing native AnyBURL rank logs
  a warning and records native.agree.
- The smoke test docstring names both tolerances. The root README says
  that the bitwise tests hold in the recorded environment. The guide no
  longer cites the design document.
- Both command lines show their exit codes in --help, and the help of
  dicee_pretrain_synthetic shows no RST markup.
- PROTOCOL and the README of registered/ lose two badly wrapped lines.
dicee's KG loads tiktoken's GPT-2 encoding, also for the split files of
the real source graphs. On a machine with an empty tiktoken cache the
first run of sources needs the network. The guide says so. A lazy load
of tiktoken in KG belongs to kgfm_cqd.
…H100 NVL

On 30 October 2026 test_check_graph_ranks_equal_the_research_ranks passed
on kiowl's NVIDIA H100 NVL in a new MERIT environment: the raw ranks of
the five MERIT systems on both check graphs equal merit-ref-v1/check.
README, PROTOCOL, the guide and the CHANGELOG drop the unverified status.
The README lists the install commands and the package versions of that
environment. The check ranks hold for the H100 NVL only, and the test
keeps its GPU skip.
The research branch, its tags and the snapshot repository stay local and
will not be pushed. The guide, PROTOCOL, the README of registered/, the
golden README and a test comment no longer name them. The guide says
that the research code and notes are archived outside this repository,
and that the goldens and the registered files document the equivalence.
evaluate: _group_views, _group_rows, _metrics_of, _view_metrics,
_shard_blocks, _identity_of, _record_of. cells: _rules_r1, _rules_r2,
_peer_scores, _query_cells. anyburl: _write_properties, _step_failed,
_compress. kgpfn: _run_worker. ultra: _check_options. checkpoint:
_state_shapes. PROTOCOL names the private functions of its steps.
…of the benchmark

Every public function and method of benchmarks/proofpfn/, of the three
adapters and of the KGPFN worker now has Args, Returns and Raises where
they apply, and ULTRA.load_checkpoint too. write_json no longer calls its
default JSON layout compact. The code is unchanged (the ASTs without
docstrings equal the previous ones).
… KGPFN record

- select names an unknown view also when graphs are given; before, the
  command line printed the bare view name.
- A rows-only key gets its own message, and a family key must start with
  a letter, as the message says (a key of digits got the message of an
  upper-case key).
- The KGPFN host removes the outputs of an earlier attempt before each
  attempt, so an error record never shows the out-of-memory file of
  attempt 1 for a later failure.
…_digest

The worker kept its own block loop. hashlib.file_digest does the same
in the Python of the image (3.12.13) and in every supported Python.
The MERIT sentence made one line of the paragraph 207 characters long.
verify looked up each dumped row in the query groups, so a row of no
query raised a KeyError. It now raises the ValueError of its docstring.
The test also covers a dump that lacks a row.
The training and GPU goldens skip outside the recorded environment.
G7 and the real source graphs hold under every scipy and torch version,
because the cells follow the input order and the sources pin their
content.
… newer torch

deterministic_execution read the legacy TF32 flag of cuDNN. torch 2.9
and later refuse that read once the convolution and RNN settings of
cuDNN differ, and earlier tests of the suite leave them so. Fifteen
ProofPFN tests failed in a CI-like run (torch 2.14.1, pytest 9.2).

With per-backend settings, the block now sets fp32_precision = 'ieee'
on every backend that float32_precision_backends names, and restores
the previous values. torch 2.5.1, the recorded environment, keeps the
same legacy calls in the same order, so the goldens do not change.
On torch 2.14, the training call before the last refusal prints a
UserWarning about sparse tensors to stderr. The test now clears the
captured output after that call, so it checks only the refusal.
The merge brings in the develop merge dc38201 and e40dd5e. e40dd5e restores the precision settings of the kernel tests without monkeypatch, so that they pass under pytest 9.2.
@LckyLke LckyLke changed the title ProofPFN: ULTRA pretrained on a synthetic prior, with its zero-shot benchmark ProofPFN: pretrain ULTRA on synthetic graphs Oct 30, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant