Skip to content

Prototype shared structural preparation across equation-connected outputs - #477

Closed
j-emberton wants to merge 4 commits into
mainfrom
prototype/shared-structural-preparation-outputs
Closed

j-emberton wants to merge 4 commits into
mainfrom
prototype/shared-structural-preparation-outputs

Conversation

@j-emberton

@j-emberton j-emberton commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Outcome: prototype not recommended for merge

This experiment explored reusing completed structural expansions across different requested outputs. Reuse was restricted to matching supplied-input sets and a conservative reversible-equation path, with separate target-specific DM validation, matching, planning and execution.

Measured CYOF performance

Same-runner paired benchmark / all 940 CYOF tests on identical CYOF #584 source, three fresh-process runs for each case/variant:

Case Baseline cold median Prototype cold median Reduction
0 481.81 ms 463.62 ms 3.8%
30 477.18 ms 466.57 ms 2.2%
59 475.01 ms 466.78 ms 1.7%

One case-30 prototype timing (660.64 ms) was an outlier; median is shown. Warm median stayed ~27 ms. The full CYOF suite passed 940/940.

Important limitation

Latest Bilt CI: 2,867 passed, one failed, test_nested_requests_share_queries_and_next_request_starts_fresh[False]. The test expects fresh structural inspection for a new independent analysis, even within the same evaluation context. The proposed analyser-level cache persists across that boundary. This is a real mismatch with the existing request-isolation design; it should not be bypassed by weakening the test.

Decision

Close without merging. A 2–4% cold benefit is insufficient to justify changing the cache lifetime, cross-request retention and added correctness risk. The existing output-specific structural preparation is retained on main. A more promising follow-up would be an explicitly grouped multi-output preparation API with a clearly defined lifetime, or reducing per-request validation/graph construction while leaving structural evidence scoped to the request. Keep the collected tests and measurements as reference, not as merged production code.

@j-emberton j-emberton closed this Nov 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant