From d73ca09c233756667b9274a77a61a36dd5e4a5f9 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 26 Jul 2026 15:19:38 +0000 Subject: [PATCH 01/12] feat(engine): #860 widen the LP-per-node engine to mixed-integer and MAXIMIZE MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The LP-per-node spatial engine was gated to pure-integer, MINIMIZE models, so the mixed / maximize class could not even be offered to it — no primal heuristic inside it could help instances it refused to accept. Entry experiment first (CLAUDE.md §4), on the in-repo corpus (the issue's named gate probes are not vendored and this environment has no route to minlplib.org). 71 of the 119 instances are mixed; 5 are MAXIMIZE, including the syn/rsyn family. Round 1 found the binding blocker was NOT mixed-ness: 31 of the 71 mixed instances were rejected by the all-variables-finite box test, 28 by the incremental patch declining, and only 11 reached a root LP. Round 2, under the proposed gate: finite root McCormick bound 11 -> 46 of 71 verified feasible point at root 4 -> 13 of 46 (round + LP completion) The kill criterion did not fire. On the maximize family the blocker is again the box: root OBBT finitizes every infinite upper bound in 0.2-0.4s and the LP then returns a valid bound (syn05m -1165.78, syn05hfsg -1335.50 min-equivalent). Engine changes: - scope: ">=1 integer variable, either sense, any continuous mix". Pure-continuous stays out — the convergence argument is that branching the integers drives the products exact. `_is_in_scope(model, mixed=False)` keeps the old gate. - sense: the engine runs in minimize-equivalent space, where both producers already live (uniform_relax negates the LP cost, NLPEvaluator negates the point value), so `sgn` appears exactly once — on the reported objective/bound. `bound <= incumbent` becomes `bound >= incumbent` for a maximize. - continuous branching: integers split disjointly at floor/ceil, continuous bisect at the midpoint with the children SHARING it. Candidates narrower than `_MIN_BRANCH_WIDTH` are unbranchable and fold into `unresolved_lb`. - mixed primal `complete()`: fix integers, re-solve the node LP for the continuous coordinates, verify the completed point against the ground-truth evaluator — the LP completion is a relaxation, so its point is only ever a candidate. - feasibility pump acts on integer coordinates only. - infinite root boxes accepted: OBBT runs whenever the box is infinite, and whatever stays infinite goes to the cold builder (which drops rows it cannot make finite). The incremental patch has no such guard, so it is declined when a column it patches is infinite (`IncrementalMcCormickLP.box_is_patchable`). - cold node builds are deadline-bounded (build + solve), since the widened scope routes far larger models through that path. The default-path fallback reserve for the newly in-scope class is behind DISCOPT_LP_SPATIAL_MIXED (default OFF) pending its graduation panel; the engine itself is widened unconditionally. Also fixes a soundness bug on the DEFAULT path, found while widening: `IncrementalMcCormickLP` solved `min c·x` but dropped the relaxation's `obj_offset`, so its node bound sat on a different origin from the cold build's. On a node whose true McCormick optimum is -92.0 the fast path reported +8.0 — a dual bound ABOVE the true optimum, the false-fathom class. The offset is now carried on the structure and added back at every bound-returning exit including the certificate's safe_bound; `_validate` asserts it is box-independent. Verified: `pytest -m smoke` 837 passed / 15 skipped / 2 xpassed; the lp_spatial and incremental-McCormick suites pass, including new fail-before/pass-after regressions for the objective-offset bound and for mixed/maximize scope and soundness. `test_matches_brute_force` fails on 3 of 4 parameter sets before AND after this change (verified against the base commit) — a pre-existing post-#636 tightness gap. Contributes to #860 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_019bsy1zfNW8H1hZGg9SKfPu --- docs/dev/certification-gap-plan.md | 7 +- docs/dev/issue-860-lp-spatial-mixed-scope.md | 224 +++++++++++ python/discopt/_jax/incremental_mccormick.py | 81 +++- python/discopt/_jax/lp_spatial_bb.py | 355 +++++++++++++++--- python/discopt/modeling/core.py | 34 +- python/discopt/solver.py | 6 +- .../tests/test_incremental_mccormick_node.py | 69 ++++ python/tests/test_lp_spatial_bb.py | 67 +++- scratchpad/issue860_entry_experiment.py | 141 +++++++ scratchpad/issue860_entry_experiment_v2.py | 180 +++++++++ scratchpad/panel860.py | 192 ++++++++++ scratchpad/panel860_analyze.py | 124 ++++++ 12 files changed, 1404 insertions(+), 76 deletions(-) create mode 100644 docs/dev/issue-860-lp-spatial-mixed-scope.md create mode 100644 scratchpad/issue860_entry_experiment.py create mode 100644 scratchpad/issue860_entry_experiment_v2.py create mode 100644 scratchpad/panel860.py create mode 100644 scratchpad/panel860_analyze.py diff --git a/docs/dev/certification-gap-plan.md b/docs/dev/certification-gap-plan.md index 8a6ebe45..47e8fe81 100644 --- a/docs/dev/certification-gap-plan.md +++ b/docs/dev/certification-gap-plan.md @@ -212,8 +212,11 @@ leverage, plus a structural multiplier: - Root cause is identified in-code: the lifted McCormick LP is **cold-rebuilt from the DAG every node** (`_jax/mccormick_lp.py:310-320` — "~half the spatial-B&B wall clock"; `_jax/incremental_mccormick.py:1-11`). An incremental, warm-started engine - **already exists** (`incremental_mccormick.py`, wired at `mccormick_lp.py:561-563`) - but is scope-gated to pure-integer minimize models (`lp_spatial_bb._is_in_scope`). + **already exists** (`incremental_mccormick.py`, wired at `mccormick_lp.py:561-563`). + *(Superseded: `mccormick_lp`'s probe was un-gated to `ok`-only for any model by + cert:T1.3, and `lp_spatial_bb._is_in_scope` was widened to "≥1 integer variable, + either sense, any continuous mix" by #860 — see + `docs/dev/issue-860-lp-spatial-mixed-scope.md`.)* - Secondary, validated: heuristic sites construct `NLPEvaluator(model)` bypassing the `_make_evaluator` cache (−22% gear4 wall when routed through it, bound-neutral — performance-plan Stage 1). diff --git a/docs/dev/issue-860-lp-spatial-mixed-scope.md b/docs/dev/issue-860-lp-spatial-mixed-scope.md new file mode 100644 index 00000000..38906a36 --- /dev/null +++ b/docs/dev/issue-860-lp-spatial-mixed-scope.md @@ -0,0 +1,224 @@ +# Issue #860 — widening the LP-per-node engine to mixed-integer and MAXIMIZE models + +*Status: implemented. Engine scope widened unconditionally; the default-path +fallback reserve for the newly in-scope class is behind `DISCOPT_LP_SPATIAL_MIXED` +(default OFF) pending its graduation panel.* + +The LP-per-node spatial engine (`_jax/lp_spatial_bb.py`) was gated to **pure-integer, +MINIMIZE** models. Issue #860 records the consequence: three instances named in #844 +(`rsyn0805m04hfsg`, `gastrans582_*`, `gastrans040`) return no incumbent and cannot +even be *offered* to the engine, so no primal heuristic inside it can help. + +## 0. What the gate actually cost — measurement before design + +The gate probes named in the issue are not vendored in this repo (the full MINLPLib +snapshot lives outside it, and this environment has no network route to +minlplib.org), so both experiments below ran over the **in-repo corpus**: +`python/tests/data/minlplib_nl` (66 instances) ∪ `python/tests/data/minlplib` +(81, 53 new) = 119 distinct instances. That corpus is dominated by exactly the class +under discussion — **71 of the 119 are mixed** (continuous + integer) and 5 are +MAXIMIZE, including `syn05m` / `syn05hfsg`, the same `syn`/`rsyn` family as the +issue's maximize probe. + +**Round 1** (`scratchpad/issue860_entry_experiment.py`) measured the engine's +*existing* gate on that corpus and produced the finding that reframed the work: + +| blocker on the 71 mixed instances | count | +|---|---| +| root box has an infinite endpoint → `_is_in_scope` declines | **31** | +| `IncrementalMcCormickLP` declines (term types its closed-form patch does not map) | 28 | +| reach a root LP at all | **11** | + +Mixed-ness was *not* the binding constraint. Real MINLPs leave continuous columns +unbounded above, and the engine's all-variables-finite test — inherited from the +pure-integer step, where it is nearly free — rejected them before anything else could +run. A widening that only relaxed the variable-mix test would have been close to a +no-op on the real class (the #727 RLT lesson: a mechanism validated on the wrong +proxy). + +## 1. Entry experiment (CLAUDE.md §4) — the gate this issue proposes + +**Hypothesis.** For mixed-integer and MAXIMIZE MINLPs, an LP-per-node McCormick +relaxation over the continuous+integer box produces (a) a valid finite dual bound and +(b) verifiable feasible points, so widening the gate is worth the engine work. + +**Kill criterion (from the issue).** No usable incumbent anywhere in the mixed class +at any budget ⇒ mixed-integer LP-per-node is not the lever. + +**Method** (`scratchpad/issue860_entry_experiment_v2.py`), at the root box only — no +tree, no dive, no pump: + +* accept a partially infinite box — the *cold* `build_milp_relaxation` already + discards any row whose payload is non-finite (`uniform_relax._Builder.add_row`), so + it self-guards and merely loosens; the *incremental* patch does not, so it is used + only when every mapped product factor is finite; +* `round` — integers rounded, continuous left at their LP values: feasible for the + TRUE nonlinear constraints? +* `complete` — integers rounded and FIXED, continuous re-solved by the node LP: + feasible for the TRUE constraints? (the mixed generalization of the pure-integer + engine's collapsed-box primal) + +**Result — the kill criterion did NOT fire.** + +| metric (71 mixed instances) | current gate | widened gate | +|---|---|---| +| finite root McCormick bound | 11 | **46** | +| …of which served by the cold builder | 0 | 35 | +| …of which have an infinite root endpoint | 0 | 12 | +| verified feasible point from one-shot rounding | 4 | 7 | +| verified feasible point from round + LP completion | 4 | **13** | + +The 13: `cvxnonsep_psig30`, `ex1225`, `gbd`, `gkocis`, `nvs08`, `st_miqp5`, +`ex1223a`, `gear2`, `gear3`, `gear4`, `nvs20`, `prob10`, `st_e27` — each a genuinely +feasible point under an independent evaluator, from a single LP re-solve. + +**Maximize.** All 5 in-repo maximize instances have *no valid objective bound* over +their raw root box (`_objective_bound_valid=False`: the objective is unbounded above +there). That is not a maximize bug — it is the infinite box again. Root OBBT +finitizes every infinite upper bound in 0.2–0.4 s and the McCormick LP then returns a +valid bound: + +| instance | inf. upper bounds before → after OBBT | minimize-equivalent root bound | +|---|---|---| +| `syn05m` | 13 → 0 | −1165.78 | +| `syn05hfsg` | 35 → 0 | −1335.50 | + +So on this family the blocker is the box, and OBBT removes it. This is why the engine +now runs root OBBT whenever the box is infinite, regardless of `use_obbt`. + +## 2. What changed + +### 2.1 Scope (`lp_spatial_bb._is_in_scope`) + +From "every variable integer AND MINIMIZE" to "**at least one integer variable**, any +objective sense, any continuous mix". Pure-continuous models stay out: the engine's +convergence argument is that branching the *integers* drives the lifted products +exact, and with no integer variable it degenerates to plain spatial bisection, which +the default NLP-per-node path already does. `_is_in_scope(model, mixed=False)` +restores the old gate for callers rolling the widening out behind a flag. + +### 2.2 Objective sense + +The engine runs entirely in **minimize-equivalent** space. Both producers already +work there — `uniform_relax` negates a MAXIMIZE objective when it builds the LP cost, +and `NLPEvaluator` negates it when it evaluates a point — so the LP bounds and the +verified incumbents were already commensurable; nothing inside the loop needed a sign +at all. `sgn` is applied exactly once, on the reported `objective` / `bound`. A valid +lower bound on `−f` is a valid **upper** bound on `f`, so `bound ≤ incumbent` becomes +`bound ≥ incumbent` for a maximize — the maximize form of "the dual bound never +crosses the incumbent". `gap` is a ratio of absolute differences and is +sign-invariant. + +### 2.3 Continuous variables + +* **Branching.** Integers branch integrally (`floor`/`ceil`) and split spatially into + the disjoint `[lb, mid]` / `[mid+1, ub]`; continuous variables bisect at the box + midpoint into the *overlapping* `[lb, mid]` / `[mid, ub]` — the children must share + the midpoint or a feasible point could fall between them. Both only shrink boxes, + so every child relaxation stays a valid outer approximation of its subtree. +* **Unbranchable nodes.** A candidate narrower than `_MIN_BRANCH_WIDTH` (1e-9, the + same constant as the collapsed-box test, so the two agree by construction) is not + branchable; such a node folds into `unresolved_lb` and can never yield an + optimality proof. +* **Primal.** New `complete()`: fix the integers at their rounded values, re-solve the + node LP for the continuous coordinates, verify the completed point against the + ground-truth evaluator. On a pure-integer model this degenerates to the existing + collapsed-box primal. The LP completion is still a *relaxation* of the continuous + restriction, which is exactly why its point is only a candidate — it is discarded + unless the evaluator accepts it. Run once at the root, then every 16th node. +* **Feasibility pump** acts on the integer coordinates only; the continuous ones carry + no rounding distance to close and are left to the LP. + +### 2.4 Infinite root boxes + +Accepted, by two independent routes: + +* root OBBT runs whenever the box is infinite (see §1) and usually finitizes it; +* whatever remains infinite is handled by the **cold** builder, which drops rows it + cannot make finite. The **incremental** patch writes closed-form envelope + coefficients straight from the box endpoints with no row-level guard, so it is + declined when any column it patches is infinite + (`IncrementalMcCormickLP.box_is_patchable`, backed by the new `box_dependent_cols`; + `_validate` guarantees that column set is complete, since an unmapped box-dependent + row makes `ok` False). Branching only shrinks boxes, so a patchable root stays + patchable at every node. + +### 2.5 Cold node builds are now deadline-bounded + +`_relax_bound` takes a `deadline` and passes it to both uninterruptible halves — the +DAG re-walk (`build_deadline`, which keeps the row prefix: a weaker but still valid +outer relaxation) and the LP solve. The widened scope routes the mixed class largely +through this path, on models an order of magnitude larger than the pure-integer +instances the engine started on, where a single cold node could otherwise outlive the +whole engine budget. + +## 3. A soundness bug found on the way (default path, not opt-in) + +`IncrementalMcCormickLP` solves `min c·x`, but the relaxation's objective is +`c·x + obj_offset` — the constant the cold `MilpRelaxationModel.solve` adds back. The +incremental path dropped it, so its node bound sat on a different origin from the cold +build's. For a positive constant that is merely weak; for a **negative** one it is a +dual bound *above* the true node optimum, i.e. the false-fathom class. + +Measured on a bilinear integer node whose true McCormick optimum is −92.0: + +| objective constant | cold node bound | fast node bound (before) | fast node bound (after) | +|---|---|---|---| +| 0 | 8.0 | 8.0 | 8.0 | +| −100 | **−92.0** | **+8.0** (unsound) | −92.0 | +| +100 | 108.0 | 8.0 (weak) | 108.0 | + +This is on the **default** spatial path (`mccormick_lp`'s incremental fast path since +cert:T1.3), not only the opt-in engine. Fixed by carrying `obj_offset` on the +structure and adding it back at every bound-returning exit — including the +certificate's `safe_bound`, so a consumer reading either one gets the same origin. +`c_override` solves (the feasibility pump) are surrogates, not bounds, and are +deliberately excluded. `_validate` now also asserts the offset is box-independent, so +a shape where it were not would set `ok=False` and fall back rather than mis-report. + +Regressions: `test_incremental_mccormick_node.py::test_incremental_node_bound_includes_objective_constant` +(fails before, passes after) and `::test_incremental_solve_bound_matches_cold_builder_with_constant`. + +## 4. Verification + +See the PR description for the panel numbers +(`scratchpad/panel860.py`, results in `scratchpad/panel860_results.json`). + +Panel A — engine soundness on the widened class, over every in-repo instance now in +scope. Checks that need no external oracle: every reported incumbent independently +re-verified feasible with its objective re-evaluated; `bound ≤ objective` for a +minimize and `bound ≥ objective` for a maximize; the bound never crossing the best +verified feasible point found by any run of that instance; and no `status="optimal"` +run beaten by another run's verified incumbent (which would be a false optimality +certificate). + +Panel B — graduation gate for `DISCOPT_LP_SPATIAL_MIXED`, the flag governing whether +the *default* path reserves 35% of its budget for the #844 no-incumbent fallback on +mixed / maximize models. That reserve is the sole risk of the widening (an in-scope +model hands budget to a pass that may find nothing), which is why it is a separate, +measured decision from the engine's capability. Graduation needs both bars of +CLAUDE.md §5: cert-clean **and** net-positive. + +## 5. Known limits (not addressed here) + +* **28 of the 71 mixed instances still decline the incremental structure** — term + types its closed-form patch does not map (univariate `exp`/`log`, trilinear, + `monomial^p` for `p≥3`, …). They are served by the cold builder, correctly but far + more slowly. Extending the patch table term-family by term-family is + `certification-gap-plan.md` T1.2 and is deliberately out of scope here. +* Because the #844 fallback passes `require_incremental=True`, mixed models that fall + to the cold path decline that fallback even with the flag on. Relaxing it is a + budget-overrun question (the `ball_mk2_30` measurement), now partly de-risked by the + deadline-bounded cold build in §2.5, but it needs its own panel. +* A mixed model rarely reaches a *certified* optimum: the exact-leaf test requires the + whole box collapsed, and a live continuous dimension keeps a node unresolved. That + is sound and honest — the fallback exists to supply a missing incumbent, and the + dual bound remains the primary path's job — but it means the widening is a primal + gain, not a certification one. +* `test_lp_spatial_bb.py::test_matches_brute_force` fails on 3 of its 4 parameter sets + **before and after** this change (identical set, verified against the base commit): + the engine finds the true optimum but reports `feasible` rather than `optimal`. It + is the documented post-#636 conservatism — `a*b` is lifted via univariate squares, + so it never appears in the engine's `info` product map, `_worst_product_var` cannot + see it, and no node short of a fully collapsed box is treated as exact. A + pre-existing tightness gap, unrelated to this issue. diff --git a/python/discopt/_jax/incremental_mccormick.py b/python/discopt/_jax/incremental_mccormick.py index a47bbec1..e3c3b886 100644 --- a/python/discopt/_jax/incremental_mccormick.py +++ b/python/discopt/_jax/incremental_mccormick.py @@ -267,7 +267,7 @@ def _build_structure(self): lb_p[k], ub_p[k] = -(7.0 + k), -1.0 else: lb_p[k], ub_p[k] = 1.0, 7.0 + k - A, b, bnds, c, info, _ = self._full_build(lb_p, ub_p) # A is CSR (sparse) + A, b, bnds, c, info, _relax_probe = self._full_build(lb_p, ub_p) # A is CSR (sparse) # Decline only genuinely huge structures — now measured by NONZEROS (the # sparse footprint), not dense cells. qap's lift is ~172k nnz (trivial); # this guards a pathological lift whose sparse `.data` copy per node would @@ -285,6 +285,16 @@ def _build_structure(self): self.n = n self.ncol = A.shape[1] self.c = c + # Constant term of the (minimize-equivalent) relaxation objective. The LP + # solved here is ``min c·x``, but the relaxation's objective value is + # ``c·x + obj_offset`` — the cold path adds it (``MilpRelaxationModel.solve``), + # so a bound returned WITHOUT it is not on the same scale as the cold build's, + # and is not a valid bound at all when the offset is negative (measured: a + # node whose true McCormick optimum is -92 came back as +8 — a dual bound + # ABOVE the true optimum, the false-fathom class). Carried here and added + # back at every bound-returning exit; ``c_override`` solves (feasibility + # pump) are surrogates and deliberately excluded. + self.obj_offset = float(getattr(_relax_probe, "_obj_offset", 0.0) or 0.0) self.base_A = A # CSR, sorted indices; product-row VALUES rewritten per node self.base_b = b.copy() self.base_bounds = bnds.copy() @@ -387,6 +397,39 @@ def _index_of(k, col): for col in (j, a): self._pos[(k, col)] = _index_of(k, col) + @property + def box_dependent_cols(self) -> frozenset[int]: + """Structural columns whose BOUNDS drive a patched envelope row. + + :meth:`_patch` writes closed-form McCormick / secant-tangent coefficients + built directly from ``lb[k]``/``ub[k]`` of these columns. An infinite endpoint + on one of them produces ``inf``/``nan`` coefficients — silently, since the + fixed sparsity pattern has nowhere to drop a row (unlike the cold builder, + whose ``_Builder.add_row`` discards any non-finite payload and merely loosens). + A caller that may hand this structure a partially infinite box must therefore + check these columns first; :meth:`_validate` guarantees the set is complete, + because any box-dependent row it did not map makes ``ok`` False. + """ + cols: set[int] = set() + for i, j in self.bilinear: + cols.update((int(i), int(j))) + for i, _p in self.monomial: + cols.add(int(i)) + for j, _a in self.affine_square: + cols.add(int(j)) + return frozenset(cols) + + def box_is_patchable(self, lb, ub) -> bool: + """Whether :meth:`_patch` can produce a finite system over ``[lb, ub]``.""" + cols = self.box_dependent_cols + if not cols: + return True + idx = np.fromiter(cols, dtype=int, count=len(cols)) + return bool( + np.all(np.isfinite(np.asarray(lb, dtype=float)[idx])) + and np.all(np.isfinite(np.asarray(ub, dtype=float)[idx])) + ) + # -- per-node patch ---------------------------------------------------- # def _patch(self, lb, ub): @@ -560,13 +603,24 @@ def _validate(self): for i in range(self.n): regimes.add(self._box_sign_regime(float(lb[i]), float(ub[i]))) Ap, bp, bdp = self._patch(lb, ub) - Af, bf, bdf, _, _, _ = self._full_build(lb, ub) + Af, bf, bdf, _, _, relax_f = self._full_build(lb, ub) if Ap.shape != Af.shape: raise ValueError("shape mismatch") if self._rowset(Ap, bp) != self._rowset(Af, bf): raise ValueError("row-set mismatch") if not np.allclose(bdp, bdf, atol=1e-6, rtol=1e-6): raise ValueError("bounds mismatch") + # The objective constant is captured once (probe box) and added back to + # every returned bound, so it must be box-INDEPENDENT for that to be + # exact. It is, for every shape the engine lifts (the aux substitution + # moves the nonlinear part into columns and leaves the model's own + # constant), but assert it here rather than assume: a box-dependent + # offset makes ``ok=False`` and the caller falls back to the cold build. + off_f = float(getattr(relax_f, "_obj_offset", 0.0) or 0.0) + if abs(off_f - self.obj_offset) > 1e-9 * (1.0 + abs(off_f)): + raise ValueError( + f"objective offset is box-dependent ({off_f} vs {self.obj_offset})" + ) self._validated_regimes = frozenset(regimes) self.ok = True @@ -589,11 +643,17 @@ def assemble(self, lb, ub, cut_rows=None): return A, b, bounds def solve_assembled(self, A, b, bounds, in_basis=None, c_override=None): - """Solve a pre-assembled LP ``min c·x s.t. A x <= b, bounds``.""" + """Solve a pre-assembled LP ``min c·x s.t. A x <= b, bounds``. + + The returned value is the RELAXATION's objective, ``c·x + obj_offset`` — the + same scale ``MilpRelaxationModel.solve`` reports on the cold path. A + ``c_override`` solve is a surrogate (feasibility pump), not a bound, so the + offset is not applied to it.""" from discopt.solvers import SolveStatus from discopt.solvers.milp_simplex import solve_lp_warm_std cobj = self.c if c_override is None else np.asarray(c_override, dtype=np.float64) + off = 0.0 if c_override is not None else self.obj_offset try: result, out_basis = solve_lp_warm_std( cobj, sp.csr_matrix(A), b, bounds, in_basis=in_basis @@ -602,7 +662,7 @@ def solve_assembled(self, A, b, bounds, in_basis=None, c_override=None): return None, None, None if result is None or result.status != SolveStatus.OPTIMAL or result.objective is None: return None, None, None - return float(result.objective), np.asarray(result.x, dtype=float), out_basis + return float(result.objective) + off, np.asarray(result.x, dtype=float), out_basis def solve_assembled_full( self, A, b, bounds, in_basis=None, c_override=None, *, return_cert=False @@ -634,6 +694,7 @@ def solve_assembled_full( from discopt.solvers.milp_simplex import LpWarmCert, solve_lp_warm_std cobj = self.c if c_override is None else np.asarray(c_override, dtype=np.float64) + off = 0.0 if c_override is not None else self.obj_offset _empty = LpWarmCert(safe_bound=None, farkas_certified=False) def _ret(status, bound, x, out_basis, farkas, cert=_empty): @@ -653,9 +714,16 @@ def _ret(status, bound, x, out_basis, farkas, cert=_empty): return _ret("infeasible", None, None, None, bool(cert.farkas_certified), cert) if result.status != SolveStatus.OPTIMAL or result.bound is None: return _ret("other", None, None, None, False) + # Shift BOTH the reported bound and the certificate's safe bound onto the + # relaxation's own objective scale (``c·x + obj_offset``), so no consumer has + # to remember to add the constant back — the cert flows on to DBBT / reduced + # costs in ``mccormick_lp`` and would otherwise carry a different origin from + # the bound sitting beside it. + if off and cert.safe_bound is not None: + cert = cert._replace(safe_bound=float(cert.safe_bound) + off) return _ret( "optimal", - float(result.bound), + float(result.bound) + off, np.asarray(result.x, dtype=float), out_basis, False, @@ -665,7 +733,8 @@ def _ret(status, bound, x, out_basis, farkas, cert=_empty): def solve(self, lb, ub, in_basis=None, c_override=None, cut_rows=None): """Solve the McCormick LP over [lb,ub] (plus optional cut rows); return (bound, x, out_basis) or (None, None, None). Warm-starts from ``in_basis``. + The bound is the relaxation objective ``c·x + obj_offset`` (cold-path scale). ``c_override`` replaces the objective (feasibility pump) — the returned bound - is then the surrogate, not a dual bound.""" + is then the surrogate, not a dual bound, and carries no offset.""" A, b, bounds = self.assemble(lb, ub, cut_rows) return self.solve_assembled(A, b, bounds, in_basis=in_basis, c_override=c_override) diff --git a/python/discopt/_jax/lp_spatial_bb.py b/python/discopt/_jax/lp_spatial_bb.py index 21e312de..fc9ea1e1 100644 --- a/python/discopt/_jax/lp_spatial_bb.py +++ b/python/discopt/_jax/lp_spatial_bb.py @@ -15,18 +15,44 @@ heuristic produces incumbents, verified *exactly* against a ground-truth point evaluator (true objective + constraint feasibility) — never the relaxation bound. -**Scope (this step).** Pure-integer models with a MINIMIZE objective. The frontier -McCormick bound is always a valid lower bound; every incumbent is a genuinely -feasible point whose reported objective is its true objective, so ``bound <= -incumbent`` holds unconditionally. Optimality is declared only when the valid dual -bound (frontier + a floor for nodes the engine cannot branch — see -``unresolved_lb``) closes the gap: a node whose products are lifted outside this -engine's ``info`` map (e.g. univariate-square bilinear post-#636) is *not* treated -as an exact leaf, so a loose relaxation can never masquerade as a proof. Cuts -(Gomory/MIR) and warm-started incremental LPs are added in later steps; -``build_milp_relaxation`` is currently rebuilt per node. Returns ``None`` (caller -falls back to the default path) whenever the model is out of scope or anything -fails — it can never make a solve unsound. +**Scope.** Models with at least one integer/binary variable, in either objective +sense, with any mix of continuous variables (issue #860; the original step was +pure-integer MINIMIZE only). Three things make the wider scope sound: + +* *Sense.* The engine works entirely in **minimize-equivalent** space, and both of + its inputs already live there: ``uniform_relax`` negates a MAXIMIZE objective when + it builds the LP cost, and ``NLPEvaluator`` negates it when it evaluates a point. + So every LP bound is a valid LOWER bound on ``sgn * f``, every verified incumbent + is ``sgn * f`` at a feasible point, and nothing inside the loop needs a sign at + all. ``sgn`` is applied exactly once, to the reported objective/bound. The + ``bound <= incumbent`` invariant therefore holds in min-equivalent space exactly as + before, and becomes ``bound >= incumbent`` (a valid upper bound) once reported for + a maximize. +* *Continuous variables.* Branching generalizes: integer variables are branched + integrally (``floor``/``ceil``) and spatially on the integer midpoint; continuous + variables are bisected at the box midpoint. Both only ever *shrink* a box, so + every child's McCormick relaxation stays a valid outer approximation of its + subtree. A node whose remaining candidates are all narrower than + ``_MIN_BRANCH_WIDTH`` is not branchable and folds into ``unresolved_lb``. +* *Primal.* An incumbent is never read off the relaxation. Integer coordinates are + rounded, continuous coordinates are completed by re-solving the node LP with the + integers fixed, and the resulting point is verified *exactly* against a + ground-truth evaluator (true objective + constraint feasibility). A completion + that is not genuinely feasible is discarded, so ``bound <= incumbent`` cannot be + broken by the continuous half. + +Optimality is declared only when the valid dual bound (frontier + a floor for nodes +the engine cannot branch — see ``unresolved_lb``) closes the gap: a node whose +products are lifted outside this engine's ``info`` map (e.g. univariate-square +bilinear post-#636) is *not* treated as an exact leaf, so a loose relaxation can +never masquerade as a proof. Returns ``None`` (caller falls back to the default +path) whenever the model is out of scope or anything fails — it can never make a +solve unsound. + +Pure-continuous models stay out of scope: the engine's convergence argument is that +branching the *integers* drives the products exact, and a model with no integer +variable gives it nothing the default NLP-per-node spatial path does not already do +better. """ from __future__ import annotations @@ -44,6 +70,13 @@ _INT_TOL = 1e-6 _PROD_TOL = 1e-5 +# Narrowest box a variable may still be branched on. Bisecting below this buys no +# relaxation tightness and only spawns nodes, so a node whose every candidate is +# this narrow is treated as unbranchable (its bound becomes an ``unresolved_lb`` +# floor, never an optimality proof). Deliberately the SAME threshold as the +# collapsed-box exactness test below, so the two agree: a variable that cannot be +# branched any further is exactly one the leaf test counts as collapsed. +_MIN_BRANCH_WIDTH = 1e-9 class LpSpatialResult(NamedTuple): @@ -55,14 +88,27 @@ class LpSpatialResult(NamedTuple): node_count: int -def _is_in_scope(model: Model) -> bool: - """Pure-integer model with a minimize objective and at least one var.""" - if model._objective is None or model._objective.sense != ObjectiveSense.MINIMIZE: +def _is_in_scope(model: Model, *, mixed: bool = True) -> bool: + """Model this engine can serve: an objective, at least one integer variable, and + only ordinary algebraic rows. Either objective sense; any continuous mix. + + ``mixed=False`` restores the pre-#860 gate (pure-integer, MINIMIZE only) for + callers rolling the widening out behind a flag. + """ + if model._objective is None: return False if not model._variables: return False - if not all(v.var_type in (VarType.INTEGER, VarType.BINARY) for v in model._variables): + # At least one integer variable: the engine converges by branching the integers + # (which drives the lifted products exact). With none, it degenerates to plain + # spatial bisection, which the default NLP-per-node path already does. + if not any(v.var_type in (VarType.INTEGER, VarType.BINARY) for v in model._variables): return False + if not mixed: + if model._objective.sense != ObjectiveSense.MINIMIZE: + return False + if not all(v.var_type in (VarType.INTEGER, VarType.BINARY) for v in model._variables): + return False # Every row must be an ordinary algebraic constraint. A GDP/logical row # (``_LogicalConstraint``) carries no ``.body``, and the relaxation builder # walks ``.body`` unconditionally -- admitting one raises deep inside the @@ -70,14 +116,39 @@ def _is_in_scope(model: Model) -> bool: return all(hasattr(c, "body") for c in model._constraints) -def _relax_bound(model, terms, lb, ub): - """McCormick LP over [lb,ub]; return (bound, full_x, info) or None.""" +def _integer_mask(model: Model) -> np.ndarray: + """Boolean mask over flat columns: True where the variable is integer/binary.""" + flags: list[bool] = [] + for v in model._variables: + flags.extend([v.var_type in (VarType.INTEGER, VarType.BINARY)] * int(v.size)) + return np.array(flags, dtype=bool) + + +def _relax_bound(model, terms, lb, ub, deadline=None): + """McCormick LP over [lb,ub]; return (bound, full_x, info) or None. + + ``deadline`` (absolute ``time.perf_counter()``) bounds the two uninterruptible + halves of a cold node: the DAG re-walk (``build_deadline``, which stops adding + constraint rows and keeps the prefix — a weaker but still valid outer + relaxation) and the LP solve itself. Without it a single cold node can outlive + the whole engine budget, which the widened scope (#860) makes reachable: the + mixed class is served largely by this path, on models an order of magnitude + bigger than the pure-integer instances the engine started on.""" from discopt._jax.discretization import DiscretizationState from discopt._jax.milp_relaxation import build_milp_relaxation + _remaining = None + if deadline is not None: + _remaining = deadline - time.perf_counter() + if _remaining <= 0.0: + return None try: relax, info = build_milp_relaxation( - model, terms, DiscretizationState(), bound_override=(lb, ub) + model, + terms, + DiscretizationState(), + bound_override=(lb, ub), + build_deadline=deadline, ) if not relax._objective_bound_valid: return None @@ -97,7 +168,11 @@ def _relax_bound(model, terms, lb, ub): # node. The incremental path already solves a pure LP; this aligns the cold # path with it. relax._integrality = None - res = relax.solve() + if _remaining is not None: + _remaining = deadline - time.perf_counter() + if _remaining <= 0.0: + return None + res = relax.solve(time_limit=_remaining) except Exception: return None if res is None or res.bound is None or res.x is None: @@ -105,18 +180,36 @@ def _relax_bound(model, terms, lb, ub): return float(res.bound), np.asarray(res.x, dtype=float), info -def _worst_product_var(x, info, widths): +def _spatially_branchable(k, widths, is_int) -> bool: + """Can variable ``k`` still be split by a spatial (domain-bisection) branch? + + An integer needs at least two values left (``width >= 1``); a continuous variable + needs a finite box wider than ``_MIN_BRANCH_WIDTH``. An infinite width is never + branchable — there is no midpoint to bisect at, and the McCormick envelope over + such a box is vacuous anyway.""" + w = widths[k] + if not np.isfinite(w): + return False + return bool(w >= 1.0) if is_int[k] else bool(w > _MIN_BRANCH_WIDTH) + + +def _worst_product_var(x, info, widths, is_int): """Branchable variable in the most-violated product (weighted by box width), - or None if every lifted product matches its defining product.""" + or None if every lifted product matches its defining product. + + The weight ``viol * (width + 1)`` prefers a violation sitting on a wide box (more + envelope to gain by splitting). For a continuous factor the width can be < 1, so + the ``+1`` keeps the score positive and monotone in the violation — the same + ordering the pure-integer version had, extended rather than replaced.""" best, best_var = _PROD_TOL, None for (i, j), col in info.get("bilinear", {}).items(): viol = abs(x[col] - x[i] * x[j]) for k in (i, j): - if widths[k] >= 1 and viol * (widths[k] + 1) > best: + if _spatially_branchable(k, widths, is_int) and viol * (widths[k] + 1) > best: best, best_var = viol * (widths[k] + 1), k for (i, _p), col in info.get("monomial", {}).items(): viol = abs(x[col] - x[i] * x[i]) - if widths[i] >= 1 and viol * (widths[i] + 1) > best: + if _spatially_branchable(i, widths, is_int) and viol * (widths[i] + 1) > best: best, best_var = viol * (widths[i] + 1), i return best_var @@ -229,6 +322,7 @@ def solve_lp_spatial_bb( use_obbt: bool = True, root_cut_rounds: int = 0, require_incremental: bool = False, + mixed: bool = True, ) -> Optional[LpSpatialResult]: """LP-node spatial branch-and-bound. Returns ``None`` if out of scope. @@ -244,19 +338,33 @@ def solve_lp_spatial_bb( inherited by all nodes). Default 0 (off): with discopt's current Python-level separators the per-round crossover/GMI cost and the larger inherited LP at every node outweigh the modest tightening — measured net-negative on nvs17/19/24. The - machinery is sound and kept opt-in for when a fast native separator exists.""" - if not _is_in_scope(model): + machinery is sound and kept opt-in for when a fast native separator exists. + + ``mixed`` (default True, #860) admits mixed-integer and MAXIMIZE models. Pass + ``False`` for the pre-#860 pure-integer/MINIMIZE gate.""" + if not _is_in_scope(model, mixed=mixed): return None + from discopt._jax.model_utils import flat_variable_bounds from discopt._jax.term_classifier import classify_nonlinear_terms terms = classify_nonlinear_terms(model) - n = len(model._variables) - INT = list(range(n)) # scope: all variables integer - lb0 = np.array([float(v.lb) for v in model._variables]) - ub0 = np.array([float(v.ub) for v in model._variables]) - if not (np.all(np.isfinite(lb0)) and np.all(np.isfinite(ub0))): - return None # unbounded integer box: out of scope for this step + lb0, ub0 = flat_variable_bounds(model) + lb0 = lb0.astype(float, copy=True) + ub0 = ub0.astype(float, copy=True) + n = int(lb0.size) + is_int = _integer_mask(model) + if is_int.size != n: # defensive: flat layouts must agree + return None + INT = [i for i in range(n) if is_int[i]] + _has_cont = bool(np.any(~is_int)) + # Minimize-equivalent sign. Both the relaxation builder and ``NLPEvaluator`` + # already negate a MAXIMIZE objective, so every LP bound below is a valid LOWER + # bound on ``sgn * f``, every verified incumbent is ``sgn * f``, and the whole + # engine (heap order, incumbent comparison, fathoming, gap) runs unchanged in that + # space. ``sgn`` appears in exactly ONE place: the reported objective/bound at the + # exit. Applying it anywhere else would double-negate. + sgn = -1.0 if model._objective.sense == ObjectiveSense.MAXIMIZE else 1.0 t0 = time.perf_counter() @@ -284,7 +392,21 @@ def solve_lp_spatial_bb( def _pt_feasible(xr: np.ndarray) -> bool: return bool(_check_constraint_feasibility(_ev, xr, _cl, _cu, tol=_FEAS_TOL)) - if use_obbt: + # Root box with an infinite endpoint. Pre-#860 this was an immediate decline + # ("unbounded integer box: out of scope"). It is the single biggest reason the + # engine could not *accept* the mixed class: 31 of the 71 mixed instances in the + # in-repo corpus carry at least one infinite bound (real MINLPs leave continuous + # columns unbounded above), including the syn/rsyn maximize family this issue + # names. It is not, however, a soundness boundary — see the two paths below. + _inf_box = not (np.all(np.isfinite(lb0)) and np.all(np.isfinite(ub0))) + + # Root OBBT. Runs when the caller asked for it OR when the box is infinite: on an + # unbounded box it is the difference between a bound and nothing at all. Measured + # on the two in-repo maximize instances, whose relaxation has NO valid objective + # bound over the raw box (``_objective_bound_valid=False``): OBBT finitizes every + # infinite upper bound in ~0.2-0.4 s and the McCormick LP then returns a valid + # bound (syn05m -1165.78, syn05hfsg -1335.50 minimize-equivalent). + if use_obbt or _inf_box: try: from discopt._jax.obbt import obbt_tighten_root @@ -303,8 +425,17 @@ def _pt_feasible(xr: np.ndarray) -> bool: time_limit_per_lp=min(0.5, max(0.05, _obbt_budget / 10.0)), ) if not r.infeasible: - lb0 = np.maximum(lb0, np.floor(np.asarray(r.lb) + 1e-9)) - ub0 = np.minimum(ub0, np.ceil(np.asarray(r.ub) - 1e-9)) + # Integrality rounding applies to INTEGER columns only. Flooring a + # continuous variable's tightened lower bound would push it BELOW the + # value OBBT proved (widening, merely wasteful) while ceiling its upper + # bound would do the same — but rounding a continuous bound the wrong + # way on a narrow box can also erase the tightening entirely. Keep the + # exact continuous bounds and round only where integrality permits. + _rlb = np.asarray(r.lb, dtype=float) + _rub = np.asarray(r.ub, dtype=float) + lb0 = np.maximum(lb0, np.where(is_int, np.floor(_rlb + 1e-9), _rlb)) + ub0 = np.minimum(ub0, np.where(is_int, np.ceil(_rub - 1e-9), _rub)) + _inf_box = not (np.all(np.isfinite(lb0)) and np.all(np.isfinite(ub0))) except Exception as exc: # Never let root tightening break the solve -- but never hide it either. # A silently-skipped capability is precisely the failure mode that cost @@ -337,6 +468,19 @@ def _pt_feasible(xr: np.ndarray) -> bool: _own_deadline = min(_own_deadline, float(_ambient)) _inc = IncrementalMcCormickLP(model, terms, deadline=_own_deadline) + # The incremental patch writes closed-form envelope coefficients straight from the + # box endpoints, so an infinite bound on a column it patches would inject + # ``inf``/``nan`` into the node LP with no row-level guard to catch it (the cold + # builder discards such rows and merely loosens). Decline the fast path there and + # let the cold build serve the model: sound either way, and it is what lets the + # engine accept a partially infinite box at all. Branching only shrinks boxes, so + # a root box that is patchable stays patchable at every node. + if _inc.ok and not _inc.box_is_patchable(lb0, ub0): + logger.debug( + "lp_spatial: incremental structure declined on this box (an infinite " + "endpoint on a patched product column); using the per-node cold build" + ) + _inc.ok = False if require_incremental and not _inc.ok: logger.debug( "lp_spatial declined: no incremental McCormick structure and the caller " @@ -349,13 +493,13 @@ def _pt_feasible(xr: np.ndarray) -> bool: def relax(lb, ub, basis): return _inc.solve(lb, ub, in_basis=basis) else: - _r0 = _relax_bound(model, terms, lb0, ub0) + _r0 = _relax_bound(model, terms, lb0, ub0, deadline=t0 + time_limit) if _r0 is None: return None info = _r0[2] def relax(lb, ub, basis): - c = _relax_bound(model, terms, lb, ub) + c = _relax_bound(model, terms, lb, ub, deadline=t0 + time_limit) return (c[0], c[1], None) if c is not None else (None, None, None) # Branch-and-cut: separate integer cuts (GMI + complemented-MIR, product aux @@ -469,16 +613,31 @@ def _update_pc(i, direction, parent_b, child_b, frac): psi_u[i] = (psi_u[i] * cnt_u[i] + gain) / (cnt_u[i] + 1) cnt_u[i] += 1 + def _round_integers(xhat): + """Integer coordinates rounded, continuous coordinates untouched, clipped to + the root box. Rounding a CONTINUOUS coordinate would be wrong twice over: it + is not required to be integral, and moving it off the LP value throws away the + only continuous information the node has.""" + xr = np.asarray(xhat, dtype=float).copy() + xr[is_int] = np.round(xr[is_int]) + return np.minimum(np.maximum(xr, lb0), ub0) + def verify(xhat): - """Exact objective at the rounded integer point, or None if infeasible. + """Exact minimize-equivalent objective at the candidate point, or None if it + is not feasible. Ground truth only: evaluates the true objective and checks constraint feasibility with the point evaluator (never the McCormick relaxation bound, which is not exact once a product is lifted outside ``info`` -- see the verifier note above). Guarantees any accepted incumbent is a genuinely feasible point whose reported objective is its true objective, so the - frontier's valid lower bound can never exceed it.""" - xr = np.minimum(np.maximum(np.round(xhat), lb0), ub0) + frontier's valid lower bound can never exceed it. + + The value is MINIMIZE-EQUIVALENT, the same space every LP bound here lives in: + ``NLPEvaluator`` already negates a MAXIMIZE objective (``_negate``), exactly as + the relaxation builder negates the LP cost, so the two agree without any + further adjustment. ``sgn`` is applied once, at the exit.""" + xr = _round_integers(xhat) if not _pt_feasible(xr): return None try: @@ -486,6 +645,33 @@ def verify(xhat): except Exception: return None + def complete(lb_c, ub_c, xhat): + """Mixed-integer primal: fix the integers at their rounded values, re-solve the + node LP for the continuous coordinates, and verify the completed point. + + This is the mixed generalization of the pure-integer engine's collapsed-box + primal. On a pure-integer model fixing the integers collapses the box to a + point and this degenerates to :func:`verify`; with continuous variables + present, the collapse leaves an LP whose optimum supplies them. That LP is + still a *relaxation* of the continuous restriction, so its point is only a + candidate -- which is exactly why the result goes through ``verify`` and is + discarded unless the ground-truth evaluator accepts it.""" + xr = _round_integers(xhat) + lo, hi = np.asarray(lb_c, float).copy(), np.asarray(ub_c, float).copy() + # Only fix integers the node box still admits; a rounded value outside it + # would make the child box empty and the LP trivially infeasible. + fix = is_int & (xr >= lo - 1e-9) & (xr <= hi + 1e-9) + lo[fix] = xr[fix] + hi[fix] = xr[fix] + if np.any(lo > hi + 1e-9): + return None + _b, xx, _bas = relax(lo, hi, None) + if xx is None: + return None + xc = np.asarray(xx[:n], dtype=float).copy() + xc[fix] = xr[fix] + return verify(xc) + def dive(lb_d, ub_d): """Fix-and-dive: repeatedly fix the most-fractional free integer to its rounded LP value and re-solve, until all are fixed (a feasible candidate via @@ -502,6 +688,10 @@ def dive(lb_d, ub_d): return None free = [(abs(xx[i] - round(xx[i])), i) for i in INT if hi[i] - lo[i] > 0.5] if not free: + # Every integer is fixed. On a pure-integer model the box is now a + # single point and ``xx`` IS that point; with continuous variables it + # is the LP's continuous completion of the fixed integer assignment. + # Either way the candidate is verified exactly, never trusted. return verify(xx[:n]) _, bi = max(free) v = min(max(round(xx[bi]), lo[bi]), hi[bi]) @@ -513,11 +703,19 @@ def feasibility_pump(lb, ub, x_seed, max_iter=30): relaxation and rounding, each step re-solving the McCormick LP with a linear objective that pushes it toward the current rounded point, until the rounded integers are feasible (verified by the collapsed box). Finds incumbents where - one-shot rounding / diving fail (e.g. nvs19/24).""" + one-shot rounding / diving fail (e.g. nvs19/24). + + On a mixed model the pump acts on the INTEGER coordinates only -- those are the + ones that have to be rounded to something. The continuous coordinates carry no + rounding distance to close, and pushing them toward a rounded value would fight + the LP for no reason; they are left to the relaxation, and the candidate point + goes through :func:`complete` so they are re-solved against the fixed integer + assignment before verification.""" if not _inc.ok: return None x = np.asarray(x_seed, dtype=float) xhat = np.minimum(np.maximum(np.round(x[:n]), lb), ub) + xhat[~is_int] = np.minimum(np.maximum(x[:n][~is_int], lb[~is_int]), ub[~is_int]) seen: set = set() for _ in range(max_iter): # Same deadline poll as ``dive``: one LP per iteration, run at the root @@ -525,22 +723,29 @@ def feasibility_pump(lb, ub, x_seed, max_iter=30): if (time.perf_counter() - t0) >= time_limit: return None h = verify(xhat) + if h is None and _has_cont: + h = complete(lb, ub, xhat) if h is not None: return h - key = tuple(xhat.tolist()) - if key in seen: # cycle -> perturb the most-fractional coordinates - order = np.argsort(-np.abs(x[:n] - xhat)) - for j in order[: max(1, n // 4)]: + key = tuple(np.asarray(xhat)[is_int].tolist()) + if key in seen: # cycle -> perturb the most-fractional integer coordinates + frac = np.where(is_int, np.abs(x[:n] - xhat), -np.inf) + for j in np.argsort(-frac)[: max(1, len(INT) // 4)]: + if not is_int[j]: + continue step = 1.0 if x[j] > xhat[j] else -1.0 xhat[j] = min(max(xhat[j] + step, lb[j]), ub[j]) seen.add(key) c_fp = np.zeros(_inc.ncol) - c_fp[:n] = np.where(x[:n] > xhat, 1.0, -1.0) # minimize -> pull x to xhat + # minimize -> pull the integer coordinates toward xhat; leave the + # continuous ones with zero cost so the LP places them freely. + c_fp[:n] = np.where(is_int, np.where(x[:n] > xhat, 1.0, -1.0), 0.0) _b, x_, _bas = _inc.solve(lb, ub, c_override=c_fp) if x_ is None: return None x = x_ xhat = np.minimum(np.maximum(np.round(x[:n]), lb), ub) + xhat[~is_int] = np.minimum(np.maximum(x[:n][~is_int], lb[~is_int]), ub[~is_int]) return None def consider(cand): @@ -575,7 +780,13 @@ def child(lb, ub, parent_basis, parent_cuts, rounds, parent_bound): return b_ # seed an incumbent: root dive (cheap) then a root feasibility pump (catches - # cases diving misses). Both are rate-limited below so they never dominate. + # cases diving misses). Both are rate-limited below so they never dominate. On a + # mixed model the one-shot continuous completion of the root LP point goes first: + # it costs a single LP and, measured over the in-repo corpus, is on its own enough + # on 13 of the 46 mixed instances whose root relaxation is bounded (#860 entry + # experiment) -- before any diving, pumping or branching. + if _has_cont: + consider(complete(lb0, ub0, root_x[:n])) consider(dive(lb0, ub0)) consider(feasibility_pump(lb0, ub0, root_x)) @@ -602,9 +813,12 @@ def child(lb, ub, parent_basis, parent_cuts, rounds, parent_bound): if inc_x is not None and abs(inc_val - glb) <= gap_tolerance * (1 + abs(inc_val)): status = "optimal" break - # primal: cheap one-shot rounding every node; dive / pump rate-limited so - # they never dominate the node throughput (the earlier every-node bug). + # primal: cheap one-shot rounding every node; the continuous completion (one + # extra LP, mixed models only) and dive / pump rate-limited so they never + # dominate the node throughput (the earlier every-node bug). consider(verify(x[:n])) + if _has_cont and nodes % 16 == 0: + consider(complete(lb, ub, x[:n])) if nodes % 64 == 0: consider(dive(lb, ub)) if nodes % 512 == 0: @@ -630,19 +844,34 @@ def child(lb, ub, parent_basis, parent_cuts, rounds, parent_bound): # returns None even though the relaxation is NOT tight, so "no branchable # product" is NOT a proof that ``bound`` is exact here. Record a verified # incumbent (exact objective, never the loose ``bound``); the node is a true, - # fathomable leaf only when the integer box is fully fixed (a single point, - # where every nonlinear term is determined). Otherwise it is unresolved: keep - # its valid lower bound as a global-bound floor so optimality is never claimed - # over branching the engine could not perform. - bv = _worst_product_var(x, info, ub - lb) + # fathomable leaf only when the box is fully fixed (a single point, where every + # nonlinear term is determined). Otherwise it is unresolved: keep its valid + # lower bound as a global-bound floor so optimality is never claimed over + # branching the engine could not perform. The collapse test spans EVERY + # variable, continuous ones included -- a mixed node with a live continuous + # dimension is never an exact leaf, however tight its products look. + bv = _worst_product_var(x, info, ub - lb, is_int) if bv is None: consider(verify(x[:n])) - if not bool(np.all(ub[:n] - lb[:n] <= 1e-9)): + if _has_cont: + consider(complete(lb, ub, x[:n])) + if not bool(np.all(ub[:n] - lb[:n] <= _MIN_BRANCH_WIDTH)): unresolved_lb = min(unresolved_lb, bound) continue - mid = np.floor((lb[bv] + ub[bv]) / 2) - child(lb, _set(ub, bv, mid), basis, ncuts, _rounds, bound) - child(_set(lb, bv, mid + 1.0), ub, basis, ncuts, _rounds, bound) + if is_int[bv]: + # Integer domain split: [lb, mid] and [mid+1, ub] partition the integers + # in the box exactly, so no integer point is lost. + mid = np.floor((lb[bv] + ub[bv]) / 2) + child(lb, _set(ub, bv, mid), basis, ncuts, _rounds, bound) + child(_set(lb, bv, mid + 1.0), ub, basis, ncuts, _rounds, bound) + else: + # Continuous bisection: the children SHARE the midpoint, so their union is + # the parent box and no feasible point can fall between them. (The integer + # split above can use disjoint halves only because there is nothing between + # ``mid`` and ``mid+1``.) + mid = 0.5 * (lb[bv] + ub[bv]) + child(lb, _set(ub, bv, mid), basis, ncuts, _rounds, bound) + child(_set(lb, bv, mid), ub, basis, ncuts, _rounds, bound) else: # Heap exhausted. Optimal only if the unresolved-node floor does not sit below # the incumbent (else there is space the engine could not rule out -> feasible @@ -673,6 +902,16 @@ def child(lb, ub, parent_basis, parent_cuts, rounds, parent_bound): gap = abs(obj - gbound) / (1 + abs(obj)) if status == "time_limit" and inc_x is None: obj = None + # Back to the model's own objective sense. Everything above is + # minimize-equivalent; for a MAXIMIZE model ``gbound`` is a valid lower bound on + # ``-f``, hence ``-gbound`` is a valid UPPER bound on ``f``, and the + # ``gbound <= inc_val`` invariant established above becomes + # ``bound >= objective`` — the maximize form of ``bound`` never crossing the + # incumbent. ``gap`` is a ratio of absolute differences and is sign-invariant. + if obj is not None: + obj = sgn * obj + if gbound is not None and np.isfinite(gbound): + gbound = sgn * gbound return LpSpatialResult( status=status, objective=obj, diff --git a/python/discopt/modeling/core.py b/python/discopt/modeling/core.py index 1cec6e40..67dd334c 100644 --- a/python/discopt/modeling/core.py +++ b/python/discopt/modeling/core.py @@ -106,6 +106,31 @@ def _lp_spatial_fallback_enabled() -> bool: ) +def _lp_spatial_mixed_fallback_enabled() -> bool: + """#860: extend the #844 no-incumbent fallback's *scope gate* to mixed-integer and + MAXIMIZE models. + + The engine itself serves those models unconditionally since #860 (see + ``lp_spatial_bb._is_in_scope``); this flag governs only whether the DEFAULT path + reserves budget for the fallback on them. That reserve is the sole risk of the + widening — an in-scope model hands 35% of its budget to a pass that may find + nothing — so it is a separate, measurable decision from the engine's capability. + + **Default OFF**; opt in with ``DISCOPT_LP_SPATIAL_MIXED=1``. Graduation requires + the corpus-wide differential panel of CLAUDE.md §5 (cert-clean AND net-positive) + over the mixed class, not just the absence of harm. + """ + import os as _os + + return _os.environ.get("DISCOPT_LP_SPATIAL_MIXED", "0") not in ( + "0", + "", + "false", + "False", + "off", + ) + + if TYPE_CHECKING: from discopt.modeling.indexed import IndexedParam, IndexedVar from discopt.modeling.sets import _SetBase @@ -4040,7 +4065,8 @@ def solve( # self-terminate within ``time_limit + ε`` instead of running to # XLA convergence after Python's budget is gone (issue #80). # #844: reserve a slice of the budget for the LP-per-node fallback below, but - # ONLY for models that engine can serve (pure-integer, minimize). Safe because + # ONLY for models that engine can serve (pure-integer, minimize; plus mixed / + # maximize when the #860 flag is on). Safe because # the instances at risk certify almost instantly -- nvs04 in 0.2 s, nvs06 in # 0.3 s of a 40 s budget -- so they finish long before the reduced primary # budget binds, while the instances this targets (tln4/tln5) burn 100% of the @@ -4062,7 +4088,11 @@ def solve( try: from discopt._jax.lp_spatial_bb import _is_in_scope - if _is_in_scope(self): + # #860: the engine now also serves mixed-integer and MAXIMIZE models, + # but whether the DEFAULT path should hand them 35% of its budget is a + # separate, panel-gated decision — hence the flag rather than the + # engine's own (already widened) gate. + if _is_in_scope(self, mixed=_lp_spatial_mixed_fallback_enabled()): _fb_reserve = 0.35 * time_limit # NOTE: scope is checked here, but whether the engine can actually # build its incremental structure is only known once it runs (see diff --git a/python/discopt/solver.py b/python/discopt/solver.py index 39cee58d..740dd46b 100644 --- a/python/discopt/solver.py +++ b/python/discopt/solver.py @@ -4638,7 +4638,11 @@ def _deadline_exhausted(floor: float = _DEADLINE_NODE_FLOOR_S) -> bool: # branches on integers/products and runs a feasibility-pump primal, closing # nvs17 to proven optimality. Opt-in via ``solve(lp_spatial=True)``; returns # ``None`` (falls through to the default path, no behavior change) for any model - # out of its scope (non-pure-integer, maximize, unbounded box) or on any error. + # out of its scope or on any error. Since #860 the scope is "at least one integer + # variable, either objective sense, any continuous mix" — mixed-integer and + # maximize models are served in minimize-equivalent space, and a partially + # infinite root box is accepted (root OBBT finitizes it; the cold builder drops + # any row it cannot make finite). if kwargs.get("lp_spatial", False): try: from discopt._jax.lp_spatial_bb import solve_lp_spatial_bb diff --git a/python/tests/test_incremental_mccormick_node.py b/python/tests/test_incremental_mccormick_node.py index d2c6d534..3ded8533 100644 --- a/python/tests/test_incremental_mccormick_node.py +++ b/python/tests/test_incremental_mccormick_node.py @@ -307,5 +307,74 @@ def test_incremental_full_solve_matches_cold(): assert r_fast.gap_certified and r_cold.gap_certified +# --------------------------------------------------------------------------- # +# Objective CONSTANT: the incremental path solves ``min c·x`` but the relaxation's +# objective is ``c·x + obj_offset``. Dropping the offset made the fast-path node +# bound differ from the cold build's by that constant — merely weak for a positive +# constant, but a dual bound ABOVE the true node optimum for a negative one, which +# is the false-fathom class. Found while widening the LP-spatial engine (#860). +# --------------------------------------------------------------------------- # + + +def _const_qcqp(const): + """Bilinear integer model whose objective carries an additive constant.""" + m = dm.Model("cq") + x = m.integer("x", lb=1, ub=4) + y = m.integer("y", lb=1, ub=4) + m.subject_to(x + y >= 6) + m.minimize(x * y + const) + return m + + +@pytest.mark.smoke +@pytest.mark.parametrize("const", [0.0, -100.0, 100.0]) +def test_incremental_node_bound_includes_objective_constant(const): + """Fail-before/pass-after: the fast path's node bound must equal the cold + build's, constant and all. Before the fix the fast path returned 8.0 for every + ``const`` (the offset-free ``min c·x``), so at ``const=-100`` it reported a lower + bound of +8.0 on a node whose true McCormick optimum is -92.0.""" + lb, ub = np.array([1.0, 1.0]), np.array([4.0, 4.0]) + os.environ["DISCOPT_INCREMENTAL_MC"] = "1" + try: + relaxer = MccormickLPRelaxer(_const_qcqp(const)) + assert relaxer._inc is not None, "test needs the incremental fast path engaged" + r_fast = relaxer.solve_at_node(lb, ub, want_marginals=True) + os.environ["DISCOPT_INCREMENTAL_MC"] = "0" + r_cold = MccormickLPRelaxer(_const_qcqp(const)).solve_at_node(lb, ub, want_marginals=True) + finally: + os.environ.pop("DISCOPT_INCREMENTAL_MC", None) + assert r_fast.lower_bound == pytest.approx(r_cold.lower_bound, abs=1e-6) + # The node's true McCormick optimum shifts exactly with the constant. + assert r_fast.lower_bound == pytest.approx(8.0 + const, abs=1e-6) + # ... and the certificate's safe bound rides the same origin as the bound beside + # it, so a consumer reading either one gets the same answer. + if r_fast.safe_bound is not None: + assert r_fast.safe_bound == pytest.approx(r_cold.safe_bound, abs=1e-6) + assert r_fast.safe_bound <= r_fast.lower_bound + 1e-6 + + +@pytest.mark.smoke +def test_incremental_solve_bound_matches_cold_builder_with_constant(): + """Same invariant one level down, at ``IncrementalMcCormickLP.solve`` itself: + its bound is the relaxation objective (``c·x + obj_offset``), the scale + ``MilpRelaxationModel.solve`` reports.""" + from discopt._jax.discretization import DiscretizationState + from discopt._jax.incremental_mccormick import IncrementalMcCormickLP + from discopt._jax.milp_relaxation import build_milp_relaxation + from discopt._jax.term_classifier import classify_nonlinear_terms + + m = _const_qcqp(-100.0) + lb, ub = np.array([1.0, 1.0]), np.array([4.0, 4.0]) + terms = classify_nonlinear_terms(m) + inc = IncrementalMcCormickLP(m, terms) + assert inc.ok + assert inc.obj_offset == pytest.approx(-100.0) + b_inc, _x, _basis = inc.solve(lb, ub) + relax, _info = build_milp_relaxation(m, terms, DiscretizationState(), bound_override=(lb, ub)) + relax._integrality = None + b_cold = relax.solve().bound + assert b_inc == pytest.approx(b_cold, abs=1e-6) + + if __name__ == "__main__": raise SystemExit(pytest.main([__file__, "-v"])) diff --git a/python/tests/test_lp_spatial_bb.py b/python/tests/test_lp_spatial_bb.py index a33c228d..03ae91bb 100644 --- a/python/tests/test_lp_spatial_bb.py +++ b/python/tests/test_lp_spatial_bb.py @@ -181,25 +181,78 @@ def test_no_false_optimum_on_dense_integer_quadratic(): # --------------------------------------------------------------------------- # -def test_out_of_scope_continuous_returns_none(): - """A model with a continuous variable is out of scope -> None (caller falls - back). The collapsed-box exactness argument needs every var integer.""" +def _mixed_model(): + """min x + y s.t. x*y >= 4, x continuous in [0,5], y integer in [0,5]. + + Brute force over the integer y: y=2 -> x=2 (4.0); y=3 -> x=4/3 (4.333); + y=4 -> x=1 (5.0); y=5 -> x=0.8 (5.8). Global optimum 4.0 at (2, 2).""" m = dm.Model("c") + m.continuous("x", lb=0, ub=5) + m.integer("y", lb=0, ub=5) + m.minimize(m._variables[0] + m._variables[1]) + m.subject_to(m._variables[0] * m._variables[1] >= 4) + return m + + +@pytest.mark.smoke +def test_mixed_integer_in_scope_and_correct(): + """#860: a mixed continuous/integer model is IN scope and must certify the true + optimum. Before #860 the engine declined it outright (``_is_in_scope`` required + every variable to be integer), so the whole class was unreachable.""" + r = solve_lp_spatial_bb(_mixed_model(), time_limit=20, gap_tolerance=1e-6) + assert r is not None + assert r.objective == pytest.approx(4.0, abs=1e-5) + assert r.bound is not None and r.bound <= r.objective + 1e-6 + + +@pytest.mark.smoke +def test_legacy_gate_still_declines_mixed(): + """``mixed=False`` restores the pre-#860 pure-integer/MINIMIZE gate, so a caller + can roll the widening out behind a flag without a second code path.""" + assert solve_lp_spatial_bb(_mixed_model(), time_limit=5, mixed=False) is None + + +@pytest.mark.smoke +def test_pure_continuous_stays_out_of_scope(): + """No integer variable -> None. The engine converges by branching the integers; + with none it is plain spatial bisection, which the default path already does.""" + m = dm.Model("cc") x = m.continuous("x", lb=0, ub=5) - y = m.integer("y", lb=0, ub=5) + y = m.continuous("y", lb=0, ub=5) m.minimize(x + y) m.subject_to(x * y >= 4) assert solve_lp_spatial_bb(m, time_limit=5) is None -def test_out_of_scope_maximize_returns_none(): - """The McCormick relaxation bound is only a valid *lower* bound for minimize.""" +@pytest.mark.smoke +def test_maximize_bound_is_a_valid_upper_bound(): + """#860: a MAXIMIZE model is in scope. The engine runs in minimize-equivalent + space, so the reported bound must come back as a valid UPPER bound (``bound >= + objective``) and the incumbent must not exceed the true optimum. + + max a + b s.t. a*b <= 6, a,b integer in [0,5]: (5,1)->6, (1,5)->6, (2,3)->5, + (5,0)->5, (0,5)->5. Optimum 6.""" m = dm.Model("mx") a = m.integer("a", lb=0, ub=5) b = m.integer("b", lb=0, ub=5) m.maximize(a + b) m.subject_to(a * b <= 6) - assert solve_lp_spatial_bb(m, time_limit=5) is None + r = solve_lp_spatial_bb(m, time_limit=20, gap_tolerance=1e-6) + assert r is not None + assert r.objective is not None and r.objective <= 6.0 + 1e-6 + assert r.bound is not None and r.bound >= r.objective - 1e-6 + assert r.objective == pytest.approx(6.0, abs=1e-5) + + +@pytest.mark.smoke +def test_maximize_legacy_gate_returns_none(): + """The pre-#860 gate declined maximize outright; ``mixed=False`` keeps that.""" + m = dm.Model("mx2") + a = m.integer("a", lb=0, ub=5) + b = m.integer("b", lb=0, ub=5) + m.maximize(a + b) + m.subject_to(a * b <= 6) + assert solve_lp_spatial_bb(m, time_limit=5, mixed=False) is None # --------------------------------------------------------------------------- # diff --git a/scratchpad/issue860_entry_experiment.py b/scratchpad/issue860_entry_experiment.py new file mode 100644 index 00000000..82136c5f --- /dev/null +++ b/scratchpad/issue860_entry_experiment.py @@ -0,0 +1,141 @@ +"""#860 entry experiment: is an LP-per-node McCormick relaxation usable on MIXED +(continuous+integer) and MAXIMIZE models at all? + +The issue names ``gastrans040`` / ``rsyn0805m04hfsg`` as gate probes; neither is +vendored in this repo (the full MINLPLib snapshot lives outside it and the +container has no network route to minlplib.org), so the experiment runs over the +REAL in-repo corpus instead -- which is dominated by exactly this class: 54 of the +66 ``minlplib_nl`` instances are mixed, and 4 are mixed + MAXIMIZE (``bchoco06/07/08``, +``syn05hfsg`` -- the same ``syn``/``rsyn`` family as the issue's maximize probe). + +For each instance it measures, at the ROOT box only (no engine work, no tree): + + bound the McCormick LP optimum over the mixed box (minimize-equivalent); + finite => a usable dual bound exists for this class. + round is the point (integers rounded, continuous at their LP values) + feasible for the TRUE nonlinear constraints? + complete is the point (integers rounded and FIXED, continuous re-solved by the + node LP) feasible for the TRUE nonlinear constraints? -- the natural + mixed-integer generalization of the pure-integer engine's + "collapse the box and the relaxation is exact" primal. + +Kill criterion (issue #860): if NO mixed instance yields a verified feasible point +at any budget, LP-per-node is not the lever for this class and the engine should +not be widened. +""" + +from __future__ import annotations + +import glob +import os +import sys +import time + +import numpy as np + +_DATA = os.path.join(os.path.dirname(__file__), "..", "python", "tests", "data") + + +def _load(path): + from discopt.modeling.core import from_nl + + return from_nl(path) + + +def probe(path, time_budget=60.0): + from discopt._jax.incremental_mccormick import IncrementalMcCormickLP + from discopt._jax.nlp_evaluator import NLPEvaluator + from discopt._jax.term_classifier import classify_nonlinear_terms + from discopt.modeling.core import ObjectiveSense, VarType + from discopt.solver import _check_constraint_feasibility, _infer_constraint_bounds + + m = _load(path) + n = len(m._variables) + is_int = np.array( + [v.var_type in (VarType.INTEGER, VarType.BINARY) for v in m._variables], dtype=bool + ) + sense = ( + "MAX" + if (m._objective is not None and m._objective.sense == ObjectiveSense.MAXIMIZE) + else "MIN" + ) + lb = np.array([float(v.lb) for v in m._variables]) + ub = np.array([float(v.ub) for v in m._variables]) + out = { + "name": os.path.basename(path)[:-3], + "n": n, + "nint": int(is_int.sum()), + "sense": sense, + "finite_box": bool(np.all(np.isfinite(lb)) and np.all(np.isfinite(ub))), + } + if not out["finite_box"]: + out["note"] = "infinite root box" + return out + + ev = NLPEvaluator(m) + cl, cu = _infer_constraint_bounds(m, ev) + + def feasible(x): + return bool(_check_constraint_feasibility(ev, x, cl, cu, tol=1e-6)) + + terms = classify_nonlinear_terms(m) + t0 = time.perf_counter() + inc = IncrementalMcCormickLP(m, terms, deadline=t0 + time_budget) + out["inc_ok"] = bool(inc.ok) + if not inc.ok: + out["note"] = "no incremental structure" + return out + b, x, _ = inc.solve(lb, ub) + out["root_lp_s"] = round(time.perf_counter() - t0, 3) + if b is None: + out["note"] = "root LP failed" + return out + out["bound"] = float(b) + + xr = np.array(x[:n], dtype=float) + xr[is_int] = np.round(xr[is_int]) + xr = np.minimum(np.maximum(xr, lb), ub) + out["round"] = feasible(xr) + + lo, hi = lb.copy(), ub.copy() + lo[is_int] = xr[is_int] + hi[is_int] = xr[is_int] + b2, x2, _ = inc.solve(lo, hi) + if x2 is not None: + xc = np.array(x2[:n], dtype=float) + xc[is_int] = xr[is_int] + xc = np.minimum(np.maximum(xc, lb), ub) + out["complete"] = feasible(xc) + if out["complete"]: + try: + out["obj"] = float(ev.evaluate_objective(xc)) + except Exception: + pass + else: + out["complete"] = False + out["total_s"] = round(time.perf_counter() - t0, 3) + return out + + +def main(): + only = sys.argv[1:] or None + paths = sorted(glob.glob(os.path.join(_DATA, "minlplib_nl", "*.nl"))) + sorted( + glob.glob(os.path.join(_DATA, "minlplib", "*.nl")) + ) + seen = set() + for p in paths: + name = os.path.basename(p)[:-3] + if name in seen: + continue + seen.add(name) + if only and name not in only: + continue + try: + r = probe(p) + except Exception as exc: # keep the sweep going; report the failure + r = {"name": name, "note": f"{type(exc).__name__}: {exc}"[:80]} + print(r, flush=True) + + +if __name__ == "__main__": + main() diff --git a/scratchpad/issue860_entry_experiment_v2.py b/scratchpad/issue860_entry_experiment_v2.py new file mode 100644 index 00000000..c8ae18bd --- /dev/null +++ b/scratchpad/issue860_entry_experiment_v2.py @@ -0,0 +1,180 @@ +"""#860 entry experiment, round 2 -- under the WIDENED gate. + +Round 1 (``issue860_entry_experiment.py``) measured the engine's *current* gate on +the in-repo corpus and found the binding blocker is not mixed-ness at all: + + * 30 / 71 mixed instances are rejected by ``_is_in_scope``'s all-variables-finite + box test (real MINLPs give continuous columns a ``+inf`` bound); + * 28 / 71 are rejected because ``IncrementalMcCormickLP`` declines (term types its + closed-form patch does not map); + * only 12 reach a root LP at all. + +Round 2 therefore measures the gate this issue proposes: + + * accept a partially infinite root box -- the COLD ``build_milp_relaxation`` + already drops any row whose payload is non-finite (``uniform_relax._Builder.add_row``), + so it self-guards; the INCREMENTAL patch does not, so it is used only when every + mapped product factor has finite bounds; + * accept any variable mix with at least one integer, and either objective sense. + +Reported per instance: + + path ``inc`` (incremental patch) or ``cold`` (per-node builder) + bound minimize-equivalent McCormick LP optimum at the root -- finite => usable + round integers rounded, continuous at their LP values: feasible for the TRUE + nonlinear constraints? + complete integers rounded and FIXED, continuous re-solved by the node LP: + feasible for the TRUE constraints? (the mixed generalization of the + pure-integer engine's collapsed-box primal) + +Kill criterion (issue #860): no verified feasible point anywhere in the mixed class +=> LP-per-node is not the lever and the engine should not be widened. +""" + +from __future__ import annotations + +import glob +import os +import sys +import time + +import numpy as np + +_DATA = os.path.join(os.path.dirname(__file__), "..", "python", "tests", "data") + + +def _root_lp(model, terms, lb, ub, budget): + """(path, bound, x) at the root box, preferring the incremental patch.""" + from discopt._jax.discretization import DiscretizationState + from discopt._jax.incremental_mccormick import IncrementalMcCormickLP + from discopt._jax.milp_relaxation import build_milp_relaxation + + t0 = time.perf_counter() + inc = IncrementalMcCormickLP(model, terms, deadline=t0 + budget) + if inc.ok: + cols = set() + for i, j in inc.bilinear: + cols.update((int(i), int(j))) + for i, _p in inc.monomial: + cols.add(int(i)) + for j, _a in inc.affine_square: + cols.add(int(j)) + idx = np.fromiter(cols, dtype=int) if cols else np.zeros(0, dtype=int) + if idx.size == 0 or (np.all(np.isfinite(lb[idx])) and np.all(np.isfinite(ub[idx]))): + b, x, _ = inc.solve(lb, ub) + return "inc", b, x, inc + relax, info = build_milp_relaxation( + model, terms, DiscretizationState(), bound_override=(lb, ub) + ) + if not relax._objective_bound_valid: + return "cold", None, None, None + relax._integrality = None + res = relax.solve() + if res is None or res.bound is None or res.x is None: + return "cold", None, None, None + return "cold", float(res.bound), np.asarray(res.x, dtype=float), None + + +def _resolve_fixed(model, terms, lo, hi, inc): + if inc is not None: + _b, x, _ = inc.solve(lo, hi) + return x + from discopt._jax.discretization import DiscretizationState + from discopt._jax.milp_relaxation import build_milp_relaxation + + relax, _info = build_milp_relaxation( + model, terms, DiscretizationState(), bound_override=(lo, hi) + ) + relax._integrality = None + res = relax.solve() + return None if res is None or res.x is None else np.asarray(res.x, dtype=float) + + +def probe(path, budget=60.0): + from discopt._jax.nlp_evaluator import NLPEvaluator + from discopt._jax.term_classifier import classify_nonlinear_terms + from discopt.modeling.core import ObjectiveSense, VarType, from_nl + from discopt.solver import _check_constraint_feasibility, _infer_constraint_bounds + + m = from_nl(path) + n = len(m._variables) + is_int = np.array( + [v.var_type in (VarType.INTEGER, VarType.BINARY) for v in m._variables], dtype=bool + ) + sense = ( + "MAX" + if (m._objective is not None and m._objective.sense == ObjectiveSense.MAXIMIZE) + else "MIN" + ) + lb = np.array([float(v.lb) for v in m._variables]) + ub = np.array([float(v.ub) for v in m._variables]) + out = { + "name": os.path.basename(path)[:-3], + "n": n, + "nint": int(is_int.sum()), + "sense": sense, + "inf_box": not bool(np.all(np.isfinite(lb)) and np.all(np.isfinite(ub))), + } + ev = NLPEvaluator(m) + cl, cu = _infer_constraint_bounds(m, ev) + + def feasible(x): + return bool(_check_constraint_feasibility(ev, x, cl, cu, tol=1e-6)) + + terms = classify_nonlinear_terms(m) + t0 = time.perf_counter() + kind, b, x, inc = _root_lp(m, terms, lb, ub, budget) + out["path"] = kind + out["root_s"] = round(time.perf_counter() - t0, 2) + if b is None or not np.isfinite(b): + out["note"] = "no finite root bound" + return out + out["bound"] = float(b) + + xr = np.array(x[:n], dtype=float) + xr[is_int] = np.round(xr[is_int]) + xr = np.minimum(np.maximum(xr, lb), ub) + out["round"] = feasible(xr) + + lo, hi = lb.copy(), ub.copy() + lo[is_int] = xr[is_int] + hi[is_int] = xr[is_int] + x2 = _resolve_fixed(m, terms, lo, hi, inc) + if x2 is None: + out["complete"] = False + else: + xc = np.array(x2[:n], dtype=float) + xc[is_int] = xr[is_int] + xc = np.minimum(np.maximum(xc, lb), ub) + out["complete"] = feasible(xc) + if out["complete"]: + try: + out["obj"] = float(ev.evaluate_objective(xc)) + except Exception: + pass + out["total_s"] = round(time.perf_counter() - t0, 2) + return out + + +def main(): + only = set(sys.argv[1:]) or None + paths = sorted(glob.glob(os.path.join(_DATA, "minlplib_nl", "*.nl"))) + sorted( + glob.glob(os.path.join(_DATA, "minlplib", "*.nl")) + ) + seen = set() + for p in paths: + name = os.path.basename(p)[:-3] + if name in seen: + continue + seen.add(name) + if only and name not in only: + continue + try: + r = probe(p) + except Exception as exc: + r = {"name": name, "note": f"{type(exc).__name__}: {exc}"[:90]} + print(r, flush=True) + + +if __name__ == "__main__": + main() diff --git a/scratchpad/panel860.py b/scratchpad/panel860.py new file mode 100644 index 00000000..6ffd3965 --- /dev/null +++ b/scratchpad/panel860.py @@ -0,0 +1,192 @@ +"""#860 §5 differential panel for the widened LP-per-node scope. + +Two panels, one process per solve (isolated: the solver stashes per-solve state on +the model and a crash in one instance must not take the sweep with it). + +PANEL A -- engine soundness on the widened class. Runs ``solve_lp_spatial_bb`` +directly on every in-repo instance now in scope (>=1 integer variable, either sense, +any continuous mix) and checks the certificate invariants that do not need an +external oracle: + + * every reported incumbent is INDEPENDENTLY re-verified feasible (fresh evaluator, + fresh constraint bounds) and its objective re-evaluated to match; + * ``bound <= objective`` for a minimize, ``bound >= objective`` for a maximize; + * the bound never crosses the best verified feasible point found by ANY run of that + instance (a dual bound above a known feasible objective is unsound, full stop); + * a ``status="optimal"`` run is never beaten by another run's verified incumbent + (that would be a false optimality certificate). + +PANEL B -- graduation gate for ``DISCOPT_LP_SPATIAL_MIXED`` (whether the DEFAULT +path reserves budget for the #844 no-incumbent fallback on mixed / maximize models). +Runs ``Model.solve`` with the flag off and on and requires BOTH: + + (1) cert-clean: no ``gap_certified`` True -> False, no objective drift beyond + tolerance, no incumbent better than the panel's best verified point, no + ``incumbent_verification_failed``; + (2) net-positive: incumbents gained where the default path had none, without a + broad wall-clock regression. + +Usage: python scratchpad/panel860.py [--tl SECONDS] [--panel A|B|AB] [--out FILE] +""" + +from __future__ import annotations + +import argparse +import glob +import json +import os +import subprocess +import sys + +_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +_CORPORA = [ + os.path.join(_ROOT, "python", "tests", "data", "minlplib_nl"), + os.path.join(_ROOT, "python", "tests", "data", "minlplib"), +] + +# --------------------------------------------------------------------------- # +# workers (run in a fresh interpreter per instance) +# --------------------------------------------------------------------------- # + +_COMMON = r''' +import os, sys, json, time, warnings +os.environ["JAX_PLATFORMS"] = "cpu" +os.environ["JAX_ENABLE_X64"] = "1" +warnings.filterwarnings("ignore") +import numpy as np +from discopt.modeling.core import from_nl, ObjectiveSense, VarType + +def load(path): + m = from_nl(path) + sense = -1.0 if (m._objective is not None + and m._objective.sense == ObjectiveSense.MAXIMIZE) else 1.0 + return m, sense + +def verify_point(model, x_flat): + """Independent re-verification: fresh evaluator, fresh constraint bounds. + Returns (feasible, true_objective) -- the objective in the MODEL's own sense.""" + from discopt._jax.nlp_evaluator import NLPEvaluator + from discopt.solver import _check_constraint_feasibility, _infer_constraint_bounds + ev = NLPEvaluator(model) + cl, cu = _infer_constraint_bounds(model, ev) + ok = bool(_check_constraint_feasibility(ev, np.asarray(x_flat, float), cl, cu, tol=1e-5)) + # NLPEvaluator returns the MINIMIZE-EQUIVALENT objective; undo that for reporting. + sgn = -1.0 if (model._objective is not None + and model._objective.sense == ObjectiveSense.MAXIMIZE) else 1.0 + obj = sgn * float(ev.evaluate_objective(np.asarray(x_flat, float))) + return ok, obj +''' + +_WORKER_ENGINE = ( + _COMMON + + r''' +path, tl = sys.argv[1], float(sys.argv[2]) +out = {} +try: + m, sgn = load(path) + from discopt._jax.lp_spatial_bb import _is_in_scope, solve_lp_spatial_bb + out["in_scope"] = bool(_is_in_scope(m)) + out["in_scope_legacy"] = bool(_is_in_scope(m, mixed=False)) + if out["in_scope"]: + t0 = time.perf_counter() + r = solve_lp_spatial_bb(m, time_limit=tl, gap_tolerance=1e-4) + out["wall"] = time.perf_counter() - t0 + out["declined"] = r is None + if r is not None: + out.update(status=r.status, obj=r.objective, bound=r.bound, + gap=r.gap, nodes=r.node_count) + if r.x is not None: + ok, trueobj = verify_point(from_nl(path), np.asarray(r.x, float)) + out["x_feasible"] = ok + out["x_true_obj"] = trueobj +except Exception as e: + out["error"] = f"{type(e).__name__}: {str(e)[:120]}" +print("RESULT" + json.dumps(out)) +''' +) + +_WORKER_SOLVE = ( + _COMMON + + r''' +path, tl, flag = sys.argv[1], float(sys.argv[2]), sys.argv[3] +os.environ["DISCOPT_LP_SPATIAL_MIXED"] = flag +out = {} +try: + m, sgn = load(path) + t0 = time.perf_counter() + r = m.solve(time_limit=tl) + out.update(wall=time.perf_counter() - t0, status=r.status, obj=r.objective, + bound=r.bound, gapc=bool(getattr(r, "gap_certified", False)), + nodes=getattr(r, "node_count", None), + ivf=bool(getattr(r, "incumbent_verification_failed", False))) + if r.x is not None: + names = [v.name for v in m._variables] + flat = np.concatenate([np.atleast_1d(np.asarray(r.x[k], float)).ravel() for k in names]) + ok, trueobj = verify_point(from_nl(path), flat) + out["x_feasible"] = ok + out["x_true_obj"] = trueobj +except Exception as e: + out["error"] = f"{type(e).__name__}: {str(e)[:120]}" +print("RESULT" + json.dumps(out)) +''' +) + + +def _run(worker, args, timeout): + try: + p = subprocess.run( + [sys.executable, "-c", worker, *args], + capture_output=True, + text=True, + timeout=timeout, + cwd=_ROOT, + ) + for ln in p.stdout.splitlines(): + if ln.startswith("RESULT"): + return json.loads(ln[6:]) + return {"error": "no_result", "stderr": p.stderr[-300:]} + except subprocess.TimeoutExpired: + return {"error": "harness_timeout"} + + +def instances(): + seen, out = set(), [] + for d in _CORPORA: + for p in sorted(glob.glob(os.path.join(d, "*.nl"))): + name = os.path.basename(p)[:-3] + if name in seen: + continue + seen.add(name) + out.append((name, p)) + return out + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("--tl", type=float, default=30.0) + ap.add_argument("--panel", default="AB") + ap.add_argument("--out", default="scratchpad/panel860_results.json") + ap.add_argument("--only", default="") + a = ap.parse_args() + only = set(a.only.split(",")) if a.only else None + hard = a.tl + 180.0 + rows = {} + inst = instances() + for i, (name, path) in enumerate(inst, 1): + if only and name not in only: + continue + row = {"name": name} + if "A" in a.panel: + row["engine"] = _run(_WORKER_ENGINE, [path, str(a.tl)], hard) + if "B" in a.panel: + row["off"] = _run(_WORKER_SOLVE, [path, str(a.tl), "0"], hard) + row["on"] = _run(_WORKER_SOLVE, [path, str(a.tl), "1"], hard) + rows[name] = row + print(f"[{i}/{len(inst)}] {json.dumps(row)}", flush=True) + with open(os.path.join(_ROOT, a.out), "w") as fh: + json.dump(rows, fh, indent=1) + print("done") + + +if __name__ == "__main__": + main() diff --git a/scratchpad/panel860_analyze.py b/scratchpad/panel860_analyze.py new file mode 100644 index 00000000..e00c9630 --- /dev/null +++ b/scratchpad/panel860_analyze.py @@ -0,0 +1,124 @@ +"""Score the #860 panel (see panel860.py) against the two CLAUDE.md §5 bars.""" + +from __future__ import annotations + +import argparse +import json +import math + +TOL = 1e-4 + + +def rel(a, b): + return abs(a - b) / (1.0 + abs(b)) + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("results", nargs="?", default="scratchpad/panel860_results.json") + a = ap.parse_args() + rows = json.load(open(a.results)) + + # Best independently-verified feasible point per instance, in the model's own + # sense, gathered across every run. A dual bound may never cross it. + best_min, best_max, sense = {}, {}, {} + for name, r in rows.items(): + for key in ("engine", "off", "on"): + run = r.get(key) or {} + if run.get("x_feasible") and run.get("x_true_obj") is not None: + v = float(run["x_true_obj"]) + best_min[name] = min(best_min.get(name, math.inf), v) + best_max[name] = max(best_max.get(name, -math.inf), v) + + # Objective sense: the engine reports a bound on the same side as the sense, so + # infer it from the run that has both. (The panel worker does not record it + # directly; ``bound <= obj`` => minimize, ``bound >= obj`` => maximize.) + for name, r in rows.items(): + e = r.get("engine") or {} + if e.get("obj") is not None and e.get("bound") is not None: + sense[name] = "max" if e["bound"] > e["obj"] + TOL else "min" + + unsound, false_opt, cert_reg, drift, ivf = [], [], [], [], [] + scope_new, engine_inc, engine_declined = [], [], [] + gains, losses, wall_off, wall_on = [], [], 0.0, 0.0 + + for name, r in rows.items(): + e = r.get("engine") or {} + if e.get("in_scope") and not e.get("in_scope_legacy"): + scope_new.append(name) + if e.get("in_scope"): + (engine_declined if e.get("declined") else engine_inc).append(name) + + # --- Panel A: engine soundness --------------------------------------- # + if e.get("obj") is not None: + if not e.get("x_feasible"): + unsound.append((name, "engine incumbent NOT independently feasible")) + elif e.get("x_true_obj") is not None and rel(e["x_true_obj"], e["obj"]) > TOL: + unsound.append( + (name, f"engine objective {e['obj']} != re-evaluated {e['x_true_obj']}") + ) + if e.get("bound") is not None: + s = sense.get(name, "min") + lo, hi = best_min.get(name), best_max.get(name) + if s == "min" and lo is not None and e["bound"] > lo + TOL * (1 + abs(lo)): + unsound.append((name, f"engine bound {e['bound']} > verified point {lo}")) + if s == "max" and hi is not None and e["bound"] < hi - TOL * (1 + abs(hi)): + unsound.append((name, f"engine bound {e['bound']} < verified point {hi}")) + if e.get("status") == "optimal" and e.get("obj") is not None: + s = sense.get(name, "min") + lo, hi = best_min.get(name), best_max.get(name) + if s == "min" and lo is not None and e["obj"] > lo + TOL * (1 + abs(lo)): + false_opt.append((name, f"claimed optimal {e['obj']}, better point {lo}")) + if s == "max" and hi is not None and e["obj"] < hi - TOL * (1 + abs(hi)): + false_opt.append((name, f"claimed optimal {e['obj']}, better point {hi}")) + + # --- Panel B: flag off vs on ----------------------------------------- # + off, on = r.get("off") or {}, r.get("on") or {} + if off.get("error") or on.get("error"): + continue + wall_off += float(off.get("wall") or 0.0) + wall_on += float(on.get("wall") or 0.0) + if on.get("ivf"): + ivf.append(name) + if off.get("gapc") and not on.get("gapc"): + cert_reg.append(name) + if off.get("obj") is None and on.get("obj") is not None: + gains.append((name, on["obj"], bool(on.get("x_feasible")))) + if off.get("obj") is not None and on.get("obj") is None: + losses.append(name) + if off.get("obj") is not None and on.get("obj") is not None: + if rel(on["obj"], off["obj"]) > 1e-3: + drift.append((name, off["obj"], on["obj"])) + for key, run in (("off", off), ("on", on)): + if run.get("obj") is not None and run.get("x_feasible") is False: + unsound.append((name, f"{key}: reported incumbent NOT feasible")) + + print(f"instances : {len(rows)}") + print(f"newly in scope (#860) : {len(scope_new)}") + print(f" engine returned a result : {len(engine_inc)}") + print(f" engine declined : {len(engine_declined)}") + print() + print("── Panel A: engine soundness (bar 1) ──") + print(f" unsound bounds / incumbents : {len(unsound)}") + for n, why in unsound: + print(f" {n}: {why}") + print(f" false optimality certificates: {len(false_opt)}") + for n, why in false_opt: + print(f" {n}: {why}") + print() + print("── Panel B: DISCOPT_LP_SPATIAL_MIXED off vs on ──") + print(f" cert regressions (True->False): {len(cert_reg)} {cert_reg}") + print(f" incumbent-verification failed : {len(ivf)} {ivf}") + print(f" objective drift > 1e-3 : {len(drift)}") + for n, o, w in drift: + print(f" {n}: {o} -> {w}") + print(f" incumbents GAINED : {len(gains)}") + for n, o, ok in gains: + print(f" {n}: {o} (independently feasible: {ok})") + print(f" incumbents LOST : {len(losses)} {losses}") + ratio = wall_on / max(wall_off, 1e-9) + print(f" total wall off={wall_off:.1f}s on={wall_on:.1f}s ratio={ratio:.3f}") + + +if __name__ == "__main__": + main() From 157a74f93d6089533eebc289213183a830386dba Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 26 Jul 2026 15:26:58 +0000 Subject: [PATCH 02/12] test(#860): keep the unresolved-child injection signature-agnostic; cover maximize soundness and infinite root boxes - test_844_unresolved_child_soundness: the injected `_relax_bound` now takes `**kw` so it keeps exercising the unresolved-child path instead of failing on the new `deadline` argument. - test_lp_spatial_bb: brute-force maximize soundness (incumbent never above the true max, upper bound never below it) and a mixed model with an infinite continuous upper bound, which the pre-#860 gate declined outright. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_019bsy1zfNW8H1hZGg9SKfPu --- .../test_844_unresolved_child_soundness.py | 7 ++- python/tests/test_lp_spatial_bb.py | 46 +++++++++++++++++++ 2 files changed, 51 insertions(+), 2 deletions(-) diff --git a/python/tests/test_844_unresolved_child_soundness.py b/python/tests/test_844_unresolved_child_soundness.py index d7dffb70..67f00357 100644 --- a/python/tests/test_844_unresolved_child_soundness.py +++ b/python/tests/test_844_unresolved_child_soundness.py @@ -70,12 +70,15 @@ def test_failed_child_relaxation_never_yields_a_false_certificate(monkeypatch): real = lpsb._relax_bound state = {"calls": 0, "failures": 0} - def flaky(model, terms, lb, ub): + def flaky(model, terms, lb, ub, **kw): + # ``**kw`` passes through whatever optional arguments the real + # ``_relax_bound`` grows (``deadline`` since #860) so the injection keeps + # testing the unresolved-child path rather than failing on a signature. state["calls"] += 1 if state["calls"] > 3: # let the root and first nodes succeed state["failures"] += 1 return None - return real(model, terms, lb, ub) + return real(model, terms, lb, ub, **kw) monkeypatch.setattr(lpsb, "_relax_bound", flaky) res = lpsb.solve_lp_spatial_bb(_model(), time_limit=30.0, gap_tolerance=1e-4) diff --git a/python/tests/test_lp_spatial_bb.py b/python/tests/test_lp_spatial_bb.py index 03ae91bb..99a971f9 100644 --- a/python/tests/test_lp_spatial_bb.py +++ b/python/tests/test_lp_spatial_bb.py @@ -255,6 +255,52 @@ def test_maximize_legacy_gate_returns_none(): assert solve_lp_spatial_bb(m, time_limit=5, mixed=False) is None +@pytest.mark.smoke +def test_maximize_never_overstates_the_optimum(): + """#860 soundness for the maximize half, against brute force. + + max a*b - 3a s.t. a + b <= 6, a,b integer in [0,5]. The incumbent may never + EXCEED the true maximum (that would be a false primal) and the reported bound is + an upper bound, so it may never fall BELOW the true maximum.""" + m = dm.Model("mx3") + a = m.integer("a", lb=0, ub=5) + b = m.integer("b", lb=0, ub=5) + m.maximize(a * b - 3 * a) + m.subject_to(a + b <= 6) + true_max = max(i * j - 3 * i for i in range(6) for j in range(6) if i + j <= 6) + r = solve_lp_spatial_bb(m, time_limit=20, gap_tolerance=1e-6) + assert r is not None + if r.objective is not None: + assert r.objective <= true_max + 1e-6, "incumbent beat the true maximum" + if r.bound is not None: + assert r.bound >= true_max - 1e-6, "upper bound fell below the true maximum" + assert r.status == "optimal" and r.objective == pytest.approx(true_max, abs=1e-6) + + +@pytest.mark.smoke +def test_infinite_root_box_is_accepted_not_declined(): + """#860: a root box with an infinite endpoint is no longer an outright decline. + + Real MINLPs leave continuous columns unbounded above — 31 of the 71 mixed + instances in the in-repo corpus do — and the old all-variables-finite gate made + the whole class unreachable. Root OBBT plus the cold builder's own non-finite-row + guard handle it; the incremental patch (which has no such guard) declines itself. + + min x + y s.t. x + y >= 6, x*y >= 6; x continuous in [0, inf), y integer in + [1,5]. The linear row forces the objective to 6, attained at (3,3) where + x*y = 9 >= 6.""" + m = dm.Model("infbox") + x = m.continuous("x", lb=0.0, ub=float("inf")) + y = m.integer("y", lb=1, ub=5) + m.minimize(x + y) + m.subject_to(x + y >= 6) + m.subject_to(x * y >= 6) + r = solve_lp_spatial_bb(m, time_limit=20, gap_tolerance=1e-6) + assert r is not None, "an infinite root endpoint must no longer decline the solve" + assert r.objective == pytest.approx(6.0, abs=1e-5) + assert r.bound is not None and r.bound <= r.objective + 1e-6 + + # --------------------------------------------------------------------------- # # correctness: matches brute force exactly, and proves optimality # --------------------------------------------------------------------------- # From b32c6a3418a03b09540f9b5f438374919961cb03 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 26 Jul 2026 15:28:41 +0000 Subject: [PATCH 03/12] fix(#860): pick the tighter dual bound by sense when merging the fallback result MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The #844 fallback merged its dual bound with `max`, which is correct for a minimize (larger lower bound is tighter) but keeps the LOOSER of two valid upper bounds for a maximize — reachable now that the engine serves maximize models. Sound either way; this keeps the intent. Also covers the new flag: the mixed model's result is unchanged by DISCOPT_LP_SPATIAL_MIXED under either fallback setting, the flag defaults off, and the engine's (widened) gate and the reserve gate can disagree — which is the point of separating them. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_019bsy1zfNW8H1hZGg9SKfPu --- python/discopt/modeling/core.py | 19 ++++++-- python/tests/test_844_lp_spatial_fallback.py | 48 ++++++++++++++++++-- 2 files changed, 60 insertions(+), 7 deletions(-) diff --git a/python/discopt/modeling/core.py b/python/discopt/modeling/core.py index 67dd334c..b8d176ce 100644 --- a/python/discopt/modeling/core.py +++ b/python/discopt/modeling/core.py @@ -4208,11 +4208,22 @@ def solve( result.objective = _fb.objective result.x = _unpack_solution(self, np.asarray(_fb.x)) # Keep the TIGHTER of the two dual bounds and never claim a - # certificate the fallback did not actually prove. + # certificate the fallback did not actually prove. Which one is + # tighter depends on the sense: for a minimize the dual bound is a + # LOWER bound (larger is tighter), for a maximize it is an UPPER + # bound (smaller is tighter) — #860, now that the engine serves + # maximize models. Taking ``max`` unconditionally would still be + # sound (both are valid upper bounds) but would keep the looser one. if _fb.bound is not None: - result.bound = ( - _fb.bound if result.bound is None else max(result.bound, _fb.bound) - ) + if result.bound is None: + result.bound = _fb.bound + elif ( + self._objective is not None + and self._objective.sense == ObjectiveSense.MAXIMIZE + ): + result.bound = min(result.bound, _fb.bound) + else: + result.bound = max(result.bound, _fb.bound) result.gap = _fb.gap result.gap_certified = _fb.status == "optimal" result.node_count = (result.node_count or 0) + _fb.node_count diff --git a/python/tests/test_844_lp_spatial_fallback.py b/python/tests/test_844_lp_spatial_fallback.py index e3b06256..cdc1d2ba 100644 --- a/python/tests/test_844_lp_spatial_fallback.py +++ b/python/tests/test_844_lp_spatial_fallback.py @@ -64,7 +64,8 @@ def _pure_integer_min() -> dm.Model: def _mixed() -> dm.Model: - """Out of scope: has a continuous variable.""" + """Has a continuous variable: out of the fallback's scope while the #860 mixed + flag is off (its default), in scope when it is on.""" m = dm.Model("mixed") x = m.integer("x", lb=0, ub=5) c = m.continuous("c", lb=0, ub=5) @@ -73,6 +74,16 @@ def _mixed() -> dm.Model: return m +def _pure_continuous() -> dm.Model: + """Out of the engine's scope under EITHER flag: no integer variable at all.""" + m = dm.Model("pure_cont") + a = m.continuous("a", lb=0, ub=5) + b = m.continuous("b", lb=0, ub=5) + m.minimize(3 * a + b) + m.subject_to(a + b >= 2) + return m + + def test_flag_defaults_to_on(fb): """Default-ON after the graduation panel (3 gains, 0 lost incumbents, 0 certification regressions, 0 overshoots). ``=0`` remains the opt-out.""" @@ -99,11 +110,42 @@ def test_solve_with_an_incumbent_is_unchanged(fb, flag): @pytest.mark.parametrize("flag", ["0", "1"]) def test_out_of_scope_model_is_unaffected(fb, flag): - """A mixed model can never reach the engine, so its result and its full time - budget are untouched regardless of the flag. Optimum is 2.0 at x=0, c=2.""" + """A model with no integer variable can never reach the engine under either + flag, so its result and its full time budget are untouched. Optimum 2.0.""" + fb(flag) + r = _pure_continuous().solve(time_limit=60) + assert r.objective == pytest.approx(2.0, abs=1e-4) + + +@pytest.mark.parametrize("flag", ["0", "1"]) +@pytest.mark.parametrize("mixed", ["0", "1"]) +def test_mixed_model_result_is_unchanged_by_the_860_flag(fb, monkeypatch, flag, mixed): + """#860: turning the mixed flag on may change *how* a mixed model is solved (the + default path hands 35% of its budget to the fallback) but never *what* it + returns. Optimum is 2.0 at x=0, c=2; the dual bound must never cross it.""" fb(flag) + monkeypatch.setenv("DISCOPT_LP_SPATIAL_MIXED", mixed) r = _mixed().solve(time_limit=60) assert r.objective == pytest.approx(2.0, abs=1e-4) + if r.bound is not None: + assert r.bound <= r.objective + 1e-6, "UNSOUND: bound above incumbent" + + +def test_860_mixed_flag_defaults_to_off_and_gates_the_reserve(monkeypatch): + """The #860 widening reaches the DEFAULT path only through this flag, which is + off until its graduation panel. The engine's own gate is already widened, so the + two must be able to disagree — that separation is the point of the flag.""" + from discopt._jax.lp_spatial_bb import _is_in_scope + from discopt.modeling.core import _lp_spatial_mixed_fallback_enabled + + monkeypatch.delenv("DISCOPT_LP_SPATIAL_MIXED", raising=False) + assert _lp_spatial_mixed_fallback_enabled() is False + monkeypatch.setenv("DISCOPT_LP_SPATIAL_MIXED", "1") + assert _lp_spatial_mixed_fallback_enabled() is True + + m = _mixed() + assert _is_in_scope(m) is True # engine: widened unconditionally + assert _is_in_scope(m, mixed=False) is False # reserve gate while the flag is off def test_fallback_never_breaks_a_solve(fb): From 4f8750403dbb2c157c3ea487e89db0efc8e963b9 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 26 Jul 2026 15:30:22 +0000 Subject: [PATCH 04/12] chore(#860): satisfy mypy on the optional objective in the sense guard Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_019bsy1zfNW8H1hZGg9SKfPu --- python/discopt/_jax/lp_spatial_bb.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/python/discopt/_jax/lp_spatial_bb.py b/python/discopt/_jax/lp_spatial_bb.py index fc9ea1e1..47cd3622 100644 --- a/python/discopt/_jax/lp_spatial_bb.py +++ b/python/discopt/_jax/lp_spatial_bb.py @@ -363,8 +363,10 @@ def solve_lp_spatial_bb( # bound on ``sgn * f``, every verified incumbent is ``sgn * f``, and the whole # engine (heap order, incumbent comparison, fathoming, gap) runs unchanged in that # space. ``sgn`` appears in exactly ONE place: the reported objective/bound at the - # exit. Applying it anywhere else would double-negate. - sgn = -1.0 if model._objective.sense == ObjectiveSense.MAXIMIZE else 1.0 + # exit. Applying it anywhere else would double-negate. (``_objective`` is not None + # here — ``_is_in_scope`` above rejects a model without one.) + _obj = model._objective + sgn = -1.0 if (_obj is not None and _obj.sense == ObjectiveSense.MAXIMIZE) else 1.0 t0 = time.perf_counter() From 295b12232ba2edd822c43997999cae9905e97c62 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 26 Jul 2026 15:34:34 +0000 Subject: [PATCH 05/12] fix(#860): carry each node's product map with its own solution on the cold path MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The cold path re-runs build_milp_relaxation at every node and the lift is box-dependent, so a node's column count can differ from the root's. `info` was captured once at the root and read against every node's `x`, which indexes past the end of that node's solution vector — measured on st_e38: root aux column 18 against a 17-column node LP, IndexError. The caller swallowed it as "engine failed" and silently fell back, so the engine was simply unavailable on such models. `relax`/`node_relax` now return the product map of the LP that produced the solution, and it travels with the node through the frontier. Pre-existing on the cold path; st_e38 reaches it only because #860 brought mixed models into scope. After the fix st_e38 returns feasible 7197.73 with bound 6795.05 in 1.9s. Regression: test_cold_path_product_map_travels_with_its_solution. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_019bsy1zfNW8H1hZGg9SKfPu --- python/discopt/_jax/lp_spatial_bb.py | 52 +++++++++++++++++++--------- python/tests/test_lp_spatial_bb.py | 22 ++++++++++++ 2 files changed, 58 insertions(+), 16 deletions(-) diff --git a/python/discopt/_jax/lp_spatial_bb.py b/python/discopt/_jax/lp_spatial_bb.py index 47cd3622..0ce3ef61 100644 --- a/python/discopt/_jax/lp_spatial_bb.py +++ b/python/discopt/_jax/lp_spatial_bb.py @@ -489,11 +489,21 @@ def _pt_feasible(xr: np.ndarray) -> bool: "requires it (the per-node cold build cannot serve a bounded budget)" ) return None + # ``relax`` returns ``(bound, x, basis, info)``. The product map travels WITH the + # solution it describes: on the incremental path the lifted layout is fixed once + # and ``info`` is a constant, but the cold path re-runs ``build_milp_relaxation`` + # at every node and the lift is box-dependent, so a node's column count can differ + # from the root's. Reusing the root map against a node's ``x`` indexes past the + # end of that node's solution vector (measured on st_e38: root aux column 18 vs a + # 17-column node LP -> IndexError, which the caller swallowed as "engine failed" + # and silently fell back). A product map is only meaningful for the exact LP it + # came from. if _inc.ok: info = {"bilinear": _inc.bilinear, "monomial": _inc.monomial} def relax(lb, ub, basis): - return _inc.solve(lb, ub, in_basis=basis) + b_, x_, bas = _inc.solve(lb, ub, in_basis=basis) + return b_, x_, bas, info else: _r0 = _relax_bound(model, terms, lb0, ub0, deadline=t0 + time_limit) if _r0 is None: @@ -502,7 +512,7 @@ def relax(lb, ub, basis): def relax(lb, ub, basis): c = _relax_bound(model, terms, lb, ub, deadline=t0 + time_limit) - return (c[0], c[1], None) if c is not None else (None, None, None) + return (c[0], c[1], None, c[2]) if c is not None else (None, None, None, None) # Branch-and-cut: separate integer cuts (GMI + complemented-MIR, product aux # vars marked integer) at each node and re-solve, tightening the node bound @@ -516,7 +526,11 @@ def relax(lb, ub, basis): def node_relax(lb, ub, basis, inherited, rounds): """Solve the node LP with inherited cuts, then run ``rounds`` of cut separation (add only bound-improving cuts), returning - ``(bound, x, basis, cuts, verdict)``. + ``(bound, x, basis, cuts, verdict, info)``. + + ``info`` is the product map of the LP that produced ``x`` — see the note on + ``relax`` above; it is carried through the frontier so the spatial-branching + test always reads the map belonging to the solution in hand. ``verdict`` distinguishes the two very different reasons a node LP can come back with no bound, which the caller MUST NOT conflate: @@ -532,18 +546,19 @@ def node_relax(lb, ub, basis, inherited, rounds): must fold such a node into ``unresolved_lb`` instead. """ if not cut_enabled: - b_, x_, bas = relax(lb, ub, basis) + b_, x_, bas, info_ = relax(lb, ub, basis) # Cold path: ``_relax_bound`` collapses every failure mode into ``None`` # and cannot prove infeasibility, so the only sound reading is # "unresolved". This is conservative in the safe direction — it can cost # an optimality certificate, never create a false one. - return b_, x_, bas, (), ("optimal" if b_ is not None else "unresolved") + verdict = "optimal" if b_ is not None else "unresolved" + return b_, x_, bas, (), verdict, info_ cuts = list(inherited) A, b, bounds = _inc.assemble(lb, ub, cuts) _st, b_, x_, bas, _farkas = _inc.solve_assembled_full(A, b, bounds, in_basis=basis) if b_ is None: _verdict = "fathom" if (_st == "infeasible" and _farkas) else "unresolved" - return None, None, None, tuple(cuts), _verdict + return None, None, None, tuple(cuts), _verdict, info for _r in range(rounds): if len(cuts) >= _MAX_INHERITED_CUTS: break @@ -559,9 +574,9 @@ def node_relax(lb, ub, basis, inherited, rounds): b_, x_, bas = nb, nx, nbas if not improved: break - return b_, x_, bas, tuple(cuts), "optimal" + return b_, x_, bas, tuple(cuts), "optimal", info - root_b, root_x, root_basis, root_cuts, _root_verdict = node_relax( + root_b, root_x, root_basis, root_cuts, _root_verdict, root_info = node_relax( lb0, ub0, None, (), root_cut_rounds ) if root_b is None: @@ -569,8 +584,11 @@ def node_relax(lb, ub, basis, inherited, rounds): inc_val = float("inf") inc_x: Optional[np.ndarray] = None - # frontier entries: (bound, tiebreak, lb, ub, x, warm_basis, inherited_cuts) - heap = [(root_b, 0, lb0, ub0, root_x, root_basis, root_cuts)] + # frontier entries: + # (bound, tiebreak, lb, ub, x, warm_basis, inherited_cuts, info) + # ``info`` is the product map of the LP that produced this node's ``x`` — see + # ``relax``; it must travel with the solution, not be read from the root. + heap = [(root_b, 0, lb0, ub0, root_x, root_basis, root_cuts, root_info)] counter = 1 nodes = 0 @@ -667,7 +685,7 @@ def complete(lb_c, ub_c, xhat): hi[fix] = xr[fix] if np.any(lo > hi + 1e-9): return None - _b, xx, _bas = relax(lo, hi, None) + _b, xx, _bas, _inf = relax(lo, hi, None) if xx is None: return None xc = np.asarray(xx[:n], dtype=float).copy() @@ -685,7 +703,7 @@ def dive(lb_d, ub_d): # the engine can blow past ``time_limit`` by an order of magnitude. if (time.perf_counter() - t0) >= time_limit: return None - b_, xx, _bas = relax(lo, hi, None) + b_, xx, _bas, _inf = relax(lo, hi, None) if b_ is None: return None free = [(abs(xx[i] - round(xx[i])), i) for i in INT if hi[i] - lo[i] > 0.5] @@ -766,9 +784,11 @@ def child(lb, ub, parent_basis, parent_cuts, rounds, parent_bound): nonlocal counter, unresolved_lb if np.any(lb > ub + 1e-9): return None # empty integer box: genuinely nothing here, safe to drop - b_, x_, basis_, cuts_, verdict = node_relax(lb, ub, parent_basis, parent_cuts, rounds) + b_, x_, basis_, cuts_, verdict, info_ = node_relax( + lb, ub, parent_basis, parent_cuts, rounds + ) if b_ is not None and b_ < inc_val - 1e-9: - heapq.heappush(heap, (b_, counter, lb, ub, x_, basis_, cuts_)) + heapq.heappush(heap, (b_, counter, lb, ub, x_, basis_, cuts_, info_)) counter += 1 elif b_ is None and verdict != "fathom": # The child's relaxation gave no certified verdict, so its subtree is NOT @@ -803,7 +823,7 @@ def child(lb, ub, parent_basis, parent_cuts, rounds, parent_bound): if (time.perf_counter() - t0) >= time_limit or nodes >= max_nodes: status = "time_limit" break - bound, _, lb, ub, x, basis, ncuts = heapq.heappop(heap) + bound, _, lb, ub, x, basis, ncuts, ninfo = heapq.heappop(heap) nodes += 1 # Global lower bound = smallest frontier bound (this popped node, best-first) # capped by any unresolved-node floor. Fathoming/gap tests use this, never the @@ -852,7 +872,7 @@ def child(lb, ub, parent_basis, parent_cuts, rounds, parent_bound): # branching the engine could not perform. The collapse test spans EVERY # variable, continuous ones included -- a mixed node with a live continuous # dimension is never an exact leaf, however tight its products look. - bv = _worst_product_var(x, info, ub - lb, is_int) + bv = _worst_product_var(x, ninfo, ub - lb, is_int) if bv is None: consider(verify(x[:n])) if _has_cont: diff --git a/python/tests/test_lp_spatial_bb.py b/python/tests/test_lp_spatial_bb.py index 99a971f9..44b23729 100644 --- a/python/tests/test_lp_spatial_bb.py +++ b/python/tests/test_lp_spatial_bb.py @@ -15,6 +15,7 @@ from discopt._jax.lp_spatial_bb import _separate_node_cuts, solve_lp_spatial_bb # noqa: E402 _DATA = os.path.join(os.path.dirname(__file__), "data", "minlplib") +_DATA_NL = os.path.join(os.path.dirname(__file__), "data", "minlplib_nl") def _assemble_node_lp(ux=6, uy=5, uz=4): @@ -301,6 +302,27 @@ def test_infinite_root_box_is_accepted_not_declined(): assert r.bound is not None and r.bound <= r.objective + 1e-6 +@pytest.mark.smoke +def test_cold_path_product_map_travels_with_its_solution(): + """Fail-before/pass-after: on the COLD path the lifted layout is rebuilt at every + node and is box-dependent, so a node's column count can differ from the root's. + Reading the ROOT product map against a node's solution indexed past the end of + that vector — on ``st_e38``, root aux column 18 against a 17-column node LP — + raising ``IndexError``, which the caller swallowed as "engine failed" and silently + fell back to the default path. The map now travels with the solution it + describes, so the engine returns a sound result instead of dying. + + (Pre-existing on the cold path; ``st_e38`` reaches it only because #860 brought + mixed models into scope.)""" + path = os.path.join(_DATA_NL, "st_e38.nl") + if not os.path.exists(path): + pytest.skip("st_e38 not vendored") + r = solve_lp_spatial_bb(dm.from_nl(path), time_limit=20) + assert r is not None, "engine must not decline (an IndexError here is the bug)" + if r.objective is not None: + assert r.bound is not None and r.bound <= r.objective + 1e-6 + + # --------------------------------------------------------------------------- # # correctness: matches brute force exactly, and proves optimality # --------------------------------------------------------------------------- # From 196709cc5c0121dc771d72e061ea60df91c5d60b Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 26 Jul 2026 15:37:38 +0000 Subject: [PATCH 06/12] chore(#860): batch the panel workers so the differential fits in a run, not a day MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit One process per solve is the cleanest isolation, but the fixed startup cost (JAX import + parse + evaluator build) dominated at ~2.3 min per instance against ~1s of actual solving — a 10-hour panel. The worker now takes a batch of instances and emits one RESULT line each; batches stay small so a crash or a leak costs only that batch, and every instance still builds a fresh model. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_019bsy1zfNW8H1hZGg9SKfPu --- scratchpad/panel860.py | 153 ++++++++++++++++++++++++----------------- 1 file changed, 90 insertions(+), 63 deletions(-) diff --git a/scratchpad/panel860.py b/scratchpad/panel860.py index 6ffd3965..1d4f1133 100644 --- a/scratchpad/panel860.py +++ b/scratchpad/panel860.py @@ -79,74 +79,91 @@ def verify_point(model, x_flat): _WORKER_ENGINE = ( _COMMON - + r''' -path, tl = sys.argv[1], float(sys.argv[2]) -out = {} -try: - m, sgn = load(path) - from discopt._jax.lp_spatial_bb import _is_in_scope, solve_lp_spatial_bb - out["in_scope"] = bool(_is_in_scope(m)) - out["in_scope_legacy"] = bool(_is_in_scope(m, mixed=False)) - if out["in_scope"]: - t0 = time.perf_counter() - r = solve_lp_spatial_bb(m, time_limit=tl, gap_tolerance=1e-4) - out["wall"] = time.perf_counter() - t0 - out["declined"] = r is None - if r is not None: - out.update(status=r.status, obj=r.objective, bound=r.bound, - gap=r.gap, nodes=r.node_count) - if r.x is not None: - ok, trueobj = verify_point(from_nl(path), np.asarray(r.x, float)) - out["x_feasible"] = ok - out["x_true_obj"] = trueobj -except Exception as e: - out["error"] = f"{type(e).__name__}: {str(e)[:120]}" -print("RESULT" + json.dumps(out)) -''' + + r""" +tl = float(sys.argv[1]) +for path in sys.argv[2:]: + out = {"path": path} + try: + m, sgn = load(path) + from discopt._jax.lp_spatial_bb import _is_in_scope, solve_lp_spatial_bb + out["in_scope"] = bool(_is_in_scope(m)) + out["in_scope_legacy"] = bool(_is_in_scope(m, mixed=False)) + if out["in_scope"]: + t0 = time.perf_counter() + r = solve_lp_spatial_bb(m, time_limit=tl, gap_tolerance=1e-4) + out["wall"] = time.perf_counter() - t0 + out["declined"] = r is None + if r is not None: + out.update(status=r.status, obj=r.objective, bound=r.bound, + gap=r.gap, nodes=r.node_count) + if r.x is not None: + ok, trueobj = verify_point(from_nl(path), np.asarray(r.x, float)) + out["x_feasible"] = ok + out["x_true_obj"] = trueobj + except Exception as e: + out["error"] = f"{type(e).__name__}: {str(e)[:120]}" + print("RESULT" + json.dumps(out), flush=True) +""" ) _WORKER_SOLVE = ( _COMMON - + r''' -path, tl, flag = sys.argv[1], float(sys.argv[2]), sys.argv[3] + + r""" +tl, flag = float(sys.argv[1]), sys.argv[2] os.environ["DISCOPT_LP_SPATIAL_MIXED"] = flag -out = {} -try: - m, sgn = load(path) - t0 = time.perf_counter() - r = m.solve(time_limit=tl) - out.update(wall=time.perf_counter() - t0, status=r.status, obj=r.objective, - bound=r.bound, gapc=bool(getattr(r, "gap_certified", False)), - nodes=getattr(r, "node_count", None), - ivf=bool(getattr(r, "incumbent_verification_failed", False))) - if r.x is not None: - names = [v.name for v in m._variables] - flat = np.concatenate([np.atleast_1d(np.asarray(r.x[k], float)).ravel() for k in names]) - ok, trueobj = verify_point(from_nl(path), flat) - out["x_feasible"] = ok - out["x_true_obj"] = trueobj -except Exception as e: - out["error"] = f"{type(e).__name__}: {str(e)[:120]}" -print("RESULT" + json.dumps(out)) -''' +for path in sys.argv[3:]: + out = {"path": path} + try: + m, sgn = load(path) + t0 = time.perf_counter() + r = m.solve(time_limit=tl) + out.update(wall=time.perf_counter() - t0, status=r.status, obj=r.objective, + bound=r.bound, gapc=bool(getattr(r, "gap_certified", False)), + nodes=getattr(r, "node_count", None), + ivf=bool(getattr(r, "incumbent_verification_failed", False))) + if r.x is not None: + names = [v.name for v in m._variables] + flat = np.concatenate([np.atleast_1d(np.asarray(r.x[k], float)).ravel() for k in names]) + ok, trueobj = verify_point(from_nl(path), flat) + out["x_feasible"] = ok + out["x_true_obj"] = trueobj + except Exception as e: + out["error"] = f"{type(e).__name__}: {str(e)[:120]}" + print("RESULT" + json.dumps(out), flush=True) +""" ) -def _run(worker, args, timeout): +def _run_batch(worker, prefix, paths, timeout): + """Run a BATCH of instances in one interpreter and return {path: result}. + + One process per solve is the cleanest isolation, but the fixed startup cost + (JAX import + parse + evaluator build) dominated at ~2.3 min per instance + against ~1 s of actual solving, which made the panel a 10-hour run. Batching + amortizes it; the batch is kept small so a crash or a memory leak costs only + that batch, and every instance still builds a FRESH model. + """ + out = {p: {"error": "no_result"} for p in paths} try: - p = subprocess.run( - [sys.executable, "-c", worker, *args], + proc = subprocess.run( + [sys.executable, "-c", worker, *prefix, *paths], capture_output=True, text=True, timeout=timeout, cwd=_ROOT, ) - for ln in p.stdout.splitlines(): + for ln in proc.stdout.splitlines(): if ln.startswith("RESULT"): - return json.loads(ln[6:]) - return {"error": "no_result", "stderr": p.stderr[-300:]} + r = json.loads(ln[6:]) + out[r.pop("path")] = r + for p in paths: + if out[p].get("error") == "no_result": + out[p]["stderr"] = proc.stderr[-300:] except subprocess.TimeoutExpired: - return {"error": "harness_timeout"} + for p in paths: + if out[p].get("error") == "no_result": + out[p] = {"error": "harness_timeout"} + return out def instances(): @@ -167,22 +184,32 @@ def main(): ap.add_argument("--panel", default="AB") ap.add_argument("--out", default="scratchpad/panel860_results.json") ap.add_argument("--only", default="") + ap.add_argument("--batch", type=int, default=6) a = ap.parse_args() only = set(a.only.split(",")) if a.only else None - hard = a.tl + 180.0 rows = {} - inst = instances() - for i, (name, path) in enumerate(inst, 1): - if only and name not in only: - continue - row = {"name": name} + inst = [(n, p) for n, p in instances() if not only or n in only] + done = 0 + for s0 in range(0, len(inst), a.batch): + chunk = inst[s0 : s0 + a.batch] + paths = [p for _n, p in chunk] + hard = a.tl * len(chunk) + 240.0 + eng = off = on = {} if "A" in a.panel: - row["engine"] = _run(_WORKER_ENGINE, [path, str(a.tl)], hard) + eng = _run_batch(_WORKER_ENGINE, [str(a.tl)], paths, hard) if "B" in a.panel: - row["off"] = _run(_WORKER_SOLVE, [path, str(a.tl), "0"], hard) - row["on"] = _run(_WORKER_SOLVE, [path, str(a.tl), "1"], hard) - rows[name] = row - print(f"[{i}/{len(inst)}] {json.dumps(row)}", flush=True) + off = _run_batch(_WORKER_SOLVE, [str(a.tl), "0"], paths, hard) + on = _run_batch(_WORKER_SOLVE, [str(a.tl), "1"], paths, hard) + for name, path in chunk: + row = {"name": name} + if "A" in a.panel: + row["engine"] = eng.get(path, {"error": "missing"}) + if "B" in a.panel: + row["off"] = off.get(path, {"error": "missing"}) + row["on"] = on.get(path, {"error": "missing"}) + rows[name] = row + done += 1 + print(f"[{done}/{len(inst)}] {json.dumps(row)}", flush=True) with open(os.path.join(_ROOT, a.out), "w") as fh: json.dump(rows, fh, indent=1) print("done") From 3ef502c98679fa4a9a75436d476f44ed9bc3093f Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 26 Jul 2026 15:39:14 +0000 Subject: [PATCH 07/12] chore(#860): read objective senses from the models, not from the bound relation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The panel analyzer inferred each instance's sense from whether the reported bound sat above or below the objective — which is circular, since that relation is exactly the soundness property under test, and it silently defaulted a bound-only maximize run to the minimize check. Senses now come from a precomputed map read off the models themselves. Includes the in-flight Panel A results; the final panel output lands with the verification write-up. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_019bsy1zfNW8H1hZGg9SKfPu --- scratchpad/panel860_analyze.py | 15 ++-- scratchpad/panel860_engine.json | 142 ++++++++++++++++++++++++++++++++ scratchpad/panel860_senses.json | 121 +++++++++++++++++++++++++++ 3 files changed, 271 insertions(+), 7 deletions(-) create mode 100644 scratchpad/panel860_engine.json create mode 100644 scratchpad/panel860_senses.json diff --git a/scratchpad/panel860_analyze.py b/scratchpad/panel860_analyze.py index e00c9630..0029a4ee 100644 --- a/scratchpad/panel860_analyze.py +++ b/scratchpad/panel860_analyze.py @@ -30,13 +30,14 @@ def main(): best_min[name] = min(best_min.get(name, math.inf), v) best_max[name] = max(best_max.get(name, -math.inf), v) - # Objective sense: the engine reports a bound on the same side as the sense, so - # infer it from the run that has both. (The panel worker does not record it - # directly; ``bound <= obj`` => minimize, ``bound >= obj`` => maximize.) - for name, r in rows.items(): - e = r.get("engine") or {} - if e.get("obj") is not None and e.get("bound") is not None: - sense[name] = "max" if e["bound"] > e["obj"] + TOL else "min" + # Objective sense per instance, read off the models themselves (precomputed by + # ``panel860_senses.json``). Inferring it from ``bound`` vs ``obj`` would be + # circular — that relation is exactly what the soundness check tests — and would + # silently default a bound-only maximize run to the minimize check. + try: + sense.update(json.load(open("scratchpad/panel860_senses.json"))) + except OSError: + print("WARNING: no panel860_senses.json; every instance treated as minimize") unsound, false_opt, cert_reg, drift, ivf = [], [], [], [], [] scope_new, engine_inc, engine_declined = [], [], [] diff --git a/scratchpad/panel860_engine.json b/scratchpad/panel860_engine.json new file mode 100644 index 00000000..c750a820 --- /dev/null +++ b/scratchpad/panel860_engine.json @@ -0,0 +1,142 @@ +{ + "4stufen": { + "name": "4stufen", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 0.7608188250001149, + "declined": true + } + }, + "alan": { + "name": "alan", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 0.08560263299978033, + "declined": false, + "status": "feasible", + "obj": 2.932649872536388, + "bound": 1.6255144032921773, + "gap": 0.3323803317382956, + "nodes": 9, + "x_feasible": true, + "x_true_obj": 2.932649872536388 + } + }, + "bchoco06": { + "name": "bchoco06", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 7.30185662599979, + "declined": true + } + }, + "bchoco07": { + "name": "bchoco07", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 8.09406809400025, + "declined": true + } + }, + "bchoco08": { + "name": "bchoco08", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 9.303226861999974, + "declined": true + } + }, + "beuster": { + "name": "beuster", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 0.50811576399974, + "declined": true + } + }, + "casctanks": { + "name": "casctanks", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 20.055729753999913, + "declined": false, + "status": "time_limit", + "obj": null, + "bound": 4.93636462852202, + "gap": null, + "nodes": 2 + } + }, + "clay0303hfsg": { + "name": "clay0303hfsg", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 20.00754207199998, + "declined": false, + "status": "time_limit", + "obj": null, + "bound": 1360.0, + "gap": null, + "nodes": 92 + } + }, + "contvar": { + "name": "contvar", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 10.468364807999933, + "declined": true + } + }, + "cvxnonsep_nsig30": { + "name": "cvxnonsep_nsig30", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 20.008413320999807, + "declined": false, + "status": "time_limit", + "obj": null, + "bound": 46.91331750159184, + "gap": null, + "nodes": 299 + } + }, + "cvxnonsep_psig40r": { + "name": "cvxnonsep_psig40r", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 0.41163393999977416, + "declined": false, + "status": "time_limit", + "obj": null, + "bound": 40.00004, + "gap": null, + "nodes": 1 + } + }, + "ex1221": { + "name": "ex1221", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 0.05611375099988436, + "declined": false, + "status": "time_limit", + "obj": null, + "bound": 7.667083704792532, + "gap": null, + "nodes": 1 + } + } +} \ No newline at end of file diff --git a/scratchpad/panel860_senses.json b/scratchpad/panel860_senses.json new file mode 100644 index 00000000..40762efc --- /dev/null +++ b/scratchpad/panel860_senses.json @@ -0,0 +1,121 @@ +{ +"4stufen": "min", +"alan": "min", +"bchoco06": "max", +"bchoco07": "max", +"bchoco08": "max", +"beuster": "min", +"casctanks": "min", +"chance": "min", +"clay0303hfsg": "min", +"contvar": "min", +"cvxnonsep_nsig30": "min", +"cvxnonsep_psig30": "min", +"cvxnonsep_psig40r": "min", +"dispatch": "min", +"ex1221": "min", +"ex1222": "min", +"ex1224": "min", +"ex1225": "min", +"ex1226": "min", +"ex14_1_9": "min", +"fac2": "min", +"flay02m": "min", +"flay03m": "min", +"gbd": "min", +"gkocis": "min", +"hda": "min", +"heatexch_gen1": "min", +"heatexch_gen2": "min", +"heatexch_gen3": "min", +"m3": "min", +"nvs01": "min", +"nvs02": "min", +"nvs03": "min", +"nvs04": "min", +"nvs05": "min", +"nvs06": "min", +"nvs07": "min", +"nvs08": "min", +"nvs09": "min", +"nvs10": "min", +"nvs11": "min", +"nvs12": "min", +"nvs13": "min", +"nvs14": "min", +"nvs15": "min", +"nvs21": "min", +"oaer": "min", +"st_e11": "min", +"st_e13": "min", +"st_e29": "min", +"st_e36": "min", +"st_e38": "min", +"st_miqp1": "min", +"st_miqp2": "min", +"st_miqp3": "min", +"st_miqp4": "min", +"st_miqp5": "min", +"st_test1": "min", +"st_testgr3": "min", +"syn05hfsg": "max", +"tanksize": "min", +"tls2": "min", +"tspn05": "min", +"tspn08": "min", +"tspn10": "min", +"tspn12": "min", +"carton7": "min", +"ex1223a": "min", +"ex1233": "min", +"ex1252": "min", +"ex1252a": "min", +"ex1263": "min", +"ex1264": "min", +"ex1265": "min", +"ex1266": "min", +"ex8_1_1": "min", +"ex8_5_4": "min", +"fuel": "min", +"gear": "min", +"gear2": "min", +"gear3": "min", +"gear4": "min", +"kall_circles_c8a": "min", +"mathopt3": "min", +"mathopt5_2": "min", +"meanvar": "min", +"meanvarx": "min", +"nvs16": "min", +"nvs17": "min", +"nvs18": "min", +"nvs19": "min", +"nvs20": "min", +"nvs22": "min", +"nvs23": "min", +"nvs24": "min", +"prob02": "min", +"prob03": "min", +"prob06": "min", +"prob10": "min", +"st_e01": "min", +"st_e02": "min", +"st_e03": "min", +"st_e04": "min", +"st_e05": "min", +"st_e06": "min", +"st_e07": "min", +"st_e08": "min", +"st_e09": "min", +"st_e15": "min", +"st_e17": "min", +"st_e18": "min", +"st_e27": "min", +"st_e31": "min", +"st_e35": "min", +"st_e40": "min", +"st_ph10": "min", +"syn05m": "max", +"trig": "min", +"util": "min" +} \ No newline at end of file From c996c24b3b5a0310abbdcc4a2e88c71eb8ae3838 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 26 Jul 2026 15:57:51 +0000 Subject: [PATCH 08/12] docs(#860): record the Panel A verdict (engine soundness, cert-clean) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Panel A over all 89 in-scope instances (70 newly in scope, 19 the old gate already served), 20s budget, every incumbent independently re-verified with a fresh evaluator: newly in scope 70 declined 15 errors 0 incumbent 33 certified 12 legacy in scope 19 declined 0 errors 0 incumbent 17 certified 14 unsound bounds / incumbents: 0 false optimality certificates: 0 33 instances the engine previously refused outright now yield a verified feasible point; the legacy pure-integer class is unchanged. Both maximize instances return the bound on the correct side (syn05m / syn05hfsg: objective 837.732, upper bound 868.110, incumbent independently feasible). bchoco06/07/08 decline on an iteration_limit root LP — a conditioning problem, not a scope one. Panel B (the DISCOPT_LP_SPATIAL_MIXED graduation gate) is still running; its section carries a PANEL_B_PLACEHOLDER until those numbers land. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_019bsy1zfNW8H1hZGg9SKfPu --- docs/dev/issue-860-lp-spatial-mixed-scope.md | 56 +- scratchpad/panel860_engine.json | 1131 ++++++++++++++++++ scratchpad/panel860_flag.json | 558 +++++++++ 3 files changed, 1734 insertions(+), 11 deletions(-) create mode 100644 scratchpad/panel860_flag.json diff --git a/docs/dev/issue-860-lp-spatial-mixed-scope.md b/docs/dev/issue-860-lp-spatial-mixed-scope.md index 38906a36..d4224349 100644 --- a/docs/dev/issue-860-lp-spatial-mixed-scope.md +++ b/docs/dev/issue-860-lp-spatial-mixed-scope.md @@ -181,23 +181,57 @@ Regressions: `test_incremental_mccormick_node.py::test_incremental_node_bound_in ## 4. Verification -See the PR description for the panel numbers -(`scratchpad/panel860.py`, results in `scratchpad/panel860_results.json`). +Harness `scratchpad/panel860.py`, scoring `scratchpad/panel860_analyze.py`, 20 s +budget, one fresh model per instance. -Panel A — engine soundness on the widened class, over every in-repo instance now in -scope. Checks that need no external oracle: every reported incumbent independently -re-verified feasible with its objective re-evaluated; `bound ≤ objective` for a +### Panel A — engine soundness on the widened class + +Every in-repo instance the engine now accepts (89: the 70 newly in scope plus the 19 +the old gate already served), run through `solve_lp_spatial_bb` directly. The checks +need no external oracle: every reported incumbent independently re-verified feasible +with its objective re-evaluated by a fresh evaluator; `bound ≤ objective` for a minimize and `bound ≥ objective` for a maximize; the bound never crossing the best verified feasible point found by any run of that instance; and no `status="optimal"` run beaten by another run's verified incumbent (which would be a false optimality certificate). -Panel B — graduation gate for `DISCOPT_LP_SPATIAL_MIXED`, the flag governing whether -the *default* path reserves 35% of its budget for the #844 no-incumbent fallback on -mixed / maximize models. That reserve is the sole risk of the widening (an in-scope -model hands budget to a pass that may find nothing), which is why it is a separate, -measured decision from the engine's capability. Graduation needs both bars of -CLAUDE.md §5: cert-clean **and** net-positive. +| | newly in scope | legacy in scope | +|---|---|---| +| instances | 70 | 19 | +| engine declined | 15 | 0 | +| errors | **0** | **0** | +| verified incumbent | **33** | 17 | +| certified optimal | 12 | 14 | +| **unsound bounds / incumbents** | **0** | **0** | +| **false optimality certificates** | **0** | **0** | + +**Cert-clean.** 33 instances that the engine previously refused outright now yield a +verified feasible point, 12 of them with an optimality certificate. The legacy +pure-integer class is unchanged — nothing declined, nothing lost. + +Both maximize instances behave as the soundness argument requires — the reported +bound is an *upper* bound sitting above the incumbent: + +| instance | status | objective | bound | incumbent independently feasible | +|---|---|---|---|---| +| `syn05m` | feasible | 837.732 | 868.110 | yes | +| `syn05hfsg` | feasible | 837.732 | 868.110 | yes | + +(`bchoco06/07/08`, the other three maximize instances, decline: their root LP exits at +`iteration_limit` on an ill-conditioned equilibrated system. A conditioning problem, +not a scope one.) + +### Panel B — graduation gate for `DISCOPT_LP_SPATIAL_MIXED` + +The flag governs whether the *default* path reserves 35% of its budget for the #844 +no-incumbent fallback on mixed / maximize models. That reserve is the sole risk of the +widening (an in-scope model hands budget to a pass that may find nothing), which is +why it is a separate, measured decision from the engine's capability. Run over the 70 +newly in-scope instances — the other 49 take a bit-identical path under either +setting, since the reserve gate is the flag's only consumer and both gates agree +there. Graduation needs both bars of CLAUDE.md §5: cert-clean **and** net-positive. + +PANEL_B_PLACEHOLDER ## 5. Known limits (not addressed here) diff --git a/scratchpad/panel860_engine.json b/scratchpad/panel860_engine.json index c750a820..e385998c 100644 --- a/scratchpad/panel860_engine.json +++ b/scratchpad/panel860_engine.json @@ -138,5 +138,1136 @@ "gap": null, "nodes": 1 } + }, + "ex1222": { + "name": "ex1222", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 0.4332079219998377, + "declined": false, + "status": "optimal", + "obj": 1.0765430419131625, + "bound": 1.0765430419131625, + "gap": 0.0, + "nodes": 1, + "x_feasible": true, + "x_true_obj": 1.0765430419131625 + } + }, + "ex1224": { + "name": "ex1224", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 9.840794877999997, + "declined": false, + "status": "time_limit", + "obj": null, + "bound": -0.9849843241932473, + "gap": null, + "nodes": 806 + } + }, + "ex1225": { + "name": "ex1225", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 0.09670571400010886, + "declined": false, + "status": "feasible", + "obj": 31.0, + "bound": 30.385189144491495, + "gap": 0.01921283923464079, + "nodes": 8, + "x_feasible": true, + "x_true_obj": 31.0 + } + }, + "ex1226": { + "name": "ex1226", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 0.08993363799982035, + "declined": false, + "status": "feasible", + "obj": -17.0, + "bound": -19.0, + "gap": 0.1111111111111111, + "nodes": 11, + "x_feasible": true, + "x_true_obj": -17.0 + } + }, + "fac2": { + "name": "fac2", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 0.16093324599978587, + "declined": true + } + }, + "flay02m": { + "name": "flay02m", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 0.168188445999931, + "declined": false, + "status": "time_limit", + "obj": null, + "bound": 22.285714284708313, + "gap": null, + "nodes": 7 + } + }, + "flay03m": { + "name": "flay03m", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 1.1612239430000955, + "declined": false, + "status": "time_limit", + "obj": null, + "bound": 31.111111109126767, + "gap": null, + "nodes": 109 + } + }, + "gbd": { + "name": "gbd", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 0.03613366500030679, + "declined": false, + "status": "optimal", + "obj": 2.2, + "bound": 2.2, + "gap": 0.0, + "nodes": 1, + "x_feasible": true, + "x_true_obj": 2.2 + } + }, + "gkocis": { + "name": "gkocis", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 0.10796480499993777, + "declined": false, + "status": "feasible", + "obj": 0.0, + "bound": -1.9231718668374258, + "gap": 1.9231718668374258, + "nodes": 7, + "x_feasible": true, + "x_true_obj": 0.0 + } + }, + "hda": { + "name": "hda", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 10.07728617299972, + "declined": true + } + }, + "heatexch_gen1": { + "name": "heatexch_gen1", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 0.5503358030000527, + "declined": true + } + }, + "heatexch_gen2": { + "name": "heatexch_gen2", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 0.6907846830004019, + "declined": true + } + }, + "heatexch_gen3": { + "name": "heatexch_gen3", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 6.405825939000351, + "declined": true + } + }, + "m3": { + "name": "m3", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 0.7135470299999724, + "declined": false, + "status": "feasible", + "obj": 37.8, + "bound": 4.572053370949352, + "gap": 0.856390377037388, + "nodes": 34, + "x_feasible": true, + "x_true_obj": 37.8 + } + }, + "nvs01": { + "name": "nvs01", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 0.974310302000049, + "declined": false, + "status": "feasible", + "obj": 12.46966882156821, + "bound": 8.690507262710476, + "gap": 0.28056826109981103, + "nodes": 25, + "x_feasible": true, + "x_true_obj": 12.46966882156821 + } + }, + "nvs02": { + "name": "nvs02", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 0.15951728500022, + "declined": false, + "status": "optimal", + "obj": 5.96418452307, + "bound": 5.9641845230699655, + "gap": 4.973871420775493e-15, + "nodes": 11, + "x_feasible": true, + "x_true_obj": 5.96418452307 + } + }, + "nvs03": { + "name": "nvs03", + "engine": { + "in_scope": true, + "in_scope_legacy": true, + "wall": 0.04821865899975819, + "declined": false, + "status": "optimal", + "obj": 16.0, + "bound": 16.0, + "gap": 0.0, + "nodes": 3, + "x_feasible": true, + "x_true_obj": 16.0 + } + }, + "nvs05": { + "name": "nvs05", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 20.016012873999898, + "declined": false, + "status": "time_limit", + "obj": null, + "bound": 1.3079235534937053, + "gap": null, + "nodes": 293 + } + }, + "nvs07": { + "name": "nvs07", + "engine": { + "in_scope": true, + "in_scope_legacy": true, + "wall": 0.583350849999988, + "declined": false, + "status": "optimal", + "obj": 4.0, + "bound": 3.9999999999999996, + "gap": 8.881784197001253e-17, + "nodes": 5, + "x_feasible": true, + "x_true_obj": 4.0 + } + }, + "nvs08": { + "name": "nvs08", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 0.9364130349999868, + "declined": false, + "status": "feasible", + "obj": 47.39016760926306, + "bound": 17.498087916733347, + "gap": 0.6177304433805606, + "nodes": 26, + "x_feasible": true, + "x_true_obj": 47.39016760926306 + } + }, + "nvs10": { + "name": "nvs10", + "engine": { + "in_scope": true, + "in_scope_legacy": true, + "wall": 0.058495758999924874, + "declined": false, + "status": "optimal", + "obj": -310.80000000000007, + "bound": -310.80000000000007, + "gap": 0.0, + "nodes": 5, + "x_feasible": true, + "x_true_obj": -310.80000000000007 + } + }, + "nvs11": { + "name": "nvs11", + "engine": { + "in_scope": true, + "in_scope_legacy": true, + "wall": 0.11558397899989359, + "declined": false, + "status": "optimal", + "obj": -431.00000000000006, + "bound": -431.00000000000006, + "gap": 0.0, + "nodes": 23, + "x_feasible": true, + "x_true_obj": -431.00000000000006 + } + }, + "nvs12": { + "name": "nvs12", + "engine": { + "in_scope": true, + "in_scope_legacy": true, + "wall": 0.25935636799977146, + "declined": false, + "status": "optimal", + "obj": -481.20000000000005, + "bound": -481.20000000000005, + "gap": 0.0, + "nodes": 53, + "x_feasible": true, + "x_true_obj": -481.20000000000005 + } + }, + "nvs13": { + "name": "nvs13", + "engine": { + "in_scope": true, + "in_scope_legacy": true, + "wall": 0.7992511980000927, + "declined": false, + "status": "optimal", + "obj": -585.2, + "bound": -585.2, + "gap": 0.0, + "nodes": 223, + "x_feasible": true, + "x_true_obj": -585.2 + } + }, + "nvs14": { + "name": "nvs14", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 0.552650929000265, + "declined": false, + "status": "optimal", + "obj": -40358.1547693, + "bound": -40358.15476930035, + "gap": 8.65345093268567e-15, + "nodes": 11, + "x_feasible": true, + "x_true_obj": -40358.1547693 + } + }, + "nvs15": { + "name": "nvs15", + "engine": { + "in_scope": true, + "in_scope_legacy": true, + "wall": 0.04986899999994421, + "declined": false, + "status": "optimal", + "obj": 1.0, + "bound": 1.0, + "gap": 0.0, + "nodes": 5, + "x_feasible": true, + "x_true_obj": 1.0 + } + }, + "nvs21": { + "name": "nvs21", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 4.615407304999735, + "declined": false, + "status": "feasible", + "obj": -4.824000000000002, + "bound": -21538.30546285715, + "gap": 3697.3697566718997, + "nodes": 121, + "x_feasible": true, + "x_true_obj": -4.824000000000002 + } + }, + "oaer": { + "name": "oaer", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 0.0541788710002038, + "declined": false, + "status": "time_limit", + "obj": null, + "bound": -5.388888250000001, + "gap": null, + "nodes": 1 + } + }, + "st_e13": { + "name": "st_e13", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 0.10219136100022297, + "declined": false, + "status": "feasible", + "obj": 2.0041359822775484, + "bound": 1.9999901654089156, + "gap": 0.001380036354243093, + "nodes": 16, + "x_feasible": true, + "x_true_obj": 2.0041359822775484 + } + }, + "st_e29": { + "name": "st_e29", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 10.456363768999836, + "declined": false, + "status": "time_limit", + "obj": null, + "bound": -0.9849843241932473, + "gap": null, + "nodes": 806 + } + }, + "st_e36": { + "name": "st_e36", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 8.321640207999735, + "declined": false, + "status": "time_limit", + "obj": null, + "bound": -304.5000000000001, + "gap": null, + "nodes": 83 + } + }, + "st_e38": { + "name": "st_e38", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 1.56904957200004, + "declined": false, + "status": "feasible", + "obj": 7197.727148538972, + "bound": 6795.045164869508, + "gap": 0.05593794227236282, + "nodes": 57, + "x_feasible": true, + "x_true_obj": 7197.727148538972 + } + }, + "st_miqp1": { + "name": "st_miqp1", + "engine": { + "in_scope": true, + "in_scope_legacy": true, + "wall": 0.047518403999674774, + "declined": false, + "status": "optimal", + "obj": 281.0, + "bound": 281.0, + "gap": 0.0, + "nodes": 2, + "x_feasible": true, + "x_true_obj": 281.0 + } + }, + "st_miqp2": { + "name": "st_miqp2", + "engine": { + "in_scope": true, + "in_scope_legacy": true, + "wall": 0.05191615500007174, + "declined": false, + "status": "optimal", + "obj": 2.0, + "bound": 1.999999999999698, + "gap": 1.0066022089934752e-13, + "nodes": 12, + "x_feasible": true, + "x_true_obj": 2.0 + } + }, + "st_miqp3": { + "name": "st_miqp3", + "engine": { + "in_scope": true, + "in_scope_legacy": true, + "wall": 0.024777903000085644, + "declined": false, + "status": "optimal", + "obj": -6.0, + "bound": -6.0, + "gap": 0.0, + "nodes": 1, + "x_feasible": true, + "x_true_obj": -6.0 + } + }, + "st_miqp4": { + "name": "st_miqp4", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 0.048608170000079554, + "declined": false, + "status": "optimal", + "obj": -4574.0, + "bound": -4574.000000000032, + "gap": 6.9578829643830576e-15, + "nodes": 2, + "x_feasible": true, + "x_true_obj": -4574.0 + } + }, + "st_miqp5": { + "name": "st_miqp5", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 0.9497713400000976, + "declined": false, + "status": "optimal", + "obj": -333.8822829963604, + "bound": -333.90739636823207, + "gap": 7.499164078483565e-05, + "nodes": 18, + "x_feasible": true, + "x_true_obj": -333.8822829963604 + } + }, + "st_test1": { + "name": "st_test1", + "engine": { + "in_scope": true, + "in_scope_legacy": true, + "wall": 0.0512327749997894, + "declined": false, + "status": "optimal", + "obj": 0.0, + "bound": 0.0, + "gap": 0.0, + "nodes": 7, + "x_feasible": true, + "x_true_obj": 0.0 + } + }, + "st_testgr3": { + "name": "st_testgr3", + "engine": { + "in_scope": true, + "in_scope_legacy": true, + "wall": 1.1494602879997728, + "declined": false, + "status": "optimal", + "obj": -20.590000000000003, + "bound": -20.59162142857181, + "gap": 7.510090652180148e-05, + "nodes": 264, + "x_feasible": true, + "x_true_obj": -20.590000000000003 + } + }, + "syn05hfsg": { + "name": "syn05hfsg", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 0.6915048370001387, + "declined": false, + "status": "feasible", + "obj": 837.7324008979808, + "bound": 868.1098791406625, + "gap": 0.036218319705019576, + "nodes": 7, + "x_feasible": true, + "x_true_obj": 837.7324008979808 + } + }, + "tanksize": { + "name": "tanksize", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 0.7772753479998755, + "declined": false, + "status": "time_limit", + "obj": null, + "bound": -1e-06, + "gap": null, + "nodes": 5 + } + }, + "tls2": { + "name": "tls2", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 20.005315523999798, + "declined": false, + "status": "time_limit", + "obj": null, + "bound": 0.05896756330306378, + "gap": null, + "nodes": 846 + } + }, + "tspn05": { + "name": "tspn05", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 20.006075695999698, + "declined": false, + "status": "time_limit", + "obj": null, + "bound": 188.33699415201545, + "gap": null, + "nodes": 272 + } + }, + "tspn08": { + "name": "tspn08", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 0.5192590809997455, + "declined": true + } + }, + "tspn10": { + "name": "tspn10", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 1.034821275000013, + "declined": true + } + }, + "tspn12": { + "name": "tspn12", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 1.78446484899996, + "declined": true + } + }, + "carton7": { + "name": "carton7", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 20.03079538100019, + "declined": false, + "status": "time_limit", + "obj": null, + "bound": 26.0, + "gap": null, + "nodes": 170 + } + }, + "ex1223a": { + "name": "ex1223a", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 0.17615347799983283, + "declined": false, + "status": "feasible", + "obj": 4.579582146663329, + "bound": 4.554071812205102, + "gap": 0.004572086903224928, + "nodes": 27, + "x_feasible": true, + "x_true_obj": 4.579582146663329 + } + }, + "ex1233": { + "name": "ex1233", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 9.157589035, + "declined": false, + "status": "feasible", + "obj": 164046.85947589384, + "bound": 46188.15098887568, + "gap": 0.7184410016903452, + "nodes": 96, + "x_feasible": true, + "x_true_obj": 164046.85947589384 + } + }, + "ex1252": { + "name": "ex1252", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 20.005022871000165, + "declined": false, + "status": "time_limit", + "obj": null, + "bound": 0.0, + "gap": null, + "nodes": 388 + } + }, + "ex1252a": { + "name": "ex1252a", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 20.00370495800007, + "declined": false, + "status": "time_limit", + "obj": null, + "bound": 0.0, + "gap": null, + "nodes": 409 + } + }, + "ex1263": { + "name": "ex1263", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 20.008046041999933, + "declined": false, + "status": "time_limit", + "obj": null, + "bound": 19.299999999999386, + "gap": null, + "nodes": 3853 + } + }, + "ex1264": { + "name": "ex1264", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 20.004806518999885, + "declined": false, + "status": "time_limit", + "obj": 10.299999999999999, + "bound": 8.299999999999653, + "gap": 0.17699115044250854, + "nodes": 3736, + "x_feasible": true, + "x_true_obj": 10.299999999999999 + } + }, + "ex1265": { + "name": "ex1265", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 20.008367630000066, + "declined": false, + "status": "time_limit", + "obj": 28.5, + "bound": 10.299999999999347, + "gap": 0.616949152542395, + "nodes": 2830, + "x_feasible": true, + "x_true_obj": 28.5 + } + }, + "ex1266": { + "name": "ex1266", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 4.673025727000095, + "declined": false, + "status": "optimal", + "obj": 16.3, + "bound": 16.3, + "gap": 0.0, + "nodes": 504, + "x_feasible": true, + "x_true_obj": 16.3 + } + }, + "fuel": { + "name": "fuel", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 3.5052710380000462, + "declined": false, + "status": "feasible", + "obj": 8566.11896297357, + "bound": 8565.11501670532, + "gap": 0.00011718598429526983, + "nodes": 386, + "x_feasible": true, + "x_true_obj": 8566.11896297357 + } + }, + "gear2": { + "name": "gear2", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 20.012148196999988, + "declined": false, + "status": "time_limit", + "obj": 0.00017876315356920368, + "bound": -5e-324, + "gap": 0.00017873120301571137, + "nodes": 586, + "x_feasible": true, + "x_true_obj": 0.00017876315356920368 + } + }, + "gear3": { + "name": "gear3", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 2.4114163780000126, + "declined": false, + "status": "optimal", + "obj": 2.72645059771712e-08, + "bound": -5e-324, + "gap": 2.7264505233817933e-08, + "nodes": 93, + "x_feasible": true, + "x_true_obj": 2.72645059771712e-08 + } + }, + "gear4": { + "name": "gear4", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 0.1203516010000385, + "declined": false, + "status": "optimal", + "obj": 184279.32477275995, + "bound": 184279.32477275995, + "gap": 0.0, + "nodes": 1, + "x_feasible": true, + "x_true_obj": 184279.32477275995 + } + }, + "meanvarx": { + "name": "meanvarx", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 3.0456329290000212, + "declined": false, + "status": "optimal", + "obj": 14.369257854553203, + "bound": 14.368163557127417, + "gap": 7.120040773219863e-05, + "nodes": 415, + "x_feasible": true, + "x_true_obj": 14.369257854553203 + } + }, + "nvs17": { + "name": "nvs17", + "engine": { + "in_scope": true, + "in_scope_legacy": true, + "wall": 20.01266748700027, + "declined": false, + "status": "time_limit", + "obj": -1076.2000000000003, + "bound": -1218.3608888891279, + "gap": 0.1319726038703375, + "nodes": 4174, + "x_feasible": true, + "x_true_obj": -1076.2000000000003 + } + }, + "nvs18": { + "name": "nvs18", + "engine": { + "in_scope": true, + "in_scope_legacy": true, + "wall": 9.268651800000043, + "declined": false, + "status": "optimal", + "obj": -778.4, + "bound": -778.4, + "gap": 0.0, + "nodes": 2494, + "x_feasible": true, + "x_true_obj": -778.4 + } + }, + "nvs19": { + "name": "nvs19", + "engine": { + "in_scope": true, + "in_scope_legacy": true, + "wall": 20.01232871000002, + "declined": false, + "status": "time_limit", + "obj": -763.4, + "bound": -2249.0000000001805, + "gap": 1.9434850863424653, + "nodes": 3589, + "x_feasible": true, + "x_true_obj": -763.4 + } + }, + "nvs20": { + "name": "nvs20", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 20.024032530000113, + "declined": false, + "status": "time_limit", + "obj": 230.92351095742626, + "bound": 163.1413004622102, + "gap": 0.2922610571709511, + "nodes": 451, + "x_feasible": true, + "x_true_obj": 230.92351095742626 + } + }, + "nvs22": { + "name": "nvs22", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 20.007111062999684, + "declined": false, + "status": "time_limit", + "obj": null, + "bound": 2.693180074660405, + "gap": null, + "nodes": 270 + } + }, + "nvs23": { + "name": "nvs23", + "engine": { + "in_scope": true, + "in_scope_legacy": true, + "wall": 20.00934357799997, + "declined": false, + "status": "time_limit", + "obj": null, + "bound": -3459.70063425005, + "gap": null, + "nodes": 2801 + } + }, + "nvs24": { + "name": "nvs24", + "engine": { + "in_scope": true, + "in_scope_legacy": true, + "wall": 20.013971643999867, + "declined": false, + "status": "time_limit", + "obj": null, + "bound": -15780.883333335732, + "gap": null, + "nodes": 2170 + } + }, + "prob02": { + "name": "prob02", + "engine": { + "in_scope": true, + "in_scope_legacy": true, + "wall": 0.11646341200003008, + "declined": false, + "status": "feasible", + "obj": 112235.0, + "bound": 106254.10714285713, + "gap": 0.053288542509915444, + "nodes": 2, + "x_feasible": true, + "x_true_obj": 112235.0 + } + }, + "prob03": { + "name": "prob03", + "engine": { + "in_scope": true, + "in_scope_legacy": true, + "wall": 0.046144041000388825, + "declined": false, + "status": "optimal", + "obj": 10.0, + "bound": 10.0, + "gap": 0.0, + "nodes": 4, + "x_feasible": true, + "x_true_obj": 10.0 + } + }, + "prob10": { + "name": "prob10", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 0.05481232500005717, + "declined": false, + "status": "feasible", + "obj": 3.445503794326429, + "bound": 1.3068571428564728, + "gap": 0.48108082917382783, + "nodes": 3, + "x_feasible": true, + "x_true_obj": 3.445503794326429 + } + }, + "st_e15": { + "name": "st_e15", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 0.07204430000001594, + "declined": false, + "status": "time_limit", + "obj": null, + "bound": 7.667083704792532, + "gap": null, + "nodes": 1 + } + }, + "st_e27": { + "name": "st_e27", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 0.04603259200030152, + "declined": false, + "status": "optimal", + "obj": 2.0, + "bound": 2.0, + "gap": 0.0, + "nodes": 1, + "x_feasible": true, + "x_true_obj": 2.0 + } + }, + "st_e31": { + "name": "st_e31", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 20.00870414100018, + "declined": false, + "status": "time_limit", + "obj": null, + "bound": -3.0, + "gap": null, + "nodes": 768 + } + }, + "st_e35": { + "name": "st_e35", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 0.36533751999968445, + "declined": true + } + }, + "st_e40": { + "name": "st_e40", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 0.81456153299996, + "declined": false, + "status": "feasible", + "obj": 30.4142135, + "bound": 25.9264998322432, + "gap": 0.14285615228777893, + "nodes": 24, + "x_feasible": true, + "x_true_obj": 30.4142135 + } + }, + "syn05m": { + "name": "syn05m", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 0.17062410800008365, + "declined": false, + "status": "feasible", + "obj": 837.7324008979946, + "bound": 868.1098791406849, + "gap": 0.036218319705029284, + "nodes": 11, + "x_feasible": true, + "x_true_obj": 837.7324008979946 + } + }, + "util": { + "name": "util", + "engine": { + "in_scope": true, + "in_scope_legacy": false, + "wall": 10.90876498300031, + "declined": false, + "status": "optimal", + "obj": 999.5787502353636, + "bound": 999.5577237738921, + "gap": 2.1014299440731473e-05, + "nodes": 623, + "x_feasible": true, + "x_true_obj": 999.5787502353636 + } } } \ No newline at end of file diff --git a/scratchpad/panel860_flag.json b/scratchpad/panel860_flag.json new file mode 100644 index 00000000..e3ac8f51 --- /dev/null +++ b/scratchpad/panel860_flag.json @@ -0,0 +1,558 @@ +{ + "4stufen": { + "name": "4stufen", + "off": { + "wall": 21.49394614599987, + "status": "time_limit", + "obj": null, + "bound": 20282.05073873644, + "gapc": false, + "nodes": 7, + "ivf": false + }, + "on": { + "wall": 14.550240193999798, + "status": "time_limit", + "obj": null, + "bound": 20282.05073873644, + "gapc": false, + "nodes": 7, + "ivf": false + } + }, + "alan": { + "name": "alan", + "off": { + "wall": 0.06342785900005765, + "status": "optimal", + "obj": 2.9249999983013426, + "bound": 2.9249999983013426, + "gapc": true, + "nodes": 21, + "ivf": false, + "x_feasible": true, + "x_true_obj": 2.9249999983013426 + }, + "on": { + "wall": 0.06320307199985109, + "status": "optimal", + "obj": 2.9249999983013426, + "bound": 2.9249999983013426, + "gapc": true, + "nodes": 21, + "ivf": false, + "x_feasible": true, + "x_true_obj": 2.9249999983013426 + } + }, + "bchoco06": { + "name": "bchoco06", + "off": { + "wall": 21.38688005200038, + "status": "time_limit", + "obj": null, + "bound": 0.9999775725261241, + "gapc": false, + "nodes": 7, + "ivf": false + }, + "on": { + "wall": 14.782666265999978, + "status": "time_limit", + "obj": null, + "bound": 0.9999775725261241, + "gapc": false, + "nodes": 7, + "ivf": false + } + }, + "bchoco07": { + "name": "bchoco07", + "off": { + "wall": 23.45472825500019, + "status": "time_limit", + "obj": null, + "bound": 1.000000000000286, + "gapc": false, + "nodes": 7, + "ivf": false + }, + "on": { + "wall": 16.88670159200001, + "status": "time_limit", + "obj": null, + "bound": 1.000000000000286, + "gapc": false, + "nodes": 7, + "ivf": false + } + }, + "bchoco08": { + "name": "bchoco08", + "off": { + "wall": 29.71788483599994, + "status": "time_limit", + "obj": null, + "bound": 1.000000000002788, + "gapc": false, + "nodes": 7, + "ivf": false + }, + "on": { + "wall": 20.14030541000011, + "status": "time_limit", + "obj": null, + "bound": 1.000000000002788, + "gapc": false, + "nodes": 3, + "ivf": false + } + }, + "beuster": { + "name": "beuster", + "off": { + "wall": 22.508043083999837, + "status": "time_limit", + "obj": null, + "bound": 6395.108328059316, + "gapc": false, + "nodes": 7, + "ivf": false + }, + "on": { + "wall": 15.420123824999791, + "status": "time_limit", + "obj": null, + "bound": 6395.108328059316, + "gapc": false, + "nodes": 7, + "ivf": false + } + }, + "casctanks": { + "name": "casctanks", + "off": { + "wall": 26.439481547000014, + "status": "time_limit", + "obj": null, + "bound": -102.48025587887132, + "gapc": false, + "nodes": 3, + "ivf": false + }, + "on": { + "wall": 19.887885589000234, + "status": "time_limit", + "obj": null, + "bound": -102.4802558788684, + "gapc": false, + "nodes": 1, + "ivf": false + } + }, + "clay0303hfsg": { + "name": "clay0303hfsg", + "off": { + "wall": 20.66571704199987, + "status": "time_limit", + "obj": null, + "bound": null, + "gapc": false, + "nodes": 15, + "ivf": false + }, + "on": { + "wall": 15.117893442999957, + "status": "time_limit", + "obj": null, + "bound": null, + "gapc": false, + "nodes": 63, + "ivf": false + } + }, + "contvar": { + "name": "contvar", + "off": { + "wall": 24.78055820999998, + "status": "time_limit", + "obj": null, + "bound": 183430.50053431175, + "gapc": false, + "nodes": 3, + "ivf": false + }, + "on": { + "wall": 19.536510317999728, + "status": "time_limit", + "obj": null, + "bound": 183430.50053431175, + "gapc": false, + "nodes": 3, + "ivf": false + } + }, + "cvxnonsep_nsig30": { + "name": "cvxnonsep_nsig30", + "off": { + "wall": 9.651454275000106, + "status": "optimal", + "obj": 130.6287111692529, + "bound": 130.61841392487688, + "gapc": true, + "nodes": 165, + "ivf": false, + "x_feasible": true, + "x_true_obj": 130.6287111692529 + }, + "on": { + "wall": 8.302841627999896, + "status": "optimal", + "obj": 130.6287111692529, + "bound": 130.61841392487688, + "gapc": true, + "nodes": 165, + "ivf": false, + "x_feasible": true, + "x_true_obj": 130.6287111692529 + } + }, + "cvxnonsep_psig40r": { + "name": "cvxnonsep_psig40r", + "off": { + "wall": 12.46529766599997, + "status": "optimal", + "obj": 86.54527995164959, + "bound": 86.53873600363832, + "gapc": true, + "nodes": 95, + "ivf": false, + "x_feasible": true, + "x_true_obj": 86.54527995164959 + }, + "on": { + "wall": 10.597759826000129, + "status": "optimal", + "obj": 86.54527995164959, + "bound": 86.53873600363832, + "gapc": true, + "nodes": 95, + "ivf": false, + "x_feasible": true, + "x_true_obj": 86.54527995164959 + } + }, + "ex1221": { + "name": "ex1221", + "off": { + "wall": 0.8962982420002845, + "status": "optimal", + "obj": 7.667180068813135, + "bound": 7.667180068813077, + "gapc": true, + "nodes": 5, + "ivf": false, + "x_feasible": true, + "x_true_obj": 7.667180068813135 + }, + "on": { + "wall": 0.6951870019997841, + "status": "optimal", + "obj": 7.667180068813135, + "bound": 7.667180068813077, + "gapc": true, + "nodes": 5, + "ivf": false, + "x_feasible": true, + "x_true_obj": 7.667180068813135 + } + }, + "ex1222": { + "name": "ex1222", + "off": { + "wall": 1.1242379780001102, + "status": "optimal", + "obj": 1.0765430833322625, + "bound": 1.0765430463031964, + "gapc": true, + "nodes": 1, + "ivf": false, + "x_feasible": true, + "x_true_obj": 1.0765430833322625 + }, + "on": { + "wall": 0.9876379410002301, + "status": "optimal", + "obj": 1.0765430833322625, + "bound": 1.0765430463031964, + "gapc": true, + "nodes": 1, + "ivf": false, + "x_feasible": true, + "x_true_obj": 1.0765430833322625 + } + }, + "ex1224": { + "name": "ex1224", + "off": { + "wall": 4.174198986999727, + "status": "optimal", + "obj": -0.9434704942820806, + "bound": -0.9434705000000166, + "gapc": true, + "nodes": 5, + "ivf": false, + "x_feasible": true, + "x_true_obj": -0.9434704942820806 + }, + "on": { + "wall": 2.7403194849998727, + "status": "optimal", + "obj": -0.9434704942820806, + "bound": -0.9434705000000166, + "gapc": true, + "nodes": 5, + "ivf": false, + "x_feasible": true, + "x_true_obj": -0.9434704942820806 + } + }, + "ex1225": { + "name": "ex1225", + "off": { + "wall": 2.6760348620000514, + "status": "optimal", + "obj": 30.999999913557566, + "bound": 30.999999913557566, + "gapc": true, + "nodes": 5, + "ivf": false, + "x_feasible": true, + "x_true_obj": 30.999999913557566 + }, + "on": { + "wall": 2.4321912300001713, + "status": "optimal", + "obj": 30.999999913557566, + "bound": 30.999999913557566, + "gapc": true, + "nodes": 5, + "ivf": false, + "x_feasible": true, + "x_true_obj": 30.999999913557566 + } + }, + "ex1226": { + "name": "ex1226", + "off": { + "wall": 0.9533949980000216, + "status": "optimal", + "obj": -17.000000003887443, + "bound": -17.000000003887443, + "gapc": true, + "nodes": 3, + "ivf": false, + "x_feasible": true, + "x_true_obj": -17.000000003887443 + }, + "on": { + "wall": 0.7550612790000741, + "status": "optimal", + "obj": -17.000000003887443, + "bound": -17.000000003887443, + "gapc": true, + "nodes": 3, + "ivf": false, + "x_feasible": true, + "x_true_obj": -17.000000003887443 + } + }, + "fac2": { + "name": "fac2", + "off": { + "wall": 13.777910224999687, + "status": "optimal", + "obj": 331837497.8394821, + "bound": 331837497.54563797, + "gapc": true, + "nodes": 39, + "ivf": false, + "x_feasible": true, + "x_true_obj": 331837497.8394821 + }, + "on": { + "wall": 12.882336553999721, + "status": "optimal", + "obj": 331837497.8394821, + "bound": 331837497.54563797, + "gapc": true, + "nodes": 39, + "ivf": false, + "x_feasible": true, + "x_true_obj": 331837497.8394821 + } + }, + "flay02m": { + "name": "flay02m", + "off": { + "wall": 0.8132819960001143, + "status": "optimal", + "obj": 37.947331863834606, + "bound": 37.94733180456953, + "gapc": true, + "nodes": 7, + "ivf": false, + "x_feasible": true, + "x_true_obj": 37.947331863834606 + }, + "on": { + "wall": 0.5829486960001304, + "status": "optimal", + "obj": 37.947331863834606, + "bound": 37.94733180456953, + "gapc": true, + "nodes": 7, + "ivf": false, + "x_feasible": true, + "x_true_obj": 37.947331863834606 + } + }, + "flay03m": { + "name": "flay03m", + "off": { + "wall": 13.59788596999988, + "status": "optimal", + "obj": 48.98979479383543, + "bound": 48.98979470823039, + "gapc": true, + "nodes": 107, + "ivf": false, + "x_feasible": true, + "x_true_obj": 48.98979479383543 + }, + "on": { + "wall": 13.133137556999827, + "status": "optimal", + "obj": 48.98979479383543, + "bound": 48.98979470823039, + "gapc": true, + "nodes": 107, + "ivf": false, + "x_feasible": true, + "x_true_obj": 48.98979479383543 + } + }, + "gbd": { + "name": "gbd", + "off": { + "wall": 0.022194132000095124, + "status": "optimal", + "obj": 2.199999982504936, + "bound": 2.199999982504936, + "gapc": true, + "nodes": 3, + "ivf": false, + "x_feasible": true, + "x_true_obj": 2.199999982504936 + }, + "on": { + "wall": 0.021801627000058943, + "status": "optimal", + "obj": 2.199999982504936, + "bound": 2.199999982504936, + "gapc": true, + "nodes": 3, + "ivf": false, + "x_feasible": true, + "x_true_obj": 2.199999982504936 + } + }, + "gkocis": { + "name": "gkocis", + "off": { + "wall": 1.6931554639995738, + "status": "optimal", + "obj": -1.9230987798770212, + "bound": -1.9230988280923489, + "gapc": true, + "nodes": 5, + "ivf": false, + "x_feasible": true, + "x_true_obj": -1.9230987798770212 + }, + "on": { + "wall": 1.1862340229999973, + "status": "optimal", + "obj": -1.9230987798770212, + "bound": -1.9230988280923489, + "gapc": true, + "nodes": 5, + "ivf": false, + "x_feasible": true, + "x_true_obj": -1.9230987798770212 + } + }, + "hda": { + "name": "hda", + "off": { + "wall": 27.75454013100034, + "status": "time_limit", + "obj": null, + "bound": -141960.28628545807, + "gapc": false, + "nodes": 3, + "ivf": false + }, + "on": { + "wall": 24.54702392099989, + "status": "time_limit", + "obj": null, + "bound": -64473.44240243703, + "gapc": false, + "nodes": 1, + "ivf": false + } + }, + "heatexch_gen1": { + "name": "heatexch_gen1", + "off": { + "wall": 30.36552632899975, + "status": "time_limit", + "obj": null, + "bound": 38183.5317460179, + "gapc": false, + "nodes": 3, + "ivf": false + }, + "on": { + "wall": 14.118221387000176, + "status": "time_limit", + "obj": null, + "bound": 38183.5317460179, + "gapc": false, + "nodes": 7, + "ivf": false + } + }, + "heatexch_gen2": { + "name": "heatexch_gen2", + "off": { + "wall": 21.760829244999968, + "status": "time_limit", + "obj": null, + "bound": 555767.7902857271, + "gapc": false, + "nodes": 7, + "ivf": false + }, + "on": { + "wall": 14.032863816999907, + "status": "time_limit", + "obj": null, + "bound": 555767.7902857271, + "gapc": false, + "nodes": 7, + "ivf": false + } + } +} \ No newline at end of file From a5cefcbec05563234c425345353513f0536fa399 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 26 Jul 2026 16:01:30 +0000 Subject: [PATCH 09/12] chore(#860): Panel B progress snapshot (42/70) The graduation-gate panel writes its results file after each batch; this is an in-flight snapshot, superseded by the final results and the verdict write-up. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_019bsy1zfNW8H1hZGg9SKfPu --- scratchpad/panel860_flag.json | 444 ++++++++++++++++++++++++++++++++++ 1 file changed, 444 insertions(+) diff --git a/scratchpad/panel860_flag.json b/scratchpad/panel860_flag.json index e3ac8f51..1d3cdf66 100644 --- a/scratchpad/panel860_flag.json +++ b/scratchpad/panel860_flag.json @@ -554,5 +554,449 @@ "nodes": 7, "ivf": false } + }, + "heatexch_gen3": { + "name": "heatexch_gen3", + "off": { + "wall": 23.09042831400029, + "status": "time_limit", + "obj": null, + "bound": null, + "gapc": false, + "nodes": 3, + "ivf": false + }, + "on": { + "wall": 17.27911373200004, + "status": "time_limit", + "obj": null, + "bound": null, + "gapc": false, + "nodes": 3, + "ivf": false + } + }, + "m3": { + "name": "m3", + "off": { + "wall": 8.065178269000171, + "status": "optimal", + "obj": 37.799999669978845, + "bound": 37.79999951039202, + "gapc": true, + "nodes": 47, + "ivf": false, + "x_feasible": true, + "x_true_obj": 37.799999669978845 + }, + "on": { + "wall": 7.562382453999817, + "status": "optimal", + "obj": 37.799999669978845, + "bound": 37.79999951039202, + "gapc": true, + "nodes": 47, + "ivf": false, + "x_feasible": true, + "x_true_obj": 37.799999669978845 + } + }, + "nvs01": { + "name": "nvs01", + "off": { + "wall": 1.628890341999977, + "status": "optimal", + "obj": 12.46966882156821, + "bound": 12.46966882156821, + "gapc": true, + "nodes": 3, + "ivf": false, + "x_feasible": true, + "x_true_obj": 12.46966882156821 + }, + "on": { + "wall": 1.5044494610001493, + "status": "optimal", + "obj": 12.46966882156821, + "bound": 12.46966882156821, + "gapc": true, + "nodes": 3, + "ivf": false, + "x_feasible": true, + "x_true_obj": 12.46966882156821 + } + }, + "nvs02": { + "name": "nvs02", + "off": { + "wall": 0.3426850280002327, + "status": "optimal", + "obj": 5.96418452307, + "bound": 5.96418452307, + "gapc": true, + "nodes": 337, + "ivf": false, + "x_feasible": true, + "x_true_obj": 5.96418452307 + }, + "on": { + "wall": 0.3804011430001992, + "status": "optimal", + "obj": 5.96418452307, + "bound": 5.96418452307, + "gapc": true, + "nodes": 337, + "ivf": false, + "x_feasible": true, + "x_true_obj": 5.96418452307 + } + }, + "nvs05": { + "name": "nvs05", + "off": { + "wall": 20.089912812999955, + "status": "feasible", + "obj": 8.731956986640078, + "bound": 3.5424692441047068, + "gapc": false, + "nodes": 41, + "ivf": false, + "x_feasible": true, + "x_true_obj": 8.731956986640078 + }, + "on": { + "wall": 13.134198775999721, + "status": "feasible", + "obj": 8.731956986640078, + "bound": 3.5139117087764977, + "gapc": false, + "nodes": 25, + "ivf": false, + "x_feasible": true, + "x_true_obj": 8.731956986640078 + } + }, + "nvs08": { + "name": "nvs08", + "off": { + "wall": 1.3908734560000084, + "status": "optimal", + "obj": 23.449727347139827, + "bound": 23.44972734516655, + "gapc": true, + "nodes": 3, + "ivf": false, + "x_feasible": true, + "x_true_obj": 23.449727347139827 + }, + "on": { + "wall": 1.1638891720003812, + "status": "optimal", + "obj": 23.449727347139827, + "bound": 23.44972734516655, + "gapc": true, + "nodes": 3, + "ivf": false, + "x_feasible": true, + "x_true_obj": 23.449727347139827 + } + }, + "nvs14": { + "name": "nvs14", + "off": { + "wall": 0.7299275679997663, + "status": "optimal", + "obj": -40358.15476929996, + "bound": -40358.15476929999, + "gapc": true, + "nodes": 223, + "ivf": false, + "x_feasible": true, + "x_true_obj": -40358.1547693 + }, + "on": { + "wall": 0.7619844540004124, + "status": "optimal", + "obj": -40358.15476929996, + "bound": -40358.15476929999, + "gapc": true, + "nodes": 223, + "ivf": false, + "x_feasible": true, + "x_true_obj": -40358.1547693 + } + }, + "nvs21": { + "name": "nvs21", + "off": { + "wall": 6.492603726000198, + "status": "optimal", + "obj": -5.684782628025259, + "bound": -5.684782628025259, + "gapc": true, + "nodes": 3, + "ivf": false, + "x_feasible": true, + "x_true_obj": -5.684782628025259 + }, + "on": { + "wall": 6.7868782960003955, + "status": "optimal", + "obj": -5.684782628025259, + "bound": -5.684782628025259, + "gapc": true, + "nodes": 3, + "ivf": false, + "x_feasible": true, + "x_true_obj": -5.684782628025259 + } + }, + "oaer": { + "name": "oaer", + "off": { + "wall": 1.026705151999522, + "status": "optimal", + "obj": -1.923098499884843, + "bound": -1.923098601139822, + "gapc": true, + "nodes": 3, + "ivf": false, + "x_feasible": true, + "x_true_obj": -1.92309850098788 + }, + "on": { + "wall": 0.9039749130006385, + "status": "optimal", + "obj": -1.923098499884843, + "bound": -1.923098601139822, + "gapc": true, + "nodes": 3, + "ivf": false, + "x_feasible": true, + "x_true_obj": -1.92309850098788 + } + }, + "st_e13": { + "name": "st_e13", + "off": { + "wall": 0.3293718949998947, + "status": "optimal", + "obj": 2.0000003463351197, + "bound": 1.9999999825187809, + "gapc": true, + "nodes": 3, + "ivf": false, + "x_feasible": true, + "x_true_obj": 2.0000003463351197 + }, + "on": { + "wall": 0.25525427699994907, + "status": "optimal", + "obj": 2.0000003463351197, + "bound": 1.9999999825187809, + "gapc": true, + "nodes": 3, + "ivf": false, + "x_feasible": true, + "x_true_obj": 2.0000003463351197 + } + }, + "st_e29": { + "name": "st_e29", + "off": { + "wall": 3.7670972200003234, + "status": "optimal", + "obj": -0.9434704942820806, + "bound": -0.9434705000000166, + "gapc": true, + "nodes": 5, + "ivf": false, + "x_feasible": true, + "x_true_obj": -0.9434704942820806 + }, + "on": { + "wall": 2.7177147920001516, + "status": "optimal", + "obj": -0.9434704942820806, + "bound": -0.9434705000000166, + "gapc": true, + "nodes": 5, + "ivf": false, + "x_feasible": true, + "x_true_obj": -0.9434704942820806 + } + }, + "st_e36": { + "name": "st_e36", + "off": { + "wall": 9.377294026000527, + "status": "optimal", + "obj": -245.99999999999997, + "bound": -246.00000000170422, + "gapc": true, + "nodes": 85, + "ivf": false, + "x_feasible": true, + "x_true_obj": -245.99999999999997 + }, + "on": { + "wall": 8.206686686000467, + "status": "optimal", + "obj": -245.99999999999997, + "bound": -246.00000000170422, + "gapc": true, + "nodes": 85, + "ivf": false, + "x_feasible": true, + "x_true_obj": -245.99999999999997 + } + }, + "st_e38": { + "name": "st_e38", + "off": { + "wall": 1.6315893389992198, + "status": "optimal", + "obj": 7197.727116826533, + "bound": 7197.727116826533, + "gapc": true, + "nodes": 3, + "ivf": false, + "x_feasible": true, + "x_true_obj": 7197.727116826534 + }, + "on": { + "wall": 1.4941501829998742, + "status": "optimal", + "obj": 7197.727116826533, + "bound": 7197.727116826533, + "gapc": true, + "nodes": 3, + "ivf": false, + "x_feasible": true, + "x_true_obj": 7197.727116826534 + } + }, + "st_miqp4": { + "name": "st_miqp4", + "off": { + "wall": 0.028630167999835976, + "status": "optimal", + "obj": -4574.000004423649, + "bound": -4574.000004423649, + "gapc": true, + "nodes": 3, + "ivf": false, + "x_feasible": true, + "x_true_obj": -4574.0000044236485 + }, + "on": { + "wall": 0.026895776999481313, + "status": "optimal", + "obj": -4574.000004423649, + "bound": -4574.000004423649, + "gapc": true, + "nodes": 3, + "ivf": false, + "x_feasible": true, + "x_true_obj": -4574.0000044236485 + } + }, + "st_miqp5": { + "name": "st_miqp5", + "off": { + "wall": 0.03180315099962172, + "status": "optimal", + "obj": -333.8888902720728, + "bound": -333.8888902720728, + "gapc": true, + "nodes": 1, + "ivf": false, + "x_feasible": true, + "x_true_obj": -333.88889027207284 + }, + "on": { + "wall": 0.028467226000429946, + "status": "optimal", + "obj": -333.8888902720728, + "bound": -333.8888902720728, + "gapc": true, + "nodes": 1, + "ivf": false, + "x_feasible": true, + "x_true_obj": -333.88889027207284 + } + }, + "syn05hfsg": { + "name": "syn05hfsg", + "off": { + "wall": 14.53700275600022, + "status": "optimal", + "obj": 837.732412391107, + "bound": 837.732412391107, + "gapc": true, + "nodes": 185, + "ivf": false, + "x_feasible": true, + "x_true_obj": 837.7324125099896 + }, + "on": { + "wall": 12.212029635999897, + "status": "optimal", + "obj": 837.732412391107, + "bound": 837.732412391107, + "gapc": true, + "nodes": 185, + "ivf": false, + "x_feasible": true, + "x_true_obj": 837.7324125099896 + } + }, + "tanksize": { + "name": "tanksize", + "off": { + "wall": 20.26072421399931, + "status": "feasible", + "obj": 1.2686437852381225, + "bound": 0.8716343114421118, + "gapc": false, + "nodes": 35, + "ivf": false, + "x_feasible": true, + "x_true_obj": 1.2686437852381225 + }, + "on": { + "wall": 13.62224176899963, + "status": "feasible", + "obj": 1.2686437852381225, + "bound": 0.8528765311589279, + "gapc": false, + "nodes": 19, + "ivf": false, + "x_feasible": true, + "x_true_obj": 1.2686437852381225 + } + }, + "tls2": { + "name": "tls2", + "off": { + "wall": 20.072858062000705, + "status": "feasible", + "obj": 11.299999856725538, + "bound": 2.0999999901667694, + "gapc": false, + "nodes": 317, + "ivf": false, + "x_feasible": true, + "x_true_obj": 11.3 + }, + "on": { + "wall": 13.583608351000294, + "status": "time_limit", + "obj": null, + "bound": null, + "gapc": false, + "nodes": 95, + "ivf": false + } } } \ No newline at end of file From 668dead8ec1bae00841d0c7529a35d714a36f1c1 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 26 Jul 2026 16:15:08 +0000 Subject: [PATCH 10/12] =?UTF-8?q?docs(#860):=20record=20the=20Panel=20B=20?= =?UTF-8?q?verdict=20=E2=80=94=20cert-clean=20but=20NOT=20net-positive,=20?= =?UTF-8?q?flag=20stays=20OFF?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Graduation panel for DISCOPT_LP_SPATIAL_MIXED (20s budget, 70 newly in-scope in-repo instances, off vs on; the other 49 take a bit-identical path since the reserve gate is the flag's only consumer): cert-clean 0 certification regressions, 0 incumbent_verification_failed, 0 unsound bounds, 0 false optimality certificates net-positive FAILED — gains 1, improved 1, LOST 2 tspn12 no incumbent (30.6s) -> feasible 262.647 (9.3s) ex1252a feasible 183660.35 (24.5s) -> feasible 149530.99 (14.9s) tls2 feasible 11.30 (20.1s) -> NO INCUMBENT (13.6s) st_e31 feasible -2.00 (22.2s) -> NO INCUMBENT (14.8s) The widening is sound — Panel A already showed that, with 33 newly reachable instances gaining a verified incumbent and 0 unsound results. What it cannot do is pay for itself on the default path. The reserve hands 35% of the budget to the fallback BEFORE knowing whether the engine can serve the model: on tls2 and st_e31 the primary then runs out of time at 65% and returns nothing, while the fallback declines anyway (require_incremental=True + an infinite root box). Budget taken from a path that was going to succeed, given to a path that never runs. Sound but harmful stays OFF with the measurement recorded — the DISCOPT_CUT_INHERIT rule. The 0.747 wall ratio is not a win: the on-runs are faster largely because they gave up earlier, on exactly the two instances that lost their answer. The fix this points at (make the reserve conditional on buildability) is a separate change with its own panel. Also: the analyzer now classifies objective drift by sense, so a better incumbent (ex1252a) is not scored as harm. Separately, Panel B's independent verifier rejects nvs22's incumbent at tol 1e-5 under BOTH settings. Not from this branch: byte-identical against base ac9f5cf (objective 6.058219942618198, max violation 2.641e-4) — above the panel's 1e-5, below the deliberately loose 1e-3 of discopt's own final guard, so nothing flags it today. Worth its own investigation. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_019bsy1zfNW8H1hZGg9SKfPu --- docs/dev/issue-860-lp-spatial-mixed-scope.md | 60 +- python/discopt/modeling/core.py | 41 +- scratchpad/panel860_analyze.py | 12 +- scratchpad/panel860_flag.json | 676 +++++++++++++++++++ 4 files changed, 779 insertions(+), 10 deletions(-) diff --git a/docs/dev/issue-860-lp-spatial-mixed-scope.md b/docs/dev/issue-860-lp-spatial-mixed-scope.md index d4224349..68e61786 100644 --- a/docs/dev/issue-860-lp-spatial-mixed-scope.md +++ b/docs/dev/issue-860-lp-spatial-mixed-scope.md @@ -1,8 +1,9 @@ # Issue #860 — widening the LP-per-node engine to mixed-integer and MAXIMIZE models -*Status: implemented. Engine scope widened unconditionally; the default-path -fallback reserve for the newly in-scope class is behind `DISCOPT_LP_SPATIAL_MIXED` -(default OFF) pending its graduation panel.* +*Status: implemented. Engine scope widened unconditionally (Panel A: cert-clean, 33 +newly reachable instances get a verified incumbent). The default-path fallback reserve +for the newly in-scope class stays behind `DISCOPT_LP_SPATIAL_MIXED`, **default OFF**: +its graduation panel is cert-clean but not net-positive (§4, Panel B).* The LP-per-node spatial engine (`_jax/lp_spatial_bb.py`) was gated to **pure-integer, MINIMIZE** models. Issue #860 records the consequence: three instances named in #844 @@ -231,7 +232,58 @@ newly in-scope instances — the other 49 take a bit-identical path under either setting, since the reserve gate is the flag's only consumer and both gates agree there. Graduation needs both bars of CLAUDE.md §5: cert-clean **and** net-positive. -PANEL_B_PLACEHOLDER +**Verdict: cert-clean, but NOT net-positive. The flag stays default-OFF.** + +| bar | result | +|---|---| +| certification regressions (`gap_certified` True→False) | **0** | +| `incumbent_verification_failed` | **0** | +| unsound bound / false optimality certificate | **0** | +| objective improved | 1 — `ex1252a` 183660.35 → **149530.99** (minimize; both independently feasible) | +| incumbents **gained** | 1 — `tspn12` 262.647 (independently feasible) | +| incumbents **LOST** | **2 — `tls2`, `st_e31`** | + +| instance | flag off | flag on | +|---|---|---| +| `tspn12` | time_limit, no incumbent (30.6 s) | feasible **262.647** (9.3 s) | +| `ex1252a` | feasible 183660.35 (24.5 s) | feasible **149530.99** (14.9 s) | +| `tls2` | feasible **11.30** (20.1 s) | time_limit, **no incumbent** (13.6 s) | +| `st_e31` | feasible **−2.00** (22.2 s) | time_limit, **no incumbent** (14.8 s) | + +Soundness is untouched — the widening cannot produce a bad answer, exactly as Panel A +showed. What it cannot do is *pay for itself on the default path*: one incumbent +gained and one improved against two lost. Losing an incumbent is strictly worse than +improving one that already exists, so the net is negative and the flag does not +graduate. This is the `DISCOPT_CUT_INHERIT` lesson again (CLAUDE.md §5): sound ≠ +helpful, and a cert-clean but harmful flag stays off with its measurement recorded. + +**Mechanism of the two losses**, which is the useful part of the result. The reserve +hands 35% of the budget to the fallback *before* knowing whether the engine can serve +the model. On `tls2` and `st_e31` the primary then runs out of time at 65% and returns +nothing — and the fallback declines anyway, because it passes `require_incremental=True` +and both models have infinite root boxes, which decline the incremental structure +(§2.4). The reserve is pure loss there: budget taken from a path that was going to +succeed, given to a path that never runs. + +Do not read the 0.747 wall ratio as a win. The flag-on runs are faster mostly *because* +they gave up earlier — `tls2` 20.1 s → 13.6 s and `st_e31` 22.2 s → 14.8 s are exactly +the two instances that stopped producing an answer. + +**The concrete follow-up** this points at: make the reserve conditional on the engine +actually being able to build (probe buildability, or relax `require_incremental` for +the mixed class now that cold node builds are deadline-bounded per §2.5), so a model +the fallback will decline never pays for it. That is a separate change with its own +panel, not a tweak to this one. + +### A pre-existing soundness flag, not from this work + +Panel B's independent verifier rejects `nvs22`'s reported incumbent at tolerance 1e-5 +under **both** flag settings. It is not caused by the flag, and not by this branch: +run against the base commit `ac9f5cf` the result is byte-identical — objective +`6.058219942618198`, `gap_certified=True`, maximum constraint violation +`2.641e-4` on row 2. That sits above the 1e-5 the panel checks and below the +deliberately loose 1e-3 of discopt's own final `incumbent_verification_failed` guard, +so nothing flags it today. Worth its own investigation; out of scope here. ## 5. Known limits (not addressed here) diff --git a/python/discopt/modeling/core.py b/python/discopt/modeling/core.py index b8d176ce..f892792e 100644 --- a/python/discopt/modeling/core.py +++ b/python/discopt/modeling/core.py @@ -116,9 +116,44 @@ def _lp_spatial_mixed_fallback_enabled() -> bool: widening — an in-scope model hands 35% of its budget to a pass that may find nothing — so it is a separate, measurable decision from the engine's capability. - **Default OFF**; opt in with ``DISCOPT_LP_SPATIAL_MIXED=1``. Graduation requires - the corpus-wide differential panel of CLAUDE.md §5 (cert-clean AND net-positive) - over the mixed class, not just the absence of harm. + **Default OFF** — it ran its graduation panel and did NOT graduate. Opt in with + ``DISCOPT_LP_SPATIAL_MIXED=1``. + + **Graduation panel** (20 s budget, 70 newly in-scope in-repo instances, off vs on; + the other 49 take a bit-identical path since this gate is the flag's only + consumer). *Cert-clean*: 0 certification regressions, 0 + ``incumbent_verification_failed``, 0 unsound bounds. *Net-positive*: **failed**. + + ========== ============================== ============================== + instance off on + ========== ============================== ============================== + tspn12 no incumbent (30.6 s) feasible 262.647 (9.3 s) + ex1252a feasible 183660.35 (24.5 s) feasible 149530.99 (14.9 s) + tls2 feasible 11.30 (20.1 s) NO INCUMBENT (13.6 s) + st_e31 feasible -2.00 (22.2 s) NO INCUMBENT (14.8 s) + ========== ============================== ============================== + + ``gains=1 improved=1 lost_incumbents=2 cert_regressions=0 unsound=0``. + + The widening is sound (see ``lp_spatial_bb`` and the #860 Panel A: 33 newly + reachable instances get a verified incumbent, 0 unsound results); what it cannot do + is pay for itself *here*. The reserve hands 35% of the budget to the fallback before + knowing whether the engine can serve the model: on tls2 and st_e31 the primary then + runs out of time at 65% and returns nothing, while the fallback declines anyway + (``require_incremental=True`` + an infinite root box, which declines the incremental + structure). Budget taken from a path that was going to succeed, given to a path that + never runs. Sound but harmful stays OFF, with the measurement recorded — the + ``DISCOPT_CUT_INHERIT`` rule (CLAUDE.md §5). + + Do not read the 0.747 total wall ratio as a win: the on-runs are faster largely + because they gave up earlier, on exactly the two instances that lost their answer. + + **What would change the verdict**: making the reserve conditional on the engine + actually being able to build (probe buildability, or relax ``require_incremental`` + for the mixed class now that cold node builds are deadline-bounded), so a model the + fallback will decline never pays for it. Separate change, separate panel. Evidence: + ``docs/dev/issue-860-lp-spatial-mixed-scope.md`` §4, + ``scratchpad/panel860_flag.json``. """ import os as _os diff --git a/scratchpad/panel860_analyze.py b/scratchpad/panel860_analyze.py index 0029a4ee..aa1bc101 100644 --- a/scratchpad/panel860_analyze.py +++ b/scratchpad/panel860_analyze.py @@ -89,7 +89,13 @@ def main(): losses.append(name) if off.get("obj") is not None and on.get("obj") is not None: if rel(on["obj"], off["obj"]) > 1e-3: - drift.append((name, off["obj"], on["obj"])) + # Classify by sense: a drift toward the optimum is an improvement, not + # a regression. Reporting the bare magnitude would score a better + # incumbent as harm. + better = ( + on["obj"] > off["obj"] if sense.get(name) == "max" else on["obj"] < off["obj"] + ) + drift.append((name, off["obj"], on["obj"], "better" if better else "WORSE")) for key, run in (("off", off), ("on", on)): if run.get("obj") is not None and run.get("x_feasible") is False: unsound.append((name, f"{key}: reported incumbent NOT feasible")) @@ -111,8 +117,8 @@ def main(): print(f" cert regressions (True->False): {len(cert_reg)} {cert_reg}") print(f" incumbent-verification failed : {len(ivf)} {ivf}") print(f" objective drift > 1e-3 : {len(drift)}") - for n, o, w in drift: - print(f" {n}: {o} -> {w}") + for n, o, w, verdict in drift: + print(f" {n}: {o} -> {w} ({verdict})") print(f" incumbents GAINED : {len(gains)}") for n, o, ok in gains: print(f" {n}: {o} (independently feasible: {ok})") diff --git a/scratchpad/panel860_flag.json b/scratchpad/panel860_flag.json index 1d3cdf66..5219dee3 100644 --- a/scratchpad/panel860_flag.json +++ b/scratchpad/panel860_flag.json @@ -998,5 +998,681 @@ "nodes": 95, "ivf": false } + }, + "tspn05": { + "name": "tspn05", + "off": { + "wall": 20.585565102000146, + "status": "feasible", + "obj": 191.2552076848774, + "bound": 179.44337578420988, + "gapc": false, + "nodes": 43, + "ivf": false, + "x_feasible": true, + "x_true_obj": 191.2552076848774 + }, + "on": { + "wall": 13.694829500000196, + "status": "feasible", + "obj": 191.2552076848774, + "bound": 178.27199442798212, + "gapc": false, + "nodes": 33, + "ivf": false, + "x_feasible": true, + "x_true_obj": 191.2552076848774 + } + }, + "tspn08": { + "name": "tspn08", + "off": { + "wall": 17.491286255999512, + "status": "feasible", + "obj": 290.56685374735093, + "bound": null, + "gapc": false, + "nodes": 1, + "ivf": false, + "x_feasible": true, + "x_true_obj": 290.56685374735093 + }, + "on": { + "wall": 15.73233596999944, + "status": "feasible", + "obj": 290.56685374735093, + "bound": null, + "gapc": false, + "nodes": 1, + "ivf": false, + "x_feasible": true, + "x_true_obj": 290.56685374735093 + } + }, + "tspn10": { + "name": "tspn10", + "off": { + "wall": 15.337441571000454, + "status": "feasible", + "obj": 225.1260713973286, + "bound": null, + "gapc": false, + "nodes": 1, + "ivf": false, + "x_feasible": true, + "x_true_obj": 225.1260713973286 + }, + "on": { + "wall": 11.317079618999742, + "status": "feasible", + "obj": 225.1260713973286, + "bound": null, + "gapc": false, + "nodes": 1, + "ivf": false, + "x_feasible": true, + "x_true_obj": 225.1260713973286 + } + }, + "tspn12": { + "name": "tspn12", + "off": { + "wall": 30.604999670000325, + "status": "time_limit", + "obj": null, + "bound": null, + "gapc": false, + "nodes": 3, + "ivf": false + }, + "on": { + "wall": 9.307628071000181, + "status": "feasible", + "obj": 262.64739525060224, + "bound": null, + "gapc": false, + "nodes": 1, + "ivf": false, + "x_feasible": true, + "x_true_obj": 262.64739525060224 + } + }, + "carton7": { + "name": "carton7", + "off": { + "wall": 32.65056300500055, + "status": "time_limit", + "obj": null, + "bound": 26.0, + "gapc": false, + "nodes": 7, + "ivf": false + }, + "on": { + "wall": 39.27721478100011, + "status": "time_limit", + "obj": null, + "bound": 26.0, + "gapc": false, + "nodes": 7, + "ivf": false + } + }, + "ex1223a": { + "name": "ex1223a", + "off": { + "wall": 0.421073391000391, + "status": "optimal", + "obj": 4.579582406521569, + "bound": 4.579582397947032, + "gapc": true, + "nodes": 3, + "ivf": false, + "x_feasible": true, + "x_true_obj": 4.579582406521569 + }, + "on": { + "wall": 0.3097158339996895, + "status": "optimal", + "obj": 4.579582406521569, + "bound": 4.579582397947032, + "gapc": true, + "nodes": 3, + "ivf": false, + "x_feasible": true, + "x_true_obj": 4.579582406521569 + } + }, + "ex1233": { + "name": "ex1233", + "off": { + "wall": 22.867122020999886, + "status": "time_limit", + "obj": null, + "bound": 109649.41590017003, + "gapc": false, + "nodes": 63, + "ivf": false + }, + "on": { + "wall": 14.191701922999528, + "status": "time_limit", + "obj": null, + "bound": 109649.41590017003, + "gapc": false, + "nodes": 31, + "ivf": false + } + }, + "ex1252": { + "name": "ex1252", + "off": { + "wall": 20.293754636999438, + "status": "time_limit", + "obj": null, + "bound": -4.25657e-319, + "gapc": false, + "nodes": 3, + "ivf": false + }, + "on": { + "wall": 15.5271799559996, + "status": "time_limit", + "obj": null, + "bound": -4.25657e-319, + "gapc": false, + "nodes": 3, + "ivf": false + } + }, + "ex1252a": { + "name": "ex1252a", + "off": { + "wall": 24.459606012999757, + "status": "feasible", + "obj": 183660.35482126236, + "bound": -4.25657e-319, + "gapc": false, + "nodes": 3, + "ivf": false, + "x_feasible": true, + "x_true_obj": 183660.35482126236 + }, + "on": { + "wall": 14.907741936999628, + "status": "feasible", + "obj": 149530.99498512247, + "bound": -4.25657e-319, + "gapc": false, + "nodes": 3, + "ivf": false, + "x_feasible": true, + "x_true_obj": 149530.99498512247 + } + }, + "ex1263": { + "name": "ex1263", + "off": { + "wall": 1.8693513609996444, + "status": "optimal", + "obj": 19.6, + "bound": 19.6, + "gapc": true, + "nodes": 7455, + "ivf": false, + "x_feasible": true, + "x_true_obj": 19.6 + }, + "on": { + "wall": 1.7884880320007142, + "status": "optimal", + "obj": 19.6, + "bound": 19.6, + "gapc": true, + "nodes": 7455, + "ivf": false, + "x_feasible": true, + "x_true_obj": 19.6 + } + }, + "ex1264": { + "name": "ex1264", + "off": { + "wall": 2.4814531459996942, + "status": "optimal", + "obj": 8.6, + "bound": 8.6, + "gapc": true, + "nodes": 12193, + "ivf": false, + "x_feasible": true, + "x_true_obj": 8.6 + }, + "on": { + "wall": 2.4087952629997744, + "status": "optimal", + "obj": 8.6, + "bound": 8.6, + "gapc": true, + "nodes": 12193, + "ivf": false, + "x_feasible": true, + "x_true_obj": 8.6 + } + }, + "ex1265": { + "name": "ex1265", + "off": { + "wall": 2.2513344270000744, + "status": "optimal", + "obj": 10.299999999999999, + "bound": 10.299999999999999, + "gapc": true, + "nodes": 1471, + "ivf": false, + "x_feasible": true, + "x_true_obj": 10.299999999999999 + }, + "on": { + "wall": 2.3213560329995744, + "status": "optimal", + "obj": 10.299999999999999, + "bound": 10.299999999999999, + "gapc": true, + "nodes": 1471, + "ivf": false, + "x_feasible": true, + "x_true_obj": 10.299999999999999 + } + }, + "ex1266": { + "name": "ex1266", + "off": { + "wall": 5.946768261000216, + "status": "optimal", + "obj": 16.3, + "bound": 16.3, + "gapc": true, + "nodes": 7429, + "ivf": false, + "x_feasible": true, + "x_true_obj": 16.3 + }, + "on": { + "wall": 5.711828896999577, + "status": "optimal", + "obj": 16.3, + "bound": 16.3, + "gapc": true, + "nodes": 7429, + "ivf": false, + "x_feasible": true, + "x_true_obj": 16.3 + } + }, + "fuel": { + "name": "fuel", + "off": { + "wall": 3.2344341500001974, + "status": "optimal", + "obj": 8566.118923054262, + "bound": 8566.118923028069, + "gapc": true, + "nodes": 3, + "ivf": false, + "x_feasible": true, + "x_true_obj": 8566.118923054262 + }, + "on": { + "wall": 2.714808117000757, + "status": "optimal", + "obj": 8566.118923054262, + "bound": 8566.118923028069, + "gapc": true, + "nodes": 3, + "ivf": false, + "x_feasible": true, + "x_true_obj": 8566.118923054262 + } + }, + "gear2": { + "name": "gear2", + "off": { + "wall": 3.676273637000122, + "status": "optimal", + "obj": 1.155529714729433e-07, + "bound": -5e-324, + "gapc": true, + "nodes": 3, + "ivf": false, + "x_feasible": true, + "x_true_obj": 1.155529714729433e-07 + }, + "on": { + "wall": 3.619437115999972, + "status": "optimal", + "obj": 1.2492237047931615e-07, + "bound": -5e-324, + "gapc": true, + "nodes": 91, + "ivf": false, + "x_feasible": true, + "x_true_obj": 1.2492237047931615e-07 + } + }, + "gear3": { + "name": "gear3", + "off": { + "wall": 0.9851909570006683, + "status": "optimal", + "obj": 1.827380235298078e-08, + "bound": -5e-324, + "gapc": true, + "nodes": 3, + "ivf": false, + "x_feasible": true, + "x_true_obj": 1.827380235298078e-08 + }, + "on": { + "wall": 0.8173131650000869, + "status": "optimal", + "obj": 1.827380235298078e-08, + "bound": -5e-324, + "gapc": true, + "nodes": 3, + "ivf": false, + "x_feasible": true, + "x_true_obj": 1.827380235298078e-08 + } + }, + "gear4": { + "name": "gear4", + "off": { + "wall": 1.175519556000836, + "status": "optimal", + "obj": 1.6434284641271995, + "bound": 1.643427471831146, + "gapc": true, + "nodes": 3, + "ivf": false, + "x_feasible": true, + "x_true_obj": 1.6434284641271995 + }, + "on": { + "wall": 0.8973954729999605, + "status": "optimal", + "obj": 1.6434284641271995, + "bound": 1.643427471831146, + "gapc": true, + "nodes": 3, + "ivf": false, + "x_feasible": true, + "x_true_obj": 1.6434284641271995 + } + }, + "meanvarx": { + "name": "meanvarx", + "off": { + "wall": 0.3575182799995673, + "status": "optimal", + "obj": 14.369231356314272, + "bound": 14.369231356314272, + "gapc": true, + "nodes": 17, + "ivf": false, + "x_feasible": true, + "x_true_obj": 14.369231356314275 + }, + "on": { + "wall": 0.3192492649995984, + "status": "optimal", + "obj": 14.369231356314272, + "bound": 14.369231356314272, + "gapc": true, + "nodes": 17, + "ivf": false, + "x_feasible": true, + "x_true_obj": 14.369231356314275 + } + }, + "nvs20": { + "name": "nvs20", + "off": { + "wall": 20.534779711999363, + "status": "feasible", + "obj": 230.92216276477316, + "bound": 224.8139291270698, + "gapc": false, + "nodes": 43, + "ivf": false, + "x_feasible": true, + "x_true_obj": 230.92216300555503 + }, + "on": { + "wall": 13.589719059000345, + "status": "feasible", + "obj": 230.9221630055551, + "bound": 224.62037670900688, + "gapc": false, + "nodes": 29, + "ivf": false, + "x_feasible": true, + "x_true_obj": 230.9221630055551 + } + }, + "nvs22": { + "name": "nvs22", + "off": { + "wall": 12.486188273000153, + "status": "optimal", + "obj": 6.058219942618198, + "bound": 6.058219942618198, + "gapc": true, + "nodes": 35, + "ivf": false, + "x_feasible": false, + "x_true_obj": 6.05822 + }, + "on": { + "wall": 12.914577578000717, + "status": "optimal", + "obj": 6.0582199426182, + "bound": 6.0582199426182, + "gapc": true, + "nodes": 35, + "ivf": false, + "x_feasible": false, + "x_true_obj": 6.05822 + } + }, + "prob10": { + "name": "prob10", + "off": { + "wall": 1.041906168999958, + "status": "optimal", + "obj": 3.4455037948230816, + "bound": 3.445503788218336, + "gapc": true, + "nodes": 15, + "ivf": false, + "x_feasible": true, + "x_true_obj": 3.4455037948230816 + }, + "on": { + "wall": 0.9212320480000926, + "status": "optimal", + "obj": 3.4455037948230816, + "bound": 3.445503788218336, + "gapc": true, + "nodes": 15, + "ivf": false, + "x_feasible": true, + "x_true_obj": 3.4455037948230816 + } + }, + "st_e15": { + "name": "st_e15", + "off": { + "wall": 0.7076264639999863, + "status": "optimal", + "obj": 7.667180068813135, + "bound": 7.667180068813077, + "gapc": true, + "nodes": 5, + "ivf": false, + "x_feasible": true, + "x_true_obj": 7.667180068813135 + }, + "on": { + "wall": 0.7600795540001855, + "status": "optimal", + "obj": 7.667180068813135, + "bound": 7.667180068813077, + "gapc": true, + "nodes": 5, + "ivf": false, + "x_feasible": true, + "x_true_obj": 7.667180068813135 + } + }, + "st_e27": { + "name": "st_e27", + "off": { + "wall": 0.6037468499998795, + "status": "optimal", + "obj": 1.9999999826304504, + "bound": 1.999999958156458, + "gapc": true, + "nodes": 3, + "ivf": false, + "x_feasible": true, + "x_true_obj": 1.9999999826304504 + }, + "on": { + "wall": 0.788995476999844, + "status": "optimal", + "obj": 1.9999999826304504, + "bound": 1.999999958156458, + "gapc": true, + "nodes": 3, + "ivf": false, + "x_feasible": true, + "x_true_obj": 1.9999999826304504 + } + }, + "st_e31": { + "name": "st_e31", + "off": { + "wall": 22.218101848000515, + "status": "feasible", + "obj": -2.00000002363656, + "bound": -3.0, + "gapc": false, + "nodes": 73, + "ivf": false, + "x_feasible": true, + "x_true_obj": -2.00000002363656 + }, + "on": { + "wall": 14.804342561999874, + "status": "time_limit", + "obj": null, + "bound": -3.0, + "gapc": false, + "nodes": 127, + "ivf": false + } + }, + "st_e35": { + "name": "st_e35", + "off": { + "wall": 20.906576803000462, + "status": "time_limit", + "obj": null, + "bound": 13756.629739576067, + "gapc": false, + "nodes": 7, + "ivf": false + }, + "on": { + "wall": 13.881900205999955, + "status": "time_limit", + "obj": null, + "bound": 13756.629739576067, + "gapc": false, + "nodes": 7, + "ivf": false + } + }, + "st_e40": { + "name": "st_e40", + "off": { + "wall": 11.034250014000463, + "status": "optimal", + "obj": 30.4142135, + "bound": 30.4142135, + "gapc": true, + "nodes": 5, + "ivf": false, + "x_feasible": true, + "x_true_obj": 30.4142135 + }, + "on": { + "wall": 10.669773696999982, + "status": "optimal", + "obj": 30.4142135, + "bound": 30.4142135, + "gapc": true, + "nodes": 5, + "ivf": false, + "x_feasible": true, + "x_true_obj": 30.4142135 + } + }, + "syn05m": { + "name": "syn05m", + "off": { + "wall": 1.4015699870005847, + "status": "optimal", + "obj": 837.7323995316158, + "bound": 837.7324096873247, + "gapc": true, + "nodes": 9, + "ivf": false, + "x_feasible": true, + "x_true_obj": 837.7323995316158 + }, + "on": { + "wall": 1.3700668180008506, + "status": "optimal", + "obj": 837.7323995316158, + "bound": 837.7324096873247, + "gapc": true, + "nodes": 9, + "ivf": false, + "x_feasible": true, + "x_true_obj": 837.7323995316158 + } + }, + "util": { + "name": "util", + "off": { + "wall": 22.261656106999908, + "status": "time_limit", + "obj": null, + "bound": 999.2752969072262, + "gapc": false, + "nodes": 159, + "ivf": false + }, + "on": { + "wall": 16.576981659000012, + "status": "time_limit", + "obj": null, + "bound": 999.2752969072262, + "gapc": false, + "nodes": 95, + "ivf": false + } } } \ No newline at end of file From 81fd3e9b376e6116e37381129b21169ea745fb39 Mon Sep 17 00:00:00 2001 From: John Kitchin Date: Sun, 26 Jul 2026 18:39:59 -0400 Subject: [PATCH 11/12] fix(#860): gate the mixed/MAXIMIZE widening behind DISCOPT_LP_SPATIAL_MIXED Review outcome on #874. The widening is sound and stays; what changes is that it no longer ships by DEFAULT, because it is not net-positive on the default path (CLAUDE.md 5 bar 2). The flag already governed the #844 fallback's 35% budget reserve, but the ENGINE gate was unconditional, so solve(lp_spatial=True) -- a documented public kwarg -- picked up the widened scope with no opt-in. Measured on gear4, which is mixed (4 integer, 2 continuous with infinite upper bounds) and therefore admitted only under the widening, at a 25 s budget: DISCOPT_LP_SPATIAL_MIXED=0 (default): optimal, 1.6434284641, certified, 3 nodes, 1.1 s DISCOPT_LP_SPATIAL_MIXED=1: time_limit, 17.514, UNcertified, 2678 nodes, 25.0 s The engine accepts a model the default path already certified in 3 nodes, then spends the whole budget to return an incumbent ~10.7x worse with no certificate. It is SOUND -- the bound never crosses the oracle, and the false certificate seen on gear4 earlier in review was the LP-presolve INF-sentinel bug fixed in #877, not this widening -- but shipping it by default trades a certificate for a worse incumbent. Changes: * solver.py's lp_spatial=True entry point passes mixed=_lp_spatial_mixed_fallback_enabled(); the #844 fallback already did. * solve_lp_spatial_bb(mixed=...) and _is_in_scope(mixed=...) now default to False, so a NEW call site inherits the pre-#860 gate rather than silently shipping the widening. Same rationale as row_scan_is_anytime defaulting to False in the tightening rules. * The flag docstring now describes both entry points it governs and records the gear4 measurement alongside the existing Panel B table. * Five tests in test_lp_spatial_bb.py exercise the widened CAPABILITY, so they now request it explicitly with mixed=True. The two tests that assert the pre-#860 gate declines (mixed=False) are unchanged and still pass. Also in this branch: merged main and resolved two conflicts by COMPOSITION, not by choosing a side (rationale recorded in comments at both sites). * modeling/core.py -- main (#861 review) moved the fallback dual-bound merge out of the `_fb.objective is not None` block so a sound bound is reported even with no incumbent; this branch made the same merge sense-aware (min for MAXIMIZE, where the dual bound is an UPPER bound and max would keep the looser one). Kept main's placement AND this branch's sense-awareness. * _jax/incremental_mccormick.py -- main relaxed _validate to compare shape[1] with vacuity-filtered row sets, bounds checked first, which is what lets a PINNED box validate; this branch added the obj_offset box-independence assertion. Kept main's structure AND this branch's assertion. Tests: python/tests/test_860_mixed_gate_is_opt_in.py -- 6 tests pinning the GATING rather than wall-clock, so they cannot rot into machine-speed assertions: mixed=True admits mixed-minimize and integer-maximize while mixed=False declines both (with pure-integer-minimize as a control that must be admitted either way), both defaults are False, a source-level guard fails if either production call site stops threading the flag, and the env-var helper is off when unset. Fails-before/passes-after verified by stashing the three source files: 2 failed / 4 passed before, 6 passed after. Verified: pytest -m smoke -> 844 passed, 1 skipped, 2 xpassed; pytest -m slow test_adversarial_recent_fixes.py -> 10 passed; ruff check python/ clean; ruff format clean; no Rust touched. Also measured, and the reason this is not "Closes": at time_limit=60 with the flag ON, all three instances #860 names are now IN SCOPE (_is_in_scope True, where they previously failed the gate) but NONE returns an incumbent -- rsyn0805m04hfsg (MAXIMIZE) bound 9741.371 vs oracle 7174.219, gastrans040 bound 0.0, gastrans582_cold13 bound 0.0, objective None on all three, 6 soundness checks and 0 violations. #860's stated gap is that these three return no incumbent, so it remains open for the primal work. Contributes to #860 --- python/discopt/_jax/lp_spatial_bb.py | 14 +- python/discopt/modeling/core.py | 44 ++++-- python/discopt/solver.py | 25 ++- python/tests/test_860_mixed_gate_is_opt_in.py | 142 ++++++++++++++++++ python/tests/test_lp_spatial_bb.py | 10 +- 5 files changed, 211 insertions(+), 24 deletions(-) create mode 100644 python/tests/test_860_mixed_gate_is_opt_in.py diff --git a/python/discopt/_jax/lp_spatial_bb.py b/python/discopt/_jax/lp_spatial_bb.py index 92d4d054..8be89548 100644 --- a/python/discopt/_jax/lp_spatial_bb.py +++ b/python/discopt/_jax/lp_spatial_bb.py @@ -88,7 +88,7 @@ class LpSpatialResult(NamedTuple): node_count: int -def _is_in_scope(model: Model, *, mixed: bool = True) -> bool: +def _is_in_scope(model: Model, *, mixed: bool = False) -> bool: """Model this engine can serve: an objective, at least one integer variable, and only ordinary algebraic rows. Either objective sense; any continuous mix. @@ -322,7 +322,7 @@ def solve_lp_spatial_bb( use_obbt: bool = True, root_cut_rounds: int = 0, require_incremental: bool = False, - mixed: bool = True, + mixed: bool = False, ) -> Optional[LpSpatialResult]: """LP-node spatial branch-and-bound. Returns ``None`` if out of scope. @@ -351,8 +351,14 @@ def solve_lp_spatial_bb( node outweigh the modest tightening — measured net-negative on nvs17/19/24. The machinery is sound and kept opt-in for when a fast native separator exists. - ``mixed`` (default True, #860) admits mixed-integer and MAXIMIZE models. Pass - ``False`` for the pre-#860 pure-integer/MINIMIZE gate.""" + ``mixed`` (#860) admits mixed-integer and MAXIMIZE models; ``False`` is the + pre-#860 pure-integer/MINIMIZE gate. **Default False**: the widening is a real + capability but it is not net-positive on the default path (CLAUDE.md §5 bar 2 — + see ``_lp_spatial_mixed_fallback_enabled``), so both production call sites pass + ``mixed=_lp_spatial_mixed_fallback_enabled()`` rather than relying on this + default. Defaulting to ``False`` means a *new* call site inherits the + conservative gate instead of silently shipping the widening, which is the same + reason ``row_scan_is_anytime`` defaults to ``False`` in the tightening rules.""" if not _is_in_scope(model, mixed=mixed): return None diff --git a/python/discopt/modeling/core.py b/python/discopt/modeling/core.py index 53e69614..6d7cbb4f 100644 --- a/python/discopt/modeling/core.py +++ b/python/discopt/modeling/core.py @@ -123,17 +123,39 @@ def _lp_spatial_fallback_enabled() -> bool: def _lp_spatial_mixed_fallback_enabled() -> bool: - """#860: extend the #844 no-incumbent fallback's *scope gate* to mixed-integer and - MAXIMIZE models. - - The engine itself serves those models unconditionally since #860 (see - ``lp_spatial_bb._is_in_scope``); this flag governs only whether the DEFAULT path - reserves budget for the fallback on them. That reserve is the sole risk of the - widening — an in-scope model hands 35% of its budget to a pass that may find - nothing — so it is a separate, measurable decision from the engine's capability. - - **Default OFF** — it ran its graduation panel and did NOT graduate. Opt in with - ``DISCOPT_LP_SPATIAL_MIXED=1``. + """#860: extend the LP-per-node engine's *scope gate* to mixed-integer and MAXIMIZE + models. + + Governs **both** production entry points, so the widening is entirely opt-in: + + * ``solve(lp_spatial=True)`` (``solver.py``) — whether the engine accepts a mixed + or maximize model at all; and + * the #844 no-incumbent fallback (below) — whether the DEFAULT path reserves 35% of + its budget for the fallback on such a model. + + The engine's ``mixed`` parameter and ``_is_in_scope``'s keyword both default to + ``False`` as well, so a *new* call site inherits the pre-#860 gate rather than + silently shipping the widening. + + **Default OFF** — it ran its graduation panel and did NOT graduate, on *both* + counts. Opt in with ``DISCOPT_LP_SPATIAL_MIXED=1``. + + On the public ``lp_spatial=True`` path the loss is direct. ``gear4`` is mixed (4 + integer, 2 continuous with infinite upper bounds), so it is admitted only under the + widening; at a 25 s budget: + + ============== ============================================ + gate result + ============== ============================================ + pre-#860 ``optimal``, 1.6434284641, certified, 3 nodes + widened ``time_limit``, 17.514, uncertified, 2673 nodes + ============== ============================================ + + The engine accepts a model the default path already certified in 3 nodes, then + spends the entire budget to return an incumbent ~10.7x worse with no certificate. + It is *sound* — the bound never crosses the oracle, and the false certificate seen + there earlier was the LP-presolve sentinel bug fixed in #877, not this widening — + but shipping it by default would be trading a certificate for a worse incumbent. **Graduation panel** (20 s budget, 70 newly in-scope in-repo instances, off vs on; the other 49 take a bit-identical path since this gate is the flag's only diff --git a/python/discopt/solver.py b/python/discopt/solver.py index 4373e284..abf7dbc6 100644 --- a/python/discopt/solver.py +++ b/python/discopt/solver.py @@ -4681,14 +4681,30 @@ def _deadline_exhausted(floor: float = _DEADLINE_NODE_FLOOR_S) -> bool: # branches on integers/products and runs a feasibility-pump primal, closing # nvs17 to proven optimality. Opt-in via ``solve(lp_spatial=True)``; returns # ``None`` (falls through to the default path, no behavior change) for any model - # out of its scope or on any error. Since #860 the scope is "at least one integer + # out of its scope or on any error. #860 widens that scope to "at least one integer # variable, either objective sense, any continuous mix" — mixed-integer and - # maximize models are served in minimize-equivalent space, and a partially - # infinite root box is accepted (root OBBT finitizes it; the cold builder drops - # any row it cannot make finite). + # maximize models served in minimize-equivalent space, with a partially infinite + # root box accepted — but the widening is BEHIND ``DISCOPT_LP_SPATIAL_MIXED`` + # here too, not just on the #844 fallback's reserve. + # + # Why the gate is flagged and not only the reserve (#860 review): the widened gate + # is not net-positive on the default path, which is CLAUDE.md §5 bar (2). Measured + # on ``gear4`` at a 25 s budget — a MIXED model (4 integer, 2 continuous with + # infinite upper bounds), so it is admitted only under the widening: + # + # pre-#860 gate : optimal, objective = 1.6434284641, certified, 3 nodes + # widened gate : time_limit, objective = 17.514, UNcertified, 2673 nodes + # + # i.e. the engine accepts a model the default path already certified in 3 nodes, + # then spends the whole budget to return an incumbent ~10.7x worse with no + # certificate. Sound (the bound never crosses the oracle; the earlier false + # certificate there was the LP-presolve bug fixed in #877) but a clear regression, + # and ``lp_spatial=True`` is a documented public kwarg. The capability is kept and + # is opt-in; what is not shipped by default is a measured loss. if kwargs.get("lp_spatial", False): try: from discopt._jax.lp_spatial_bb import solve_lp_spatial_bb + from discopt.modeling.core import _lp_spatial_mixed_fallback_enabled _lps = solve_lp_spatial_bb( model, @@ -4696,6 +4712,7 @@ def _deadline_exhausted(floor: float = _DEADLINE_NODE_FLOOR_S) -> bool: gap_tolerance=gap_tolerance, max_nodes=max_nodes, root_cut_rounds=int(kwargs.get("lp_spatial_cut_rounds", 0)), + mixed=_lp_spatial_mixed_fallback_enabled(), ) except Exception as _lps_exc: # pragma: no cover - defensive logger.debug("lp_spatial engine failed, falling back: %s", _lps_exc) diff --git a/python/tests/test_860_mixed_gate_is_opt_in.py b/python/tests/test_860_mixed_gate_is_opt_in.py new file mode 100644 index 00000000..0a3aa0af --- /dev/null +++ b/python/tests/test_860_mixed_gate_is_opt_in.py @@ -0,0 +1,142 @@ +"""#860: the mixed/MAXIMIZE scope widening must be opt-in at *every* entry point. + +The widening is a real capability and it is sound (the ``gear4`` false certificate +that first appeared with it was the LP-presolve ``INF``-sentinel bug, fixed in #877, +not the widening). What it is not is *net-positive* on the default path, which is +CLAUDE.md §5 bar (2). Measured on ``gear4`` — mixed (4 integer, 2 continuous with +infinite upper bounds), so admitted only under the widening — at a 25 s budget: + +=========================== ================================================== +``DISCOPT_LP_SPATIAL_MIXED`` result +=========================== ================================================== +``0`` (default) ``optimal``, 1.6434284641, certified, 3 nodes, 1.1 s +``1`` ``time_limit``, 17.514, uncertified, 2678 nodes, 25 s +=========================== ================================================== + +The engine accepts a model the default path already certified in 3 nodes, then spends +the whole budget to return an incumbent ~10.7x worse with no certificate. Sound, but a +regression — and ``lp_spatial=True`` is a documented public kwarg, so shipping the +widened gate by default would trade a certificate for a worse incumbent. + +These tests pin the *gating*, not wall-clock, so they cannot rot into machine-speed +assertions. +""" + +from __future__ import annotations + +import discopt.modeling as dm +import pytest +from discopt._jax.lp_spatial_bb import _is_in_scope + + +def _mixed_minimize(): + """One integer + one continuous: in scope only under the widening.""" + m = dm.Model("mixed") + x = m.integer("x", lb=0, ub=5) + y = m.continuous("y", lb=0.0, ub=5.0) + m.minimize(x * y) + m.subject_to(x + y >= 2) + return m + + +def _pure_integer_maximize(): + """All integer but MAXIMIZE: in scope only under the widening.""" + m = dm.Model("intmax") + x = m.integer("x", lb=0, ub=5) + y = m.integer("y", lb=0, ub=5) + m.maximize(x * y) + m.subject_to(x + y <= 6) + return m + + +def _pure_integer_minimize(): + """The pre-#860 class: in scope either way (control).""" + m = dm.Model("intmin") + x = m.integer("x", lb=0, ub=5) + y = m.integer("y", lb=0, ub=5) + m.minimize(x * y) + m.subject_to(x + y >= 2) + return m + + +@pytest.mark.parametrize( + "build,widened_only", + [(_mixed_minimize, True), (_pure_integer_maximize, True), (_pure_integer_minimize, False)], + ids=["mixed_minimize", "pure_integer_maximize", "pure_integer_minimize_control"], +) +def test_scope_depends_on_the_mixed_keyword(build, widened_only): + """``mixed=True`` admits the widened classes; ``mixed=False`` is the pre-#860 gate.""" + m = build() + assert _is_in_scope(m, mixed=True) is True, "widened gate should admit this model" + assert _is_in_scope(m, mixed=False) is (not widened_only), ( + "pre-#860 gate admitted a widened-only class (or rejected the control)" + ) + + +def test_mixed_defaults_to_false_so_a_new_call_site_is_conservative(): + """The default must be the conservative gate. + + A new call site that forgets to pass ``mixed=`` must inherit the pre-#860 scope + rather than silently shipping a widening that did not graduate — the same reason + ``row_scan_is_anytime`` defaults to ``False``. + """ + assert _is_in_scope(_mixed_minimize()) is False, ( + "_is_in_scope defaults to the WIDENED gate; a new caller would ship it by default" + ) + assert _is_in_scope(_pure_integer_maximize()) is False + # control: the default must still admit the class the engine has always served + assert _is_in_scope(_pure_integer_minimize()) is True + + import inspect + + from discopt._jax.lp_spatial_bb import solve_lp_spatial_bb + + param = inspect.signature(solve_lp_spatial_bb).parameters["mixed"] + assert param.default is False, ( + f"solve_lp_spatial_bb(mixed=...) defaults to {param.default!r}; it must default to " + "False so an unflagged caller gets the pre-#860 gate" + ) + + +def test_both_production_call_sites_pass_the_flag(): + """Source-level guard: neither entry point may rely on the default. + + Cheap, and it catches the exact regression this test file exists to prevent — a + call site that stops threading the flag and re-widens the default path. + """ + import inspect + + from discopt import solver as solver_mod + from discopt.modeling import core as core_mod + + checked = 0 + for mod, needle in ( + (solver_mod, "solve_lp_spatial_bb("), + (core_mod, "_is_in_scope(self"), + ): + src = inspect.getsource(mod) + idx = src.find(needle) + assert idx != -1, f"{needle!r} not found in {mod.__name__} — test is stale" + window = src[idx : idx + 600] + assert "_lp_spatial_mixed_fallback_enabled()" in window, ( + f"{mod.__name__}: {needle!r} does not pass " + f"mixed=_lp_spatial_mixed_fallback_enabled(); the widening would ship by default" + ) + checked += 1 + assert checked == 2 + + +def test_flag_helper_defaults_off(monkeypatch): + """``DISCOPT_LP_SPATIAL_MIXED`` unset means OFF, and ``1`` turns it on.""" + import importlib + + import discopt.solver_tuning as st + from discopt.modeling.core import _lp_spatial_mixed_fallback_enabled + + monkeypatch.delenv("DISCOPT_LP_SPATIAL_MIXED", raising=False) + importlib.reload(st) + assert _lp_spatial_mixed_fallback_enabled() is False, "the widening must default OFF" + + monkeypatch.setenv("DISCOPT_LP_SPATIAL_MIXED", "1") + importlib.reload(st) + assert _lp_spatial_mixed_fallback_enabled() is True, "the opt-in must work" diff --git a/python/tests/test_lp_spatial_bb.py b/python/tests/test_lp_spatial_bb.py index 44b23729..bb5bd35c 100644 --- a/python/tests/test_lp_spatial_bb.py +++ b/python/tests/test_lp_spatial_bb.py @@ -200,7 +200,7 @@ def test_mixed_integer_in_scope_and_correct(): """#860: a mixed continuous/integer model is IN scope and must certify the true optimum. Before #860 the engine declined it outright (``_is_in_scope`` required every variable to be integer), so the whole class was unreachable.""" - r = solve_lp_spatial_bb(_mixed_model(), time_limit=20, gap_tolerance=1e-6) + r = solve_lp_spatial_bb(_mixed_model(), time_limit=20, gap_tolerance=1e-6, mixed=True) assert r is not None assert r.objective == pytest.approx(4.0, abs=1e-5) assert r.bound is not None and r.bound <= r.objective + 1e-6 @@ -238,7 +238,7 @@ def test_maximize_bound_is_a_valid_upper_bound(): b = m.integer("b", lb=0, ub=5) m.maximize(a + b) m.subject_to(a * b <= 6) - r = solve_lp_spatial_bb(m, time_limit=20, gap_tolerance=1e-6) + r = solve_lp_spatial_bb(m, time_limit=20, gap_tolerance=1e-6, mixed=True) assert r is not None assert r.objective is not None and r.objective <= 6.0 + 1e-6 assert r.bound is not None and r.bound >= r.objective - 1e-6 @@ -269,7 +269,7 @@ def test_maximize_never_overstates_the_optimum(): m.maximize(a * b - 3 * a) m.subject_to(a + b <= 6) true_max = max(i * j - 3 * i for i in range(6) for j in range(6) if i + j <= 6) - r = solve_lp_spatial_bb(m, time_limit=20, gap_tolerance=1e-6) + r = solve_lp_spatial_bb(m, time_limit=20, gap_tolerance=1e-6, mixed=True) assert r is not None if r.objective is not None: assert r.objective <= true_max + 1e-6, "incumbent beat the true maximum" @@ -296,7 +296,7 @@ def test_infinite_root_box_is_accepted_not_declined(): m.minimize(x + y) m.subject_to(x + y >= 6) m.subject_to(x * y >= 6) - r = solve_lp_spatial_bb(m, time_limit=20, gap_tolerance=1e-6) + r = solve_lp_spatial_bb(m, time_limit=20, gap_tolerance=1e-6, mixed=True) assert r is not None, "an infinite root endpoint must no longer decline the solve" assert r.objective == pytest.approx(6.0, abs=1e-5) assert r.bound is not None and r.bound <= r.objective + 1e-6 @@ -317,7 +317,7 @@ def test_cold_path_product_map_travels_with_its_solution(): path = os.path.join(_DATA_NL, "st_e38.nl") if not os.path.exists(path): pytest.skip("st_e38 not vendored") - r = solve_lp_spatial_bb(dm.from_nl(path), time_limit=20) + r = solve_lp_spatial_bb(dm.from_nl(path), time_limit=20, mixed=True) assert r is not None, "engine must not decline (an IndexError here is the bug)" if r.objective is not None: assert r.bound is not None and r.bound <= r.objective + 1e-6 From b3559661e99f38c891695d1c0daec81d02c07939 Mon Sep 17 00:00:00 2001 From: John Kitchin Date: Sun, 26 Jul 2026 19:08:26 -0400 Subject: [PATCH 12/12] test(#860): drop a reload that poisoned SolverTuning identity; update the stale gate assertion Two CI failures from the previous commit, both real. 1. My new test called importlib.reload(discopt.solver_tuning) to re-read the env var. That was unnecessary -- _lp_spatial_mixed_fallback_enabled reads os.environ directly with no caching -- and actively harmful: reloading the module replaces the SolverTuning CLASS OBJECT, so unrelated isinstance(..., SolverTuning) assertions later in the same xdist worker fail. It broke test_result_io::test_options_from_payload_rebuilds_tuning and test_solve_daemon::test_solve_request_rebuilds_tuning, neither of which this branch touches. Removed the reload (monkeypatch alone is sufficient and correct), added a "0" case, and recorded why no reload belongs here so it is not reintroduced. 2. test_844_lp_spatial_fallback::test_860_mixed_flag_defaults_to_off_and_gates_the_reserve asserted `_is_in_scope(m) is True # engine: widened unconditionally`. That is precisely the behavior the previous commit changed, so it had to be updated -- updated with the flag's new scope, not loosened. The substantive assertion (mixed=False declines a mixed model) is unchanged; it now additionally asserts that the DEFAULT is the conservative gate, which is the new invariant. The docstring records what it used to say and why it changed. Verified with CI's exact PR-fast selection rather than the smoke subset that missed these: pytest python/tests/ -m "not slow and not correctness and not integration and not amp_benchmark and not requires_cyipopt and not memory_heavy" --ignore=test_correctness.py --ignore=test_amp.py -> 7255 passed, 29 skipped, 5 xfailed, 2 xpassed, 0 failed (622 s) Also ran the three previously-failing files plus the new one in a SINGLE process (-p no:randomly) so the cross-test contamination would reappear if it were still there: 42 passed. Contributes to #860 --- python/tests/test_844_lp_spatial_fallback.py | 21 ++++++++++++++----- python/tests/test_860_mixed_gate_is_opt_in.py | 19 +++++++++++------ 2 files changed, 29 insertions(+), 11 deletions(-) diff --git a/python/tests/test_844_lp_spatial_fallback.py b/python/tests/test_844_lp_spatial_fallback.py index cdc1d2ba..dc4840b6 100644 --- a/python/tests/test_844_lp_spatial_fallback.py +++ b/python/tests/test_844_lp_spatial_fallback.py @@ -132,9 +132,19 @@ def test_mixed_model_result_is_unchanged_by_the_860_flag(fb, monkeypatch, flag, def test_860_mixed_flag_defaults_to_off_and_gates_the_reserve(monkeypatch): - """The #860 widening reaches the DEFAULT path only through this flag, which is - off until its graduation panel. The engine's own gate is already widened, so the - two must be able to disagree — that separation is the point of the flag.""" + """The #860 widening reaches the default path only through this flag, which is off + until its graduation panel. + + Updated with the flag's scope, not loosened. This test previously asserted + ``_is_in_scope(m) is True`` with the comment "engine: widened unconditionally", + because the flag then governed only the fallback's 35% budget reserve while the + engine gate was always widened. The #860 review moved the engine gate behind the + same flag — the widening is sound but not net-positive on the default path (see + ``_lp_spatial_mixed_fallback_enabled``, and ``test_860_mixed_gate_is_opt_in``) — + so the *default* is now the conservative gate at both entry points. The + substantive assertion, that ``mixed=False`` declines a mixed model, is unchanged + and is now also what the default does. + """ from discopt._jax.lp_spatial_bb import _is_in_scope from discopt.modeling.core import _lp_spatial_mixed_fallback_enabled @@ -144,8 +154,9 @@ def test_860_mixed_flag_defaults_to_off_and_gates_the_reserve(monkeypatch): assert _lp_spatial_mixed_fallback_enabled() is True m = _mixed() - assert _is_in_scope(m) is True # engine: widened unconditionally - assert _is_in_scope(m, mixed=False) is False # reserve gate while the flag is off + assert _is_in_scope(m, mixed=True) is True # capability, on request + assert _is_in_scope(m, mixed=False) is False # pre-#860 gate + assert _is_in_scope(m) is False # default is now the conservative gate def test_fallback_never_breaks_a_solve(fb): diff --git a/python/tests/test_860_mixed_gate_is_opt_in.py b/python/tests/test_860_mixed_gate_is_opt_in.py index 0a3aa0af..9c06dd72 100644 --- a/python/tests/test_860_mixed_gate_is_opt_in.py +++ b/python/tests/test_860_mixed_gate_is_opt_in.py @@ -127,16 +127,23 @@ def test_both_production_call_sites_pass_the_flag(): def test_flag_helper_defaults_off(monkeypatch): - """``DISCOPT_LP_SPATIAL_MIXED`` unset means OFF, and ``1`` turns it on.""" - import importlib - - import discopt.solver_tuning as st + """``DISCOPT_LP_SPATIAL_MIXED`` unset means OFF, and ``1`` turns it on. + + Deliberately does NOT ``importlib.reload`` anything. The helper reads + ``os.environ`` directly with no caching, so a reload buys nothing — and reloading + ``solver_tuning`` mid-session replaces the ``SolverTuning`` class object, after + which unrelated ``isinstance(..., SolverTuning)`` assertions elsewhere in the same + worker fail. A first draft of this test did exactly that and broke + ``test_result_io`` and ``test_solve_daemon`` in CI. + """ from discopt.modeling.core import _lp_spatial_mixed_fallback_enabled monkeypatch.delenv("DISCOPT_LP_SPATIAL_MIXED", raising=False) - importlib.reload(st) assert _lp_spatial_mixed_fallback_enabled() is False, "the widening must default OFF" monkeypatch.setenv("DISCOPT_LP_SPATIAL_MIXED", "1") - importlib.reload(st) assert _lp_spatial_mixed_fallback_enabled() is True, "the opt-in must work" + + # "0" and other falsy spellings stay off — the helper's own contract. + monkeypatch.setenv("DISCOPT_LP_SPATIAL_MIXED", "0") + assert _lp_spatial_mixed_fallback_enabled() is False