Add Per-Parameter Convenience Kernel Override by Scienfitz · Pull Request #904 · emdgroup/baybe

Scienfitz · 2026-08-24T17:52:14Z

This adds the per-parameter convenience override for the kernel expressed in parameter objects
Builds on PR #868

Adds an optional, kw-only kernel_override to regular parameters, TaskParameter intentionally does not expose it
Streamlines override inside the GP. New flow in _resolve_kernel:
- Collect (name, kernel) factors from param overrides and, independently, the tl override
- If there are none, resolve the surrogate kernel as before (fast path)
- Otherwise, build the residual surrogate kernel restricted to the non-overridden parameters, then multiply it by each override factor
- Override logic largely lives in a dedicated private subpackage (_override/ with parameter.py, tl.py, core.py)
Limitations:
- Applies only to GP surrogate; others ignore it and emit an UnusedObjectWarning
- Composition resulting from convenience override is fixed multiplicative
- BayBE kernels as override must have
  - param_names=None or
  - carry the exact correct parameter name
- Raw GPyTorch overrides must
  - not set active_dims
  - ard_num_dims must be None or match the parameter's comp rep width
Kernel factories are not supported
The surrogate kernel/factory must allow the overridden dimensions to be excluded; incompatible custom kernels/factories raise IncompatibleOverrideError

Notes:

Multi-column parameters (categorical/substance/custom) bind the override to their full computational block
Kernel overrides can coexist with transfer-learning override because the TaskParameter does not allow kernel overrides like other kernels
This prepared the architecture for future overrides:
- multi-fidelty
- overrides created by Symmetry-indicated symmetrization of base kernels
Since this PR starts a more streamlined variant of overrides in the resolution of components, here a rough sketch of what would be possible with that also for future work:

…a TaskParameter - Change the default transfer learning kernel from `IndexKernel` to `PositiveIndexKernel`, enforcing positive task correlations; `PositiveIndexKernel` disables BoTorch's target-task normalization (`unit_scale_for_target=False`) - Add `TransferLearningMode` enum (`INDEX_KERNEL`, `POSITIVE_INDEX_KERNEL`) in `baybe/parameters/enum.py` - Add optional `TaskParameter.override_transfer_learning_mode` field (default `None`) to override the task kernel used for transfer learning - Add `GaussianProcessSurrogate._resolve_kernel`: when an override is set, strip the task parameter, run the kernel factory on the reduced search space, and attach the requested task kernel - Add `SearchSpace._without_task_parameter()` helper to build a task-free search space - Raise the new `IncompatibleKernelError` when an override clashes with a task-aware kernel factory - Extend the `task_parameters` and kernel hypothesis strategies to cover the new field and `PositiveIndexKernel`; add tests for factory dispatch and `_resolve_kernel` - Use the default transfer-learning kernel in the transfer-learning benchmarks - Filter the "Negative variance values detected" `NumericalWarning` in `pytest.ini`, add a `gpytorch` intersphinx mapping for the docs, and update the CHANGELOG Co-authored-by: Martin Fitzner <martin.fitzner@merckgroup.com>

- build the base kernel from what the user gave us and attach the requested task kernel manually - Strip the task out of user-provided (scaled) kernels - Raise when the override can't be implementded (raw gpytorch kernels) - Add a dedicated error for blocked reduced-search-space access so we catch the right thing instead of an general AttributeError - Update the tests for the new behavior

- Undo unintended reformat - Remove test for defautls - Export TransferLearningMode at top level namespace - Improve docstring - Rename IncompatibleKErnelError to IncompatibleOverrideError - Make IndexKernel case explicit

- The override used to build the base kernel on a reduced search space, where the default factory's numerical kernel cannot resolve its active dimensions, so it raised on non-substance spaces. - Route the default factory through the ICM machinery on the full search space instead, pairing the task-excluded base kernel with the requested index kernel. - Leave the reduced-search-space path for other factories and the strip path for fixed kernels untouched, so all other behavior stays the same. - Move the default-factory override cases from the raising test to the success test.

…sk_from_kernel

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Adds a per-parameter kernel_override convenience interface to compose Gaussian process kernels at the parameter level, while keeping transfer-learning overrides compatible and warning when non-GP surrogates ignore these overrides.

Changes:

Introduces kernel_override on regular parameters (explicitly disallowed on TaskParameter) with validation and equivalence handling.
Refactors GP kernel resolution to apply (parameter + transfer-learning) overrides via a dedicated _override/ subpackage and residual-kernel composition.
Adds comprehensive tests, docs, and a changelog entry; warns when non-GP surrogates ignore configured overrides.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
tests/validation/test_parameter_validation.py	Adds constructor-time validation coverage for invalid overrides and `TaskParameter` rejection.
tests/test_surrogate.py	Verifies non-GP surrogates emit `UnusedObjectWarning` when overrides are present.
tests/test_parameter_kernel_overrides.py	New functional test suite covering override binding/composition and incompatibility cases.
tests/test_iterations.py	Adds an end-to-end iteration test exercising parameter kernel overrides.
tests/hypothesis_strategies/parameters.py	Extends parameter strategies to optionally generate valid kernel overrides.
docs/components/surrogates.md	Links surrogate kernel documentation to the new parameter override section.
docs/components/parameters.md	Documents parameter-specific kernel overrides, semantics, and limitations.
CHANGELOG.md	Records the new parameter-specific kernel override feature.
baybe/surrogates/gaussian_process/core.py	Implements the new override-aware kernel resolution flow and residual-kernel logic.
baybe/surrogates/gaussian_process/_override/init.py	Exposes the private override-resolution helpers for the GP surrogate.
baybe/surrogates/gaussian_process/_override/core.py	Adds shared helpers for reducing kernel specs and raising incompatibility errors.
baybe/surrogates/gaussian_process/_override/parameter.py	Extracts and binds per-parameter overrides (BayBE and raw GPyTorch kernels).
baybe/surrogates/gaussian_process/_override/tl.py	Extracts and builds the transfer-learning override kernel factor.
baybe/surrogates/base.py	Adds `supports_kernel_overrides` and emits warnings when unsupported surrogates ignore overrides.
baybe/parameters/categorical.py	Prevents `TaskParameter` from exposing/accepting `kernel_override`.
baybe/parameters/base.py	Adds `kernel_override` field, validation/scoping converter, and updates equivalence logic.
baybe/kernels/composite.py	Implements `_with_parameter` scoping for composite kernels to support owner rebinding.
baybe/kernels/base.py	Adds a `_with_parameter` API on kernels and implements it for basic kernels.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

AVHopp

First round of reviews - already looks quite nice :)

AVHopp · 2026-08-31T06:52:21Z

        )

+    def _with_parameter(self, name: str, /) -> Kernel:
+        """Return a copy of the kernel scoped to a single parameter.


Suggested change

"""Return a copy of the kernel scoped to a single parameter.

"""Return a copy of the kernel that acts only on the given parameter.

To mimic the style of the _without_parameter

AVHopp · 2026-08-31T07:04:37Z

+        return value._with_parameter(instance.name)
+
+    # GPyTorch kernels: no explicit active dimensions allowed anywhere in the tree.
+    if sys.modules.get("gpytorch") is not None:


Is there actually a situation where somebody would ever use this without having gpytorch installed? Or is this necessary for other reasons?

AVHopp · 2026-08-31T07:07:44Z

-        return attrs.evolve(self, name=other.name) == other
+        # The override is owner-scoped, so rebind it to the other parameter's name.
+        kernel_override = self.kernel_override
+        if isinstance(kernel_override, Kernel):


What happens in the case of a GPyTorch Kernel?

Also, Claude claims that two "seperately instantiated, but structurally identical GPyTorch overrides always compare unequal", so please double-check

AVHopp · 2026-08-31T07:14:41Z

+    Returns:
+        The reduced kernel, or ``None`` if nothing remains.
+    """
+    if not isinstance(component, Kernel):


If this is the first thing that you check here, why do you then allow general object in the type hint?

AVHopp · 2026-08-31T07:26:36Z

+    """
+    if not isinstance(component, Kernel):
+        raise_incompatible_override(excluded_names, factory)
+    spec: Kernel | None = component


Can this happen, given the check and the type hint?

AVHopp · 2026-08-31T07:30:06Z

+        yield kernel
+    elif isinstance(kernel, ScaleKernel):
+        yield from _iter_basic_kernels(kernel.base_kernel)
+    elif isinstance(kernel, (AdditiveKernel, ProductKernel)):


Shouldn't we have an "else" for potential additional subclasses/for raising errors right now? Otherwise, high danger of silently ignoring stuff we might add in the future.

AVHopp · 2026-08-31T07:31:13Z

 #   https://github.com/python-attrs/attrs/issues/164


+def _iter_basic_kernels(kernel: Kernel):


No return type annotation?

AVHopp · 2026-08-31T07:33:25Z

+        The GPyTorch kernel bound to the parameter's dimensions.
+    """
+    override = parameter.kernel_override
+    assert override is not None


Why can we make this assert here? Isn't this a very generic helper?

AVHopp · 2026-08-31T07:43:54Z

 """A strategy that generates parameter categories."""


+def _remove_kernel_parameter_names(kernel):


Doesn't this exactly replicate the helper you defined in the main part of the PR? Why don't we re-use it?

Scienfitz · 2026-08-31T09:26:31Z

With this PR its also easy to try the old idea of using an index kernel for categorical parameters, see here the result for our full lookup example:

(no prior used for index kernel)

kalama-ai · 2026-09-04T07:52:19Z

    """The individual kernels to be multiplied."""

+    @override
+    def _with_parameter(self, name: str, /) -> Kernel:


I was wondering, why we do not support _without_parameter for the ProductKernel as well? Shouldn't it be possible to combine a ProductKernel with an override by just removing the parameter from every factor in the product?

kalama-ai · 2026-09-04T08:13:58Z

        has_tl_override = context.tl_override is not None
        if (
            context.is_multitask
            and self._custom_kernel


Could you clarify whether this error can ever be triggered when combining the default kernel factory with a parameter override? My intuition was that once you attach an override, the effective kernel is no longer the plain default, but a custom one. But as far as I can tel the self._custom_kernelwill never be affected by an override. Is this the intended behaviour?

kalama-ai · 2026-09-04T08:40:36Z

+    kernel, _ = _resolve(
+        [
+            NumericalContinuousParameter("x1", (0, 1)),
+            NumericalContinuousParameter("x2", (0, 1), kernel_override=RBFKernel()),


Can we add a similar test with on override on the 2nd of three parameters for the non-default branch in _resolve_residual_kernel like a MaternKernel() to check that the indices are resolved ocrrectly?

kalama-ai · 2026-09-04T08:48:38Z

+
+
+def test_default_factory_selector_is_preserved():
+    """Partitioning preserves an explicit default-factory parameter selector."""


Can we add other success cases besides the default BayBEKernelFactory like a (scaled) reducible kernel + a parameter override:

MaternKernel()

MaternKernel(("x1","x2")) (named branch)

ScaleKernel(MaternKernel()) (unnamed branch)

kalama-ai · 2026-09-04T09:14:06Z

        default=None,
        validator=optional(is_callable()),
    )
    """The factory used to create the kernel for the Gaussian process.


Could you document the override behaviour here?

what exactly? the control flow (similar tot he picture) in words?

kalama-ai and others added 28 commits August 6, 2026 12:09

Address review comments

c75f410

- Undo unintended reformat - Remove test for defautls - Export TransferLearningMode at top level namespace - Improve docstring - Rename IncompatibleKErnelError to IncompatibleOverrideError - Make IndexKernel case explicit

Remove DISPATCHING_PLAN from version control

a01f2b8

Revert benchmarks module to upstream/main state

3074916

Minor Clean up

c3f7efd

Use converter instead of validator for override_transfer_learning_mode

2a86b07

Fix error messages

937944d

Remove legacy test

28352d7

Added task parameter only guars and simplified signature of _strip_ta…

f74e4bc

…sk_from_kernel

Catch unknown TL override

a67f4aa

Fix sphinx reference in docstring

30fb565

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Improve docstrings

c7862ad

Add tests for facrtories returning gpytorch or task-free BayBE kernels

42fc1c7

Strip task from kernels w/out names or task-aware BayBE factories

bfa8983

Move kernel reduction helper to kernel classes

ae3c5c5

Run pre-commit hooks

1d92b0c

Support scoping kernels to parameters

6c7e3f3

Add kernel overrides to parameters

bd66fdf

Generate valid parameter kernel overrides

b372ba8

Warn when surrogates ignore kernel overrides

47aa5fb

Partition GP kernels around parameter overrides

ad1f650

Test parameter-specific kernel overrides

1417922

Test parameter kernel overrides end to end

1f1e764

Document parameter-specific kernel overrides

75cda3d

Group kernel override helpers into a subpackage

f9ac6ba

Update CHANGELOG

f94fb72

Scienfitz self-assigned this Aug 24, 2026

Copilot AI lite review requested due to automatic review settings August 24, 2026 17:52

Scienfitz requested review from AVHopp and AdrianSosic as code owners August 24, 2026 17:52

Scienfitz added the new feature New functionality label Aug 24, 2026

Scienfitz requested a review from kalama-ai August 24, 2026 17:52

Copilot started reviewing on behalf of Scienfitz August 24, 2026 17:52 View session

Copilot AI reviewed Aug 24, 2026

View reviewed changes

Scienfitz mentioned this pull request Aug 28, 2026

Add TL override path to new GP factory #868

Merged

AVHopp reviewed Aug 31, 2026

View reviewed changes

Scienfitz added this to the 0.16.0 milestone Sep 1, 2026

Base automatically changed from feat/tl-override-in-gp-factory to main September 3, 2026 09:46

kalama-ai reviewed Sep 4, 2026

View reviewed changes

	"""Return a copy of the kernel scoped to a single parameter.
	"""Return a copy of the kernel that acts only on the given parameter.

		# https://github.com/python-attrs/attrs/issues/164


		def _iter_basic_kernels(kernel: Kernel):

		"""A strategy that generates parameter categories."""


		def _remove_kernel_parameter_names(kernel):



		def test_default_factory_selector_is_preserved():
		"""Partitioning preserves an explicit default-factory parameter selector."""

Uh oh!

Conversation

Scienfitz commented Aug 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

AVHopp left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Scienfitz commented Aug 31, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Scienfitz commented Aug 24, 2026 •

edited

Loading