From 7b7947fcfef6ae52a60585abb70df6a955008c1d Mon Sep 17 00:00:00 2001 From: vdsim-dev Date: Tue, 22 Sep 2026 15:23:10 +0900 Subject: [PATCH 1/2] Q21: reject a shadowed vdsim core at import time Another PyPI distribution ships a top-level vdsim/ package. Installed beside our wheel it wins the import over vdsim.cpython-*.so with no warning from pip, and the failure only surfaces later as an AttributeError on an unrelated line. python/vdsim_guard.py holds the decision once: a module bound to the name vdsim must be a compiled extension (importlib EXTENSION_SUFFIXES) and expose the core symbols. load_core() also retries an explicit build directory, so a repo checkout keeps working in an environment that has the shadowing package. vdsim_lab, vdsim_plant and vdsim_rl now go through load_core(); the other shipped modules never import the core, so they are untouched. The error names the file that was actually loaded plus a one-line cause and remedy; the check itself is structural and never tests for a distribution name. docs/VALIDATION.md drops to PENDING: python/CMakeLists.txt is part of the registration surface, so the canonical count is re-measured on this tree. --- README.ko.md | 4 + README.md | 4 + docs/VALIDATION.md | 8 +- python/CMakeLists.txt | 1 + python/vdsim_guard.py | 119 +++++++++++++++++++++++++++ python/vdsim_lab.py | 8 +- python/vdsim_plant.py | 8 +- python/vdsim_rl/__init__.py | 4 +- tests/scripts/test_experiment_api.py | 71 ++++++++++++++++ 9 files changed, 213 insertions(+), 14 deletions(-) create mode 100755 python/vdsim_guard.py diff --git a/README.ko.md b/README.ko.md index 33b60912..234d5a14 100644 --- a/README.ko.md +++ b/README.ko.md @@ -53,6 +53,10 @@ pip install "./vdsim-*.whl[plot]" vdsim-quickstart # cwd 에 run.csv + run.png 생성 ``` +깨끗한 환경에 설치할 것 — 최상위 `vdsim` 패키지를 제공하는 다른 배포판이 함께 설치되면 +그쪽이 import 를 가져간다. VDSim 은 이를 감지해 실제 로드된 파일 경로와 함께 +`CoreShadowedError` 를 던진다. + 측정 (clean conda, Python 3.11, Linux x86_64, 2026-06-25): **첫 결과까지 수 초** — pip install `[plot]` + `vdsim-quickstart` → `run.csv` + `run.png` **~6 s wall-clock** (cold; lab 네트워크, matplotlib wheel 포함). 재실행 ~1.5 s. diff --git a/README.md b/README.md index 56a9c009..f0d46349 100644 --- a/README.md +++ b/README.md @@ -55,6 +55,10 @@ pip install "./vdsim-*.whl[plot]" vdsim-quickstart # writes run.csv + run.png in the current directory ``` +Install into a clean environment: another distribution also provides a top-level +`vdsim` package, and if both are present that one wins the import. VDSim detects +this and raises `CoreShadowedError` naming the file that was loaded. + Measured on a clean conda env (Python 3.11, Linux x86_64, 2026-06-25): **first result in seconds** — pip install `[plot]` + `vdsim-quickstart` → `run.csv` + `run.png` in **~6 s wall-clock** (cold; lab network, matplotlib wheel included). Repeat quickstart ~1.5 s. diff --git a/docs/VALIDATION.md b/docs/VALIDATION.md index f7b4fdd7..db895c9d 100644 --- a/docs/VALIDATION.md +++ b/docs/VALIDATION.md @@ -20,7 +20,7 @@ the last section. | ISO 7401 step-steer / 4138 understeer / 3888-2 DLC — run + measured | — | | L1↔L2↔L3 cross-model consistency where physics overlaps | Dependent axles (twist-beam / solid beam) — configs are stubs | | FMI round-trip Δ=0 (machine precision); ISO 8608 PSD RMS per class | L3 unsprung lateral-transfer term (small) | -| Full suite: **505/505 ctest green (as of 2026-09-22)** (`validation` preset — see the currency block below) | — | +| Full suite: **PENDING** — re-measured by CI on this tree (`validation` preset — see the currency block below) | — | | Drivetrain engine inertia (open-diff carrier coupling) | — | | **ISO step-steer signature gated in CI** (`ctest -R IsoBaseline`, sedan L2 LuGre) | DLC moose gate is a preset property, not a defect (see note) | @@ -48,11 +48,13 @@ version and the new numbers have to land in the same commit. ```text -tests: 505/505 +tests: PENDING config: cmake --preset validation && cmake --build --preset validation && ctest --preset validation presets: CMakePresets.json@fb1f0c198a0201fcce2543a9f5b2242a255fb97a toolchain: cmake 3.31.10 -commit: 3a72f3c +commit: PENDING +pending: registration surface changed (python/CMakeLists.txt ships vdsim_guard.py); +pending: the canonical count is re-measured by CI on this tree before promotion. date: 2026-09-22 excluded: gui_v3_e2e (GUI v3 test group formally deferred -- VDSIM_BUILD_GUI_V3_TESTS=OFF) excluded: gui_v3_api_smoke (GUI v3 test group formally deferred -- VDSIM_BUILD_GUI_V3_TESTS=OFF) diff --git a/python/CMakeLists.txt b/python/CMakeLists.txt index 09be8a13..b32f5e2f 100644 --- a/python/CMakeLists.txt +++ b/python/CMakeLists.txt @@ -21,6 +21,7 @@ if(pybind11_FOUND) if(DEFINED SKBUILD) install(TARGETS vdsim_py LIBRARY DESTINATION ".") install(FILES + ${CMAKE_SOURCE_DIR}/python/vdsim_guard.py ${CMAKE_SOURCE_DIR}/python/vdsim_lab.py ${CMAKE_SOURCE_DIR}/python/vdsim_plant.py ${CMAKE_SOURCE_DIR}/python/vdsim_trace.py diff --git a/python/vdsim_guard.py b/python/vdsim_guard.py new file mode 100755 index 00000000..1efb491d --- /dev/null +++ b/python/vdsim_guard.py @@ -0,0 +1,119 @@ +#!/usr/bin/env python3 +"""Single check that the name ``vdsim`` resolves to VDSim's compiled core. + +Another distribution on PyPI ships a top-level ``vdsim/`` *package*. Installed +into the same environment as our wheel it wins the import over our +``vdsim.cpython-*.so`` -- pip neither warns nor overwrites, so the failure +surfaces much later as an ``AttributeError`` on an unrelated line. The modules +that need the core therefore call :func:`load_core` instead of ``import +vdsim``; the decision itself lives here once so the modules cannot drift. + +The check is structural on purpose: it asks whether the loaded module is a +compiled extension exposing the core API, never which distribution shipped it. +""" +from __future__ import annotations + +import importlib +import importlib.machinery +from pathlib import Path + +__all__ = ["CoreShadowedError", "check_core", "load_core", "diagnose"] + + +class CoreShadowedError(ImportError): + """``vdsim`` resolved to something that is not VDSim's compiled core.""" + + +#: Symbols every build of the core exposes (pybind11 ``cosim/bindings.cpp``). +#: Checked as a set so a foreign module cannot pass by coincidence on one name. +EXPECTED_SYMBOLS = ("SimSession", "VehicleParams", "TireParams", "make_sim_session") + + +def diagnose(mod): + """Return ``None`` if *mod* is the compiled core, else a one-line reason. + + :param mod: the module object bound to the name ``vdsim``. + :returns: ``None`` when the module passes both checks, otherwise a short + phrase naming which property failed. + """ + origin = getattr(mod, "__file__", None) + if origin is None: + return "it has no __file__ (a namespace package, not an extension module)" + if not any(str(origin).endswith(s) + for s in importlib.machinery.EXTENSION_SUFFIXES): + return "it is Python source, not a compiled extension module" + missing = [s for s in EXPECTED_SYMBOLS if not hasattr(mod, s)] + if missing: + return "it is missing core symbols: %s" % ", ".join( + "vdsim." + s for s in missing) + return None + + +def _message(mod, reason): + origin = getattr(mod, "__file__", None) or "" + return ( + "'import vdsim' loaded %s -- %s.\n" + "Cause: another installed distribution provides a top-level 'vdsim' " + "package, which shadows VDSim's vdsim.cpython-*.so on sys.path " + "(pip does not warn about this).\n" + "Fix: uninstall the shadowing distribution " + "(e.g. `pip uninstall vehicle-dynamics-sim`) or install VDSim into " + "its own virtual environment." % (origin, reason) + ) + + +def check_core(mod): + """Return *mod* if it is the compiled core, else raise. + + :param mod: the module object bound to the name ``vdsim``. + :raises CoreShadowedError: when :func:`diagnose` reports a mismatch. + """ + reason = diagnose(mod) + if reason is not None: + raise CoreShadowedError(_message(mod, reason)) + return mod + + +def load_core(fallback_paths=()): + """Import ``vdsim`` and return it only if it is the compiled core. + + *fallback_paths* are development build directories tried, in order, when + the ambient import is missing **or** shadowed; this keeps a repo checkout + working even in an environment that has the shadowing package installed. + + :param fallback_paths: directories prepended to ``sys.path`` on retry. + :returns: the compiled ``vdsim`` extension module. + :raises CoreShadowedError: the name resolves to a foreign module and no + fallback path yields the core. + :raises ImportError: ``vdsim`` is not importable at all. + """ + import sys + + first_error = None + try: + mod = importlib.import_module("vdsim") + except ImportError as exc: + mod, first_error = None, exc + if mod is not None and diagnose(mod) is None: + return mod + + for raw in fallback_paths: + path = str(raw) + if not Path(path).is_dir(): + continue + while path in sys.path: + sys.path.remove(path) + sys.path.insert(0, path) + sys.modules.pop("vdsim", None) + importlib.invalidate_caches() + try: + candidate = importlib.import_module("vdsim") + except ImportError: + continue + if diagnose(candidate) is None: + return candidate + mod = candidate + + if mod is None: + raise first_error + return check_core(mod) diff --git a/python/vdsim_lab.py b/python/vdsim_lab.py index 42d09d2c..7825d16b 100644 --- a/python/vdsim_lab.py +++ b/python/vdsim_lab.py @@ -20,11 +20,9 @@ from pathlib import Path REPO = Path(__file__).resolve().parent.parent -try: - import vdsim -except ImportError: - sys.path.insert(0, str(REPO / "build" / "python")) - import vdsim +from vdsim_guard import load_core # noqa: E402 (shadowed-core check, one place) + +vdsim = load_core([REPO / "build" / "python"]) def _conf_root(): diff --git a/python/vdsim_plant.py b/python/vdsim_plant.py index 82afb3d6..70e12424 100644 --- a/python/vdsim_plant.py +++ b/python/vdsim_plant.py @@ -29,11 +29,9 @@ from pathlib import Path REPO = Path(__file__).resolve().parent.parent -try: - import vdsim -except ImportError: - sys.path.insert(0, str(REPO / "build" / "python")) - import vdsim +from vdsim_guard import load_core # noqa: E402 (shadowed-core check, one place) + +vdsim = load_core([REPO / "build" / "python"]) def _conf_root(): for c in (REPO / "configs", Path.cwd() / "configs", diff --git a/python/vdsim_rl/__init__.py b/python/vdsim_rl/__init__.py index bd8c6862..e67fef08 100644 --- a/python/vdsim_rl/__init__.py +++ b/python/vdsim_rl/__init__.py @@ -24,7 +24,9 @@ import numpy as np -import vdsim +from vdsim_guard import load_core + +vdsim = load_core() __all__ = [ "EnvConfig", "VDSimVecEnv", "VDSimEnv", "make_sb3_vec_env", diff --git a/tests/scripts/test_experiment_api.py b/tests/scripts/test_experiment_api.py index 274a5cdb..9496e411 100644 --- a/tests/scripts/test_experiment_api.py +++ b/tests/scripts/test_experiment_api.py @@ -147,6 +147,74 @@ def test_ref_point_position(): assert sim_u._ref == [1.0, 0.5] +# --- Q21: the name `vdsim` must resolve to the compiled core ----------------- +# An unrelated distribution ships a top-level `vdsim/` package; installed beside +# our wheel it wins the import silently. vdsim_guard makes that an error. + +def _shadow_probe(body, shadow_dir, extra_path=()): + """Run *body* in a child whose sys.path starts with a fake `vdsim` package.""" + import os + import subprocess + env = dict(os.environ) + env["PYTHONPATH"] = os.pathsep.join( + [str(shadow_dir), str(REPO / "python")] + [str(p) for p in extra_path]) + return subprocess.run([sys.executable, "-c", body], env=env, + capture_output=True, text=True) + + +def _make_shadow(td): + pkg = Path(td) / "vdsim" + pkg.mkdir() + (pkg / "__init__.py").write_text("VERSION = 'foreign package'\n") + return Path(td) + + +def test_shadowed_core_is_rejected(): + body = ("import sys, vdsim_guard\n" + "try:\n" + " vdsim_guard.load_core([])\n" + "except vdsim_guard.CoreShadowedError as exc:\n" + " print(exc); sys.exit(0)\n" + "sys.exit('no error raised')\n") + with tempfile.TemporaryDirectory() as td: + shadow = _make_shadow(td) + r = _shadow_probe(body, shadow) + assert r.returncode == 0, f"shadowed vdsim was accepted: {r.stdout}{r.stderr}" + msg = r.stdout + assert str(shadow / "vdsim") in msg, f"message must name the real load path: {msg}" + assert "Cause:" in msg and "Fix:" in msg, f"message needs cause + remedy: {msg}" + + +def test_shadowed_core_recovers_from_build_dir(): + """A dev checkout still works: the explicit build path wins over the shadow.""" + import os + build = Path(os.environ.get("VDSIM_BUILD_DIR", REPO / "build")) / "python" + if not build.is_dir(): + build = REPO / "build" / "python" + assert build.is_dir(), f"no compiled core to fall back to ({build})" + body = ("import sys, vdsim_guard\n" + "m = vdsim_guard.load_core([sys.argv[1]])\n" + "assert hasattr(m, 'SimSession'), m\n" + "print(m.__file__)\n") + with tempfile.TemporaryDirectory() as td: + shadow = _make_shadow(td) + import os as _os + import subprocess + env = dict(_os.environ) + env["PYTHONPATH"] = _os.pathsep.join([str(shadow), str(REPO / "python")]) + r = subprocess.run([sys.executable, "-c", body, str(build)], + env=env, capture_output=True, text=True) + assert r.returncode == 0, f"fallback failed: {r.stdout}{r.stderr}" + assert str(shadow) not in r.stdout, r.stdout + + +def test_real_core_passes_the_check(): + import vdsim + import vdsim_guard + assert vdsim_guard.diagnose(vdsim) is None, vdsim.__file__ + assert vdsim_guard.check_core(vdsim) is vdsim + + if __name__ == "__main__": test_throttle_then_brake() test_step_steer_yaws() @@ -159,4 +227,7 @@ def test_ref_point_position(): test_register_metric() test_plot_comparison() test_ref_point_position() + test_shadowed_core_is_rejected() + test_shadowed_core_recovers_from_build_dir() + test_real_core_passes_the_check() print("OK test_experiment_api") From d4e6b268f83a247906a677efbbd959e134d63efc Mon Sep 17 00:00:00 2001 From: Jiwon Seok Date: Tue, 22 Sep 2026 16:39:21 +0900 Subject: [PATCH 2/2] docs(validation): promote currency block to the CI-measured tree Measured tree 7b7947f; three identical CI attempts on the same head. Numbers are in the PR body and the currency block. Co-Authored-By: Claude Opus 5 --- docs/VALIDATION.md | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/docs/VALIDATION.md b/docs/VALIDATION.md index db895c9d..331f7df9 100644 --- a/docs/VALIDATION.md +++ b/docs/VALIDATION.md @@ -20,7 +20,7 @@ the last section. | ISO 7401 step-steer / 4138 understeer / 3888-2 DLC — run + measured | — | | L1↔L2↔L3 cross-model consistency where physics overlaps | Dependent axles (twist-beam / solid beam) — configs are stubs | | FMI round-trip Δ=0 (machine precision); ISO 8608 PSD RMS per class | L3 unsprung lateral-transfer term (small) | -| Full suite: **PENDING** — re-measured by CI on this tree (`validation` preset — see the currency block below) | — | +| Full suite: **505/505 ctest green (as of 2026-09-22)** (`validation` preset — see the currency block below) | — | | Drivetrain engine inertia (open-diff carrier coupling) | — | | **ISO step-steer signature gated in CI** (`ctest -R IsoBaseline`, sedan L2 LuGre) | DLC moose gate is a preset property, not a defect (see note) | @@ -48,13 +48,11 @@ version and the new numbers have to land in the same commit. ```text -tests: PENDING +tests: 505/505 config: cmake --preset validation && cmake --build --preset validation && ctest --preset validation presets: CMakePresets.json@fb1f0c198a0201fcce2543a9f5b2242a255fb97a toolchain: cmake 3.31.10 -commit: PENDING -pending: registration surface changed (python/CMakeLists.txt ships vdsim_guard.py); -pending: the canonical count is re-measured by CI on this tree before promotion. +commit: 7b7947f date: 2026-09-22 excluded: gui_v3_e2e (GUI v3 test group formally deferred -- VDSIM_BUILD_GUI_V3_TESTS=OFF) excluded: gui_v3_api_smoke (GUI v3 test group formally deferred -- VDSIM_BUILD_GUI_V3_TESTS=OFF)