From 7b200c0c78b151cd2bfc8ac6000cf1f4088e7b5f Mon Sep 17 00:00:00 2001 From: Kristof Date: Sat, 18 Jul 2026 11:43:03 +0200 Subject: [PATCH 1/3] Split groundstation package out of simulation Move genuinely-production ground-station/flight-planner code into a new top-level groundstation/ package, keeping simulation/ as the physics/ mock-hardware framework: - groundstation/: pumping_planner.py, landing_planner.py, gcs.py, rawes_modes.py, mavlink_log.py, ekf_flags.py (moved whole), plus new winch_protocol.py (WinchCommand/WinchTelemetry wire format extracted from simulation/winch_node.py) and unified_ground.py (NvComms base + GcsComms production adapter extracted from simulation/unified_ground.py). - simulation/ keeps VirtualComms (comms.py), DirectComms/LuaComms (unified_ground.py), and the winch control-loop stand-ins (winch.py's WinchController, winch_node.py's GovernedWinchNode + Anemometer) since these simulate hardware/algorithms that will eventually run standalone (winch node firmware) rather than code that runs on the real ground station today. - Removed dead code: simulation/comms.py's MavlinkComms class was never imported/instantiated anywhere (GcsComms in unified_ground.py is the actual production adapter used by SITL stack tests); deleted along with its unused _PHASE_TO_SUB duplicate. - Updated ~25 import call-sites across simulation/, calibrate/, analysis/, and tests/ to the new module paths. - pyproject.toml: added groundstation/groundstation.* to the packages.find include list. test.sh: added groundstation to the Docker sync tar list. - Updated AGENTS.md, design/flight_stack.md, design/simulation.md, design/testing.md to reflect the new package and removed dead code. Validated: 467/467 unit tests, 14 passed/5 skipped simtests, and an isolated SITL stack run (test_pumping_cycle_lua_sitl) all pass. --- AGENTS.md | 5 +- analysis/analyse_run.py | 4 +- analysis/diagnose_sitl.py | 2 +- analysis/flight_log.py | 4 +- analysis/mavlink_jsonl_query.py | 4 +- calibrate/constants.py | 2 +- design/flight_stack.md | 16 ++-- design/simulation.md | 52 ++++++++----- design/testing.md | 6 +- groundstation/__init__.py | 0 {simulation => groundstation}/ekf_flags.py | 0 {simulation => groundstation}/gcs.py | 4 +- .../landing_planner.py | 0 {simulation => groundstation}/mavlink_log.py | 0 .../pumping_planner.py | 0 {simulation => groundstation}/rawes_modes.py | 2 +- groundstation/unified_ground.py | 60 +++++++++++++++ groundstation/winch_protocol.py | 53 +++++++++++++ pyproject.toml | 2 + simulation/comms.py | 76 ++----------------- simulation/mediator.py | 3 +- simulation/unified_ground.py | 61 ++------------- simulation/winch_node.py | 35 ++------- test.sh | 2 +- tests/common/mock_ardupilot.py | 4 +- tests/simtests/test_generate_ic.py | 2 +- tests/simtests/test_ground_liftoff.py | 2 +- tests/simtests/test_landing.py | 2 +- tests/simtests/test_landing_lua.py | 4 +- tests/simtests/test_pump_cycle_lua.py | 11 +-- tests/simtests/test_pump_cycle_unified.py | 2 +- tests/simtests/test_steady_flight.py | 2 +- tests/simtests/test_steady_flight_lua.py | 2 +- tests/simtests/test_yaw_regulation_lua.py | 2 +- tests/sitl/flight/test_pumping_cycle_sitl.py | 4 +- tests/sitl/stack_infra.py | 2 +- tests/sitl/stack_utils.py | 2 +- tests/unit/test_ap_controller.py | 2 +- tests/unit/test_full_loop_stability.py | 2 +- tests/unit/test_math_lua.py | 2 +- 40 files changed, 223 insertions(+), 217 deletions(-) create mode 100644 groundstation/__init__.py rename {simulation => groundstation}/ekf_flags.py (100%) rename {simulation => groundstation}/gcs.py (99%) rename {simulation => groundstation}/landing_planner.py (100%) rename {simulation => groundstation}/mavlink_log.py (100%) rename {simulation => groundstation}/pumping_planner.py (100%) rename {simulation => groundstation}/rawes_modes.py (98%) create mode 100644 groundstation/unified_ground.py create mode 100644 groundstation/winch_protocol.py diff --git a/AGENTS.md b/AGENTS.md index 5f29cbe..c7ca9e2 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -15,14 +15,15 @@ Current focus: ## Repository Layout The repo root is a single Python distribution (`pyproject.toml`, name `rawes`) containing -8 first-party top-level packages, installed in editable mode +9 first-party top-level packages, installed in editable mode (`pip install -e . --no-deps`, done by `setup.cmd`/`setup.sh`). Import them as plain dotted packages (`from simulation.controller import ...`, `from analysis.flight_log import ...`) — there are no `sys.path.insert()` hacks anywhere in the codebase. | Package | Contents | |---|---| -| `simulation/` | Physics/aero/EKF-adjacent simulation runtime, Lua/Python flight-stack modules (`mediator.py`, `controller.py`, `param_defaults.py`, `swashplate.py`, `frames.py`, `gcs.py`, ...), `requirements.txt`, `Dockerfile`, `logs/` | +| `simulation/` | Physics/aero/EKF-adjacent simulation runtime, Lua/Python flight-stack modules (`mediator.py`, `controller.py`, `param_defaults.py`, `swashplate.py`, `frames.py`, `winch.py`, `winch_node.py`, `comms.py` (`VirtualComms`), ...), `requirements.txt`, `Dockerfile`, `logs/` | +| `groundstation/` | Genuinely-production ground-station/flight-planner code, split out from `simulation/`: `pumping_planner.py`, `landing_planner.py`, `gcs.py` (`RawesGCS` MAVLink client), `rawes_modes.py` (protocol constants), `mavlink_log.py`, `ekf_flags.py`, `winch_protocol.py` (`WinchCommand`/`WinchTelemetry` wire format), `unified_ground.py` (`GcsComms` production comms adapter). Distinction: code that will genuinely run on the real ground station/flight planner lives here; code that stands in for not-yet-built hardware (e.g. the winch-node control loop in `simulation/winch.py`) stays in `simulation/` | | `arduloop/` | Self-contained Python port of ArduPilot's traditional-heli attitude/rate-control stack (used by the in-process mock ArduPilot) | | `calibrate/` | Bench calibration REPL/tooling for hardware bring-up | | `envelope/` | Flight-envelope map computation (`compute_map.py`) and related analysis | diff --git a/analysis/analyse_run.py b/analysis/analyse_run.py index 9ecb39a..3575a1f 100644 --- a/analysis/analyse_run.py +++ b/analysis/analyse_run.py @@ -44,8 +44,8 @@ _LOG_DIR = _SIM_DIR / "logs" from simulation.telemetry_csv import TelRow, read_csv # noqa: E402 -from simulation.mavlink_log import iter_messages as _iter_mavlink # noqa: E402 -from simulation.ekf_flags import has_warn as _ekf_has_warn # noqa: E402 +from groundstation.mavlink_log import iter_messages as _iter_mavlink # noqa: E402 +from groundstation.ekf_flags import has_warn as _ekf_has_warn # noqa: E402 from simulation.ic import load_ic_dict # noqa: E402 from analysis.flight_log import FlightLog # noqa: E402 diff --git a/analysis/diagnose_sitl.py b/analysis/diagnose_sitl.py index 5015858..cd87bb5 100644 --- a/analysis/diagnose_sitl.py +++ b/analysis/diagnose_sitl.py @@ -41,7 +41,7 @@ _LOG_DIR = _SIM_DIR / "logs" _IC_PATH = _SIM_DIR / "steady_state_starting.json" -from simulation.ekf_flags import EKF_FLAGS, has_warn, decode_flags # noqa: E402 +from groundstation.ekf_flags import EKF_FLAGS, has_warn, decode_flags # noqa: E402 from analysis.flight_log import FlightLog # noqa: E402 diff --git a/analysis/flight_log.py b/analysis/flight_log.py index ccbcf87..fb93a21 100644 --- a/analysis/flight_log.py +++ b/analysis/flight_log.py @@ -19,8 +19,8 @@ _SIM_DIR = Path(_simulation_pkg.__file__).resolve().parent # simulation/ from simulation.telemetry_csv import read_csv -from simulation.mavlink_log import iter_messages as _iter_mavlink # noqa: F401 (re-exported for callers) -from simulation.ekf_flags import ( +from groundstation.mavlink_log import iter_messages as _iter_mavlink # noqa: F401 (re-exported for callers) +from groundstation.ekf_flags import ( MAV_MODE_ARMED, ARDU_MODES, decode_flags, flag_diff, has_warn, ) diff --git a/analysis/mavlink_jsonl_query.py b/analysis/mavlink_jsonl_query.py index 9deb0fc..9721ad8 100644 --- a/analysis/mavlink_jsonl_query.py +++ b/analysis/mavlink_jsonl_query.py @@ -44,8 +44,8 @@ import simulation as _simulation_pkg _SIM_DIR = Path(_simulation_pkg.__file__).resolve().parent # simulation/ -from simulation.mavlink_log import iter_messages # noqa: E402 -from simulation.ekf_flags import MAV_MODE_ARMED, ARDU_MODES, MAV_STATE # noqa: E402 +from groundstation.mavlink_log import iter_messages # noqa: E402 +from groundstation.ekf_flags import MAV_MODE_ARMED, ARDU_MODES, MAV_STATE # noqa: E402 MAV_SEVERITY = { 0: "EMERGENCY", 1: "ALERT", 2: "CRITICAL", 3: "ERROR", diff --git a/calibrate/constants.py b/calibrate/constants.py index bf154e0..4769dcf 100644 --- a/calibrate/constants.py +++ b/calibrate/constants.py @@ -12,7 +12,7 @@ import simulation # Re-exported so submodules can do `from .constants import RawesGCS` etc. -from simulation.gcs import RawesGCS, WallClock +from groundstation.gcs import RawesGCS, WallClock from simulation.param_defaults import load_ap_params from simulation.servo_pwm import (SWASH_PWM_MIN, SWASH_PWM_NEUTRAL, SWASH_PWM_MAX, MOTOR_PWM_MIN, MOTOR_PWM_MAX) diff --git a/design/flight_stack.md b/design/flight_stack.md index c8f8da4..eca9aa9 100644 --- a/design/flight_stack.md +++ b/design/flight_stack.md @@ -199,7 +199,7 @@ So the controller is a **single proportional gain on tension error**, followed b ### 3.3 TensionCommand Protocol -Ground→AP command packet (10 Hz), carried by `VirtualComms` (simtest) or `GcsComms` (stack): +Ground→AP command packet (10 Hz), carried by `VirtualComms` (simtest) or `groundstation.unified_ground.GcsComms` (stack): ```python @dataclass(frozen=True) @@ -1052,16 +1052,18 @@ level first. | `calibrate/` (`python -m calibrate`) | Interactive calibration CLI for run/watch/motor/log/config workflows | | `simulation/controller.py` | `compute_bz_altitude_hold`, `AltitudeHoldController`, `TensionPI`, `RatePID`, `compute_rate_cmd`, `OrbitTracker` | | `simulation/ap_controller.py` | `TensionApController` (400 Hz AP side), `LandingApController` | -| `simulation/pumping_planner.py` | `TensionCommand`, `PumpingGroundController` (10 Hz phase state machine) | -| `simulation/unified_ground.py` | `DirectComms`, `LuaComms`, `GcsComms` (TensionCommand -> NAMED_VALUE_FLOAT adapters) | -| `simulation/winch.py` | `WinchController` (tension-controlled, 400 Hz) | -| `simulation/winch_node.py` | `WinchNode` + `Anemometer` (physics/planner protocol boundary) | +| `groundstation/pumping_planner.py` | `TensionCommand`, `PumpingGroundController` (10 Hz phase state machine) | +| `groundstation/unified_ground.py` | `NvComms`, `GcsComms` (production TensionCommand -> NAMED_VALUE_FLOAT adapter) | +| `simulation/unified_ground.py` | `DirectComms`, `LuaComms` (test-only TensionCommand adapters) | +| `simulation/winch.py` | `WinchController` (tension-controlled, 400 Hz; stands in for future dedicated winch-node hardware) | +| `simulation/winch_node.py` | `GovernedWinchNode` + `Anemometer` (simulated winch-node hardware stand-in) | +| `groundstation/winch_protocol.py` | `WinchCommand`, `WinchTelemetry` (ground <-> winch-node wire protocol) | | `simulation/physics_core.py` | `PhysicsCore` — shared 400 Hz physics (dynamics, aero, tether, spin ODE, kinematic) | | `simulation/mediator.py` | SITL co-simulation loop — thin wrapper around PhysicsCore | | `simulation/torque_model.py` | Hub yaw kinematics: `HubParams`, `HubState`, `step()`, `equilibrium_throttle()` | | `simulation/mediator_torque.py` | Standalone torque SITL mediator | -| `simulation/comms.py` | `VirtualComms` (simtest), `MavlinkComms` (SITL/hardware) | -| `simulation/gcs.py` | `RawesGCS` MAVLink client: arm, mode, params, `send_named_float` | +| `simulation/comms.py` | `VirtualComms` (simtest-only comms link) | +| `groundstation/gcs.py` | `RawesGCS` MAVLink client: arm, mode, params, `send_named_float` | | `simulation/sensor.py` | `PhysicalSensor` — honest NED sensors (accel, gyro, vel) | | `analysis/analyse_run.py` | Post-run report: physics + EKF/GPS + attitude per time bucket | | `analysis/analyse_landing.py` | Landing diagnosis: alt/vz/winch/tension/collective per bucket | diff --git a/design/simulation.md b/design/simulation.md index be7276a..7f8dddb 100644 --- a/design/simulation.md +++ b/design/simulation.md @@ -363,7 +363,7 @@ PumpingGroundController (10 Hz) ──TensionCommand──▶ _PumpingPythonMo - **`pumping_planner.PumpingGroundController`** (10 Hz) emits `TensionCommand(tension_target_n, alt_m, phase)` — the **commanded** tension and target altitude only. The ground closes the tension loop itself on the winch's load cell; the AP never receives the measurement. `alt_m` is smoothly ramped at phase boundaries using `hub_alt_m` telemetry received from the kite at 10 Hz: up over `t_transition` seconds entering "transition", down at the start of each next "reel_out". Sudden `alt_m` jumps are ground-controller bugs — detected by `ap_unreachable_alt`. - **`_PumpingPythonMode`** (AP side, in `mock_ardupilot.py`) uses the commanded tension (`cmd.tension_target_n`) as a feedforward into the orientation force balance (`bz_altitude_hold`) and holds altitude with an altitude PID on collective — the same law as steady mode. There is **no TensionPI on the AP**. It validates each received command via `BadEventLog`: `ap_impossible_alt` (alt_m > tether_length), `ap_unreachable_alt` (elevation gap > `slew_rate × FEASIBILITY_WINDOW_S = 1 s`). **Blame rule:** `ap_*` events → ground planner sent unreachable commands; slack/tension_spike without `ap_*` → AP tracking failure. - **`winch.WinchController`** (400 Hz) is tension-controlled: cruise speed proportional to tension error, trapezoidal accel/decel profile, virtual battery accumulates energy_out_j / energy_in_j / net_energy_j. -- **`unified_ground`** provides pluggable comms adapters that marshal `TensionCommand` to the AP: `DirectComms` (Python AP), `LuaComms` (Lua simtest), `GcsComms` (SITL stack). +- **`unified_ground`** provides pluggable comms adapters that marshal `TensionCommand` to the AP: `DirectComms` (Python AP, `simulation/unified_ground.py`), `LuaComms` (Lua simtest, `simulation/unified_ground.py`), `GcsComms` (SITL stack, `groundstation/unified_ground.py` — the production adapter). ### Critical design invariants @@ -446,10 +446,14 @@ Additional physics limitations in the current simulation: ## Module Map -Repo layout note: `simulation/` is one of 8 top-level packages (see `AGENTS.md` -> -Repository Layout). `arduloop/`, `analysis/`, `viz3d/`, `scripts/`, and `tests/` are -siblings of `simulation/` at the repo root, not nested inside it — shown here as -separate trees for clarity. +Repo layout note: `simulation/` is one of 9 top-level packages (see `AGENTS.md` -> +Repository Layout). `arduloop/`, `groundstation/`, `analysis/`, `viz3d/`, `scripts/`, +and `tests/` are siblings of `simulation/` at the repo root, not nested inside it — +shown here as separate trees for clarity. Genuinely production ground-station/ +flight-planner code (`pumping_planner.py`, `landing_planner.py`, `gcs.py`, +`rawes_modes.py`, `mavlink_log.py`, `ekf_flags.py`, the `WinchCommand`/ +`WinchTelemetry` wire protocol, and `GcsComms`) lives in `groundstation/`, not +`simulation/` — see the `groundstation/` tree below. ``` simulation/ @@ -484,21 +488,17 @@ simulation/ ├── torque_model.py Hub yaw model (kinematic + motor lag) — HubParams (rpm_scale, gear_ratio, │ motor_tau), HubState (psi, psi_dot, omega_motor), step(), equilibrium_throttle() ├── kinematic.py KinematicStartup — hub trajectory during EKF init phase -├── winch_node.py WinchNode + Anemometer (physics/planner protocol boundary) -├── gcs.py MAVLink GCS client (arm, mode, params, named-float commands). -│ recv_local_position_latest() — non-blocking poll of LOCAL_POSITION_NED. -├── comms.py MAVLink comms boundary between ground and AP. +├── winch_node.py GovernedWinchNode + Anemometer — simulated winch-node hardware stand-in. +│ WinchCommand/WinchTelemetry wire protocol lives in +│ groundstation/winch_protocol.py (imported here). +├── comms.py MAVLink comms boundary between ground and AP (simtest-only). │ VirtualComms — simtest: latency queue + optional Gaussian noise on hub_alt_m; │ inject(t, alt) at 400 Hz, receive_telemetry(t)/send_command(t,cmd)/ -│ poll_ap_command(t) at 10 Hz. -│ MavlinkComms — SITL/hardware: receive_telemetry() polls LOCAL_POSITION_NED; -│ send_command() sends RAWES_TEN + RAWES_ALT + RAWES_SUB via gcs.py. -├── unified_ground.py TensionCommand comms adapters: DirectComms (Python AP), LuaComms (Lua -│ simtest), GcsComms (SITL stack). Marshal TensionCommand -> NV float pairs. -├── pumping_planner.py TensionCommand dataclass + PumpingGroundController (10 Hz phase schedule). -│ step(t_sim, tension_measured_n, rest_length, hub_alt_m) → TensionCommand. -│ alt_m is smoothly ramped at every phase boundary using hub_alt_m telemetry. -│ winch_target_length / winch_target_tension properties for WinchController. +│ poll_ap_command(t) at 10 Hz. The production adapter is +│ groundstation.unified_ground.GcsComms (SITL/hardware). +├── unified_ground.py Test-only TensionCommand comms adapters: DirectComms (Python AP), LuaComms +│ (Lua simtest). NvComms base + the production GcsComms adapter live in +│ groundstation/unified_ground.py. ├── ap_controller.py TensionApController (400 Hz AP side): TensionPI collective + rate-limited │ elevation hold. receive_command(cmd, dt) validates alt_m against cached │ pos_ned (updated each step): ap_impossible_alt (alt > tether_length), @@ -519,6 +519,22 @@ simulation/ ├── rawes_lua_harness.py RawesLua class — runs rawes.lua in-process via lupa; shared by unit │ tests and simtests. Loads mock_ardupilot.lua then rawes.lua. Python writes │ sensor inputs to `_mock` and calls `_update_fn()` each tick. +└── (rawes_modes.py, gcs.py, mavlink_log.py, ekf_flags.py, pumping_planner.py, + landing_planner.py moved to groundstation/ — see below) + +groundstation/ +├── pumping_planner.py TensionCommand dataclass + PumpingGroundController (10 Hz phase schedule). +│ step(t_sim, tension_measured_n, rest_length, hub_alt_m) → TensionCommand. +│ alt_m is smoothly ramped at every phase boundary using hub_alt_m telemetry. +│ winch_target_length / winch_target_tension properties for WinchController. +├── landing_planner.py LandingCommand dataclass + LandingGroundController (10 Hz phase schedule). +├── winch_protocol.py WinchCommand/WinchTelemetry — ground <-> winch-node wire protocol dataclasses. +├── unified_ground.py NvComms base + _cmd_to_nv marshalling + GcsComms — production TensionCommand +│ -> NAMED_VALUE_FLOAT adapter (SITL stack / real hardware). +├── gcs.py MAVLink GCS client (arm, mode, params, named-float commands). +│ recv_local_position_latest() — non-blocking poll of LOCAL_POSITION_NED. +├── mavlink_log.py MavlinkLogWriter (live NDJSON message log) + iter_messages() (log reader). +├── ekf_flags.py EKF_STATUS_REPORT flag decode helpers used by analysis/ tooling. └── rawes_modes.py Python constants mirroring rawes.lua mode/substate numbers. scripts/ diff --git a/design/testing.md b/design/testing.md index 16e1eac..a770141 100644 --- a/design/testing.md +++ b/design/testing.md @@ -206,7 +206,7 @@ _mock = global state table (inputs/outputs bridged to Python) ```python from simulation.rawes_lua_harness import RawesLua -from simulation.rawes_modes import MODE_STEADY, PUMP_HOLD +from groundstation.rawes_modes import MODE_STEADY, PUMP_HOLD sim = RawesLua(mode=MODE_STEADY) # RAWES_MODE = 1 (pumping schedule runs in steady) sim.armed = True @@ -263,10 +263,10 @@ sim.vec_to_list(bz) # -> [x, y, z] `RAWES_MODE` is a **script-generated parameter** (registered by rawes.lua at load time; visible as `RAWES_MODE` in GCS). Substate is delivered separately via `NAMED_VALUE_FLOAT("RAWES_SUB", N)` — never encoded in the mode param. -Constants are in `simulation/rawes_modes.py` (Python) and as locals in `rawes.lua` (Lua). +Constants are in `groundstation/rawes_modes.py` (Python) and as locals in `rawes.lua` (Lua). ```python -from simulation.rawes_modes import ( +from groundstation.rawes_modes import ( MODE_NONE, MODE_STEADY, MODE_LANDING, LAND_DESCEND, LAND_FINAL_DROP, diff --git a/groundstation/__init__.py b/groundstation/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/simulation/ekf_flags.py b/groundstation/ekf_flags.py similarity index 100% rename from simulation/ekf_flags.py rename to groundstation/ekf_flags.py diff --git a/simulation/gcs.py b/groundstation/gcs.py similarity index 99% rename from simulation/gcs.py rename to groundstation/gcs.py index 4e4dfa7..286975a 100644 --- a/simulation/gcs.py +++ b/groundstation/gcs.py @@ -28,7 +28,7 @@ from pymavlink import mavutil -from simulation.mavlink_log import MavlinkLogWriter +from groundstation.mavlink_log import MavlinkLogWriter log = logging.getLogger(__name__) @@ -1080,7 +1080,7 @@ def recv_local_position_latest(self) -> tuple[float, float, float] | None: """ Non-blocking poll: drain buffered LOCAL_POSITION_NED messages and return the most recent one, or None if none are buffered. - Used by MavlinkComms.receive_telemetry() in the 10 Hz ground loop. + Used in the 10 Hz ground loop to read the downlinked hub altitude. """ latest = None while True: diff --git a/simulation/landing_planner.py b/groundstation/landing_planner.py similarity index 100% rename from simulation/landing_planner.py rename to groundstation/landing_planner.py diff --git a/simulation/mavlink_log.py b/groundstation/mavlink_log.py similarity index 100% rename from simulation/mavlink_log.py rename to groundstation/mavlink_log.py diff --git a/simulation/pumping_planner.py b/groundstation/pumping_planner.py similarity index 100% rename from simulation/pumping_planner.py rename to groundstation/pumping_planner.py diff --git a/simulation/rawes_modes.py b/groundstation/rawes_modes.py similarity index 98% rename from simulation/rawes_modes.py rename to groundstation/rawes_modes.py index 4601483..ff4a707 100644 --- a/simulation/rawes_modes.py +++ b/groundstation/rawes_modes.py @@ -14,7 +14,7 @@ Usage ----- - from simulation.rawes_modes import MODE_STEADY, PUMP_REEL_OUT, send_anchor_ned + from groundstation.rawes_modes import MODE_STEADY, PUMP_REEL_OUT, send_anchor_ned gcs.set_param("RAWES_MODE", MODE_STEADY) # set mode (pumping runs in steady) gcs.send_named_float("RAWES_SUB", PUMP_REEL_OUT) # set substate diff --git a/groundstation/unified_ground.py b/groundstation/unified_ground.py new file mode 100644 index 0000000..9cc6c2a --- /dev/null +++ b/groundstation/unified_ground.py @@ -0,0 +1,60 @@ +""" +unified_ground.py -- Ground-side pumping comms wire protocol + production adapter. + +Marshals a PumpingGroundController's TensionCommand to the AP as +NAMED_VALUE_FLOAT pairs: + + RAWES_TEN target/feed-forward tension [N] for gravity compensation + RAWES_ALT altitude target [m] + RAWES_SUB phase as integer (0=hold 1=reel-out 2=transition 3=reel-in) + + GcsComms(gcs) SITL stack test / real hardware: sends NAMED_VALUE_FLOAT via MAVLink + +The NvComms base + _cmd_to_nv marshalling here is the single source of truth +for this wire format; simulation.unified_ground's test-only adapters +(DirectComms, LuaComms) reuse NvComms from this module. +""" + +from __future__ import annotations + +from groundstation.pumping_planner import TensionCommand + +_PHASE_TO_SUB: dict[str, int] = { + "hold": 0, + "reel-out": 1, + "transition": 2, + "reel-in": 3, +} + + +def _cmd_to_nv(cmd: TensionCommand) -> list[tuple[str, float]]: + """Convert a TensionCommand to a list of (name, value) NV float pairs.""" + return [ + ("RAWES_TEN", cmd.tension_target_n), # target tension for gravity comp + ("RAWES_ALT", cmd.alt_m), + ("RAWES_SUB", float(_PHASE_TO_SUB.get(cmd.phase, 0))), + ] + + +class NvComms: + """Base for comms that marshal TensionCommand to NAMED_VALUE_FLOAT pairs.""" + + def send_nv(self, name: str, value: float) -> None: + raise NotImplementedError + + def send(self, cmd: TensionCommand, dt: float) -> None: + for name, value in _cmd_to_nv(cmd): + self.send_nv(name, value) + + +class GcsComms(NvComms): + """Sends TensionCommand via MAVLink NAMED_VALUE_FLOAT (SITL stack tests). + + gcs: object with send_named_float(name: str, value: float) — e.g. RawesGCS. + """ + + def __init__(self, gcs) -> None: + self._gcs = gcs + + def send_nv(self, name: str, value: float) -> None: + self._gcs.send_named_float(name, value) diff --git a/groundstation/winch_protocol.py b/groundstation/winch_protocol.py new file mode 100644 index 0000000..2bb58b1 --- /dev/null +++ b/groundstation/winch_protocol.py @@ -0,0 +1,53 @@ +""" +winch_protocol.py -- Wire protocol between the ground-station planner and the +winch node (a separate physical node at the anchor: winch drum + load cell + +co-located anemometer, eventually its own dedicated hardware). + + GCS / planner node --(WinchCommand: targets only)--> Winch node + Winch node --(WinchTelemetry: sensed only)-> GCS / planner node + +The fast tension-governing control loop itself (simulation.winch. +GovernedWinchController, hosted today by simulation.winch_node. +GovernedWinchNode) is NOT part of this protocol -- it stands in for winch-node +firmware that does not exist yet, so it lives in simulation/ alongside the +other not-yet-real-hardware stand-ins. Only the cable boundary -- these two +dataclasses -- is genuine production wire format, used identically whether +the winch node is real hardware or the simulated stand-in. + +No-leak guarantee: the planner can only see a WinchTelemetry, whose fields are +all quantities the drum/anchor hardware physically senses (load cell, drum +encoder, co-located anemometer). Hub altitude / position / attitude never +cross this boundary. +""" + +from __future__ import annotations + +from dataclasses import dataclass + + +@dataclass(frozen=True) +class WinchCommand: + """Down-link (planner -> winch node): targets only. + + cruise_v -- commanded cruise reel velocity [m/s], +out / -in / 0=hold + tension_target -- governor tension setpoint [N] + """ + cruise_v: float + tension_target: float + + +@dataclass(frozen=True) +class WinchTelemetry: + """Up-link (winch node -> planner): winch-measurable quantities only. + + tension_n -- load cell [N] + rest_length -- drum encoder: tether rest length [m] + speed_ms -- reel speed [m/s], signed (+out) + net_energy_j -- drum mechanical energy, integral of T*v [J] + wind_ned -- co-located anemometer reading [NED, m/s] + """ + tension_n: float + rest_length: float + speed_ms: float + net_energy_j: float + wind_ned: tuple diff --git a/pyproject.toml b/pyproject.toml index 3cb9ebf..1686e08 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -19,6 +19,8 @@ where = ["."] include = [ "simulation", "simulation.*", + "groundstation", + "groundstation.*", "arduloop", "arduloop.*", "calibrate", diff --git a/simulation/comms.py b/simulation/comms.py index ef5daf9..2a35535 100644 --- a/simulation/comms.py +++ b/simulation/comms.py @@ -1,10 +1,11 @@ """ -comms.py — MAVLink communication boundary between ground controller and AP. +comms.py — MAVLink communication boundary between ground controller and AP +(simtest-only implementation). -Two implementations share the same call pattern: - - VirtualComms — simtest: latency queue + optional Gaussian noise - MavlinkComms — SITL/hardware: real MAVLink via gcs.py +VirtualComms simulates the comms link for Python simtests (latency queue + +optional Gaussian noise). The real SITL/hardware adapter is +groundstation.unified_ground.GcsComms, which marshals commands to NAMED_VALUE_FLOAT +via MAVLink (see groundstation/unified_ground.py for the shared NvComms protocol). Simtest loop (VirtualComms): @@ -20,16 +21,6 @@ if ap_cmd: ap.receive_command(ap_cmd, DT_PLANNER) -Stack-test loop (MavlinkComms): - - comms = MavlinkComms(gcs) - # 10 Hz ground step: - tel = comms.receive_telemetry() # non-blocking poll LOCAL_POSITION_NED - cmd = ground.step(t_sim, tension, rest_length, - hub_alt_m=tel.hub_alt_m if tel else prev_alt) - comms.send_command(t_sim, cmd) # RAWES_TEN + RAWES_ALT + RAWES_SUB - # No poll_ap_command — ArduPilot/Lua handles AP side directly. - Latency model (VirtualComms): Each injected telemetry sample is tagged with t_sim. receive_telemetry(t_now) returns the most-recent sample where t_injected <= t_now - latency_s. @@ -47,8 +38,7 @@ from typing import TYPE_CHECKING if TYPE_CHECKING: - from simulation.pumping_planner import TensionCommand - from simulation.gcs import GCSClient + from groundstation.pumping_planner import TensionCommand # --------------------------------------------------------------------------- @@ -123,55 +113,3 @@ def poll_ap_command(self, t_now: float) -> "TensionCommand | None": return cmd return None - -# --------------------------------------------------------------------------- -# MavlinkComms — SITL / hardware implementation -# --------------------------------------------------------------------------- - -# Phase string → RAWES_SUB integer (matches rawes_modes.py constants) -_PHASE_TO_SUB: dict[str, int] = { - "hold": 0, # PUMP_HOLD - "reel-out": 1, # PUMP_REEL_OUT - "transition": 2, # PUMP_TRANSITION - "reel-in": 3, # PUMP_REEL_IN -} - - -class MavlinkComms: - """ - Real MAVLink comms for SITL stack tests and hardware. - - send_command() sends RAWES_TEN + RAWES_ALT + RAWES_SUB via gcs.py. - receive_telemetry() does a non-blocking poll of LOCAL_POSITION_NED. - - tension_target_n carries the target/feed-forward tension used by Lua - gravity compensation and by the local MockArdupilot Python equivalent. - - Parameters - ---------- - gcs : GCSClient open MAVLink connection (simulation/gcs.py) - """ - - def __init__(self, gcs: "GCSClient") -> None: - self._gcs = gcs - - def receive_telemetry(self) -> HubTelemetry | None: - """ - Non-blocking poll. Returns latest LOCAL_POSITION_NED as HubTelemetry, - or None if no message is buffered. - """ - pos = self._gcs.recv_local_position_latest() - if pos is None: - return None - _n, _e, ned_z = pos - return HubTelemetry(hub_alt_m=float(-ned_z)) - - def send_command(self, _t_sim: float, cmd: "TensionCommand") -> None: - """ - Send target tension, altitude target, and phase substate to the AP. - t_sim is accepted for API symmetry with VirtualComms but is not used. - """ - sub = _PHASE_TO_SUB.get(cmd.phase, 0) - self._gcs.send_named_float("RAWES_TEN", cmd.tension_target_n) - self._gcs.send_named_float("RAWES_ALT", cmd.alt_m) - self._gcs.send_named_float("RAWES_SUB", float(sub)) diff --git a/simulation/mediator.py b/simulation/mediator.py index 9072a62..14653d7 100644 --- a/simulation/mediator.py +++ b/simulation/mediator.py @@ -48,7 +48,8 @@ from simulation.sensor import make_sensor, SpinSensor from simulation.kinematic import KinematicStartup, compute_launch_position, make_smooth_trapezoid_traj # noqa: F401 from simulation.winch import GovernedWinchController -from simulation.winch_node import GovernedWinchNode, WinchCommand, Anemometer +from simulation.winch_node import GovernedWinchNode, Anemometer +from groundstation.winch_protocol import WinchCommand import simulation.config as _mcfg from dynbem import rotor_definition as _rd from simulation.telemetry_csv import COLUMNS as _TEL_COLUMNS, ASYNC_MAV_COLUMNS diff --git a/simulation/unified_ground.py b/simulation/unified_ground.py index 49feadd..99dd3a3 100644 --- a/simulation/unified_ground.py +++ b/simulation/unified_ground.py @@ -1,43 +1,21 @@ """ -unified_ground.py -- Ground-side pumping comms adapters. +unified_ground.py -- Ground-side pumping comms adapters (simtest/test-only). -Marshals a PumpingGroundController's TensionCommand to the AP via one of three -comms adapters: +Marshals a PumpingGroundController's TensionCommand to the AP via one of two +test-only comms adapters: DirectComms(ap) Python simtest: calls ap.receive_command() directly LuaComms(inject) Lua unit test: injects NV floats via send_named_float - GcsComms(gcs) SITL stack test: sends NAMED_VALUE_FLOAT via MAVLink -LuaComms and GcsComms both marshal TensionCommand to the same three NV pairs: - RAWES_TEN target/feed-forward tension [N] for gravity compensation - RAWES_ALT altitude target [m] - RAWES_SUB phase as integer (0=hold 1=reel-out 2=transition 3=reel-in) +The real production adapter (GcsComms) and the NvComms base + _cmd_to_nv wire +marshalling live in groundstation/unified_ground.py. """ from __future__ import annotations -from simulation.pumping_planner import TensionCommand +from groundstation.pumping_planner import TensionCommand +from groundstation.unified_ground import NvComms -_PHASE_TO_SUB: dict[str, int] = { - "hold": 0, - "reel-out": 1, - "transition": 2, - "reel-in": 3, -} - - -def _cmd_to_nv(cmd: TensionCommand) -> list[tuple[str, float]]: - """Convert a TensionCommand to a list of (name, value) NV float pairs.""" - return [ - ("RAWES_TEN", cmd.tension_target_n), # target tension for gravity comp - ("RAWES_ALT", cmd.alt_m), - ("RAWES_SUB", float(_PHASE_TO_SUB.get(cmd.phase, 0))), - ] - - -# --------------------------------------------------------------------------- -# Comms adapters -# --------------------------------------------------------------------------- class DirectComms: """Delivers TensionCommand directly to a local MockArdupilot Python equivalent.""" @@ -49,30 +27,6 @@ def send(self, cmd: TensionCommand, dt: float) -> None: self._ap.receive_command(cmd, dt) -class NvComms: - """Base for comms that marshal TensionCommand to NAMED_VALUE_FLOAT pairs.""" - - def send_nv(self, name: str, value: float) -> None: - raise NotImplementedError - - def send(self, cmd: TensionCommand, dt: float) -> None: - for name, value in _cmd_to_nv(cmd): - self.send_nv(name, value) - - -class GcsComms(NvComms): - """Sends TensionCommand via MAVLink NAMED_VALUE_FLOAT (SITL stack tests). - - gcs: object with send_named_float(name: str, value: float) — e.g. RawesGCS. - """ - - def __init__(self, gcs) -> None: - self._gcs = gcs - - def send_nv(self, name: str, value: float) -> None: - self._gcs.send_named_float(name, value) - - class LuaComms(NvComms): """Injects TensionCommand into Lua's named-value inbox via send_named_float. @@ -84,3 +38,4 @@ def __init__(self, inject) -> None: def send_nv(self, name: str, value: float) -> None: self._inject(name, value) + diff --git a/simulation/winch_node.py b/simulation/winch_node.py index 82995e8..96fd7c4 100644 --- a/simulation/winch_node.py +++ b/simulation/winch_node.py @@ -14,6 +14,11 @@ winch_node.exchange(WinchCommand) -> WinchTelemetry +WinchCommand/WinchTelemetry are the genuine wire protocol and live in +groundstation.winch_protocol (importable by both the real ground station and +this simulated node). GovernedWinchNode itself stands in for winch-node +firmware that does not exist yet, so it stays here in simulation/. + The mediator (physics side) additionally calls: winch_node.update_sensors(tension, wind) (feed physics outputs in) @@ -25,9 +30,9 @@ """ import numpy as np -from dataclasses import dataclass from simulation.winch import GovernedWinchController +from groundstation.winch_protocol import WinchCommand, WinchTelemetry class Anemometer: @@ -81,34 +86,6 @@ def measure(self, wind_world_ned: np.ndarray) -> np.ndarray: # altitude / position / attitude never cross this boundary. -@dataclass(frozen=True) -class WinchCommand: - """Down-link (planner -> winch node): targets only. - - cruise_v -- commanded cruise reel velocity [m/s], +out / -in / 0=hold - tension_target -- governor tension setpoint [N] - """ - cruise_v: float - tension_target: float - - -@dataclass(frozen=True) -class WinchTelemetry: - """Up-link (winch node -> planner): winch-measurable quantities only. - - tension_n -- load cell [N] - rest_length -- drum encoder: tether rest length [m] - speed_ms -- reel speed [m/s], signed (+out) - net_energy_j -- drum mechanical energy, integral of T*v [J] - wind_ned -- co-located anemometer reading [NED, m/s] - """ - tension_n: float - rest_length: float - speed_ms: float - net_energy_j: float - wind_ned: tuple - - class GovernedWinchNode: """Winch + load cell + anemometer node hosting the fast governing loop. diff --git a/test.sh b/test.sh index 01c4440..044dd5a 100644 --- a/test.sh +++ b/test.sh @@ -56,7 +56,7 @@ _sync_code() { --exclude="tests/hil" \ --exclude=".venv" \ --exclude="*.egg-info" \ - -cf - pyproject.toml simulation arduloop envelope analysis viz3d scripts tests calibrate \ + -cf - pyproject.toml simulation groundstation arduloop envelope analysis viz3d scripts tests calibrate \ | docker exec -i "$_c" tar -xf - -C /rawes/ # dynbem (the Rust-backed aero core) is installed from a pinned PyPI wheel, # not synced from the sibling ../aero source workspace -- that source tree diff --git a/tests/common/mock_ardupilot.py b/tests/common/mock_ardupilot.py index 5e28370..93303e1 100644 --- a/tests/common/mock_ardupilot.py +++ b/tests/common/mock_ardupilot.py @@ -21,9 +21,9 @@ apply_crosswind_rate_to_body_rates, slerp_body_z, update_plane_azimuth, YawTrimObserver) -from simulation.landing_planner import LandingCommand +from groundstation.landing_planner import LandingCommand from simulation.physics_core import HubObservation -from simulation.pumping_planner import TensionCommand +from groundstation.pumping_planner import TensionCommand from simulation.param_defaults import get_ap_param, load_ap_params, load_rawes_lua_constants, load_collective_phys_range from simulation.telemetry_csv import TelRow, write_csv diff --git a/tests/simtests/test_generate_ic.py b/tests/simtests/test_generate_ic.py index a790c16..b61b262 100644 --- a/tests/simtests/test_generate_ic.py +++ b/tests/simtests/test_generate_ic.py @@ -37,7 +37,7 @@ from simulation.frames import build_orb_frame from tests.simtests.simtest_runner import PhysicsRunner from tests.common.mock_ardupilot import MockArdupilot -from simulation.pumping_planner import TensionCommand +from groundstation.pumping_planner import TensionCommand from simulation.controller import compute_bz_altitude_hold from tests.simtests._rotor_helpers import ( load_default_rotor, dynamics_kwargs, BODY_Z_SLEW_RATE_RAD_S, diff --git a/tests/simtests/test_ground_liftoff.py b/tests/simtests/test_ground_liftoff.py index f03bf19..d6a9870 100644 --- a/tests/simtests/test_ground_liftoff.py +++ b/tests/simtests/test_ground_liftoff.py @@ -31,7 +31,7 @@ from tests.simtests.simtest_runner import PhysicsRunner from tests.common.mock_ardupilot import MockArdupilot from simulation.rawes_lua_harness import RawesLua -from simulation.rawes_modes import MODE_STEADY, send_anchor_ned +from groundstation.rawes_modes import MODE_STEADY, send_anchor_ned from tests.simtests._rotor_helpers import load_default_rotor # ── Physical constants ──────────────────────────────────────────────────────── diff --git a/tests/simtests/test_landing.py b/tests/simtests/test_landing.py index 7a17738..3964857 100644 --- a/tests/simtests/test_landing.py +++ b/tests/simtests/test_landing.py @@ -32,7 +32,7 @@ from tests.simtests.simtest_ic import load_ic from tests.simtests.simtest_runner import PhysicsRunner from tests.common.mock_ardupilot import MockArdupilot -from simulation.landing_planner import LandingGroundController +from groundstation.landing_planner import LandingGroundController from tests.simtests._rotor_helpers import load_default_rotor, BODY_Z_SLEW_RATE_RAD_S _IC = load_ic() diff --git a/tests/simtests/test_landing_lua.py b/tests/simtests/test_landing_lua.py index b37bdc3..5a6ecc3 100644 --- a/tests/simtests/test_landing_lua.py +++ b/tests/simtests/test_landing_lua.py @@ -27,9 +27,9 @@ from tests.simtests.simtest_ic import load_ic from tests.simtests.simtest_runner import PhysicsRunner from tests.common.mock_ardupilot import MockArdupilot -from simulation.landing_planner import LandingGroundController +from groundstation.landing_planner import LandingGroundController from simulation.rawes_lua_harness import RawesLua -from simulation.rawes_modes import MODE_LANDING, LAND_FINAL_DROP +from groundstation.rawes_modes import MODE_LANDING, LAND_FINAL_DROP from tests.simtests._rotor_helpers import load_default_rotor, BODY_Z_SLEW_RATE_RAD_S _IC = load_ic() diff --git a/tests/simtests/test_pump_cycle_lua.py b/tests/simtests/test_pump_cycle_lua.py index d7bac5f..b63a6a2 100644 --- a/tests/simtests/test_pump_cycle_lua.py +++ b/tests/simtests/test_pump_cycle_lua.py @@ -31,14 +31,15 @@ pytest.mark.timeout(int(os.environ.get("RAWES_PUMP_TIMEOUT", "600")))] from simulation.winch import GovernedWinchController -from simulation.winch_node import GovernedWinchNode, WinchCommand, Anemometer +from simulation.winch_node import GovernedWinchNode, Anemometer +from groundstation.winch_protocol import WinchCommand from tests.simtests.simtest_ic import load_ic from simulation.simtest_log import BadEventLog from tests.simtests.simtest_runner import PhysicsRunner from tests.common.mock_ardupilot import MockArdupilot -from simulation.pumping_planner import PumpingGroundController +from groundstation.pumping_planner import PumpingGroundController from simulation.rawes_lua_harness import RawesLua -from simulation.rawes_modes import MODE_STEADY, send_anchor_ned +from groundstation.rawes_modes import MODE_STEADY, send_anchor_ned from simulation.unified_ground import LuaComms from tests.simtests._rotor_helpers import load_default_rotor @@ -284,8 +285,8 @@ def test_lua_pumping_constants(): sim = RawesLua(mode=MODE_STEADY) f = sim.fns # NV names are ≤ 10 chars (MAVLink hard limit) - from simulation.unified_ground import _cmd_to_nv - from simulation.pumping_planner import TensionCommand + from groundstation.unified_ground import _cmd_to_nv + from groundstation.pumping_planner import TensionCommand dummy = TensionCommand( tension_target_n=300.0, alt_m=38.0, diff --git a/tests/simtests/test_pump_cycle_unified.py b/tests/simtests/test_pump_cycle_unified.py index 58b2e82..b176f11 100644 --- a/tests/simtests/test_pump_cycle_unified.py +++ b/tests/simtests/test_pump_cycle_unified.py @@ -31,7 +31,7 @@ from tests.simtests.simtest_ic import load_ic from tests.simtests.simtest_runner import PhysicsRunner from tests.common.mock_ardupilot import MockArdupilot -from simulation.pumping_planner import TensionCommand +from groundstation.pumping_planner import TensionCommand from simulation.comms import VirtualComms from tests.simtests._rotor_helpers import load_default_rotor, BODY_Z_SLEW_RATE_RAD_S from simulation.param_defaults import thrust_to_coll_rad diff --git a/tests/simtests/test_steady_flight.py b/tests/simtests/test_steady_flight.py index b634f8d..18aa504 100644 --- a/tests/simtests/test_steady_flight.py +++ b/tests/simtests/test_steady_flight.py @@ -34,7 +34,7 @@ import simulation.mediator as _mediator_module from tests.simtests.simtest_runner import PhysicsRunner from tests.common.mock_ardupilot import MockArdupilot -from simulation.pumping_planner import TensionCommand +from groundstation.pumping_planner import TensionCommand from tests.simtests.simtest_ic import load_ic from tests.simtests._rotor_helpers import ( load_default_rotor, dynamics_kwargs, BODY_Z_SLEW_RATE_RAD_S, diff --git a/tests/simtests/test_steady_flight_lua.py b/tests/simtests/test_steady_flight_lua.py index 53f1776..4bb2913 100644 --- a/tests/simtests/test_steady_flight_lua.py +++ b/tests/simtests/test_steady_flight_lua.py @@ -30,7 +30,7 @@ from tests.simtests.simtest_runner import PhysicsRunner from tests.common.mock_ardupilot import MockArdupilot from simulation.rawes_lua_harness import RawesLua -from simulation.rawes_modes import MODE_STEADY, send_anchor_ned +from groundstation.rawes_modes import MODE_STEADY, send_anchor_ned from tests.simtests._rotor_helpers import load_default_rotor _IC = load_ic() diff --git a/tests/simtests/test_yaw_regulation_lua.py b/tests/simtests/test_yaw_regulation_lua.py index 3c6f630..9e59a2b 100644 --- a/tests/simtests/test_yaw_regulation_lua.py +++ b/tests/simtests/test_yaw_regulation_lua.py @@ -37,7 +37,7 @@ import simulation.torque_model as _m from simulation.rawes_lua_harness import RawesLua -from simulation.rawes_modes import MODE_PASSIVE +from groundstation.rawes_modes import MODE_PASSIVE # --------------------------------------------------------------------------- # Plant constants (torque_model defaults — GB4008 66KV, 80:44 gear, 4S LiPo) diff --git a/tests/sitl/flight/test_pumping_cycle_sitl.py b/tests/sitl/flight/test_pumping_cycle_sitl.py index 77ae4e5..22bec30 100644 --- a/tests/sitl/flight/test_pumping_cycle_sitl.py +++ b/tests/sitl/flight/test_pumping_cycle_sitl.py @@ -42,8 +42,8 @@ assert_no_mediator_criticals, assert_procs_alive, ) from simulation.telemetry_csv import read_csv -from simulation.pumping_planner import PumpingGroundController -from simulation.unified_ground import _cmd_to_nv +from groundstation.pumping_planner import PumpingGroundController +from groundstation.unified_ground import _cmd_to_nv from tests.simtests._rotor_helpers import load_default_rotor _ROTOR = load_default_rotor() diff --git a/tests/sitl/stack_infra.py b/tests/sitl/stack_infra.py index 542cadb..004f58e 100644 --- a/tests/sitl/stack_infra.py +++ b/tests/sitl/stack_infra.py @@ -83,7 +83,7 @@ def _project_default_rotor(): ) from pymavlink import mavutil as _mavutil -from simulation.gcs import GUIDED, GUIDED_NOGPS, STABILIZE, RawesGCS +from groundstation.gcs import GUIDED, GUIDED_NOGPS, STABILIZE, RawesGCS from simulation.mediator_events import MediatorEventLog from simulation.controller import make_hold_controller from simulation.ic import load_ic_dict, IC_JSON_PATH diff --git a/tests/sitl/stack_utils.py b/tests/sitl/stack_utils.py index 6dfdbde..0da2630 100644 --- a/tests/sitl/stack_utils.py +++ b/tests/sitl/stack_utils.py @@ -16,7 +16,7 @@ import numpy as np from simulation.frames import build_gps_yaw_frame -from simulation.rawes_modes import ( +from groundstation.rawes_modes import ( MOCK_ORIGIN_LAT_DEG as HOME_LAT_DEG, MOCK_ORIGIN_LON_DEG as HOME_LON_DEG, MOCK_ORIGIN_ALT_M as HOME_ALT_M, diff --git a/tests/unit/test_ap_controller.py b/tests/unit/test_ap_controller.py index 9508cab..a44699f 100644 --- a/tests/unit/test_ap_controller.py +++ b/tests/unit/test_ap_controller.py @@ -7,7 +7,7 @@ from simulation.physics_core import HubObservation -from simulation.pumping_planner import TensionCommand +from groundstation.pumping_planner import TensionCommand from tests.common.mock_ardupilot import MockArdupilot # ── Shared helpers ──────────────────────────────────────────────────────────── diff --git a/tests/unit/test_full_loop_stability.py b/tests/unit/test_full_loop_stability.py index c8f684f..558a08b 100644 --- a/tests/unit/test_full_loop_stability.py +++ b/tests/unit/test_full_loop_stability.py @@ -558,7 +558,7 @@ def _run_elastic_free_flight_with_python_ap( """ from types import SimpleNamespace from arduloop import HeliParams - from simulation.pumping_planner import TensionCommand + from groundstation.pumping_planner import TensionCommand from tests.common.mock_ardupilot import MockArdupilot from tests.simtests.simtest_ic import load_ic from tests.simtests.simtest_runner import PhysicsRunner diff --git a/tests/unit/test_math_lua.py b/tests/unit/test_math_lua.py index c30f2de..6cc8050 100644 --- a/tests/unit/test_math_lua.py +++ b/tests/unit/test_math_lua.py @@ -25,7 +25,7 @@ compute_bz_altitude_hold, ) from simulation.rawes_lua_harness import RawesLua -from simulation.rawes_modes import send_anchor_ned +from groundstation.rawes_modes import send_anchor_ned # ── Module-level harness ────────────────────────────────────────────────────── From a18cc05ded74c497a1041a25ff9f333c21f83090 Mon Sep 17 00:00:00 2001 From: Kristof Date: Sat, 18 Jul 2026 12:14:22 +0200 Subject: [PATCH 2/3] docs: fix stale simulation/gcs.py path reference in mavlink_jsonl_query.py docstring --- analysis/mavlink_jsonl_query.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/analysis/mavlink_jsonl_query.py b/analysis/mavlink_jsonl_query.py index 9721ad8..0dba093 100644 --- a/analysis/mavlink_jsonl_query.py +++ b/analysis/mavlink_jsonl_query.py @@ -2,8 +2,8 @@ """ mavlink_jsonl_query.py -- Standard CLI tool for querying *.mavlink.jsonl logs. -These logs are written by MavlinkLogWriter (simulation/mavlink_log.py) from -both simulation/gcs.py (RawesGCS.start_mavlog) and the bench calibration tool +These logs are written by MavlinkLogWriter (groundstation/mavlink_log.py) from +both groundstation/gcs.py (RawesGCS.start_mavlog) and the bench calibration tool (calibrate/run.py). Every line is one JSON object: {"_t_wall": , "_dir": "rx"|"tx", "mavpackettype": "", ...fields...} From 0839afe8221fd0a4126149ed134ba55d5466271a Mon Sep 17 00:00:00 2001 From: Kristof Date: Sat, 18 Jul 2026 13:41:44 +0200 Subject: [PATCH 3/3] docs: trim AGENTS.md to pointers, move detailed content to owner docs - Remove stale 'Current focus' section. - RAWES_* parameter mapping and DShot config tables were duplicated from design/flight_stack.md and design/dshot.md; replaced with short pointers per AGENTS.md's own 'no duplicated tables' policy. - Moved the long /tmp Git-Bash-vs-WSL gotcha and the long-running-command efficiency guidance out of AGENTS.md into design/sitl_testing.md (their natural owner doc per the Documentation Ownership table), leaving short pointers behind. --- AGENTS.md | 147 +++++++---------------------------------- design/sitl_testing.md | 65 ++++++++++++++++++ 2 files changed, 88 insertions(+), 124 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index c7ca9e2..b815588 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -8,10 +8,6 @@ Detailed design and implementation content lives in `design/*.md` and module-lev RAWES is a tethered, 4-blade autorotating rotor kite (no drive motor on the rotor). Wind drives autorotation; cyclic steers; tether tension during reel-out drives a ground generator. -Current focus: -- Run `bash test.sh stack -n 1 -k test_lua_flight_steady_sitl` to validate the steady flight SITL stack. -- After steady stack passes, validate pumping and landing stack tests. - ## Repository Layout The repo root is a single Python distribution (`pyproject.toml`, name `rawes`) containing @@ -144,94 +140,25 @@ Parameter-reference ownership note: Do not introduce `pitch, roll` ordering unless an external interface explicitly requires it; if so, add an inline comment at that boundary. -## RAWES_* Parameter Mapping - -All Lua configuration uses RAWES_* script-generated params (param:add_table key 77, prefix "RAWES_"). -Canonical parameter list (mirrors `param:add_param` calls in rawes.lua): - -| Param | Default | Purpose | -|---|---|---| -| RAWES_MODE | 0 | Mode selector (0=none,1=steady,3=passive,4=landing) | -| RAWES_YAW_SLP | 0 | Yaw motor slope [RPM/µs], 0=bench default | -| RAWES_KP_ALT | 0.0263 | Altitude P gain [thrust/m] | -| RAWES_KI_ALT | 0.0026 | Altitude I gain [thrust/m·s] | -| RAWES_KD_VZ | 0.105 | Vertical-speed damping [thrust/(m/s)] | -| RAWES_KP_EL | 2.5 | In-plane (elevation) position rate-P [rad/s per m] | -| RAWES_KP_AZ | 0.5 | Crosswind (azimuth) position rate-P [rad/s per m] | -| RAWES_KD_EL | 0.0 | In-plane position rate-D [rad/s per (m/s)] | -| RAWES_CWMAX | 0.6 | Position rate saturation [rad/s] | -| RAWES_SLW | 0.40 | Elevation/body_z slew rate limit [rad/s] | -| RAWES_TEL_HZ | 2.0 | Diagnostic NVF telemetry emission rate [Hz] | -| RAWES_YFF_MAX | 0.7 | Yaw trim clamp upper bound [throttle] | -| RAWES_YFF_TAU | 0.3 | Yaw trim low-pass time constant [s] | -| RAWES_TRP | 2.0 | Tension feedforward ramp time constant [s] | - -Ground→Lua NAMED_VALUE_FLOAT interface (not AP params): - -| NVF key | Purpose | -|---|---| -| RAWES_ALT | Target altitude [m] above anchor | -| RAWES_TEN | Target/feed-forward tether tension [N] | -| RAWES_SUB | Substate (0=hold,1=reel_out,2=transition,3=reel_in,4=transition_back) | -| RAWES_ARM | Optional disarm timer [ms until forced disarm] | -| RAWES_THR | IC thrust [0..1] (passive seed; committed atomically with RIC/PIC) | -| RAWES_RIC | IC roll [rad] | -| RAWES_PIC | IC pitch [rad] | -| RAWES_YIC | Optional fixed yaw target [rad] for MODE_PASSIVE | - -Ground→Lua NAMED_VALUE_INT interface (one-shot anchor location, sent once post-arm): - -| NVI key | Purpose | -|---|---| -| RAWES_LAT | Anchor latitude [deg * 1e7] | -| RAWES_LON | Anchor longitude [deg * 1e7] | -| RAWES_AAL | Anchor altitude [cm, AMSL] | - -Sent as NAMED_VALUE_INT (not FLOAT) for ArduPilot's Location int32 precision -(~1 cm) end-to-end. Lua resolves this to an EKF-local NED anchor offset on -board via `Location:get_vector_from_origin_NEU_m()` (see `_try_resolve_anchor()` -in `rawes.lua`); MODE_STEADY does not initialise altitude hold until all three -ints have arrived AND that resolution has succeeded. Python helper: -`rawes_modes.send_anchor_ned(sim_or_gcs, dn_m, de_m, dd_m)`. - -Set RAWES_MODE per-test; other RAWES_* are in rawes_common_defaults.parm. - -RAWES mode -> vehicle control API (current `rawes.lua` behavior): - - -| RAWES_MODE | Mode | Vehicle API used | -|---|---|---| -| 0 | none | none | -| 3 | passive | `vehicle:set_target_rate_and_throttle` for thrust-only seed (`RAWES_THR` without `RAWES_RIC/PIC`); `vehicle:set_target_angle_and_rate_and_throttle` once full IC seed is present | -| 1 | steady | `vehicle:set_target_angle_and_rate_and_throttle` | +## RAWES_* Parameter Reference -Steady command basis (`RAWES_MODE=1`): -- roll/pitch: derived from `bz_altitude_hold(rel, el_rad, tension_n, az_ref)` and converted by `bz_ned_to_roll_pitch(...)`. -- throttle: altitude PID around IC thrust (`RAWES_THR` as trim/seed), with vertical-speed damping and thrust slew limiting before sending to ArduPilot. +The full RAWES_* script-generated parameter table, the NAMED_VALUE_FLOAT/INT +wire interface, and the RAWES_MODE → vehicle-API mapping are owned by +`design/flight_stack.md` (see its "RAWES_\* script-generated parameters" table +and §4.2b–4.5) — do not duplicate those tables here. Live defaults for +non-RAWES_* AP params are in `tests/sitl/rawes_common_defaults.parm`; set +`RAWES_MODE` per-test. For signs, frame details, EKF gating, and mixer conventions, read the primary docs in the ownership table. ## DShot Setup (Agent Critical) -Use this as the quick contract-level reference for the yaw-motor ESC path. -Canonical long-form owner doc is `design/dshot.md`. - -Active hardware-default parameters (from `tests/sitl/rawes_common_defaults.parm`): -- `SERVO9_FUNCTION=36` (Motor4 on output 9), `SERVO9_MIN=1000`, `SERVO9_MAX=2000`, `SERVO9_TRIM=1000` -- `SERVO_BLH_MASK=256` (output 9) -- `SERVO_BLH_BDMASK=256` (bidirectional DShot on output 9) -- `SERVO_BLH_AUTO=0` (manual masks) -- `SERVO_BLH_OTYPE=5` (DShot300) -- `SERVO_BLH_POLES=22` -- `SERVO_DSHOT_ESC=1` (AM32 telemetry decode) -- `SERVO_DSHOT_RATE=0` -- `BRD_IO_DSHOT=0` (FMU output path) -- `RPM1_TYPE=5`, `RPM1_ESC_MASK=256` (ESC telemetry routed from output 9) - -SITL behavior: -- These BLHeli/DShot params are intentionally excluded from SITL boot verification - (`tests/sitl/stack_utils.py` -> `SITL_UNSUPPORTED_PARAMS`) because - ArduCopter-heli SITL does not compile the BLHeli backend and drives output 9 as PWM. +Canonical owner doc: `design/dshot.md` (full parameter tables, wiring, RPM +conversion). One SITL-specific gotcha not covered there: BLHeli/DShot params +(`SERVO9_*`, `SERVO_BLH_*`, `SERVO_DSHOT_*`, `RPM1_*`) are intentionally +excluded from SITL boot verification (`tests/sitl/stack_utils.py` -> +`SITL_UNSUPPORTED_PARAMS`) because ArduCopter-heli SITL does not compile the +BLHeli backend and drives output 9 as plain PWM — see `design/sitl_testing.md`. ## Workflow Rules @@ -262,23 +189,11 @@ SITL behavior: exist only to work around circular imports are a sign of bad architecture — fix the circular dependency by refactoring (e.g. extract a shared module, invert the dependency) rather than papering over it with a local import. -- `/tmp` is NOT one shared filesystem on this box — Git Bash (mingw/MSYS2, the default terminal) - and WSL2 (used for `docker`/`test.sh stack`) each have their own separate `/tmp`: - - A bare `> /tmp/foo.log` redirection in a Git Bash command writes to Git Bash's own `/tmp` - (really `C:\Users\\AppData\Local\Temp\foo.log` — check with `cygpath -w /tmp/foo.log`). - - A command run via `wsl -e bash -lc "... > /tmp/foo.log"` writes inside WSL's `/tmp` - (`\\wsl$\...\tmp\foo.log` from Windows, or `/tmp/foo.log` from *inside* another `wsl -e` call) - — NOT reachable from a later plain Git Bash `cat /tmp/foo.log`. - - If the redirection is written OUTSIDE the `wsl -e bash -lc "..."` quoted string (e.g. - `wsl -e bash -lc "cmd" > /tmp/foo.log`), it's the OUTER (Git Bash) shell that owns the - redirect, not WSL — easy to mix up. - - A native Windows executable (e.g. `.venv/Scripts/python.exe`) invoked from Git Bash does NOT - understand `/tmp/...` paths passed as arguments (it's a Windows process, not MSYS2-aware) — - convert with `cygpath -w /tmp/foo.log` first, or it'll fail with `FileNotFoundError` even - though `ls /tmp/foo.log` (from Git Bash) shows the file existing. - - Rule of thumb: know which shell environment (Git Bash vs WSL) is actually creating/reading - a `/tmp` path before assuming a file exists or is missing; don't conclude "no output was - produced" just because a naive `cat`/`find` from the wrong shell doesn't see it. +- `/tmp` is NOT one shared filesystem on this box — Git Bash and WSL2 (used for + `docker`/`test.sh stack`) each have their own separate `/tmp`, and native Windows + executables invoked from Git Bash can't resolve `/tmp/...` paths at all. Full + gotcha writeup (redirection ownership, `cygpath -w` conversion, diagnosis tips) + is in `design/sitl_testing.md`. ## Test Entry Points @@ -315,27 +230,11 @@ the tail of each failure, including the assertion error). The full output is als saved — read it directly if needed: `Get-Content simulation/logs//worker.log | Select-String "ERROR|CRITICAL|Traceback|assert|FAIL" | Select-Object -First 30` -Long-running test commands (agent-critical, efficiency): -- Unit and simtest runs can take 1-5+ minutes. Do NOT pipe them through `| tail -N` when - running in a mode that may background the command — a slow/idle command piped through - `tail` produces no output until the pipeline's stdout closes, so a background poll via - `get_terminal_output` just returns the same stale snapshot every time (wastes calls, - looks like a hang). Run the bare command first; only pipe through `tail`/`grep` after - confirming the run is fast enough to complete synchronously, or redirect to a file - (`... > /tmp/out.log 2>&1`) and read/grep the file instead. -- Once a command has been moved to background, do not repeatedly call `get_terminal_output` - in a tight loop — it will not return new content until the process actually produces more - output or exits. Wait for the automatic completion notification instead of polling. -- Do NOT call `get_terminal_output` immediately after a command moves to background "just to - check progress". It returns a byte-limited tail of the WHOLE terminal scrollback, not just - the new command's output — if the new command has only printed a little so far, the tail - can still be dominated by leftover output from earlier unrelated commands in the same - terminal, which looks like stale/wrong output but is really just "not enough new output yet - to push the old stuff out of the tail window". End the turn and wait for the automatic - completion notification instead; only poll if genuinely unsure whether the process is hung - after a long silence. -- Prefer `bash test.sh stack -n 1 -k ` (single test) while iterating; only widen - to `-n 4`/full suite once the targeted test is confirmed passing, to keep turnaround short. +Long-running test commands (agent-critical, efficiency): unit/simtest/stack runs can +take 1-5+ minutes. Do not pipe a possibly-backgrounded command through `tail`/`grep`, +and do not poll `get_terminal_output` in a tight loop — full guidance (why piping +hides output, why polling wastes calls, and the preferred iterate-in-isolation +pattern) is in `design/sitl_testing.md`. ## Visualization diff --git a/design/sitl_testing.md b/design/sitl_testing.md index 9ebc78b..102d86a 100644 --- a/design/sitl_testing.md +++ b/design/sitl_testing.md @@ -51,6 +51,71 @@ runs in its own fresh Docker container, one per test file. --- +## Efficient long-running command handling (agent-critical) + +Unit/simtest/stack runs can take 1-5+ minutes. Handle them like this: + +- Do NOT pipe a command through `| tail -N` if it might run in a mode that can + background — a slow/idle command piped through `tail` produces no output until + the pipeline's stdout closes, so a background poll via `get_terminal_output` + just returns the same stale snapshot every time (wastes calls, looks like a + hang). Run the bare command first; only pipe through `tail`/`grep` once you've + confirmed the run completes synchronously, or redirect to a file + (`... > /tmp/out.log 2>&1`) and read/grep the file instead. +- Once a command has moved to background, do not repeatedly call + `get_terminal_output` in a tight loop — it will not return new content until + the process actually produces more output or exits. Wait for the automatic + completion notification instead of polling. +- Do NOT call `get_terminal_output` immediately after a command backgrounds + "just to check progress". It returns a byte-limited tail of the WHOLE + terminal scrollback, not just the new command's output — if the new command + has only printed a little so far, the tail can still be dominated by + leftover output from earlier unrelated commands in the same terminal, which + looks like stale/wrong output but really just means "not enough new output + yet to push the old stuff out of the tail window". End the turn and wait for + the automatic completion notification instead; only poll if genuinely unsure + whether the process is hung after a long silence. +- Prefer `bash test.sh stack -n 1 -k ` (single test) while iterating; + only widen to `-n 4`/full suite once the targeted test is confirmed passing, + to keep turnaround short. + +## Git Bash vs WSL `/tmp` gotcha + +`/tmp` is NOT one shared filesystem on Windows dev boxes — Git Bash (mingw/MSYS2, +the default terminal) and WSL2 (used for `docker`/`test.sh stack`) each have their +own separate `/tmp`: + +- A bare `> /tmp/foo.log` redirection in a Git Bash command writes to Git Bash's + own `/tmp` (really `C:\Users\\AppData\Local\Temp\foo.log` — check with + `cygpath -w /tmp/foo.log`). +- A command run via `wsl -e bash -lc "... > /tmp/foo.log"` writes inside WSL's + `/tmp` (`\\wsl$\...\tmp\foo.log` from Windows, or `/tmp/foo.log` from *inside* + another `wsl -e` call) — NOT reachable from a later plain Git Bash + `cat /tmp/foo.log`. +- If the redirection is written OUTSIDE the `wsl -e bash -lc "..."` quoted + string (e.g. `wsl -e bash -lc "cmd" > /tmp/foo.log`), it's the OUTER (Git + Bash) shell that owns the redirect, not WSL — easy to mix up. +- A native Windows executable (e.g. `.venv/Scripts/python.exe`) invoked from + Git Bash does NOT understand `/tmp/...` paths passed as arguments (it's a + Windows process, not MSYS2-aware) — convert with `cygpath -w /tmp/foo.log` + first, or it'll fail with `FileNotFoundError` even though + `ls /tmp/foo.log` (from Git Bash) shows the file existing. +- Rule of thumb: know which shell environment (Git Bash vs WSL) is actually + creating/reading a `/tmp` path before assuming a file exists or is missing; + don't conclude "no output was produced" just because a naive `cat`/`find` + from the wrong shell doesn't see it. + +## DShot/BLHeli params excluded from SITL boot verification + +BLHeli/DShot params (`SERVO9_*`, `SERVO_BLH_*`, `SERVO_DSHOT_*`, `RPM1_*` — +full table in [design/dshot.md](dshot.md)) are intentionally excluded from +SITL boot-param verification via `SITL_UNSUPPORTED_PARAMS` in +[tests/sitl/stack_utils.py](../tests/sitl/stack_utils.py), because +ArduCopter-heli SITL does not compile the BLHeli backend and drives output 9 +as plain PWM instead of DShot. + +--- + ## Post-run diagnosis workflow **ALWAYS run `diagnose_sitl.py` FIRST after ANY SITL stack run, before making any