From e9ccf236518ca28b7bad46d56972cb65267a1564 Mon Sep 17 00:00:00 2001 From: Ilya Tikhonyuk Date: Wed, 15 Jul 2026 15:19:03 +0300 Subject: [PATCH 1/2] Add testing reference to wb-dev MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit New references/testing.md: how tests are organized in WB Python services — tests/ layout with captured fixture data and golden files, pytest black-box style, WB fixture patterns (fake sysroot in tmp_path, factory mocks, autouse state reset, boundary patches), and the three run levels (local pytest, pybuild during .deb build, Jenkins coverage flags). Distilled from wb-cloud-agent, wb-nm-helper, wb-diag-collect and wb-ai-skills; the service template itself ships no tests. SKILL.md gets a Testing section pointing at the reference. Co-Authored-By: Claude Fable 5 --- .claude-plugin/plugin.json | 2 +- debian/changelog | 14 ++++ pyproject.toml | 2 +- wb-plc/skills/wb-dev/SKILL.md | 6 ++ wb-plc/skills/wb-dev/references/testing.md | 86 ++++++++++++++++++++++ wb_cli/__init__.py | 2 +- 6 files changed, 109 insertions(+), 3 deletions(-) create mode 100644 wb-plc/skills/wb-dev/references/testing.md diff --git a/.claude-plugin/plugin.json b/.claude-plugin/plugin.json index c0b7e50..805b867 100644 --- a/.claude-plugin/plugin.json +++ b/.claude-plugin/plugin.json @@ -1,7 +1,7 @@ { "name": "wb-plc", "description": "Wiren Board controller skills — operate, configure, troubleshoot WB hardware via SSH and wb-cli.", - "version": "1.11.1", + "version": "1.13.0", "author": { "name": "Wiren Board", "url": "https://wirenboard.com" diff --git a/debian/changelog b/debian/changelog index 0f45da8..517808f 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,3 +1,17 @@ +wb-cli (1.8.14) stable; urgency=medium + + * wb-plc plugin bumped to 1.13.0. wb-dev: new references/testing.md — + how tests are organized in WB Python services (tests/ layout with + golden files in tests/data, pytest black-box style, WB fixture + patterns: fake sysroot, factory mocks, autouse state reset; run + levels: local pytest, pybuild during .deb build, Jenkins coverage + flags). Patterns distilled from wb-cloud-agent, wb-nm-helper, + wb-diag-collect, wb-ai-skills. SKILL.md gets a Testing section + pointing at it. Version assumes the open 1.8.11-1.8.13 PRs merge + first. + + -- Ilya Tikhonyuk Tue, 15 Jul 2026 14:00:00 +0300 + wb-cli (1.8.10) stable; urgency=medium * wb-plc/wb-zigbee: fix converter-detection probe — unescape pipe in diff --git a/pyproject.toml b/pyproject.toml index c0a24fe..c4ebc8b 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta" [project] name = "wb-cli" -version = "1.8.10" +version = "1.8.14" description = "Command-line interface to a Wiren Board controller" requires-python = ">=3.9" authors = [{name = "Wiren Board Team", email = "info@wirenboard.com"}] diff --git a/wb-plc/skills/wb-dev/SKILL.md b/wb-plc/skills/wb-dev/SKILL.md index 71cccb8..0e220e8 100644 --- a/wb-plc/skills/wb-dev/SKILL.md +++ b/wb-plc/skills/wb-dev/SKILL.md @@ -122,6 +122,12 @@ Canonical configs (clang-format, pyproject.toml, find-python-files) live in /controls/` topic space; the web UI, wb-rules, and other services consume that namespace. All `/meta` topics are retained; `/meta/error` doubles as the LWT. Specific typed controls (`temperature`, `voltage`) are deprecated — use `type: value` + `units`. diff --git a/wb-plc/skills/wb-dev/references/testing.md b/wb-plc/skills/wb-dev/references/testing.md new file mode 100644 index 0000000..525a67a --- /dev/null +++ b/wb-plc/skills/wb-dev/references/testing.md @@ -0,0 +1,86 @@ +# Testing WB Python services + +How tests are organized in Wiren Board Python projects. The service template +(`wb-python-service-template`) does not ship tests — the patterns below are distilled from +production repos: `wb-cloud-agent`, `wb-nm-helper`, `wb-diag-collect`, `wb-ai-skills`. +Process rules (what must be tested before a PR) live in the `wb-git`/`wb-development` skills; +this is the engineering how-to. + +## Layout + +``` +tests/ +├── __init__.py ← required: pybuild discovers the suite by package +├── conftest.py ← shared fixtures for the whole suite +├── data/ ← captured real-world inputs + expected outputs +│ ├── wb-mqtt-serial.conf +│ └── wb-mqtt-serial.conf.filtered ← golden file: expected result next to input +└── test_.py ← one file per production module +``` + +- `tests/` sits at the repo root, next to the production package. +- One `test_.py` per production module (`wb-nm-helper`: `test_connection_checker.py`, + `test_dns_resolver.py`, ...). Plain pytest functions, no test classes. +- `tests/data/` holds **captured artefacts from real controllers** (configs, RPC responses, + `/proc` contents) — not hand-invented minimal samples. Expected outputs live next to inputs + as golden files (`wb-diag-collect`: `20bridge.conf` → `20bridge.conf.filtered`). + +## Test style + +Black-box through the public API: call the function, assert the returned value or the raised +exception — not internal call sequences (same principle as `fw-unittests` for firmware). + +```python +def test_do_curl_success_response(settings, mock_subprocess): + mock_subprocess(status.OK, '{"result": "success"}') + data, code = do_curl(settings) + assert data == {"result": "success"} + assert code == 200 + +def test_do_curl_invalid_method(settings): + with pytest.raises(ValueError, match="Invalid method"): + do_curl(settings, method="invalid") +``` + +## Fixtures — the WB patterns + +Everything at the process boundary is mocked; tests never touch a real broker, network, +hardware, or systemd. + +| Pattern | What it solves | Example | +|---|---|---| +| **Factory fixture** | One fixture builds parameterized mocks | `shell_returning(rc, stdout, stderr)` returns a configured `MagicMock` (`wb-ai-skills`) | +| **Fake sysroot in `tmp_path`** | Code reads controller paths (`/etc/wb-release`, `/proc/uptime`) | fixture copies captured files from `tests/data/` into `tmp_path` mirroring the real layout, code gets the root injected (`wb-ai-skills` `controller_root`) | +| **`autouse` state reset** | Module-level state leaks between tests | clear module dicts/threads before and after each test (`wb-cloud-agent` `_clear_metrics_monitor_state`) | +| **`patch` at the boundary** | subprocess / curl / MQTT client / file paths | `patch("...METRICS_COLLECTOR_TEMPLATE_PATH", str(tmp_file))`, `patch` on `subprocess.run` (`wb-cloud-agent`) | +| **`monkeypatch` for CLI args** | Testing `main()` entry points | `monkeypatch.setattr(sys, "argv", [...])` (`wb-cloud-agent` `set_argv`) | + +Shared settings objects come from a `settings` fixture constructing the service's real config +class with test values — not a dict imitation. + +## Running + +| Where | How | +|---|---| +| Locally (devenv/venv) | `pytest`; with coverage — commands from `codestyle/python.ru.md` (`pytest --cov --cov-config=.coveragerc --cov-fail-under=`) | +| During .deb build | pybuild runs the suite automatically (`tests/` is a package); for coverage inside `wbdev ndeb`/`cdeb`: `export WBDEV_PYBUILD_TEST_ARGS="--cov ... --cov-fail-under="` | +| CI (Jenkins) | flags in the one-line Jenkinsfile, e.g. `buildDebSbuild defaultRunPythonChecks: true, defaultAngryPylint: true, defaultRunCoverage: true, defaultCoverageMin: "59", defaultDoCoverallsReporting: true` | + +- `defaultCoverageMin` is per-repo (`wb-cloud-agent`: 59, `wb-nm-helper`: 80) — when adding + tests to an existing repo, don't lower it; when bootstrapping, set a reachable floor and + raise it as coverage grows. +- Every Python repo carries `tests/test_version.py` (from `package-bootstrap`) locking + `debian/changelog` ↔ `pyproject.toml` ↔ `__init__.py` — don't remove it, it fails the build + on version drift. + +## What NOT to do + +- **No real I/O in unit tests** — no live MQTT broker, no network, no SSH to a controller. + Integration tests against real hardware are a separate story (custom Jenkinsfile, see + `package-bootstrap` "When to ask the user"). +- **Don't edit a failing test to make it pass** — surface it; the user decides whether the + test or the production code is wrong (per `wb-development` coder rules). +- **Don't hand-invent fixture data** when a real controller is available — capture the real + config/output into `tests/data/` and cut it down. +- **Don't skip the docstring** on a non-trivial test (multi-step setup, async) — reviewers + flag it (`code-review-orchestrator` project rules). diff --git a/wb_cli/__init__.py b/wb_cli/__init__.py index d3d41ce..d6578c9 100644 --- a/wb_cli/__init__.py +++ b/wb_cli/__init__.py @@ -1 +1 @@ -__version__ = "1.8.10" +__version__ = "1.8.14" From 29c2562ce1931ff04040b438b75d91f2da977dab Mon Sep 17 00:00:00 2001 From: Ilya Tikhonyuk Date: Wed, 15 Jul 2026 15:37:54 +0300 Subject: [PATCH 2/2] Clarify that tests are added when needed, not unconditionally The service template intentionally ships without tests; the guide applies when the need arises, its absence is not a per-project gap. Co-Authored-By: Claude Fable 5 --- wb-plc/skills/wb-dev/SKILL.md | 2 ++ wb-plc/skills/wb-dev/references/testing.md | 12 +++++++----- 2 files changed, 9 insertions(+), 5 deletions(-) diff --git a/wb-plc/skills/wb-dev/SKILL.md b/wb-plc/skills/wb-dev/SKILL.md index 0e220e8..2108b56 100644 --- a/wb-plc/skills/wb-dev/SKILL.md +++ b/wb-plc/skills/wb-dev/SKILL.md @@ -124,6 +124,8 @@ For naming rules, exact CLI invocations and config paths — see **`references/c ## Testing +Tests are added **when the service has logic worth protecting** — they are not a mandatory part of every project, and the service template intentionally ships without them. When the need arises, follow **`references/testing.md`**. + Tests are pytest, live in `tests/` at the repo root (with `__init__.py` — pybuild discovers the suite by package), and mock everything at the process boundary: no live MQTT broker, network, or hardware in unit tests. Fixture data is captured from real controllers into `tests/data/`, not hand-invented. The suite runs at three levels: locally (`pytest`), by pybuild during the .deb build, and in CI via the Jenkinsfile coverage flags (`defaultRunCoverage`, `defaultCoverageMin`). For the layout, WB fixture patterns (fake sysroot in `tmp_path`, factory mocks, autouse state reset, golden files), run commands and coverage thresholds — see **`references/testing.md`**. diff --git a/wb-plc/skills/wb-dev/references/testing.md b/wb-plc/skills/wb-dev/references/testing.md index 525a67a..dd68bba 100644 --- a/wb-plc/skills/wb-dev/references/testing.md +++ b/wb-plc/skills/wb-dev/references/testing.md @@ -1,10 +1,12 @@ # Testing WB Python services -How tests are organized in Wiren Board Python projects. The service template -(`wb-python-service-template`) does not ship tests — the patterns below are distilled from -production repos: `wb-cloud-agent`, `wb-nm-helper`, `wb-diag-collect`, `wb-ai-skills`. -Process rules (what must be tested before a PR) live in the `wb-git`/`wb-development` skills; -this is the engineering how-to. +How tests are organized in Wiren Board Python projects — **when they are needed**. Tests are +added as the need arises (non-trivial logic, parsing, regressions worth locking), not +unconditionally: the service template (`wb-python-service-template`) intentionally ships +without them, and their absence is not a gap to "fix" on every project. When you do add +tests, follow the patterns below — distilled from production repos: `wb-cloud-agent`, +`wb-nm-helper`, `wb-diag-collect`, `wb-ai-skills`. Process rules (what must be tested before +a PR) live in the `wb-git`/`wb-development` skills; this is the engineering how-to. ## Layout