diff --git a/CHANGELOG.md b/CHANGELOG.md index f20113d..6ba77c1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,7 +8,7 @@ Every tag has its own section, release candidates included, and the release work ## [Unreleased] -Nothing has been released yet. This section records what `main` holds after the September 2026 stack (#13, #16 to #21, #23), compared with the May 2026 tree (`aa7a5a8`) that anyone following `main` before it had. +Nothing has been released yet. This section records what `main` holds after the pull requests merged in September 2026 (#13 to #68), compared with the May 2026 tree (`aa7a5a8`) that anyone following `main` before them had. ### Breaking @@ -38,6 +38,18 @@ Nothing has been released yet. This section records what `main` holds after the - New checks built in CI: the NixOS VM tests `vmTest-0_6`, `vmTestWithHapp-0_6` (#16), `vmTestConductorMetrics-0_6`, `vmTestWindtunnel` (#17) and `vmTestGateway` (#18), and `conductorMetricsJq` (#21). - The example fleet runs Holochain 0.6.3 with hREA happ-0.4.0-beta, Kando v0.17.5 and Requests & Offers v0.5.2, fetched by hash. The hardware stubs of the example and of the `fleet` template carry the HoloPort disk layout (GPT with a `bios_grub` partition and an ESP, GRUB for both firmwares). (#19, #21) - `CONTRIBUTING.md`, with the VM-test and option-reference rules. (#18) +- `services.holochain-edgenode.binaryCache.enable`, on by default: a host importing the module gets the Holochain Foundation cache (`holochain-ci.cachix.org`) and its key in `nix.settings`, so `holochain` and `hc` download prebuilt after the first switch. (#34) +- Per-DHT series from `dump-network-metrics --include-dht-summary`: `holochain_dht_*`, one set per cell, labelled `conductor`, `app_id`, `role` and `dna`, and named for dashboards by `holochain_dht_info` and `holochain_app_info`, from the new `happs..displayName` and `roleNames`. (#47, #63) +- `packages..holochain-conductor-exporter` and `holochain-conductor-exporter-0_6`, the one program that writes `holochain_*` series for any conductor on a machine. Every series carries a `conductor` label, from the new `conductorMetrics.name` (default `"Holochain"`). (#63) +- `holochain-grafana`: recording rules that compute every state once (`modules/holochain-rules.nix`), with their thresholds in `states`; `scrapeTargets` as an attribute set keyed by node name, with an optional `site` (a list still works); `overviewUnits` with the name a person reads for each unit; and `room`. (#35, #64) +- Five dashboards, each titled with its reader's question and all tagged `holochain`: `holochain-home` ("What is this machine running?", Grafana's home page, opening on the machine Grafana runs on), `holochain-now`, `holochain-fleet` (now "Which Holochain node needs attention?"), `holochain-node` and `holochain-network`. (#35, #65, #67) +- `services.holochain-services`: every module lists the units it installs, by name and with the version it runs, and the dashboards show each one's state; a timer reads the bootstrap server's `/health`. (#66, #67) +- `nixosModules.holochain-bootstrap`, the Kitsune2 bootstrap and relay server as a service, with `packages..bootstrap-srv` and `bootstrap-srv-0_6`. The edgenode gains `relayAllowPlainText`, `requestTimeoutS`, `dbSyncLevel` and `wasmBackend`. (#62) +- `nixosModules.sensorica-event-node`, the Sensorica workshop profile: Holochain 0.6.3 with hREA, Kando and Requests & Offers on one network seed. Not part of `nixosModules.default`. (#59) +- `packages.x86_64-linux.wdocker-0_15`, Moss `wdocker` from tag v0.15.8 with the Holochain 0.6.1 it pins (#60), and `nixosModules.holochain-moss-node`, a Moss group's always-online node as a service, with its readings and its Grafana page (#67). Not part of `nixosModules.default`. +- `packages.x86_64-linux.holoport-install`, which erases one disk, lays it out as ADR-017 says and installs a system that boots on legacy BIOS and on UEFI. (#61) +- More checks built in CI: the VM tests `vmTestWdocker` (#60), `vmTestHoloportInstall` (#61), `vmTestBootstrap` (#62), `vmTestServices` and `vmTestServices-noBootstrap` (#66) and `vmTestMossNode` (#67); without a VM, `edgenodeBinaryCache` (#34), `dhtMetricsJq` (#47), `edgenodeConfigRender` (#62), `metricsHelpAgreement`, `metricsNameShape` and `edgenodeNamesWiring` (#63), `holochainRules` and `grafanaProvisioning` (#64), `dashboardLabels`, `dashboardWords` and `dashboardQueries` (#65), `moss-dashboard` and `moss-names` (#67). +- The MIT `LICENSE` (#36); `SECURITY.md`, issue forms and a pull request template (#48); this changelog and a tag-driven release workflow (#49); the design record in `docs/adr/` (#51); and the documentation book, built from `docs/` and published at (#67). ### Changed @@ -47,11 +59,15 @@ Nothing has been released yet. This section records what `main` holds after the - The boot loader moved out of the placeholder `hardware-configuration.nix` files into `configuration.nix` and `hosts/common.nix`, so replacing a stub with `nixos-generate-config` output keeps it. `#minimal` targets a stock UEFI install with systemd-boot. (#21) - The fleet template and the example put one `operatorKeys` list on the operator account and on root, so Colmena can log in. (#21) - The example fleet's lock follows `main` after the stack instead of the May tree. (#23) +- The `holochain_conductor_*_total` counters are running totals across closed connections instead of sums over the connections open at that moment, and the metrics timer also calls `list-apps` and `dump-network-metrics`. (#35, #47) +- `nixosModules.default` also imports `holochain-bootstrap`, which is off until enabled. (#62) +- The example fleet's hosts are `sensorica-holoport-01` to `sensorica-holoport-05`, take their Holochain line and hApps from `sensorica-event-node`, and gain an operator desk, per-host switches and the Moss node on `sensorica-holoport-01`. Its `nixos-holochain` input follows `main`. (#59, #67, #68) ### Fixed - The hApp installer treats a failed `list-apps` while the conductor compiles wasm as "not yet" instead of ending the unit, and no fixed start timeout caps `installerTimeout`. (#16, #21) - The metrics jq sums `blocked_message_counts` at any depth; it used to write a JSON object into the textfile, which made node_exporter drop every `holochain_*` series. (#21) - The gateway's `--address` is shell-escaped. (#21) +- The metrics timer no longer hands the whole `list-apps` reply to jq as one argument, which Linux caps at 128 KiB: a large reply stopped the script before it wrote anything. (#47) [Unreleased]: https://github.com/Sensorica/nixos-holochain/commits/main diff --git a/README.md b/README.md index b726155..d670deb 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ > A declarative substrate for running Holochain edgenodes, hApps, and developer environments. Built at Sensorica, intended for the Holochain community. -**Status:** the modules work and are VM-tested. A conductor and its hApps come up at boot on both supported Holochain lines (0.7.0 and 0.6.3), a fleet's traffic is on a provisioned Grafana dashboard, and an HTTP gateway serves zome reads over HTTP. Ten NixOS VM tests run in CI. What is still open is hardware: the five-machine fleet has not been deployed to real Holoports yet (issues [#8](https://github.com/Sensorica/nixos-holochain/issues/8) to [#12](https://github.com/Sensorica/nixos-holochain/issues/12)). +**Status:** the modules work and are VM-tested. A conductor and its hApps come up at boot on both supported Holochain lines (0.7.0 and 0.6.3), a fleet's traffic is on a provisioned Grafana dashboard, and an HTTP gateway serves zome reads over HTTP. Fourteen NixOS VM tests run in CI. What is still open is hardware: one Holoport, `sensorica-holoport-01`, was installed from the [runbook](docs/deployment.md#installing-on-a-holoport-legacy-bios) on 2026-09-27, and the five-machine fleet is not deployed yet (issues [#8](https://github.com/Sensorica/nixos-holochain/issues/8) to [#12](https://github.com/Sensorica/nixos-holochain/issues/12)). **License:** [MIT](LICENSE), the license of nixpkgs, so any module here can be reused in other flakes or proposed upstream to nixpkgs as it is. The hApps these modules run keep their own licenses (Holochain itself and Moss are CAL-1.0, hREA is Apache-2.0). **Origin:** Successor to the archived [Sensorica/holoports-workshop](https://github.com/Sensorica/holoports-workshop), pivoting from HolOS appliance-image deployment to vanilla NixOS authorship. **Documentation:** the book at [sensorica.github.io/nixos-holochain](https://sensorica.github.io/nixos-holochain/), built from `docs/` with mdBook. @@ -97,44 +97,32 @@ See [`docs/deployment.md`](docs/deployment.md) for the deployment guide, [`docs/ ``` nixos-holochain/ -├── flake.nix # Entry point: inputs, modules, templates, packages, VM checks -├── modules/ -│ ├── holochain-edgenode.nix # Core: conductor + lair + hApp installer + metrics -│ ├── conductor-metrics.jq # dump-network-stats → Prometheus text -│ ├── conductor-counters.jq # running byte and message totals across closed connections -│ ├── dht-metrics.jq # list-apps + dump-network-metrics → per-DHT Prometheus text -│ ├── holochain-grafana.nix # Prometheus + Grafana for a fleet -│ ├── dashboards/ # Provisioned Grafana dashboards -│ ├── holochain-windtunnel.nix # Opt-in: donate the machine to the Foundation's Nomad cluster -│ ├── holochain-http-gateway.nix # HTTP gateway in front of the conductor -│ └── default.nix # Module aggregator -├── packages/ -│ ├── holochain-http-gateway.nix # hc-http-gw build, one release per Holochain line -│ └── wdocker.nix # Moss always-online node, with the Holochain it expects +├── flake.nix # Entry point: inputs, modules, templates, packages, checks +├── modules/ # The NixOS modules, with the exporters, recording rules and dashboards they ship +├── packages/ # hc-http-gw per Holochain line, the conductor exporter, Moss wdocker ├── templates/ │ ├── minimal/ # nix flake init -t …#minimal: one edgenode │ └── fleet/ # nix flake init -t …#fleet: five nodes, Grafana, live ISO ├── examples/ -│ └── sensorica-fleet/ # The Sensorica Lab fleet: its own flake, five hosts, ISO, colmena hive -│ ├── flake.nix -│ ├── hosts/common.nix # shared host config, operator SSH keys -│ ├── hosts/sensorica-holoport-01..05/ # configuration.nix + hardware-configuration.nix per machine -│ ├── hosts/workshop-iso/ # Live ISO for participants -│ └── README.md +│ └── sensorica-fleet/ # The Sensorica Lab fleet: its own flake, five Holoports, ISO, Colmena hive (layout in its README) +├── tests/ # Fixtures and the checks that need no VM ├── happs/ # .happ bundles (not committed, see happs/README.md) ├── secrets/ # private material only, gitignored except *.example ├── CHANGELOG.md # One section per release, published as its release note ├── scripts/ -│ └── changelog-section.sh # Prints one version's CHANGELOG section +│ ├── changelog-section.sh # Prints one version's CHANGELOG section +│ └── holoport-install.sh # Erases one disk and installs a system that boots on a Holoport ├── workshop/ │ ├── facilitator-guide.md │ ├── participant-handout.md │ └── preflight-checklist.md +├── book.toml # The documentation book, built from docs/ with mdBook └── docs/ - ├── architecture.md + ├── introduction.md # The book's first page; SUMMARY.md is its table of contents + ├── architecture.md # How the pieces fit, with every file under modules/ and packages/ ├── module-options.md # generated by `nix build .#options-doc` ├── deployment.md - ├── moss-node.md # running the packaged Moss node by hand + ├── moss-node.md # the Moss always-online node, as a service and by hand ├── releasing.md # How a maintainer cuts a release candidate and a release ├── adr/ # architecture decision records, one per file ├── images/ # dashboard screenshots @@ -152,6 +140,8 @@ nixos-holochain/ | `holochain-http-gateway` | `hc-http-gw` in front of the conductor, exposing named zome functions over HTTP. Nothing is exposed by default. | | `holochain-windtunnel` | Opt-in, off by default: joins the machine to the Holochain Foundation's Nomad cluster to run their Wind Tunnel scenarios. | | `holochain-bootstrap` | The Kitsune2 bootstrap and relay server on your own machine, so a fleet finds itself without the Foundation's test server or the internet. See [Running your own bootstrap and relay](docs/deployment.md#running-your-own-bootstrap-and-relay). | +| `holochain-moss-node` | A Moss group's always-online node as a service, beside the edgenode, with its readings and its own Grafana page. Not part of `nixosModules.default`, because it runs a second conductor. See [`docs/moss-node.md`](docs/moss-node.md). | +| `sensorica-event-node` | The Sensorica workshop's profile layered on `holochain-edgenode`: Holochain 0.6.3, hREA, Kando and Requests & Offers on one network seed. Not part of `nixosModules.default`. See [`examples/sensorica-fleet/README.md`](examples/sensorica-fleet/README.md#holochain-line-and-happs). | Key options for `services.holochain-edgenode`: @@ -168,13 +158,13 @@ Key options for `services.holochain-edgenode`: | `conductorMetrics.enable` | `false` | The conductor's own `holochain_*` series | | `openFirewall` | `false` | Open firewall ports | -The full reference for all five modules is [`docs/module-options.md`](docs/module-options.md), generated from the declarations by `nix build .#options-doc`. +The full reference for `holochain-edgenode`, `holochain-grafana`, `holochain-http-gateway`, `holochain-windtunnel`, `holochain-bootstrap` and `holochain-services` (the list of services every module feeds to the dashboards) is [`docs/module-options.md`](docs/module-options.md), generated from the declarations by `nix build .#options-doc`. The Moss node's options are in [`docs/moss-node.md`](docs/moss-node.md#as-a-nixos-service); `sensorica-event-node` declares none. --- ## Tests -Ten NixOS VM tests and a conductor config check, all built in CI: +Fourteen NixOS VM tests, all built in CI: thirteen in the `nix flake check` job, and `vmTestHoloportInstall` in a job of its own because it copies a whole system onto a virtual disk ([`.github/workflows/ci.yml`](.github/workflows/ci.yml)). | Check | What it proves | |---|---| @@ -182,11 +172,15 @@ Ten NixOS VM tests and a conductor config check, all built in CI: | `vmTestWithHapp` / `vmTestWithHapp-0_6` | A hApp installs once, stays enabled, and survives a cold boot on both lines, and every one of its cells has its `holochain_dht_*` series on `/metrics` | | `vmTestConductorMetrics-0_6` | The conductor's gauges appear on `/metrics` on the 0.6 line | | `vmTestGrafana` | Conductor and per-DHT series reach Prometheus, the five dashboards are provisioned with their data source and "What is this machine running?" is Grafana's home page, opening on this machine, every service its modules list carries its version, every panel query answers through Grafana's own query API (three are only required not to error: the two temperature panels, since a VM has no sensor, and "Same data everywhere", which needs two nodes), and the pages name a failed unit, a dead node and a stale, silent or unreadable conductor as such | +| `vmTestServices` / `vmTestServices-noBootstrap` | The node page names exactly the services the enabled modules installed, each Running; a bootstrap server frozen, stopped or failing reads Not answering, Stopped or Failed, and the room screen's tile reads "A service is down"; without the server, the same list less that one | | `vmTestGateway` | A zome read answers 200 with JSON through the HTTP gateway, and a function outside the allow list answers 403 | | `vmTestWindtunnel` | The generated container unit carries the flags the runner requires, and stays stopped when `autoStart = false` | | `vmTestWdocker` | The packaged Moss `wdocker` starts its pinned Holochain 0.6.1 conductor through `wdaemon` in an offline VM, downloads nothing into its `bins` directory, and `wdocker stop` ends the conductor | +| `vmTestMossNode` | The Moss node service starts with no terminal, reads its password from a credential, survives a restart and reports `holochain_conductor_up{conductor="Moss"} 1`; with no password file it never starts, and an empty one is refused | | `vmTestBootstrap` | Two 0.6 edgenodes with no internet find each other through a `holochain-bootstrap` server and its plain-HTTP relay; its falsifier, with one node on the wrong port, must fail | -| `edgenodeConfigRender` | `relayAllowPlainText`, `requestTimeoutS`, `dbSyncLevel` and `wasmBackend` render on each line, and that line's real conductor starts on the result | +| `vmTestHoloportInstall` | `holoport-install` lays out an empty SATA disk the ADR-017 way and installs `sensorica-holoport-01` on it; the disk then boots under SeaBIOS, which is legacy BIOS like a Holoport, with GRUB for both firmwares, the conductor active and the three workshop hApps enabled once each | + +The checks without a VM are built in the same job: `edgenodeConfigRender` (`relayAllowPlainText`, `requestTimeoutS`, `dbSyncLevel` and `wasmBackend` render on each line, and that line's real conductor starts on the result), the exporter checks (`conductorMetricsJq`, `dhtMetricsJq`, `metricsHelpAgreement`, `metricsNameShape`, `edgenodeNamesWiring`), the Grafana checks (`holochainRules`, `grafanaProvisioning`, `dashboardLabels`, `dashboardWords`, `dashboardQueries`), the Moss checks (`moss-dashboard`, `moss-names`), and `edgenodeBinaryCache`, which `nix flake check` settles at evaluation. ```bash nix flake check --no-build --all-systems @@ -227,7 +221,7 @@ Each ticked item names the pull request that closed it. - [ ] Validated on a physical machine ([#8](https://github.com/Sensorica/nixos-holochain/issues/8)) **Phase 2: workshop ready** -- [x] `holochain-grafana`: Prometheus and Grafana with the "Holochain Fleet" dashboard and its data source provisioned (#17) +- [x] `holochain-grafana`: Prometheus and Grafana with the fleet dashboard (`holochain-fleet`) and its data source provisioned (#17) - [x] `conductorMetrics`: the conductor's own network stats as `holochain_*` series, on both lines (#17) - [x] The example fleet exports metrics on all five nodes, with the Wind Tunnel runner off in writing (#17) - [x] `holochain-windtunnel`: the Foundation's runner image, off by default, with what enabling it costs written into the option (#17) @@ -245,7 +239,7 @@ Each ticked item names the pull request that closed it. - [x] `CONTRIBUTING.md` with the VM-test and options-doc rules (#18) - [ ] hAppenings Community Substack announcement - [ ] hREA module (composable with the edgenode module) -- [ ] Documentation site +- [x] Documentation site: the book at [sensorica.github.io/nixos-holochain](https://sensorica.github.io/nixos-holochain/) (#67) **Phase 4: production hardening** - [ ] sops-nix integration for secrets diff --git a/SECURITY.md b/SECURITY.md index 4146db4..3cac746 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -20,7 +20,7 @@ Please do not open a public issue, pull request or discussion for a vulnerabilit A useful report says: -- which module and which options are involved (`services.holochain-edgenode`, `services.holochain-grafana`, `services.holochain-http-gateway` or `services.holochain-windtunnel`), +- which module and which options are involved (`services.holochain-edgenode`, `services.holochain-grafana`, `services.holochain-http-gateway`, `services.holochain-windtunnel`, `services.holochain-bootstrap` or `services.holochain-moss-node`), - the nixos-holochain commit your flake is locked to (`nix flake metadata` lists it under Inputs), - what an attacker can do, from where, and what they need first, - the steps or configuration that reproduce it. diff --git a/docs/adr/0017-holoport-legacy-bios-target.md b/docs/adr/0017-holoport-legacy-bios-target.md index 4e953f8..cdb9bcc 100644 --- a/docs/adr/0017-holoport-legacy-bios-target.md +++ b/docs/adr/0017-holoport-legacy-bios-target.md @@ -25,3 +25,5 @@ From #15, section 8, open question: "Whether the HoloPort firmware offers UEFI o ## Later record - [#21](https://github.com/Sensorica/nixos-holochain/pull/21) moved the boot loader out of the hardware stubs, because `nixos-generate-config` output never contains one: it now lives in `hosts/common.nix` of the `#fleet` template and of the example, so the stubs describe filesystems only. The `#minimal` template targets a stock NixOS UEFI install with systemd-boot, and a comment in its `configuration.nix` gives the GRUB lines for legacy BIOS. The Holoport layout of this ADR stays with `#fleet` and the example. +- [#61](https://github.com/Sensorica/nixos-holochain/pull/61) wrote the install sequence of this decision as one script, `scripts/holoport-install.sh`, published as `packages.x86_64-linux.holoport-install`: the layout above with 8 GiB of swap at the end of the disk, `nixos-install`, then `grub-install --target=i386-pc` for the BIOS half, while NixOS writes the EFI half from `hosts/common.nix`. `checks.x86_64-linux.vmTestHoloportInstall` runs it on an empty SATA disk and boots that disk under SeaBIOS, which is legacy BIOS like the Holoport. The runbook is [docs/deployment.md § Installing on a Holoport (legacy BIOS)](../deployment.md#installing-on-a-holoport-legacy-bios). +- The first install on a real Holoport, `sensorica-holoport-01` on 2026-09-27, came into `main` with [#67](https://github.com/Sensorica/nixos-holochain/pull/67). The runbook records what it showed: Esc at power-on opens the base HoloPort's firmware boot menu, its live system reads the stick reliably only from a USB 2 port, and a graphical desktop freezes on its Intel HD 610. The BIOS setup key, and whether UEFI or Secure Boot exist, are still unknown, so the open question above stands and issue #8 is open. diff --git a/docs/architecture.md b/docs/architecture.md index ddf981c..76286ee 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -21,6 +21,7 @@ The decisions behind this layout are recorded one per file in [`adr/`](adr/READM flake.nix ├── modules/ │ ├── holochain-edgenode.nix ← core: conductor + lair + hApp installer + metrics +│ ├── families.jq ← the one place a holochain_* family's HELP and TYPE are written │ ├── conductor-metrics.jq ← dump-network-stats → Prometheus text │ ├── conductor-counters.jq ← running byte and message totals across closed connections │ ├── dht-metrics.jq ← list-apps + dump-network-metrics → per-DHT Prometheus text @@ -31,9 +32,15 @@ flake.nix │ ├── holochain-windtunnel.nix ← optional: donate the machine to the Foundation's Nomad cluster │ ├── holochain-http-gateway.nix ← optional: HTTP gateway in front of the conductor │ ├── holochain-bootstrap.nix ← optional: Kitsune2 bootstrap and relay server -│ └── default.nix ← aggregator +│ ├── holochain-moss-node.nix ← optional: a Moss group's always-online node (not in default) +│ ├── moss-node-names.jq ← the Moss node's names file for the exporter +│ ├── dashboards-moss/ ← the Moss node's Grafana page +│ ├── sensorica-event-node.nix ← the Sensorica workshop profile (not in default) +│ ├── sensorica-happs.nix ← the three workshop hApps, fetched by hash +│ └── default.nix ← aggregator: edgenode, grafana, windtunnel, http-gateway, bootstrap ├── packages/ │ ├── holochain-http-gateway.nix ← the hc-http-gw build, one release per Holochain line +│ ├── holochain-conductor-exporter.nix ← the one program that writes holochain_* series, for any conductor │ └── wdocker.nix ← Moss wdocker, with the Holochain it pins └── templates/ ├── minimal/ ← nix flake init -t …#minimal: one edgenode @@ -58,6 +65,8 @@ Modules are independent. Import only what you need. | `holochain-bootstrap.service` | simple, `DynamicUser`, no state directory | `holochain-bootstrap.enable` | | `holochain-service-health.service` | oneshot, driven by the timer: runs every declared health check and writes `holochain-service-health.prom` | a health check is declared (the bootstrap server declares one) and `services.holochain-services.textfileDirectory` is set | | `holochain-service-health.timer` | `OnBootSec` / `OnUnitActiveSec` = 30 s | as above | +| `moss-node.service` | simple, its own `moss-node` user, the password as a credential | `holochain-moss-node.enable` ([moss-node.md](moss-node.md#as-a-nixos-service)) | +| `moss-node-metrics.service`, `moss-node-metrics.timer` | oneshot driven by a 30 s timer: the conductor exporter under the name `Moss` | `holochain-moss-node.enable` | ## Service dependency graph diff --git a/docs/archive/README.md b/docs/archive/README.md index b94d6d6..7092055 100644 --- a/docs/archive/README.md +++ b/docs/archive/README.md @@ -2,6 +2,6 @@ *Original HolOS-based workshop, December 2025. Preserved for context. Current workshop uses NixOS — see `workshop/facilitator-guide.md`.* -The December 2025 event at Sensorica lab installed HolOS and edgenode on 5 Holoports. Key lessons learned are captured in the facilitator guide for the August 2026 NixOS workshop. +The December 2025 event at Sensorica lab installed HolOS and edgenode on 5 Holoports. Key lessons learned are captured in the facilitator guide for the 2026 NixOS workshop. Original files from [Sensorica/holoports-workshop](https://github.com/Sensorica/holoports-workshop) (archived) will be placed here if migrated. diff --git a/docs/deployment.md b/docs/deployment.md index 5890aa8..deac8bf 100644 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -33,7 +33,7 @@ Before running `colmena apply` from `examples/sensorica-fleet`, each host must h 1. A real `hardware-configuration.nix` replacing the committed placeholder, generated on the target machine: ```bash - sudo nixos-generate-config --show-hardware-config > examples/sensorica-fleet/hosts/edgenode-XX/hardware-configuration.nix + sudo nixos-generate-config --show-hardware-config > examples/sensorica-fleet/hosts/sensorica-holoport-0N/hardware-configuration.nix ``` 2. The facilitator's SSH public key in `examples/sensorica-fleet/hosts/common.nix` in the `operatorKeys` list at the top (used for the `sensorica` account and for root, which Colmena connects as; public keys are committed, a flake never sees untracked files). @@ -168,7 +168,7 @@ journalctl -u holochain-happ-installer --no-pager | grep 'Enabled app' curl -s -u "admin:NEW_PASSWORD" 'localhost:3000/api/search?query=Holochain' ``` -The first boot compiles three hApps, so `holochain-happ-installer` can take several minutes to finish on a Holoport (about a minute in the VM check). The conductor and the Moss node should each answer `active`; the journal should end with `hc-sandbox: Enabled app: "hrea"`, `"kando"` and `"requests-and-offers"`; and the last line should return the **Holochain Fleet** dashboard, which is also at `http://HOLOPORT_IP:3000` from a laptop on the same network. [Verifying the deployment](#verifying-the-deployment) has the metrics checks. +The first boot compiles three hApps, so `holochain-happ-installer` can take several minutes to finish on a Holoport (about a minute in the VM check). The conductor and the Moss node should each answer `active`; the journal should end with `hc-sandbox: Enabled app: "hrea"`, `"kando"` and `"requests-and-offers"`; and the last line should list the dashboards whose titles name Holochain, among them the fleet page, **Which Holochain node needs attention?** Grafana is also at `http://HOLOPORT_IP:3000` from a laptop on the same network, where it opens on **What is this machine running?** [Verifying the deployment](#verifying-the-deployment) has the metrics checks. The Moss node hosts no group until it joins one. Once per machine, as root on the Holoport, run `moss-node join "INVITE_LINK"` with an invite from the Sensorica group in Moss, starting the line with a space so the link stays out of shell history, as described in [Moss always-online node](moss-node.md#as-a-nixos-service); `moss-node status` then lists the group. @@ -364,7 +364,7 @@ On the monitor node: # every configured scrape target should be "health":"up" curl -s localhost:9090/api/v1/targets | jq '.data.activeTargets[] | {scrapeUrl, health, lastError}' -# the four provisioned dashboards should be there +# the five provisioned dashboards should be there, six where the Moss page is # export GRAFANA_ADMIN_PASSWORD first; on a node that kept the module # default it is the workshop password curl -s -u "admin:$GRAFANA_ADMIN_PASSWORD" 'localhost:3000/api/search?tag=holochain' | jq -r '.[].uid' @@ -446,7 +446,7 @@ Next to a conductor's gigabyte this is noise, so one Holoport can carry the serv ```bash # Roll back to the previous NixOS generation -sudo nixos-rebuild --rollback +sudo nixos-rebuild switch --rollback # List all generations sudo nix-env --list-generations --profile /nix/var/nix/profiles/system diff --git a/docs/introduction.md b/docs/introduction.md index 1223528..97aee41 100644 --- a/docs/introduction.md +++ b/docs/introduction.md @@ -12,7 +12,7 @@ These modules are that middle path. One `nixos-rebuild` brings up a conductor wi ## What is in the repository -- **Five NixOS modules.** `holochain-edgenode` is the core: conductor, in-process lair keystore, an idempotent hApp installer and optional metrics, on both the 0.7 and the 0.6 Holochain lines from one option set. `holochain-grafana` adds Prometheus and Grafana for a fleet, `holochain-http-gateway` serves chosen zome functions over HTTP, `holochain-bootstrap` runs your own bootstrap and relay server, and `holochain-windtunnel` lends a machine to the Holochain Foundation's test cluster, off by default. +- **Seven NixOS modules.** `holochain-edgenode` is the core: conductor, in-process lair keystore, an idempotent hApp installer and optional metrics, on both the 0.7 and the 0.6 Holochain lines from one option set. `holochain-grafana` adds Prometheus and Grafana for a fleet, `holochain-http-gateway` serves chosen zome functions over HTTP, `holochain-bootstrap` runs your own bootstrap and relay server, and `holochain-windtunnel` lends a machine to the Holochain Foundation's test cluster, off by default. `holochain-moss-node` runs a Moss group's always-online node beside the edgenode, and `sensorica-event-node` is the Sensorica workshop's profile (Holochain line, hApps and network seed) layered on it. - **Two flake templates.** `nix flake init -t github:Sensorica/nixos-holochain#minimal` writes one edgenode; `#fleet` writes five nodes with Grafana, a Colmena hive and a live ISO. diff --git a/happs/README.md b/happs/README.md index 9c5412f..12be9b8 100644 --- a/happs/README.md +++ b/happs/README.md @@ -41,9 +41,6 @@ Neither test is conditional. An earlier version of `vmTestWithHapp` was gated on ## Workshop bundles -| Bundle | Source | Purpose | -|---|---|---| -| Wind Tunnel | [holochain/wind-tunnel](https://github.com/holochain/wind-tunnel) | Observable traffic for the Grafana moment (slice 3) | -| Moss | [lightningrodlabs/moss](https://github.com/lightningrodlabs/moss) | Participants join the group from their own laptop after the workshop | +The Sensorica workshop fleet runs hREA, Kando and Requests & Offers on the 0.6 line (ADR-015), fetched by hash in `modules/sensorica-happs.nix` and installed by the `sensorica-event-node` profile. Versions and bundles are listed once, in the Sensorica fleet README (`examples/sensorica-fleet/README.md`, "Holochain line and hApps"). Participants join the Sensorica Moss group from their own laptop with [Moss](https://github.com/lightningrodlabs/moss), and the group's always-online node runs on the monitor Holoport (`holochain-moss-node`). Wind Tunnel is not a workshop bundle: the `holochain-windtunnel` module lends a machine to the Foundation's test cluster and feeds nothing to Grafana. Check each project's releases for a bundle built against the Holochain line the fleet runs. As of this writing Wind Tunnel, hREA, Requests & Offers and Nondominium all still publish 0.6.x bundles; only Moss 0.16-dev targets 0.7. diff --git a/holochain-nixos-plan.md b/holochain-nixos-plan.md index ca81582..ad6e3ef 100644 --- a/holochain-nixos-plan.md +++ b/holochain-nixos-plan.md @@ -1,5 +1,7 @@ # holochain-nixos +> **Superseded:** this is the May 2026 plan the repository started from, kept as a record. Where it differs from the tree, the tree wins: the project is `nixos-holochain`, licensed MIT, `modules/pai.nix` was removed, the fleet lives in `examples/sensorica-fleet/`, and the current state is in [README.md](README.md) and the [book](https://sensorica.github.io/nixos-holochain/). + > A declarative substrate for running Holochain edgenodes, hApps, and developer environments. Built at Sensorica, intended for the Holochain community. **Status:** Pre-alpha. Workshop substrate under construction. diff --git a/templates/fleet/README.md b/templates/fleet/README.md index 2756e2c..d8ff043 100644 --- a/templates/fleet/README.md +++ b/templates/fleet/README.md @@ -73,7 +73,7 @@ sync ## Monitoring -`node-01` serves Grafana on `:3000` with the "Holochain Fleet" dashboard provisioned, scraping every node's `node_exporter` and the conductor metrics timer. It logs in as `admin` with the password in `/var/lib/secrets/grafana-admin-password`, which you create on the node before the first deploy (root-owned, mode 0400; systemd hands it to Grafana); `services.holochain-grafana.adminPasswordFile` in the option reference gives the commands. The module's `adminPassword` default is a lab convenience and lands world-readable in the Nix store, so it is not used here. +`node-01` serves Grafana on `:3000` with the five Holochain dashboards provisioned, "What is this machine running?" as its home page, scraping every node's `node_exporter` and the conductor metrics timer. It logs in as `admin` with the password in `/var/lib/secrets/grafana-admin-password`, which you create on the node before the first deploy (root-owned, mode 0400; systemd hands it to Grafana); `services.holochain-grafana.adminPasswordFile` in the option reference gives the commands. The module's `adminPassword` default is a lab convenience and lands world-readable in the Nix store, so it is not used here. ## Option reference diff --git a/workshop/facilitator-guide.md b/workshop/facilitator-guide.md index 2be48b3..c955a48 100644 --- a/workshop/facilitator-guide.md +++ b/workshop/facilitator-guide.md @@ -13,9 +13,9 @@ |------|---------|------| | 0:00 to 0:30 | **Conceptual intro** | Declarative vs imperative. Why this matters for Holochain. Fractal sovereignty framing if the room is receptive. | | 0:30 to 1:15 | **Flake walkthrough** | Open the repo in Kate. Walk through `flake.nix`, the module, a host config. Show option discovery via `nix repl`. | -| 1:15 to 2:15 | **First deploy** | Each participant boots, clones the repo, runs `nixos-rebuild switch --flake .#minimal`. Conductor visible via `systemctl status`. | -| 2:15 to 3:15 | **Add Wind Tunnel + observe** | Flip `services.holochain-windtunnel.enable = true`. Redeploy. Open Grafana, watch fleet traffic light up. | -| 3:15 to 3:45 | **Modify, rollback, join Moss** | Change a hApp property, redeploy, then `nixos-rebuild --rollback`. This is where the "aha" usually lands. Then have participants open Moss on their laptop and join the group hosted by the fleet. | +| 1:15 to 2:15 | **First deploy** | Each participant boots, clones the repo, runs `sudo nixos-rebuild switch --flake .#sensorica-holoport-0N` from `examples/sensorica-fleet` (`rebuild` on an installed Holoport). Conductor visible via `systemctl status`. | +| 2:15 to 3:15 | **Observe** | Open Grafana's room screen and watch the fleet's traffic: the five conductors running hREA, Kando and Requests & Offers on one network seed. Wind Tunnel is not part of this: it feeds nothing to Grafana (see docs/architecture.md § What the Wind Tunnel runner is, and is not). | +| 3:15 to 3:45 | **Modify, rollback, join Moss** | Change a hApp property, redeploy, then `sudo nixos-rebuild switch --rollback`. This is where the "aha" usually lands. Then have participants open Moss on their laptop and join the group hosted by the fleet. | | 3:45 to 4:00 | **Q&A + next steps** | How to extend the module. How to contribute back. Where the project goes from here. | --- @@ -34,9 +34,9 @@ Workshop nodes ship with KDE Plasma 6 as the desktop. Reasoning: ## Facilitation notes - **Option A vs B trade-off.** Option A (pre-baked module, participants are users) is what this workshop does. Option B (live module authoring) is more interesting but riskier and only works for groups already comfortable with Nix. For 5-machine fleets with mixed audiences, A wins. -- **Deployment tool.** `colmena apply --on @all` for parallel deploys. Plain `nixos-rebuild switch --target-host` if colmena feels like too much. +- **Deployment tool.** `colmena apply --impure --on @all` for parallel deploys (`--impure`: see the fleet README). Plain `nixos-rebuild switch --target-host` if colmena feels like too much. - **Network reality.** Test the workshop network in advance. The December 2025 HolOS workshop was bitten by this. Bring a dedicated router. -- **Grafana moment.** This is the high point of the workshop. Make sure Wind Tunnel is generating visible traffic before flipping the dashboard to the big screen. +- **Grafana moment.** This is the high point of the workshop. Make sure the room screen shows the three hApps In step on every machine before flipping it to the big screen. --- @@ -44,10 +44,10 @@ Workshop nodes ship with KDE Plasma 6 as the desktop. Reasoning: | Symptom | Likely cause | Fix | |---------|-------------|-----| -| `holochain-conductor.service` fails immediately | Lair keystore not initialized | Check `journalctl -u holochain-conductor` for lair errors; may need a first-boot init step | -| `happ-installer.service` fails silently | hApp file not found at path | Verify `happs/` contains the `.happ` files before ISO build | +| `holochain-conductor.service` fails immediately | Conductor or keystore error | Check `journalctl -u holochain-conductor`; the unit creates its lair passphrase itself on first boot, so no init step is missing | +| `holochain-happ-installer.service` fails | An app did not install or enable within `installerTimeout` (900 s on the fleet); the first boot compiles three hApps | `journalctl -u holochain-happ-installer`, then `systemctl restart holochain-happ-installer`. Bundles are fetched by hash when the system is built, never read from `happs/` | | Participants can't see each other's nodes | Firewall closed | Ensure `openFirewall = true` and router is not blocking DHT traffic | -| `colmena apply` can't reach nodes | SSH keys not set up | Add facilitator SSH key to each host config before building | +| `colmena apply` can't reach nodes | SSH keys not set up | Add the facilitator's SSH key to `operatorKeys` in `examples/sensorica-fleet/hosts/common.nix` before building | | Live USB drops to emergency mode, "Expecting device /dev/disk/by-label/nixos-graphical-…" | Stick made with Ventoy | Write the ISO with `dd` and check it with `cmp`; see docs/deployment.md § Rescuing an install | | Installer fails on `cache.nixos.org … after 0 ms` | Router DNS answers IPv6 only, no IPv6 route | Public DNS on the live session with `nmcli`, then retry; see docs/deployment.md § Rescuing an install | | Installer offers only manual partitioning on retry | Previous failed run still mounted | Unmount `/tmp/calamares-root-*` and `swapoff -a`, relaunch | diff --git a/workshop/participant-handout.md b/workshop/participant-handout.md index 1465350..fd889cd 100644 --- a/workshop/participant-handout.md +++ b/workshop/participant-handout.md @@ -1,6 +1,6 @@ # Participant Handout — Holochain Edgenode Workshop -**Sensorica Lab, August 2026** +**Sensorica Lab, 2026** --- @@ -10,7 +10,7 @@ By the end of this session you will have: - A working Holochain edgenode running on a real machine, declared entirely in a single Nix file - Deployed that node into a 5-machine fleet using a single command -- Watched live P2P traffic between all nodes using Wind Tunnel + Grafana +- Watched live P2P traffic between all nodes on Grafana - Rolled back a configuration change in under 10 seconds --- @@ -25,7 +25,7 @@ sudo nixos-rebuild switch --flake .#sensorica-holoport-0X systemctl status holochain-conductor ↓ # if something breaks: -sudo nixos-rebuild --rollback +sudo nixos-rebuild switch --rollback ``` That is the whole practice. Everything else is understanding what lives in `configuration.nix`. @@ -50,7 +50,7 @@ nixos-holochain/ | Command | What it does | |---------|-------------| | `nixos-rebuild switch --flake .#sensorica-holoport-01` | Rebuild and switch to new config | -| `nixos-rebuild --rollback` | Roll back to previous generation | +| `nixos-rebuild switch --rollback` | Roll back to previous generation | | `systemctl status holochain-conductor` | Check conductor health | | `journalctl -u holochain-conductor -f` | Follow conductor logs | | `nix repl --file ''` | Explore available options interactively | diff --git a/workshop/preflight-checklist.md b/workshop/preflight-checklist.md index 1dd3a0d..f0209b1 100644 --- a/workshop/preflight-checklist.md +++ b/workshop/preflight-checklist.md @@ -29,7 +29,7 @@ You do not need to know Nix before the workshop. We will walk through the flake - [ ] Flash 5 USB keys with the workshop ISO - [ ] Test ISO boots on at least one Holoport / NUC - [ ] Verify `colmena apply` reaches all 5 nodes over the local network -- [ ] Confirm `windtunnel.happ` and `moss.happ` are in `happs/` and the hApp installer service starts cleanly +- [ ] Confirm the hApp installer enabled hREA, Kando and Requests & Offers on every node (`journalctl -u holochain-happ-installer | grep 'Enabled app'`); the bundles are fetched by hash, nothing goes in `happs/` - [ ] Bring a dedicated router (tested) — do not rely on Sensorica lab wifi alone - [ ] Print or share the participant handout - [ ] Have Grafana dashboard URL ready on a shared screen