oblinder · Oct 19, 2026 · Oct 14, 2026 · Oct 14, 2026 · Oct 15, 2026 · Oct 15, 2026
diff --git a/CLAUDE.md b/CLAUDE.md
@@ -21,6 +21,13 @@ offline unit lane.
 - To write a new test (placement, markers, clean-skip), see
   `docs/testing/testing-strategy.md`.
 
+**Dependency direction.** Tests of any type (`test/`, `eval/`) must not depend on
+`demo/` or point to a demo doc (for example `demo.md`). They set up what they need
+themselves. The demo may use the test harness, but the harness must not use the demo.
+One exception: `demo/assets/` (the `github-agent` and `github-tool` workloads, their
+manifests, Dockerfiles and `kind-load.sh`). The system tests deploy these two
+workloads, so they may use `demo/assets/` and nothing else under `demo/`.
+
 ## Python environment
 
 Always use the `.venv` virtual environment (`.venv/bin/python`, `.venv/bin/pytest`)

diff --git a/demo/use-cases/onboarding/demo.md b/demo/use-cases/onboarding/demo.md
@@ -221,7 +221,8 @@ To tear the forward down afterwards: `pkill -f 'port-forward .*keycloak-service'
 
 If the realm was freshly provisioned, the `rossoctl` client needs Direct Access Grants enabled.
 Without it, token minting fails with `unauthorized_client`. This is platform state, so the demo does
-not set it:
+not set it. The snippet also removes the attribute `aiac.test.direct-access-grants-by-harness`, so a
+system-test run does not turn the grant off at its end:
 
 ```bash
 KC=http://keycloak.localtest.me:8080
@@ -232,7 +233,7 @@ ADMIN=$(curl -s -X POST "$KC/realms/master/protocol/openid-connect/token" \
 CID=$(curl -s -H "Authorization: Bearer $ADMIN" "$KC/admin/realms/rossoctl/clients?clientId=rossoctl" \
   | python3 -c 'import sys,json;print(json.load(sys.stdin)[0]["id"])')
 curl -s -H "Authorization: Bearer $ADMIN" "$KC/admin/realms/rossoctl/clients/$CID" \
-  | python3 -c 'import sys,json;d=json.load(sys.stdin);d["directAccessGrantsEnabled"]=True;print(json.dumps(d))' \
+  | python3 -c 'import sys,json;d=json.load(sys.stdin);d["directAccessGrantsEnabled"]=True;d.setdefault("attributes",{})["aiac.test.direct-access-grants-by-harness"]="";print(json.dumps(d))' \
   | curl -s -o /dev/null -w "enable DAG HTTP %{http_code}\n" -X PUT -H "Authorization: Bearer $ADMIN" \
     -H "Content-Type: application/json" "$KC/admin/realms/rossoctl/clients/$CID" --data-binary @-
 ```
@@ -817,8 +818,8 @@ make teardown ARGS=--include-opa   # also revert step 2's overlay and combiner (
 - **the `team1` namespace itself** — the installer owns it
   ([`INSTALL.md`](../../assets/INSTALL.md): "a precondition, not an output"). A fresh install gives
   you an *empty* `team1`, so emptying it *is* the post-install state.
-- **the realm fix-up** (Direct Access Grants) — one-time cluster state that this demo and the
-  `-m system` suite both depend on. The client scope `aiac-username-sub` and its links (D31) are
+- **the realm fix-up** (Direct Access Grants) — one-time cluster state that this demo depends on.
+  (The `-m system` suite enables the grant itself.) The client scope `aiac-username-sub` and its links (D31) are
   AIAC's; AIAC makes them again at its next start and onboarding.
 - **the operator's `*-aud` audience client scopes**, which it owns and recreates.
 - **step 2's OPA overlay**, unless you pass `--include-opa`.

diff --git a/docs/agents/deploy.md b/docs/agents/deploy.md
@@ -1,48 +1,11 @@
-# Deploy parameters and preconditions
-
-Names the deploy-time parameters and preconditions of this repo for the `deploy`
-skill. The deploy flow is in `k8s/aiac-deployment-guide.md`. The commands for each
-parameter are in the linked doc — this file copies none of them.
-
-## Precondition: OPA on the Kind platform
-
-AIAC needs OPA in the AuthBridge pipeline, `bundle-service`, and the changed
-global combiner (D20). Without them the AIAC Controller does not start.
-
-| | |
-|---|---|
-| Applies to | Every deploy of an AIAC unit in `aiac-system`, and a parameter-only deploy |
-| Check | `./k8s/opa-kind-verify.sh` (read-only, exits non-zero on the first failed check) |
-| Install | `OPERATOR_DIR=… ROSSOCTL_DIR=… CORTEX_DIR=… ./k8s/opa-kind-enable.sh`, from the repo root. Details: `k8s/opa-kind-runbook.md` → Enable |
-| Install warning | Two image builds (a few minutes). It runs `helm upgrade` on the rossoctl chart and restarts the agent and tool pods in `team1`. |
-| Inputs | Sibling clones `operator`, `rossoctl`, `cortex`. Defaults are `../operator`, `../rossoctl`, `../cortex`, relative to the repo root. If one is missing, stop and ask for its path. |
-| Re-check | After any `kubectl apply` of a platform manifest or an operator-chart upgrade. The check also fails when the stock combiner is back. |
-
-Enable runs the verify script as its last step, so after an install the check
-has already passed once.
-
-`k8s/opa-kind-enable.sh` and `k8s/opa-kind-restore.sh` run `helm upgrade` on the
-rossoctl chart. This can put the stock Keycloak image back. After either script,
-do the Keycloak SPI check below.
-
-## Precondition: Keycloak event-listener SPI
-
-AIAC learns of a new Keycloak client from the `aiac-event-listener` SPI, which
-publishes to the NATS broker. Without it no registration event reaches the
-Controller, and the onboarding path (and the system lane) does not work.
-
-| | |
-|---|---|
-| Applies to | Every deploy of an AIAC unit in `aiac-system`, and a parameter-only deploy |
-| Check | `./k8s/keycloak-spi-verify.sh` (read-only, exits non-zero on the first failed check) |
-| Install | `./k8s/keycloak-spi-install.sh`, from the repo root |
-| Install warning | One image build (maven container, then a derived Keycloak image). It restarts the Keycloak StatefulSet once, so Keycloak is down for a short time. On the realm it adds `aiac-event-listener` to the event listeners and turns on admin events. The other event settings stay. |
-| Inputs | The OPA precondition above first (the platform must be on Kind). A broker in `aiac-system` is not needed to install, but the listener drops events until `aiac-event-broker` runs. Both scripts read `KEYCLOAK_ADMIN_USERNAME` / `KEYCLOAK_ADMIN_PASSWORD` from the environment, else from `.env`, else from secret `aiac-system/keycloak-admin-secret`, else use `admin`. |
-| Re-install | The install records the stock Keycloak image in the StatefulSet annotation `aiac.rossoctl.dev/keycloak-base-image`, so a re-run derives from the same image. If the live image is the derived one and the annotation is missing (an install by an older script), the install stops: set `KEYCLOAK_IMAGE_BASE` to the stock image. |
-| Re-check | After any change of the Keycloak StatefulSet. It can put the stock Keycloak image back. |
-
-Install runs the verify script as its last step, so after an install the check
-has already passed once. Do the OPA precondition first.
+# Deploy parameters
+
+Names the deploy-time parameters of this repo for the `deploy` skill. The deploy
+flow and its platform preconditions (OPA on the Kind platform, the Keycloak
+event-listener SPI) are in `k8s/aiac-deployment-guide.md` → Platform
+preconditions. Check them before every deploy, also a parameter-only deploy. The
+commands for each parameter are in the linked doc — this file copies none of
+them.
 
 ## `ac-side` — access-control scheme (enforcement side)
 

diff --git a/docs/agents/test.md b/docs/agents/test.md
@@ -94,12 +94,16 @@ assert the allow/deny of the **deployed OPA plugin**. They do not use
 
 Preconditions:
 
-- A rossoctl/Kind cluster with the AuthBridge OPA pipeline wired into both legs.
-- The AIAC global combiner. It denies a pod that has no `AuthorizationPolicy`
-  CR, and the Controller does not start without it.
-- `k8s/opa-kind-enable.sh` does both steps (run it one time).
-- The event path wired: the NATS broker and the `aiac-event-listener` Keycloak
-  SPI (see `docs/testing/uc1-onboarding-pipeline.md`, "Preconditions").
+- AIAC deployed on a rossoctl/Kind cluster that meets the platform
+  preconditions in `k8s/aiac-deployment-guide.md` → Platform preconditions: OPA
+  on the Kind platform (with the AIAC global combiner, which denies a pod that
+  has no `AuthorizationPolicy` CR) and the Keycloak event-listener SPI.
+- An LLM key that the endpoint accepts, in the Controller's Secret
+  `aiac-system/aiac-agent-secret`. The in-cluster Controller does not read
+  `.env`. If the endpoint refuses the key, each onboarding rolls back, and the
+  fixtures time out with "agent did not converge" or "tool did not converge".
+  The Controller log then shows `LLMCredentialsError`. To check or change the
+  key: `k8s/aiac-deployment-guide.md`, step 3b, "Check or change the API key".
 
 Do **not** deploy the demo `github-agent` / `github-tool` yourself. The session
 fixture (`onboarded_stack` in `test/system/uc1_onboard.py`) builds and loads the
@@ -114,12 +118,17 @@ test both sides, run the lane, do a clean redeploy with the other side (see
 `docs/agents/deploy.md`), and run the lane again.
 
 The platform prerequisites, wiring, verification and the setting of the side:
-`k8s/opa-kind-runbook.md`. The realm fix-up (Direct Access Grants) and the manual
-probes of the scenario: `demo/use-cases/onboarding/demo.md`. The per-loop shape:
-`test/system/uc1_onboard.py`.
+`k8s/opa-kind-runbook.md`. The fixture enables Direct Access Grants on the login
+client `rossoctl` itself, once per session, and turns it off again at session end
+if it turned it on, so the lane needs no manual realm step. A client attribute
+records that the harness turned it on, so a run that is killed is undone by the
+next run. A
+manual probe with the password grant outside the lane needs the grant set by hand:
+see the runbook section "Direct Access Grants on the login client". The per-loop
+shape: `test/system/uc1_onboard.py`.
 
 ```bash
-k8s/opa-kind-enable.sh          # one time: wire the OPA plugin and the AIAC combiner into Kind
+k8s/opa-kind-verify.sh && k8s/keycloak-spi-verify.sh   # the platform preconditions
 set -a; . .env; set +a
 .venv/bin/pytest -m system
 ```

diff --git a/docs/analysis/user-subject-across-token-exchange.md b/docs/analysis/user-subject-across-token-exchange.md
@@ -190,7 +190,7 @@ So:
 - **Agent side:** one hop works, which is why the runbook's agent-side example (B.5) shows
   `dev-user`. It fails at the second agent of a chain, in both that agent's inbound and its
   outbound. AIAC models such a chain: the delegation scenario `dispatch-agent` → `customs-agent`
-  (`test/system/scenario_eval_agent_delegation.py`).
+  (`eval/scenarios/scenario_eval_agent_delegation.py`).
 
 The agent-side rows come from the code paths. They are not run live (§9).
 

diff --git a/docs/evaluation/policy-eval-correctness-prb.md b/docs/evaluation/policy-eval-correctness-prb.md
@@ -191,8 +191,7 @@ covered exactly as `policy-eval-scenarios.md` already documents them
 
 ## Policy-text rewrite
 
-As part of this ticket, all 8 scenarios' policy `.md` files (`eval/scenarios/policy.eval_*.md`,
-`test/system/policy.eval_agent_delegation.md`) and their 8 `eval/scenarios_perturbed/`
+As part of this ticket, all 8 scenarios' policy `.md` files (`eval/scenarios/policy.eval_*.md`) and their 8 `eval/scenarios_perturbed/`
 semantic siblings were rewritten to read like real human-authored access-control policy rather
 than the pipeline's own internal vocabulary reflected back into the document a policy author would
 supposedly have written. The previous text opened with the literal internal-modeling phrase

diff --git a/docs/evaluation/policy-eval-robustness-consistency.md b/docs/evaluation/policy-eval-robustness-consistency.md
@@ -46,8 +46,8 @@ scenarios, and reuse that family's scenario corpus rather than defining their ow
 six siblings, plus their `policy.eval_*.md` files) were moved from `eval/` directly into a new
 `eval/scenarios/` subpackage, so the growing `eval/` directory doesn't flatten test modules,
 scenario-data modules, and (now) two more scenario-data variants (`scenarios_perturbed/`) into one
-namespace. `scenario_eval_agent_delegation.py`/`policy.eval_agent_delegation.md` are unaffected —
-they already lived at `test/system/` top level and stay there. `test_policy_pipeline_eval.py`'s
+namespace. `scenario_eval_agent_delegation.py`/`policy.eval_agent_delegation.md` were first left
+at `test/system/` top level; they later moved to `eval/scenarios/` too. `test_policy_pipeline_eval.py`'s
 imports were updated accordingly; no test logic changed.
 
 ## Description
@@ -426,7 +426,7 @@ chart over time, not just show up as a one-off failed run with no lasting record
   robustness tests *and* the consistency suite identically for this scenario, both invariance
   (unedited) and sensitivity (edited) — proving it was a corpus gap, not an edit-mechanism bug.
   Fixed in two places: the committed digested `.md` (both original and perturbed sibling) now
-  states the read grant explicitly, and `agent-role-dispatcher`'s own description (`test/system/
+  states the read grant explicitly, and `agent-role-dispatcher`'s own description (`eval/scenarios/
   scenario_eval_agent_delegation.py` and its perturbed counterpart) now says "reading, creating,
   and updating" instead of just "creating and updating" — the same cascade-needs-its-own-textual-
   grounding principle this file already documents for `unreachable_resources`/`empty_descriptions`

diff --git a/docs/evaluation/policy-eval-scenarios.md b/docs/evaluation/policy-eval-scenarios.md
@@ -15,21 +15,16 @@
 
 Two independent groups of files, split by cost tier:
 
-**Heavy scenarios (1, 3, 4, 6-10) — full pipeline, `eval` marker — under `eval/`,
-except `agent_delegation`:**
+**Heavy scenarios (1, 3, 4, 6-10) — full pipeline, `eval` marker — under `eval/`:**
 - `eval/test_policy_pipeline_eval.py` — the test module, `@pytest.mark.eval`.
 - `eval/scenarios/scenario_eval_baseline.py` (Scenario 1),
+  `scenario_eval_agent_delegation.py` (Scenario 3),
   `scenario_eval_unreachable_resources.py` (Scenario 4), `scenario_eval_ambiguous_clause.py`
   (Scenario 6), `scenario_eval_wildcard_grant.py` (Scenario 7),
   `scenario_eval_misleading_descriptions.py` (Scenario 8), `scenario_eval_confusable_agents.py`
   (Scenario 9), `scenario_eval_empty_descriptions.py` (Scenario 10) — pure-data scenario modules
   (mirroring `scenario.py`'s shape, generalized to lists/dicts of many entities), each isolating
   exactly one aspect at the minimal entity count that aspect needs.
-- **`test/system/scenario_eval_agent_delegation.py`** (Scenario 3) — the one exception:
-  lives at the **top level** of `test/system/` (sibling of `launcher.py`/`scenario_uc1.py`),
-  not under `eval/` like the other seven. It isolates the agent-to-agent delegation
-  (`delegation_scopes`) mechanism, which is conceptually closer to the top-level fixed-scenario family than to
-  the `eval/` catalog's silent-gap/ambiguity/adversarial-authoring aspects.
 - A matching `policy.eval_<name>.md` next to each scenario module above — the scenario's *source*
   policy text. Not read directly at runtime: the harness's `pipeline` fixture points
   `AIAC_POLICY_FILE` at that scenario's **committed digested** counterpart in
@@ -324,8 +319,7 @@ See each scenario module's own module docstring (`scenario_eval_baseline.py`,
 `scenario_eval_empty_descriptions.py`) for the full entity list and role→access facts — these are
 the single source of truth (`INBOUND_PAIRS`/`OUTBOUND_SUBJECT_PAIRS`/`OUTBOUND_PAIRS`, plus
 `EXPECT_NO_REGO`/`IDENTITY_CONFUSION_PROBES` where applicable), not a second hand-maintained copy in
-this document. Note that `scenario_eval_agent_delegation.py` lives at the top level of
-`test/system/`, not under `eval/` like the other seven (see [Location](#location)). Scenarios
+this document. Scenarios
 2 and 5's cast is defined inline in their test modules' `_POLICY`/`_USER_ROLES`/`_ROLES` constants —
 see [Location](#location) for why the standalone `.md` mirrors are not what the tests actually
 read.
@@ -340,7 +334,7 @@ scenarios (`KEYCLOAK_URL`, `KEYCLOAK_ADMIN_USERNAME`/`PASSWORD`, `AIAC_PDP_CONFI
 | Variable | Difference from `policy-pipeline.md` |
 |----------|----------------------------------------|
 | `KEYCLOAK_REALM` | Set per scenario module (`scenario.REALM_DEFAULT`), not a single fixed realm — eight distinct realms across the session. |
-| `AIAC_POLICY_FILE` | Set per scenario to its **digested** policy in `eval/scenarios_digested/` (heavy scenarios only) — `digested_policy_path(scenario)`, same filename as `scenario.POLICY_FILE` regardless of which directory that scenario's source lives in, so `scenario_eval_agent_delegation.py` living one level up from the rest (see [Location](#location)) needs no special-casing here. |
+| `AIAC_POLICY_FILE` | Set per scenario to its **digested** policy in `eval/scenarios_digested/` (heavy scenarios only) — `digested_policy_path(scenario)`, same filename as `scenario.POLICY_FILE` regardless of which directory that scenario's source lives in. |
 
 The light scenarios (2, 5) need only `LLM_BASE_URL`/`LLM_MODEL`/`LLM_API_KEY` — no Keycloak, store,
 or OPA URLs, no `opa` binary.