Repository navigation
docs(eventing): KubeCon NA knowledge base — abstract, status, plan, slides - #893
Conversation
…lides A reference for the accepted KubeCon NA talk, kept beside the designs it cites so its claims can be checked against the code the same way theirs can. Four parts: the abstract as submitted with its five demo beats broken out and numbered; the status of each against the tree as of 2026-10-05; the W1-W6 plan to close the gaps; and the 30-minute slide and demo shape. Two of the four seams in the abstract name SPIRE, and there is no SPIRE in the implementation -- seven mentions, all prose. The egress block the demo list promises has no NetworkPolicy, Istio or sidecar behind it at all. Those are the two gaps the plan is ordered around, and SPIRE is its own section because the choice between wiring it and reframing it decides both the plan and the slides. The status section was written by running the decision functions rather than reading them, which is DESIGN_PHASE2.md section 8.6's rule. It changed two conclusions: a forged group.completed is refused by the verifier and then applied anyway (#885, traced through kafka_in.py to on_group_event), and enforcement with no keyset refuses nothing while printing "response verification OFF" (#888). An unsigned non-terminal frame is also accepted, but that one is the signing policy on the other side, not a hole -- recorded as such. Also recorded: the signed path has never run on a cluster, and k8s_demo_flow.py cannot drive the kind-signed overlay, so there is no evidence it survives a real broker. Section 4.4 step 2 is the precedent -- the signer/verifier policy mismatch was found against a live broker and by no unit test. Section 7 keeps what the review found that no single document states, including a false negative of its own: the first group-forgery harness used wrong event-type constants and reported nothing. A negative result from a hand-written harness is worth little until the harness has produced a positive one. No code, no manifests, no tests. Adds a row and a reading-order entry to the agentdocs index. Assisted-By: Claude Code Signed-off-by: Mariusz Sabath <mrsabath@gmail.com>
Two changes to KUBECON_NA_2026.md. **A new section 5: the M1-M4 MVP.** Section 4 is the full plan and stays as the record of why each item matters; section 5 is the subset that makes the talk honest and demo-backed, which is a smaller target than finishing Phase 2. Four items, roughly two days: fix #888 then #885, run kind-signed end to end, substitute the egress beat, and clear #889 plus the two stale doc claims. It cuts SPIRE and Phase 3 explicitly, defers #887 to an operational mitigation, and states the talk it buys -- three seams delivered and demo-backed, one named as the upgrade path with its obstacles. M2 is marked as the item that must not slip, because it is what converts "we wrote a verifier" into "we ran one". Section 2.6 is the argument: 575 passing tests did not catch #885 or #888, and section 4.4 step 2 records that the one comparable bug was found against a live broker and by no unit test. **A correction to W3.** The first version recommended a NetworkPolicy as the no-SPIRE MVP for the egress beat. That was written without checking the CNI. k8s/kind/kind-cluster.yaml creates a default Kind cluster, and kindnetd accepts NetworkPolicy objects while enforcing none of them -- so the demo would have shown a policy that appeared to work and blocked nothing. Three options are tabulated instead (swap the CNI, record on ykt1, or cut the beat), and M3 takes the third: substitute the #888 asymmetry, which is a real measured trust-scope failure needing no new infrastructure. Recorded in section 8 as observation 6, because it is the same shape as the thirteen -- a control claim true of the configuration imagined rather than the one that exists -- and because this document cited section 8.5.1 while breaking it. "Which code path, by name?" has to include which CNI, which broker, which cluster. The caveat note also marks the CNI finding as read rather than run, by the same rule. Renumbering: the 30-minute shape moves to section 6, risks to 7, observations to 8, change log to 9. Internal cross-references and the index entry follow. Rebased onto #883. No code, no manifests, no tests. Assisted-By: Claude Code Signed-off-by: Mariusz Sabath <mrsabath@gmail.com>
dce0264 to
3468bea
Compare
aslom
left a comment
There was a problem hiding this comment.
Docs-only: one new 714-line knowledge base plus an index row and a reading-order entry. No code, no manifests, no tests — matching the description. All 11 checks pass, DCO included, and both commits carry Signed-off-by and Assisted-By per CLAUDE.md.
I reviewed this the way the document asks to be reviewed — by checking its claims against the code rather than reading them — and the substance holds up unusually well. Spot-verified and confirmed:
- #888 —
kafka_in.pyreally does setok, why = True, "not checked"and then gate the decision behindif self._keyset is not None:, sorequireis unreachable without a keyset. Exactly as described. - #885 — the rejection rewrite builds a new
d, but routing isif ce.is_group_event(evt)on the original event, so a refusedgroup.completedstill reacheson_group_event. Also as described, including the reasoning that the fix is not to striptype/groupid— the code comment gives the same forensic rationale. - §2.7 —
signing.pyrejects anyalgbutEdDSA, andkeyset.loadrequires each value to be a string decoding to 32 bytes, so a JWKS document fails onmust be a string. EdDSA-only and JWKS-rejection both hold. - §2.2 canonicalization — length-prefixed fields and the newline-collision rationale match
canonical()precisely, down to the example. (IMPLEMENTATION_REPORT1.md§9 is the stale one here, still describing the oldkey=valueform — out of scope for this PR.) - §4/W3 CNI —
k8s/kind/kind-cluster.yamlhas nodisableDefaultCNIand there is no CNI override anywhere ineventing/k8s/, so the kindnetd conclusion is right. k8s_demo_flow.py—--overlaychoices really are exactly("test", "kind", "demo").- §2.5 — all five issues #885–#889 are open, with titles matching the table.
- Citations to
IMPLEMENTATION_REPORT1.md§7/§8,DESIGN_PHASE2.md§8.5.5 and §7, andemit.py's terminal-only signing policy all check out, quotes included.
That is a high hit rate, and §8 observation 6 — catching its own NetworkPolicy recommendation for skipping §8.5.1 on infrastructure — is the kind of thing that makes this document worth having in the tree.
The numbers are where it needs another pass. Every finding below is a figure or a pointer, not an argument, and they cluster around one root cause plus one sourcing gap:
1. The review was done against the pre-#883 tree, and the rebase invalidated the citations. All five code citations — kafka_in.py:93, :122, :136 and __main__.py:76 — match commit 3264e8d exactly, which is the commit immediately before #883. At this PR's actual base they are 109/110, 138, 152 and 80. The substance is unaffected; the pointers are off by +16 and +4. Since this file is meant to be the citable reference and M1 says "both call sites were read", a reader following :93 lands in a comment block.
2. The test count is contradicted by this PR's own CI. The doc says 575 passed, 5 skipped. The eventing-test job on 3468bea collected 904 items and reported 896 passed, 8 skipped. 575/5 is very likely the pre-#883 number too — #883 added the Phase 3 tenancy tests. This one matters more than a stray figure, because 575 is load-bearing twice: it is M2's argument ("575 passing tests did not catch #885 or #888") and M4's instruction to overwrite DESIGN_PHASE2.md §6.
3. The headline cluster timings are not sourced. 1.7–3.0 s, ~14 s and 22 s appear nowhere else in the repository, and IMPLEMENTATION_REPORT1.md §3 records materially different measurements for the same things. The Caveat says cluster claims come from that document; these do not, and §6's four-minute demo budget is built on them.
4. An estimate is quoted where a measurement exists. ~150–200 ms traces to emit.py/DESIGN_PHASE2.md prose, but §9 of IMPLEMENTATION_REPORT1.md measured 222 ms / 227 ms — above the quoted range. §6.6 and M2 both read as though no measurement exists.
Given the talk's thesis, (2) and (4) are the ones I would not want an attendee to find first: a stale test count in the document arguing that test counts are the cheapest claim to keep honest, and a docstring estimate preferred over a recorded measurement in a document whose §8.6 rule is "run it, don't read it."
Two smaller things worth fixing in the same pass: the SPIRE-absence claim is scoped to "the tree" when it is only true of eventing/ (this repo ships a SPIFFE-header-authenticated MCP tool), and the document defers its own <!-- VERIFY --> markers to W5 while M4's honesty pass omits this file — which matters because M3 depends on #888 staying demonstrable.
Approving. None of this is structural, and none of it is a reason to hold the document out of the tree — it is far more useful in agentdocs/ being corrected than sitting in a branch. Scope, placement and rationale are right, the separate-PR argument is convincing, and I agree with Option A and with M3's substitution.
Treat the two [high] items as follow-ups to land before anything in this file reaches a slide: re-derive the five code citations and the test count against 0667270. The figures in (3) and (4) want a source or a correction in the same pass. Everything else is optional.
Inline comments below, severity-tagged.
| Test suite at review time: **575 passed, 5 skipped**. (`DESIGN_PHASE2.md` §6 says | ||
| 490/5 — the suite has grown; the doc is stale, not wrong.) |
There was a problem hiding this comment.
[high] The test count is contradicted by this PR's own CI run.
eventing-test on 3468bea collected 904 items and reported 896 passed, 8 skipped:
rootdir: /home/runner/work/examples/examples/eventing
collecting ... collected 904 items
================== 896 passed, 8 skipped in 86.69s (0:01:26) ===================
575/5 looks like the pre-#883 number, same as the code citations below — #883 added the Phase 3 tenancy tests.
This one is worth more than a figure fix, because 575 is load-bearing twice: §5/M2 argues "575 passing tests did not catch #885 or #888", and §4/W5 + §5/M4 tell a maintainer to propagate it into DESIGN_PHASE2.md §6. The argument survives at 896 — it gets stronger — but the number has to be right in a document whose own thesis is that a test count is the cheapest claim to keep honest.
| Test suite at review time: **575 passed, 5 skipped**. (`DESIGN_PHASE2.md` §6 says | |
| 490/5 — the suite has grown; the doc is stale, not wrong.) | |
| Test suite at review time: **896 passed, 8 skipped** (`eventing-test` on this PR's head; | |
| `python -m pytest -v` with rootdir `eventing/`, 904 collected). | |
| (`DESIGN_PHASE2.md` §6 says 490/5 — the suite has grown; see §5/M4 before editing it.) |
There was a problem hiding this comment.
Fixed in 8846e9f, and running it myself turned up something worth recording: the pass count is environment-dependent, so neither 575/5 nor 896/8 is reproducible on its own.
At this PR's base, on a laptop with claude on PATH:
904 tests collected
899 passed, 5 skipped in 63.95s
CI reports 896/8 on the same tree. The three-test gap is two gates that pass locally and skip on the runner:
test_agentspec.py:370—skipif(shutil.which("claude") is None), the pinned-CLI checktest_graceful_shutdown.py:46— needs a reachable Kafka onlocalhost:9092
The five constant skips are test_manifests.py's no reachable cluster cases, which is the same set in both environments.
So the document now quotes 904 collected plus both environments and why they differ, rather than a bare pass figure. Your point that 575 was the pre-#883 number is right — it has the same provenance as the line citations below, and #883 added the Phase 3 tenancy tests.
Both load-bearing uses now read "~900 passing tests did not catch #885 or #888". You noted the argument survives at the higher number; it does, and I would say it reads harder — nine hundred tests is a more uncomfortable sentence than five hundred.
|
|
||
| | Seam | Named CNCF integration | Code status | | ||
| |---|---|---| | ||
| | Cold-start identity gap | SPIRE → KEDA | ❌ **No SPIRE/SPIFFE/SVID in the implementation.** 7 mentions total, all prose, all aspirational | |
There was a problem hiding this comment.
[medium] "seven mentions across the tree, all prose, all aspirational" is true of eventing/, but not of the tree — and the gap is the kind a sharp attendee greps for.
Within eventing/, excluding this file, there are 6 pre-existing mentions (DESIGN_PHASE2.md ×3, DESIGN_PHASE3.md ×2, keyset.py ×1) and all are prose. Correct and on point.
Repo-wide it is ~84, because mcp/wiki_memory_tool/ ships SPIFFE workload identity as executable code, not prose — wiki_service.py parses X-Spiffe-Id headers and resolves topics from the SPIFFE ID, k8s/acl-configmap.yaml lists spiffe://… identities as ACL subjects, and there are tests. Its own README says "No real SPIRE SVID verification (header-based, not mTLS)", so it does not undercut §3's decision — but "no SPIFFE in the tree, all prose" is checkable in one grep and does not hold.
Scoping the claim fixes it, and the sibling component is probably worth one sentence in §3 so the question is answered before it is asked.
| | Cold-start identity gap | SPIRE → KEDA | ❌ **No SPIRE/SPIFFE/SVID in the implementation.** 7 mentions total, all prose, all aspirational | | |
| | Cold-start identity gap | SPIRE → KEDA | ❌ **No SPIRE/SPIFFE/SVID in `eventing/`.** 6 mentions, all prose, all aspirational. (`mcp/wiki_memory_tool/` does have header-simulated SPIFFE identity — a different component, and its own README notes there is no real SVID verification) | |
There was a problem hiding this comment.
Scoped in 8846e9f, with the sibling component named as you suggested.
Counted it both ways. In eventing/, excluding this file: 8 mentions across 4 files — DESIGN_PHASE2.md ×3, agentdocs/README.md ×2, DESIGN_PHASE3.md ×2, shared/keyset.py ×1 — all prose. Repo-wide it is ~95, and mcp/wiki_memory_tool/ is nearly all of it: 21 in wiki_service.py, 9 in k8s/acl-configmap.yaml, plus skills and tests.
Your framing of why it matters is the part I took: it is checkable in one grep, and an attendee who runs that grep finds executable SPIFFE code in a repo whose slide says there is none. The row now says eventing/ and notes that the sibling component's own README disclaims real SVID verification, so §3's decision stands either way.
| - **KEDA scale-to-zero, measured on two clusters.** OpenShift and Kind. POST → KEDA | ||
| decision 1.7–3.0 s; POST → pod Ready ~14 s; full cycle back to zero 22 s. |
There was a problem hiding this comment.
[medium] These three figures appear nowhere else in the repository, and IMPLEMENTATION_REPORT1.md §3 records materially different numbers for the same measurements.
§3's "Measurements against the design's assumptions":
| Measured there | Stated here | |
|---|---|---|
| Wake latency (RQ-4) | 1.0–5.0 s on ykt1, 2.2–4.3 s on Kind | KEDA decision 1.7–3.0 s |
| Scale-to-zero | 30–31 s with cooldownPeriod: 30 |
full cycle back to zero 22 s |
| POST → pod Ready | (no such figure) | ~14 s |
| Image pull | 5.1 s cold (245 MB), 454 ms–2.45 s warm | (§6.1: "~12 s of the ~14 s wake") |
The Caveat (§8) says everything about cluster behaviour comes from IMPLEMENTATION_REPORT1.md — these do not, and I could not find them in any other agentdocs/ file or anywhere in the tree.
This propagates: §6's opening uses ~14 s and 22 s to justify pre-recording, and §6.1's "image pull is ~12 s of the ~14 s wake" conflicts with §3's 5.1 s cold pull. The pre-record recommendation is right regardless — a 6-minute mock run and a 429 risk carry it on their own — so only the numbers need changing, but they are headed for a slide.
Worth also saying which cluster each number is from, since §4's own sequencing note warns Kind and cluster timings are not comparable.
There was a problem hiding this comment.
Replaced with §3's measured figures in 8846e9f. I could not find a source for the three either, and a figure nobody can trace does not belong in a document whose §8 caveat says cluster claims come from IMPLEMENTATION_REPORT1.md.
§2.2 now carries §3's numbers with their cluster: wake 1.0–5.0 s on ykt1, 2.2–4.3 s on Kind; back to zero 30–31 s at cooldownPeriod: 30; image pull 5.1 s cold (245 MB), 454 ms–2.45 s warm. The D5 row reads 30–31 s rather than 22 s.
The two downstream uses are fixed too, and your catch on §6.1 was the sharper one — "image pull is ~12 s of the ~14 s wake" conflicted with §3's 5.1 s cold pull, so the derived figure was wrong in a second way beyond being unsourced. It now gives the cold and warm pull separately.
I left the three superseded numbers named in §2.2 rather than deleting them silently, so they do not come back the next time someone drafts a slide from an older copy. Per your note on §4's sequencing warning, each figure now says which cluster it is from.
| **(D) is a deliberate policy, not a bug.** `emit()` signs terminal events only, because | ||
| it runs per `stdout` frame and Ed25519 costs ~150–200 ms here; signing every frame would | ||
| add minutes to a chatty run. The verifier matches that policy. The honest limit: this | ||
| proves *who finished a run*, not *what it said along the way*. Say that on stage. |
There was a problem hiding this comment.
[medium] ~150–200 ms is the docstring estimate; the repository has a measurement, and it is higher.
IMPLEMENTATION_REPORT1.md §9 measured on-laptop:
sign_event() |
222 ms |
verify_signature() |
227 ms |
and draws the per-turn consequence: "At ~3-10 response events per turn that is 0.7-2.2 s of pure signing overhead per turn."
~150–200 ms traces to emit.py and DESIGN_PHASE2.md prose, so quoting it is faithful to the docstring — but it understates the one number that was actually run, which is the inversion §8.6 warns about. It appears four times here (§2.4, §4/W2, §5/M2, §6.6), and §6.6's prepared answer "~150–200 ms/op. Get the on-cluster number in M2" reads as though no measurement exists. M2 should be framed as confirming 222/227 ms on-cluster, not as first contact.
There was a problem hiding this comment.
All four sites now cite the measurement (8846e9f), and §6.6 no longer reads as though none exists. One wrinkle worth recording, because it slightly changes the framing.
I re-measured rather than copying either number — driving sign_event / verify_signature 20× each on this laptop:
sign_event min 188.8 ms median 190.6 ms max 194.3 ms
verify_signature min 194.5 ms median 197.8 ms max 244.4 ms
So §9's 222/227 ms is real but hardware-dependent, and on this machine the median lands inside the ~150–200 ms range rather than above it. Your conclusion holds — quoting a docstring estimate over a recorded measurement is the inversion §8.6 warns about, and §9's figure is the one with a method behind it — but "the estimate understates the one number that was actually run" is only true on the hardware §9 was run on.
The document now gives 222/227 ms as the recorded measurement and ~190–230 ms as the pure-Python band with hardware as the variable, so whoever runs M2 knows a cluster number outside the range is not necessarily a regression. M2 is framed as confirming on cluster hardware, not first contact, per your point.
cryptography doing this in microseconds stays the real argument, and that is unaffected either way.
| **(E) is the one that will bite on stage.** `signing.response_decision` correctly refuses | ||
| the forged `group.completed`. But `kafka_in.py:122` rewrites only `phase` and `data` — it |
There was a problem hiding this comment.
[high] Stale line numbers — this and §5/M1's citations were all derived against the tree before #883, which is this PR's base.
Every one of them matches commit 3264e8d (the commit immediately preceding #883) exactly:
| Cited | At 3264e8d |
At this PR's base 0667270 |
|---|---|---|
kafka_in.py:93 (§5/M1) |
✅ ok, why = True, "not checked" |
109 |
kafka_in.py:94 (§5/M1) |
✅ if self._keyset is not None: |
110 |
kafka_in.py:122 (here) |
✅ d = dict(d, phase="error", data={ |
138 |
kafka_in.py:136 (§2.4, §5/M1) |
✅ if ce.is_group_event(evt): |
152 |
__main__.py:76 (§5/M1) |
✅ response verification OFF |
80 |
Consistent +16 in kafka_in.py, +4 in __main__.py — "Rebased onto #883" moved the code without re-deriving the pointers.
The findings themselves all still hold at the new lines; I verified each against the rebased code. But this file is meant to be the citable reference, §5/M1 says "both call sites were read", and :93 now lands inside the comment block explaining why the try is not belt-and-braces. Worth a sweep for any other path:line in the file.
| **(E) is the one that will bite on stage.** `signing.response_decision` correctly refuses | |
| the forged `group.completed`. But `kafka_in.py:122` rewrites only `phase` and `data` — it | |
| **(E) is the one that will bite on stage.** `signing.response_decision` correctly refuses | |
| the forged `group.completed`. But `kafka_in.py:138` rewrites only `phase` and `data` — it |
There was a problem hiding this comment.
Re-derived in 8846e9f. Your 3264e8d diagnosis is exactly right, including the +16 / +4 offsets — I verified each against the base rather than applying the offset:
| Cited | Now | Content at the new line |
|---|---|---|
kafka_in.py:93 |
109 | ok, why = True, "not checked" |
kafka_in.py:94 |
110 | if self._keyset is not None: |
kafka_in.py:122 |
138 | d = dict(d, phase="error", data={ |
kafka_in.py:136 |
152 | if ce.is_group_event(evt): |
__main__.py:76 |
80 | response verification OFF |
§5/M1 now states the commit the pointers are against — 0667270, which git merge-base confirms is this PR's base and is #883 — and says to re-derive rather than adjust after a rebase. Adjusting by a constant offset is how a citation ends up plausibly wrong instead of obviously wrong, which is worse for a file meant to be the citable reference.
That :93 landed inside the comment block explaining why the try is not belt-and-braces is the detail that makes this more than cosmetic — a reader following the pointer concludes the finding is about error handling.
I swept the file; those five were all of them.
| - Add the `<!-- VERIFY -->` comments §8.5.5 prescribes to any statement in this file that | ||
| depends on #885–#889 staying open. That is this document eating its own cooking. |
There was a problem hiding this comment.
[medium] This file ships without the markers it prescribes for itself, and M4's honesty pass does not include it.
There are no <!-- VERIFY --> comments anywhere in the document — the only two VERIFY hits are EB_VERIFY_KEYSET_PATH at line 198 and this line. Meanwhile a large share of the document expires when #885–#889 are fixed: all of §2.4's five-line table, §2.5, §5/M1, and §6.3's slide.
The sharpest case is M3. It cuts D3 and substitutes the #888 asymmetry as a demo beat, so M3 depends on #888 staying reproducible — and M1 fixes #888. The sequencing resolves it (demo the fix landing), but nothing in the file says so to a maintainer who fixes #888 first and finds the egress beat substituted with a bug that no longer reproduces.
Since §4/W5 is scheduled after the talk and §5/M4's honesty pass lists only keyset.py and the test count, "this document eating its own cooking" never actually happens. Adding the markers now is cheap — per §8.5.5 they should name the release and the issue, and say what should change:
| - Add the `<!-- VERIFY -->` comments §8.5.5 prescribes to any statement in this file that | |
| depends on #885–#889 staying open. That is this document eating its own cooking. | |
| - Add the `<!-- VERIFY -->` comments §8.5.5 prescribes to any statement in this file that | |
| depends on #885-#889 staying open — §2.4's table, §2.5, §5/M1 and §6.3's slide, and in | |
| particular §5/M3, whose substituted demo beat stops reproducing once M1 lands. That is | |
| this document eating its own cooking, and it is cheap enough to do now rather than in W5. |
There was a problem hiding this comment.
Markers added in 8846e9f, and moved from W5 into M4 — which was the real bug you found here. W5 is scheduled after the talk, so a document whose §5/M3 depends on #888 staying open had its own expiry note landing after the expiry.
Six <!-- VERIFY --> comments now, each naming the release, the issue and what should change per DESIGN_PHASE2.md §8.5.5:
- §2.4 — the five-line table; says what changes when 🐛 eventing: EB_REQUIRE_RESPONSE_SIGNATURE without a keyset refuses nothing and reports nothing #888 lands, and not to delete the table because it is M3's demo beat
- §2.5 — "all still open" is the shortest-lived claim in the file
- §5/M1 — the citation provenance
- §5/M3 — the one you identified
- §6.3 — say "thirteen found", not "thirteen outstanding", so the slide number survives the fixes
M3's is worded as you implied it should be: run M1 and M3 in order and the beat becomes "watch the fix land"; find #888 already closed and reframe as before/after rather than delete, because the asymmetry is the point, not the bug. That is the instruction a maintainer needs and the one a vague marker would lose.
W5 now points at M4 having done it rather than repeating the instruction.
| - **Docs:** correct `keyset.py`'s "the upgrade path changes no verification logic" claim | ||
| (false per §8.3 — EdDSA-only against SPIRE's EC/RSA, and `keyset.load` rejects JWKS), | ||
| and `DESIGN_PHASE2.md` §6's `490/5` test count. Small, but in a repository whose §8 is | ||
| about claims that quietly stopped being true, a test count is the cheapest one to keep | ||
| honest. |
There was a problem hiding this comment.
[low] Replacing §6's test count in place would damage a historical record and break its arithmetic.
DESIGN_PHASE2.md §6 reads: "Tests: 490 passed, 5 skipped. Baseline on the same tree is 450/5, so this adds 40 and regresses nothing." That is a delta measurement of what Phase 2 contributed, not a live claim about the suite — overwrite 490 with the current number and the "adds 40 over 450/5" sentence stops making sense.
It also runs against the convention the agentdocs README states for exactly this: "where Phase 1 supersedes them it says so explicitly rather than editing them in place."
§2 already has the right instinct — "the doc is stale, not wrong" — but a historical measurement is not stale at all. A footnote preserves both:
| - **Docs:** correct `keyset.py`'s "the upgrade path changes no verification logic" claim | |
| (false per §8.3 — EdDSA-only against SPIRE's EC/RSA, and `keyset.load` rejects JWKS), | |
| and `DESIGN_PHASE2.md` §6's `490/5` test count. Small, but in a repository whose §8 is | |
| about claims that quietly stopped being true, a test count is the cheapest one to keep | |
| honest. | |
| - **Docs:** correct `keyset.py`'s "the upgrade path changes no verification logic" claim | |
| (false per §8.3 — EdDSA-only against SPIRE's EC/RSA, and `keyset.load` rejects JWKS). | |
| Leave `DESIGN_PHASE2.md` §6's `490/5` as the Phase 2 measurement it is — its "adds 40 | |
| over a 450/5 baseline" arithmetic depends on it, and the `agentdocs` README says phase | |
| records are not edited in place. Add a footnote pointing at §2's current count instead. |
There was a problem hiding this comment.
Taken as suggested in 8846e9f — and you are right that this is not a stale claim at all.
Confirmed the arithmetic before changing it. DESIGN_PHASE2.md §6 reads:
Tests: 490 passed, 5 skipped. Baseline on the same tree is 450/5, so this adds 40 and regresses nothing.
Overwrite 490 with ~900 and "adds 40 over 450/5" becomes nonsense, so the instruction I wrote would have destroyed a measurement to fix something that was never wrong. A delta measured against a stated baseline on a stated tree stays true permanently; that is the opposite of the §8 failure mode, and I had it filed under the same heading.
Both W5 and M4 now call for a footnote pointing at §2's current count, and cite the agentdocs README convention you quoted — a phase record is superseded explicitly rather than edited in place.
| The CNI finding in §8.6 was reached by reading `k8s/kind/kind-cluster.yaml`, not by | ||
| applying a NetworkPolicy to a running Kind cluster and watching it fail to block. That | ||
| is the right direction for the conclusion — kindnetd's lack of NetworkPolicy support is | ||
| documented upstream — but by this document's own §8.6 rule it is still a claim that has | ||
| been read rather than run. Confirm it before spending anything on either CNI option. |
There was a problem hiding this comment.
[low] §8.6 does not resolve in this document, and it means two different things in adjacent sentences.
§8 is a flat six-item numbered list with no subsections, so there is no §8.6 here — the CNI finding is §8 observation 6. The commit message gets this right ("Recorded in section 8 as observation 6"). The next sentence's "this document's own §8.6 rule" then means DESIGN_PHASE2.md §8.6 ("Run it — reading the code is not enough"), which is not this document's and is a different §8.6 from the one three lines up.
Everywhere else in the file §8.6 unambiguously means Phase 2's (§2 line 78, §5/M1 line 420), so this is the only ambiguous pair — likely fallout from the §7→§8 renumbering.
| The CNI finding in §8.6 was reached by reading `k8s/kind/kind-cluster.yaml`, not by | |
| applying a NetworkPolicy to a running Kind cluster and watching it fail to block. That | |
| is the right direction for the conclusion — kindnetd's lack of NetworkPolicy support is | |
| documented upstream — but by this document's own §8.6 rule it is still a claim that has | |
| been read rather than run. Confirm it before spending anything on either CNI option. | |
| The CNI finding in §8 observation 6 was reached by reading `k8s/kind/kind-cluster.yaml`, | |
| not by applying a NetworkPolicy to a running Kind cluster and watching it fail to block. | |
| That is the right direction for the conclusion — kindnetd's lack of NetworkPolicy support | |
| is documented upstream — but by `DESIGN_PHASE2.md` §8.6's rule it is still a claim that | |
| has been read rather than run. Confirm it before spending anything on either CNI option. |
There was a problem hiding this comment.
Fixed in 8846e9f, and I widened it slightly: every cross-document § reference in the file is now qualified, not just this pair.
Confirmed the diagnosis first — this file's §8 is a flat six-item numbered list with no subsections, so there is no §8.6 here at all. The CNI sentence now reads "§8 observation 6" (matching the commit message, as you noted) and the next one reads "DESIGN_PHASE2.md §8.6's rule".
The same bare-§8.6 ambiguity was at three other sites — §2's method line, §4/W4's test rule, and §6.3's two-rules line — all meaning Phase 2's. Those are now qualified too. Likely the same §7→§8 renumbering fallout you suspected, since before it this document had no §8 and a bare reference was unambiguous.
| | [`IMPLEMENTATION_REPORT1.md`](IMPLEMENTATION_REPORT1.md) | What Phase 1 built, the measured results on both clusters, 28 findings including two real bugs only a scale-to-zero deployment could expose, and an honest account of what is still blocked and why. | | ||
| | [`README_PHASE1.md`](README_PHASE1.md) | The Phase 1 runbook in full detail, covering all four ways to run it: locally without containers, under Docker, on a Kind cluster, and on OpenShift. | | ||
| | [`DESIGN_PHASE2.md`](DESIGN_PHASE2.md) | The Phase 2 design — identity on the event path. Why GitHub's opaque user token rules out local verification and forces a `GET /user` lookup plus a load-bearing cache; why `401` and `403` are kept distinct; what `ce_submitter` is and is not worth, and what a signature over it does and does not prove; how the `kid` and the approved-key set prove *which agent* answered, why group events are pinned to EventBridge's own key, and why a rejected response is stored as an error rather than dropped. **§8 records the thirteen claims across §§2.6, 3, 4.3 and 4.4 that a docs review found do not hold in the code, with the issue filed for each — and §8.5-§8.8 the procedure that came out of them: the five questions to ask of any control claim, in any phase, before it ships.** | | ||
| | [`KUBECON_NA_2026.md`](KUBECON_NA_2026.md) | **The KubeCon NA knowledge base.** The accepted abstract and the five demo beats it commits to; the status of each against the code as of 2026-10-05, including what was confirmed by *running* the decision functions rather than reading them; the SPIRE decision that shapes everything else (two of the four seams name it and there is no SPIRE); the W1-W6 plan and the **M1-M4 MVP** cut of it; and the 30-minute slide and demo shape, with the recommendation to pre-record. Cites the other documents rather than restating them. | |
There was a problem hiding this comment.
[low] Since this row is the index entry a reader lands on first, the "as of 2026-10-05" date will go stale at a different rate from the rest of the table — the other rows describe finished phases, this one describes a living document with a deadline.
Worth marking it as such here, the way the Phase 3 rows carry their Partly implemented / Partial caveats, so the index itself says this entry expires after the talk. The document's own header says "knowledge base, living document"; the index does not.
Not blocking — and if the status figures get corrected per the other comments, this row's "the status of each against the code as of 2026-10-05" stays accurate either way.
There was a problem hiding this comment.
Marked in 8846e9f. The row now ends:
A living document with a deadline, unlike the phase records above — its status figures and its "all still open" bug list expire at the talk.
Your framing is the useful bit: every other row describes a finished phase, so a reader reasonably assumes the whole table ages at the same rate. Saying it in the index rather than only in the document's own header is what makes the difference, since the index is where someone decides whether to trust a figure.
Dobra uwaga — trzeba było to powiedzieć tam, gdzie czytelnik patrzy najpierw. (Good catch — it needed saying where the reader looks first.)
M1 and M2 are filed as #895 and #896. Section 5 gains a tracking table so the MVP's status is visible without reading all four subsections, and the two subsections that now have issues link to them. M3 has no issue by design -- it is a decision about what to demo, not work to schedule. M4 tracks against the existing #889. Assisted-By: Claude Code Signed-off-by: Mariusz Sabath <mrsabath@gmail.com>
…nst the base All eight findings from @aslom's #893 review. Every figure in this file was either re-measured or traced to the document that records it; none was carried over from the draft. **The code citations were derived against the pre-#883 tree.** `kafka_in.py:93`, `:94`, `:122`, `:136` and `__main__.py:76` all match `3264e8d`, the commit before #883 — this PR's base is `0667270`. Re-derived at the base: 109/110, 138, 152 and 80. The findings hold at the new lines; only the pointers were wrong, and `:93` landed inside a comment block. §5/M1 now states the commit the pointers are against and says to re-derive rather than adjust them after a rebase. **The test count was stale and the replacement is environment-dependent.** 575/5 was also a pre-#883 number. Running the suite at this base gives **899 passed, 5 skipped** on a laptop, where CI reports 896/8 — the three extra CI skips are `test_agentspec.py`'s pinned-CLI check (needs `claude` on `PATH`) and `test_graceful_shutdown.py` (needs a reachable Kafka); the five constant ones are `test_manifests.py`'s `no reachable cluster` cases. The document now quotes the collected count, 904, plus the environment, since a bare pass figure is not reproducible. The two load-bearing uses ("575 passing tests did not catch #885 or #888") read ~900, which strengthens the argument. **Three cluster timings were unsourced.** `1.7–3.0 s`, `~14 s` and `22 s` appear in no other file in the repository, while §8's own caveat says cluster claims come from `IMPLEMENTATION_REPORT1.md`. Replaced with §3's measured figures — wake 1.0–5.0 s on ykt1, 2.2–4.3 s on Kind, 30–31 s back to zero at `cooldownPeriod: 30`, image pull 5.1 s cold / 454 ms–2.45 s warm — each attributed to its cluster, per §4's own note that the two are not comparable. §6's demo budget and §6.1's pull figure follow. The superseded numbers are named so they do not reappear. **An estimate was quoted where a measurement exists.** `~150–200 ms` is the docstring estimate; `IMPLEMENTATION_REPORT1.md` §9 measured 222 ms / 227 ms. Re-running it here gave 190/198 ms median, so the band is ~190–230 ms and the hardware is the variable — worth saying, because the review framed the estimate as understating the measurement and on this machine it does not. All four sites now cite the measurement, and §6.6's prepared answer no longer reads as though none exists. M2 is framed as confirming on cluster hardware, not first contact. **The SPIFFE claim was scoped to the tree.** True of `eventing/` (8 mentions across 4 files, all prose); repo-wide it is ~95, because `mcp/wiki_memory_tool/` ships header-simulated SPIFFE identity as executable code. Scoped, with the sibling component named so the question is answered before an attendee greps. **The file prescribed `<!-- VERIFY -->` markers it did not carry.** Added at the five places that expire when #885–#889 land: §2.4, §2.5, §5/M1, §5/M3 and §6.3. §5/M3 is the sharp one — it substitutes the #888 asymmetry as a demo beat, and §5/M1 fixes #888, so the marker says to reframe rather than delete. Moved from W5 (after the talk) into M4. **Overwriting `DESIGN_PHASE2.md` §6's test count would have broken it.** `490/5` is Phase 2's delta measurement — "adds 40 over a 450/5 baseline on the same tree" — not a live claim, and the `agentdocs` README says a phase record is superseded explicitly rather than edited in place. W5 and M4 now call for a footnote. Also: `§8.6` meant this document's §8 observation 6 in one sentence and Phase 2's §8.6 in the next; every cross-document reference is now qualified. The `agentdocs/README.md` row says this is a living document with a deadline, unlike the phase records around it. 899 passed, 5 skipped at this base, unchanged. Docs only. Assisted-By: Claude Code Signed-off-by: Mariusz Sabath <mrsabath@gmail.com>
…ermission (#897) * docs(eventing): Phase 4 design — enrollment, and asking a human for permission Two gaps Phase 2 and Phase 3 leave, which are the two halves of one idea: the notification channel is bidirectional. Phase 3 §4.2 derives an unguessable per-user ntfy topic and §4.3 provisions the reader account, but both are operator-side — a user who signs in with GitHub has no way to learn their own topic, and the token is printed to the *operator's* terminal. And every notification so far has been an announcement: Phase 3 §5.1 fixes each agent's permission mode and tool allowlist before the run, so an agent that needs a permission it was not granted cannot ask and a human cannot answer. There is no approval vocabulary anywhere in the tree. That second gap is what makes Phase 3's triggers risky to enable. §7.4 concedes a trigger's rendered prompt is attacker-influenced text and §5.1 answers with default-deny tools; default-deny with no escalation path means the operator's rational move is to widen the allowlist permanently for a capability needed once. An approval path is what keeps a narrow allowlist narrow. §2 is the measured part, and the rest depends on it Phase 2 §8.6's rule earned its keep twice. Against the CLI: a PreToolUse hook blocks 75 s with the tool not running (93 s wall clock), honours a per-hook "timeout": 120, refuses on `deny`, and delivers permissionDecisionReason to the agent — which then explained the refusal in its answer. `allow` runs the tool. Two negative findings, both attractive wrong turns. `--permission-prompt-tool` does not exist; the flag is `--permission-prompts <host|none>` and its own help defines `host` as the SDK host, so it needs the Agent SDK in-process — as does `canUseTool`. EventRunner spawns a subprocess, so the hook is the only mechanism available to it. And `plan` mode never attempts the guarded call at all: the agent wrote a plan and reached for ExitPlanMode, so the hook adjudicated nothing. Phase 3 §5.1 proposes `plan` as the safe default; for an agent subject to approval it is the wrong mode. The hook needs no new plumbing — AgentSpec already carries `settings_file`, resolved under the agent dir by `_resolve_under` and passed as `--settings`, and Phase 3 §5.2 already requires that directory be unwritable by the agent. The control that matters most Phase 3 §4.4 MACs `userkey|corr|exp`, which is right for continuing a conversation. Authorising an *action* has to bind the action: the approval key covers a digest of the canonical tool input, and the hook re-checks that digest before returning `allow`. Without it a key that approved `echo hi` could be replayed against a rewritten command on the same approval id. The verdict is a row read from the store, never an attribute off the event — Phase 2 §8.5.4, whose cautionary tale is group-member push suppression being defeated by omitting the attribute it reads. Once-only is a guarded UPDATE on `decided_utc IS NULL`, the pattern mark_member_finished already uses. A timeout is recorded as `timeout`, not `deny`: "nobody was awake" and "a human said no" need different fixes, and collapsing them hides which is happening. The budget inequality is pinned, because a killed hook is a non-blocking error — execution proceeds, i.e. the tool runs unapproved. Three ntfy action buttons is a hard ceiling and `_publish` already fills it, so an approval cannot be the result notification with a button added. §9 answers Phase 2 §8.5's five questions before this ships, including the hole that remains: the hook runs in the pod the agent runs in, so the mitigation is a filesystem permission rather than a cryptographic control. Design only — no code, no manifests, no tests, matching #892/#893 and the convention that a phase design lands as a reviewable record first. Nothing in it is implemented; §11 orders the tasks behind Phase 3's ntfy isolation and capability key. Not verified: no cluster access, and §2 was measured on CLI 2.1.270 while Dockerfile-eventrunner-claude pins 2.1.278 — §10 says so, and §9.1's pinned-CLI test is the first thing to build. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Mariusz Sabath <mrsabath@gmail.com> * docs(eventing): Phase 4 — address review of #897 Review: #897 (approved), five inline comments — two substantive, three nits. - §5.7's budget chain now carries §6's bound: the hook's wait, not just the turn, fits inside ER_DRAIN_TIMEOUT_S < terminationGracePeriodSeconds. The paragraph says why that link matters — ER_APPROVAL_TIMEOUT_S is the knob an operator raises when two minutes is not enough, and at 600 it outlives the drain budget, so a rolling update lands a SIGKILL mid-approval and §6's late-approval path replays a turn whose partition was taken away mid-flight. The startup check prints all five values; test_manifests.py's drain test is named as gaining the one-more-assert (ER_APPROVAL_TIMEOUT_S < ER_DRAIN_TIMEOUT_S, in the rendered ConfigMap). - §7.2's ER_APPROVAL_TIMEOUT_S row: "below the per-hook timeout AND below ER_DRAIN_TIMEOUT_S", pointing at §5.7's full chain. - §5.6 gains the missing 503 row (EB_CAPABILITY_SECRET_PATH empty, §7.1), distinguishing it from 404: the endpoint exists, the remedy is the operator's. - §3.4 and §6: dropped the duplicated "Phase 1" / "Phase 3" across the line wraps. - §9.5's VERIFY marker: dropped the ```markdown fence so both instances render the same (invisible HTML comments); grep-ability unchanged. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Mariusz Sabath <mrsabath@gmail.com> --------- Signed-off-by: Mariusz Sabath <mrsabath@gmail.com>
Phase / scope
agentdocsindex. No code, no manifests, no tests.DESIGN_PHASE4.mdwould make the phase sequence say something false. It also has a different lifecycle: a living document that goes stale after the talk, where the designs are records that do not.DESIGN_PHASE2.md§8) — disjoint files.Adds
eventing/agentdocs/KUBECON_NA_2026.md. Four parts: the abstract as submittedwith its five demo beats broken out and numbered; the status of each against the tree;
the W1-W6 plan; and the 30-minute slide and demo shape.
Kept here rather than in a slide deck or a scratch directory for the reason the
agentdocsREADME already gives: so the claims in it can be checked against the codethey describe. A talk abstract is a set of control claims with a deadline.
The two gaps the plan is ordered around
Two of the abstract's four seams name SPIRE — "the agent egresses before SPIRE issues
an SVID" and "sidecar policy reading SVIDs at runtime". There is no SPIRE, SPIFFE or
SVID in the implementation: seven mentions across the tree, all prose, all
aspirational. And the demo list promises "blocked egress on a wrong-scope call", which
has no
NetworkPolicy, Istio,AuthorizationPolicyor sidecar behind it at all.§3 is therefore its own section: the choice between wiring a minimal SPIRE and
reframing it as the named upgrade path decides both the plan and the slides, and it
cannot be deferred. It recommends reframing, and says why in one line — the talk's
thesis is that unchecked control claims are the trap, so overselling the guarantee in a
talk about overselling guarantees is the one failure mode with no recovery.
§2.7 also records, from §8.3, that the upgrade path does not currently hold as
keyset.pystates it: the verifier accepts EdDSA only, SPIRE issues EC or RSA, andkeyset.loadrejects a JWKS document outright. The kid lookup survives; the algorithmand the file format do not.
The status section was run, not read
§8.6's rule, applied to this document's own claims. It changed two conclusions that
reading the docstrings had produced:
group.completedwith enforcement on is refused by the verifier and thenapplied anyway — traced from
kafka_in.py:122(which rewritesphaseanddatabut preserves
typeandgroupid) throughis_group_event(evt)on the originalevent, to
on_group_event, which reads only those two fields and callscomplete_group(). 🐛 eventing: response-signature enforcement is advisory — rejected events still end batches and overwrite verified answers #885, confirmed end to end rather than inferred.EB_REQUIRE_RESPONSE_SIGNATURE=truewith no keyset refuses nothing while printingresponse verification OFF, where the request side with no key refuses everything.🐛 eventing: EB_REQUIRE_RESPONSE_SIGNATURE without a keyset refuses nothing and reports nothing #888.
An unsigned non-terminal frame is also accepted, and §2.4 records it as the signing
policy on the other side rather than as a hole —
emit()signs terminals only becauseEd25519 costs ~150-200 ms here and signing every
stdoutframe would add minutes to achatty run.
Two things it records that no document currently states
k8s/overlays/kind-signed/exists,but no document records a run and
k8s_demo_flow.py --overlayaccepts onlytest/kind/demo, so it cannot drive it. §4.4 step 2 is the precedent that makesthis matter: the signer/verifier policy mismatch was found against a live broker and
by no unit test.
DESIGN_PHASE2.md§6 says 490. Stale rather thanwrong, but flagged in W5 — in a repository whose §8 is about claims that quietly
stopped being true, a test count is the cheapest one to keep honest.
§7 keeps a false negative of its own
The first group-forgery harness used
io.rossoctl.…event types instead ofce.TYPE_GROUP_COMPLETEDand reported no issue. The finding only appeared on the secondattempt. Recorded with the generalisation, because it is the same shape as §8.8's
"a suggested wording is a hypothesis": a negative result from a hand-written harness is
worth little until the harness has produced a positive one — and whoever re-verifies
that case should not trust a clean run.
Caveat
Written without cluster access, so everything stated about cluster behaviour comes from
IMPLEMENTATION_REPORT1.mdrather than observation. §7 says so, and W2's exit criteriaare what would replace it.