Skip to content

Phase 7: Kubernetes learning track on local kind (manifests + HPA + ingress + CI/CD) - #11

Merged
tomnguyen103 merged 14 commits into
mainfrom
phase-7-impl
Jun 2, 2026
Merged

Phase 7: Kubernetes learning track on local kind (manifests + HPA + ingress + CI/CD)#11
tomnguyen103 merged 14 commits into
mainfrom
phase-7-impl

Conversation

@tomnguyen103

@tomnguyen103 tomnguyen103 commented Jun 2, 2026

Copy link
Copy Markdown
Owner

Phase 7 — Kubernetes learning track (local kind), then torn down

Translates the 8-service prod Compose stack to real Kubernetes manifests (deploy/k8s/),
proves it on a free multi-node kind cluster, and tears the cluster down — K8s is a learning
track, NOT the production runtime
(prod stays the single-VPS Compose stack, ADR-0011/0012).
Plan: docs/phase-7-plan.md · Decisions: ADR-0014 · Evidence: docs/k8s-evidence/.

Verification here is "apply manifest → assert rollout/health/scale" (not pytest-TDD); each layer
was applied + verified + committed, with text evidence captured.

What shipped (deploy/k8s/)

  • Postgres StatefulSet + 1Gi PVC (headless Service) + a one-shot migrate Job (alembic upgrade head direct to the DB — split out of the api's compose command, D3)
  • pgbouncer (env-configured so no userlist.txt/password is committed, D12), redis
  • api (uvicorn only; CPU requests for HPA), worker (--loop), frontend (NEXT_PUBLIC
    baked at build, D11)
  • ingress-nginx host routing (api.second-brain.local / second-brain.local)
  • metrics-server + HPA on api CPU; Prometheus + Grafana (configs reused --from-file, DRY)
  • kustomization.yaml (one-shot apply -k), secret.example.yaml (template only, D4),
    deploy/k8s/README.md
  • New CI .github/workflows/k8s.yml (kind-action): build+load → apply → wait rollouts → smoke
    /health + UI through ingress → teardown. The eval-gated ci.yml is untouched (D8).

Verified live (multi-node kind, v1.35.0)

  • All pods Ready; migrate Job Complete (schema 0004); SELECT 1 through pgbouncer (scram-sha-256)
  • api /health db:ok; worker drained an enqueued briefing job (queued→done, Briefing written)
  • ingress: /health 200 + UI 200; frontend client bundle carries the ingress API host (D11)
  • HPA scaled api 1→4 under hey load (CPU peaked 400%/50%, 36,293 reqs all 200, pods spread
    across both workers) and back 4→1
  • Prometheus scrapes the api (up{job=second-brain-api}=1); Grafana /api/health 200
  • Teardown (D10): kind delete cluster → nothing left running ($0). No managed cloud created (D9).

Notable / off-spec

  • .dockerignore (latent Phase-6 bug): without it the repo-root build context shipped the host
    backend/.venv (1.3G) + frontend/node_modules (660M, wrong-OS) into the images, bloating the
    backend and breaking the frontend. The prod images were only compose config-linted before, never
    built — this surfaced on first real build.
  • D11 additive build ARG on Dockerfile.frontend (Next inlines NEXT_PUBLIC_* at build time).
  • D12 pgbouncer env-config keeps the credential in the Secret. DSNs assembled per-pod via $(VAR).
  • D13 HPA load-scaling proven locally (evidence 08); CI stays deterministic (no load assertion).

Full detail in docs/implementation-notes.md (Phase 7 section) and docs/PROGRESS.md.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Add full Kubernetes learning-track to run the stack locally (multi-node kind, ingress, services, deployments, DB, pgbouncer, redis, HPA) and bake frontend API base at build time.
    • Add CI workflow to create kind cluster, deploy stack, run smoke tests, and capture diagnostics.
    • Add Prometheus + Grafana monitoring.
  • Documentation

    • Phase 7 implementation plan, ADR, runbook, and extensive evidence files documenting deploy, verify, autoscaling, and teardown.
  • Chores

    • Update Docker build args and add Docker/Git ignore entries to exclude local tooling/artifacts.

tomnguyen103 and others added 12 commits June 2, 2026 11:03
Translate the 8 prod-compose services to real K8s manifests on local kind:
StatefulSet+PVC postgres, migrate Job, api/worker/frontend Deployments,
ingress-nginx host routing, metrics-server+HPA, reused Prometheus/Grafana,
plus a kind CI workflow. Verification = apply->rollout/health/scale (not
pytest-TDD). D1-D10 are the goal defaults; D11-D13 added after reading the
stack (NEXT_PUBLIC build-time bake, pgbouncer env-config, local HPA evidence).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 control-plane + 2 workers (HPA spread + ingress-on-labelled-node are real).
control-plane labelled ingress-ready=true and maps host 80/443 for the
ingress-nginx kind provider. Namespace second-brain. Verified: 3 nodes Ready
(v1.35.0), namespace Active (evidence 01).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Non-secret config (POSTGRES_USER/DB, LLM provider=fake for the keyless $0
learning-track demo) in a ConfigMap. Secret SHAPE documented in
secret.example.yaml; the real Secret is created imperatively so no credential
touches disk/git. .gitignore guards deploy/k8s/secret.yaml. Per D11,
NEXT_PUBLIC_API_BASE_URL is build-time baked, not a runtime ConfigMap key.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Postgres (pgvector:pg16) as StatefulSet + 1Gi PVC (local-path), headless
Service for stable DNS. Migrations run as a one-shot Job (alembic upgrade
head) against db:5432 directly, not inline in the api (D3). Added root
.dockerignore so host .venv (1.3G) / node_modules (660M) are not shipped into
or copied by the images (latent Phase-6 bug; images were only config-linted
before). Verified: db pod Ready, PVC Bound, Job Complete, schema at 0004
(evidence 03).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
pgbouncer configured by env (edoburu auto-generates pgbouncer.ini+userlist),
so the DB password stays in the Secret and no userlist.txt is committed.
SESSION pool mode (psycopg3 prepared statements). redis as in-memory LRU
cache. Verified: SELECT 1 through pgbouncer:6432 (scram-sha-256 auth OK),
redis PONG (evidence 04).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
api Deployment (CPU requests for HPA) + Service; runs only uvicorn, reaches
Postgres via pgbouncer (migrations are the Job's, D3). worker Deployment runs
the resident --loop draining the jobs queue. Verified: api /health db:ok;
worker drained an enqueued briefing job (queued->done, Briefing row written)
end-to-end on the cluster (evidence 05). Note: worker eager-loads the embedder
at startup, api lazy-loads.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
frontend Deployment + Service. Dockerfile.frontend gains an additive, default-
preserving ARG NEXT_PUBLIC_API_BASE_URL so the K8s image bakes the ingress API
host (Next inlines NEXT_PUBLIC_* at build time; runtime config would be ignored).
Verified: serves HTML; api.second-brain.local present in both client + server
bundles (evidence 06).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Host-based Ingress: api.second-brain.local -> api:8000, second-brain.local ->
frontend:3000. ingress-nginx kind provider pinned to controller-v1.12.3.
Verified through localhost (host 80 -> control-plane node): api /health 200
db:ok; UI / -> 307 /chat -> 200 HTML (evidence 07).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
HPA on api CPU (50% of 250m request, min 1 / max 4). metrics-server v0.7.2
with --kubelet-insecure-tls (kind). Verified under in-cluster hey load: api
scaled 1->4 (CPU peaked 400%/50%, pods spread across both workers, 36293 reqs
all 200), then 4->1 after load removed (evidence 08).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Prometheus + Grafana as simple Deployments; configs sourced --from-file from
the Phase 6 deploy/prometheus/* + deploy/grafana/* (DRY, single source of
truth). Verified: both Ready; Prometheus scrapes api /metrics (up{job=
second-brain-api}=1); Grafana /api/health 200 with provisioned datasource +
dashboard (evidence 09).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New .github/workflows/k8s.yml (separate from the untouched eval-gated ci.yml):
multi-node kind (pinned v0.31.0/v1.35.0), build+load images, create secret +
monitoring configmaps, install ingress-nginx v1.12.3 + metrics-server v0.7.2,
kubectl apply -k, wait all rollouts, smoke /health + UI through ingress (with
retry), kind-action tears the cluster down (D10). HPA load stays local (D13).
Adds kustomization.yaml (one-shot apply) + deploy/k8s/README.md (run/verify).
Cluster name now via --name flag only (config name removed) so local + CI
align without kind's both-set conflict. Stack-overview evidence captured.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ADR-0014 (K8s learning track, D1-D13) + index. PROGRESS Phase 7 -> COMPLETE
(dated entry; managed-cluster parking-lot item resolved via D9). implementation-
notes: .dockerignore latent fix, D11 build-time NEXT_PUBLIC, D12 pgbouncer
env-config, migrate-Job + $(VAR) DSN, worker eager vs api lazy embedder, D13
local HPA, kind name-via-flag, pinned versions. README roadmap -> all phases
complete + a Phase 7 run/verify section. Teardown evidence (11): cluster
deleted, nothing left running ($0).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jun 2, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 7d1034a0-3d94-4a88-92ad-ad036e4a7371

📥 Commits

Reviewing files that changed from the base of the PR and between d360565 and cbb2829.

📒 Files selected for processing (1)
  • .github/workflows/k8s.yml
🚧 Files skipped from review as they are similar to previous changes (1)
  • .github/workflows/k8s.yml

📝 Walkthrough

Walkthrough

Phase 7 adds a local Kubernetes learning track: 25+ k8s manifests (namespace, Postgres StatefulSet, migrate Job, pgbouncer, redis, api/worker/frontend, ingress, HPA, Prometheus/Grafana), a GitHub Actions kind-based CI smoke pipeline, Docker/git tweaks, ADR and runbooks, and extensive verification evidence.

Changes

Phase 7 Kubernetes Learning Track

Layer / File(s) Summary
Build Foundation & Cluster Infrastructure
.dockerignore, deploy/Dockerfile.frontend, deploy/k8s/kind-cluster.yaml, deploy/k8s/namespace.yaml, deploy/k8s/secret.example.yaml, deploy/k8s/configmap.yaml, deploy/k8s/kustomization.yaml
Docker build context ignores added; frontend Dockerfile bakes NEXT_PUBLIC_API_BASE_URL at build-time. Kind cluster YAML defines multi-node topology with ingress-ready control-plane and host port mappings; namespace, example Secret, and ConfigMap establish base config.
Data Persistence & Migrations
deploy/k8s/postgres-statefulset.yaml, deploy/k8s/migrate-job.yaml, deploy/k8s/pgbouncer.yaml, deploy/k8s/redis.yaml
Postgres StatefulSet with PVC and headless Service; migration Job runs alembic upgrade head against DB; PgBouncer configured via env + Secret on port 6432; Redis deployment for caching with resource limits.
Application Services & Deployments
deploy/k8s/api.yaml, deploy/k8s/worker.yaml, deploy/k8s/frontend.yaml
API Deployment (second-brain-api:phase7) exposes 8000 with /health probes and pgbouncer wiring; worker runs python -m app.jobs.worker --loop; frontend image (second-brain-web:phase7) serves on 3000 with build-time API host baked in. All include resource requests/limits.
Ingress & Host-Based Routing
deploy/k8s/ingress.yaml
NGINX Ingress routes api.second-brain.localapi:8000 and second-brain.localfrontend:3000 using host-based rules; kind maps host ports for local smoke tests.
Observability & Autoscaling
deploy/k8s/api-hpa.yaml, deploy/k8s/monitoring/prometheus.yaml, deploy/k8s/monitoring/grafana.yaml
HPA for api targets 50% CPU with min 1/max 4; Prometheus and Grafana Deployments added with ConfigMap-backed provisioning; Grafana admin password comes from Secret.
Kustomization & Manifest Composition
deploy/k8s/kustomization.yaml
Kustomize base enumerates namespace, config, data services, app workloads, ingress/HPA, and monitoring for unified kubectl apply -k deploy/k8s.
CI/CD Pipeline & Smoke Testing
.github/workflows/k8s.yml, .gitignore
GitHub Actions workflow (k8s-kind) provisions kind, builds/loads images (frontend build ARG for API host), applies namespace/ConfigMaps/Secrets, installs ingress-nginx and metrics-server, applies manifests, waits for rollouts/migration, runs ingress-based HTTP smoke tests, dumps state, and tears down. .gitignore excludes deploy/k8s/secret.yaml.
Implementation Plan & Architecture Decisions
docs/phase-7-plan.md, docs/adr/0014-kubernetes-learning-track.md, docs/adr/README.md
Phase 7 plan and ADR-0014 document scope, Compose→Kubernetes mapping, 13 decisions (D1–D13), and the learning-track (non-production) constraints and consequences.
Verification Evidence & Runbooks
deploy/k8s/README.md, docs/k8s-evidence/*
Runbook and evidence files (00-stack-overview.txt11-teardown.txt) capture commands and outputs for cluster bring-up, ConfigMap/Secret, Postgres/migrations, pgbouncer/redis, API/worker/frontend verification, ingress smoke tests, HPA autoscaling proof, Prometheus/Grafana checks, and teardown.
README, Progress, & Implementation Notes
README.md, docs/PROGRESS.md, docs/implementation-notes.md
README marks Phases 1–7 complete and adds deploy/ layout; PROGRESS.md logs Phase 7 completion; implementation notes record build-time API bake, .dockerignore, pgbouncer/Secret strategy, migrations-as-Job, embedder load behavior, and pinned tooling versions.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Poem

🐰 A rabbit hops through kind clusters bright,
Manifests stacked like carrots in the night,
DBs and probes and ingress paths aligned,
Ephemeral learning, evidence signed.
Tear it down, then hop away — hooray!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and specifically summarizes the main change: adding a Kubernetes learning track on local kind with manifests, HPA, ingress, and CI/CD integration, which aligns with the comprehensive changeset.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch phase-7-impl

Comment @coderabbitai help to get the list of available commands and usage tips.

tomnguyen103 and others added 2 commits June 2, 2026 11:51
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
kind-smoke failed at 'kind load' with 'no space left on device' — docker save
of the multi-GB backend image (CUDA torch wheels) overflows the hosted
runner's ~14GB disk. Reclaim ~20GB of unused preinstalled toolchains (Android
SDK, .NET, GHC, CodeQL) before building. CPU-only torch (slim the image
itself) remains the deferred optimization (ADR-0014).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 9

🧹 Nitpick comments (3)
deploy/k8s/monitoring/grafana.yaml (1)

48-81: 💤 Low value

Optional: add a minimal pod/container securityContext.

Static analysis flags the default security context (root-capable, privilege escalation allowed). For a throwaway learning cluster the risk is low, but runAsNonRoot + allowPrivilegeEscalation: false are safe to add. Note readOnlyRootFilesystem: true would break Grafana here since it writes its SQLite DB to /var/lib/grafana (no writable mount), so leave that off unless you add an emptyDir.

♻️ Suggested hardening
       containers:
         - name: grafana
           image: grafana/grafana:latest
+          securityContext:
+            allowPrivilegeEscalation: false
+            runAsNonRoot: true
+            capabilities:
+              drop: ["ALL"]
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@deploy/k8s/monitoring/grafana.yaml` around lines 48 - 81, Add a minimal
container-level securityContext to the grafana container to harden privileges:
in the container spec for the container named "grafana" set runAsNonRoot: true
(optionally set runAsUser to a non-root UID if known) and set
allowPrivilegeEscalation: false; do NOT add readOnlyRootFilesystem (Grafana
needs to write to /var/lib/grafana) unless you also add a writable volume (e.g.,
emptyDir or persistentVolumeClaim). This will address the static analysis
warning while keeping Grafana writable storage intact.
deploy/k8s/monitoring/prometheus.yaml (1)

46-78: 💤 Low value

Optional: harden the container securityContext.

Static analysis flags the default (root) security context. Unlike Grafana, Prometheus only writes to the mounted /prometheus emptyDir, so readOnlyRootFilesystem: true is viable here alongside runAsNonRoot.

♻️ Suggested hardening
         - name: prometheus
           image: prom/prometheus:latest
+          securityContext:
+            allowPrivilegeEscalation: false
+            runAsNonRoot: true
+            readOnlyRootFilesystem: true
+            capabilities:
+              drop: ["ALL"]
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@deploy/k8s/monitoring/prometheus.yaml` around lines 46 - 78, Add a hardened
securityContext to the Prometheus container spec (container name "prometheus")
so it doesn't run as root: set securityContext.runAsNonRoot: true, set a
non-root runAsUser (e.g., a numeric UID consistent with Prometheus image), and
enable securityContext.readOnlyRootFilesystem: true while keeping the mounted
/prometheus volume writable via the existing volumeMount "data"; update
securityContext.capabilities/drop and fsGroup as needed to allow writing to
/prometheus (reference the "prometheus" container, the "data" volumeMount, and
properties readOnlyRootFilesystem, runAsNonRoot, runAsUser, fsGroup,
capabilities).
deploy/k8s/redis.yaml (1)

56-62: ⚡ Quick win

Align the memory request with Redis's configured ceiling.

This pod advertises a 64Mi request but is configured to keep up to 256mb in cache and can grow to a 320Mi limit. That under-reserves memory for the scheduler and makes eviction pressure more likely on the small kind nodes this track targets.

Example adjustment
           resources:
             requests:
               cpu: "50m"
-              memory: "64Mi"
+              memory: "256Mi"
             limits:
               cpu: "200m"
               memory: "320Mi"

If you want to keep a smaller request, please verify the expected steady-state cache footprint stays below it in local and CI runs.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@deploy/k8s/redis.yaml` around lines 56 - 62, The pod's memory request
(resources.requests.memory = "64Mi") is under-provisioned relative to the
configured Redis cache ceiling (256mb) and the limit (resources.limits.memory =
"320Mi"); update resources.requests.memory to align with the expected
steady-state footprint (e.g., set resources.requests.memory to "256Mi" or
another value that covers the steady-state cache usage) and ensure the requests
<= limits; if you intend to keep a smaller request, run local/CI measurements to
confirm the steady-state cache stays below the chosen request and adjust
accordingly.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/k8s.yml:
- Around line 29-39: The workflow uses mutable action tags for actions/checkout
and helm/kind-action and lacks least-privilege permissions; update the checkout
step (actions/checkout) and the kind step (helm/kind-action) to pinned commit
SHAs instead of v4/v1 tags, set persist-credentials: false on the checkout step,
and add a top-level or job-level permissions block granting only the minimal
scopes required for the job (e.g., contents: read, id-token: write if needed) so
the kind cluster creation (Create kind cluster) runs with least privilege.

In `@deploy/k8s/api.yaml`:
- Around line 39-104: Add a pod- and container-level securityContext for the api
container: set podSecurityContext with runAsNonRoot: true, runAsUser (e.g. 1000)
and fsGroup; add a container-level securityContext on the container named "api"
with allowPrivilegeEscalation: false, capabilities.drop: ["ALL"],
readOnlyRootFilesystem: true, seccompProfile.type: RuntimeDefault (and
runAsNonRoot: true). If the app needs writable temp/cache paths, keep
readOnlyRootFilesystem: true and mount an emptyDir at the writable paths instead
of making the root filesystem writable.
- Around line 82-97: readinessProbe currently uses httpGet /health (which is
DB-aware) and livenessProbe also uses httpGet /health causing pod restarts when
DB/PgBouncer is slow; change the livenessProbe to a lightweight tcpSocket probe
on port 8000 (keep readinessProbe as-is pointing at path: /health) and adjust
liveness settings if needed (initialDelaySeconds, periodSeconds, timeoutSeconds)
so liveness only checks that the process is accepting TCP connections rather
than invoking db.execute — update the livenessProbe block to use tcpSocket: {
port: 8000 } and remove the httpGet path to decouple liveness from DB checks.

In `@deploy/k8s/frontend.yaml`:
- Around line 38-65: Add a hardened securityContext to the frontend
Pod/Container (container name "frontend"): set pod-level runAsNonRoot: true and
runAsUser (e.g., 1000), and in the container securityContext set
allowPrivilegeEscalation: false, capabilities.drop: ["ALL"],
seccompProfile.type: RuntimeDefault, and readOnlyRootFilesystem: true; if
Next.js needs runtime write access, add explicit writable emptyDir volumes
(e.g., for /tmp or .next/cache) and mount them into the "frontend" container
instead of leaving the root filesystem writable.

In `@deploy/k8s/monitoring/grafana.yaml`:
- Line 50: Replace the floating image reference "image: grafana/grafana:latest"
in the Grafana Deployment/StatefulSet spec with a specific Grafana release tag
(for example "grafana/grafana:13.1.0-25469333600" or your chosen
X.Y.Z-<buildID>) to ensure reproducible deployments; update the manifest entry
that currently contains image: grafana/grafana:latest so the container spec uses
the pinned tag and commit the change.

In `@deploy/k8s/monitoring/prometheus.yaml`:
- Line 48: Replace the floating image tag to a fixed, reproducible Prometheus
version: locate the manifest entry where the container image is set to
"prom/prometheus:latest" and change it to the pinned stable tag
"prom/prometheus:v3.12.0" so deployments use a deterministic Prometheus release.

In `@deploy/k8s/pgbouncer.yaml`:
- Around line 42-43: The image reference for the pgbouncer container currently
uses the mutable tag "edoburu/pgbouncer:latest"; update the image field for the
container named "pgbouncer" to a fixed, immutable reference by replacing
":latest" with a specific version tag and the matching digest (e.g.,
edoburu/pgbouncer:<version>`@sha256`:<digest>) so deployments are reproducible and
platform-specific; fetch the correct version and sha256 for your target
architecture and substitute it into the image line.

In `@deploy/k8s/postgres-statefulset.yaml`:
- Around line 41-90: The StatefulSet pod/container lacks securityContext causing
the Postgres container (container name "postgres") to run with default
root-capable privileges; add a pod-level securityContext (fsGroup and
runAsNonRoot:true) and a container-level securityContext for the "postgres"
container that sets runAsUser and runAsGroup to the UID/GID expected by the
pgvector/pgvector:pg16 image, set allowPrivilegeEscalation: false,
readOnlyRootFilesystem: false only if PGDATA writable, and drop all Linux
capabilities (capabilities.drop: ["ALL"]) to enforce least privilege for the
mounted PVC (volumeMount name "data") and access to secrets
(second-brain-secrets); verify the chosen runAsUser/runAsGroup keep
/var/lib/postgresql/data/pgdata writable and adjust FS permissions if needed.

In `@deploy/k8s/worker.yaml`:
- Around line 22-62: The pod template for the worker container (container name
"worker", command "python -m app.jobs.worker") lacks a securityContext; add a
pod-level securityContext and container-level securityContext to harden it: set
podSecurityContext.runAsNonRoot=true and runAsUser to a non-root UID (e.g.,
1000), and on the container set securityContext.readOnlyRootFilesystem=true,
allowPrivilegeEscalation=false, drop all capabilities and only add needed ones
if any; if the worker requires writable scratch space, add an emptyDir volume
and mount it at the specific writable path instead of leaving the root FS
writable (reference the "worker" container and the pod template to locate these
changes).

---

Nitpick comments:
In `@deploy/k8s/monitoring/grafana.yaml`:
- Around line 48-81: Add a minimal container-level securityContext to the
grafana container to harden privileges: in the container spec for the container
named "grafana" set runAsNonRoot: true (optionally set runAsUser to a non-root
UID if known) and set allowPrivilegeEscalation: false; do NOT add
readOnlyRootFilesystem (Grafana needs to write to /var/lib/grafana) unless you
also add a writable volume (e.g., emptyDir or persistentVolumeClaim). This will
address the static analysis warning while keeping Grafana writable storage
intact.

In `@deploy/k8s/monitoring/prometheus.yaml`:
- Around line 46-78: Add a hardened securityContext to the Prometheus container
spec (container name "prometheus") so it doesn't run as root: set
securityContext.runAsNonRoot: true, set a non-root runAsUser (e.g., a numeric
UID consistent with Prometheus image), and enable
securityContext.readOnlyRootFilesystem: true while keeping the mounted
/prometheus volume writable via the existing volumeMount "data"; update
securityContext.capabilities/drop and fsGroup as needed to allow writing to
/prometheus (reference the "prometheus" container, the "data" volumeMount, and
properties readOnlyRootFilesystem, runAsNonRoot, runAsUser, fsGroup,
capabilities).

In `@deploy/k8s/redis.yaml`:
- Around line 56-62: The pod's memory request (resources.requests.memory =
"64Mi") is under-provisioned relative to the configured Redis cache ceiling
(256mb) and the limit (resources.limits.memory = "320Mi"); update
resources.requests.memory to align with the expected steady-state footprint
(e.g., set resources.requests.memory to "256Mi" or another value that covers the
steady-state cache usage) and ensure the requests <= limits; if you intend to
keep a smaller request, run local/CI measurements to confirm the steady-state
cache stays below the chosen request and adjust accordingly.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 351dee5b-87e1-44a5-bd06-9cebd8fb037b

📥 Commits

Reviewing files that changed from the base of the PR and between 355d727 and d360565.

📒 Files selected for processing (38)
  • .dockerignore
  • .github/workflows/k8s.yml
  • .gitignore
  • README.md
  • deploy/Dockerfile.frontend
  • deploy/k8s/README.md
  • deploy/k8s/api-hpa.yaml
  • deploy/k8s/api.yaml
  • deploy/k8s/configmap.yaml
  • deploy/k8s/frontend.yaml
  • deploy/k8s/ingress.yaml
  • deploy/k8s/kind-cluster.yaml
  • deploy/k8s/kustomization.yaml
  • deploy/k8s/migrate-job.yaml
  • deploy/k8s/monitoring/grafana.yaml
  • deploy/k8s/monitoring/prometheus.yaml
  • deploy/k8s/namespace.yaml
  • deploy/k8s/pgbouncer.yaml
  • deploy/k8s/postgres-statefulset.yaml
  • deploy/k8s/redis.yaml
  • deploy/k8s/secret.example.yaml
  • deploy/k8s/worker.yaml
  • docs/PROGRESS.md
  • docs/adr/0014-kubernetes-learning-track.md
  • docs/adr/README.md
  • docs/implementation-notes.md
  • docs/k8s-evidence/00-stack-overview.txt
  • docs/k8s-evidence/01-cluster-and-namespace.txt
  • docs/k8s-evidence/02-config-and-secrets.txt
  • docs/k8s-evidence/03-postgres-and-migrate.txt
  • docs/k8s-evidence/04-pgbouncer-and-redis.txt
  • docs/k8s-evidence/05-api-and-worker.txt
  • docs/k8s-evidence/06-frontend.txt
  • docs/k8s-evidence/07-ingress.txt
  • docs/k8s-evidence/08-hpa-autoscaling.txt
  • docs/k8s-evidence/09-monitoring.txt
  • docs/k8s-evidence/11-teardown.txt
  • docs/phase-7-plan.md

Comment thread .github/workflows/k8s.yml
Comment on lines +29 to +39
- uses: actions/checkout@v4

# Multi-node cluster from the committed config (ingress-ready label + host 80/443 maps).
# Versions pinned to match local (kind v0.31.0 -> node v1.35.0). Name via flag only.
- name: Create kind cluster
uses: helm/kind-action@v1
with:
version: v0.31.0
node_image: kindest/node:v1.35.0
cluster_name: second-brain
config: deploy/k8s/kind-cluster.yaml

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🌐 Web query:

latest commit SHA for actions/checkout v4 and helm/kind-action v1 releases

💡 Result:

actions/checkout v4 (latest as of 2026-06-02): v4.1.5 → commit SHA is not retrievable from the sources I found (I could confirm v4.1.4 and its associated tag, but not the full SHA for the latest v4 release) [1][2]. actions/checkout v4.1.4 (latest fully evidenced in sources): 0ad4b8fadaa221de15dcec353f45205ec38ea70b [1][3]. helm/kind-action v1 (latest as of 2026-06-02): v1.14.0 (the latest release shown in sources) [4]. However, the sources I found do not include the full commit SHA corresponding to tag v1.14.0, only tag/release information and some downstream dependabot notes with short SHAs that are insufficient for an authoritative full 40-character commit SHA [4][5].

Citations:


🏁 Script executed:

#!/bin/bash
set -euo pipefail
FILE=".github/workflows/k8s.yml"
echo "== File =="
ls -la "$FILE"
echo
echo "== Snippet around lines 1-120 =="
nl -ba "$FILE" | sed -n '1,140p'
echo
echo "== Search for permissions / checkout options =="
rg -n "^(permissions:|jobs:|steps:|- uses: actions/checkout@|persist-credentials:)" "$FILE" || true

Repository: tomnguyen103/second-brain

Length of output: 250


🏁 Script executed:

#!/bin/bash
set -euo pipefail
FILE=".github/workflows/k8s.yml"

echo "== File =="
ls -la "$FILE"
echo

echo "== First 140 lines (numbered) =="
python3 - <<'PY'
from pathlib import Path
p = Path(".github/workflows/k8s.yml")
lines = p.read_text().splitlines()
for i, line in enumerate(lines[:140], start=1):
    print(f"{i:4d}\t{line}")
PY
echo

echo "== Search for permissions / checkout options =="
rg -n "^(permissions:|jobs:|steps:|- uses: actions/checkout@|persist-credentials:|helm/kind-action@)" "$FILE" || true

Repository: tomnguyen103/second-brain

Length of output: 6882


🏁 Script executed:

#!/bin/bash
set -euo pipefail
FILE=".github/workflows/k8s.yml"

echo "== Search for top-level permissions block =="
rg -n "^\s*permissions\s*:" "$FILE" || true

echo
echo "== Search for persist-credentials / checkout uses =="
rg -n "persist-credentials:|actions/checkout@" "$FILE" || true

echo
echo "== Search for helm/kind-action uses =="
rg -n "helm/kind-action@" "$FILE" || true

Repository: tomnguyen103/second-brain

Length of output: 282


🌐 Web query:

actions/checkout v4.1.5 commit SHA

💡 Result:

The commit SHA associated with the actions/checkout version v4.1.5 release is 44c2b7a [1]. This 7-character short SHA represents the specific point in the actions/checkout repository history tagged as v4.1.5 [1].

Citations:


🌐 Web query:

helm/kind-action v1.14.0 commit SHA

💡 Result:

The release v1.14.0 of helm/kind-action was published on February 17, 2026 [1][2]. While the release tag v1.14.0 points to a specific commit in the helm/kind-action repository, the exact 40-character commit SHA is not explicitly indexed in the search results provided. To obtain the precise commit SHA for v1.14.0, you can view the official release page on GitHub at https://github.com/helm/kind-action/releases/tag/v1.14.0 [3][2][4]. Alternatively, you can use the following Git command if you have the repository cloned locally: git rev-parse v1.14.0 This release included updates such as bumping actions/checkout to 6.0.2, updating kind to v0.31.0, and updating Kubernetes to v1.35.0 [5][2][6].

Citations:


Harden k8s kind-smoke workflow supply chain: pin actions + least-privilege permissions.

actions/checkout@v4 and helm/kind-action@v1 are used by mutable tags, and the workflow has no top-level permissions scope nor persist-credentials: false on checkout. Pin both actions to full commit SHAs and add least-privilege permissions for this job.

♻️ Suggested changes
+permissions:
+  contents: read
+
 jobs:
   kind-smoke:
     runs-on: ubuntu-latest
     timeout-minutes: 30
     steps:
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@<commit-sha> # v4
+        with:
+          persist-credentials: false
@@
       - name: Create kind cluster
-        uses: helm/kind-action@v1
+        uses: helm/kind-action@<commit-sha> # v1
🧰 Tools
🪛 zizmor (1.25.2)

[warning] 29-32: credential persistence through GitHub Actions artifacts (artipacked): does not set persist-credentials: false

(artipacked)


[error] 29-29: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)

(unpinned-uses)


[error] 34-34: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)

(unpinned-uses)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/k8s.yml around lines 29 - 39, The workflow uses mutable
action tags for actions/checkout and helm/kind-action and lacks least-privilege
permissions; update the checkout step (actions/checkout) and the kind step
(helm/kind-action) to pinned commit SHAs instead of v4/v1 tags, set
persist-credentials: false on the checkout step, and add a top-level or
job-level permissions block granting only the minimal scopes required for the
job (e.g., contents: read, id-token: write if needed) so the kind cluster
creation (Create kind cluster) runs with least privilege.

Comment thread deploy/k8s/api.yaml
Comment on lines +39 to +104
spec:
containers:
- name: api
image: second-brain-api:phase7
imagePullPolicy: IfNotPresent # use the kind-loaded image; never pull from a registry (D2)
ports:
- name: http
containerPort: 8000
env:
- name: POSTGRES_USER
valueFrom:
configMapKeyRef:
name: second-brain-config
key: POSTGRES_USER
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: second-brain-secrets
key: POSTGRES_PASSWORD
- name: POSTGRES_DB
valueFrom:
configMapKeyRef:
name: second-brain-config
key: POSTGRES_DB
# App traffic goes through PgBouncer (session pooling). Assembled via $(VAR) so the
# password lives only in the Secret.
- name: SECOND_BRAIN_DATABASE_URL
value: postgresql+psycopg://$(POSTGRES_USER):$(POSTGRES_PASSWORD)@pgbouncer:6432/$(POSTGRES_DB)
- name: SECOND_BRAIN_LLM_PROVIDER
valueFrom:
configMapKeyRef:
name: second-brain-config
key: SECOND_BRAIN_LLM_PROVIDER
- name: SECOND_BRAIN_GEMINI_API_KEY
valueFrom:
secretKeyRef:
name: second-brain-secrets
key: SECOND_BRAIN_GEMINI_API_KEY
- name: SECOND_BRAIN_ADMIN_TOKEN
valueFrom:
secretKeyRef:
name: second-brain-secrets
key: SECOND_BRAIN_ADMIN_TOKEN
readinessProbe:
httpGet:
path: /health
port: 8000
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 12
livenessProbe:
httpGet:
path: /health
port: 8000
initialDelaySeconds: 30
periodSeconds: 15
timeoutSeconds: 5
failureThreshold: 6
resources:
requests:
cpu: "250m" # HPA target is a % of this
memory: "512Mi"
limits:
cpu: "1000m"
memory: "1Gi"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Locate and show relevant portions of the manifest
FILE="deploy/k8s/api.yaml"
if [ ! -f "$FILE" ]; then
  echo "Missing file: $FILE"
  exit 1
fi

echo "=== File: $FILE (security-related snippets) ==="
# show lines around deployment/pod/container securityContext keywords if present
rg -n "securityContext|readOnlyRootFilesystem|allowPrivilegeEscalation|capabilities:|seccompProfile" "$FILE" || true

echo
echo "=== Lines 1-140 (for context) ==="
nl -ba "$FILE" | sed -n '1,140p'

Repository: tomnguyen103/second-brain

Length of output: 228


🏁 Script executed:

#!/bin/bash
set -euo pipefail

FILE="deploy/k8s/api.yaml"

echo "=== rg security keywords ==="
rg -n "securityContext|readOnlyRootFilesystem|allowPrivilegeEscalation|capabilities:|seccompProfile" "$FILE" || true

echo
echo "=== Print relevant deployment section (lines 1-140) ==="
# use cat -n (nl not available) and keep within reasonable bounds
cat -n "$FILE" | sed -n '1,140p'

echo
echo "=== Print env/probes/resources section context (lines ~30-120) ==="
cat -n "$FILE" | sed -n '30,120p'

Repository: tomnguyen103/second-brain

Length of output: 7435


Harden the API Deployment security context (pod + container).

deploy/k8s/api.yaml has no securityContext at the pod or container level for the api container, so Kubernetes defaults apply (privilege escalation/seccomp/non-root are not explicitly restricted).

Suggested manifest hardening
     spec:
+      securityContext:
+        runAsNonRoot: true
+        seccompProfile:
+          type: RuntimeDefault
       containers:
         - name: api
+          securityContext:
+            allowPrivilegeEscalation: false
+            capabilities:
+              drop: ["ALL"]
+            readOnlyRootFilesystem: true
           image: second-brain-api:phase7

If the image needs writable temp/cache paths, keep readOnlyRootFilesystem: true and mount an emptyDir for those paths instead of leaving the whole root FS writable.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
spec:
containers:
- name: api
image: second-brain-api:phase7
imagePullPolicy: IfNotPresent # use the kind-loaded image; never pull from a registry (D2)
ports:
- name: http
containerPort: 8000
env:
- name: POSTGRES_USER
valueFrom:
configMapKeyRef:
name: second-brain-config
key: POSTGRES_USER
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: second-brain-secrets
key: POSTGRES_PASSWORD
- name: POSTGRES_DB
valueFrom:
configMapKeyRef:
name: second-brain-config
key: POSTGRES_DB
# App traffic goes through PgBouncer (session pooling). Assembled via $(VAR) so the
# password lives only in the Secret.
- name: SECOND_BRAIN_DATABASE_URL
value: postgresql+psycopg://$(POSTGRES_USER):$(POSTGRES_PASSWORD)@pgbouncer:6432/$(POSTGRES_DB)
- name: SECOND_BRAIN_LLM_PROVIDER
valueFrom:
configMapKeyRef:
name: second-brain-config
key: SECOND_BRAIN_LLM_PROVIDER
- name: SECOND_BRAIN_GEMINI_API_KEY
valueFrom:
secretKeyRef:
name: second-brain-secrets
key: SECOND_BRAIN_GEMINI_API_KEY
- name: SECOND_BRAIN_ADMIN_TOKEN
valueFrom:
secretKeyRef:
name: second-brain-secrets
key: SECOND_BRAIN_ADMIN_TOKEN
readinessProbe:
httpGet:
path: /health
port: 8000
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 12
livenessProbe:
httpGet:
path: /health
port: 8000
initialDelaySeconds: 30
periodSeconds: 15
timeoutSeconds: 5
failureThreshold: 6
resources:
requests:
cpu: "250m" # HPA target is a % of this
memory: "512Mi"
limits:
cpu: "1000m"
memory: "1Gi"
spec:
securityContext:
runAsNonRoot: true
seccompProfile:
type: RuntimeDefault
containers:
- name: api
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
readOnlyRootFilesystem: true
image: second-brain-api:phase7
imagePullPolicy: IfNotPresent # use the kind-loaded image; never pull from a registry (D2)
ports:
- name: http
containerPort: 8000
env:
- name: POSTGRES_USER
valueFrom:
configMapKeyRef:
name: second-brain-config
key: POSTGRES_USER
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: second-brain-secrets
key: POSTGRES_PASSWORD
- name: POSTGRES_DB
valueFrom:
configMapKeyRef:
name: second-brain-config
key: POSTGRES_DB
# App traffic goes through PgBouncer (session pooling). Assembled via $(VAR) so the
# password lives only in the Secret.
- name: SECOND_BRAIN_DATABASE_URL
value: postgresql+psycopg://$(POSTGRES_USER):$(POSTGRES_PASSWORD)`@pgbouncer`:6432/$(POSTGRES_DB)
- name: SECOND_BRAIN_LLM_PROVIDER
valueFrom:
configMapKeyRef:
name: second-brain-config
key: SECOND_BRAIN_LLM_PROVIDER
- name: SECOND_BRAIN_GEMINI_API_KEY
valueFrom:
secretKeyRef:
name: second-brain-secrets
key: SECOND_BRAIN_GEMINI_API_KEY
- name: SECOND_BRAIN_ADMIN_TOKEN
valueFrom:
secretKeyRef:
name: second-brain-secrets
key: SECOND_BRAIN_ADMIN_TOKEN
readinessProbe:
httpGet:
path: /health
port: 8000
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 12
livenessProbe:
httpGet:
path: /health
port: 8000
initialDelaySeconds: 30
periodSeconds: 15
timeoutSeconds: 5
failureThreshold: 6
resources:
requests:
cpu: "250m" # HPA target is a % of this
memory: "512Mi"
limits:
cpu: "1000m"
memory: "1Gi"
🧰 Tools
🪛 Checkov (3.2.530)

[medium] 21-104: Containers should not run with allowPrivilegeEscalation

(CKV_K8S_20)


[medium] 21-104: Minimize the admission of root containers

(CKV_K8S_23)

🪛 Trivy (0.69.3)

[error] 40-103: Root file system is not read-only

Container 'api' of Deployment 'api' should set 'securityContext.readOnlyRootFilesystem' to true

Rule: KSV-0014

Learn more

(IaC/Kubernetes)


[error] 40-103: Default security context configured

container api in second-brain namespace is using the default security context

Rule: KSV-0118

Learn more

(IaC/Kubernetes)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@deploy/k8s/api.yaml` around lines 39 - 104, Add a pod- and container-level
securityContext for the api container: set podSecurityContext with runAsNonRoot:
true, runAsUser (e.g. 1000) and fsGroup; add a container-level securityContext
on the container named "api" with allowPrivilegeEscalation: false,
capabilities.drop: ["ALL"], readOnlyRootFilesystem: true, seccompProfile.type:
RuntimeDefault (and runAsNonRoot: true). If the app needs writable temp/cache
paths, keep readOnlyRootFilesystem: true and mount an emptyDir at the writable
paths instead of making the root filesystem writable.

Comment thread deploy/k8s/api.yaml
Comment on lines +82 to +97
readinessProbe:
httpGet:
path: /health
port: 8000
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 12
livenessProbe:
httpGet:
path: /health
port: 8000
initialDelaySeconds: 30
periodSeconds: 15
timeoutSeconds: 5
failureThreshold: 6

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Verify whether the FastAPI /health handler touches DB/PgBouncer before changing probe semantics.
rg -n -C3 '/health|health' --glob '*.py'
rg -n -C3 'SELECT 1|db:ok|database|pgbouncer|session.execute|engine.connect' --glob '*.py'

Repository: tomnguyen103/second-brain

Length of output: 9341


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Inspect the referenced k8s probe configuration
if [ -f deploy/k8s/api.yaml ]; then
  sed -n '1,200p' deploy/k8s/api.yaml | nl -ba | sed -n '70,120p'
else
  echo "Missing file: deploy/k8s/api.yaml"
fi

# Show the FastAPI /health implementation (already partially found, but include full function body)
if [ -f backend/app/api/health.py ]; then
  sed -n '1,120p' backend/app/api/health.py | nl -ba
else
  echo "Missing file: backend/app/api/health.py"
fi

Repository: tomnguyen103/second-brain

Length of output: 112


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== deploy/k8s/api.yaml (lines ~70-120) =="
if [ -f deploy/k8s/api.yaml ]; then
  # cat -n is usually available; fallback to sed without line numbers
  if command -v cat >/dev/null 2>&1; then
    cat -n deploy/k8s/api.yaml | sed -n '70,120p'
  else
    sed -n '70,120p' deploy/k8s/api.yaml
  fi
else
  echo "Missing file: deploy/k8s/api.yaml"
fi

echo
echo "== backend/app/api/health.py =="
if [ -f backend/app/api/health.py ]; then
  if command -v cat >/dev/null 2>&1; then
    cat -n backend/app/api/health.py | sed -n '1,200p'
  else
    sed -n '1,200p' backend/app/api/health.py
  fi
else
  echo "Missing file: backend/app/api/health.py"
fi

Repository: tomnguyen103/second-brain

Length of output: 2299


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Locate deps.get_db
rg -n "def get_db|get_db\s*=" backend/app -S

# Print the deps module around get_db
f="$(rg -l "def get_db" backend/app)"
echo "== $f =="
sed -n '1,220p' "$f"

Repository: tomnguyen103/second-brain

Length of output: 693


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Show health-related tests
rg -n "test_health|/health" backend/tests -S
echo
rg -n "`@router.get`\\(\"/health\"\\)" -S backend/app/api/health.py
echo

# Inspect deps.get_db caller and embedder dependency used by /health
rg -n "def get_embedder" backend/app -S
f="$(rg -l "def get_embedder" backend/app)"
echo "== $f =="
sed -n '1,240p' "$f"

# Inspect app/deps.py or app/deps module for get_embedder / get_db exports
if [ -f backend/app/deps.py ]; then
  echo
  echo "== backend/app/deps.py =="
  sed -n '1,260p' backend/app/deps.py
fi

# Also show how deps.get_db is imported (backend/app/api/health.py imports "from app import deps")
ls -la backend/app | head -n 50

Repository: tomnguyen103/second-brain

Length of output: 4717


🏁 Script executed:

#!/bin/bash
set -euo pipefail
sed -n '1,120p' backend/app/config.py | cat -n

Repository: tomnguyen103/second-brain

Length of output: 3318


🏁 Script executed:

#!/bin/bash
set -euo pipefail
rg -n "/health|db:ok|db_status|SELECT 1" -S .

Repository: tomnguyen103/second-brain

Length of output: 7359


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== Probe variants in deploy/k8s/api.yaml =="
rg -n "startupProbe|readinessProbe|livenessProbe|httpGet|tcpSocket" deploy/k8s/api.yaml

echo
echo "== backend/app/embeddings/encoder.py (Embedder init/load behavior) =="
f="$(rg -l "class Embedder" backend/app/embeddings backend/app -S)"
echo "Found: $f"
sed -n '1,240p' "$f"

Repository: tomnguyen103/second-brain

Length of output: 493


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== Locate Embedder class =="
rg -n "class Embedder" backend/app -S

echo
echo "== Show backend/app/embeddings module tree (lightweight) =="
# Use fd if available to avoid assumptions
if command -v fd >/dev/null 2>&1; then
  fd -t f "encoder.py" backend/app/embeddings || true
  fd -t f "encoder.*" backend/app/embeddings || true
else
  ls -R backend/app/embeddings 2>/dev/null || true
fi

echo
echo "== deploy/k8s/api.yaml probe area with surrounding context =="
sed -n '1,140p' deploy/k8s/api.yaml

Repository: tomnguyen103/second-brain

Length of output: 3707


🏁 Script executed:

#!/bin/bash
set -euo pipefail

sed -n '1,260p' backend/app/embeddings/encoder.py | cat -n

Repository: tomnguyen103/second-brain

Length of output: 1001


Decouple liveness from /health (DB/PgBouncer)

  • deploy/k8s/api.yaml configures both readinessProbe and livenessProbe as httpGet to /health.
  • /health runs db.execute(text("SELECT 1")); even though DB exceptions are caught and the endpoint still returns HTTP 200 ("db":"down"), a slow/hung DB/PgBouncer call can exceed timeoutSeconds and cause liveness to fail, restarting otherwise healthy pods.
  • Keep /health for readiness (DB-aware) but make liveness a lightweight process check (tcpSocket), as shown:
Safer probe split
           readinessProbe:
             httpGet:
               path: /health
               port: 8000
             initialDelaySeconds: 10
             periodSeconds: 10
             timeoutSeconds: 5
             failureThreshold: 12
+          startupProbe:
+            tcpSocket:
+              port: 8000
+            periodSeconds: 5
+            timeoutSeconds: 5
+            failureThreshold: 12
           livenessProbe:
-            httpGet:
-              path: /health
-              port: 8000
-            initialDelaySeconds: 30
+            tcpSocket:
+              port: 8000
             periodSeconds: 15
             timeoutSeconds: 5
             failureThreshold: 6
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
readinessProbe:
httpGet:
path: /health
port: 8000
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 12
livenessProbe:
httpGet:
path: /health
port: 8000
initialDelaySeconds: 30
periodSeconds: 15
timeoutSeconds: 5
failureThreshold: 6
readinessProbe:
httpGet:
path: /health
port: 8000
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 12
startupProbe:
tcpSocket:
port: 8000
periodSeconds: 5
timeoutSeconds: 5
failureThreshold: 12
livenessProbe:
tcpSocket:
port: 8000
periodSeconds: 15
timeoutSeconds: 5
failureThreshold: 6
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@deploy/k8s/api.yaml` around lines 82 - 97, readinessProbe currently uses
httpGet /health (which is DB-aware) and livenessProbe also uses httpGet /health
causing pod restarts when DB/PgBouncer is slow; change the livenessProbe to a
lightweight tcpSocket probe on port 8000 (keep readinessProbe as-is pointing at
path: /health) and adjust liveness settings if needed (initialDelaySeconds,
periodSeconds, timeoutSeconds) so liveness only checks that the process is
accepting TCP connections rather than invoking db.execute — update the
livenessProbe block to use tcpSocket: { port: 8000 } and remove the httpGet path
to decouple liveness from DB checks.

Comment thread deploy/k8s/frontend.yaml
Comment on lines +38 to +65
spec:
containers:
- name: frontend
image: second-brain-web:phase7
imagePullPolicy: IfNotPresent # use the kind-loaded image; never pull from a registry (D2)
ports:
- name: http
containerPort: 3000
readinessProbe:
httpGet:
path: /
port: 3000
initialDelaySeconds: 5
periodSeconds: 10
failureThreshold: 12
livenessProbe:
httpGet:
path: /
port: 3000
initialDelaySeconds: 20
periodSeconds: 15
resources:
requests:
cpu: "100m"
memory: "256Mi"
limits:
cpu: "500m"
memory: "512Mi"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# 1) Show the current frontend manifest (with line numbers)
echo "=== deploy/k8s/frontend.yaml ==="
nl -ba deploy/k8s/frontend.yaml | sed -n '1,200p'

# 2) Find all securityContext occurrences under deploy/k8s
echo
echo "=== rg securityContext in deploy/k8s ==="
rg -n "securityContext:" deploy/k8s || true

# 3) Compare with backend hardening patterns (common keys)
echo
echo "=== rg hardening keywords in deploy/k8s ==="
rg -n "runAsNonRoot|readOnlyRootFilesystem|allowPrivilegeEscalation|seccompProfile|capabilities:\s*$|drop:\s*\[\"ALL\"\]|drop:\s*\[ALL\]|capDrop|capabilities:" deploy/k8s || true

# 4) Check whether frontend has volumes/mounts that would be impacted by readOnlyRootFilesystem
echo
echo "=== frontend volumes/mounts ==="
rg -n "volumes:|volumeMounts:" deploy/k8s/frontend.yaml || true

Repository: tomnguyen103/second-brain

Length of output: 163


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# 0) Locate the file
echo "=== locating frontend manifest ==="
fd -t f "frontend.yaml" deploy . || true
fd -t f "frontend*.y*ml" deploy . || true

# 1) Show the current frontend manifest (with line numbers)
FRONTEND_PATH="deploy/k8s/frontend.yaml"
if [ -f "$FRONTEND_PATH" ]; then
  echo
  echo "=== $FRONTEND_PATH ==="
  cat -n "$FRONTEND_PATH" | sed -n '1,220p'
else
  echo
  echo "WARN: $FRONTEND_PATH not found; listing deploy/k8s tree (limited) for context."
  fd -t f "*" deploy/k8s -a || true
fi

# 2) Find all securityContext occurrences under deploy/k8s
echo
echo "=== rg securityContext in deploy/k8s (if present) ==="
if [ -d "deploy/k8s" ]; then
  rg -n "securityContext:" deploy/k8s || true
fi

# 3) Compare with backend hardening patterns (common keys)
echo
echo "=== rg hardening keywords in deploy/k8s ==="
if [ -d "deploy/k8s" ]; then
  rg -n "runAsNonRoot|readOnlyRootFilesystem|allowPrivilegeEscalation|seccompProfile|capabilities:\s*$|drop:\s*\[\"ALL\"\]|drop:\s*\[ALL\]|capabilities:" deploy/k8s || true
fi

# 4) Check whether frontend has volumes/mounts that would be impacted by readOnlyRootFilesystem
echo
echo "=== frontend volumes/mounts ==="
if [ -f "$FRONTEND_PATH" ]; then
  rg -n "volumes:|volumeMounts:" "$FRONTEND_PATH" || true
fi

Repository: tomnguyen103/second-brain

Length of output: 2449


Harden the frontend container security context before it becomes the default K8s path

deploy/k8s/frontend.yaml’s frontend Deployment has no pod/container securityContext at all (no runAsNonRoot, seccompProfile, capability drops, or readOnlyRootFilesystem), leaving it exposed to the same default root/privilege-escalation risk the scanners flag for other workloads.

Suggested manifest hardening
     spec:
+      securityContext:
+        runAsNonRoot: true
+        seccompProfile:
+          type: RuntimeDefault
       containers:
         - name: frontend
+          securityContext:
+            allowPrivilegeEscalation: false
+            capabilities:
+              drop: ["ALL"]
+            readOnlyRootFilesystem: true
           image: second-brain-web:phase7

If next start needs runtime writes (Next.js cache, /tmp, etc.), mount those explicitly with writable emptyDir volumes rather than keeping the whole root filesystem writable.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
spec:
containers:
- name: frontend
image: second-brain-web:phase7
imagePullPolicy: IfNotPresent # use the kind-loaded image; never pull from a registry (D2)
ports:
- name: http
containerPort: 3000
readinessProbe:
httpGet:
path: /
port: 3000
initialDelaySeconds: 5
periodSeconds: 10
failureThreshold: 12
livenessProbe:
httpGet:
path: /
port: 3000
initialDelaySeconds: 20
periodSeconds: 15
resources:
requests:
cpu: "100m"
memory: "256Mi"
limits:
cpu: "500m"
memory: "512Mi"
spec:
securityContext:
runAsNonRoot: true
seccompProfile:
type: RuntimeDefault
containers:
- name: frontend
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
readOnlyRootFilesystem: true
image: second-brain-web:phase7
imagePullPolicy: IfNotPresent # use the kind-loaded image; never pull from a registry (D2)
ports:
- name: http
containerPort: 3000
readinessProbe:
httpGet:
path: /
port: 3000
initialDelaySeconds: 5
periodSeconds: 10
failureThreshold: 12
livenessProbe:
httpGet:
path: /
port: 3000
initialDelaySeconds: 20
periodSeconds: 15
resources:
requests:
cpu: "100m"
memory: "256Mi"
limits:
cpu: "500m"
memory: "512Mi"
🧰 Tools
🪛 Checkov (3.2.530)

[medium] 20-65: Containers should not run with allowPrivilegeEscalation

(CKV_K8S_20)


[medium] 20-65: Minimize the admission of root containers

(CKV_K8S_23)

🪛 Trivy (0.69.3)

[error] 39-64: Root file system is not read-only

Container 'frontend' of Deployment 'frontend' should set 'securityContext.readOnlyRootFilesystem' to true

Rule: KSV-0014

Learn more

(IaC/Kubernetes)


[error] 39-64: Default security context configured

container frontend in second-brain namespace is using the default security context

Rule: KSV-0118

Learn more

(IaC/Kubernetes)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@deploy/k8s/frontend.yaml` around lines 38 - 65, Add a hardened
securityContext to the frontend Pod/Container (container name "frontend"): set
pod-level runAsNonRoot: true and runAsUser (e.g., 1000), and in the container
securityContext set allowPrivilegeEscalation: false, capabilities.drop: ["ALL"],
seccompProfile.type: RuntimeDefault, and readOnlyRootFilesystem: true; if
Next.js needs runtime write access, add explicit writable emptyDir volumes
(e.g., for /tmp or .next/cache) and mount them into the "frontend" container
instead of leaving the root filesystem writable.

spec:
containers:
- name: grafana
image: grafana/grafana:latest

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🌐 Web query:

What is the latest stable release version of the grafana/grafana Docker image?

💡 Result:

The latest stable release version for the grafana/grafana Docker image is 13.1.0, as indicated by the Grafana docs saying to use the “latest stable version of Grafana” and by the Docker Hub grafana/grafana tags showing the most recent stable version tag as 13.1.0 (with the corresponding “latest” tag pointing to the same line of releases). [1][2]

Citations:


🌐 Web query:

What is the current latest stable release version shown for the grafana/grafana Docker image on Docker Hub (and/or Grafana docs)?

💡 Result:

On Docker Hub, the grafana/grafana Docker image tag that represents the latest stable release is latest. The current latest stable version shown for grafana/grafana:latest is 13.1.0-25469333600 (as indicated by the latest tag pointing to that specific version tag on the Docker Hub tags page). [1][2] For running Grafana via Docker, Grafana’s own Docker documentation instructs users to “run the latest stable version of Grafana” using the grafana/grafana image (i.e., the latest tag). [3]

Citations:


🌐 Web query:

Does the grafana/grafana Docker image have a stable version tag exactly 13.1.0(not just13.1.0-*), and what is the recommended tag format for pinning Grafana to a specific version?

💡 Result:

No: the official grafana/grafana Docker tag is not shown as a stable exact tag 13.1.0; instead, tags are published like 13.1.0-<buildID> (for example 13.1.0-25858739282) and variants like 13.1.0-25858739282-ubuntu.[1] Recommended way to pin Grafana to a specific version: use an explicit version tag (i.e., the exact tag that corresponds to the release you want), rather than floating tags like latest or main. Grafana’s Docker documentation notes that for recent releases there are “minor” version tags such as grafana/grafana-enterprise:12.1 or grafana/grafana-enterprise:12.1-ubuntu, and it also describes using prerelease/versioned build tags like 12.2.0-1234 for main-branch builds.[2] For stable production pinning, prefer the exact release/version tag you intend to run (or, if you only want to track a minor line, use the documented minor tag format like 12.1).[2]

Citations:


Pin the Grafana image to a fixed release tag (avoid grafana/grafana:latest).

grafana/grafana:latest currently resolves to grafana/grafana:13.1.0-25469333600, and Grafana publishes stable release tags as X.Y.Z-<buildID> (not plain X.Y.Z). Pin to the exact release tag you want for reproducible deploys.

♻️ Suggested change
-          image: grafana/grafana:latest
+          image: grafana/grafana:13.1.0-25469333600
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@deploy/k8s/monitoring/grafana.yaml` at line 50, Replace the floating image
reference "image: grafana/grafana:latest" in the Grafana Deployment/StatefulSet
spec with a specific Grafana release tag (for example
"grafana/grafana:13.1.0-25469333600" or your chosen X.Y.Z-<buildID>) to ensure
reproducible deployments; update the manifest entry that currently contains
image: grafana/grafana:latest so the container spec uses the pinned tag and
commit the change.

spec:
containers:
- name: prometheus
image: prom/prometheus:latest

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🌐 Web query:

What is the latest stable release version of the prom/prometheus Docker image?

💡 Result:

The latest stable release Prometheus Docker image version (for the prom/prometheus image) is 3.12.0, i.e. use: prom/prometheus:v3.12.0 (and prom/prometheus:latest points to the default “busybox” variant). [1][2] Evidence: - The upstream Prometheus project’s latest release is v3.12.0 (dated 2026-05-28). [2] - The prom/prometheus Docker image provides versioned tags such as v3.11.3 and (by extension of their version-tagging scheme) includes v3.12.0; additionally Docker Hub shows “latest” is a tag mapped to the default busybox variant. [3][4]

Citations:


Pin the Prometheus image to a fixed version.

prom/prometheus:latest can drift between runs; pin to the current stable release tag prom/prometheus:v3.12.0.

♻️ Suggested change
-          image: prom/prometheus:latest
+          image: prom/prometheus:v3.12.0
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
image: prom/prometheus:latest
image: prom/prometheus:v3.12.0
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@deploy/k8s/monitoring/prometheus.yaml` at line 48, Replace the floating image
tag to a fixed, reproducible Prometheus version: locate the manifest entry where
the container image is set to "prom/prometheus:latest" and change it to the
pinned stable tag "prom/prometheus:v3.12.0" so deployments use a deterministic
Prometheus release.

Comment thread deploy/k8s/pgbouncer.yaml
Comment on lines +42 to +43
- name: pgbouncer
image: edoburu/pgbouncer:latest

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🌐 Web query:

What immutable tag and digest should be used for the Docker image edoburu/pgbouncerthat supports theDB_HOST, DB_PORT, DB_USER, DB_PASSWORD, DB_NAME, POOL_MODE, AUTH_TYPE, MAX_CLIENT_CONN, DEFAULT_POOL_SIZE, and LISTEN_PORT environment variables used here?

💡 Result:

Answer: Use an immutable tag that maps to a specific PgBouncer version (not latest) and pin by digest from Docker Hub. However, the exact immutable tag/digest cannot be uniquely determined from the sources we retrieved in this research run, because Docker Hub provides multiple digests for different platforms/tags and we did not capture the specific tag that corresponds to your desired option set or the current default latest. What we can state with evidence: - The edoburu/docker-pgbouncer project documents configuration via environment variables, including options like DB_HOST, DB_PORT, DB_USER, DB_PASSWORD, DB_NAME, and PgBouncer settings such as POOL_MODE and MAX_CLIENT_CONN that are passed through to pgbouncer.ini-equivalent configuration.[1][2] - Docker Hub exposes digests for edoburu/pgbouncer under the tags page, meaning you should pin by digest to make it immutable.[3] How to get the correct immutable tag+digest for your deployment (recommended): 1) Pick the exact immutable tag you want (e.g., a semver tag rather than latest). The repository discusses that it provides versioned tags mapped to the upstream pgbouncer versions.[4] 2) Then resolve that exact tag to its immutable digest on Docker Hub and pin using @sha256:<digest>. - Use Docker Hub’s tag/digest listing for edoburu/pgbouncer and copy the digest for your chosen tag/architecture.[3] If you tell me which immutable tag you were using originally (for example edoburu/pgbouncer:1.15.0 or similar) and your target platform (linux/amd64 vs linux/arm64), I can give you the precise edoburu/pgbouncer:<tag>@sha256:<digest> pin for that exact image variant using the Docker Hub digest table.[3]

Citations:


Pin PgBouncer to an immutable image reference (remove latest).

edoburu/pgbouncer:latest makes this manifest non-reproducible since the resolved image can change over time (and digests vary by tag/architecture). Pin a specific PgBouncer version and include the matching @sha256:... digest for the target platform.

Minimal fix
-          image: edoburu/pgbouncer:latest
+          image: edoburu/pgbouncer:<immutable-tag>`@sha256`:<sha256-digest>
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@deploy/k8s/pgbouncer.yaml` around lines 42 - 43, The image reference for the
pgbouncer container currently uses the mutable tag "edoburu/pgbouncer:latest";
update the image field for the container named "pgbouncer" to a fixed, immutable
reference by replacing ":latest" with a specific version tag and the matching
digest (e.g., edoburu/pgbouncer:<version>`@sha256`:<digest>) so deployments are
reproducible and platform-specific; fetch the correct version and sha256 for
your target architecture and substitute it into the image line.

Comment on lines +41 to +90
spec:
containers:
- name: postgres
image: pgvector/pgvector:pg16
ports:
- name: postgres
containerPort: 5432
env:
- name: POSTGRES_USER
valueFrom:
configMapKeyRef:
name: second-brain-config
key: POSTGRES_USER
- name: POSTGRES_DB
valueFrom:
configMapKeyRef:
name: second-brain-config
key: POSTGRES_DB
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: second-brain-secrets
key: POSTGRES_PASSWORD
# Init into a subdir so the PV mount root (which may carry a lost+found) is never PGDATA.
- name: PGDATA
value: /var/lib/postgresql/data/pgdata
readinessProbe:
exec:
command: ["sh", "-c", 'pg_isready -U "$POSTGRES_USER" -d "$POSTGRES_DB"']
initialDelaySeconds: 5
periodSeconds: 5
timeoutSeconds: 3
failureThreshold: 12
livenessProbe:
exec:
command: ["sh", "-c", 'pg_isready -U "$POSTGRES_USER" -d "$POSTGRES_DB"']
initialDelaySeconds: 20
periodSeconds: 10
timeoutSeconds: 3
failureThreshold: 6
resources:
requests:
cpu: "100m"
memory: "256Mi"
limits:
cpu: "500m"
memory: "512Mi"
volumeMounts:
- name: data
mountPath: /var/lib/postgresql/data

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Locate and print the postgres StatefulSet manifest section around the reported lines
sed -n '1,220p' deploy/k8s/postgres-statefulset.yaml | nl -ba | sed -n '1,220p'
echo "----"

# Search for any existing securityContext usage in deploy/k8s
rg -n "securityContext:" deploy/k8s || true
echo "----"

# Display configmap/secret references if they exist, for UID/GID clues (less likely but may help)
ls -la deploy/k8s || true
for f in deploy/k8s/configmap.yaml deploy/k8s/secret.example.yaml deploy/k8s/secret.yaml; do
  if [ -f "$f" ]; then
    echo "== $f =="
    sed -n '1,260p' "$f" | nl -ba | sed -n '1,260p'
    echo "----"
  fi
done

Repository: tomnguyen103/second-brain

Length of output: 112


🏁 Script executed:

#!/bin/bash
set -euo pipefail

FILE="deploy/k8s/postgres-statefulset.yaml"
if [ -f "$FILE" ]; then
  echo "== $FILE (lines 35-120) =="
  sed -n '35,120p' "$FILE" | awk '{printf "%5d  %s\n", NR+34, $0}'
else
  echo "Missing file: $FILE" >&2
fi

echo "----"
echo "== securityContext occurrences in deploy/k8s =="
rg -n "securityContext:" deploy/k8s || true

echo "----"
echo "== securityContext occurrences in repo (limited) =="
rg -n "securityContext:" . || true

echo "----"
echo "== show configmap.yaml =="
if [ -f "deploy/k8s/configmap.yaml" ]; then
  sed -n '1,120p' deploy/k8s/configmap.yaml
else
  echo "Missing deploy/k8s/configmap.yaml" >&2
fi

echo "----"
echo "== show secret.example.yaml =="
if [ -f "deploy/k8s/secret.example.yaml" ]; then
  sed -n '1,200p' deploy/k8s/secret.example.yaml
else
  echo "Missing deploy/k8s/secret.example.yaml" >&2
fi

Repository: tomnguyen103/second-brain

Length of output: 4855


Add least-privilege securityContext to the Postgres StatefulSet

deploy/k8s/postgres-statefulset.yaml sets no pod/container securityContext, so the container runs with default root-capable and privilege-escalation/capability settings while mounting the PVC and reading second-brain-secrets.

Suggested hardening baseline
     spec:
+      securityContext:
+        runAsNonRoot: true
+        fsGroup: 999
       containers:
         - name: postgres
           image: pgvector/pgvector:pg16
+          securityContext:
+            allowPrivilegeEscalation: false
+            capabilities:
+              drop: ["ALL"]
           ports:
             - name: postgres
               containerPort: 5432

If you also pin runAsUser, ensure it matches the UID/GID expected by pgvector/pgvector:pg16 and that the PGDATA directory remains writable.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
spec:
containers:
- name: postgres
image: pgvector/pgvector:pg16
ports:
- name: postgres
containerPort: 5432
env:
- name: POSTGRES_USER
valueFrom:
configMapKeyRef:
name: second-brain-config
key: POSTGRES_USER
- name: POSTGRES_DB
valueFrom:
configMapKeyRef:
name: second-brain-config
key: POSTGRES_DB
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: second-brain-secrets
key: POSTGRES_PASSWORD
# Init into a subdir so the PV mount root (which may carry a lost+found) is never PGDATA.
- name: PGDATA
value: /var/lib/postgresql/data/pgdata
readinessProbe:
exec:
command: ["sh", "-c", 'pg_isready -U "$POSTGRES_USER" -d "$POSTGRES_DB"']
initialDelaySeconds: 5
periodSeconds: 5
timeoutSeconds: 3
failureThreshold: 12
livenessProbe:
exec:
command: ["sh", "-c", 'pg_isready -U "$POSTGRES_USER" -d "$POSTGRES_DB"']
initialDelaySeconds: 20
periodSeconds: 10
timeoutSeconds: 3
failureThreshold: 6
resources:
requests:
cpu: "100m"
memory: "256Mi"
limits:
cpu: "500m"
memory: "512Mi"
volumeMounts:
- name: data
mountPath: /var/lib/postgresql/data
spec:
securityContext:
runAsNonRoot: true
fsGroup: 999
containers:
- name: postgres
image: pgvector/pgvector:pg16
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
ports:
- name: postgres
containerPort: 5432
env:
- name: POSTGRES_USER
valueFrom:
configMapKeyRef:
name: second-brain-config
key: POSTGRES_USER
- name: POSTGRES_DB
valueFrom:
configMapKeyRef:
name: second-brain-config
key: POSTGRES_DB
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: second-brain-secrets
key: POSTGRES_PASSWORD
# Init into a subdir so the PV mount root (which may carry a lost+found) is never PGDATA.
- name: PGDATA
value: /var/lib/postgresql/data/pgdata
readinessProbe:
exec:
command: ["sh", "-c", 'pg_isready -U "$POSTGRES_USER" -d "$POSTGRES_DB"']
initialDelaySeconds: 5
periodSeconds: 5
timeoutSeconds: 3
failureThreshold: 12
livenessProbe:
exec:
command: ["sh", "-c", 'pg_isready -U "$POSTGRES_USER" -d "$POSTGRES_DB"']
initialDelaySeconds: 20
periodSeconds: 10
timeoutSeconds: 3
failureThreshold: 6
resources:
requests:
cpu: "100m"
memory: "256Mi"
limits:
cpu: "500m"
memory: "512Mi"
volumeMounts:
- name: data
mountPath: /var/lib/postgresql/data
🧰 Tools
🪛 Trivy (0.69.3)

[error] 42-89: Root file system is not read-only

Container 'postgres' of StatefulSet 'db' should set 'securityContext.readOnlyRootFilesystem' to true

Rule: KSV-0014

Learn more

(IaC/Kubernetes)


[error] 42-89: Default security context configured

container db in second-brain namespace is using the default security context

Rule: KSV-0118

Learn more

(IaC/Kubernetes)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@deploy/k8s/postgres-statefulset.yaml` around lines 41 - 90, The StatefulSet
pod/container lacks securityContext causing the Postgres container (container
name "postgres") to run with default root-capable privileges; add a pod-level
securityContext (fsGroup and runAsNonRoot:true) and a container-level
securityContext for the "postgres" container that sets runAsUser and runAsGroup
to the UID/GID expected by the pgvector/pgvector:pg16 image, set
allowPrivilegeEscalation: false, readOnlyRootFilesystem: false only if PGDATA
writable, and drop all Linux capabilities (capabilities.drop: ["ALL"]) to
enforce least privilege for the mounted PVC (volumeMount name "data") and access
to secrets (second-brain-secrets); verify the chosen runAsUser/runAsGroup keep
/var/lib/postgresql/data/pgdata writable and adjust FS permissions if needed.

Comment thread deploy/k8s/worker.yaml
Comment on lines +22 to +62
spec:
containers:
- name: worker
image: second-brain-api:phase7
imagePullPolicy: IfNotPresent
command: ["python", "-m", "app.jobs.worker", "--loop"]
env:
- name: POSTGRES_USER
valueFrom:
configMapKeyRef:
name: second-brain-config
key: POSTGRES_USER
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: second-brain-secrets
key: POSTGRES_PASSWORD
- name: POSTGRES_DB
valueFrom:
configMapKeyRef:
name: second-brain-config
key: POSTGRES_DB
- name: SECOND_BRAIN_DATABASE_URL
value: postgresql+psycopg://$(POSTGRES_USER):$(POSTGRES_PASSWORD)@pgbouncer:6432/$(POSTGRES_DB)
- name: SECOND_BRAIN_LLM_PROVIDER
valueFrom:
configMapKeyRef:
name: second-brain-config
key: SECOND_BRAIN_LLM_PROVIDER
- name: SECOND_BRAIN_GEMINI_API_KEY
valueFrom:
secretKeyRef:
name: second-brain-secrets
key: SECOND_BRAIN_GEMINI_API_KEY
resources:
requests:
cpu: "100m"
memory: "512Mi"
limits:
cpu: "500m"
memory: "1Gi"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# 1) Show the full relevant section of worker.yaml around the snippet
echo "=== worker.yaml: lines 1-140 (for context) ==="
nl -ba deploy/k8s/worker.yaml | sed -n '1,140p'

# 2) Search for securityContext in worker.yaml
echo "=== worker.yaml: securityContext occurrences ==="
rg -n "securityContext" deploy/k8s/worker.yaml || true

# 3) Check all k8s manifests for securityContext usage (API counterpart too)
echo "=== deploy/k8s: securityContext occurrences across manifests ==="
rg -n "securityContext" deploy/k8s || true

# 4) Identify the API manifest and show its securityContext (heuristic: name contains api / endpoint)
echo "=== deploy/k8s: candidate API manifests ==="
ls -1 deploy/k8s || true

Repository: tomnguyen103/second-brain

Length of output: 177


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Additional targeted inspection:
# Find manifests that look like API deployments and show their pod securityContext/container securityContext.
for f in $(fd -e yaml deploy/k8s | sort); do
  if rg -q "(kind: Deployment|kind: StatefulSet|name: api|image: .*api|worker)" "$f"; then
    echo "=== $f: kind + metadata.name + securityContext blocks ==="
    rg -n "kind:|metadata:|name:|securityContext|containers:" "$f" | head -n 200
  fi
done

Repository: tomnguyen103/second-brain

Length of output: 415


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Ensure target exists
ls -la deploy/k8s || true

# 1) Print worker.yaml with line numbers (nl unavailable)
echo "=== worker.yaml: first 200 lines ==="
awk '{printf "%5d %s\n", NR, $0}' deploy/k8s/worker.yaml | sed -n '1,200p'

# 2) Find securityContext occurrences in worker.yaml
echo "=== worker.yaml: securityContext occurrences ==="
rg -n "securityContext" deploy/k8s/worker.yaml || true

# 3) List YAML manifests under deploy/k8s (fix fd usage)
echo "=== YAML manifests under deploy/k8s ==="
fd --type f -e yaml . deploy/k8s | sort | head -n 200

# 4) Show securityContext usage across deploy/k8s manifests
echo "=== deploy/k8s: securityContext occurrences across all manifests ==="
rg -n "securityContext" deploy/k8s -S || true

# 5) Show deployments/containers that reference worker or api images
echo "=== deploy/k8s: manifests referencing worker/api keywords (context) ==="
rg -n --no-heading "(name: worker|name: api|worker\.yaml|image: .*api|app\.jobs\.worker)" deploy/k8s -S || true

Repository: tomnguyen103/second-brain

Length of output: 5166


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "=== api.yaml security hardening keys ==="
rg -n "securityContext|runAsNonRoot|readOnlyRootFilesystem|allowPrivilegeEscalation|seccompProfile" deploy/k8s/api.yaml || true

echo "=== migrate-job.yaml security hardening keys ==="
rg -n "securityContext|runAsNonRoot|readOnlyRootFilesystem|allowPrivilegeEscalation|seccompProfile" deploy/k8s/migrate-job.yaml || true

echo "=== any of these keys anywhere under deploy/k8s ==="
rg -n "securityContext|runAsNonRoot|readOnlyRootFilesystem|allowPrivilegeEscalation|seccompProfile" deploy/k8s -S || true

Repository: tomnguyen103/second-brain

Length of output: 211


Harden the worker pod securityContext to match the API security posture.

deploy/k8s/worker.yaml’s worker Deployment pod template has no securityContext, so it can run with default/root permissions and a writable root filesystem.

Suggested manifest hardening
     spec:
+      securityContext:
+        runAsNonRoot: true
+        seccompProfile:
+          type: RuntimeDefault
       containers:
         - name: worker
+          securityContext:
+            allowPrivilegeEscalation: false
+            capabilities:
+              drop: ["ALL"]
+            readOnlyRootFilesystem: true
           image: second-brain-api:phase7

If the worker needs scratch space, add an emptyDir mount for that path instead of leaving the full root FS writable.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
spec:
containers:
- name: worker
image: second-brain-api:phase7
imagePullPolicy: IfNotPresent
command: ["python", "-m", "app.jobs.worker", "--loop"]
env:
- name: POSTGRES_USER
valueFrom:
configMapKeyRef:
name: second-brain-config
key: POSTGRES_USER
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: second-brain-secrets
key: POSTGRES_PASSWORD
- name: POSTGRES_DB
valueFrom:
configMapKeyRef:
name: second-brain-config
key: POSTGRES_DB
- name: SECOND_BRAIN_DATABASE_URL
value: postgresql+psycopg://$(POSTGRES_USER):$(POSTGRES_PASSWORD)@pgbouncer:6432/$(POSTGRES_DB)
- name: SECOND_BRAIN_LLM_PROVIDER
valueFrom:
configMapKeyRef:
name: second-brain-config
key: SECOND_BRAIN_LLM_PROVIDER
- name: SECOND_BRAIN_GEMINI_API_KEY
valueFrom:
secretKeyRef:
name: second-brain-secrets
key: SECOND_BRAIN_GEMINI_API_KEY
resources:
requests:
cpu: "100m"
memory: "512Mi"
limits:
cpu: "500m"
memory: "1Gi"
spec:
securityContext:
runAsNonRoot: true
seccompProfile:
type: RuntimeDefault
containers:
- name: worker
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
readOnlyRootFilesystem: true
image: second-brain-api:phase7
imagePullPolicy: IfNotPresent
command: ["python", "-m", "app.jobs.worker", "--loop"]
env:
- name: POSTGRES_USER
valueFrom:
configMapKeyRef:
name: second-brain-config
key: POSTGRES_USER
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: second-brain-secrets
key: POSTGRES_PASSWORD
- name: POSTGRES_DB
valueFrom:
configMapKeyRef:
name: second-brain-config
key: POSTGRES_DB
- name: SECOND_BRAIN_DATABASE_URL
value: postgresql+psycopg://$(POSTGRES_USER):$(POSTGRES_PASSWORD)`@pgbouncer`:6432/$(POSTGRES_DB)
- name: SECOND_BRAIN_LLM_PROVIDER
valueFrom:
configMapKeyRef:
name: second-brain-config
key: SECOND_BRAIN_LLM_PROVIDER
- name: SECOND_BRAIN_GEMINI_API_KEY
valueFrom:
secretKeyRef:
name: second-brain-secrets
key: SECOND_BRAIN_GEMINI_API_KEY
resources:
requests:
cpu: "100m"
memory: "512Mi"
limits:
cpu: "500m"
memory: "1Gi"
🧰 Tools
🪛 Checkov (3.2.530)

[medium] 4-62: Containers should not run with allowPrivilegeEscalation

(CKV_K8S_20)


[medium] 4-62: Minimize the admission of root containers

(CKV_K8S_23)

🪛 Trivy (0.69.3)

[error] 24-62: Root file system is not read-only

Container 'worker' of Deployment 'worker' should set 'securityContext.readOnlyRootFilesystem' to true

Rule: KSV-0014

Learn more

(IaC/Kubernetes)


[error] 24-62: Default security context configured

container worker in second-brain namespace is using the default security context

Rule: KSV-0118

Learn more

(IaC/Kubernetes)


[error] 22-62: Default security context configured

deployment worker in second-brain namespace is using the default security context, which allows root privileges

Rule: KSV-0118

Learn more

(IaC/Kubernetes)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@deploy/k8s/worker.yaml` around lines 22 - 62, The pod template for the worker
container (container name "worker", command "python -m app.jobs.worker") lacks a
securityContext; add a pod-level securityContext and container-level
securityContext to harden it: set podSecurityContext.runAsNonRoot=true and
runAsUser to a non-root UID (e.g., 1000), and on the container set
securityContext.readOnlyRootFilesystem=true, allowPrivilegeEscalation=false,
drop all capabilities and only add needed ones if any; if the worker requires
writable scratch space, add an emptyDir volume and mount it at the specific
writable path instead of leaving the root FS writable (reference the "worker"
container and the pod template to locate these changes).

@tomnguyen103
tomnguyen103 merged commit fdc2885 into main Jun 2, 2026
9 checks passed
@tomnguyen103
tomnguyen103 deleted the phase-7-impl branch June 26, 2026 19:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant