Skip to content
Draft
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 17 additions & 2 deletions .agent/skills/webstatus-backend/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,21 @@ This skill provides guidance for developing the Go-based backend API for `websta

For a technical deep-dive into the backend implementation patterns, request flows, and auth middleware, see [references/architecture.md](references/architecture.md).

## Development Environments

The project supports two primary development environments:

- **VS Code DevContainer**: A Docker-based environment with all tools pre-installed.
- **Nix (Alternative)**: A lightweight environment with pinned tool versions (Go 1.26.1). Enter via `nix develop`.

### Tool Management

- **Core Tools**: Go and Node are provided by the environment (Nix or DevContainer).
- **CLI Tools**: Tools like `wrench` and `oapi-codegen` are managed via `tools/go.mod` and executed via `make` targets (which use `go tool`).

> [!IMPORTANT]
> A **container runtime** (Docker or Podman) is required on the host machine in both environments to run Spanner integration tests (via `testcontainers-go`) and the local emulator.

## Guides

- **[Add a New API Endpoint](references/add-api-endpoint.md)**: Mandatory spec-first process.
Expand Down Expand Up @@ -52,15 +67,15 @@ We use a Hexagonal-style **Adapter Pattern** to decouple application logic from

## Testing & Linting

- **Precommit Suite**: Run `make precommit` to execute the full suite of Go tests, formatting, and linting.
- **Precommit Suite**: Run `make precommit` to execute the full suite of Go tests, formatting, and linting. This uses the pinned tool versions provided by your environment.
- **Linting**: Run `make go-lint` to lint all Go code using `golangci-lint`.
- **Quick Test Iteration**: Because this project uses a multi-module workspace (`go.work`), to run tests quickly for a single package without running the whole suite, execute `go test` from _within_ the specific module directory, or provide the full module path:
```bash
cd backend && go test -v ./pkg/...
# Or
go test -v github.com/GoogleChrome/webstatus.dev/lib/gcpspanner/...
```
- **Integration Tests**: Any changes to `lib/gcpspanner` **MUST** include integration tests using `testcontainers-go` against the Spanner emulator.
- **Integration Tests**: Any changes to `lib/gcpspanner` **MUST** include integration tests using `testcontainers-go` against the Spanner emulator. Remember that this requires a working container runtime on your host!

## Documentation Updates

Expand Down
11 changes: 11 additions & 0 deletions .agent/skills/webstatus-e2e/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,17 @@ For a detailed technical guide on the local development environment (Skaffold/Mi
- **Retries**: Playwright tests are configured to retry twice on failure only when running in a CI environment. If you want to simulate this locally and test flakiness, you can prefix your command with `CI=true` (e.g., `CI=true make playwright-test`).
- **Browsers**: If you ever need to test against new browsers (e.g., mobile viewports, branded Edge/Chrome), modify the `projects` array within `playwright.config.ts`.

## Nix Environment & Docker Browser

When working in the Nix development environment (via `nix develop` or `direnv`), special handling is required for Playwright:

- **Isolated Browsers**: Browsers are isolated to `.nix/browsers` to avoid affecting host installations.
- **Docker Browser Server**: To guarantee 100% font parity for screenshots, the environment defaults to `USE_DOCKER_BROWSER=true`. This automatically spins up a Docker container and connects to it via WebSocket.
- **Known Limitation**: The "Show browsers" option in the VS Code Playwright extension is **broken** when using the Docker server.
- **Workarounds**:
- Use UI Mode (`make playwright-ui` or `npx playwright test --ui`).
- Temporarily disable Docker to use local Nix-patched browsers: `USE_DOCKER_BROWSER=false npx playwright test --headed`.

## Execution & Debugging

- For detailed instructions on rapid iteration, debugging CI failures, and using traces, see [references/execution-and-debugging.md](references/execution-and-debugging.md).
Expand Down
14 changes: 13 additions & 1 deletion .agent/skills/webstatus-frontend/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,13 @@ This skill provides architectural guidance and conventions for the `frontend/` d
- **State Management**: Uses **Lit Context** for dependency injection and state management via a service container pattern (`<webstatus-services-container>`).
- **API Interaction**: Communicates with the Go backend using TypeScript types generated from the OpenAPI specification (`make node-openapi`).

## Development Environments

The project supports two primary development environments:

- **VS Code DevContainer**: A Docker-based environment with all tools pre-installed.
- **Nix (Alternative)**: A lightweight environment with pinned tool versions (Node.js 24.14.0). Enter via `nix develop`.

## Architecture

For a technical breakdown of the Lit component hierarchy, frontend identity flows, and theming patterns, see [references/architecture.md](references/architecture.md).
Expand All @@ -36,7 +43,11 @@ For a technical breakdown of the Lit component hierarchy, frontend identity flow

## Testing & Linting

- **Test Execution**: `npm run test -w frontend`.
- **Test Execution**: `npm run test -w frontend` or `make node-test`.
- **Nix Environment**:
- Browsers are isolated to `.nix/browsers`.
- If browsers are missing, run `npx playwright install` within the `nix develop` shell.
- Note: Frontend unit tests always use local browsers, even in Nix (unlike E2E tests which may default to a Docker-based browser).
- **Linting**: Run `make node-lint` to run ESLint and Prettier for the frontend code, or `make lint-fix` to attempt auto-fixing. `make style-lint` is also available for CSS.
- **ES Module Testing**: When testing components that use ES module exports directly (e.g. Firebase Auth), use a helper property (e.g. `credentialGetter`) that can be overridden with a Sinon stub.
- **Typing**: Use generic arguments for `querySelector` in tests (e.g. `querySelector<HTMLSlotElement>(...)`) for type safety.
Expand All @@ -54,6 +65,7 @@ If a frontend unit test is timing out or failing mysteriously, Web Test Runner's

- **Watch Mode**: Instruct the user to run `npm run test:watch -w frontend` in their own terminal.
- **Visual Debugging**: Ask the user to open the provided localhost URL (e.g., `http://localhost:8000/`) in their web browser and inspect the developer console/DOM to see where the test is getting stuck.
- **Wayland Support**: In the Nix environment, Firefox is automatically configured to run natively if Wayland is detected (`MOZ_ENABLE_WAYLAND=1`).
- **DON'T** arbitrarily increase the timeout in `web-test-runner.config.mjs` to fix timeout issues. Address the root cause of the hang instead.

## Documentation Updates
Expand Down
12 changes: 11 additions & 1 deletion .agent/skills/webstatus-ingestion/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,16 @@ This skill provides guidance for developing and deploying the scheduled data ing

For a detailed map of data sources, Spanner target tables, and job orchestration patterns, see [references/architecture.md](references/architecture.md).

## Development Environments

The project supports two primary development environments:

- **VS Code DevContainer**: A Docker-based environment with all tools pre-installed.
- **Nix (Alternative)**: A lightweight environment with pinned tool versions. Enter via `nix develop`.

> [!IMPORTANT]
> Even when using the Nix environment, a **container runtime** (Docker or Podman) is still required on your host machine to run the local Spanner emulator and test local job execution via Skaffold.

## Infrastructure Abstraction (The Adapter Pattern)

Ingestion jobs must be decoupled from the core DB logic and the "Backend" API.
Expand All @@ -44,7 +54,7 @@ Ingestion jobs must be decoupled from the core DB logic and the "Backend" API.

## Testing & Linting

- **Precommit Suite**: Run `make precommit` to execute the full suite of Go tests, formatting, and linting.
- **Precommit Suite**: Run `make precommit` to execute the full suite of Go tests, formatting, and linting. Ensure you are inside your chosen environment (Nix or DevContainer) to use the correct tool versions.
- **Linting**: Run `make go-lint` to lint all Go code using `golangci-lint`.
- **Quick Test Iteration**: Because this project uses a multi-module workspace (`go.work`), to run tests quickly for a single package without running the whole suite, execute `go test` from _within_ the specific module directory:
```bash
Expand Down
24 changes: 21 additions & 3 deletions .agent/skills/webstatus-maintenance/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: webstatus-maintenance
description: Use when upgrading toolchain versions (Go, Node.js, Terraform, Playwright) or updating the DevContainer and Github CI configurations.
description: Use when upgrading toolchain versions (Go, Node.js, Terraform, Playwright), updating the DevContainer, Nix environment, or Github CI configurations.
---

# webstatus-maintenance
Expand All @@ -27,6 +27,7 @@ The Go version must be kept in sync across:
2. `.github/workflows/ci.yml` (`GO_VERSION` environment variable)
3. `.github/workflows/devcontainer.yml` (`GO_VERSION` environment variable)
4. `images/go_service.Dockerfile` (`FROM golang:X.Y.Z-alpine...`)
5. `flake.nix` (If pinning a specific `nixpkgs` commit for Go)

After updating the files, you should run `make go-update && make go-tidy` to ensure the `go.mod` dependencies are compatible with the new version.

Expand All @@ -39,6 +40,7 @@ The Node.js version must be kept in sync across:
3. `.github/workflows/ci.yml` (`NODE_VERSION` environment variable)
4. `.github/workflows/devcontainer.yml` (`NODE_VERSION` environment variable)
5. `images/nodejs_service.Dockerfile` (`FROM node:X.Y.Z-alpine...`)
6. `flake.nix` (If pinning a specific `nixpkgs` commit for Node.js)

After updating, run `make node-update` and test the frontend build.

Expand All @@ -48,17 +50,33 @@ Playwright requires its NPM package and OS-level dependencies to stay in sync:

1. Update `playwright` and `@web/test-runner-playwright` in `frontend/package.json`.
2. Update the system dependencies in `.github/workflows/ci.yml` (the `npx playwright install --with-deps` step).
3. On Nix: Playwright browsers are cached in `.nix/browsers` and patched automatically by the shell hook in `flake.nix`.

### Upgrading via Nix

The Nix environment provides an alternative toolchain.

- **Bumping All Tools**: Run `nix flake update` to update all tools to their latest versions in `nixpkgs-unstable`.
- **Pinning Versions**: To pin a specific tool version, update `flake.nix` to use a specific `nixpkgs` commit hash (see `docs/nix-setup.md`).

### Upgrading Go-based Tools

Tools like `wrench`, `oapi-codegen`, and `golines` are managed via `tools/go.mod`.

- To upgrade a tool, navigate to the `tools/` directory and run `go get <package>@<version>`.
- Run `go mod tidy` in the `tools/` directory.

### Upgrading DevContainer Features

Other DevContainer tools (Terraform, Skaffold, Shellcheck, GitHub CLI) are managed within `.devcontainer/devcontainer.json`.
Other DevContainer tools (Skaffold, Shellcheck, GitHub CLI) are managed within `.devcontainer/devcontainer.json`.

- Find the relevant feature under the `features` object and update its `"version"`.
- If modifying Terraform, also ensure the `.terraform-version` file (if one exists) or CI checks match the new version.
- If modifying Terraform, also ensure `infra/providers.tf` (`required_version`) and CI checks match the new version.

## Documentation Updates

If you change how versions are managed or introduce a new critical dependency:

- Update `docs/maintenance.md` to reflect the new update path.
- Update `docs/nix-setup.md` if the Nix environment logic changes.
- Trigger the "Updating the Knowledge Base" prompt in `GEMINI.md` to ensure I am aware of the changes.
6 changes: 5 additions & 1 deletion .agent/skills/webstatus-pr-creation/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,14 @@ When you have finished implementing a feature or fixing a bug and the user asks

1. **Verify the State**: Run `git status` to see what files have been modified.
2. **Review the Diffs**: Before committing, use `git diff` to quickly review the changes. Ensure you haven't left any stray `console.log` statements, commented-out debugger code, or unresolved merge conflict markers.
3. **Format, Lint, and Style**: Run standard project linters (`make precommit`, `make go-lint`, or `make node-lint`). Additionally, cross-reference your changes against the project's specific skills (e.g., `webstatus-backend`, `webstatus-frontend`) and standard Google Style Guides (e.g., Google Go Style Guide, Google TypeScript Style Guide). If you are unsure about a style rule, you may search for it or ask the user.
3. **Format, Lint, and Style**: Run standard project linters (`make precommit`, `make go-lint`, or `make node-lint`).
- **Environment**: Ensure you are in the `nix develop` shell or the project's DevContainer so you use the correct tool versions.
- **Regeneration**: If you modified OpenAPI specs, JSON schemas, or ANTLR grammars, you MUST run `make gen` before committing to ensure generated code is in sync.
- **Style Guides**: Cross-reference your changes against the project's specific skills (e.g., `webstatus-backend`, `webstatus-frontend`) and standard Google Style Guides. If you are unsure about a style rule, you may search for it or ask the user.
4. **Create a New Branch**: NEVER commit directly to `main`.
- Run `git checkout -b feature/<short-description>` or `git checkout -b fix/<short-description>`
5. **Stage Files**: Add the specific files you modified using `git add <file1> <file2>`. Try to avoid `git add .` unless you are absolutely certain no unrelated files (like local IDE configs) are present.
- **Nix**: If you modified `flake.nix`, ensure you also run `nix flake update` and stage the updated `flake.lock`.
6. **Write a Descriptive Commit Message**: You MUST use the Conventional Commits format (`type(scope): subject`). Make sure to prefix the commit with `feat:`, `fix:`, `chore:`, `docs:`, `test:`, or `refactor:`.
- _Example_: `feat: implement dark mode component`
- _Example_: `fix(api): handle missing search name in payload`
Expand Down
9 changes: 9 additions & 0 deletions .agent/skills/webstatus-search-grammar/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,15 @@ For a technical breakdown of the ANTLR grammar, search node transformation, and
- The canonical source of truth for the search syntax is `antlr/FeatureSearch.g4`.
- **DON'T** edit the generated parser files in `lib/gen/featuresearch/parser/` directly.

## Tooling & Environment

The search grammar relies on **ANTLR v4**. The project is configured to handle tool execution seamlessly across different development environments:

- **Nix**: The environment provides the `antlr4` package and sets the `ANTLR=antlr4` environment variable in the shell.
- **DevContainer**: The environment vendors the ANTLR JAR at `/usr/local/lib/antlr-4.13.2-complete.jar` and the `Makefile` defaults to this path if `ANTLR` is not set.

**Always use `make antlr-gen`** to regenerate the parser. This target abstracts the environment differences and ensures the correct Go-specific flags (`-Dlanguage=Go -visitor -no-listener`) and output directories are used.

## General Guidelines

- **DO** cross-reference all Go and test code against the official Google Go Style Guide. If you are unsure about a specific style rule, DO NOT assume; you MUST ask the user for clarification.
Expand Down
10 changes: 10 additions & 0 deletions .agent/skills/webstatus-workers/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,16 @@ Workers must use the shared structs in [lib/workertypes/types.go](../../lib/work
- The workers run locally via Skaffold and connect to local emulators for Spanner (`spanner:9010`) and Pub/Sub (`pubsub:8060`).
- The `FRONTEND_BASE_URL` locally is usually `http://localhost:5555`.

## Development Environments

The project supports two primary development environments:

- **VS Code DevContainer**: A Docker-based environment with all tools pre-installed.
- **Nix (Alternative)**: A lightweight environment with pinned tool versions. Enter via `nix develop`.

> [!IMPORTANT]
> Even when using the Nix environment, a **container runtime** (Docker or Podman) is still required on your host machine to run the local emulators and test local execution via Skaffold.

## Infrastructure Abstraction (The Adapter Pattern)

All workers must be decoupled from GCP-specific SDKs.
Expand Down
105 changes: 105 additions & 0 deletions .github/workflows/ci-nix.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

name: 'ci-nix'
on:
pull_request:
merge_group:
workflow_dispatch:

jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Maximize build space
uses: easimon/maximize-build-space@fc881a613ad2a34aca9c9624518214ebc21dfc0c
with:
remove-dotnet: 'true'
remove-android: 'true'
remove-haskell: 'true'
remove-codeql: 'true'
root-reserve-mb: 2048
overprovision-lvm: 'true'
build-mount-path: '/home/runner/work/webstatus.dev'
continue-on-error: true

- name: Create Workspace
run: mkdir -p ${{ github.workspace }}

- name: Checkout (GitHub)
uses: actions/checkout@v6

- name: Move Docker
run: |
sudo service docker stop
mkdir -p ../docker-storage
sudo rm -rf /var/lib/docker
sudo ln -s $(realpath ../docker-storage) /var/lib/docker
sudo service docker start

- name: Install Nix
uses: DeterminateSystems/nix-installer-action@ef8a148080ab6020fd15196c2084a2eea5ff2d25

- name: Magic Nix Cache
uses: DeterminateSystems/magic-nix-cache-action@565684385bcd71bad329742eefe8d12f2e765b39

- name: Run precommit target for CI via Nix
run: nix develop --command make precommit

playwright:

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {contents: read}
Comment thread
github-advanced-security[bot] marked this conversation as resolved.
Fixed
runs-on: ubuntu-latest
steps:
- name: Maximize build space
uses: easimon/maximize-build-space@fc881a613ad2a34aca9c9624518214ebc21dfc0c
with:
remove-dotnet: 'true'
remove-android: 'true'
remove-haskell: 'true'
remove-codeql: 'true'
root-reserve-mb: 2048
overprovision-lvm: 'true'
build-mount-path: '/home/runner/work/webstatus.dev'
continue-on-error: true

- name: Create Workspace
run: mkdir -p ${{ github.workspace }}

- name: Checkout (GitHub)
uses: actions/checkout@v6

- name: Move Docker
run: |
sudo service docker stop
mkdir -p ../docker-storage
sudo rm -rf /var/lib/docker
sudo ln -s $(realpath ../docker-storage) /var/lib/docker
sudo service docker start

- name: Install Nix
uses: DeterminateSystems/nix-installer-action@ef8a148080ab6020fd15196c2084a2eea5ff2d25

- name: Magic Nix Cache
uses: DeterminateSystems/magic-nix-cache-action@565684385bcd71bad329742eefe8d12f2e765b39

- name: Run playwright-test target for CI via Nix
run: nix develop --command make playwright-test
env:
CI: 'true'

- uses: actions/upload-artifact@v7
if: ${{ !cancelled() }}
with:
name: playwright-report-nix
path: playwright-report/
retention-days: 30

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {contents: read}
Comment thread
github-advanced-security[bot] marked this conversation as resolved.
Fixed
7 changes: 7 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -74,3 +74,10 @@ tsconfig.tsbuildinfo

# Docker build log files
infra/*.log

# Direnv
.envrc
.direnv/

# Nix
.nix/
10 changes: 5 additions & 5 deletions DEVELOPMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,11 @@

## Requirements

The recommended way to do development is through the provided devcontainer. To
run a devcontainer, check out this
[web page](https://code.visualstudio.com/docs/devcontainers/containers#_system-requirements)
for the latest requirements to run devcontainer. The devcontainer will have
everything pre-installed.
The recommended way to do development is through the provided **VS Code DevContainer**. The devcontainer will have everything pre-installed.

Alternatively, you can use **Nix** to set up your development environment. See the [Nix Setup Guide](./docs/nix-setup.md) for detailed instructions.

For either path, a container runtime (like Docker Desktop) is still required on your host machine to run the local services.

## Project Structure

Expand Down
Loading
Loading