Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,3 +37,8 @@ jobs:
run: docker build --tag llmgate:ci .
- name: Verify installed image over loopback HTTP
run: python e2e/docker_smoke.py llmgate:ci
- name: Verify exact local-demo Compose recipe
run: |
docker version
docker compose version
python e2e/compose_smoke.py llmgate:ci
4 changes: 2 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
# llmgate demo container: the gateway plus two simulated providers, no real upstream.
#
# Build: docker build -t llmgate .
# Run: docker run --rm -p 8000:8000 llmgate
# Run: docker run --rm -p 127.0.0.1:8000:8000 llmgate
# curl -H 'Authorization: Bearer demo-key' -H 'content-type: application/json' \
# -d '{"model":"gpt-sim","messages":[{"role":"user","content":"hi"}]}' \
# localhost:8000/v1/chat/completions
#
# Real upstreams: mount a config (see examples.config.json) and override the
# command with the config factory. Secrets come from the environment only:
# docker run --rm -p 8000:8000 \
# docker run --rm -p 127.0.0.1:8000:8000 \
# -e LLMGATE_CONFIG=/cfg/prod.json -e LLMGATE_KEYS -e PRIMARY_API_KEY \
# -v "$PWD/my.json:/cfg/prod.json:ro" \
# llmgate python -m uvicorn --factory llmgate.config:app --host 0.0.0.0 --port 8000
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ original Actas tests and the gateway's separate regression checks.

## Demo and limits
https://llmgate-demo.vercel.app/
The public demo labels simulated and real-model paths. Redis-backed limits are optional; breaker and latency state are per process. Usage telemetry is not billing-grade. The default simulated Docker image was built and checked on hosted CI on 11 November 2026, including installed-package imports, non-root execution, authentication, streaming and forced-primary failover. Docker Compose and mounted real-provider configurations remain unverified.
The public demo labels simulated and real-model paths. Redis-backed limits are optional; breaker and latency state are per process. Usage telemetry is not billing-grade. The default simulated Docker image was built and checked on hosted CI on 11 November 2026, including installed-package imports, non-root execution, authentication, streaming and forced-primary failover. The exact default Docker Compose recipe was also checked on hosted CI on 11 November 2026 with Docker Engine 28.0.4 and Compose v2.38.2: runtime inspection confirmed only `127.0.0.1:8000`, and the healthy simulated HTTP checks passed. Compose forced-failover and mounted real-provider configurations remain unverified.

## Repository setup
GitHub is the source of truth for project changes. The active
Expand Down
4 changes: 2 additions & 2 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
# One-command local demo: docker compose up --build
# Gateway on http://localhost:8000 with the same two simulated providers as
# `uvicorn llmgate.demo_app:app`. Demo keys: demo-key / admin-key.
# Nothing here talks to a real provider or leaves localhost.
# No real providers. The published host port is explicitly bound to IPv4 loopback.
services:
gateway:
build: .
image: llmgate:local
ports:
- "8000:8000"
- "127.0.0.1:8000:8000"
environment:
SIM_A_FAIL: "0" # set to "1" to watch traffic fail over to sim-b
DEMO_RATE: "50" # demo-key request budget per second (burst = 2x)
Expand Down
14 changes: 11 additions & 3 deletions docs/DEPLOYMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ uvicorn llmgate.demo_app:app --port 8000

```bash
docker build -t llmgate .
docker run --rm -p 8000:8000 llmgate
docker run --rm -p 127.0.0.1:8000:8000 llmgate
```

The image is `python:3.11-slim` with the package installed, running as a
Expand All @@ -31,7 +31,7 @@ the command with the config factory. Upstream secrets and client keys come
from environment variables only - never from the image or the config file:

```bash
docker run --rm -p 8000:8000 \
docker run --rm -p 127.0.0.1:8000:8000 \
-e LLMGATE_CONFIG=/cfg/prod.json -e LLMGATE_KEYS -e PRIMARY_API_KEY \
-v "$PWD/my.json:/cfg/prod.json:ro" \
llmgate python -m uvicorn --factory llmgate.config:app --host 0.0.0.0 --port 8000
Expand All @@ -43,6 +43,14 @@ docker run --rm -p 8000:8000 \
docker compose up --build # gateway on http://localhost:8000
```

The local recipes explicitly publish `127.0.0.1:8000:8000`, rather than all
host interfaces. Keep this loopback binding: the demo has public, fixed client
and admin keys and no TLS. Do not use it as a public service. Docker releases
older than 28.0.0 have a [documented localhost-publishing caveat](https://github.com/docker/docs/blob/main/content/manuals/engine/network/port-publishing.md)
that can allow access from the same network segment; use a current engine.
This host binding is not an outbound-network sandbox or a replacement for
firewalls, authentication or production hardening.

`docker-compose.yml` is a single service in demo mode. Set `SIM_A_FAIL=1` in
its environment to watch traffic fail over to the backup provider.

Expand Down Expand Up @@ -156,7 +164,7 @@ section and the demo against `gpt-sim` / `gpt-sim-pro`.

## Honest limits

- On 11 November 2026, [hosted CI](https://github.com/Rajatendu1/llmgate/actions/runs/42977091731/job/130378576049?pr=8) built the default Python 3.11 simulated image and checked installed gateway/Actas imports from site-packages, actual non-root execution, loopback HTTP health, authentication, completion, SSE, unknown-model rejection, admin isolation and usage in healthy and forced-primary-failure modes. This verifies the default demo image only. Docker Compose and mounted real-provider configurations remain unverified; no image was pushed or deployed.
- On 11 November 2026, [hosted CI](https://github.com/Rajatendu1/llmgate/actions/runs/42977091731/job/130378576049?pr=8) built the default Python 3.11 simulated image and checked installed gateway/Actas imports from site-packages, actual non-root execution, loopback HTTP health, authentication, completion, SSE, unknown-model rejection, admin isolation and usage in healthy and forced-primary-failure modes. This verifies the default demo image only. Separately, on the same date, [Compose CI](https://github.com/Rajatendu1/llmgate/actions/runs/42979140919/job/130385152959?pr=9) ran the exact default recipe with Docker Engine 28.0.4 and Compose v2.38.2, inspected its published HostIp as 127.0.0.1 only, confirmed the tested image and installed non-root packages, and passed nine healthy simulated HTTP assertions. Compose forced-failover and mounted real-provider configurations remain unverified; no image was pushed or deployed.
- Usage metering and rate limits are per process by default and can use a shared Redis store when configured. Rate limiting fails closed when the store is unreachable. Circuit-breaker and latency state remain per process. Usage telemetry is not billing-grade.
- No TLS inside the container; put it behind a proxy that terminates TLS.
- Demo keys are hard-coded in `llmgate/demo_app.py`; real deployments use
Expand Down
63 changes: 63 additions & 0 deletions e2e/compose_smoke.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
#!/usr/bin/env python3
"""Check the exact local-demo Compose recipe; reuse an already built image.

No registry push, real providers, mounted configs or production changes.
"""
import json
from pathlib import Path
import subprocess
import sys
import uuid

from docker_smoke import docker, check, verify_http


ROOT = Path(__file__).resolve().parents[1]


def main():
if len(sys.argv) != 2:
raise SystemExit("usage: compose_smoke.py BUILT_IMAGE")
image = sys.argv[1]
expected_image = docker("image", "inspect", "--format", "{{.Id}}", image)
# The checked-in recipe uses this local-only tag. No build or registry pull.
docker("tag", image, "llmgate:local")
project = "llmgate-smoke-" + uuid.uuid4().hex[:12]
command = ["docker", "compose", "--project-name", project,
"--file", str(ROOT / "docker-compose.yml")]

def compose(*args):
return subprocess.check_output(command + list(args), text=True, timeout=90).strip()

try:
config = json.loads(compose("config", "--format", "json"))
ports = config["services"]["gateway"]["ports"]
check(len(ports) == 1 and ports[0].get("host_ip") == "127.0.0.1"
and int(ports[0]["target"]) == 8000
and str(ports[0]["published"]) == "8000",
"exact Compose recipe publishes only IPv4 loopback port8000")
compose("up", "--detach", "--no-build", "--pull", "never")
container = compose("ps", "--quiet", "gateway")
check(bool(container) and "\n" not in container, "one Compose gateway started")
inspect = json.loads(docker("inspect", container))[0]
check(inspect["Image"] == expected_image, "Compose runs the tested image, no pull or rebuild")
bindings = inspect["NetworkSettings"]["Ports"]["8000/tcp"]
check(len(bindings) == 1 and bindings[0]["HostIp"] == "127.0.0.1"
and bindings[0]["HostPort"] == "8000",
"running Compose container is published to127.0.0.1 only")
docker("exec", "--workdir", "/tmp", container, "python", "-c",
"import os, llmgate, actas; assert os.getuid() != 0; "
"assert all('/site-packages/' in m.__file__ for m in (llmgate, actas))")
check(True, "Compose runs installed packages as a non-root user")
verify_http("http://127.0.0.1:8000", 0)
print("PASS exact default Compose recipe and loopback HTTP, simulated only")
except BaseException:
subprocess.run(command + ["logs", "--no-color"], check=False, timeout=30)
raise
finally:
subprocess.run(command + ["down", "--volumes", "--remove-orphans"],
check=True, timeout=90)


if __name__ == "__main__":
main()
Loading