Add structured logging support - #326
Conversation
Use official and maintained ruff action, run only on changed python files.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
replace prints with structured logging with support for early boot Vector log aggregation
|
To test it with Vector/Loki/Grafana, simply clone ynput/ayon-vector and run in docker compose up -dthat should start the whole stack with some sane example configuration that should work almost out-of-the-box (well except Grafana which you need to configure yourself) |
There was a problem hiding this comment.
Pull request overview
This PR replaces ad-hoc print-style output in the launcher boot path with structured logging (structlog), including optional early-boot forwarding of JSON logs to Vector via HTTP to support downstream aggregation (e.g., Loki).
Changes:
- Introduces a shared logging configuration (
common/ayon_common/logging.py) and initializes it early instart.py. - Converts a number of startup messages and error paths in
start.pyto structuredlogger.*calls with contextual fields. - Adds new runtime dependencies (
structlog,rich) and updates the lockfile.
Reviewed changes
Copilot reviewed 3 out of 4 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
start.py |
Initializes structlog-based startup logger and replaces multiple prints with structured log events. |
common/ayon_common/logging.py |
Adds structlog + stdlib logging configuration and an optional Vector HTTP handler with async queueing. |
pyproject.toml |
Adds structlog and rich as dependencies. |
uv.lock |
Lockfile regeneration to include structlog and rich (plus related resolver changes). |
Suppressed comments (2)
start.py:1121
logger.info(..., info=info)passes aRuntimeInfodataclass instance into the event dict. When Vector logging is enabled,JSONRendererusesjson.dumpsand will raiseTypeErroron non-JSON-serializable objects (like dataclass instances).
info = get_info(
use_staging=is_staging_enabled(),
use_dev=is_dev_mode_enabled()
)
logger.info("AYON launcher initialized", info=info)
start.py:1217
- The
get_infodocstring no longer matches behavior: it now returns aRuntimeInfoobject and doesn't print to the console. Updating the docstring will prevent confusion for future callers.
def get_info(use_staging=None, use_dev=None) -> RuntimeInfo:
"""Print additional information to console."""
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
|
Note: I had to merge in #325 because the linter action was checking the whole codebase producing tons of errors. |
Changelog Description
Replace prints with structured logging with support for early boot Vector log aggregation.
Additional info
What are the benefits of structured logging and possible processing it with Vector/Loki stack? Structured logging treats log entries as queryable data rather than readable sentences.
Standard logging uses flat text strings, filtering logs can be done but with rather fragile regexes, it has cumbersome context propagation and has miserable machine readability. Structured logging addresses all of it while it maintains human readable format.
Additionally pairing Python's structured output with a Vector and Loki pipeline creates a highly scalable, heavily cost-optimized observability stack:
Vector acts as a lightweight agent that can intercept your Python JSON logs, extract specific fields, and enrich them (such as geo-tagging IP addresses) before forwarding them to storage.
You can configure Vector to drop noisy debug logs at the edge or compress payloads heavily, drastically reducing network and storage costs before the data even reaches your database.
You can connect Vector to other systems, one of them for example Loki for storing and querying the logs.
Note
Currently hardcoded Vector related code is not optimal but it provides early sending of logs to Loki for example. Maybe there is a better modular way.
Warning
This needs companion PR in ynput/ayon-core#2029
Testing notes:
--debugFuture Steps
Handle silencing of noise when using
--debugusing some configuration or CLI argument.