Skip to content

Add structured logging support - #326

Open
antirotor wants to merge 8 commits into
developfrom
feature/structured-logging
Open

Add structured logging support#326
antirotor wants to merge 8 commits into
developfrom
feature/structured-logging

Conversation

@antirotor

@antirotor antirotor commented Aug 28, 2026

Copy link
Copy Markdown
Member

Changelog Description

Replace prints with structured logging with support for early boot Vector log aggregation.

Additional info

What are the benefits of structured logging and possible processing it with Vector/Loki stack? Structured logging treats log entries as queryable data rather than readable sentences.

Standard logging uses flat text strings, filtering logs can be done but with rather fragile regexes, it has cumbersome context propagation and has miserable machine readability. Structured logging addresses all of it while it maintains human readable format.

Additionally pairing Python's structured output with a Vector and Loki pipeline creates a highly scalable, heavily cost-optimized observability stack:

Vector acts as a lightweight agent that can intercept your Python JSON logs, extract specific fields, and enrich them (such as geo-tagging IP addresses) before forwarding them to storage.
You can configure Vector to drop noisy debug logs at the edge or compress payloads heavily, drastically reducing network and storage costs before the data even reaches your database.

You can connect Vector to other systems, one of them for example Loki for storing and querying the logs.

Note

Currently hardcoded Vector related code is not optimal but it provides early sending of logs to Loki for example. Maybe there is a better modular way.

Warning

This needs companion PR in ynput/ayon-core#2029

Testing notes:

  • Re-create the launcher venv (it pulls in structlog)
  • Run with or without --debug

Future Steps

Handle silencing of noise when using --debug using some configuration or CLI argument.

antirotor and others added 3 commits August 26, 2026 11:24
Use official and maintained ruff action, run only on changed python files.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
replace prints with structured logging with support for early boot Vector log aggregation
@antirotor
antirotor requested a lite review from Copilot August 28, 2026 17:56
@antirotor antirotor self-assigned this Aug 28, 2026
@antirotor antirotor added the type: feature Adding something new and exciting to the product label Aug 28, 2026
@antirotor

Copy link
Copy Markdown
Member Author

To test it with Vector/Loki/Grafana, simply clone ynput/ayon-vector and run in ./services:

docker compose up -d

that should start the whole stack with some sane example configuration that should work almost out-of-the-box (well except Grafana which you need to configure yourself)

@antirotor
antirotor requested review from BigRoy and iLLiCiTiT August 28, 2026 18:02

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR replaces ad-hoc print-style output in the launcher boot path with structured logging (structlog), including optional early-boot forwarding of JSON logs to Vector via HTTP to support downstream aggregation (e.g., Loki).

Changes:

  • Introduces a shared logging configuration (common/ayon_common/logging.py) and initializes it early in start.py.
  • Converts a number of startup messages and error paths in start.py to structured logger.* calls with contextual fields.
  • Adds new runtime dependencies (structlog, rich) and updates the lockfile.

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 5 comments.

File Description
start.py Initializes structlog-based startup logger and replaces multiple prints with structured log events.
common/ayon_common/logging.py Adds structlog + stdlib logging configuration and an optional Vector HTTP handler with async queueing.
pyproject.toml Adds structlog and rich as dependencies.
uv.lock Lockfile regeneration to include structlog and rich (plus related resolver changes).
Suppressed comments (2)

start.py:1121

  • logger.info(..., info=info) passes a RuntimeInfo dataclass instance into the event dict. When Vector logging is enabled, JSONRenderer uses json.dumps and will raise TypeError on non-JSON-serializable objects (like dataclass instances).
        info = get_info(
            use_staging=is_staging_enabled(),
            use_dev=is_dev_mode_enabled()
        )
        logger.info("AYON launcher initialized", info=info)

start.py:1217

  • The get_info docstring no longer matches behavior: it now returns a RuntimeInfo object and doesn't print to the console. Updating the docstring will prevent confusion for future callers.
def get_info(use_staging=None, use_dev=None) -> RuntimeInfo:
    """Print additional information to console."""

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread common/ayon_common/logging.py
Comment thread common/ayon_common/logging.py Outdated
Comment thread start.py
Comment thread start.py
Comment thread pyproject.toml
Comment thread common/ayon_common/logging.py Outdated
@antirotor

Copy link
Copy Markdown
Member Author

Note: I had to merge in #325 because the linter action was checking the whole codebase producing tons of errors.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type: feature Adding something new and exciting to the product

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants