Calculation dependency graph

README.md

@@ -177,6 +177,26 @@ curl -X POST http://localhost:8000/api/v1/scripts \
         }'
 ```
 
+**Calculations dependency graph:**
+
+Turn DPM-XL operations into a single self-contained HTML dependency graph —
+one node per operation, one arrow per dependency, roots highlighted,
+click-to-inspect, search and re-layout. The rendering libraries are embedded
+inline so the file opens offline. Dependencies are always resolved by the
+engine against the DPM dictionary — ranges and wildcards expand to exact
+cells — so `--database` is required in every mode. Three input sources:
+
+```bash
+# A Code,Expression CSV, resolved against the dictionary
+dpmcore generate-graph calculations_script.csv --database sqlite:///dpm.db -o graph.html
+
+# Inline expressions (no file needed)
+dpmcore generate-graph -e "calc1={tA, r1, c1} <- {tB, r1, c1}" --database sqlite:///dpm.db -o graph.html
+
+# The DPM dictionary directly (filter to keep the graph readable)
+dpmcore generate-graph --database sqlite:///dpm.db --table C_01.00 -o graph.html
+```
+
 **Data dictionary queries:**
 
 Release-aware methods accept `release_id` (integer ID) or
@@ -592,7 +612,8 @@ src/dpmcore/
 │   ├── meili_build.py     MeiliBuildService (end-to-end pipeline)
 │   ├── meili_json.py      MeiliJsonService (JSON generation)
 │   ├── database_update.py DatabaseUpdateService (atomic DB update)
-│   └── layout_exporter/   LayoutExporterService (tables → .xlsx)
+│   ├── layout_exporter/   LayoutExporterService (tables → .xlsx)
+│   └── calculations_graph/ CalculationsGraphService (engine-resolved HTML graph)
 ├── loaders/               data-loading (mutates the DB)
 │   └── migration.py       MigrationService (Access import)
 ├── server/                FastAPI REST app (optional, [server] extra)

docs/cli.rst

@@ -469,3 +469,150 @@ Each generated workbook contains:
     categorisations (``Dimension = Member``)
   - Outline groups for expanding/collapsing hierarchical rows and columns
   - Frozen panes at the data-area origin
+
+
+``dpmcore generate-graph``
+--------------------------
+
+Build a portable, self-contained HTML **dependency graph** from a DPM-XL
+calculations script. The graph shows the execution order of the operations:
+one node per operation, one arrow per dependency. The output is a single
+``.html`` file with its rendering libraries (Cytoscape.js + the dagre layout)
+embedded inline, so it opens offline and can be attached to a ticket or sent
+to another person as a single file.
+
+**``--database URL`` is always required.** The engine resolves every
+selection's cells against the DPM dictionary, so dependencies are exact in
+every mode — row/column ranges and wildcards are expanded to the concrete
+cells they cover, never approximated. Choose the operations to graph with one
+input source:
+
+#. a ``Code,Expression`` **CSV file** (the ``CSV`` argument);
+#. one or more inline **``-e CODE=EXPRESSION``** operations (handy for a
+   quick, ad-hoc graph without authoring a file); or
+#. **neither** — read the DPM dictionary directly (filter with ``--module``
+   / ``--table``).
+
+``--release`` selects the release to resolve against (default: the latest).
+
+.. code-block:: text
+
+   dpmcore generate-graph CSV --database URL [--release R] [-o OUT] [-t TITLE]
+   dpmcore generate-graph -e CODE=EXPRESSION [-e ...] --database URL
+                          [--release R] [-o OUT] [-t TITLE]
+   dpmcore generate-graph --database URL [--module C] [--table T]
+                          [--release R] [-o OUT] [-t TITLE]
+
+**Arguments / Options:**
+
+.. list-table::
+   :header-rows: 1
+   :widths: 32 68
+
+   * - Argument / Option
+     - Description
+   * - ``CSV``
+     - Path to the calculations-script CSV.
+   * - ``-e, --expression CODE=EXPRESSION``
+     - An inline operation, given as a ``Code`` and its DPM-XL expression
+       separated by ``=`` (split on the first ``=``). Repeatable.
+   * - ``--database TEXT`` (required)
+     - SQLAlchemy database URL. The engine resolves every selection's cells
+       against this dictionary, so dependencies are exact in all modes.
+   * - ``--module TEXT``
+     - Dictionary mode only (no CSV / ``-e``): restrict to operations in this
+       module version code.
+   * - ``--table TEXT``
+     - Dictionary mode only (no CSV / ``-e``): restrict to operations
+       referencing this table code.
+   * - ``--release TEXT``
+     - Release code to resolve against (e.g. ``4.2``). Defaults to the latest
+       release.
+   * - ``-o, --output PATH``
+     - Output HTML path. Defaults to ``calculations_graph.html``.
+   * - ``-t, --title TEXT``
+     - Graph title. Defaults to a title derived from the input.
+
+**How dependencies are resolved:**
+
+The engine expands every selection to the concrete ``VariableID`` set it
+covers — ranges, wildcards and the sheet dimension included — and draws an
+implicit edge only on an *exact* variable match, so overlapping ranges never
+produce a false dependency.
+
+In the **CSV and inline modes** each ``<lhs> <- ...: (<rhs>)`` assignment is
+resolved against ``--database`` at ``--release``: the left-hand selection is
+the operation's output and the right-hand side its inputs.
+
+In the **dictionary mode** the graph is built from the operations already in
+the DPM dictionary. Those are stored as validations/equalities
+(``with {scope}: {LHS} = {RHS}``); for an ``=`` operation the ``left`` side is
+the output and the ``right`` the inputs. The full dictionary has thousands of
+operations, so a ``--module`` / ``--table`` / ``--release`` filter is
+recommended to keep the graph readable.
+
+**Input format:**
+
+A CSV with a ``Code,Expression`` header. Each row is one operation:
+
+- ``Code`` is the bare operation code. It must **not** start with ``o`` —
+  explicit references use that prefix (see below).
+- ``Expression`` is a DPM-XL assignment of the form
+  ``<lhs selection> <- with {default:..., interval:...}: (<rhs expression>)``.
+  The left-hand side selection is the operation's output; the right-hand side
+  reads its inputs through selection operators ``{...}``. Because expressions
+  contain commas, quote the ``Expression`` field.
+
+**Dependencies:**
+
+An arrow ``A -> B`` is drawn when operation ``B`` depends on ``A``:
+
+- **Implicit** — ``B`` reads a cell that ``A`` writes (matched on the
+  engine-resolved ``VariableID``).
+- **Explicit** — ``B`` references ``A`` directly via ``{o<A-code>}``.
+
+Operations with no incoming arrow are **roots** and are shown in a distinct
+colour. Click a node to see its output cell and full expression; the panel
+also offers search/filter and ``Fit`` / ``Re-layout`` controls.
+
+.. note::
+
+   An operation whose selection cannot be resolved (an unknown table, or a
+   grey cell that carries no variable) is not fatal: its node still renders,
+   its unresolved dependencies are skipped, and a warning is printed.
+   Explicit ``{o<code>}`` references need no cell resolution and are always
+   drawn.
+
+**Examples:**
+
+.. code-block:: bash
+
+   # CSV script, resolved against the dictionary (default output)
+   dpmcore generate-graph calculations_script.csv \
+       --database sqlite:///dpm.db
+
+   # Custom output path, title and release
+   dpmcore generate-graph calculations_script.csv \
+       --database sqlite:///dpm.db \
+       --release 4.2 \
+       -o output/graph.html \
+       -t "COREP calculations"
+
+   # Inline expressions, no CSV file needed
+   dpmcore generate-graph \
+       -e "calc1={tK_1.00, r0010, c0010} <- {tK_2.00, r0010, c0010}" \
+       -e "calc2={tK_3.00, r0010, c0010} <- {tK_1.00, r0010, c0010} + 1" \
+       --database sqlite:///dpm.db \
+       -o graph.html
+
+   # Dictionary mode: graph the operations already in the DPM (one table)
+   dpmcore generate-graph \
+       --database sqlite:///dpm.db \
+       --table C_01.00 \
+       -o graph.html
+
+**Vendored libraries:**
+
+The embedded JavaScript (Cytoscape.js, dagre, cytoscape-dagre — all MIT) is
+vendored under ``dpmcore/services/calculations_graph/assets/``. Their pinned
+versions, source URLs and update steps are recorded in ``assets/VENDOR.md``.

src/dpmcore/cli/main.py

@@ -7,6 +7,12 @@
     dpmcore generate-script --expressions ./rules.json \
         --module-code COREP_Con --module-version 2.0.1 \
         --database sqlite:///dpm.db --output ./script.json
+    dpmcore generate-graph ./calculations_script.csv \
+        --database sqlite:///dpm.db -o ./graph.html
+    dpmcore generate-graph -e "c1={tA,r1,c1} <- {tB,r1,c1}" \
+        --database sqlite:///dpm.db -o ./graph.html
+    dpmcore generate-graph --database sqlite:///dpm.db --table C_01.00 \
+        -o ./graph.html
     dpmcore --version
 """
 
@@ -702,3 +708,196 @@ def export_layout(
             )
 
     click.echo(f"Exported to {path}")
+
+
+def _validate_graph_inputs(
+    database: str | None,
+    csv: str | None,
+    expressions: tuple[str, ...],
+    module_code: str | None,
+    table_code: str | None,
+) -> tuple[str, list[tuple[str, str]]]:
+    """Validate ``generate-graph`` inputs and parse inline expressions.
+
+    Returns ``(database, rows)`` where ``rows`` are the ``(code, expression)``
+    pairs from ``--expression`` (empty for the CSV and dictionary modes).
+    Raises ``click.UsageError`` on any invalid combination.
+    """
+    if database is None:  # required=True guarantees this; narrows for mypy
+        raise click.UsageError("--database is required.")
+    if csv is not None and expressions:
+        raise click.UsageError(
+            "Provide either a CSV path argument or --expression options, "
+            "not both."
+        )
+    reads_dictionary = csv is None and not expressions
+    if not reads_dictionary and (module_code or table_code):
+        raise click.UsageError(
+            "--module / --table only apply when reading the dictionary "
+            "directly (omit the CSV argument and --expression)."
+        )
+    rows: list[tuple[str, str]] = []
+    for item in expressions:
+        code, sep, expr = item.partition("=")
+        if not sep:
+            raise click.UsageError(
+                f"Invalid --expression {item!r}; expected CODE=EXPRESSION."
+            )
+        rows.append((code.strip(), expr.strip()))
+    return database, rows
+
+
+@main.command("generate-graph")
+@click.argument(
+    "csv",
+    required=False,
+    type=click.Path(exists=True, dir_okay=False, path_type=str),
+)
+@click.option(
+    "--expression",
+    "-e",
+    "expressions",
+    multiple=True,
+    metavar="CODE=EXPRESSION",
+    help=(
+        "Inline operation as CODE=EXPRESSION (repeatable). Use instead "
+        "of the CSV argument for a quick, ad-hoc graph. Resolved against "
+        "--database like the CSV mode."
+    ),
+)
+@click.option(
+    "--output",
+    "-o",
+    "output_path",
+    default="calculations_graph.html",
+    show_default=True,
+    type=click.Path(dir_okay=False, path_type=str),
+    help="Output HTML path.",
+)
+@click.option(
+    "--database",
+    default=None,
+    required=True,
+    help=(
+        "SQLAlchemy database URL (required). The engine resolves every "
+        "selection's cells against this DPM dictionary, so dependencies "
+        "are exact in all input modes."
+    ),
+)
+@click.option(
+    "--module",
+    "module_code",
+    default=None,
+    help=(
+        "Dictionary mode only (no CSV/-e): restrict to operations in this "
+        "module version code."
+    ),
+)
+@click.option(
+    "--table",
+    "table_code",
+    default=None,
+    help=(
+        "Dictionary mode only (no CSV/-e): restrict to operations "
+        "referencing this table code."
+    ),
+)
+@click.option(
+    "--release",
+    "release_code",
+    default=None,
+    help="Resolve against this release code (default: the latest release).",
+)
+@click.option(
+    "--title",
+    "-t",
+    default=None,
+    help="Graph title (default: derived from the input).",
+)
+def generate_graph(
+    csv: str | None,
+    expressions: tuple[str, ...],
+    database: str | None,
+    module_code: str | None,
+    table_code: str | None,
+    release_code: str | None,
+    output_path: str,
+    title: str | None,
+) -> None:
+    r"""Build a self-contained HTML dependency graph of a calculations script.
+
+    ``--database URL`` is always required: the engine resolves every
+    selection's cells against the DPM dictionary, so dependencies are exact
+    (ranges and wildcards are expanded, never approximated). Choose the
+    operations to graph with one input source:
+
+    \b
+    * a ``Code,Expression`` CSV file (the CSV argument);
+    * one or more inline ``-e CODE=EXPRESSION`` operations; or
+    * neither — read the DPM dictionary directly (filter with
+      ``--module`` / ``--table``).
+
+    ``--release`` selects the release to resolve against (default: latest).
+    The output is a single HTML file with its rendering libraries embedded
+    inline, so it opens offline.
+    """
+    from pathlib import Path
+
+    try:
+        from rich.console import Console
+    except ImportError:
+        click.echo(
+            "Install 'rich' for pretty output: pip install dpmcore[cli]",
+            err=True,
+        )
+        sys.exit(1)
+
+    from dpmcore.errors import DpmCoreError
+    from dpmcore.services.calculations_graph import CalculationsGraphService
+
+    database, rows = _validate_graph_inputs(
+        database, csv, expressions, module_code, table_code
+    )
+
+    console = Console()
+
+    from dpmcore.connection import connect
+
+    svc = CalculationsGraphService()
+    try:
+        with connect(database) as db:
+            if csv is not None:
+                result = svc.generate(
+                    Path(csv), db.session, title, release_code
+                )
+            elif expressions:
+                result = svc.generate_from_rows(
+                    rows,
+                    db.session,
+                    title or "Execution graph",
+                    release_code,
+                )
+            else:
+                result = svc.generate_from_database(
+                    db.session,
+                    release_code=release_code,
+                    module_code=module_code,
+                    table_code=table_code,
+                    title=title,
+                )
+    except DpmCoreError as exc:
+        console.print(f"[red]Error:[/red] {exc}")
+        sys.exit(1)
+
+    out = Path(output_path)
+    out.parent.mkdir(parents=True, exist_ok=True)
+    out.write_text(result.html, encoding="utf-8")
+
+    size_kib = out.stat().st_size // 1024
+    console.print(
+        f"[green]Wrote graph to[/green] {out} "
+        f"({result.n_nodes} operations, {result.n_edges} dependencies, "
+        f"{result.n_roots} roots; {size_kib} KiB, self-contained)."
+    )
+    for warning in result.warnings:
+        console.print(f"[yellow]Warning:[/yellow] {warning}")