Calculation dependency graph

README.md

@@ -177,6 +177,25 @@ curl -X POST http://localhost:8000/api/v1/scripts \
         }'
 ```
 
+**Calculations dependency graph:**
+
+Turn DPM-XL operations into a single self-contained HTML dependency graph —
+one node per operation, one arrow per dependency, roots highlighted,
+click-to-inspect, search and re-layout. The rendering libraries are embedded
+inline so the file opens offline. Three input sources:
+
+```bash
+# A Code,Expression CSV (no database; dependencies from the DPM-XL AST)
+dpmcore generate-graph calculations_script.csv -o graph.html
+
+# Inline expressions (no file needed)
+dpmcore generate-graph -e "calc1={tA, r1, c1} <- {tB, r1, c1}" -o graph.html
+
+# Engine mode: read the DPM dictionary directly, using the engine's resolved
+# operand cells (recommended for real data — filter to keep it readable)
+dpmcore generate-graph --database sqlite:///dpm.db --table C_01.00 -o graph.html
+```
+
 **Data dictionary queries:**
 
 Release-aware methods accept `release_id` (integer ID) or
@@ -585,7 +604,8 @@ src/dpmcore/
 │   ├── meili_build.py     MeiliBuildService (end-to-end pipeline)
 │   ├── meili_json.py      MeiliJsonService (JSON generation)
 │   ├── database_update.py DatabaseUpdateService (atomic DB update)
-│   └── layout_exporter/   LayoutExporterService (tables → .xlsx)
+│   ├── layout_exporter/   LayoutExporterService (tables → .xlsx)
+│   └── calculations_graph/ CalculationsGraphService (CSV → HTML graph, no DB)
 ├── loaders/               data-loading (mutates the DB)
 │   └── migration.py       MigrationService (Access import)
 ├── server/                FastAPI REST app (optional, [server] extra)

docs/cli.rst

@@ -400,3 +400,135 @@ Each generated workbook contains:
     categorisations (``Dimension = Member``)
   - Outline groups for expanding/collapsing hierarchical rows and columns
   - Frozen panes at the data-area origin
+
+
+``dpmcore generate-graph``
+--------------------------
+
+Build a portable, self-contained HTML **dependency graph** from a DPM-XL
+calculations script. The graph shows the execution order of the operations:
+one node per operation, one arrow per dependency. The output is a single
+``.html`` file with its rendering libraries (Cytoscape.js + the dagre layout)
+embedded inline, so it opens offline and can be attached to a ticket or sent
+to another person as a single file.
+
+Provide exactly **one** input source:
+
+#. a ``Code,Expression`` **CSV file** (the ``CSV`` argument);
+#. one or more inline **``-e CODE=EXPRESSION``** operations (handy for a
+   quick, ad-hoc graph without authoring a file); or
+#. **``--database URL``** to read the DPM dictionary directly (the *engine
+   mode*; see below).
+
+The CSV and inline modes need no database — dependencies are derived from the
+DPM-XL AST. The engine mode reads the dictionary's pre-resolved operands.
+
+.. code-block:: text
+
+   dpmcore generate-graph CSV [-o OUTPUT] [-t TITLE]
+   dpmcore generate-graph -e CODE=EXPRESSION [-e ...] [-o OUTPUT] [-t TITLE]
+   dpmcore generate-graph --database URL [--module C] [--table T]
+                          [--release R] [-o OUTPUT] [-t TITLE]
+
+**Arguments / Options:**
+
+.. list-table::
+   :header-rows: 1
+   :widths: 32 68
+
+   * - Argument / Option
+     - Description
+   * - ``CSV``
+     - Path to the calculations-script CSV.
+   * - ``-e, --expression CODE=EXPRESSION``
+     - An inline operation, given as a ``Code`` and its DPM-XL expression
+       separated by ``=`` (split on the first ``=``). Repeatable.
+   * - ``--database TEXT``
+     - SQLAlchemy database URL. Engine mode: builds the graph from the DPM
+       dictionary using the engine's resolved operand cells.
+   * - ``--module TEXT``
+     - Engine mode only: restrict to operations in this module version code.
+   * - ``--table TEXT``
+     - Engine mode only: restrict to operations referencing this table code.
+   * - ``--release TEXT``
+     - Engine mode only: restrict to operations active in this release code
+       (e.g. ``4.2``).
+   * - ``-o, --output PATH``
+     - Output HTML path. Defaults to ``calculations_graph.html``.
+   * - ``-t, --title TEXT``
+     - Graph title. Defaults to a title derived from the input.
+
+**Engine mode (``--database``):**
+
+Real DPM dictionaries store each operation as a validation/equality
+(``with {scope}: {LHS} = {RHS}``), not a ``<-`` assignment, so the text modes
+above find no dependencies in them. Engine mode instead reads the engine's
+already-resolved operand tree: for an ``=`` operation the ``left`` side is the
+**output** cell and the ``right`` side the **inputs**, and operations are
+linked when one writes a variable another reads (exact ``VariableID`` match,
+with ranges/wildcards/sheets already expanded by the engine). The full
+dictionary has thousands of operations, so a ``--module`` / ``--table`` /
+``--release`` filter is recommended to keep the graph readable.
+
+**Input format:**
+
+A CSV with a ``Code,Expression`` header. Each row is one operation:
+
+- ``Code`` is the bare operation code. It must **not** start with ``o`` —
+  explicit references use that prefix (see below).
+- ``Expression`` is a DPM-XL assignment of the form
+  ``<lhs selection> <- with {default:..., interval:...}: (<rhs expression>)``.
+  The left-hand side selection is the operation's output; the right-hand side
+  reads its inputs through selection operators ``{...}``. Because expressions
+  contain commas, quote the ``Expression`` field.
+
+**Dependencies:**
+
+An arrow ``A -> B`` is drawn when operation ``B`` depends on ``A``:
+
+- **Implicit** — ``B`` reads a cell that ``A`` writes (matched on
+  table/row/column/sheet).
+- **Explicit** — ``B`` references ``A`` directly via ``{o<A-code>}``.
+
+Operations with no incoming arrow are **roots** and are shown in a distinct
+colour. Click a node to see its output cell and full expression; the panel
+also offers search/filter and ``Fit`` / ``Re-layout`` controls.
+
+.. note::
+
+   In the CSV / inline modes dependency detection runs without a database, so
+   concrete cell-range expansion (the exact row codes a ``0010-0050`` range
+   covers) is approximated by numeric range *overlap*. Wildcards, the sheet
+   dimension and operation references are handled exactly. Engine mode
+   (``--database``) avoids the approximation by using the dictionary's
+   pre-resolved cells.
+
+**Examples:**
+
+.. code-block:: bash
+
+   # Default output (calculations_graph.html)
+   dpmcore generate-graph calculations_script.csv
+
+   # Custom output path and title
+   dpmcore generate-graph calculations_script.csv \
+       -o output/graph.html \
+       -t "COREP calculations"
+
+   # Inline expressions, no CSV file needed
+   dpmcore generate-graph \
+       -e "calc1={tK_1.00, r0010, c0010} <- {tK_2.00, r0010, c0010}" \
+       -e "calc2={tK_3.00, r0010, c0010} <- {tK_1.00, r0010, c0010} + 1" \
+       -o graph.html
+
+   # Engine mode: read the DPM dictionary directly (filter to one table)
+   dpmcore generate-graph \
+       --database sqlite:///dpm.db \
+       --table C_01.00 \
+       -o graph.html
+
+**Vendored libraries:**
+
+The embedded JavaScript (Cytoscape.js, dagre, cytoscape-dagre — all MIT) is
+vendored under ``dpmcore/services/calculations_graph/assets/``. Their pinned
+versions, source URLs and update steps are recorded in ``assets/VENDOR.md``.

src/dpmcore/cli/main.py

@@ -7,6 +7,10 @@
     dpmcore generate-script --expressions ./rules.json \
         --module-code COREP_Con --module-version 2.0.1 \
         --database sqlite:///dpm.db --output ./script.json
+    dpmcore generate-graph ./calculations_script.csv -o ./graph.html
+    dpmcore generate-graph -e "c1={tA,r1,c1} <- {tB,r1,c1}" -o ./graph.html
+    dpmcore generate-graph --database sqlite:///dpm.db --table C_01.00 \
+        -o ./graph.html
     dpmcore --version
 """
 
@@ -702,3 +706,156 @@ def export_layout(
             )
 
     click.echo(f"Exported to {path}")
+
+
+@main.command("generate-graph")
+@click.argument(
+    "csv",
+    required=False,
+    type=click.Path(exists=True, dir_okay=False, path_type=str),
+)
+@click.option(
+    "--expression",
+    "-e",
+    "expressions",
+    multiple=True,
+    metavar="CODE=EXPRESSION",
+    help=(
+        "Inline operation as CODE=EXPRESSION (repeatable). Use instead "
+        "of the CSV argument for a quick, ad-hoc graph."
+    ),
+)
+@click.option(
+    "--output",
+    "-o",
+    "output_path",
+    default="calculations_graph.html",
+    show_default=True,
+    type=click.Path(dir_okay=False, path_type=str),
+    help="Output HTML path.",
+)
+@click.option(
+    "--database",
+    default=None,
+    help=(
+        "SQLAlchemy database URL. Builds the graph from the DPM "
+        "dictionary using the engine's resolved operand cells."
+    ),
+)
+@click.option(
+    "--module",
+    "module_code",
+    default=None,
+    help="Engine mode: restrict to operations in this module version code.",
+)
+@click.option(
+    "--table",
+    "table_code",
+    default=None,
+    help="Engine mode: restrict to operations referencing this table code.",
+)
+@click.option(
+    "--release",
+    "release_code",
+    default=None,
+    help="Engine mode: restrict to operations active in this release code.",
+)
+@click.option(
+    "--title",
+    "-t",
+    default=None,
+    help="Graph title (default: derived from the input).",
+)
+def generate_graph(
+    csv: str | None,
+    expressions: tuple[str, ...],
+    database: str | None,
+    module_code: str | None,
+    table_code: str | None,
+    release_code: str | None,
+    output_path: str,
+    title: str | None,
+) -> None:
+    r"""Build a self-contained HTML dependency graph of a calculations script.
+
+    Provide exactly one input source:
+
+    \b
+    * a ``Code,Expression`` CSV file (the CSV argument);
+    * one or more inline ``-e CODE=EXPRESSION`` operations; or
+    * ``--database URL`` to read the DPM dictionary directly, deriving
+      dependencies from the engine's resolved operand cells (filter with
+      ``--module`` / ``--table`` / ``--release``).
+
+    The output is a single HTML file with its rendering libraries embedded
+    inline, so it opens offline.
+    """
+    from pathlib import Path
+
+    try:
+        from rich.console import Console
+    except ImportError:
+        click.echo(
+            "Install 'rich' for pretty output: pip install dpmcore[cli]",
+            err=True,
+        )
+        sys.exit(1)
+
+    from dpmcore.errors import DpmCoreError
+    from dpmcore.services.calculations_graph import CalculationsGraphService
+
+    sources = [csv is not None, bool(expressions), database is not None]
+    if sum(sources) != 1:
+        raise click.UsageError(
+            "Provide exactly one input source: a CSV path argument, one or "
+            "more --expression options, or --database."
+        )
+    if database is None and (module_code or table_code or release_code):
+        raise click.UsageError(
+            "--module / --table / --release only apply with --database."
+        )
+
+    rows: list[tuple[str, str]] = []
+    for item in expressions:
+        code, sep, expr = item.partition("=")
+        if not sep:
+            raise click.UsageError(
+                f"Invalid --expression {item!r}; expected CODE=EXPRESSION."
+            )
+        rows.append((code.strip(), expr.strip()))
+
+    console = Console()
+
+    svc = CalculationsGraphService()
+    try:
+        if database is not None:
+            from dpmcore.connection import connect
+
+            with connect(database) as db:
+                result = svc.generate_from_database(
+                    db.session,
+                    release_code=release_code,
+                    module_code=module_code,
+                    table_code=table_code,
+                    title=title,
+                )
+        elif csv is not None:
+            result = svc.generate(Path(csv), title)
+        else:
+            result = svc.generate_from_rows(rows, title or "Execution graph")
+    except DpmCoreError as exc:
+        console.print(f"[red]Error:[/red] {exc}")
+        sys.exit(1)
+
+    out = Path(output_path)
+    out.parent.mkdir(parents=True, exist_ok=True)
+    out.write_text(result.html, encoding="utf-8")
+
+    size_kib = out.stat().st_size // 1024
+    console.print(
+        f"[green]Wrote graph to[/green] {out} "
+        f"({result.n_nodes} operations, {result.n_edges} dependencies, "
+        f"{result.n_roots} roots; {size_kib} KiB, self-contained)."
+    )
+    for warning in result.warnings:
+        console.print(f"[yellow]Warning:[/yellow] {warning}")

src/dpmcore/connection.py

@@ -29,6 +29,7 @@
 if TYPE_CHECKING:
     from dpmcore.loaders.migration import MigrationService
     from dpmcore.services.ast_generator import ASTGeneratorService
+    from dpmcore.services.calculations_graph import CalculationsGraphService
     from dpmcore.services.data_dictionary import DataDictionaryService
     from dpmcore.services.dpm_xl import DpmXlService
     from dpmcore.services.explorer import ExplorerService
@@ -74,6 +75,17 @@ def syntax(self) -> SyntaxService:
         service: SyntaxService = self._cache["syntax"]
         return service
 
+    @property
+    def calculations_graph(self) -> CalculationsGraphService:
+        from dpmcore.services.calculations_graph import (
+            CalculationsGraphService,
+        )
+
+        if "calculations_graph" not in self._cache:
+            self._cache["calculations_graph"] = CalculationsGraphService()
+        service: CalculationsGraphService = self._cache["calculations_graph"]
+        return service
+
     @property
     def semantic(self) -> SemanticService:
         from dpmcore.services.semantic import SemanticService

src/dpmcore/services/__init__.py

@@ -14,6 +14,7 @@
 
 from dpmcore.loaders.migration import MigrationService
 from dpmcore.services.ast_generator import ASTGeneratorService
+from dpmcore.services.calculations_graph import CalculationsGraphService
 from dpmcore.services.data_dictionary import DataDictionaryService
 from dpmcore.services.dpm_xl import DpmXlService
 from dpmcore.services.ecb_validations_import import EcbValidationsImportService
@@ -36,6 +37,7 @@
     "ExplorerService",
     "HierarchyService",
     "DpmXlService",
+    "CalculationsGraphService",
     "MigrationService",
     "StructureService",
     "ExportCsvService",

src/dpmcore/services/calculations_graph/__init__.py

@@ -0,0 +1,15 @@
+"""Calculations-script dependency graph service."""
+
+from dpmcore.services.calculations_graph.service import (
+    CalculationsGraphResult,
+    CalculationsGraphService,
+    GraphEdge,
+    GraphNode,
+)
+
+__all__ = [
+    "CalculationsGraphService",
+    "CalculationsGraphResult",
+    "GraphNode",
+    "GraphEdge",
+]