Tables tab: delete a Table from the row, published ones included

benchmarks/tables_tab/README.md

@@ -5,14 +5,16 @@ SPDX-License-Identifier: AGPL-3.0-or-later
 
 # Profile dashboard tables tab: row cost and column widths
 
-Two measurements spec #2551 owes for the tables tab (#2555), neither of which a
-test can take:
+Three measurements spec #2551 owes for the tables tab, none of which a test can
+take:
 
 - **`run.py`**: what one row of the list costs the server, against the old
   profile cards' 6-10 ms each;
 - **`widths.mjs`**: how wide the list must be for each set of columns, which
   sets the container-query thresholds in `login/static/login/tables_tab.css`.
   happy-dom has no layout, so this runs in a real browser.
+- **`delete_cost.py`**: what deleting one Table costs (#2562), which sets the
+  delete ceiling `CEILINGS["delete"]` in `api/services/table_actions.py`.
 
 Both use accounts shaped like production (`seed.py`, a port of the WF-06
 prototype's generator, sized from WF-01's census): `p90` (130 Tables), `max`
@@ -68,3 +70,36 @@ The numbers it produced, and the thresholds taken from them, are written beside
 the container queries in `tables_tab.css`. A slice that adds a column re-runs it
 and moves the thresholds; with `STAND_INS=1` it also measures the complete row,
 standing in only for the columns that are still missing.
+
+## Delete cost
+
+Same throwaway database as the row cost; the OEDB tables go into the sandbox
+schema under names nobody else uses, and the measurement drops them itself.
+
+```bash
+python -m benchmarks.tables_tab.delete_cost
+python -m benchmarks.tables_tab.delete_cost --rows 10000000 --tables 2
+```
+
+Each size creates `--tables` Tables filled with that many rows (an OEDB table
+with a primary key and three data columns, its three meta tables, 6 KB of
+metadata, a grant, a Topic and a Dataset membership) and deletes them all in one
+`table_actions.execute` call, the dashboard's path. Results append to
+`benchmarks/results/tables_tab_delete.csv`. Keep the 1M- and 10M-row batches
+small: filling them is what takes the time.
+
+Measured 2026-10-02, local Postgres 14 (`shared_buffers` 128 MB), three rounds:
+
+| rows per Table      | per Table, total | of which the drop | slowest single drop |
+| ------------------- | ---------------- | ----------------- | ------------------- |
+| 0 (16 KB)           | 17-18 ms         | 9-10 ms           | 13 ms               |
+| 100,000 (8 MB)      | 17-18 ms         | 10-11 ms          | 14 ms               |
+| 1,000,000 (83 MB)   | 33-38 ms         | 22-24 ms          | 1,020 ms (once)     |
+| 10,000,000 (0.8 GB) | 0.64 s           | 0.62 s            | 1,126 ms            |
+
+The drop is most of it and grows with the table's size; a single drop
+occasionally takes about a second (one 1M-row drop in three rounds). The
+production timeout is `Timeout 300` / `socket-timeout=300` (read on the host
+2026-10-02). At a worst case of 1.2 s per Table, 50 Tables take 60 s: the
+ceiling is 50, a safety factor of 5, which covers production's OEDB sitting on
+another host. The reasoning is beside the constant.

benchmarks/tables_tab/delete_cost.py

@@ -0,0 +1,185 @@
+"""What deleting one Table costs, to set the delete ceiling (#2562).
+
+    # the default: 10 Tables each at 0, 100,000 and 1,000,000 rows
+    python -m benchmarks.tables_tab.delete_cost
+
+    python -m benchmarks.tables_tab.delete_cost --rows 0,1000000 --tables 5
+
+Spec #2551 owes this number: the dashboard deletes in one request, with no
+task queue, so the most Tables one request may name (``CEILINGS["delete"]``
+in ``api/services/table_actions.py``) has to finish inside the host's
+request timeout, with a stated safety factor.
+
+Like ``run.py`` this never touches production or a developer database: it
+asks Django's test runner for a throwaway database. The OEDB tables go into
+the sandbox schema of the configured OEDB, on names nobody else uses, and are
+dropped by the measurement itself (and again on the way out, whatever
+happened).
+
+For each row count it creates ``--tables`` Tables shaped like a real one (an
+OEDB table with a primary key and three data columns, filled with that many
+rows, plus its three meta tables; on the Django side 6 KB of metadata, a
+Table admin grant, a Topic and a Dataset membership), then deletes them all
+in one ``table_actions.execute`` call, the path the dashboard takes, and
+reports per Table:
+
+- ``total``: the whole call divided by the Tables in it;
+- ``drop``: the OEDB drops alone (four ``DROP TABLE`` each), timed one by one;
+- ``django``: the rest (the re-check under a row lock, the cascade delete,
+  the log lines).
+"""
+
+from __future__ import annotations
+
+import argparse
+import csv
+import statistics
+import time
+import uuid
+from datetime import datetime, timezone
+from pathlib import Path
+
+from benchmarks.tables_tab.run import bootstrap
+
+DEFAULT_RESULTS = Path("benchmarks/results/tables_tab_delete.csv")
+METADATA_KB = 6
+
+
+def parse_args(argv=None):
+    p = argparse.ArgumentParser(
+        prog="python -m benchmarks.tables_tab.delete_cost",
+        description="Measure what deleting one Table costs, locally.",
+    )
+    p.add_argument("--rows", default="0,100000,1000000")
+    p.add_argument("--tables", type=int, default=10)
+    p.add_argument("--results", type=Path, default=DEFAULT_RESULTS)
+    p.add_argument("--no-results", action="store_true")
+    return p.parse_args(argv)
+
+
+def main(argv=None) -> int:
+    args = parse_args(argv)
+    runner, old_config = bootstrap()
+
+    from django.db import connection
+    from sqlalchemy import text
+
+    from api.services import table_actions
+    from dataedit.models import Dataset, Table, Topic
+    from login.models import myuser
+    from oedb.connection import _get_engine
+
+    engine = _get_engine()
+    owner = myuser.objects.create(
+        name=f"bench_delete_{uuid.uuid4().hex[:6]}",
+        email=f"bench_{uuid.uuid4().hex[:6]}@example.org",
+        did_agree=True,
+        is_mail_verified=True,
+    )
+    dataset = Dataset.objects.create(name="bench_delete", creator=owner)
+    topic = Topic.objects.exclude(name="draft").first()
+    metadata = {"title": "x", "description": "y" * (METADATA_KB * 1024)}
+    columns = [
+        {"name": "id", "data_type": "bigserial", "primary_key": True},
+        {"name": "region", "data_type": "text"},
+        {"name": "year", "data_type": "integer"},
+        {"name": "value", "data_type": "float"},
+    ]
+    schema = Table.get_oedb_schema(is_sandbox=True)
+    created = []
+    stamp = datetime.now(timezone.utc).isoformat(timespec="seconds")
+    results = []
+    try:
+        for rows in [int(r) for r in args.rows.split(",")]:
+            names = []
+            for _ in range(args.tables):
+                name = f"bench_del_{uuid.uuid4().hex[:10]}"
+                table = Table.create_with_oedb_table(
+                    name=name,
+                    is_sandbox=True,
+                    user=owner,
+                    column_definitions=columns,
+                    constraints_definitions=[],
+                )
+                created.append(table)
+                names.append(name)
+                Table.objects.filter(pk=table.pk).update(oemetadata=metadata)
+                table.topics.add(topic)
+                dataset.tables.add(table)
+                if rows:
+                    with engine.begin() as conn:
+                        conn.execute(
+                            text(
+                                f'INSERT INTO "{schema}"."{name}" '
+                                "(region, year, value) "
+                                "SELECT 'region_' || (g % 400), 2000 + g % 50, "
+                                "random() FROM generate_series(1, :n) g"
+                            ),
+                            n=rows,
+                        )
+            with engine.connect() as conn:
+                size = conn.execute(
+                    text("SELECT pg_total_relation_size(:t)"),
+                    t=f'"{schema}"."{names[0]}"',
+                ).scalar()
+
+            drops = []
+            real = Table.drop_oedb_table
+
+            def timed(instance):
+                start = time.perf_counter()
+                real(instance)
+                drops.append((time.perf_counter() - start) * 1000)
+
+            Table.drop_oedb_table = timed
+            table_actions.CEILINGS[table_actions.DELETE] = None
+            connection.close()  # a fresh connection, like a request's
+            start = time.perf_counter()
+            try:
+                table_actions.execute(
+                    owner,
+                    table_actions.DELETE,
+                    names,
+                    {"confirm": str(len(names))},
+                    via="benchmark",
+                )
+            finally:
+                Table.drop_oedb_table = real
+            total_ms = (time.perf_counter() - start) * 1000
+            row = {
+                "run_utc": stamp,
+                "rows": rows,
+                "bytes_per_table": size,
+                "tables": len(names),
+                "total_per_table_ms": round(total_ms / len(names), 2),
+                "drop_per_table_ms": round(sum(drops) / len(names), 2),
+                "drop_median_ms": round(statistics.median(drops), 2),
+                "drop_max_ms": round(max(drops), 2),
+                "django_per_table_ms": round((total_ms - sum(drops)) / len(names), 2),
+            }
+            results.append(row)
+            print(
+                "{rows:>9,} rows ({bytes_per_table:,} B): per Table "
+                "{total_per_table_ms} ms = drop {drop_per_table_ms} (median "
+                "{drop_median_ms}, max {drop_max_ms}) + django "
+                "{django_per_table_ms}".format(**row)
+            )
+    finally:
+        for table in created:
+            table.drop_oedb_table()
+        runner.teardown_databases(old_config)
+
+    if results and not args.no_results:
+        args.results.parent.mkdir(parents=True, exist_ok=True)
+        exists = args.results.exists()
+        with args.results.open("a", newline="", encoding="utf-8") as fh:
+            writer = csv.DictWriter(fh, fieldnames=list(results[0]))
+            if not exists:
+                writer.writeheader()
+            writer.writerows(results)
+        print(f"\nappended {len(results)} rows to {args.results}")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

dataedit/models.py

@@ -208,7 +208,20 @@ class Meta:
         unique_together = (("name",),)
 
     def delete(self, *args, **kwargs):
-        super().delete(*args, **kwargs)
+        self.delete_record(*args, **kwargs)
+        self.drop_oedb_table()
+
+    def delete_record(self, *args, **kwargs):
+        """Delete the Django rows only (this Table and everything that
+        cascades from it), leaving the OEDB table in place. For a caller
+        that drops it once its own transaction has committed, as the table
+        action service does: the two databases share no transaction."""
+        return super().delete(*args, **kwargs)
+
+    def drop_oedb_table(self):
+        """Drop the OEDB table and its meta tables, if they exist. Needs
+        only the name and the schema, so it works once the Django row is
+        gone."""
         # ensure oedb tables are deleted, so we use ADMIN_PERM
         self._get_oeb_table_proxy_w_permission(
             permission_level=ADMIN_PERM

login/static/login/__tests__/tables_tab.test.js

@@ -548,6 +548,18 @@ describe("showToast", () => {
     expect(toast.isConnected).toBe(false);
   });
 
+  it("puts a warning in the assertive region and leaves it until dismissed", () => {
+    const schedule = vi.fn();
+    const toast = showToast(document, "Deleted “Go”. The database table…", {
+      warning: true,
+      schedule,
+    });
+    const assertive = document.getElementById("tables-toasts-assertive");
+    expect(assertive.contains(toast)).toBe(true);
+    expect(toast.classList.contains("dash-toast--warning")).toBe(true);
+    expect(schedule).not.toHaveBeenCalled();
+  });
+
   it("writes the message as text, never as markup", () => {
     const toast = showToast(document, "<img src=x>", { error: true });
     expect(toast.querySelector("img")).toBeNull();
@@ -677,6 +689,30 @@ describe("bindTablesTab, actions", () => {
     expect(document.activeElement.id).toBe("tables-search");
   });
 
+  it("after a delete focuses the list heading, the row being gone", () => {
+    const entry = document.getElementById("menu-8-publish");
+    swapDialog(entry);
+    serverTrigger(entry, "tables-changed", { message: "Deleted “Go”." });
+    swapRegion(REGION("1 table", {}, ROW_MENU(7)));
+    expect(document.activeElement.id).toBe("tables-heading");
+  });
+
+  it("keeps a delete whose database table stayed behind on screen", () => {
+    const entry = document.getElementById("menu-8-publish");
+    swapDialog(entry);
+    serverTrigger(entry, "tables-changed", {
+      message: "Deleted “Go”. The database table of “Go” (go) could not…",
+      warning: true,
+    });
+    expect(dialog.closed).toBe(1);
+    const assertive = document.getElementById("tables-toasts-assertive");
+    expect(assertive.textContent).toContain("could not");
+    expect(assertive.querySelector(".dash-toast--warning")).not.toBeNull();
+    expect(document.getElementById("tables-toasts-polite").textContent).toBe(
+      "",
+    );
+  });
+
   it("keeps a refusal on screen, assertively", () => {
     serverTrigger(document.body, "tables-refused", {
       message: "Nothing was changed: Already published (“t”).",

login/static/login/tables_tab.css

@@ -471,6 +471,11 @@
 
 /* an action above the user's role: still focusable, so the reason is
    reachable by keyboard; aria-disabled, not Bootstrap's .disabled */
+/* Delete, last in the menu */
+.dash-menu__danger {
+  color: #b02a37;
+}
+
 .dash-menu__off,
 .dash-menu__off:hover,
 .dash-menu__off:focus {
@@ -512,6 +517,11 @@
   border-left: 4px solid #dc3545;
 }
 
+/* done, but not cleanly: a delete whose database table stayed behind */
+.dash-toast--warning {
+  border-left: 4px solid #ffc107;
+}
+
 .dash-toast .toast-body a {
   font-weight: 600;
 }