From 77e0277c2cf1b0e8e46d1a27459b2d59daeab23c Mon Sep 17 00:00:00 2001 From: Kyle Knoepfel Date: Fri, 17 Jul 2026 15:53:37 -0500 Subject: [PATCH 1/3] Migrate remaining 'product name' instances to 'product suffix' --- doc/ch_conceptual_design/data_organization.rst | 2 +- doc/ch_conceptual_design/hofs/transforms.rst | 15 ++++++++------- 2 files changed, 9 insertions(+), 8 deletions(-) diff --git a/doc/ch_conceptual_design/data_organization.rst b/doc/ch_conceptual_design/data_organization.rst index d981bcf..1ce928e 100644 --- a/doc/ch_conceptual_design/data_organization.rst +++ b/doc/ch_conceptual_design/data_organization.rst @@ -92,7 +92,7 @@ Such metadata include: - the *creator*, the name of the algorithm that created the data product - an identifier for the *data cells* with which the data product is associated (e.g. `Spill`, `Run`, `Calibration Interval`, or other experiment-defined layer) - the *stage name*, an identifier for the job in which the data product was created -- an individual *name* for the data product (which may be empty), to distinguish between multiple products of the same type created by the same algorithm. +- an individual *suffix* for the data product (which may be empty), to distinguish between multiple products of the same type created by the same algorithm. In addition to these metadata, a data product is also specified by its *type*. diff --git a/doc/ch_conceptual_design/hofs/transforms.rst b/doc/ch_conceptual_design/hofs/transforms.rst index 886bf8c..1d74532 100644 --- a/doc/ch_conceptual_design/hofs/transforms.rst +++ b/doc/ch_conceptual_design/hofs/transforms.rst @@ -53,7 +53,7 @@ To illustrate the different ways a transform's algorithm can be registered with // Return type: first number = number of good hits // second number = number of all hits -**Transform with one argument (default output product name)** +**Transform with one argument (default output-product suffix)** .. code:: c++ @@ -63,7 +63,7 @@ To illustrate the different ways a transform's algorithm can be registered with .input_family(product_selector{.suffix = "Waveforms", .layer = "APA"}); } -**Transform with one argument (user-specified output product name)** +**Transform with one argument (user-specified output-product suffix)** *As shown in* :numref:`workflow` *and described in* :numref:`ch_conceptual_design/registration:Framework Registration` @@ -76,7 +76,7 @@ To illustrate the different ways a transform's algorithm can be registered with .output_product_suffixes("GoodHits"); } -**Transform with two arguments (default output product name)** +**Transform with two arguments (default output-product suffix)** *As shown in* :numref:`workflow` *and described in* :numref:`ch_conceptual_design/registration:Data Products from Different Data Layers` @@ -88,11 +88,10 @@ To illustrate the different ways a transform's algorithm can be registered with .input_family( product_selector{.suffix = "Geometry", .layer = "Job"}, product_selector{.suffix = "GoodTracks", .layer = "APA"} - ) - .output_product_suffixes("Vertices"); + ); } -**Transform creating two data products (user-specified output product names)** +**Transform creating two data products (user-specified output-product suffixes)** .. code:: c++ @@ -100,5 +99,7 @@ To illustrate the different ways a transform's algorithm can be registered with { m.transform("hit_counter", count_good_hits, concurrency::unlimited) .input_family(product_selector{.suffix = "GoodHits", .layer = "APA"}) - .output_product_suffixes("NumGoodHits", "NumAllHits"); // <= One name per tuple slot + .output_product_suffixes( + "NumGoodHits", "NumAllHits" // <= One suffix per tuple slot + ); } From 17c3babf54b1d2e9dbcdd07d8d39bd3ff18a1ea3 Mon Sep 17 00:00:00 2001 From: Kyle Knoepfel Date: Fri, 17 Jul 2026 16:53:19 -0500 Subject: [PATCH 2/3] Describe logical vs. in-memory containment --- doc/ch_conceptual_design/data_organization.rst | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/doc/ch_conceptual_design/data_organization.rst b/doc/ch_conceptual_design/data_organization.rst index 1ce928e..a2ade80 100644 --- a/doc/ch_conceptual_design/data_organization.rst +++ b/doc/ch_conceptual_design/data_organization.rst @@ -30,7 +30,7 @@ The `Job` layer always includes a single data cell. Rectangles with labels :math:`\textsf{Run}_i`, :math:`\textsf{Spill}_{i,j}`, and :math:`\textsf{APA}_{i,j,k}` represent data cells. The pale green rectangles show two data-cell families; these are identified as families because they are the result of executing the :mathfunc:`unfold(into_apas)` node shown in :numref:`workflow`. A solid line from one data cell to another data cell represents a logical association between the two data cells. - The bottom rectangle shows that :math:`\textsf{Waveforms}_{1,1,1}` is in the data cell :math:`\textsf{APA}_{1,1,1}`, etc. + Each dashed line between a data product and a data cell (e.g., between :math:`\textsf{Waveforms}_{1,1,1}` and :math:`\textsf{APA}_{1,1,1}`) indicates that the data product is logically contained by the data cell (see text). Each pale purple rectangle indicates the data-product family created by unfolding each :product:`SimDepos` object as shown in :numref:`workflow`. :name: data-organization @@ -70,6 +70,13 @@ As illustrated in :numref:`data-organization`, data products are organized into They can be unfolded into finer-grained units, enabling detailed analysis or reprocessing at different scales :need:`DUNE 43`. This provides the ability to process data too large to fit into memory at one time :need:`DUNE 25`. +.. admonition:: Logical vs. in-memory containment + + From a mathematical perspective, data cells *logically* contain data products. + This does not mean, however, that the implementation must represent a given data cell as an object that keeps all of its logically-contained data products in memory at one time. + Instead, the implementation is free to retain in memory only those data products necessary for performing a particular task (see :numref:`ch_conceptual_design/data_organization:Data Product Management`). + It is from this mathematical viewpoint that a data product equivalently *belongs to*, *is associated with*, *is contained by*, *is in*, or *is a member of* one or more data cells. + Data Product Management ----------------------- From 034b368ef4fcd17fe379c94d021e967fae9a59e6 Mon Sep 17 00:00:00 2001 From: Kyle Knoepfel Date: Wed, 29 Jul 2026 11:49:50 -0500 Subject: [PATCH 3/3] Fix fold operator for arithmetic mean --- doc/ch_preliminaries/functional_programming.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/ch_preliminaries/functional_programming.rst b/doc/ch_preliminaries/functional_programming.rst index 5d95a71..dfd408a 100644 --- a/doc/ch_preliminaries/functional_programming.rst +++ b/doc/ch_preliminaries/functional_programming.rst @@ -86,10 +86,10 @@ Such a HOF is called a map or *transform*: In such a scenario, the average :math:`\overline{c}` could be expressed as: .. math:: - \overline{c} = \frac{1}{n}\ \fold{+}{0}\ \transform{w}\ \fami{e} = \frac{1}{n}\ \fold{+ \comp w}{0}\ \fami{e} + \overline{c} = \frac{1}{n}\ \fold{+}{0}\ \transform{w}\ \fami{e} = \frac{1}{n}\ \fold{+ \comp (w \times w)}{0}\ \fami{e} The second equality holds by the fold-map fusion law [Bird]_, which states that the application of a :math:`\text{transform}` followed by a :math:`\text{fold}` can be reduced to a single :math:`\text{fold}`. -The operator to this single fold is ':math:`+ \comp w`', indicating that the function :math:`w` should be applied first before invoking the :math:`+` operation. +The operator to this single fold is ':math:`+ \comp (w \times w)`', indicating that the function :math:`w` is applied to both input arguments of the :math:`+` operation. Relying on such mathematical laws permits the replacement of chained calculations with a single calculation, often leading to efficiency improvements without affecting the result. .. _hofs_in_phlex: