From 114248617b175e6ba01e1204da677dc16121cbaa Mon Sep 17 00:00:00 2001 From: markus Date: Mon, 3 Aug 2026 16:24:40 +0200 Subject: [PATCH] Improve SoATemplate README --- DataFormats/SoATemplate/README.md | 226 ++++++++++++++++++------------ 1 file changed, 137 insertions(+), 89 deletions(-) diff --git a/DataFormats/SoATemplate/README.md b/DataFormats/SoATemplate/README.md index bad5b0003f5f3..81bf9d859019e 100644 --- a/DataFormats/SoATemplate/README.md +++ b/DataFormats/SoATemplate/README.md @@ -1,7 +1,7 @@ # Structure of array (SoA) generation The header file [`SoALayout.h`](SoALayout.h) defines preprocessor macros that -allow generating SoA classes. The SoA classes generate multiple, aligned column from a memory buffer. The memory +allow generating SoA classes. The SoA classes generate multiple, aligned columns from a memory buffer. The memory buffer is allocated separately by the user, and can be located in a memory space different from the local one (for example, a SoA located in a GPU device memory can be fully pre-defined on the host and the resulting structure is passed to the GPU kernel). @@ -13,17 +13,18 @@ Additionally, templation of the layout and view classes allows compile-time vari verification of alignment and corresponding compiler hinting, cache strategy (non-coherent, streaming with immediate invalidation), range checking. -Macro generation allows generating code that provides a clear and concise access of data when used. The code -generation uses the Boost Preprocessing library. +The implementation relies on the Boost Preprocessor library to generate the required boilerplate. +This approach keeps the user-facing code concise while providing a natural (AoS-like) interface for accessing SoA data. ## Layout -`SoALayout` is a macro generated templated class that subdivides a provided buffer into a collection of columns, -Eigen columns and scalars. The buffer is expected to be aligned with a selectable alignment defaulting to the CUDA -GPU cache line (128 bytes). All columns and scalars within a `SoALayout` will be individually aligned, leaving -padding at the end of each if necessary. Eigen columns have each component of the vector or matrix properly aligned -in individual column (by defining the stride between components). Only compile-time sized Eigen vectors and matrices -are supported. Scalar members are members of layout with one element, irrespective of the size of the layout. +`SoALayout` is a macro-generated templated class that subdivides a provided buffer into a collection of columns, +Eigen columns and scalars. The buffer is expected to be aligned with a selectable alignment in bytes. See [Template +parameters section](#template-parameters) for more information. All columns and scalars within a `SoALayout` will +be individually aligned, leaving padding at the end of each if necessary. Eigen columns have each component of +the vector or matrix properly aligned in individual columns (by defining the stride between components). +Only compile-time sized Eigen vectors and matrices are supported. Scalar members are members of the layout with one +element, irrespective of the size of the layout. Static utility functions automatically compute the byte size of a layout, taking into account all its columns and alignment. @@ -31,106 +32,132 @@ alignment. ## View Layout classes also define a `View` and `ConstView` subclass that provide access to each column and -scalar of the layout. In addition to those fully parametrized templates, two others levels of parametrization are +scalar of the layout. In addition to those fully parametrized templates, two other levels of parametrization are provided: `ViewTemplate`, `ViewViewTemplateFreeParams` and respectively `ConstViewTemplate`, `ConstViewTemplateFreeParams`. The parametrization of those templates is explained in the [Template parameters section](#template-parameters). -The view can be generated in a constant and non-constant flavors. All view flavors provide with the same -interface where scalar elements are accessed with an `operator()`: `soa.scalar()` while columns (Eigen or not) are -accessed via a array of structure (AoS) -like syntax: `soa[index].x()`. The "struct" object returned by `operator[]` -can be used as a shortcut: `auto si = soa[index]; si.z() = si.x() + si.y();` +The view can be generated in constant (`ConstView`) and non-constant (`View`) flavors. All view flavors provide the +same interface where scalar elements are accessed with an `operator()`: `soa.scalar()` while columns (Eigen or not) are +accessed via an array of structure (AoS)-like syntax: `soa[index].x()`. The proxy object returned by `operator[]` +can be stored and reused as a convenient shorthand: `auto si = soa[index]; si.z() = si.x() + si.y();`. It is also +possible to access the data in a more SoA-natural way: `soa.x()[index]` or `soa.x(index)`. -A view can be instanciated by being passed the corresponding layout or passing from the [Metarecords subclass](#metarecords-subclass). -This view can point to data belonging to different SoAs and thus not contiguous in memory. +A view can be constructed either from the corresponding layout or from the Metarecords subclass of other layouts. +Since a view is non-owning, its columns may refer to data belonging to different SoAs constructed from different +memory buffers. Consequently, the columns referenced by a view are not required to be contiguous in memory. ## Descriptor -The nested class `ConstDescriptor` can only be instantiated passing a `View` or a `ConstView` and provides access to columns -and related information. This class should be considered an internal implementation detail, -used solely by the SoA and EDM frameworks for performing heterogeneous memory operations. It is used to implement the -`deepCopy` from a `View` referencing different memory buffers, as shown in +The nested class `ConstDescriptor` can only be instantiated by passing a `View` or a `ConstView`. +It provides access to columns and related information. This class should be considered an internal +implementation detail, used solely by the SoA and EDM frameworks for performing heterogeneous memory operations. +It is used to implement the `deepCopy` from a `View` referencing different memory buffers, as shown in [`PortableHostCollection`](../../DataFormats/Portable/README.md#portablehostCollection) and [`PortableDeviceCollection`](../../DataFormats/Portable/README.md#portabledeviceCollection) sections. -More specifically, it provides access to the each column through a `std::tuple...>` accessible via `descriptor.buff` -as well as the types of the columns via a `std::tuple` accessible via `descriptor.columnTypes`. +Specifically, it exposes: +- the columns as an `std::tuple...>` accessible via `descriptor.buff` +- the corresponding column types as an `std::tuple` through `descriptor.columnTypes`. ## Metadata subclass -In order to no clutter the namespace of the generated class, a subclass name `Metadata` is generated. It is -instanciated with the `metadata()` member function and contains various utility functions, like `size()` (number -of elements in the SoA), `byteSize()`, `byteAlignment()`, `data()` (a pointer to the buffer). A `nextByte()` -function computes the first byte of a structure right after a layout, allowing using a single buffer for multiple -layouts. +To avoid cluttering the namespace of the generated layout class, a subclass called `Metadata` is generated. It is +instantiated with the `metadata()` member function and provides information about the layout and +its underlying storage, including: + +- `size()`: The number of elements per column in the SoA +- `byteSize()`: The total size of the buffer required by the layout +- `alignment()`: The alignment in bytes applied to each column +- `data()`: Returns a pointer to the `std::byte` buffer of the layout +- `nextByte()`: Returns the next byte after a layout, used for creating multiple layouts from a single buffer +- `cloneToNewAddress()`: Creates a new layout using a new buffer but the same number of elements per column ## Metarecords subclass The nested type `Metarecords` describes the elements of the SoA. It can be instantiated by the `records()` member function of a `View` or `ConstView`. Every object contains the address of the first element of the column, the number -of elements per column, and the stride for the Eigen columns. These are used to validate the columns size at run time +of elements per column, and the stride for the Eigen columns. These are used to validate the column size at run time and to build a generic `View` as described in [View](#view). ## Customized methods -It is possible to generate methods inside the `element` and `const_element` nested structs using the `SOA_ELEMENT_METHODS` -and `SOA_CONST_ELEMENT_METHODS` macros. Each of these macros can be called only once, and can define multiple methods. -Note that `SOA_ELEMENT_METHODS` and `SOA_CONST_ELEMENT_METHODS` should be prefixed with the macro SOA_HOST_DEVICE. -This ensures that the methods can also be executed in device kernels. -[An example is showed below.](#examples) +It is possible to generate methods inside the `element` and `const_element` nested structs using the +`SOA_ELEMENT_METHODS` and `SOA_CONST_ELEMENT_METHODS` macros. Each of these macros can be called only once, +and can define multiple methods. Note that `SOA_ELEMENT_METHODS` and `SOA_CONST_ELEMENT_METHODS` should be prefixed +with the macro SOA_HOST_DEVICE. This ensures that the methods can also be executed in device kernels. +[An example is shown below.](#examples) ## Blocks -`SoABlocks` is a macro-generated templated class that enables structured composition of multiple `SoALayouts` into a single -container, referred to as "blocks". Each block is a Layout, and the structure itself looks like multiple contigous memory -buffers of different sizes. The alignment is ensured to be the same for every block. `SoABlocks` also supports -`View` and `ConstView` classes. In addition to those fully parametrized templates, two further levels of parametrization are provided: +`SoABlocks` is a macro-generated templated class that enables structured composition of multiple `SoALayouts` +into a single container, referred to as "blocks". Each block is a Layout, and the structure itself +looks like multiple contiguous memory buffers of different sizes. +The block of an `SoABlock` layout can be an `SoABlock` in itself. Like this, nested SoA-layouts can be created. +Classes generated by the `GENERATE_SOA_BLOCKS` macro have the same template arguments as normal SoA-layouts. +The template arguments are passed to each block to ensure that, for example, the alignment is the same for every block. +`SoABlocks` also supports `View` and `ConstView` classes. +In addition to those fully parametrized templates, two further levels of parametrization are provided: `ViewTemplate`, `ViewTemplateFreeParams` and respectively `ConstViewTemplate`, `ConstViewTemplateFreeParams`, -mirroring the structure of the underlying structs. The blocks are built via composition and access to individual layouts -and views is provided by name. +mirroring the structure of the underlying structs. The blocks are built via composition, +and access to individual layouts and views is provided by name. TODOs: - Add introspection utilities to print the structure and layout of a `SoABlocks` instance. -- Implement support for heterogeneous `deepCopy()` operations between different but compatible `SoABlocks` configurations. -[An example of utilization is showed below.](#examples) +[An example of utilization is shown below.](#examples) ## ROOT serialization and de-serialization -Layouts can be serialized and de-serialized with ROOT. In order to generate the ROOT dictionary, separate +Layouts can be serialized and de-serialized with ROOT. To generate the ROOT dictionary, separate `clases_def.xml` and `classes.h` should be prepared. `classes.h` ensures the inclusion of the proper header files to get the definition of the serialized classes, and `classes_def.xml` needs to define the fixed list of members that ROOT should ignore, plus the list of all the columns. [An example is provided below.](#examples) -Serialization of Eigen data is not yet supported. - ## Template parameters -The template shared by layouts and parameters are: -- Byte aligment (defaulting to the nVidia GPU cache line size (128 bytes)) -- Alignment enforcement (`relaxed` or `enforced`). When enforced, the alignment will be checked at construction - time.~~, and the accesses are done with compiler hinting (using the widely supported `__builtin_assume_aligned` - intrinsic).~~ It turned out that hinting `nvcc` for alignement removed the benefit of more important `__restrict__` - hinting. The `__builtin_assume_aligned` is hence currently not use. - -In addition, the views also provide access parameters: -- Restrict qualify: add restrict hints to read accesses, so that the compiler knows it can relax accesses to the - data and assume it will not change. On nVidia GPUs, this leads to the generation of instruction using the faster - non-coherent cache. -- Range checking: add index checking on each access. As this is a compile time parameter, the cost of the feature at - run time is null if turned off. When turned on, the accesses will be slowed down by checks. Uppon error detection, +The template arguments of the generated SoA-layouts are: +- `ALIGNMENT` (default: 128 bytes): The byte alignment of each column, Eigen column, and scalar. + While the optimal alignment depends on the target hardware, using the same alignment across all devices in a + heterogeneous environment is generally preferable. This ensures that the entire backing buffer has the same memory + layout everywhere, allowing it to be transferred between devices in a single operation. If different alignments + are used, each column must instead be transferred individually. +- `ALIGNMENT_ENFORCEMENT` (default: `relaxed`): When enforced, the alignment of the whole buffer will be + checked at construction time of the layout, and the alignment of each column will be checked at construction + time of a view. Possible arguments are `enforced` (true) or `relaxed` (false) + +The template arguments of the Views are: +- `RESTRICT_QUALIFY` (default: true): + Adds `__restrict__` qualifiers to the column pointers, allowing the compiler + to assume that they do not alias. This enables more aggressive optimizations like SIMD vectorisation, or + for example on NVIDIA GPUs it results in the generation of load instructions that use the faster non-coherent cache. +- `RANGE_CHECKING` (default: `cms::soa::RangeChecking::Default`): + Adds out-of-bounds index checking on each access at runtime when using `enabled` or `extended`. `extended` + additionally outputs the file and line number of where `[]-operator` was called with a faulty index. + This is achieved by using `std::source_location`. As this is a compile-time parameter, the cost of the feature at + run time is null if turned off. When turned on, the accesses will be slowed down by checks. Upon error detection, an exception is launched (on the CPU side) or the kernel is made to crash (on the GPU side). This feature can help the debugging of index issues at runtime, but of course requires a recompilation. -The trivial views subclasses come in a variety of parametrization levels: `View` uses the same byte -alignement and alignment enforcement as the layout, and defaults (off) for restrict qualifying and range checking. -`ViewTemplate` template allows setting of restrict qualifying and range checking, while -`ViewTemplateFreeParams` allows full re-customization of the template parameters. +Several predefined view types are generated with different levels of template parameterization: +- `View`: uses the same template for `ALIGNMENT` and `ALIGNMENT_ENFORCEMENT` as the corresponding layout, + while using the default settings for `RESTRICT_QUALIFY` and `RANGE_CHECKING`. +- `ViewTemplate`: additionally exposes `RESTRICT_QUALIFY` and `RANGE_CHECKING`. +- `ViewTemplateFreeParams`: exposes all template parameters, allowing complete customization of the view. + +Note that the same variants for const access are available through +`ConstView`, `ConstViewTemplate`, `ConstViewTemplateFreeParams`. +Views are lightweight, trivially copyable objects. Consequently, +converting between views with different template parameters is inexpensive. ## Using SoA layouts and views with GPUs -Instanciation of views and layouts is preferably done on the CPU side. The view object is lightweight, with only one -pointer per column, plus the global number of elements. Extra view class can be generated to restrict this number of -pointers to the strict minimum in scenarios where only a subset of columns are used in a given GPU kernel. +An SoA layout is a host-side object and cannot be used directly inside a GPU kernel. +A view, on the other hand, is a lightweight object containing only one pointer per column and +the total number of elements. Views are typically constructed on the host and passed to GPU kernels by value, +although they can also be constructed on the device if needed. + +Additional view types can be generated that expose only a selected subset of columns, +reducing the number of stored pointers for kernels that access only part of the SoA. ## Examples @@ -140,24 +167,25 @@ A layout can be defined as: #include "DataFormats/SoALayout.h" GENERATE_SOA_LAYOUT(SoA1LayoutTemplate, - // predefined static scalars - // size_t size; - // size_t alignment; - - // columns: one value per element + // Columns: one value per SoA element. The element type may be a + // fundamental type, struct, or class. SOA_COLUMN(double, x), SOA_COLUMN(double, y), SOA_COLUMN(double, z), - SOA_EIGEN_COLUMN(Eigen::Vector3d, a), - SOA_EIGEN_COLUMN(Eigen::Vector3d, b), - SOA_EIGEN_COLUMN(Eigen::Vector3d, r), SOA_COLUMN(uint16_t, color), SOA_COLUMN(int32_t, value), SOA_COLUMN(double *, py), SOA_COLUMN(uint32_t, count), SOA_COLUMN(uint32_t, anotherCount), - // scalars: one value for the whole structure + // Eigen columns: fixed-size Eigen vectors or matrices stored in a + // columnar layout using one SoA column per component. + SOA_EIGEN_COLUMN(Eigen::Vector3d, a), + SOA_EIGEN_COLUMN(Eigen::Vector3d, b), + SOA_EIGEN_COLUMN(Eigen::Vector3d, r), + + // Scalars: a single value shared by the entire SoA, independent of + // the number of elements. SOA_SCALAR(const char *, description), SOA_SCALAR(uint32_t, someNumber) ); @@ -168,7 +196,8 @@ GENERATE_SOA_LAYOUT(SoA1LayoutTemplate, // > using SoA1Layout = SoA1LayoutTemplate<>; -using SoA1LayoutAligned = SoA1LayoutTemplate; +using SoA1LayoutAligned = SoA1LayoutTemplate; ``` It is possible to declare methods that operate on the SoA elements: @@ -211,11 +240,23 @@ The buffer of the proper size is allocated, and the layout is populated with: // Allocation of aligned size_t elements = 100; using AlignedBuffer = std::unique_ptr; -AlignedBuffer h_buf (reinterpret_cast(aligned_alloc(SoA1LayoutAligned::alignment, SoA1LayoutAligned::computeDataSize(elements))), std::free); +AlignedBuffer h_buf (reinterpret_cast(aligned_alloc(SoA1LayoutAligned::alignment, + SoA1LayoutAligned::computeDataSize(elements))), + std::free); SoA1LayoutAligned soaLayout(h_buf.get(), elements); ``` -The mutable and const views with the exact same set of columns and their parametrized variants are provided from the layout as: +The SoA provides an overloaded operator<< that outputs a detailed representation of its memory layout, +including column offsets, sizes, padding, and scalar fields. +This facilitates inspection and verification of the SoA layout through standard C++ output streams. + +```C++ +// Introspection of an SoALayout +std::cout << soaLayout +``` + +The mutable and const views with the same set of columns and their +parametrized variants are provided from the layout as: ```C++ // (Pseudo-code) @@ -244,12 +285,16 @@ template; -using SoAConstViewExtended = SoA::ConstViewTemplate; +using SoAViewExtended = SoA::ViewTemplate; +using SoAConstViewExtended = SoA::ConstViewTemplate; ``` The SoA by blocks can be created in this way: @@ -326,18 +373,19 @@ blocksView.scalars().energy() = 100.0f; ### Available features -- The layout and views support scalars and columns, alignment and alignment enforcement and hinting (linked). +- The layout and views support scalars and columns, alignment and alignment enforcement, and hinting (linked). - Automatic `__restrict__` compiler hinting is supported and can be enabled where appropriate. - Automatic creation of trivial views and const views derived from a single layout. - Cache access style, which was explored, was abandoned as this not-yet-used feature interferes with `__restrict__` - support (which is already in used in existing code). It could be made available as a separate tool that can be used + support (which is already in use in existing code). It could be made available as a separate tool that can be used directly by the module developer, orthogonally from SoA. -- Optional (compile time) range checking validates the index of every column access, throwing an exception on the +- Optional (compile-time) range checking validates the index of every column access, throwing an exception on the CPU side and forcing a segmentation fault to halt kernels. When not enabled, it has no impact on performance (code not compiled). Using `RangeChecking::extended` causes a capture of the source location using `std::source_location`, - when an integer index is passed to access the data. When an out-of-bounds error is thrown, this leads to more information - in the error message, including the file name and line number where the out-of-bounds index was passed to the SoA. -- Eigen columns are also suported, with both const and non-const flavors. + when an integer index is passed to access the data. When an out-of-bounds error is thrown, + this leads to more information in the error message, including the file name and line number + where the out-of-bounds index was passed to the SoA. +- Eigen columns are also supported, with both const and non-const flavors. - ROOT serialization and deserialization is supported. In CMSSW, it is planned to be used through the memory managing `PortableCollection` family of classes. - An `operator<<()` is provided to print the layout of an SoA to standard streams.