Add Gaudi Functional C++ Class Generator

README.md

@@ -118,3 +118,51 @@ each one of the above-mentioned algorithms. In addition, there are tests that
 have either multiple inputs and / or multiple outputs (like
 `ExampleFunctionalProducerMultiple`) that can be used as a template for the more
 typical case when working with multiple inputs or outputs.
+
+### Generating boilerplate with generateFunctional
+
+`k4FWCore/helpers/generateFunctional` is a code generator that produces the C++
+boilerplate for a new functional algorithm. The functional type (Consumer,
+Producer, Transformer, MultiTransformer, FilterPredicate) is inferred
+automatically from the number of inputs and outputs, or can be set explicitly.
+
+Requirements: Python ≥ 3.9 and [jinja2](https://pypi.org/project/Jinja2/).
+With [uv](https://github.com/astral-sh/uv) installed, dependencies are
+resolved automatically via the PEP 723 script block.
+
+```bash
+# Producer with one output collection and one property
+python3 k4FWCore/helpers/generateFunctional MyProducer \
+    -o 'edm4hep::MCParticleCollection:OutputCollection' \
+    -p 'int:ExampleInt:3:An example integer property'
+
+# Transformer (inferred from 1 input + 1 output)
+python3 k4FWCore/helpers/generateFunctional MyTransformer \
+    -i 'edm4hep::MCParticleCollection:InputCollection' \
+    -o 'edm4hep::MCParticleCollection:OutputCollection' \
+    --private-properties
+
+# MultiTransformer with type aliases
+python3 k4FWCore/helpers/generateFunctional MyMulti \
+    -i 'edm4hep::MCParticleCollection:Particles' \
+       'edm4hep::TrackCollection:Tracks' \
+    -o 'edm4hep::MCParticleCollection:NewParticles' \
+       'podio::UserDataCollection<int>:Counter' \
+    --type-aliases
+
+# FilterPredicate (type must be specified explicitly)
+python3 k4FWCore/helpers/generateFunctional MyFilter filter \
+    -i 'edm4hep::MCParticleCollection:InputCollection'
+
+# Consumer with runtime (variable-length) input collections
+python3 k4FWCore/helpers/generateFunctional MyConsumer \
+    -i 'edm4hep::MCParticleCollection:Inputs' \
+    --runtime-inputs 'edm4hep::MCParticleCollection:Inputs:MCParticles0,MCParticles1'
+
+# Also emit a CMakeLists.txt skeleton
+python3 k4FWCore/helpers/generateFunctional MyProducer \
+    -o 'edm4hep::MCParticleCollection:OutputCollection' \
+    --cmake
+```
+
+Run `python3 k4FWCore/helpers/generateFunctional --help` for the full list of options.

k4FWCore/CMakeLists.txt

@@ -18,6 +18,8 @@ limitations under the License.
 ]]
 
 gaudi_install(SCRIPTS)
+install(PROGRAMS helpers/generateFunctional
+  DESTINATION ${CMAKE_INSTALL_BINDIR})
 
 gaudi_add_library(k4FWCore
                   SOURCES src/KeepDropSwitch.cpp

k4FWCore/helpers/AGENT.md

@@ -0,0 +1,162 @@
+# AGENT.md — k4FWCore/helpers
+
+Context for AI agents working on `generateFunctional` and its test suite.
+
+---
+
+## What this directory contains
+
+| File/Dir | Purpose |
+|---|---|
+| `generateFunctional` | Code generator: produces Gaudi Functional C++ boilerplate from CLI arguments |
+| `README.md` | Full user-facing reference (arguments, examples, exit codes) |
+| `tests/` | Bash build-tests: generate → cmake → build for each algorithm type |
+| `tests/_test_common.sh` | Shared helpers sourced by every `test_*.sh` |
+| `tests/run_all_tests.sh` | Runs all `test_*.sh` and reports a pass/fail summary |
+
+---
+
+## Architecture of generateFunctional
+
+The script is intentionally structured so that all string parsing happens once at the CLI boundary and never again:
+
+```
+CLI args
+  └─ _build_spec()          Parses strings → AlgorithmSpec dataclass
+       └─ generate()         AlgorithmSpec → (cpp_source, cmake_source)
+            ├─ _build_includes()
+            ├─ _build_constructor()   k4FWCore style (with brace rules below)
+            ├─ _build_constructor_gaudi()
+            ├─ _build_op_signature()
+            ├─ _build_op_body()
+            └─ Jinja2 template (_CPP_TEMPLATE, _CMAKE_TEMPLATE)
+```
+
+### Key data classes
+
+- **`DataSpec`** — one input or output collection: `type_name`, `key`, `is_vector`.
+  - `edm4hep_header` property returns `TypeCollection.h` (keep "Collection" in filename — a past bug stripped it).
+  - `_default_key()` derives a key from the type name by stripping namespace and `Collection`, e.g. `edm4hep::MCParticleCollection` → `MCParticles`.
+
+- **`RuntimeInputSpec`** — a `DataSpec` with additional default location names for `KeyValues`.
+
+- **`PropertySpec`** — a `Gaudi::Property<T>` member.
+  - `member_name` lowercases the first character after `m_`: `Offset` → `m_offset`, not `m_Offset`.
+
+- **`AlgorithmSpec`** — the single object passed through all generation functions. Contains all inputs, outputs, options, and derived properties used by templates.
+
+### Constructor brace rules
+
+This is the trickiest part of code generation. The k4FWCore constructors follow different conventions per type:
+
+| Type | Inputs | Outputs |
+|---|---|---|
+| `Consumer` / `FilterPredicate` | bare for single: `KeyValue(...)` | — |
+| `Consumer` / `FilterPredicate` | braced for multiple: `{KeyValue(...), ...}` | — |
+| `Producer` | `{}` (always empty) | bare for single: `KeyValue(...)` |
+| `Producer` | `{}` | braced for multiple: `{KeyValues(...), ...}` |
+| `Transformer` / `MultiTransformer` | **always braced**: `{KeyValue(...)}` | **always braced**: `{KeyValue(...)}` |
+
+In `_build_constructor`, `_brace_block()` always wraps in `{}` (used for transformer/producer), while `_bare_block()` leaves a single item unwrapped (used for consumer/filter).
+
+### Template structure (_CPP_TEMPLATE)
+
+Order of sections in the generated `.cpp`:
+
+1. `// Generated by ...` header comment with full command line
+2. `#include` directives
+3. Optional `using BaseClass_t = ...` (Gaudi framework only)
+4. Optional `using retType = std::tuple<...>` (k4FWCore multi-output)
+5. Optional `using XxxColl = ...` type aliases (`--type-aliases`)
+6. Optional `namespace X {`
+7. Class definition:
+   - Constructor
+   - `StatusCode initialize()` (only when vector inputs are present)
+   - `operator()`
+   - `StatusCode finalize()` (only with `--event-context`) — **must be before `private:`**
+   - Optional `private:` label (when `--private-properties` or `--event-context`)
+   - Properties
+   - `mutable std::set<unsigned long> m_eventNumbersSeen{}` and `m_mutex` (only with `--event-context`)
+8. `DECLARE_COMPONENT(ClassName)`
+
+### Functional type inference
+
+```
+inputs > 0, outputs == 0  →  consumer
+inputs == 0, outputs > 0  →  producer
+inputs > 0, outputs == 1  →  transformer
+inputs > 0, outputs > 1   →  multitransformer
+filter                    →  never inferred; must be explicit
+```
+
+`transformer` auto-promotes to `multitransformer` when multiple `--outputs` are given.
+
+---
+
+## Known remaining gaps vs. test examples
+
+These are design limitations, not bugs:
+
+1. **`KeyValue` default location = key name.** The script emits `KeyValue("OutputCollection", "OutputCollection")` but test examples have `KeyValue("OutputCollection", "MCParticles")`. There is no CLI argument for a separate default location value.
+
+2. **Include order.** Script: `k4FWCore/` first, then `Gaudi/Property.h`, then `edm4hep/`. Test examples: `Gaudi/Property.h` first, then `edm4hep/`, then `k4FWCore/`.
+
+3. **Multi-transformer output aliases.** Test examples define individual `using Counter = ...; using Particle = ...;` aliases for each output type. The script emits a single `using retType = std::tuple<...>` with raw types.
+
+4. **No license header.** The script emits `// Generated by ...`; test examples carry the Apache 2.0 block.
+
+---
+
+## Test scripts
+
+Each script in `tests/` covers one feature axis:
+
+| Script | Feature |
+|---|---|
+| `test_producer.sh` | Single output, property |
+| `test_consumer.sh` | Single input, property |
+| `test_transformer.sh` | Single in/out, `--private-properties` |
+| `test_multitransformer.sh` | Multiple in/out, `--type-aliases`, `podio::UserDataCollection` |
+| `test_filter.sh` | `FilterPredicate` |
+| `test_runtime_consumer.sh` | `--runtime-inputs` / `KeyValues` vector input |
+| `test_runtime_transformer.sh` | `--runtime-outputs` / `std::vector<T>` return |
+| `test_event_context.sh` | `--event-context`, `finalize()` placement |
+| `test_gaudi_framework.sh` | `--framework gaudi`, `--namespace` |
+
+Each script sources `_test_common.sh` which:
+- Finds `generateFunctional` (installed on `PATH` first, then `../generateFunctional` fallback)
+- Creates a `mktemp -d` sandbox, cleaned up on `EXIT`
+- Provides `run_cmake_build <ClassName> [args...]` that runs generate → cmake configure → cmake build
+
+Tests require a Key4hep environment (`k4FWCore`, `EDM4HEP`, `Gaudi` on `CMAKE_PREFIX_PATH`). Source the Key4hep setup before running:
+
+```bash
+source /cvmfs/sw.hsf.org/key4hep/setup.sh
+bash k4FWCore/helpers/tests/run_all_tests.sh
+```
+
+---
+
+## Installation
+
+`generateFunctional` is installed to `CMAKE_INSTALL_BINDIR` via `k4FWCore/CMakeLists.txt`:
+
+```cmake
+install(PROGRAMS helpers/generateFunctional
+  DESTINATION ${CMAKE_INSTALL_BINDIR})
+```
+
+After `cmake --install`, `generateFunctional` is on `PATH` in the Key4hep environment.
+
+---
+
+## Common mistakes to avoid
+
+- **Do not strip `Collection` from edm4hep header filenames.** `edm4hep::MCParticleCollection` → `edm4hep/MCParticleCollection.h`, not `edm4hep/MCParticle.h`. See `DataSpec.edm4hep_header`.
+- **Do not wrap `Consumer`/`FilterPredicate` single inputs in braces.** Only `Transformer`/`Producer` use `_brace_block()`.
+- **`finalize()` must be emitted before `private:`.** The Jinja2 template places `finalize()` in its own block before the `{% if spec.private_props or spec.event_context %}private:{% endif %}` block.
+- **Property member names must be lowercase after `m_`.** `PropertySpec.member_name` lowercases `n[0]`; do not change this or generated names diverge from k4FWCore conventions.
+- **`--runtime-outputs` is k4FWCore-only.** The parser enforces this, but the cmake template only adds podio explicitly for `--framework gaudi`; for k4fwcore it is a transitive dependency of `k4FWCore::k4FWCore`.
+- **Do not link `Gaudi::GaudiAlgLib` for `--framework gaudi`.** This target was removed in Gaudi 40.x. The cmake template links only `Gaudi::GaudiKernel`.
+- **Native Gaudi constructor takes separate input and output arguments, not a single merged list.** `_build_constructor_gaudi` passes `_arg(in_kvs), _arg(out_kvs)` as separate arguments. A single KV is bare; multiple KVs are `{kv1, kv2, ...}`.
+- **`DECLARE_COMPONENT` must use the fully qualified name when `--namespace` is set.** The template emits `DECLARE_COMPONENT(Ns::ClassName)` outside the namespace block.

k4FWCore/helpers/AGENT_USAGE.md

@@ -0,0 +1,107 @@
+# Using an AI Agent with generateFunctional
+
+You can ask an AI agent (such as Claude in Cowork or via the API) to run
+`generateFunctional` for you. Instead of memorising flags, describe your
+algorithm in plain language and the agent handles the rest.
+
+---
+
+## How it works
+
+1. You describe the algorithm you need.
+2. The agent translates your description into a `generateFunctional` command.
+3. The agent runs the command and shows you the generated `.cpp` (and
+   optionally `CMakeLists.txt`).
+4. You ask for changes; the agent re-runs with updated flags.
+
+---
+
+## What to tell the agent
+
+The more detail you provide, the closer the first attempt will be to what you
+want. Cover these points:
+
+| What | Example |
+|---|---|
+| **Algorithm name** | `MyParticleSelector` |
+| **Inputs** — type and key name | `edm4hep::MCParticleCollection` named `InputParticles` |
+| **Outputs** — type and key name | `edm4hep::MCParticleCollection` named `SelectedParticles` |
+| **Properties** — C++ type, name, default, description | `float` named `MinPt`, default `0.5`, "Minimum transverse momentum" |
+| **Private properties?** | Yes / No |
+| **EventContext needed?** | Yes / No |
+| **Type aliases?** | Yes / No |
+| **CMake file too?** | Yes / No |
+| **Framework** | `k4fwcore` (default) or `gaudi` |
+| **C++ namespace** | e.g. `MyExperiment` |
+
+You do not need to know any flags — just describe what you want.
+
+---
+
+## Example prompts
+
+### Minimal — let the agent fill in the gaps
+
+> Generate a transformer called `TrackFilter` that reads
+> `edm4hep::TrackCollection` and writes a filtered
+> `edm4hep::TrackCollection`.
+
+### With properties
+
+> Generate a transformer `EnergyThresholdFilter` that takes
+> `edm4hep::MCParticleCollection:InputParticles` as input and returns
+> `edm4hep::MCParticleCollection:OutputParticles`. Add a float property
+> `MinEnergy` with default `1.0` and description "Minimum particle energy in
+> GeV". Put properties under `private:`. Also emit a `CMakeLists.txt`.
+
+### Multiple inputs and outputs
+
+> I need a MultiTransformer `JetBuilder` with two inputs —
+> `edm4hep::MCParticleCollection:Particles` and
+> `edm4hep::TrackCollection:Tracks` — and two outputs —
+> `edm4hep::ReconstructedParticleCollection:Jets` and
+> `podio::UserDataCollection<float>:JetPt`. Use type aliases.
+
+### Runtime (variable-length) inputs
+
+> Generate a consumer `MultiCollectionReader` that reads a variable number
+> of `edm4hep::MCParticleCollection` inputs at runtime, with default names
+> `MCParticles0` and `MCParticles1`.
+
+### From an existing example
+
+> Look at `ExampleFunctionalTransformerRuntimeCollections.cpp` in the test
+> folder and generate something similar for `edm4hep::TrackCollection`.
+
+### Refinement after seeing the output
+
+> That looks good. Can you add an `int` property `MaxParticles` with default
+> `100`, and regenerate with `--force`?
+
+---
+
+## What the agent can do automatically
+
+- Infer the functional type (`Consumer`, `Producer`, `Transformer`,
+  `MultiTransformer`) from your inputs and outputs.
+- Derive default key names from collection types when you don't specify them
+  (e.g. `edm4hep::MCParticleCollection` → key `MCParticles`).
+- Add the correct `#include` directives for all edm4hep and podio types.
+- Emit `DECLARE_COMPONENT()` and a ready-to-build `CMakeLists.txt`.
+- Re-run with `--force` to overwrite after you request changes.
+
+---
+
+## Tips
+
+- **`filter` must be explicit.** The agent cannot infer `FilterPredicate` from
+  I/O counts alone — say "FilterPredicate" or "filter type" in your prompt.
+- **Key names matter.** If your steering file already names the collections,
+  tell the agent the exact keys so the generated `KeyValue` strings match.
+- **Iterate freely.** Generated code is cheap to redo. Ask the agent to tweak
+  property types, add an `EventContext`, switch to `--use-class`, or change
+  the namespace — it will re-run the generator rather than hand-editing the
+  output.
+- **Review before committing.** Check the generated constructor argument order
+  and `operator()` signature against your project's conventions before adding
+  the file to git.