copilot was used to parse my notes below into an issue
Background
While developing the experimental Hinton plot work in PR #2419, the original implementation assumed a fixed number of variables (five). However, the preferred direction is to allow users to configure an arbitrary number of variables for inclusion in the Hinton plot, rather than enforcing a hard-coded limit.
To achieve this, an operator was introduced to perform iteration over variables/cases, as recipe YAML currently lacks a mechanism for looping over collections. This operator was temporarily placed in scoreswrapper.py as part of prototyping the end-to-end workflow (draft form here: https://github.com/MetOffice/CSET/pull/2419/changes#diff-42de57b369cdd291381c081213b5a4970b790f359ff365d878a9d5a35ad69ebeR1106).
This raises a broader architectural question around the separation of responsibilities between recipes, operators, and loaders within CSET.
Problem
The current solution requires an operator that effectively orchestrates repeated operations across multiple variables/cases. Conceptually, this feels closer to what a recipe should be responsible for, but recipes currently cannot express looping behaviour.
As a result:
- Logic that feels "recipe-like" is being moved into operators.
- The distinction between recipes, operators, and loaders becomes less clear.
- Future use cases may encounter the same limitation when attempting to handle dynamic collections of inputs.
An alternative approach could be for the loader to expand templates into fully populated YAML recipes by generating repeated sections automatically. However, this could make generated recipes harder to understand, debug, and maintain.
Another approach is to generalise the scope of the operators further to take multiple variables, multiple forecasts, multiple models, but this would require a significant generalisation of operators.
This might be quite a unique example, that we do not encounter again for this kind of plot.
copilot was used to parse my notes below into an issue
Background
While developing the experimental Hinton plot work in PR #2419, the original implementation assumed a fixed number of variables (five). However, the preferred direction is to allow users to configure an arbitrary number of variables for inclusion in the Hinton plot, rather than enforcing a hard-coded limit.
To achieve this, an operator was introduced to perform iteration over variables/cases, as recipe YAML currently lacks a mechanism for looping over collections. This operator was temporarily placed in scoreswrapper.py as part of prototyping the end-to-end workflow (draft form here: https://github.com/MetOffice/CSET/pull/2419/changes#diff-42de57b369cdd291381c081213b5a4970b790f359ff365d878a9d5a35ad69ebeR1106).
This raises a broader architectural question around the separation of responsibilities between recipes, operators, and loaders within CSET.
Problem
The current solution requires an operator that effectively orchestrates repeated operations across multiple variables/cases. Conceptually, this feels closer to what a recipe should be responsible for, but recipes currently cannot express looping behaviour.
As a result:
An alternative approach could be for the loader to expand templates into fully populated YAML recipes by generating repeated sections automatically. However, this could make generated recipes harder to understand, debug, and maintain.
Another approach is to generalise the scope of the operators further to take multiple variables, multiple forecasts, multiple models, but this would require a significant generalisation of operators.
This might be quite a unique example, that we do not encounter again for this kind of plot.