Update OverlayTiming to support BIB random mixing by madbaron · Pull Request #413 · key4hep/k4FWCore

madbaron · 2026-07-06T13:11:13Z

BEGINRELEASENOTES

Extend OverlayTiming with random background-file mixing: the new RandomMixBackgroundFiles option treats each file in a background group as an independent event source and picks a random set of files for every overlaid event. BackgroundFileNames entries may now be directories (their .root files are used).
Add the MergeMCParticles option to OverlayTiming (default true); when false, background MCParticles are not stored, tracker hits keep the momentum of their originating particle and calorimeter contributions get an empty particle.
Serialize all background ROOT I/O on a dedicated worker thread so OverlayTiming is safe to run with intra-event multithreading.

ENDRELEASENOTES

This PR updates OverlayTiming with the logic used by the muon collider software to overlay the BIB pseudo-events (from https://github.com/MuonColliderSoft/k4Reco/blob/main/k4Reco/Overlay/components/OverlayTimingRandomMix.cpp).
I opted for porting the changes over rather than asking to include a second algorithm, since the code was 95% the same.

The updated algorithm uses TBB for intra-event multithreading in processing the thousands of inputs for the overlay.

madbaron · 2026-07-07T06:42:43Z

The downstream build failure doesn't seem related to the changes - but I can't retrigger it.

andread3vita · 2026-07-24T11:49:42Z

Hi! I tried this implementation, and if I simply set overlay.RandomMixBackgroundFiles = True, the overlay works using just a single background file instead of 40:

from Gaudi.Configuration import INFO

from k4FWCore import ApplicationMgr
from k4FWCore import IOSvc
from Configurables import EventDataSvc
from Configurables import OverlayTiming
from Configurables import UniqueIDGenSvc

from pathlib import Path


background_base = Path("/eos/experiment/fcc/ee/simulation/key4hep_2026_04_20/91GeV/IDEA_o1_v03/IPC_Z_background")
background_file_list = []
for d in sorted(background_base.iterdir()):
    background_file_list.append(str(d))
            
id_service = UniqueIDGenSvc("UniqueIDGenSvc")
eds = EventDataSvc("EventDataSvc")
iosvc = IOSvc()
# iosvc.Input = "/afs/cern.ch/user/a/aloeschc/fcc_fullsim_testing_grounds/data/IDEA/IDEA_o1_v03/physics_events/p8_ee_Z_qqbar_ud/1000_91.188GeV_ISR_FSR/000/IDEA_o1_v03_1000_p8_ee_Z_qqbar_ud_91.188GeV_ISR_FSR.root"
iosvc.Input = "/afs/cern.ch/work/a/adevita/public/testBIB/IDEA_test/muonGunIDEAv3o1.root" 

iosvc.Output = "IDEA_o1_v03_OverlayIPC_test.root"

overlay = OverlayTiming()
overlay.MCParticles = "MCParticles"
overlay.BackgroundMCParticleCollectionName = "MCParticles"
overlay.SimTrackerHits = ["DCHCollection", "MuonSystemCollection", "SiWrDCollection", "SiWrBCollection", "VertexBarrelCollection", "VertexEndcapCollection", "PreshowerSystemCollection"]
overlay.SimCalorimeterHits = []
overlay.OutputSimTrackerHits = ["OverlayDCHCollection", "OverlayMuonSystemCollection", "OverlaySiWrDCollection", "OverlaySiWrBCollection", "OverlayVertexBarrelCollection", "OverlayVertexEndcapCollection", "OverlayPreshowerSystemCollection"]
overlay.OutputSimCalorimeterHits = []
overlay.OutputMCParticles = "OverlayMCParticles"
overlay.OutputCaloHitContributions = []
overlay.AllowReusingBackgroundFiles = True
overlay.CopyCellIDMetadata = True
overlay.NBunchtrain = 41          # total BX in train
overlay.NumberBackground = [1]    # one background event per BX
overlay.Delta_t = 20              # ns between BX
overlay.PhysicsBX = 21            # puts physics at 21 with 20 before & 30 after (allow for hits 200ns after event time)
overlay.Poisson_random_NOverlay = [False]
overlay.StartBackgroundEventIndex = -1
# overlay.BackgroundFileNames = [
#       background_file_list
# ]

overlay.RandomMixBackgroundFiles = True
overlay.BackgroundFileNames = [["/eos/experiment/fcc/ee/simulation/key4hep_2026_04_20/91GeV/IDEA_o1_v03/IPC_Z_background"]]

overlay.TimeWindows = {"MCParticles": [-400, 400], "DCHCollection": [-400, 400], "MuonSystemCollection": [-20, 0], "SiWrDCollection": [-20, 0],"SiWrBCollection": [-20, 0], "VertexBarrelCollection": [-20, 0],"VertexEndcapCollection": [-20, 0], "PreshowerSystemCollection": [-20, 0]}

iosvc.outputCommands = ["drop *", "keep OverlayDCHCollection*", "keep OverlaySiWrDCollection*", "keep OverlaySiWrBCollection*", "keep OverlayVertexBarrelCollection*", "keep OverlayVertexEndcapCollection*", "keep OverlayMC*", "keep *EventHeader*"]


ApplicationMgr(TopAlg=[overlay],
               EvtSel="NONE",
               EvtMax=1,
               ExtSvc=[eds],
               OutputLevel=INFO,
               )

ArinaPon · 2026-07-28T11:17:45Z

Hello! while testing the implementation, I think I might have an idea why only one background file was being used across the bunch train.

fileIndices is shuffled before the BX loop, but k starts again from 0 for every BX. With NumberBackground = [1], every BX therefore selects:

fileIndices[0]

And I guess this means that the same randomly selected background file is reused for all BXs of the signal event.

I tried adding a counter outside the BX loop:

size_t fileCursor = 0;

and changing the file selection to:

const int fileIndex =
    m_randomMix ? fileIndices[fileCursor++ % fileIndices.size()] : 0;

With this change, the code continues through the shuffled list instead of starting again from the first file for every BX.

I rebuilt and tested this with 120 IPC background files and the overlay completed successfully, the different events showed different background patterns.

madbaron · 2026-07-31T21:33:06Z

Thanks for the checks @andread3vita and the suggestion for a fix @ArinaPon.
I ended up implementing it ~1:1, with the main difference that on wrap of the fileCursor I reshuffle so that there is no recurring pattern in overlaid background events.

…Core into add_overlay_BIB_random_mix

tmadlener · 2026-08-11T19:14:37Z

+With many large background files the algorithm is dominated by reading and
+decompressing them. Set `OverlayThreads` to a value greater than 1 to read and
+decompress the background files of a single event on several threads:


Conceptually this might interfere with Gaudis internal scheduling (even if we also use tbb to do our multithreading). It's unclear to me whether the Gaudi internal tbb bits communicate with the tbb bits here.

There is precedent for doing this though as the CKF in k4ActsTracking also does some internal multithreading. This might need some policy discussion as it could imply different usage patterns for different community (e.g. run the general chain on a single thread but branch out to multi-threading in dedicated algorithms vs. running the full chain on multiple threads with Gaudi scheduling but no algorithm-internal multi-threading).

Indeed. In the ideal world you might want to allow users to do a combination of both, if possible.
For now, especially in colliders that are computationally challenging per event, being able to use MT inside the same event is much more important than multi-threading over events, which can be done trivially in batch jobs anyway.

Maybe @jmcarcell knows if functional algorithms can already propagate that to the Gaudi scheduler somehow. Otherwise the potential interplay will for now just be another thing to document.

With the help of Claude, I had a look at the Gaudi sources. My understanding is that there's no way for a functional algorithm to declare or hand its internal parallelism to the scheduler. But the two TBB layers aren't independent either.

The good news is we can't oversubscribe the machine; the bad news is the scheduler counts algorithms in flight, not threads, so it keeps dispatching while we're fanned out and conversely with ThreadPoolSize=1 the pipeline only gets ~2 threads regardless of OverlayThreads (ntokens bounds in-flight items, not concurrency).

One thing I've added as a precaution: the pipeline now runs inside tbb::this_task_arena::isolate(). The reasoning is that while the calling thread blocks in parallel_pipeline, TBB may steal another AlgTask onto it, and AlgTask::operator() sets the thread-local EventContext and calls whiteboard()->selectStore(slot) without restoring either. While I haven't observed this, it costs nothing, so I'd rather keep it than rely on arena timing.

I've documented the OverlayThreads / ThreadPoolSize interplay in doc/OverlayTiming.md: they draw from one pool, OverlayThreads > 1 pays off when ThreadPoolSize is small, and raising both just repartitions the same threads.

madbaron added 8 commits July 6, 2026 12:57

merging randomMix functionalities into overlaytiming

5c76efa

docs

54afad5

more docs

4c17050

undo over-eager release notes

9ef1063

phrasing of pseudo-events

fdfde27

use TBB

a315f27

clang format

24b2107

update MT docs

b517240

Merge branch 'main' into add_overlay_BIB_random_mix

2fb0c4f

madbaron added 5 commits July 31, 2026 15:38

Merge branch 'main' into add_overlay_BIB_random_mix

d3758ff

implement suggestion from ArinaPon

a983a90

Fix clang-format in OverlayTiming.cpp

9aab53b

updated comments and added shuffle on wrap

349e589

make pre-commit happy again

b8b4f57

madbaron mentioned this pull request Aug 9, 2026

Tracking-related preparation for release 3.1 MuonColliderSoft/MAIAConfig#6

Merged

madbaron added 4 commits August 10, 2026 13:41

merge with upstream

0fad14e

Merge branch 'add_overlay_BIB_random_mix' of github.com:madbaron/k4FW…

33f9829

…Core into add_overlay_BIB_random_mix

preempt clang tidy

6e6b20c

fix clang-tidy

3b80ce7

tmadlener reviewed Aug 12, 2026

View reviewed changes

madbaron added 2 commits August 14, 2026 14:18

Merge branch 'main' into add_overlay_BIB_random_mix

a25b300

Merge branch 'main' into add_overlay_BIB_random_mix

598f020

This was referenced Aug 29, 2026

Rebase of muon collider fork key4hep/k4Reco#50

Closed

Add entry-level BIB overlay MuonColliderSoft/k4Reco#20

Closed

madbaron added 2 commits September 1, 2026 12:48

add isolate

caa0afb

Merge branch 'main' into add_overlay_BIB_random_mix

1264570

This was referenced Sep 1, 2026

switch to OverlayTiming MuonColliderSoft/MAIAConfig#43

Open

fix: protect against out of bounds access #425

Open

Wishlist for release 3.2 MuonColliderSoft/mucoll-spack#95

Open

Navigation Menu

Uh oh!

Update OverlayTiming to support BIB random mixing - #413

Update OverlayTiming to support BIB random mixing#413
madbaron wants to merge 22 commits into
key4hep:mainfrom
madbaron:add_overlay_BIB_random_mix

madbaron commented Jul 6, 2026 •

edited

Loading

Uh oh!

madbaron commented Jul 7, 2026

Uh oh!

andread3vita commented Jul 24, 2026

Uh oh!

ArinaPon commented Jul 28, 2026

Uh oh!

madbaron commented Jul 31, 2026

Uh oh!

tmadlener Aug 11, 2026

Uh oh!

madbaron Aug 14, 2026

Uh oh!

tmadlener Aug 14, 2026

Uh oh!

madbaron Sep 1, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

madbaron commented Jul 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

madbaron commented Jul 7, 2026

Uh oh!

andread3vita commented Jul 24, 2026

Uh oh!

ArinaPon commented Jul 28, 2026

Uh oh!

madbaron commented Jul 31, 2026

Uh oh!

tmadlener Aug 11, 2026

Choose a reason for hiding this comment

Uh oh!

madbaron Aug 14, 2026

Choose a reason for hiding this comment

Uh oh!

tmadlener Aug 14, 2026

Choose a reason for hiding this comment

Uh oh!

madbaron Sep 1, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

madbaron commented Jul 6, 2026 •

edited

Loading