Skip to content

Update TICL & HGCal simulation truth validation to use new simulation truth collections, and simplify validation to use only SimCluster dataformat - #51047

Open
tcuisset wants to merge 4 commits into
cms-sw:masterfrom
tcuisset:caloTruth-pr2-rebase2
Open

Conversation

@tcuisset

Copy link
Copy Markdown
Contributor

PR description:

This is a follow-up on #50578, which had updated CaloTruthAccumulator to produce multiple SimCluster collections, with the intent to use only one dataformat for calorimeter truth information (instead of CaloParticle / SimCluster dataformat dichotomy).

This PR updates the TICL/HGCal validation to use the new SimCluster collections produced in CaloTruthAccumulator.
Use of CaloParticle dataformat is replaced nearly everywhere with the use of caloparticles in SimCluster dataformat (with just some remaining usage related to simTime).

Summary of the setup after this PR:
Screenshot from 2026-05-27 11-47-07

Documentation
Slides at TICL meeting
Slide 7 of HGCAL presentation at NGT MC truth workshop

Details:

  • Associators with CaloParticle are removed (SimCluster associators are used for the "CaloParticle" SimCluster collection). They were mostly duplicated code
  • SimTrackstersProducer is split into two producers for clarity : SimTrackstersProducers (only for simtracksters) and SimTICLCandidateProducer (produces SimTICLCandidates)
  • SimTrackstersProducer is now configurable to produce an arbitrary number of simTrackster collections (instead of CP/SC only). Each collection has also a RefVector to the SimCluster that was used to produce the simTrackster.
  • SimTICLCandidateProducer is also configurable wrt the SimTrackster collections used
  • all the code designed to deal with the rare special cases where a SimTrackster is made from SimClusters that do not have a SimTrack crossing boundary is removed, since this corner case cannot happen with "boundary" SimClusters
  • validation configs for HLT and TICLbarrel are updated accordingly

PR validation:

Tested with various Phase 2 workflows:

  • 29634.0 (TTbar), 29634.758 (+HLT timing ticl_barrel), 29634.773 (+NGT scouting Nano val), 29634.7591 (+HLTNanoValid),
  • 34553.209 (CloseByPGun_Barrel_Front+Run4D121_ticl_barrel)

There should not be any validation differences in reco objects. Very minor differences in validation could be caused by the switch from "legacy" to "boundary" SimCluster (which could change slightly validation results for status=1 photons that do not convert before reaching the calorimeter)

FYI @felicepantaleo @waredjeb

@cmsbuild

cmsbuild commented May 27, 2026

Copy link
Copy Markdown
Contributor

cms-bot internal usage

@cmsbuild

Copy link
Copy Markdown
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51047/49491

@cmsbuild

Copy link
Copy Markdown
Contributor

A new Pull Request was created by @tcuisset for master.

It involves the following packages:

  • DataFormats/HGCalReco (reconstruction)
  • HLTrigger/NGTScouting (hlt)
  • RecoHGCal/TICL (reconstruction)
  • SimCalorimetry/HGCalAssociatorProducers (simulation)
  • SimCalorimetry/HGCalSimProducers (simulation)
  • SimDataFormats/Associations (simulation)
  • Validation/Configuration (dqm, simulation)
  • Validation/HGCalValidation (dqm)

@Martin-Grunewald, @Moanwar, @civanch, @cmsbuild, @ctarricone, @gabrielmscampos, @jfernan2, @kpedro88, @mandrenguyen, @mdhildreth, @mmusich, @rseidita, @srimanob can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @apsallid, @bsunanda, @cseez, @denizsun, @fabiocos, @felicepantaleo, @forthommel, @hatakeyamak, @lecriste, @lgray, @martinamalberti, @missirol, @mmusich, @pfs, @rovere, @salimcerci, @sameasy, @sethzenz, @sobhatta, @vandreev11 this is something you requested to watch as well.
@ftenchini, @mandrenguyen, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

@felicepantaleo

Copy link
Copy Markdown
Contributor

@cmsbuild please test

@mmusich

mmusich commented May 27, 2026

Copy link
Copy Markdown
Contributor

@cmsbuild, please abort

@mmusich

mmusich commented May 27, 2026

Copy link
Copy Markdown
Contributor

test parameters:

  • workflows = ph2_hlt

@mmusich

mmusich commented May 27, 2026

Copy link
Copy Markdown
Contributor

@cmsbuild, please test

@cmsbuild

Copy link
Copy Markdown
Contributor

-1

Failed Tests: RelVals
Size: This PR adds an extra 136KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-becf79/53501/summary.html
COMMIT: a66a777
CMSSW: CMSSW_17_0_X_2026-05-26-2300/el8_amd64_gcc13
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/51047/53501/install.sh to create a dev area with all the needed externals and cmssw changes.

Failed RelVals

----- Begin Fatal Exception 27-May-2026 13:43:14 CEST-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'validation_step14'
   [2] Prefetching for module HGCalValidator/'hltHgcalValidator'
   [3] Calling method for module SimTrackstersProducer/'hltTiclSimTracksters'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: std::vector<SimCluster>
Looking for module label: mixData
Looking for productInstanceName: MergedCaloTruthBoundaryTrackSimCluster

   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "TryToContinue = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

@jfernan2

Copy link
Copy Markdown
Contributor

please test

@cmsbuild

Copy link
Copy Markdown
Contributor

+1

Size: This PR adds an extra 24KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-becf79/54716/summary.html
COMMIT: 65c86e3
CMSSW: CMSSW_20_1_X_2026-07-12-2300/el9_amd64_gcc13
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/51047/54716/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • You potentially removed 5 lines from the logs
  • ROOTFileChecks: Some differences in event products or their sizes found
  • Reco comparison results: 65 differences found in the comparisons
  • DQMHistoTests: Total files compared: 59
  • DQMHistoTests: Total histograms compared: 3842662
  • DQMHistoTests: Total failures: 37533
  • DQMHistoTests: Total nulls: 3
  • DQMHistoTests: Total successes: 3805108
  • DQMHistoTests: Total skipped: 18
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 2664.352999999999 KiB( 58 files compared)
  • DQMHistoSizes: changed ( 34434.0,... ): 779.092 KiB HGCAL/HGCalValidator
  • DQMHistoSizes: changed ( 34434.0,... ): -64.793 KiB HLT/HGCAL
  • DQMHistoSizes: changed ( 34434.0,... ): -0.004 KiB MessageLogger/Errors
  • DQMHistoSizes: changed ( 34434.0,... ): -0.004 KiB MessageLogger/Warnings
  • Checked 249 log files, 203 edm output root files, 59 DQM output files
  • TriggerResults: found differences in 19 / 58 workflows

Max Memory Comparisons exceeding threshold

@cms-sw/core-l2 , I found 1 workflow step(s) with memory usage exceeding the error threshold:

Expand to see workflows ...
  • Error: Workflow 34634.0_TTbar_14TeV+Run4D121PU step3 max memory diff 102.8 exceeds +/- 30.0 MiB

@Moanwar

Moanwar commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Hi @tcuisset, Thanks a lot for fixing and updating the PR! Could you please have a look at the memory increase? It seems to be ~70 MB above the threshold. Is that expected?

…om eventContent to reduce the memory usage when running on pileup
@tcuisset

Copy link
Copy Markdown
Contributor Author

Hi @tcuisset, Thanks a lot for fixing and updating the PR! Could you please have a look at the memory increase? It seems to be ~70 MB above the threshold. Is that expected?

Hi @Moanwar , I've reduced the number of associators that run by default in the config, so it should reduce the memory usage when running with pileup now.

@cmsbuild

Copy link
Copy Markdown
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51047/50208

@Moanwar

Moanwar commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Hi @tcuisset, Thanks a lot for fixing and updating the PR! Could you please have a look at the memory increase? It seems to be ~70 MB above the threshold. Is that expected?

Hi @Moanwar , I've reduced the number of associators that run by default in the config, so it should reduce the memory usage when running with pileup now.

Thanks a lot @tcuisset , sounds good , lets test again

@Moanwar

Moanwar commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

@cmsbuild please test

@cmsbuild

Copy link
Copy Markdown
Contributor

Pull request #51047 was updated. @Martin-Grunewald, @Moanwar, @battibass, @civanch, @ctarricone, @ftorrresd, @gabrielmscampos, @jfernan2, @kpedro88, @mandrenguyen, @mdhildreth, @mmusich, @rseidita, @srimanob can you please check and sign again.

@cmsbuild

Copy link
Copy Markdown
Contributor

+1

Size: This PR adds an extra 36KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-becf79/54772/summary.html
COMMIT: c885ee3
CMSSW: CMSSW_20_1_X_2026-07-15-1100/el9_amd64_gcc13
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/51047/54772/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • You potentially removed 5 lines from the logs
  • ROOTFileChecks: Some differences in event products or their sizes found
  • Reco comparison results: 69 differences found in the comparisons
  • DQMHistoTests: Total files compared: 59
  • DQMHistoTests: Total histograms compared: 3842950
  • DQMHistoTests: Total failures: 35849
  • DQMHistoTests: Total nulls: 3
  • DQMHistoTests: Total successes: 3807080
  • DQMHistoTests: Total skipped: 18
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: -1533.7920000000008 KiB( 58 files compared)
  • DQMHistoSizes: changed ( 34434.0,... ): -64.793 KiB HLT/HGCAL
  • DQMHistoSizes: changed ( 34434.0,... ): -60.537 KiB HGCAL/HGCalValidator
  • DQMHistoSizes: changed ( 34434.0,... ): -0.004 KiB MessageLogger/Errors
  • DQMHistoSizes: changed ( 34434.0,... ): -0.004 KiB MessageLogger/Warnings
  • Checked 249 log files, 203 edm output root files, 59 DQM output files
  • TriggerResults: found differences in 20 / 58 workflows

@Moanwar

Moanwar commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Hi @tcuisset, Thanks a lot for fixing and updating the PR! Could you please have a look at the memory increase? It seems to be ~70 MB above the threshold. Is that expected?

Hi @Moanwar , I've reduced the number of associators that run by default in the config, so it should reduce the memory usage when running with pileup now.

Thanks @tcuisset, it seems to be working.
Just for my understanding, after this change, is what we have in the description still valid? And is the diagram still correct as well?

@tcuisset

Copy link
Copy Markdown
Contributor Author

Hi @tcuisset, Thanks a lot for fixing and updating the PR! Could you please have a look at the memory increase? It seems to be ~70 MB above the threshold. Is that expected?

Hi @Moanwar , I've reduced the number of associators that run by default in the config, so it should reduce the memory usage when running with pileup now.

Thanks @tcuisset, it seems to be working. Just for my understanding, after this change, is what we have in the description still valid? And is the diagram still correct as well?

Mostly the diagram is still accurate. For the left part (SimCluster part), there is no change. For the right part, the SimTracksters for "merged" and "legacy" are disabled by default (but the "CaloParticle" and "Boundary-track" are still enabled). But you just need to uncomment a few lines of python config to enable them back. The related associators are also disabled (they were not shown on the diagram).
All this can be re-recoed if needed.

@Moanwar

Moanwar commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

+1

@civanch

civanch commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

+simulation

@gabrielmscampos

Copy link
Copy Markdown
Member

+dqm

@mmusich

mmusich commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

(the branch unfortunately needs a rebase, now)

Comment on lines +35 to +38
eVector0_y=Var(
"eigenvectors()[0].z", "float", doc="Trackster PCA principal axis, y component"),
eVector0_z=Var(
"eigenvectors()[0].y", "float", doc="Trackster PCA principal axis, z component"),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the swap between z and y intended?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Btw, apologies for the delay

@battibass

Copy link
Copy Markdown

enable nano

@mmusich

mmusich commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

code-checks

@cmsbuild

cmsbuild commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.