Skip to content

Add scouting to Phase-2 HLT menu - #51424

Merged
cmsbuild merged 2 commits into
cms-sw:masterfrom
silviodonato:addScoutingToPh2Menu
Jul 9, 2026
Merged

Add scouting to Phase-2 HLT menu#51424
cmsbuild merged 2 commits into
cms-sw:masterfrom
silviodonato:addScoutingToPh2Menu

Conversation

@silviodonato

@silviodonato silviodonato commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

PR description:

As discussed at the TSG coordination meetings [1] and [2], here a PR to add scouting to the Phase-2 HLT menu.
As this scouting will reconstruct almost all HLT objects in all events, it will be added only to HLT_75e33_cff.py, and not to HLT_75e33_timing_cff.py.
Note that in HLT_75e33_cff.py we are already reconstructing tracks and unseeded electrons in all events or so (see MC_TRK and MC_Ele5_Open_Unseeded), so I do not expect a big increase in timing of step2 due to this PR

PR validation:

I ran

cmsDriver.py step2  -s DIGI:pdigi_valid,L1TrackTrigger,L1,L1P2GT,DIGI2RAW,HLT:@relvalRun4 --conditions auto:phase2_realistic_T35 --datatier GEN-SIM-DIGI-RAW -n 10 --eventcontent FEVTDEBUGHLT --geometry ExtendedRun4D121 --era Phase2C22I13M9 --filein file:/eos/cms/store/relval/CMSSW_20_0_0_pre1/RelValTTbar_14TeV/GEN-SIM/150X_mcRun4_realistic_v1_STD_D121_RegeneratedGS_PU-v1/2590000/7dad7759-584f-4f99-8fdc-cdc531cebf4c.root

in CMSSW_20_0_0_pre1 with and without the changes. I checked that after the change the trigger bit DST_PFScouting appeared

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

No backport requested (as the 20_0_X production is supposed to start very soon)

@cmsbuild

cmsbuild commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

cms-bot internal usage

@cmsbuild

cmsbuild commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51424/50085

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR enables the existing DST_PFScouting HLT path in the Phase-2 HLT_75e33 menu by loading the path fragment and adding the path to the menu schedule, so it runs and appears in TriggerResults for that menu configuration.

Changes:

  • Load HLTrigger/Configuration/HLT_75e33/paths/DST_PFScouting_cfi in the HLT_75e33_cff.py fragment.
  • Add fragment.DST_PFScouting to fragment.schedule so the path is actually executed.
Comments suppressed due to low confidence (1)

HLTrigger/Configuration/python/HLT_75e33_cff.py:281

  • There is stray whitespace on the blank line after adding fragment.DST_PFScouting to the schedule (line contains indentation spaces only). This can create avoidable diff noise and may trip whitespace/formatting checks; keep the blank line empty (or remove it).
fragment.load("HLTrigger/Configuration/HLT_75e33/sequences/HLTVertexRecoSequence_cfi")
fragment.load("HLTrigger/Configuration/HLT_75e33/services/DQMStore_cfi")
fragment.load("HLTrigger/Configuration/HLT_75e33/services/FastTimerService_cfi")

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@cmsbuild

cmsbuild commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

A new Pull Request was created by @silviodonato for master.

It involves the following packages:

  • HLTrigger/Configuration (hlt)

@Martin-Grunewald, @cmsbuild, @mmusich can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @SohamBhattacharya, @VourMa, @missirol, @mmusich, @rovere this is something you requested to watch as well.
@ftenchini, @mandrenguyen, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

@mmusich

mmusich commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

test parameters:

  • enable = hlt_p2_integration
  • workflows = ph2_hlt

@mmusich

mmusich commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

@cmsbuild, please test

@cmsbuild

cmsbuild commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

+1

Size: This PR adds an extra 28KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-206301/54539/summary.html
COMMIT: 943cd7d
CMSSW: CMSSW_20_1_X_2026-07-05-2300/el9_amd64_gcc13
Additional Tests: HLT_P2_INTEGRATION
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/51424/54539/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-206301/54539/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-206301/54539/git-merge-result

Comparison Summary

Summary:

  • You potentially removed 10 lines from the logs
  • ROOTFileChecks: Some differences in event products or their sizes found
  • Reco comparison results: 29 differences found in the comparisons
  • DQMHistoTests: Total files compared: 59
  • DQMHistoTests: Total histograms compared: 4005665
  • DQMHistoTests: Total failures: 1984
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 4003663
  • DQMHistoTests: Total skipped: 18
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 58 files compared)
  • Checked 249 log files, 203 edm output root files, 59 DQM output files
  • TriggerResults: no differences found

Max Memory Comparisons exceeding threshold

@cms-sw/core-l2 , I found 1 workflow step(s) with memory usage exceeding the error threshold:

Expand to see workflows ...
  • Error: Workflow 34634.0_TTbar_14TeV+Run4D121PU step2 max memory diff 32.1 exceeds +/- 30.0 MiB

@mmusich

mmusich commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

@silviodonato can you please rebase to get rid of the merge commit?

@rovere

rovere commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

@silviodonato I want to make sure I understand how this fits with the NGT Scouting plans. The path here looks like the current NGT variant, where we've been assuming we'd keep a lot of freedom in the reconstruction approach. Promoting it as "standard scouting" raises a question for me about how that flexibility is preserved going forward — could you say a bit about how you're thinking about the scope? Keen to find a framing that works for both the menu and NGT.

@mmusich

mmusich commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Keen to find a framing that works for both the menu and NGT.

My minimal proposal would be to duplicate the paths, call the one used in the NGTScouting menu DST_NGTScouting and thus being free in the future to change in it anything without fear of interfering with the "standard" program. The con is obviously the duplication of configuration code.

@silviodonato

Copy link
Copy Markdown
Contributor Author

Phase-2 scouting is part of the standard CMS program, so I think it is normal to have DST_PFScouting in the standard HLT_75e33_cff.py.
I also fully agree that NGT should have a standalone menu with the freedom to implement whatever NGT wants
@mmusich's proposal to have two independent paths sounds as the natural solution.
I can implement it directly in this PR.

Probably the misunderstanding comes from

The path here looks like the current NGT variant

I thought in DST_PFScouting we run the standard phase-2 reconstruction

@mmusich

mmusich commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

@silviodonato

I thought in DST_PFScouting we run the standard phase-2 reconstruction

As a matter of fact we do, and the NGT-specific sequences are achieved via process modifiers (namely ngtScouting). As we evolve further the program we might want at some point to be free to implement more radical changes directly without impinging on the standard menu and dealing with process modifier acrobatics. That's at least how I read @rovere's remark

@cmsbuild

cmsbuild commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51424/50120

@cmsbuild

cmsbuild commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Pull request #51424 was updated. @Martin-Grunewald, @cmsbuild, @ctarricone, @gabrielmscampos, @mmusich, @rseidita can you please check and sign again.

@silviodonato

Copy link
Copy Markdown
Contributor Author

Thanks for spotting it. I checked that there are no other mention of DST_PFScouting in CMSSW

git grep DST_PFScouting | grep -v DST_PFScouting_
HLTrigger/Configuration/python/HLT_NGTScouting_cff.py:    fragment.DST_PFScouting,
Validation/HLTrigger/python/HLTGenValidation_cff.py:                             hltPathsToCheck = ['DST_PFScouting'])

@mmusich

mmusich commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

@cmsbuild, please test

@cmsbuild

cmsbuild commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

+1

Size: This PR adds an extra 24KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-206301/54596/summary.html
COMMIT: 758a925
CMSSW: CMSSW_20_1_X_2026-07-07-2300/el9_amd64_gcc13
Additional Tests: HLT_P2_INTEGRATION
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/51424/54596/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • You potentially removed 10 lines from the logs
  • ROOTFileChecks: Some differences in event products or their sizes found
  • Reco comparison results: 25 differences found in the comparisons
  • DQMHistoTests: Total files compared: 59
  • DQMHistoTests: Total histograms compared: 4005581
  • DQMHistoTests: Total failures: 1981
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 4003582
  • DQMHistoTests: Total skipped: 18
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.094 KiB( 58 files compared)
  • DQMHistoSizes: changed ( 34434.771,... ): 2.184 KiB HLTGenVal/ele__DST_NGTScouting__hltHpsPFTauTrack__2Dvsetaphi
  • DQMHistoSizes: changed ( 34434.771,... ): -2.183 KiB HLTGenVal/ele__DST_PFScouting__hltHpsPFTauTrack__2Dvsetaphi
  • DQMHistoSizes: changed ( 34434.771,... ): 2.173 KiB HLTGenVal/ele__DST_NGTScouting__GEN__2Dvseta__phi
  • DQMHistoSizes: changed ( 34434.771,... ): -2.172 KiB HLTGenVal/ele__DST_PFScouting__GEN__2Dvseta__phi
  • DQMHistoSizes: changed ( 34434.771,... ): 1.748 KiB HLTGenVal/AK4jet__DST_NGTScouting__hltHpsPFTauTrack__2Dvspteta
  • DQMHistoSizes: changed ( 34434.771,... ): 1.748 KiB HLTGenVal/tauHAD__DST_NGTScouting__hltHpsPFTauTrack__2Dvspteta
  • DQMHistoSizes: changed ( 34434.771,... ): 1.747 KiB HLTGenVal/AK4HT__DST_NGTScouting__hltHpsPFTauTrack__2Dvspteta
  • DQMHistoSizes: changed ( 34434.771,... ): -1.747 KiB HLTGenVal/AK4jet__DST_PFScouting__hltHpsPFTauTrack__2Dvspteta
  • DQMHistoSizes: changed ( 34434.771,... ): -1.747 KiB HLTGenVal/tauHAD__DST_PFScouting__hltHpsPFTauTrack__2Dvspteta
  • DQMHistoSizes: changed ( 34434.771,... ): -1.746 KiB HLTGenVal/AK4HT__DST_PFScouting__hltHpsPFTauTrack__2Dvspteta
  • DQMHistoSizes: changed ( 34434.771 ): ...
  • Checked 249 log files, 203 edm output root files, 59 DQM output files
  • TriggerResults: no differences found

Max Memory Comparisons exceeding threshold

@cms-sw/core-l2 , I found 1 workflow step(s) with memory usage exceeding the error threshold:

Expand to see workflows ...
  • Error: Workflow 34634.0_TTbar_14TeV+Run4D121PU step2 max memory diff 33.3 exceeds +/- 30.0 MiB

@mmusich

mmusich commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

DQMHistoSizes: changed ( 34434.771,... ): 2.184 KiB HLTGenVal/ele__DST_NGTScouting__hltHpsPFTauTrack__2Dvsetaphi

this is expected because of the path renaming.

DQMHistoTests: Total failures: 1981

it looks like most of the failures come from the fact that now tau reconstruction runs "unseeded" due to the scouting path, which I think it's reasonable as the other MC paths already reconstructed most of the physics objects in an unseeded way.

@mmusich

mmusich commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

+hlt

@gabrielmscampos

Copy link
Copy Markdown
Member

+dqm

@cmsbuild

cmsbuild commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @sextonkennedy, @mandrenguyen, @ftenchini (and backports should be raised in the release meeting by the corresponding L2)

@mandrenguyen

Copy link
Copy Markdown
Contributor

+1

@cmsbuild
cmsbuild merged commit 26f87c1 into cms-sw:master Jul 9, 2026
12 checks passed
NJManganelli pushed a commit to NJManganelli/cmssw that referenced this pull request Jul 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants