Intergration of TOoLLiP_v3. by Brainz22 · Pull Request #51057 · cms-sw/cmssw

Brainz22 · 2026-05-27T22:19:16Z

PR description:

This PR integrates TOoLLiP_v3 into CMSSW, which involves the newest development of the Level-1 long-lived-particle jet trigger. The official TOoLLiP development repository is found here.
- The submodule with all relevant files is found under TOoLLiP/TOoLLiP_v3 and linked here, e.g. ROC curves, HLS firmware TOoLLiP_v3 folder, Python to HLS conversion scripts, and layer tracing, and model plots.
The PR to pass in the correct TOoLLiP version on cms-sw/cmsdist can be found here

PR validation:

This PR has been validated through the emulation of the trigger using the fastPuppi ntuple producer. The validation method is as follows:

cmsrel CMSSW_17_0_0_pre1
cd CMSSW_17_0_0_pre1/src
cmsenv
git cms-init 
git branch 
git cms-checkout-topic -u Brainz22:prtestingbranch

git cms-addpkg L1Trigger/Phase2L1ParticleFlow
git cms-addpkg L1Trigger/Configuration
git cms-addpkg DataFormats
git cms-addpkg L1Trigger/TrackTrigger
git cms-addpkg SimTracker/TrackTriggerAssociation
scram b -j8 

#steps for model binary
git clone https://github.com/cms-hls4ml/hls4mlEmulatorExtras.git 
cd hls4mlEmulatorExtras 
git checkout -b v1.1.3 tags/v1.1.3
make install
cd ..

git clone --quiet https://github.com/Xilinx/HLS_arbitrary_Precision_Types.git hls

git clone git@github.com:cms-hls4ml/TOoLLiP.git
cd TOoLLiP
git checkout -b v3.1.0 tags/v3.1.0
make install
export TOOLLIP_PATH=$PWD/TOoLLiP_v3 #need because binary is not in cmsdist yet
cd ..

#add testing
git clone git@github.com:Brainz22/FastPUPPI.git -b 15_1_X_LLPtagging #clones LLPtagging branch
cd FastPUPPI
git checkout 04aa685101d9454eab134d079599178b3099ee89 # checks out and specific commit
cd ..
scram b -j8

#produce nanoAOD file
cmsRun FastPUPPI/NtupleProducer/python/runPerformanceNTuple.py

perfNano.root will appeat at $PWD. This ntuple contains the branch scPuppiL1TTOOLLIPJet_llpTagScore as defined by the lines here.

TOoLLiP_v3 emulation checks have been performed on signal and minbias samples, where inputs are taken from cmssw directly and used to test qkeras and HLS models. Their predictions are compared against the emulation predictions. Plot are shown below (original_score axis has distribution of qkeras model predictions):

The qkeras model outputs differ from hls (and thus, cmssw) outputs due to arithmetic precision and quantization mismatches in the shift from qkeras to hls. This conclusion can be inferred from the spread in the hls and qkeras model final layer outputs:

No backports needed. The workflow works fine for CMSSW_17_0_0_pre1.

cmsbuild · 2026-05-27T22:19:42Z

cms-bot internal usage

cmsbuild · 2026-05-27T22:21:17Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51057/49508

cmsbuild · 2026-05-28T22:59:01Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51057/49561

cmsbuild · 2026-05-28T22:59:27Z

Pull request #51057 was updated.

quinnanm · 2026-06-02T01:06:48Z

please test

cmsbuild · 2026-06-02T01:07:17Z

A new Pull Request was created by @Brainz22 for master.

It involves the following packages:

L1Trigger/Phase2L1ParticleFlow (l1)

@BenjaminRS, @quinnanm can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @missirol, @mmusich this is something you requested to watch as well.
@ftenchini, @mandrenguyen, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

cmsbuild · 2026-06-02T02:05:30Z

-1

Failed Tests: RelVals
Size: This PR adds an extra 16KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-c94f12/53638/summary.html
COMMIT: b3ac961
CMSSW: CMSSW_17_0_X_2026-06-01-2300/el8_amd64_gcc13
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/51057/53638/install.sh to create a dev area with all the needed externals and cmssw changes.

Failed RelVals

34434.911

----- Begin Fatal Exception 02-Jun-2026 03:51:29 CEST-----------------------
An exception of category 'StdException' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=TOoLLiPProducer label='l1tTOoLLiPProducer'
Exception Message:
A std::exception was thrown.
hls4ml emulator model library dlopen failure!
----- End Fatal Exception -------------------------------------------------

34434.0

----- Begin Fatal Exception 02-Jun-2026 03:46:34 CEST-----------------------
An exception of category 'StdException' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=TOoLLiPProducer label='l1tTOoLLiPProducer'
Exception Message:
A std::exception was thrown.
hls4ml emulator model library dlopen failure!
----- End Fatal Exception -------------------------------------------------

34434.75

----- Begin Fatal Exception 02-Jun-2026 03:47:04 CEST-----------------------
An exception of category 'StdException' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=TOoLLiPProducer label='l1tTOoLLiPProducer'
Exception Message:
A std::exception was thrown.
hls4ml emulator model library dlopen failure!
----- End Fatal Exception -------------------------------------------------

Expand to see more relval errors ...

Brainz22 · 2026-06-02T22:52:20Z

I've seen the error happen when the model binary cannot be found. Probably because PR#10591 has not been yet merged in this case?

quinnanm · 2026-06-04T13:50:25Z

please test with cms-sw/cmsdist#10591

makortel · 2026-06-05T18:31:56Z

Just checking, is this PR intended for Phase 2 (20_1_X) or Run 3 legacy (17_0_X) or both?

Brainz22 · 2026-06-08T17:27:54Z

This is a Phase 2 upgrade @makortel, but does it hurt to have a working version on both 20_1_x and 17_0_X?

makortel · 2026-06-08T17:35:01Z

This is a Phase 2 upgrade @makortel, but does it hurt to have a working version on both 20_1_x and 17_0_X?

That would be up to @cms-sw/l1-l2. The 17_0_X will need a separate backport PR as usual (and you can use the same topic branch as in this PR, as long as it is not rebased to anything newer than 17_0_0_pre2).

BenjaminRS · 2026-06-08T19:41:28Z

Hi Matti and Russell - we will just have this Phase 2 PR going to 20_1_X.

quinnanm · 2026-06-08T20:54:27Z

please test with cms-sw/cmsdist#10591

cmsbuild · 2026-06-08T23:06:08Z

+1

Size: This PR adds an extra 16KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-c94f12/53745/summary.html
COMMIT: 07cd729
CMSSW: CMSSW_20_0_X_2026-06-08-1100/el8_amd64_gcc13
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/51057/53745/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

@Dr15Jones Fix finding operator<< for reco::io_v1::PFTau #51134

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-c94f12/53745/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-c94f12/53745/git-merge-result

Comparison Summary

Summary:

You potentially added 8 lines to the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 44
DQMHistoTests: Total histograms compared: 3248390
DQMHistoTests: Total failures: 42
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3248330
DQMHistoTests: Total skipped: 18
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 43 files compared)
Checked 191 log files, 159 edm output root files, 44 DQM output files
TriggerResults: no differences found

cmsbuild · 2026-06-09T07:28:11Z

Milestone for this pull request has been moved to CMSSW_20_1_X. Please open a backport if it should also go in to CMSSW_20_0_X.

cmsbuild · 2026-06-10T08:10:13Z

REMINDER @sextonkennedy, @ftenchini, @mandrenguyen: This PR was tested with cms-sw/cmsdist#10591, please check if they should be merged together

BenjaminRS · 2026-07-01T09:02:51Z

@@ -31,7 +31,7 @@ class JetId {

  void setNNVectorVar();
  float EvaluateNN();
-  ap_fixed<16, 6> EvaluateNNFixed();
+  ap_fixed<14, 8, AP_TRN, AP_SAT, 0> EvaluateNNFixed();


Should

cmssw/L1Trigger/Phase2L1ParticleFlow/interface/JetId.h

Line 36 in b3ac961

ap_fixed<16, 6> computeFixed(const l1t::PFJet &iJet, float vz, bool useRawPt);

also be changed to <14, 8, AP_TRN, AP_SAT, 0> or it is ok to keep that as <16, 6>?

Thanks for the catch. I checked things and I found no difference in going from <16, 6> to <14, 8, AP_TRN, AP_SAT, 0>. However, I updated the precision accordingly to keep things consistent.

cmsbuild · 2026-09-09T04:55:54Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51057/50874

cmsbuild · 2026-09-09T04:56:18Z

Pull request #51057 was updated. @BenjaminRS, @cmsbuild, @quinnanm can you please check and sign again.

Intergration of TOoLLiP_v3.

dce5b49

cmsbuild added this to the CMSSW_17_0_X milestone May 27, 2026

cmsbuild added l1-pending pending-signatures tests-pending orp-pending code-checks-pending labels May 27, 2026

cmsbuild added code-checks-approved and removed code-checks-pending labels May 27, 2026

Brainz22 mentioned this pull request May 28, 2026

Updated TOoLLiP to version 3.1.0. cms-sw/cmsdist#10591

Merged

Cleaned up producer to resemble original and changed v1 to v3.

b3ac961

cmsbuild added code-checks-pending and removed code-checks-approved labels May 28, 2026

cmsbuild added code-checks-approved and removed code-checks-pending labels May 28, 2026

Brainz22 marked this pull request as ready for review June 1, 2026 16:30

cmsbuild added tests-started and removed tests-pending labels Jun 2, 2026

cmsbuild added tests-rejected and removed tests-started labels Jun 2, 2026

cmsbuild added the requires-external label Jun 4, 2026

cmsbuild added tests-started requires-external and removed tests-pending labels Jun 8, 2026

cmsbuild added tests-approved and removed tests-started labels Jun 8, 2026

cmsbuild modified the milestones: CMSSW_20_0_X, CMSSW_20_1_X Jun 9, 2026

cmsbuild modified the milestones: CMSSW_20_0_X, CMSSW_20_1_X Jun 10, 2026

BenjaminRS reviewed Jul 1, 2026

View reviewed changes

Updated precision of top function.

5f177ba

cmsbuild added tests-pending code-checks-pending and removed tests-approved requires-external code-checks-approved labels Sep 9, 2026

cmsbuild added code-checks-approved and removed code-checks-pending labels Sep 9, 2026

Uh oh!

Conversation

Brainz22 commented May 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR description:

PR validation:

Uh oh!

cmsbuild commented May 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cmsbuild commented May 27, 2026

Uh oh!

cmsbuild commented May 28, 2026

Uh oh!

cmsbuild commented May 28, 2026

Uh oh!

quinnanm commented Jun 2, 2026

Uh oh!

cmsbuild commented Jun 2, 2026

Uh oh!

cmsbuild commented Jun 2, 2026

Failed RelVals

Uh oh!

Brainz22 commented Jun 2, 2026

Uh oh!

quinnanm commented Jun 4, 2026

Uh oh!

makortel commented Jun 5, 2026

Uh oh!

Brainz22 commented Jun 8, 2026

Uh oh!

makortel commented Jun 8, 2026

Uh oh!

BenjaminRS commented Jun 8, 2026

Uh oh!

quinnanm commented Jun 8, 2026

Uh oh!

cmsbuild commented Jun 8, 2026

Comparison Summary

Uh oh!

cmsbuild commented Jun 9, 2026

Uh oh!

cmsbuild commented Jun 10, 2026

Uh oh!

BenjaminRS Jul 1, 2026

Choose a reason for hiding this comment

Uh oh!

Brainz22 Sep 9, 2026

Choose a reason for hiding this comment

Uh oh!

cmsbuild commented Sep 9, 2026

Uh oh!

cmsbuild commented Sep 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Brainz22 commented May 27, 2026 •

edited

Loading

cmsbuild commented May 27, 2026 •

edited

Loading