Track Purity DNN for Phase-2 HLT - #51084
Conversation
|
cms-bot internal usage |
|
+code-checks Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51084/49552
|
|
A new Pull Request was created by @jchismar for master. It involves the following packages:
@Martin-Grunewald, @Moanwar, @cmsbuild, @davidlange6, @fabiocos, @ftenchini, @jfernan2, @mandrenguyen, @mmusich, @srimanob can you please review it and eventually sign? Thanks. cms-bot commands are listed here |
|
Hi @jchismar , thanks, which workflows needed to test this PR ? |
|
for the record, the needed model is at cms-data/RecoTracker-FinalTrackSelectors#15 (it wold be nice to link the two) |
| import FWCore.ParameterSet.Config as cms | ||
|
|
||
| # This modifier sets the use of a deep neural network for high purity track selection | ||
| trackTorchClassifier = cms.Modifier() |
There was a problem hiding this comment.
for my understanding why is this proposed via a modifier and not directly in the "production" workflow?
There was a problem hiding this comment.
Circling back to this, I am fine with having this by default in the HLT phase-2 reconstruction with the current level of performance. Should the modifier be removed and the changes applied directly in the menu?
| HLTInitialStepHPSelectionSequence = cms.Sequence( | ||
| hltInitialStepTrackCutClassifier | ||
| +hltInitialStepTrackSelectionHighPurity | ||
| ) |
| +hltInitialStepTrackTorchClassifierOutput | ||
| +hltInitialStepTrackCutClassifier | ||
| +hltInitialStepTrackSelectionHighPurity | ||
| ) |
|
@jchismar what is the cost in terms of timing and GPU memory consumption of these developments? |
|
the modifier solution was mainly motivated by much earlier interpretation that PyTorchAlpaka carries a significant memory cost (a significant fraction of 1 GB). It sounds from the pixel track DNN that the cost is much smaller. The timing costs were rather small ( @mmusich cmssw/Configuration/Eras/python/Era_Phase2_cff.py Lines 24 to 25 in 2cc100f Once the tests run we can decide if it's OK to move on for production or keep as a modifier |
Thanks @slava77
FWIW, that is fine with me. |
|
+code-checks Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51084/49603
|
the CPU backend memory average is up by 2.6 GB (and by 2.8 GB on CPU with the GPU backend running; sort of consistent considering the code in this PR is now resident on CPU). IIUC the event data should be small, 15 float features per track, about/under 2K tracks per event is ~100KB/event and just 15MB with 8job*16stream. |
For information, the reason we do that is due to the interplay between the framework view of what accelerators are available ( Using process.moduleInstance = cms.EDProducer("moduleInstance@alpaka")and setting process.moduleInstance.alpaka.backend = "serial_sync"tells the resolver to use the CPU serial backend instead of the preferred one; however the CPU backend still needs to be a valid accelerator according to the framework. Instead using process.moduleInstance = cms.EDProducer("alpaka_serial_sync::moduleInstance")bypasses the alpaka module resolver and uses unconditionally the CPU serial version. |
can you please implement this in the next push? |
|
+code-checks Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51084/50054
|
|
Pull request #51084 was updated. @Martin-Grunewald, @Moanwar, @cmsbuild, @davidlange6, @fabiocos, @ftenchini, @jfernan2, @mandrenguyen, @mmusich, @srimanob can you please check and sign again. |
|
@cmsbuild, please test |
|
+1 Size: This PR adds an extra 44KB to repository HLT P2 Timing: chart Comparison SummarySummary:
Max Memory Comparisons exceeding threshold@cms-sw/core-l2 , I found 21 workflow step(s) with memory usage exceeding the error threshold: Expand to see workflows ...
|
|
+code-checks Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51084/50133
|
|
Pull request #51084 was updated. @Martin-Grunewald, @Moanwar, @cmsbuild, @davidlange6, @fabiocos, @ftenchini, @jfernan2, @mandrenguyen, @mmusich, @srimanob can you please check and sign again. |
|
@cmsbuild please test |
|
+1 Size: This PR adds an extra 52KB to repository The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
You can see more details here: HLT P2 Timing: chart Comparison SummarySummary:
Max Memory Comparisons exceeding threshold@cms-sw/core-l2 , I found 21 workflow step(s) with memory usage exceeding the error threshold: Expand to see workflows ...
|
|
@jchismar may I ask you to squash the commits to a minimum before final sign-off? |
Co-authored-by: Jade Chismar <jchismar@ucsd.edu>
|
+code-checks Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51084/50149
|
|
Pull request #51084 was updated. @Martin-Grunewald, @Moanwar, @cmsbuild, @davidlange6, @fabiocos, @ftenchini, @jfernan2, @mandrenguyen, @mmusich, @srimanob can you please check and sign again. |
|
@cmsbuild please test ... for completeness |
|
+1 Size: This PR adds an extra 20KB to repository The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
You can see more details here: HLT P2 Timing: chart Comparison SummarySummary:
Max Memory Comparisons exceeding threshold@cms-sw/core-l2 , I found 21 workflow step(s) with memory usage exceeding the error threshold: Expand to see workflows ...
|
|
@cmsbuild, please test
|
|
+1 Size: This PR adds an extra 16KB to repository The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic: You can see more details here: HLT P2 Timing: chart Comparison SummarySummary:
Max Memory Comparisons exceeding threshold@cms-sw/core-l2 , I found 21 workflow step(s) with memory usage exceeding the error threshold: Expand to see workflows ...
|
|
+hlt |
|
Sorry for not following this closely, but did we resolve the memory increase of ~30 MB than the threshold in step2? |
The previous commit did not aim at resolving any memory increase - a memory increase of a few tens of MBs when running an ML model is expected and has been seen in other developments. As a result, the summary from the bot is consistent and expected (at least for HLT, which is what PR currently targets). |
|
+1
|
|
This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @ftenchini, @mandrenguyen, @sextonkennedy (and backports should be raised in the release meeting by the corresponding L2) |
|
+1 |
12 participants
Implementation of a track purity DNN used for high purity selection for HLT tracks. Initial results were presented at the tracking POG meeting on 15 Dec 2025. Since then, the model has been retrained with the latest version of LST, and a separate threshold has been implemented for displaced tracks (|dxy| > 0.5) to improve displaced track efficiency. This threshold is set at a target efficiency of 99.5% calculated on tracks with |dxy| > 0.5. For tracks with |dxy|$\le$ 0.5, the threshold is set at a target efficiency of 99.5% calculated on all tracks. Additionally, the number of input features has been reduced from 29 to 15 with no loss of performance. The DNN is configured to run in the HLTInitialStepSequence after the hltInitialStepTracks step when the trackTorchClassifier procModifier is used.
MTV performance on TT+PU=200 is shown below.
