Skip to content

Heterogeneous Alpaka implementation of electron pixel seed-matching - #51200

Open
ckoraka wants to merge 13 commits into
cms-sw:masterfrom
ckoraka:pixelSeedingAlpaka_20_1_X
Open

ckoraka wants to merge 13 commits into
cms-sw:masterfrom
ckoraka:pixelSeedingAlpaka_20_1_X

Conversation

@ckoraka

@ckoraka ckoraka commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

PR description:

This PR introduces a heterogeneous implementation of the electron pixel seed-matching algorithm, intended to replace the legacy ElectronNHitSeedProducer. It introduces the following developments:

  • Two new SoA data formats in DataFormats/EgammaReco:

    • ElectronSeedSoA: describes the pixel-matching seed information (hit positions, detector surfaces/rotations, and pixel-match variables dPhi/dRZ) in a portable layout.
    • SuperClusterSoA: describes the minimal supercluster information needed for matching (position and energy) in a memory-efficient portable layout.
  • Portable utility functions for helix track propagation, ported to be usable on device: helixBarrelPlaneCrossingByCircle, helixForwardPlaneCrossing, helixArbitraryPlaneCrossing(and 2nd order), ftsFromVertexToPointPortable, EleRelPointPairPortable, and Plane.

  • A dedicated Alpaka kernel that performs the seed-to-supercluster matching on the device.

Two new producers implementing the new workflow:

  • ElectronNHitSeedAlpakaProducer: fills the ElectronSeedSoA/SuperClusterSoA host collections from the legacy TrajectorySeed and SuperCluster collections, copies them to the device, and launches the matching kernel.
  • ElectronSeedConverter: converts the matched seeds back to the legacy format, and additionally performs a doublet-rejection algorithm on CPU that discards doublet seeds with a high number of valid trajectory layers which is a known source of fake-electron contributions.

PR validation:

Timing and performance studies have been conducted to validate these developments. The results can be found here:

  • Using the Phase-2 menu: [Link]
  • Using the Run-3 menu: [Link]

Note One can run these developments using :
runTheMatrix.py -w upgrade -l 37634.7511

@cmsbuild

cmsbuild commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

cms-bot internal usage

@cmsbuild

Copy link
Copy Markdown
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51200/49727

@cmsbuild

Copy link
Copy Markdown
Contributor

A new Pull Request was created by @ckoraka for master.

It involves the following packages:

  • DataFormats/EgammaReco (reconstruction)
  • RecoEgamma/Configuration (reconstruction)
  • RecoEgamma/EgammaElectronAlgos (reconstruction)
  • RecoEgamma/EgammaElectronProducers (reconstruction)

@Moanwar, @cmsbuild, @jfernan2, @mandrenguyen, @srimanob can you please review it and eventually sign? Thanks.
@Prasant1993, @Sam-Harper, @a-kapoor, @afiqaize, @jainshilpi, @lgray, @missirol, @mmusich, @ram1123, @rovere, @sameasy, @sobhatta, @valsdav, @varuns23 this is something you requested to watch as well.
@ftenchini, @mandrenguyen, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

@Moanwar

Moanwar commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

enable gpu

@Moanwar

Moanwar commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

@cmsbuild, please test

@fwyzard

fwyzard commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

assign heterogeneous

@cmsbuild

Copy link
Copy Markdown
Contributor

New categories assigned: heterogeneous

@fwyzard,@makortel you have been requested to review this Pull request/Issue and eventually sign? Thanks

Comment on lines +11 to +12
namespace ALPAKA_ACCELERATOR_NAMESPACE {
namespace reco {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could be simplified to

Suggested change
namespace ALPAKA_ACCELERATOR_NAMESPACE {
namespace reco {
namespace ALPAKA_ACCELERATOR_NAMESPACE::reco {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

corrected in ckoraka@4552547

}

template <typename TAcc>
ALPAKA_FN_ACC T partial_norm(const TAcc& acc) const {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

partial_norm() and partial_norm2() are not very descriptive.

How about r() and r2() ?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

renamed to follow ROOT conventions, along with several other functions.


Phys3DVector(const Phys3DVector<T>&) = default;

constexpr Phys3DVector(const T& value) : m_data{} {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably better to pass it by value:

Suggested change
constexpr Phys3DVector(const T& value) : m_data{} {
constexpr Phys3DVector(const T value) : m_data{} {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addressed with 2d78844


namespace cms::alpakatools::math {

template <class T>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a short documentation, explaining that Phys3DVector represents a vector in 3D space in cartesian (x, y, z) or cylindrical (r, z) coordinates ?

@fwyzard fwyzard Jun 11, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, how much more complicated would it be to use an Eigen::Matrix<T, 3, 1> here ?
It might give better performance due to how Eigen combines multiple operations.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Eigen was attempted earlier, but that was hit by #48928

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A related bug has been reported to NVIDIA and fixed in CUDA 13.1, so this may have been resolved by now.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, did anything change in the implementation ?
Now this looks like only Cartesian coordinates are used.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fwyzard as far as I can tell, there were no essential changes to the implementation itself (some methods were simply migrated to free functions). But it still provides conversion methods that are used elsewhere, e.g. in interface/EleRelPointPairPortable.h.
Note that the header itself is now isolated in RecoEgamma/EgammaElectronAlgos/interface, so it is solely associated with Egamma algorithms. This makes the implementation more consistent with the overall code organization.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regarding CUDA 13.1.0, we have not tried that yet. However, even if the issue no longer appears in the current demo code, there is no guarantee that it will not reappear elsewhere. Without a clear understanding of the root cause, I would not be confident that the problem is fully resolved.
I think migrating to Eigen would require some amount of efforts from the developers, which does not seem justified in this case: the Phys3DVector structure is now tightly coupled to the Egamma implementation and is not exposed as a central/common component.

};

template <typename T>
inline constexpr Phys3DVector<T> ax(const T a, const Phys3DVector<T>& x) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not operator* ?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addressed with 2d78844

}

template <typename T>
inline constexpr Phys3DVector<T> xmy(const Phys3DVector<T>& x, const Phys3DVector<T>& y) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not operator- ?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addressed with 2d78844

@cmsbuild

Copy link
Copy Markdown
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51200/49731

@cmsbuild

cmsbuild commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51200/50849

@cmsbuild

cmsbuild commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

…ng propagators/FreeTrajectoryState to use a template type instead of defaulting to double, Renaming of ftsFromVertexToPointPortable to FreeTrajectoryState, Not performing an explicit copy when device is same as host. Updates also to the Phys3DVector to use alpaka functions.
@cmsbuild

cmsbuild commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51200/50850

@cmsbuild

cmsbuild commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

@ckoraka

ckoraka commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

-hlt

* [Heterogeneous Alpaka implementation of electron pixel seed-matching #51200 (comment)](https://github.com/cms-sw/cmssw/pull/51200#issuecomment-5265629162) remains unaddressed

* to remove it from the queue while comments are addressed

Implemented in 7b1adbb

@mmusich

mmusich commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

@cmsbuild, please test

@cmsbuild

cmsbuild commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

-1

Failed Tests: Build ClangBuild
Size: This PR adds an extra 40KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-401477/55744/summary.html
COMMIT: 471711b
CMSSW: CMSSW_20_1_X_2026-09-07-2300/el9_amd64_gcc14
Additional Tests: GPU,HLT_P2_INTEGRATION,AMD_MI300X,AMD_W7900,NVIDIA_H100,NVIDIA_L40S,NVIDIA_T4
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/51200/55744/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-401477/55744/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-401477/55744/git-merge-result

Failed Build

I found compilation error when building:

>> Compiling  src/FWCore/Utilities/src/ConvertException.cc
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_amd64_gcc14/external/gcc/14.3.1-724da22786638848892aa9ded8fcd995/bin/c++ -c -DCMS_MICRO_ARCH='x86-64-v3' -DGNU_GCC -D_GNU_SOURCE -DTBB_USE_GLIBCXX_VERSION=140301 -DTBB_SUPPRESS_DEPRECATED_MESSAGES -DTBB_PREVIEW_RESUMABLE_TASKS=1 -DTBB_PREVIEW_TASK_GROUP_EXTENSIONS=1 -DBOOST_SPIRIT_THREADSAFE -DPHOENIX_THREADSAFE -DBOOST_MATH_DISABLE_STD_FPCLASSIFY -DBOOST_UUID_RANDOM_PROVIDER_FORCE_POSIX -DBOOST_MPL_IGNORE_PARENTHESES_WARNING -DCMSSW_GIT_HASH='CMSSW_20_1_X_2026-09-07-2300' -DPROJECT_NAME='CMSSW' -DPROJECT_VERSION='CMSSW_20_1_X_2026-09-07-2300' -Isrc -Ipoison -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02958/el9_amd64_gcc14/cms/cmssw-patch/CMSSW_20_1_X_2026-09-07-2300/src -isystem/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_amd64_gcc14/external/boost/1.92.0-cf2c8871c7faab645542060b9ce21934/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_amd64_gcc14/external/libuuid/2.40-30397f5d4641a42f70e122f23e043a5a/include -isystem/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_amd64_gcc14/external/tbb/v2022.3.0-2eeb77487c539dd6a4a6662dba22a4d3/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_amd64_gcc14/external/md5/1.0.0-4c04fac6c42256d53fbdabb58aa703f3/include -O3 -pthread -pipe -Werror=main -Werror=pointer-arith -Werror=overlength-strings -Wno-vla -Werror=overflow -std=c++20 -ftree-vectorize -Werror=array-bounds -Werror=format-contains-nul -Werror=type-limits -fvisibility-inlines-hidden -fno-math-errno --param vect-max-version-for-alias-checks=50 -Xassembler --compress-debug-sections -Wno-error=array-bounds -Warray-bounds -fuse-ld=bfd -march=x86-64-v3 -felide-constructors -fmessage-length=0 -Wall -Wno-non-template-friend -Wno-long-long -Wreturn-type -Wextra -Wpessimizing-move -Wclass-memaccess -Wno-cast-function-type -Wno-unused-but-set-parameter -Wno-ignored-qualifiers -Wno-unused-parameter -Wunused -Wparentheses -Werror=return-type -Werror=unused-value -Werror=unused-result -Werror=unused-label -Werror=address -Werror=format -Werror=sign-compare -Werror=write-strings -Werror=delete-non-virtual-dtor -Werror=strict-aliasing -Werror=narrowing -Werror=unused-but-set-variable -Werror=reorder -Werror=unused-variable -Werror=conversion-null -Werror=return-local-addr -Wnon-virtual-dtor -Werror=switch -fdiagnostics-show-option -Wno-unused-local-typedefs -Wno-attributes -Wno-psabi -Wno-error=unused-variable -DBOOST_DISABLE_ASSERTS -flto=auto -fipa-icf -flto-odr-type-merging -fno-fat-lto-objects -Wodr -fPIC -MMD -MF tmp/el9_amd64_gcc14/src/FWCore/Utilities/src/FWCoreUtilities/ConvertException.cc.d src/FWCore/Utilities/src/ConvertException.cc -o tmp/el9_amd64_gcc14/src/FWCore/Utilities/src/FWCoreUtilities/ConvertException.cc.o
>> Compiling  src/FWCore/Utilities/src/Digest.cc
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_amd64_gcc14/external/gcc/14.3.1-724da22786638848892aa9ded8fcd995/bin/c++ -c -DCMS_MICRO_ARCH='x86-64-v3' -DGNU_GCC -D_GNU_SOURCE -DTBB_USE_GLIBCXX_VERSION=140301 -DTBB_SUPPRESS_DEPRECATED_MESSAGES -DTBB_PREVIEW_RESUMABLE_TASKS=1 -DTBB_PREVIEW_TASK_GROUP_EXTENSIONS=1 -DBOOST_SPIRIT_THREADSAFE -DPHOENIX_THREADSAFE -DBOOST_MATH_DISABLE_STD_FPCLASSIFY -DBOOST_UUID_RANDOM_PROVIDER_FORCE_POSIX -DBOOST_MPL_IGNORE_PARENTHESES_WARNING -DCMSSW_GIT_HASH='CMSSW_20_1_X_2026-09-07-2300' -DPROJECT_NAME='CMSSW' -DPROJECT_VERSION='CMSSW_20_1_X_2026-09-07-2300' -Isrc -Ipoison -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02958/el9_amd64_gcc14/cms/cmssw-patch/CMSSW_20_1_X_2026-09-07-2300/src -isystem/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_amd64_gcc14/external/boost/1.92.0-cf2c8871c7faab645542060b9ce21934/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_amd64_gcc14/external/libuuid/2.40-30397f5d4641a42f70e122f23e043a5a/include -isystem/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_amd64_gcc14/external/tbb/v2022.3.0-2eeb77487c539dd6a4a6662dba22a4d3/include -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el9_amd64_gcc14/external/md5/1.0.0-4c04fac6c42256d53fbdabb58aa703f3/include -O3 -pthread -pipe -Werror=main -Werror=pointer-arith -Werror=overlength-strings -Wno-vla -Werror=overflow -std=c++20 -ftree-vectorize -Werror=array-bounds -Werror=format-contains-nul -Werror=type-limits -fvisibility-inlines-hidden -fno-math-errno --param vect-max-version-for-alias-checks=50 -Xassembler --compress-debug-sections -Wno-error=array-bounds -Warray-bounds -fuse-ld=bfd -march=x86-64-v3 -felide-constructors -fmessage-length=0 -Wall -Wno-non-template-friend -Wno-long-long -Wreturn-type -Wextra -Wpessimizing-move -Wclass-memaccess -Wno-cast-function-type -Wno-unused-but-set-parameter -Wno-ignored-qualifiers -Wno-unused-parameter -Wunused -Wparentheses -Werror=return-type -Werror=unused-value -Werror=unused-result -Werror=unused-label -Werror=address -Werror=format -Werror=sign-compare -Werror=write-strings -Werror=delete-non-virtual-dtor -Werror=strict-aliasing -Werror=narrowing -Werror=unused-but-set-variable -Werror=reorder -Werror=unused-variable -Werror=conversion-null -Werror=return-local-addr -Wnon-virtual-dtor -Werror=switch -fdiagnostics-show-option -Wno-unused-local-typedefs -Wno-attributes -Wno-psabi -Wno-error=unused-variable -DBOOST_DISABLE_ASSERTS -flto=auto -fipa-icf -flto-odr-type-merging -fno-fat-lto-objects -Wodr -fPIC -MMD -MF tmp/el9_amd64_gcc14/src/FWCore/Utilities/src/FWCoreUtilities/Digest.cc.d src/FWCore/Utilities/src/Digest.cc -o tmp/el9_amd64_gcc14/src/FWCore/Utilities/src/FWCoreUtilities/Digest.cc.o
In file included from src/FWCore/Utilities/src/Digest.cc:6:
src/FWCore/Utilities/interface/Digest.h:4:10: fatal error: edm_md5.h: No such file or directory
    4 | #include 
      |          ^~~~~~~~~~~
compilation terminated.
gmake: *** [tmp/el9_amd64_gcc14/src/FWCore/Utilities/src/FWCoreUtilities/Digest.cc.o] Error 1
>> Compiling  src/FWCore/Utilities/src/EDMException.cc


Failed Clang Build

I found compilation error while trying to compile with clang. Command used:

USER_CUDA_FLAGS='--expt-relaxed-constexpr' USER_CXXFLAGS='-Wno-register -fsyntax-only' /usr/bin/time -v scram build -k -j 32 COMPILER='llvm compile'

>> Entering Package SimDataFormats/ValidationFormats
>> Entering Package SimDataFormats/Vertex
>> Entering Package Utilities/General
>> Entering Package Utilities/PPS
>> Compile sequence completed for CMSSW CMSSW_20_1_X_2026-09-07-2300
gmake: *** [There are compilation/build errors. Please see the detail log above.] Error 1
Command exited with non-zero status 1
	Command being timed: "scram build -k -j 32 COMPILER=llvm compile BUILD_LOG=yes"
	User time (seconds): 94256.98
	System time (seconds): 6770.43
	Percent of CPU this job got: 968%


@fwyzard

fwyzard commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

please test with cms-sw/cmsdist#10825

I think this is necessary until tonight's full build.

@cmsbuild

cmsbuild commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

+1

Size: This PR adds an extra 16KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-401477/55756/summary.html
COMMIT: 471711b
CMSSW: CMSSW_20_1_X_2026-09-08-1100/el9_amd64_gcc14
Additional Tests: GPU,HLT_P2_INTEGRATION,AMD_MI300X,AMD_W7900,NVIDIA_H100,NVIDIA_L40S,NVIDIA_T4
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/51200/55756/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • You potentially removed 1 lines from the logs
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 45
  • DQMHistoTests: Total histograms compared: 3592269
  • DQMHistoTests: Total failures: 24
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3592227
  • DQMHistoTests: Total skipped: 18
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 44 files compared)
  • Checked 195 log files, 163 edm output root files, 45 DQM output files
  • TriggerResults: no differences found

AMD_MI300X Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 72 differences found in the comparisons
  • DQMHistoTests: Total files compared: 7
  • DQMHistoTests: Total histograms compared: 173739
  • DQMHistoTests: Total failures: 9761
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 163978
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
  • Checked 25 log files, 20 edm output root files, 7 DQM output files
  • TriggerResults: no differences found

AMD_W7900 Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 68 differences found in the comparisons
  • DQMHistoTests: Total files compared: 7
  • DQMHistoTests: Total histograms compared: 173739
  • DQMHistoTests: Total failures: 8674
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 165065
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
  • Checked 25 log files, 20 edm output root files, 7 DQM output files
  • TriggerResults: no differences found

NVIDIA_H100 Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 85 differences found in the comparisons
  • DQMHistoTests: Total files compared: 7
  • DQMHistoTests: Total histograms compared: 173739
  • DQMHistoTests: Total failures: 8782
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 164957
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
  • Checked 25 log files, 20 edm output root files, 7 DQM output files
  • TriggerResults: no differences found

NVIDIA_L40S Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 111 differences found in the comparisons
  • DQMHistoTests: Total files compared: 7
  • DQMHistoTests: Total histograms compared: 173739
  • DQMHistoTests: Total failures: 9174
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 164565
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
  • Checked 25 log files, 20 edm output root files, 7 DQM output files
  • TriggerResults: no differences found

NVIDIA_T4 Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 78 differences found in the comparisons
  • DQMHistoTests: Total files compared: 7
  • DQMHistoTests: Total histograms compared: 173739
  • DQMHistoTests: Total failures: 8241
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 165498
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
  • Checked 25 log files, 20 edm output root files, 7 DQM output files
  • TriggerResults: no differences found

Max Memory Comparisons exceeding threshold

@cms-sw/core-l2 , I found 56 workflow step(s) with memory usage exceeding the error threshold:

Expand to see workflows ...
  • Error: Workflow 4.22_RunCosmics2011A step4 max memory diff 611.4 exceeds +/- 30.0 MiB
  • Error: Workflow 4.22_RunCosmics2011A step2 max memory diff 608.1 exceeds +/- 30.0 MiB
  • Error: Workflow 4.22_RunCosmics2011A step3 max memory diff 610.5 exceeds +/- 30.0 MiB
  • Error: Workflow 4.53_RunPhoton2012B step3 max memory diff 613.0 exceeds +/- 30.0 MiB
  • Error: Workflow 5.1_TTbarFS step1 max memory diff 614.9 exceeds +/- 30.0 MiB
  • Error: Workflow 7.3_CosmicsSPLoose2018 step3 max memory diff 608.3 exceeds +/- 30.0 MiB
  • Error: Workflow 7.3_CosmicsSPLoose2018 step4 max memory diff 610.8 exceeds +/- 30.0 MiB
  • Error: Workflow 8.0_BeamHalo step3 max memory diff 608.9 exceeds +/- 30.0 MiB
  • Error: Workflow 8.0_BeamHalo step4 max memory diff 609.9 exceeds +/- 30.0 MiB
  • Error: Workflow 9.0_Higgs200ChargedTaus step3 max memory diff 613.0 exceeds +/- 30.0 MiB
  • Error: Workflow 25.0_TTbar step3 max memory diff 613.0 exceeds +/- 30.0 MiB
  • Error: Workflow 135.4_ZEEFS_13 step1 max memory diff 614.9 exceeds +/- 30.0 MiB
  • Error: Workflow 135.4_ZEEFS_13 step4 max memory diff 577.5 exceeds +/- 30.0 MiB
  • Error: Workflow 135.4_ZEEFS_13 step3 max memory diff 615.7 exceeds +/- 30.0 MiB
  • Error: Workflow 136.731_RunSinglePh2016B step3 max memory diff 611.5 exceeds +/- 30.0 MiB
  • Error: Workflow 136.793_RunDoubleEG2017C step3 max memory diff 616.2 exceeds +/- 30.0 MiB
  • Error: Workflow 136.874_RunEGamma2018C step3 max memory diff 614.8 exceeds +/- 30.0 MiB
  • Error: Workflow 139.001_RunMinimumBias2021 step3 max memory diff 615.9 exceeds +/- 30.0 MiB
  • Error: Workflow 140.56_RunHI2018 step2 max memory diff 602.8 exceeds +/- 30.0 MiB
  • Error: Workflow 1000.0_RunMinBias2011A step3 max memory diff 605.6 exceeds +/- 30.0 MiB
  • Error: Workflow 1000.0_RunMinBias2011A step2 max memory diff 610.4 exceeds +/- 30.0 MiB
  • Error: Workflow 1001.0_RunMinBias2011A step2 max memory diff 611.4 exceeds +/- 30.0 MiB
  • Error: Workflow 1306.0_SingleMuPt1_UP15 step3 max memory diff 612.9 exceeds +/- 30.0 MiB
  • Error: Workflow 1330.0_ZMM_13 step3 max memory diff 613.0 exceeds +/- 30.0 MiB
  • Error: Workflow 1330.0_ZMM_13 step5 max memory diff 576.4 exceeds +/- 30.0 MiB
  • Error: Workflow 2022.0010001_RunTau2022D_10k step3 max memory diff 578.3 exceeds +/- 30.0 MiB
  • Error: Workflow 2023.0020001_RunJetMET02023D_10k step3 max memory diff 578.3 exceeds +/- 30.0 MiB
  • Error: Workflow 2024.0000001_RunZeroBias2024B_10k step3 max memory diff 578.3 exceeds +/- 30.0 MiB
  • Error: Workflow 2024.0010001_RunJetMET02024C_10k step3 max memory diff 578.3 exceeds +/- 30.0 MiB
  • Error: Workflow 2024.0020001_RunEGamma02024D_10k step3 max memory diff 578.3 exceeds +/- 30.0 MiB
  • Error: Workflow 2024.0030001_RunDisplacedJet2024E_10k step3 max memory diff 578.3 exceeds +/- 30.0 MiB
  • Error: Workflow 2024.0040001_RunPark2MuonLowMass02024F_10k step3 max memory diff 578.3 exceeds +/- 30.0 MiB
  • Error: Workflow 2024.0050001_RunBTagMu2024G_10k step3 max memory diff 578.3 exceeds +/- 30.0 MiB
  • Error: Workflow 2024.0060001_RunMuon02024H_10k step3 max memory diff 578.3 exceeds +/- 30.0 MiB
  • Error: Workflow 2024.0070001_RunTau2024I_10k step3 max memory diff 578.3 exceeds +/- 30.0 MiB
  • Error: Workflow 2025.0000002_RunZeroBias2025B_10k step3 max memory diff 499.5 exceeds +/- 30.0 MiB
  • Error: Workflow 2025.0010001_RunJetMET02025C_10k step3 max memory diff 497.6 exceeds +/- 30.0 MiB
  • Error: Workflow 11634.0_TTbar_14TeV+2022 step3 max memory diff 569.7 exceeds +/- 30.0 MiB
  • Error: Workflow 12434.0_TTbar_14TeV+2023 step3 max memory diff 569.7 exceeds +/- 30.0 MiB
  • Error: Workflow 12834.0_TTbar_14TeV+2024 step3 max memory diff 569.7 exceeds +/- 30.0 MiB
  • Error: Workflow 12846.0_ZEE_14+2024 step3 max memory diff 569.7 exceeds +/- 30.0 MiB
  • Error: Workflow 13234.0_TTbar_14TeV+2022FS step2 max memory diff 569.8 exceeds +/- 30.0 MiB
  • Error: Workflow 14034.0_TTbar_14TeV+2023FS step2 max memory diff 569.8 exceeds +/- 30.0 MiB
  • Error: Workflow 16834.0_TTbar_14TeV+2025 step3 max memory diff 569.7 exceeds +/- 30.0 MiB
  • Error: Workflow 17034.96_TTbar_14TeV+2025PU_HybridPUS1S2 step4 max memory diff 569.7 exceeds +/- 30.0 MiB
  • Error: Workflow 18434.0_TTbar_14TeV+2026 step3 max memory diff 497.3 exceeds +/- 30.0 MiB
  • Error: Workflow 37634.0_TTbar_14TeV+Run4D127 step5 max memory diff 495.2 exceeds +/- 30.0 MiB
  • Error: Workflow 37634.0_TTbar_14TeV+Run4D127 step3 max memory diff 488.3 exceeds +/- 30.0 MiB
  • Error: Workflow 37634.75_TTbar_14TeV+Run4D127_HLT75e33Timing step2 max memory diff 488.5 exceeds +/- 30.0 MiB
  • Error: Workflow 37634.911_TTbar_14TeV+Run4D127_DD4hep step5 max memory diff 495.2 exceeds +/- 30.0 MiB
  • Error: Workflow 37634.911_TTbar_14TeV+Run4D127_DD4hep step3 max memory diff 488.3 exceeds +/- 30.0 MiB
  • Error: Workflow 37696.0_CloseByPGun_CE_E_Front_120um+Run4D127 step3 max memory diff 488.3 exceeds +/- 30.0 MiB
  • Error: Workflow 37696.0_CloseByPGun_CE_E_Front_120um+Run4D127 step5 max memory diff 495.2 exceeds +/- 30.0 MiB
  • Error: Workflow 37700.0_CloseByPGun_CE_H_Coarse_Scint+Run4D127 step3 max memory diff 488.4 exceeds +/- 30.0 MiB
  • Error: Workflow 37700.0_CloseByPGun_CE_H_Coarse_Scint+Run4D127 step5 max memory diff 495.2 exceeds +/- 30.0 MiB
  • Error: Workflow 37834.0_TTbar_14TeV+Run4D127PU step3 max memory diff 488.2 exceeds +/- 30.0 MiB

Max Memory Comparisons exceeding threshold NVIDIA_H100

@cms-sw/core-l2 , I found 2 workflow step(s) with memory usage exceeding the error threshold:

Expand to see workflows ...
  • Error: Workflow 37634.7503_TTbar_14TeV+Run4D127_HLTHeterogeneousValid step2 max memory diff 271.1 exceeds +/- 30.0 MiB
  • Error: Workflow 37634.751_TTbar_14TeV+Run4D127_HLT75e33TimingAlpaka step2 max memory diff 488.3 exceeds +/- 30.0 MiB

Max Memory Comparisons exceeding threshold NVIDIA_L40S

@cms-sw/core-l2 , I found 2 workflow step(s) with memory usage exceeding the error threshold:

Expand to see workflows ...
  • Error: Workflow 37634.7503_TTbar_14TeV+Run4D127_HLTHeterogeneousValid step2 max memory diff 678.8 exceeds +/- 30.0 MiB
  • Error: Workflow 37634.751_TTbar_14TeV+Run4D127_HLT75e33TimingAlpaka step2 max memory diff 487.7 exceeds +/- 30.0 MiB

Max Memory Comparisons exceeding threshold NVIDIA_T4

@cms-sw/core-l2 , I found 2 workflow step(s) with memory usage exceeding the error threshold:

Expand to see workflows ...
  • Error: Workflow 37634.7503_TTbar_14TeV+Run4D127_HLTHeterogeneousValid step2 max memory diff 478.9 exceeds +/- 30.0 MiB
  • Error: Workflow 37634.751_TTbar_14TeV+Run4D127_HLT75e33TimingAlpaka step2 max memory diff 488.3 exceeds +/- 30.0 MiB

… add deduction guide in EleRelPointPairPortable
@cmsbuild

Copy link
Copy Markdown
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51200/50903

@cmsbuild

Copy link
Copy Markdown
Contributor

@makortel

Copy link
Copy Markdown
Contributor

@cmsbuild, please test

@makortel

Copy link
Copy Markdown
Contributor

@smuzaffar Tests seem to be stuck? (they should be restarted anyhow to avoid the unrelated unit test failures)

@smuzaffar

Copy link
Copy Markdown
Contributor

@cmsbuild, please test

}

// Print per-event doublet rejection summary
edm::LogPrint("ElectronSeedConverter") << "[ElectronSeedConverter] matched=" << nMatched << " doublets=" << nDoublets

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be a LogDebug to avoid flooding the HLT output stream?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.