PerfTools/Allocmon add circles plots for ExternalWork and Transform modules by gartung · Pull Request #51608 · cms-sw/cmssw

gartung · 2026-07-31T23:24:33Z

No description provided.

cmsbuild · 2026-07-31T23:24:58Z

cms-bot internal usage

cmsbuild · 2026-07-31T23:26:55Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51608/50404

cmsbuild · 2026-07-31T23:27:16Z

A new Pull Request was created by @gartung for master.

It involves the following packages:

PerfTools/AllocMonitor (core)

@Dr15Jones, @cmsbuild, @makortel, @smuzaffar can you please review it and eventually sign? Thanks.
@felicepantaleo, @makortel this is something you requested to watch as well.
@ftenchini, @mandrenguyen, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

gartung · 2026-07-31T23:27:34Z

enable profiling

gartung · 2026-07-31T23:27:50Z

please test

cmsbuild · 2026-08-01T01:21:43Z

+1

Size: This PR adds an extra 36KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-1504a9/55036/summary.html
COMMIT: a5c7104
CMSSW: CMSSW_20_1_X_2026-07-31-1100/el9_amd64_gcc13
Additional Tests: PROFILING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/51608/55036/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 45
DQMHistoTests: Total histograms compared: 3588680
DQMHistoTests: Total failures: 4
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3588658
DQMHistoTests: Total skipped: 18
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 44 files compared)
Checked 195 log files, 163 edm output root files, 45 DQM output files
TriggerResults: no differences found

makortel · 2026-08-03T18:50:22Z

test parameters:

enable = profiling
workflows_profiling = 18434.0

makortel · 2026-08-03T18:50:28Z

@cmsbuild, please test

cmsbuild · 2026-08-03T20:39:14Z

+1

Size: This PR adds an extra 16KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-1504a9/55059/summary.html
COMMIT: a5c7104
CMSSW: CMSSW_20_1_X_2026-08-03-1100/el9_amd64_gcc13
Additional Tests: PROFILING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/51608/55059/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 1 lines to the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 45
DQMHistoTests: Total histograms compared: 3588680
DQMHistoTests: Total failures: 13
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3588649
DQMHistoTests: Total skipped: 18
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 44 files compared)
Checked 195 log files, 163 edm output root files, 45 DQM output files
TriggerResults: no differences found

gartung · 2026-08-10T21:03:39Z

please test

cmsbuild · 2026-08-10T22:14:44Z

-1

Failed Tests: UnitTests RelVals
Size: This PR adds an extra 36KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-1504a9/55234/summary.html
COMMIT: 179dec2
CMSSW: CMSSW_20_1_X_2026-08-10-1100/el9_amd64_gcc13
Additional Tests: PROFILING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/51608/55234/install.sh to create a dev area with all the needed externals and cmssw changes.

Failed Unit Tests

I found 1 errors in the following unit tests:

---> test TestPerfToolsModuleAllocMonitor had ERRORS

Failed RelVals

37834.0

----- Begin Fatal Exception 10-Aug-2026 23:38:05 CEST-----------------------
An exception of category 'NoSecondaryFiles' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=MixingModule label='mix'
Exception Message:
RootEmbeddedFileSequence no input files specified for secondary input source.
----- End Fatal Exception -------------------------------------------------

cmsbuild · 2026-08-11T15:06:52Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51608/50548

cmsbuild · 2026-08-11T15:07:14Z

Pull request #51608 was updated. @Dr15Jones, @cmsbuild, @makortel, @smuzaffar can you please check and sign again.

gartung · 2026-08-11T16:34:24Z

please test

cmsbuild · 2026-08-11T17:57:57Z

+1

Size: This PR adds an extra 24KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-1504a9/55252/summary.html
COMMIT: f0dda92
CMSSW: CMSSW_20_1_X_2026-08-11-1100/el9_amd64_gcc13
Additional Tests: PROFILING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/51608/55252/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 2 lines to the logs
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 44
DQMHistoTests: Total histograms compared: 3433258
DQMHistoTests: Total failures: 57
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3433183
DQMHistoTests: Total skipped: 18
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 43 files compared)
Checked 191 log files, 159 edm output root files, 44 DQM output files
TriggerResults: no differences found

cmsbuild · 2026-08-11T19:07:41Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51608/50552

cmsbuild · 2026-08-11T19:08:00Z

Pull request #51608 was updated. @Dr15Jones, @cmsbuild, @makortel, @smuzaffar can you please check and sign again.

cmsbuild · 2026-08-11T19:33:49Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51608/50553

cmsbuild · 2026-08-11T19:34:16Z

Pull request #51608 was updated. @Dr15Jones, @cmsbuild, @makortel, @smuzaffar can you please check and sign again.

cmsbuild · 2026-08-11T19:46:50Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51608/50554

cmsbuild · 2026-08-11T19:47:16Z

Pull request #51608 was updated. @Dr15Jones, @cmsbuild, @makortel, @smuzaffar can you please check and sign again.

cmsbuild · 2026-08-11T20:01:41Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51608/50555

cmsbuild · 2026-08-11T20:02:01Z

Pull request #51608 was updated. @Dr15Jones, @cmsbuild, @makortel, @smuzaffar can you please check and sign again.

cmsbuild · 2026-08-11T20:10:10Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51608/50556

cmsbuild · 2026-08-11T20:10:41Z

Pull request #51608 was updated. @Dr15Jones, @cmsbuild, @makortel, @smuzaffar can you please check and sign again.

cmsbuild · 2026-08-11T20:30:01Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51608/50557

cmsbuild · 2026-08-11T20:30:23Z

Pull request #51608 was updated. @Dr15Jones, @cmsbuild, @makortel, @smuzaffar can you please check and sign again.

gartung · 2026-08-11T21:08:40Z

please test

cmsbuild · 2026-08-11T22:34:04Z

+1

Size: This PR adds an extra 36KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-1504a9/55259/summary.html
COMMIT: 5159954
CMSSW: CMSSW_20_1_X_2026-08-11-1100/el9_amd64_gcc13
Additional Tests: PROFILING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/51608/55259/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

@felicepantaleo RecoEgamma: drop the TICLGeom ES producers from the HGC egamma PAT task #51668

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-1504a9/55259/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-1504a9/55259/git-merge-result

Comparison Summary

Summary:

You potentially added 1 lines to the logs
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 44
DQMHistoTests: Total histograms compared: 3433258
DQMHistoTests: Total failures: 0
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3433240
DQMHistoTests: Total skipped: 18
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 43 files compared)
Checked 191 log files, 159 edm output root files, 44 DQM output files
TriggerResults: no differences found

makortel · 2026-08-12T18:36:37Z

Looks good. Could you squash the commits again to just one with a title of e.g. "Add circles plots for ExternalWork and Transform modules"? To me the details of the code review (as in the current commits) are not important to be kept and could be dropped from the commit message.

cmsbuild · 2026-08-12T20:33:35Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51608/50571

cmsbuild · 2026-08-12T20:34:05Z

Pull request #51608 was updated. @Dr15Jones, @cmsbuild, @makortel, @smuzaffar can you please check and sign again.

makortel · 2026-08-12T20:38:41Z

@cmsbuild, please test

cmsbuild · 2026-08-12T22:01:09Z

+1

Size: This PR adds an extra 32KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-1504a9/55293/summary.html
COMMIT: 032e3bc
CMSSW: CMSSW_20_1_X_2026-08-12-1100/el9_amd64_gcc13
Additional Tests: PROFILING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/51608/55293/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 44
DQMHistoTests: Total histograms compared: 3435544
DQMHistoTests: Total failures: 0
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3435526
DQMHistoTests: Total skipped: 18
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 43 files compared)
Checked 191 log files, 159 edm output root files, 44 DQM output files
TriggerResults: no differences found

makortel · 2026-08-13T14:56:16Z

+core

cmsbuild · 2026-08-13T14:56:43Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @sextonkennedy, @ftenchini, @mandrenguyen (and backports should be raised in the release meeting by the corresponding L2)

mandrenguyen · 2026-08-14T06:55:28Z

+1

cmsbuild added this to the CMSSW_20_1_X milestone Jul 31, 2026

cmsbuild added core-pending pending-signatures tests-pending orp-pending code-checks-pending labels Jul 31, 2026

gartung linked an issue Jul 31, 2026 that may be closed by this pull request

Extend ModuleAllocModule circles plots to cover ExternalWork and Transformer modules cms-sw/framework-team#2352

Closed

cmsbuild added code-checks-approved and removed code-checks-pending labels Jul 31, 2026

cmsbuild added tests-started and removed tests-pending labels Jul 31, 2026

cmsbuild added tests-approved and removed tests-started labels Aug 1, 2026

cmsbuild added tests-started and removed tests-approved labels Aug 3, 2026

makortel reviewed Aug 3, 2026

View reviewed changes

Comment thread PerfTools/AllocMonitor/scripts/edmModuleAllocMonitorAnalyze.py Outdated

Comment thread PerfTools/AllocMonitor/scripts/edmModuleAllocMonitorAnalyze.py Outdated

Comment thread PerfTools/AllocMonitor/scripts/edmModuleAllocJsonToCircles.py Outdated

cmsbuild added tests-approved tests-pending and removed tests-started tests-approved code-checks-approved labels Aug 3, 2026

makortel reviewed Aug 11, 2026

View reviewed changes

Add circles plots for ExternalWork and Transform modules

032e3bc

Uh oh!

Conversation

gartung commented Jul 31, 2026

Uh oh!

cmsbuild commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cmsbuild commented Jul 31, 2026

Uh oh!

cmsbuild commented Jul 31, 2026

Uh oh!

gartung commented Jul 31, 2026

Uh oh!

gartung commented Jul 31, 2026

Uh oh!

cmsbuild commented Aug 1, 2026

Comparison Summary

Uh oh!

makortel commented Aug 3, 2026

Uh oh!

makortel commented Aug 3, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cmsbuild commented Aug 3, 2026

Comparison Summary

Uh oh!

gartung commented Aug 10, 2026

Uh oh!

cmsbuild commented Aug 10, 2026

Failed Unit Tests

Failed RelVals

Uh oh!

cmsbuild commented Aug 11, 2026

Uh oh!

cmsbuild commented Aug 11, 2026

Uh oh!

gartung commented Aug 11, 2026

Uh oh!

cmsbuild commented Aug 11, 2026

Comparison Summary

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cmsbuild commented Aug 11, 2026

Uh oh!

cmsbuild commented Aug 11, 2026

Uh oh!

cmsbuild commented Aug 11, 2026

Uh oh!

cmsbuild commented Aug 11, 2026

Uh oh!

cmsbuild commented Aug 11, 2026

Uh oh!

cmsbuild commented Aug 11, 2026

Uh oh!

cmsbuild commented Aug 11, 2026

Uh oh!

cmsbuild commented Aug 11, 2026

Uh oh!

cmsbuild commented Aug 11, 2026

Uh oh!

cmsbuild commented Aug 11, 2026

Uh oh!

cmsbuild commented Aug 11, 2026

Uh oh!

cmsbuild commented Aug 11, 2026

Uh oh!

gartung commented Aug 11, 2026

Uh oh!

cmsbuild commented Aug 11, 2026

Comparison Summary

Uh oh!

makortel commented Aug 12, 2026

cmsbuild commented Jul 31, 2026 •

edited

Loading