HCAL code review, continued #567

mariadalfonso · 2020-11-09T13:50:02Z

added config/analyzer used to make the scatter plot for presentation at RECO meeting 30 oct
https://indico.cern.ch/event/968978/#11-patatrack-hcal-reconstructi page 6

move common function in MultifitComputations.h

fwyzard · 2020-11-09T18:30:34Z

@mariadalfonso

if the analyzer is to be integrated upstream, could you move it to RecoLocalCalo/HcalRecProducers/ ?
is the chi2 function also used in ECAL code? should we simplify it somewhere ?

…nto hcalREVIEWcontinue

fwyzard · 2020-11-15T20:57:04Z

Validation summary

Reference release CMSSW_11_2_0_pre8 at 3c9bd1b
Development branch cms-patatrack/CMSSW_11_2_X_Patatrack at f7775e7
Testing branch cms-patatrack/CMSSW_11_2_X_Patatrack at f7775e7 with PRs:

HCAL code review, continued #567 at d271a4f

Validation plots

/RelValTTbar_14TeV/CMSSW_11_2_0_pre7-PU_112X_mcRun3_2021_realistic_v8-v1/GEN-SIM-DIGI-RAW

tracking validation plots and summary for workflow 11634.5
tracking validation plots and summary for workflow 11634.501
tracking validation plots and summary for workflow 11634.502
⚠️ tracking validation plots and summary for workflow 11634.505 are missing
tracking validation plots and summary for workflow 11634.506

/RelValZMM_14/CMSSW_11_2_0_pre7-112X_mcRun3_2021_realistic_v8-v2/GEN-SIM-DIGI-RAW

tracking validation plots and summary for workflow 11634.5
tracking validation plots and summary for workflow 11634.501
tracking validation plots and summary for workflow 11634.502
tracking validation plots and summary for workflow 11634.505
tracking validation plots and summary for workflow 11634.506

/RelValZEE_14/CMSSW_11_2_0_pre7-112X_mcRun3_2021_realistic_v8-v1/GEN-SIM-DIGI-RAW

tracking validation plots and summary for workflow 11634.5
tracking validation plots and summary for workflow 11634.501
tracking validation plots and summary for workflow 11634.502
tracking validation plots and summary for workflow 11634.505
tracking validation plots and summary for workflow 11634.506

Validation plots (CPU vs GPU)

/RelValTTbar_14TeV/CMSSW_11_2_0_pre7-PU_112X_mcRun3_2021_realistic_v8-v1/GEN-SIM-DIGI-RAW

⚠️ tracking validation plots and summary for workflows 11634.502 and 11634.501 are missing
⚠️ tracking validation plots and summary for workflows 11634.506 and 11634.505 are missing

/RelValZMM_14/CMSSW_11_2_0_pre7-112X_mcRun3_2021_realistic_v8-v2/GEN-SIM-DIGI-RAW

tracking validation plots and summary for workflows 11634.502 and 11634.501
tracking validation plots and summary for workflows 11634.506 and 11634.505

/RelValZEE_14/CMSSW_11_2_0_pre7-112X_mcRun3_2021_realistic_v8-v1/GEN-SIM-DIGI-RAW

tracking validation plots and summary for workflows 11634.502 and 11634.501
tracking validation plots and summary for workflows 11634.506 and 11634.505

Throughput plots

/EphemeralHLTPhysics1/Run2018D-v1/RAW run=323775 lumi=53

logs and `nvprof`/`nvvp` profiles

/RelValTTbar_14TeV/CMSSW_11_2_0_pre7-PU_112X_mcRun3_2021_realistic_v8-v1/GEN-SIM-DIGI-RAW

reference release, workflow 11634.5
- ✔️ step3.py: log
development release, workflow 11634.5
- ✔️ step3.py: log
development release, workflow 11634.501
- ❌ step3.py: log
development release, workflow 11634.502
- ✔️ step3.py: log
- ✔️ profile.py: log
- ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
- ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
- ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
development release, workflow 11634.505
- ❌ step3.py: log
development release, workflow 11634.506
- ✔️ step3.py: log
- ✔️ profile.py: log
- ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
- ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
- ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
development release, workflow 11634.511
- ✔️ step3.py: log
development release, workflow 11634.512
- ✔️ step3.py: log
- ✔️ profile.py: log
- ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
- ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
- ❌ cuda-memcheck --tool synccheck (report, log) found no CUDA-MEMCHECK results
development release, workflow 11634.521
- ✔️ step3.py: log
development release, workflow 11634.522
- ✔️ step3.py: log
- ✔️ profile.py: log
- ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
- ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
- ❌ cuda-memcheck --tool synccheck (report, log) found no CUDA-MEMCHECK results
development release, workflow 136.885502
development release, workflow 136.885512
development release, workflow 136.885522
testing release, workflow 11634.5
- ✔️ step3.py: log
testing release, workflow 11634.501
- ✔️ step3.py: log
testing release, workflow 11634.502
- ✔️ step3.py: log
- ✔️ profile.py: log
- ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
- ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
- ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
testing release, workflow 11634.505
- ❌ step3.py: log
testing release, workflow 11634.506
- ✔️ step3.py: log
- ✔️ profile.py: log
- ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
- ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
- ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
testing release, workflow 11634.511
- ✔️ step3.py: log
testing release, workflow 11634.512
- ✔️ step3.py: log
- ✔️ profile.py: log
- ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
- ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
- ❌ cuda-memcheck --tool synccheck (report, log) found no CUDA-MEMCHECK results
testing release, workflow 11634.521
- ✔️ step3.py: log
testing release, workflow 11634.522
- ✔️ step3.py: log
- ✔️ profile.py: log
- ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
- ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
- ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
testing release, workflow 136.885502
testing release, workflow 136.885512
testing release, workflow 136.885522

/RelValZMM_14/CMSSW_11_2_0_pre7-112X_mcRun3_2021_realistic_v8-v2/GEN-SIM-DIGI-RAW

reference release, workflow 11634.5
- ✔️ step3.py: log
development release, workflow 11634.5
- ✔️ step3.py: log
development release, workflow 11634.501
- ✔️ step3.py: log
development release, workflow 11634.502
- ✔️ step3.py: log
- ✔️ profile.py: log
- ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
- ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
- ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
development release, workflow 11634.505
- ✔️ step3.py: log
development release, workflow 11634.506
- ✔️ step3.py: log
- ✔️ profile.py: log
- ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
- ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
- ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
development release, workflow 11634.511
- ✔️ step3.py: log
development release, workflow 11634.512
- ✔️ step3.py: log
- ✔️ profile.py: log
- ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
- ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
- ❌ cuda-memcheck --tool synccheck (report, log) found no CUDA-MEMCHECK results
development release, workflow 11634.521
- ✔️ step3.py: log
development release, workflow 11634.522
- ✔️ step3.py: log
- ✔️ profile.py: log
- ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
- ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
- ❌ cuda-memcheck --tool synccheck (report, log) found no CUDA-MEMCHECK results
development release, workflow 136.885502
development release, workflow 136.885512
development release, workflow 136.885522
testing release, workflow 11634.5
- ✔️ step3.py: log
testing release, workflow 11634.501
- ✔️ step3.py: log
testing release, workflow 11634.502
- ✔️ step3.py: log
- ✔️ profile.py: log
- ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
- ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
- ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
testing release, workflow 11634.505
- ✔️ step3.py: log
testing release, workflow 11634.506
- ✔️ step3.py: log
- ✔️ profile.py: log
- ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
- ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
- ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
testing release, workflow 11634.511
- ✔️ step3.py: log
testing release, workflow 11634.512
- ✔️ step3.py: log
- ✔️ profile.py: log
- ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
- ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
- ❌ cuda-memcheck --tool synccheck (report, log) found no CUDA-MEMCHECK results
testing release, workflow 11634.521
- ✔️ step3.py: log
testing release, workflow 11634.522
- ✔️ step3.py: log
- ✔️ profile.py: log
- ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
- ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
- ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
testing release, workflow 136.885502
testing release, workflow 136.885512
testing release, workflow 136.885522

/RelValZEE_14/CMSSW_11_2_0_pre7-112X_mcRun3_2021_realistic_v8-v1/GEN-SIM-DIGI-RAW

reference release, workflow 11634.5
- ✔️ step3.py: log
development release, workflow 11634.5
- ✔️ step3.py: log
development release, workflow 11634.501
- ✔️ step3.py: log
development release, workflow 11634.502
- ✔️ step3.py: log
- ✔️ profile.py: log
- ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
- ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
- ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
development release, workflow 11634.505
- ✔️ step3.py: log
development release, workflow 11634.506
- ✔️ step3.py: log
- ✔️ profile.py: log
- ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
- ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
- ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
development release, workflow 11634.511
- ✔️ step3.py: log
development release, workflow 11634.512
- ✔️ step3.py: log
- ✔️ profile.py: log
- ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
- ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
- ❌ cuda-memcheck --tool synccheck (report, log) found no CUDA-MEMCHECK results
development release, workflow 11634.521
- ✔️ step3.py: log
development release, workflow 11634.522
- ✔️ step3.py: log
- ✔️ profile.py: log
- ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
- ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
- ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
development release, workflow 136.885502
development release, workflow 136.885512
development release, workflow 136.885522
testing release, workflow 11634.5
- ✔️ step3.py: log
testing release, workflow 11634.501
- ✔️ step3.py: log
testing release, workflow 11634.502
- ✔️ step3.py: log
- ✔️ profile.py: log
- ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
- ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
- ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
testing release, workflow 11634.505
- ✔️ step3.py: log
testing release, workflow 11634.506
- ✔️ step3.py: log
- ✔️ profile.py: log
- ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
- ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
- ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
testing release, workflow 11634.511
- ✔️ step3.py: log
testing release, workflow 11634.512
- ✔️ step3.py: log
- ✔️ profile.py: log
- ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
- ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
- ❌ cuda-memcheck --tool synccheck (report, log) found no CUDA-MEMCHECK results
testing release, workflow 11634.521
- ✔️ step3.py: log
testing release, workflow 11634.522
- ✔️ step3.py: log
- ✔️ profile.py: log
- ✔️ cuda-memcheck --tool initcheck (report, log) did not find any errors
- ✔️ cuda-memcheck --tool memcheck --leak-check full --report-api-errors all (report, log) did not find any errors
- ✔️ cuda-memcheck --tool synccheck (report, log) did not find any errors
testing release, workflow 136.885502
testing release, workflow 136.885512
testing release, workflow 136.885522

Logs

The full log is available at https://patatrack.web.cern.ch/patatrack/validation/pulls/bb7469747f2cdd46625e2141beb05ce625eab0a8/log .

fwyzard · 2020-11-16T08:08:04Z

Add an analyser for the GPU vs CPU comparison of the HCAL results.
Factor out the chi2 computation of the ECAL and HCAL code and move it to MultifitComputations.

fwyzard · 2020-11-16T08:24:45Z

Merged by hand, creating two commits:

b77f4d6 Refactor ECAL and HCAL chi2 code (HCAL code review, continued #567)
4b0cb83 Add HCAL analyser for GPU vs CPU comparison (HCAL code review, continued #567)

Factor out the chi2 computation from the ECAL multifit and HCAL MAHI code, and move it to MultifitComputations.

mariadalfonso added 2 commits November 9, 2020 14:43

add config/analyzer to run both cpu and gpu

49f91e5

generalize chi2 function (common ECAL,HCAL)

cfa2046

fwyzard mentioned this pull request Nov 9, 2020

Patatrack integration - HCAL local reconstruction (8/N) cms-sw/cmssw#31720

Merged

generalize chi2 to ECAL

746d7c2

fwyzard mentioned this pull request Nov 10, 2020

Address more HCAL review comments #568

Merged

fwyzard added HCAL HCAL-related developments ECAL ECAL-related developments labels Nov 10, 2020

fwyzard changed the base branch from master to CMSSW_11_2_X_Patatrack November 10, 2020 22:15

fwyzard changed the title ~~Hcal review continue~~ HCAL code review, continued Nov 10, 2020

fwyzard force-pushed the CMSSW_11_2_X_Patatrack branch from e67e333 to 7447a4a Compare November 10, 2020 22:18

mariadalfonso added 3 commits November 11, 2020 20:35

factorize a bit

c12f118

Merge remote-tracking branch 'cms-patatrack/CMSSW_11_2_X_Patatrack' i…

27b0b28

…nto hcalREVIEWcontinue

fix compilation, address Egpu=0

d271a4f

fwyzard changed the base branch from CMSSW_11_2_X_Patatrack to master November 16, 2020 08:08

fwyzard changed the base branch from master to CMSSW_11_2_X_Patatrack November 16, 2020 08:08

fwyzard closed this Nov 16, 2020

fwyzard added the done label Nov 16, 2020

fwyzard pushed a commit that referenced this pull request Nov 16, 2020

Refactor ECAL and HCAL chi2 code (#567)

b77f4d6

Factor out the chi2 computation from the ECAL multifit and HCAL MAHI code, and move it to MultifitComputations.

fwyzard pushed a commit that referenced this pull request Nov 16, 2020

Add HCAL analyser for GPU vs CPU comparison (#567)

4b0cb83

fwyzard pushed a commit that referenced this pull request Nov 16, 2020

Refactor ECAL and HCAL chi2 code (#567)

3c8937f

Factor out the chi2 computation from the ECAL multifit and HCAL MAHI code, and move it to MultifitComputations.

fwyzard pushed a commit that referenced this pull request Nov 16, 2020

Refactor ECAL and HCAL chi2 code (#567)

e16944c

Factor out the chi2 computation from the ECAL multifit and HCAL MAHI code, and move it to MultifitComputations.

fwyzard mentioned this pull request Nov 16, 2020

Patatrack integration - ECAL local reconstruction (7/N) cms-sw/cmssw#31719

Merged

fwyzard added a commit that referenced this pull request Nov 26, 2020

Refactor ECAL and HCAL chi2 code (#567)

54381b0

Factor out the chi2 computation from the ECAL multifit and HCAL MAHI code, and move it to MultifitComputations.

fwyzard added a commit that referenced this pull request Nov 26, 2020

Refactor ECAL and HCAL chi2 code (#567)

bbd4cca

Factor out the chi2 computation from the ECAL multifit and HCAL MAHI code, and move it to MultifitComputations.

fwyzard pushed a commit that referenced this pull request Dec 25, 2020

Refactor ECAL and HCAL chi2 code (#567)

f19564c

Factor out the chi2 computation from the ECAL multifit and HCAL MAHI code, and move it to MultifitComputations.

fwyzard pushed a commit that referenced this pull request Dec 29, 2020

Refactor ECAL and HCAL chi2 code (#567)

16b46a5

Factor out the chi2 computation from the ECAL multifit and HCAL MAHI code, and move it to MultifitComputations.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HCAL code review, continued #567

HCAL code review, continued #567

mariadalfonso commented Nov 9, 2020

fwyzard commented Nov 9, 2020

fwyzard commented Nov 15, 2020 •

edited

Loading

fwyzard commented Nov 16, 2020

fwyzard commented Nov 16, 2020

HCAL code review, continued #567

HCAL code review, continued #567

Conversation

mariadalfonso commented Nov 9, 2020

fwyzard commented Nov 9, 2020

fwyzard commented Nov 15, 2020 • edited Loading

Validation summary

Validation plots

/RelValTTbar_14TeV/CMSSW_11_2_0_pre7-PU_112X_mcRun3_2021_realistic_v8-v1/GEN-SIM-DIGI-RAW

/RelValZMM_14/CMSSW_11_2_0_pre7-112X_mcRun3_2021_realistic_v8-v2/GEN-SIM-DIGI-RAW

/RelValZEE_14/CMSSW_11_2_0_pre7-112X_mcRun3_2021_realistic_v8-v1/GEN-SIM-DIGI-RAW

Validation plots (CPU vs GPU)

/RelValTTbar_14TeV/CMSSW_11_2_0_pre7-PU_112X_mcRun3_2021_realistic_v8-v1/GEN-SIM-DIGI-RAW

/RelValZMM_14/CMSSW_11_2_0_pre7-112X_mcRun3_2021_realistic_v8-v2/GEN-SIM-DIGI-RAW

/RelValZEE_14/CMSSW_11_2_0_pre7-112X_mcRun3_2021_realistic_v8-v1/GEN-SIM-DIGI-RAW

Throughput plots

/EphemeralHLTPhysics1/Run2018D-v1/RAW run=323775 lumi=53

logs and nvprof/nvvp profiles

/RelValTTbar_14TeV/CMSSW_11_2_0_pre7-PU_112X_mcRun3_2021_realistic_v8-v1/GEN-SIM-DIGI-RAW

/RelValZMM_14/CMSSW_11_2_0_pre7-112X_mcRun3_2021_realistic_v8-v2/GEN-SIM-DIGI-RAW

/RelValZEE_14/CMSSW_11_2_0_pre7-112X_mcRun3_2021_realistic_v8-v1/GEN-SIM-DIGI-RAW

Logs

fwyzard commented Nov 16, 2020

fwyzard commented Nov 16, 2020

fwyzard commented Nov 15, 2020 •

edited

Loading

logs and `nvprof`/`nvvp` profiles