HDF5 interface for LHE-HDF5 event files

[smrenna]

PR description:

This is a copy of LHESource to handle event generator files in the HDF5 format.
This uses the header-only HighFive library for handling the HDF5 files and information.

PR validation:

I have used the two tests in the LHEInterface tests directory: testReader_cfg.py, which reads the hdf5 file and makes a root edm file and dumpLHE_cfg.py that prints out the stored information.

I can put some hdf5 files on lxplus for testing.

Before submitting your pull requests, make sure you followed this checklist:

• verify that the PR is really intended for the chosen branch
• verify that changes follow CMS Naming, Coding, And Style Rules
• verify that the PR passes the basic test procedure suggested in the CMSSW PR instructions

[cmsbuild]

The code-checks are being triggered in jenkins.

[smrenna]

@bendavid might be interested in this

[cmsbuild]

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-30312/16282

• This PR adds an extra 60KB to repository

• Found files with invalid states:

  • GeneratorInterface/LHEInterface/src/HighFive/.gitmodules:
    • Added: 8104a8b
    • Deleted: 5a9447e
  • GeneratorInterface/LHEInterface/src/HighFive/.travis.yml:
    • Added: 8104a8b
    • Deleted: 5a9447e
  • GeneratorInterface/LHEInterface/src/HighFive/.gitignore:
    • Added: 8104a8b
    • Deleted: 5a9447e
  • GeneratorInterface/LHEInterface/src/HighFive/.clang-format:
    • Added: 8104a8b
    • Deleted: 5a9447e

Code check has found code style and quality issues which could be resolved by applying following patch(s)

• code-checks:
  https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-30312/16282/code-checks.patch
  e.g. curl https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-30312/16282/code-checks.patch | patch -p1
  You can also run scram build code-checks to apply code checks directly
• code-format:
  https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-30312/16282/code-format.patch
  e.g. curl https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-30312/16282/code-format.patch | patch -p1
  You can also run scram build code-format to apply code format directly

[cmsbuild]

The code-checks are being triggered in jenkins.

[cmsbuild]

The code-checks are being triggered in jenkins.

[cmsbuild]

-code-checks

ERROR: Build errors found during clang-tidy run.

GeneratorInterface/LHEInterface/plugins/LH5Source.cc:38:39: error: too few arguments to function call, single argument 'iCatalog' was not specified [clang-diagnostic-error]
      reader_(new LH5Reader(fileNames(),
                                      ^
/cvmfs/cms-ib.cern.ch/nweek-02633/slc7_amd64_gcc820/cms/cmssw/CMSSW_11_2_X_2020-06-19-1100/src/FWCore/Sources/interface/FromFiles.h:22:5: note: 'fileNames' declared here
--
gmake: *** [config/SCRAM/GMake/Makefile.coderules:128: code-checks] Error 2
gmake: *** [There are compilation/build errors. Please see the detail log above.] Error 2

[smrenna]

I need some help here. This is a direct copy of the structure of LHEReader, so why does one trigger a problem, but not the other?

Couldn't find the definition of iCatalog in CMSSW lxr.

[cmsbuild]

The code-checks are being triggered in jenkins.

[cmsbuild]

-code-checks

ERROR: Build errors found during clang-tidy run.

GeneratorInterface/LHEInterface/plugins/LH5Source.cc:38:39: error: too few arguments to function call, single argument 'iCatalog' was not specified [clang-diagnostic-error]
      reader_(new LH5Reader(fileNames(),params.getUntrackedParameter<unsigned int>("skipEvents", 0))),
                                      ^
/cvmfs/cms-ib.cern.ch/nweek-02633/slc7_amd64_gcc820/cms/cmssw/CMSSW_11_2_X_2020-06-19-1100/src/FWCore/Sources/interface/FromFiles.h:22:5: note: 'fileNames' declared here
--
gmake: *** [config/SCRAM/GMake/Makefile.coderules:128: code-checks] Error 2
gmake: *** [There are compilation/build errors. Please see the detail log above.] Error 2

[cmsbuild]

The code-checks are being triggered in jenkins.

[cmsbuild]

-code-checks

ERROR: Build errors found during clang-tidy run.

GeneratorInterface/LHEInterface/plugins/LHESource.cc:35:39: error: too few arguments to function call, single argument 'iCatalog' was not specified [clang-diagnostic-error]
      reader_(new LHEReader(fileNames(), params.getUntrackedParameter<unsigned int>("skipEvents", 0))),
                                      ^
/cvmfs/cms-ib.cern.ch/nweek-02633/slc7_amd64_gcc820/cms/cmssw/CMSSW_11_2_X_2020-06-19-1100/src/FWCore/Sources/interface/FromFiles.h:22:5: note: 'fileNames' declared here
--
GeneratorInterface/LHEInterface/plugins/LH5Source.cc:38:39: error: too few arguments to function call, single argument 'iCatalog' was not specified [clang-diagnostic-error]
      reader_(new LH5Reader(fileNames(),params.getUntrackedParameter<unsigned int>("skipEvents", 0))),
                                      ^
/cvmfs/cms-ib.cern.ch/nweek-02633/slc7_amd64_gcc820/cms/cmssw/CMSSW_11_2_X_2020-06-19-1100/src/FWCore/Sources/interface/FromFiles.h:22:5: note: 'fileNames' declared here
--
gmake: *** [config/SCRAM/GMake/Makefile.coderules:128: code-checks] Error 2
gmake: *** [There are compilation/build errors. Please see the detail log above.] Error 2

[cmsbuild]

-1

Tested at: 00b5cc6

• Build:

I found compilation error when building:

ERROR: /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/slc7_amd64_gcc820/external/tensorflow-sources/2.1.0-e4ba21/tensorflow-2.1.0/tensorflow/tools/pip_package/BUILD:223:1 Executing genrule //tensorflow/python/keras/api:keras_python_api_gen_compat_v1 failed (Exit 1)
INFO: Elapsed time: 3080.834s, Critical Path: 206.47s
INFO: 9091 processes: 9091 local.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully
error: Bad exit status from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.6HZwWJ (%build)


RPM build errors:
Bad exit status from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.6HZwWJ (%build)



You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e4ba21/8644/summary.html

[cmsbuild]

The tests are being triggered in jenkins.
Tested with other pull request(s) cms-sw/cmsdist#6160

• CMSSW_11_2_X_2020-08-07-1100/slc7_amd64_gcc820: https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/8659/console Started: 2020/08/07 22:30

[cmsbuild]

+1
Tested at: 00b5cc6
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3c5417/8659/summary.html
CMSSW: CMSSW_11_2_X_2020-08-07-1100
SCRAM_ARCH: slc7_amd64_gcc820

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3c5417/8659/summary.html

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 35
• DQMHistoTests: Total histograms compared: 2612401
• DQMHistoTests: Total failures: 1
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2612352
• DQMHistoTests: Total skipped: 48
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 34 files compared)
• Checked 149 log files, 22 edm output root files, 35 DQM output files

[silviodonato]

After having merged cms-sw/cmsdist#6160 the tests are ok. Do you have further comments @smrenna @cms-sw/generators-l2 ?

[SiewYan]

Hello, its looks all good to me. Unless @smrenna has other comments on this.

[silviodonato]

Kind reminder @cms-sw/generators-l2

[agrohsje]

Thanks @smrenna for the PR. As there was no additional feedback I am ok with the PR being merged.

[agrohsje]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

[qliphy]

+1