Where to place loading of CUDAService?

[makortel]

PR #28537 introduces CUDAService that needs to be loaded in any configuration that may include CUDA EDModules. The service works also on machines without GPUs, and is used to communicate that fact to elsewhere in CMSSW.

The question is then where exactly the process.load("HeterogeneousCore.CUDAServices.CUDAService_cfi") should be placed? In the patatrack fork we modified ConfigBuilder to always include it in the cmsDriver.py-generated configuration files.

An alternative could be to import CUDAService into all cfi/cff files that define CUDA EDModules. On the other hand, we don't tend to do that with other services (like DQMStore or RandomNumberGeneratorService).

Requiring the framework to always load CUDAService internally does not sound a good solution either (at this stage at least).

Are there any better ideas?

[cmsbuild]

A new Issue was created by @makortel Matti Kortelainen.

@davidlange6, @Dr15Jones, @smuzaffar, @fabiocos, @kpedro88 can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

[fabiocos]

assign operations,core

[cmsbuild]

New categories assigned: operations,core

@Dr15Jones,@smuzaffar,@fabiocos,@franzoni,@davidlange6,@kpedro88 you have been requested to review this Pull request/Issue and eventually sign? Thanks

[fabiocos]

@makortel so practically what you did was to update https://github.com/cms-sw/cmssw/blob/master/Configuration/StandardSequences/python/Services_cff.py ? This sounds as a natural entry point, unless you want to have it conditional to some specific flag

[makortel]

@fabiocos I put a process.load() in ConfigBuilder

cmssw/Configuration/Applications/python/ConfigBuilder.py

Lines 912 to 925 in 692e725

	def define_Configs(self):
	if len(self.stepMap):
	self.loadAndRemember('Configuration/StandardSequences/Services_cff')
	if self._options.particleTable not in defaultOptions.particleTableList:
	print('Invalid particle table provided. Options are:')
	print(defaultOptions.particleTable)
	sys.exit(-1)
	else:
	if len(self.stepMap):
	self.loadAndRemember('SimGeneral.HepPDTESSource.'+self._options.particleTable+'_cfi')
	self.loadAndRemember('FWCore/MessageService/MessageLogger_cfi')
	# Eventually replace with some more generic file to load
	self.loadAndRemember('HeterogeneousCore/CUDAServices/CUDAService_cfi')

Thanks for pointing out the Configuration.StandardSequences.Services_cff, it indeed looks a better placement. The intention indeed is that CUDAService could be loaded by any job, and the service internally handles the case of no GPUs.

[makortel]

Based on the discussion yesterday in the core meeting (given #28576), probably it would make sense (as an interim solution) to load CUDAService_cfi only if the gpu modifier (#28522) is given to the cmsDriver.py. Therefore the loadAndRemember() call would stay in the ConfigBuilder, protected with an "if gpu". @fabiocos, @fwyzard, what do you think?

[fwyzard]

I understood that if gpu is deprecated... apart from that, I'm fine with this approach :-)

[makortel]

The actual line would not be if gpu.isChosen(), but along "if gpu in --processModifiers".

[makortel]

I agree it is not pretty, but would limit the set of broken workflows in ppc64le only to the ones really using CUDA.

[makortel]

Thinking further

def loadCUDAService(process):
    process.load("HeterogeneousCore.CUDAServices.CUDAService_cff")
modifyConfigurationStandardSequencesServicesCUDAService = gpu.makeProcessModifier(loadCUDAService)

in Configuration/StandardSequences/Services_cff should work as well and wouldn't require touching ConfigBuilder.

[fwyzard]

Look clean enough to me ...

[fabiocos]

@makortel from a quick test I did

from Configuration.ProcessModifiers.gpu_cff import *
def loadCUDAService(process):
    process.load("HeterogeneousCore.CUDAServices.CUDAService_cfi")
modifyConfigurationStandardSequencesServicesCUDAService_ = gpu.makeProcessModifier(loadCUDAService)

is correctly uploaded in case the process modifier gpu is part of the Era