How LRP is used in ResNet?

[syomantak]

This is actually a theoretical question. How are the values propagated back in ResNet where there is skip connection?
Is the value divided equally between the previous layer neuron and the neurons of the layer from which skip connection is done? If so, then the assertion that sum of the relevance of all neurons in every layer is equal would be wrong as some relevance has 'leaked' to the initial layers due to skip connection.

[bernerprzemek]

I don't know if this is correct, but in my case I just add branches relevance an simply divide by 2. Theoretically both branches have same input (which later are added together) so relevance should be sum of branches values divided by number of branches.
But this is very interesting question how to treat this situation. And also (I'm trying LRP on EfficientNet) how to treat skip connections with multiplication on the end. I was trying to find answer for some time, but seams that LRP is usually used to explain sequential architectures.

[syomantak]

@bernerprzemek but if what you say is true then the so-called 'identity' that sum of relevance values for each layer remains constant would no longer be true! Anyway, I have used a different XAI method for my research so I do not need the answer as such.

[bernerprzemek]

Yes probably not, but keep in mid that residual connection isn't simply y=f(x) but y=f(x)+x, and this fact has to be included in conservation rule, the same happen in batchnormalization layer where sum around all relevance values are no longer same as in previous layer.