I observe that the validation phase is much slower than the training phase on large validation sets and multi-GPU machines

I observe that the validation phase is much slower than the training phase on large validation sets and multi-GPU machines #27073