model.train(False) affects gradient tracking?

[MaverickMeerkat]

In this tutorial here it says in the comment that "# We don't need gradients on to do reporting". From what I understand the train flag only affects layers such as dropout and batch-normalization. Does it also affect gradient calculations, or is this comment wrong?

tutorials/beginner_source/introyt/trainingyt.py

Line 293 in 6bd30cf

# We don't need gradients on to do reporting

cc @suraj813

[chedatomasz]

This comment, and the accompanying code, is wrong. A more correct pattern is shown at https://pytorch.org/tutorials/beginner/basics/optimization_tutorial.html#full-implementation, although this one misses the train/eval switches and thus does not generalize to models with batchnorm and dropout.

[chedatomasz]

Related issues: #2083 #1756 #208

[JoseLuisC99]

/assigntome