Support for retain_graph=True

[momeara]

Is your feature request related to a problem? Please describe.
Some models require retain_graph=True, but it's not possible to set it in the .backward() call inside of Trainer.__run_training_batch(...)

Describe the solution you'd like
Add train_graph member function the LightningModule have the trainer read this option and then pass it into the .backward() call.

Describe alternatives you've considered
Driving a version of Trainer to support retain_graph=True is tough because __run_training_batch and other functions are name-mangled.

[williamFalcon]

why don’t we put a hook for the backward pass and you can override if you need retain_graph? that keeps it flexible for anything else someone wants to do later

[momeara]

perfect! that keeps with the overall philosophy, which has been very useful btw

…

On Oct 12, 2019, at 5:47 AM, William Falcon ***@***.***> wrote:  why don’t we put a hook for the backward pass and you can override if you need retain_graph? that keeps it flexible for anything else someone wants to do later — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[williamFalcon]

@yl1991 @momeara did you guys want to submit a PR for this?