added override for hparams in load_from_ckpt

[williamFalcon]

Fixes #1474

Enable the following... (ran into a bunch of issues with transfer learning when moving across clusters)... ugh.

I think @tullie has suggested something like this?

# override some of the params with new values
MyLightningModule.load_from_checkpoint(
    PATH,
    hparam_overrides={'num_layers': 128, 'pretrained_ckpt_path': NEW_PATH}
)

[justusschock]

This doesn't work, if hparams is a nested dict, right?

Meaning if I have original hparams like

{'a': {'b': ..., 'c': ...}, 'd':...}
I cannot replace only a.b but I'd have to specify a completely, right?

IMO we could think about flattening the hpramas dict on save and reverse this on load to support this, but this would require some more changes...

[williamFalcon]

good point. i guess we can support replacing the nested hparams by passing nested updates here? match key by key within the tree

[justusschock]

Should work and probably is the same effort as having to flatten the dict.
While we're on it: Can tensorboard log nested hparams? If not, we may want to flatten the dict anyway after the network is initialized?

[williamFalcon]

you mean in the setter for hparams in the pl module?

[justusschock]

Yes maybe and also save it flattened (then you could acess the entries with something like hparams['a.b'] in the example above) and unflatten it right before model init when laoding from checkpoint...

[williamFalcon]

@Borda is this how we get doc tests?

[williamFalcon]

@justusschock but if we flatten for the user it might be unexpected no?

1. maybe a better behavior is that on hparams setter in the module it becomes a namespace so that users don't have to use ['notation'].

2. for loggers we flatten all of them with the dot notation?

[Borda]

@Borda is this how we get doc tests?

What do you mean?

[williamFalcon]

I added the >>> on the docs.
Do I need something special to run as doc tests?

[Borda]

I added the >>> on the docs.
Do I need something special to run as doc tests?

it depends in it is python file, you do not need anything extra
if it is the rest file then you do not use >>> but put it to testcode

.. testcode::

    a = 1

then all these are detected automatically and executed as well as other standard tests
cc: @awaelchli

[mergify]

This pull request is now in conflict... :(

[codecov]

Codecov Report

Merging #1797 into master will increase coverage by 0%.
The diff coverage is 91%.

@@          Coverage Diff           @@
##           master   #1797   +/-   ##
======================================
  Coverage      88%     88%           
======================================
  Files          71      71           
  Lines        4394    4404   +10     
======================================
+ Hits         3862    3871    +9     
- Misses        532     533    +1     

[tullie]

LGTM