Refactor model summary + generalize example input array #1773

awaelchli · 2020-05-11T00:22:12Z

Before submitting

Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
Did you read the contributor guideline, Pull Request section?
Did you make sure to update the docs?
Did you write any new necessary tests?
If you made a notable change (that affects users), did you update the CHANGELOG?

What does this PR do?

Fixes #1556
~~currently blocked by #1756~~

Before: Model summary with "example_input_array" would break if order defined layers is not consistent with order of execution in forward, e.g.,

self.layer1 = nn.Linear(5, 10)
self.layer2 = nn.Linear(10, 20)

if layers are called in the order self.layer2(self.layer1(input))
but if we switched the order to

self.layer2 = nn.Linear(10, 20)
self.layer1 = nn.Linear(5, 10)

it would break.

Now: We solve this issue by installing a PyTorch hook on each layer. The hook will collect the input and output shape in the order of execution.

Supports example_input_array in order of execution
Adds many tests
Better code quality, improved readability
Better documentation

Open questions:

What should we display for layers that are unused in the forward pass? Currently I print "?" for the input-/output shapes (see tests).

pl_examples/domain_templates/generative_adversarial_net.py

tests/core/test_memory.py

codecov · 2020-05-20T00:49:23Z

Codecov Report

Merging #1773 into master will increase coverage by 1%.
The diff coverage is 95%.

@@          Coverage Diff           @@
##           master   #1773   +/-   ##
======================================
+ Coverage      87%     88%   +1%     
======================================
  Files          68      68           
  Lines        5221    5235   +14     
======================================
+ Hits         4544    4590   +46     
+ Misses        677     645   -32

Borda · 2020-05-24T17:29:28Z

Let's make this example input area as property..

awaelchli · 2020-05-24T19:23:44Z

What would be the benefit of that? Could you clarify

Borda · 2020-05-24T19:44:06Z

What would be the benefit of that? Could you clarify

it would be read-only and also it MAY resolve soma attribute issue...

awaelchli · 2020-06-06T22:10:56Z

@Borda getting this failed import "assert_speed_parity". Which one should I use, absolute or relative speed parity?

awaelchli · 2020-06-06T22:11:31Z

strange that master branch is passing the tests

variant a variant b add test revert rename add changelog docs move changelog entry to top use hooks wip wipp layer summary clean up, refactor type hints rename remove obsolete code rename unused imports simplify formatting of table and increase readability doctest superclass object update examples print unknown sizes more docs and doctest testing unknown layers add rnn test remove main restore train mode test device wip device constant simplify model forward transfer return summary object in method extend tests fix summary for empty module extend tests refactor and added hook variant a variant b add test revert rename add changelog docs move changelog entry to top remove hardcoded string simplify test unknown shapes and all others comments for tests fix hparams attribute

pytorch_lightning/core/memory.py

mergify · 2020-06-08T11:22:24Z

This pull request is now in conflict... :(

tests/core/test_memory.py

pytorch_lightning/core/memory.py

Borda · 2020-06-08T12:54:10Z

pytorch_lightning/core/memory.py

+        shape = [parse_batch_shape(el) for el in batch]
+        return shape
+
+    return UNKNOWN_SIZE


 def _format_summary_table(*cols) -> str:


we already have a table printer for Profiler, so lets general that function and use it here too

there is no utility for table printing in Profiler that can be extracted.
the shape, columns and rows of the table are all hardcoded and differ between Simple and AdvancedProfiler.

not sure how to extract this, seems this is a larger effort that does not fit well into this pr here.

https://github.com/PyTorchLightning/pytorch-lightning/blob/d6646e151a5c8c1ccfa4c4b3e6c859b0f81a1166/pytorch_lightning/profiler/profilers.py#L138-L151

yes that's exactly what I was looking at.

mergify · 2020-06-12T18:41:04Z

This pull request is now in conflict... :(

mergify · 2020-06-13T16:02:40Z

This pull request is now in conflict... :(

Borda

LGTM 🤖

SkafteNicki

LGTM

mergify bot requested a review from a team May 11, 2020 00:22

awaelchli changed the title ~~Refactor model summary + generalize example input array~~ [WIP] Refactor model summary + generalize example input array May 11, 2020

awaelchli removed the request for review from a team May 11, 2020 00:31

mergify bot requested a review from a team May 11, 2020 00:31

awaelchli removed the request for review from a team May 11, 2020 00:31

mergify bot requested a review from a team May 11, 2020 00:31

awaelchli mentioned this pull request May 14, 2020

Better visualization of the model using torchsummaryX #1833

Closed

Borda added the feature Is an improvement or enhancement label May 14, 2020

Borda reviewed May 14, 2020

View reviewed changes

pl_examples/domain_templates/generative_adversarial_net.py Show resolved Hide resolved

tests/core/test_memory.py Outdated Show resolved Hide resolved

mergify bot requested a review from a team May 14, 2020 11:55

awaelchli force-pushed the bugfix/example-input-array branch 2 times, most recently from ab1f83c to 405fcf1 Compare May 20, 2020 00:29

awaelchli mentioned this pull request May 24, 2020

replace Hparams by init args #1896

Merged

Borda added this to the 0.7.7 milestone May 24, 2020

Borda modified the milestones: 0.7.7, 0.8.0 May 26, 2020

awaelchli force-pushed the bugfix/example-input-array branch from 405fcf1 to fbf3eff Compare May 26, 2020 22:40

awaelchli added the bug Something isn't working label May 28, 2020

awaelchli force-pushed the bugfix/example-input-array branch from 632a456 to cf303ba Compare June 6, 2020 18:38

awaelchli added this to in Progress in Key features - Roadmap v1.0 Jun 6, 2020

awaelchli added 4 commits June 7, 2020 01:25

update default

022fc14

unused import

0a07f18

clean up

18a1ce5

add changelog

85230c1

Borda reviewed Jun 7, 2020

View reviewed changes

pytorch_lightning/core/memory.py Outdated Show resolved Hide resolved

mergify bot requested a review from a team June 7, 2020 09:05

awaelchli added 2 commits June 7, 2020 19:25

extract constant

4d68892

setter and getter

88c158c

Key features - Roadmap v1.0 automation moved this from in Progress to in Review Jun 8, 2020

Borda requested changes Jun 8, 2020

View reviewed changes

mergify bot requested a review from a team June 8, 2020 12:55

Borda modified the milestones: 0.8.0, 0.9.0 Jun 9, 2020

Borda added the waiting on author Waiting on user action, correction, or update label Jun 9, 2020

awaelchli added 3 commits June 12, 2020 18:05

move parity models to test folder

aa3a44f

parameterize mode

ac2875f

Merge branch 'master' into bugfix/example-input-array

27e1b68

Merge branch 'master' into bugfix/example-input-array

9753215

awaelchli removed the waiting on author Waiting on user action, correction, or update label Jun 13, 2020

Borda approved these changes Jun 15, 2020

View reviewed changes

mergify bot requested a review from a team June 15, 2020 12:19

SkafteNicki approved these changes Jun 15, 2020

View reviewed changes

mergify bot requested a review from a team June 15, 2020 12:33

Merge branch 'master' into bugfix/example-input-array

ddc4974

williamFalcon merged commit 7dc58bd into master Jun 15, 2020

Key features - Roadmap v1.0 automation moved this from in Review to Done Jun 15, 2020

Borda deleted the bugfix/example-input-array branch June 15, 2020 21:11

awaelchli mentioned this pull request Jun 17, 2020

ModelSummary fails on the "generative_adversarial_net" example #2211

Closed

Borda modified the milestones: 0.9.0, 0.8.0 Jun 18, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor model summary + generalize example input array #1773

Refactor model summary + generalize example input array #1773

awaelchli commented May 11, 2020 •

edited

Loading

codecov bot commented May 20, 2020 •

edited

Loading

Borda commented May 24, 2020

awaelchli commented May 24, 2020

Borda commented May 24, 2020

awaelchli commented Jun 6, 2020

awaelchli commented Jun 6, 2020 •

edited

Loading

mergify bot commented Jun 8, 2020

Borda Jun 8, 2020

awaelchli Jun 12, 2020

Borda Jun 12, 2020

awaelchli Jun 12, 2020

mergify bot commented Jun 12, 2020

mergify bot commented Jun 13, 2020

Borda left a comment

SkafteNicki left a comment

Refactor model summary + generalize example input array #1773

Refactor model summary + generalize example input array #1773

Conversation

awaelchli commented May 11, 2020 • edited Loading

Before submitting

What does this PR do?

codecov bot commented May 20, 2020 • edited Loading

Codecov Report

Borda commented May 24, 2020

awaelchli commented May 24, 2020

Borda commented May 24, 2020

awaelchli commented Jun 6, 2020

awaelchli commented Jun 6, 2020 • edited Loading

mergify bot commented Jun 8, 2020

Borda Jun 8, 2020

Choose a reason for hiding this comment

awaelchli Jun 12, 2020

Choose a reason for hiding this comment

Borda Jun 12, 2020

Choose a reason for hiding this comment

awaelchli Jun 12, 2020

Choose a reason for hiding this comment

mergify bot commented Jun 12, 2020

mergify bot commented Jun 13, 2020

Borda left a comment

Choose a reason for hiding this comment

SkafteNicki left a comment

Choose a reason for hiding this comment

awaelchli commented May 11, 2020 •

edited

Loading

codecov bot commented May 20, 2020 •

edited

Loading

awaelchli commented Jun 6, 2020 •

edited

Loading