SpectralNormalization #17648

markub3327 · 2023-03-08T23:13:57Z

Hello,

here is the adapted SpectralNormalization from TF-Addons implementation.

Thanks.

fchollet · 2023-03-12T20:31:07Z

Thanks for the PR!

We have recently ported a number of classes from TFAddons to core Keras. However for each class we ended up doing pretty extensive style changes and we increased test coverage. If we were to bring SpectralNormalization into Keras it would require further work.

However I am not sure we should do it. Our observation was that SpectralNormalization was getting very little usage, in terms of pageviews to the TFAddons docs. It didn't seem to meet our threshold for the kind of APIs we host in core Keras. Are you aware of any important use cases?

markub3327 · 2023-03-13T07:29:22Z

@fchollet

I think it is a useful layer. Please look at paper: https://arxiv.org/abs/1802.05957.
It's cited by 3797.

PyTorch includes by default: https://paperswithcode.com/paper/spectral-normalization-for-generative

fchollet

Sounds good -- the citation count is pretty convincing.

keras/layers/normalization/spectral_normalization.py

fchollet · 2023-03-13T19:38:52Z

keras/layers/normalization/spectral_normalization.py

+            )
+
+    def get_config(self):
+        config = {"power_iterations": self.power_iterations}


If it doesn't include the layer being wrapped, it won't properly deserialize. Is this tested?

Only power_iterations was tested.

keras/layers/normalization/spectral_normalization_test.py

keras/layers/normalization/spectral_normalization.py

fchollet

Thanks for the update! Looking good. Some minor nits.

keras/layers/normalization/spectral_normalization.py

fchollet

LGTM, thank you for the contribution!

markub3327 · 2023-03-28T19:46:46Z

@haifeng-jin All is repaired. Thanks.

Imported from GitHub PR #17648 Hello, here is the adapted `SpectralNormalization` from TF-Addons implementation. Thanks. Copybara import of the project: -- 0ae1ae7 by Martin Kubovcik <markub3327@gmail.com>: + SpectralNormalization -- 7f91062 by Martin Kubovcik <markub3327@gmail.com>: fixes -- 5fbe19e by Martin Kubovcik <markub3327@gmail.com>: fix -- f57b45a by Martin Kubovcik <markub3327@gmail.com>: update -- 46269ea by Martin Kubovcik <markub3327@gmail.com>: update tests Merging this change closes #17648 FUTURE_COPYBARA_INTEGRATE_REVIEW=#17648 from markub3327:keras-update 46269ea PiperOrigin-RevId: 519773515

PiperOrigin-RevId: 520171696

Imported from GitHub PR keras-team/keras#17648 Hello, here is the adapted `SpectralNormalization` from TF-Addons implementation. Thanks. Copybara import of the project: -- 0ae1ae70024b548fc2aee47c976ca4c30530157f by Martin Kubovcik <markub3327@gmail.com>: + SpectralNormalization -- 7f91062eea8d23f6407a2c3bc253e82e48a52e30 by Martin Kubovcik <markub3327@gmail.com>: fixes -- 5fbe19ecadbdf69acb13f1b3aac3c411b1427c83 by Martin Kubovcik <markub3327@gmail.com>: fix -- f57b45aa5a635a3d75ec957e9ca3f7ce5984421c by Martin Kubovcik <markub3327@gmail.com>: update -- 46269ea5fc5d404051c47a7eadcd42acd2a62ad6 by Martin Kubovcik <markub3327@gmail.com>: update tests Merging this change closes #17648 PiperOrigin-RevId: 520171696

rchao · 2023-03-30T17:26:26Z

@haifeng-jin mind taking a look what's being blocked here?

haifeng-jin · 2023-03-30T17:54:22Z

@rchao Seems it is merged by copybara here, but somehow the PR is not merged.

haifeng-jin · 2023-03-30T17:55:51Z

Probably, we should just click "Merge pull request" here since copybara missed it.
Anyone who has the permission may help.

milmor · 2023-06-27T20:42:04Z

Are you sure this spectral normalization works correctly? It seems like it's the same tfa implementation, which is wrong @fchollet

tensorflow/addons#2414

+ SpectralNormalization

0ae1ae7

google-ml-butler bot added the size:L label Mar 8, 2023

google-ml-butler bot assigned gbaned Mar 8, 2023

gbaned requested a review from fchollet March 9, 2023 17:12

google-ml-butler bot added the keras-team-review-pending Pending review by a Keras team member. label Mar 9, 2023

qlzh727 removed the keras-team-review-pending Pending review by a Keras team member. label Mar 9, 2023

fchollet requested changes Mar 13, 2023

View reviewed changes

fixes

7f91062

markub3327 requested a review from fchollet March 13, 2023 23:05

google-ml-butler bot added the keras-team-review-pending Pending review by a Keras team member. label Mar 13, 2023

fix

5fbe19e

mattdangerw removed the keras-team-review-pending Pending review by a Keras team member. label Mar 16, 2023

fchollet reviewed Mar 20, 2023

View reviewed changes

keras/layers/normalization/spectral_normalization.py Outdated Show resolved Hide resolved

keras/layers/normalization/spectral_normalization.py Outdated Show resolved Hide resolved

update

f57b45a

markub3327 requested a review from fchollet March 21, 2023 09:02

google-ml-butler bot added the keras-team-review-pending Pending review by a Keras team member. label Mar 21, 2023

fchollet approved these changes Mar 23, 2023

View reviewed changes

google-ml-butler bot added kokoro:force-run ready to pull Ready to be merged into the codebase labels Mar 23, 2023

kokoro-team removed the kokoro:force-run label Mar 23, 2023

fchollet removed the keras-team-review-pending Pending review by a Keras team member. label Mar 23, 2023

update tests

46269ea

markub3327 requested a review from fchollet March 24, 2023 05:20

google-ml-butler bot added keras-team-review-pending Pending review by a Keras team member. and removed ready to pull Ready to be merged into the codebase labels Mar 24, 2023

Update spectral_normalization.py

29d3178

markub3327 requested a review from haifeng-jin March 28, 2023 19:47

google-ml-butler bot added the keras-team-review-pending Pending review by a Keras team member. label Mar 28, 2023

haifeng-jin approved these changes Mar 28, 2023

View reviewed changes

google-ml-butler bot added the kokoro:force-run label Mar 28, 2023

kokoro-team removed the kokoro:force-run label Mar 28, 2023

copybara-service bot pushed a commit that referenced this pull request Mar 28, 2023

Merge pull request #17648 from markub3327:keras-update

42047cc

PiperOrigin-RevId: 520171696

rchao removed the keras-team-review-pending Pending review by a Keras team member. label Mar 30, 2023

gbaned removed the ready to pull Ready to be merged into the codebase label Apr 6, 2023

gbaned approved these changes Apr 6, 2023

View reviewed changes

google-ml-butler bot added kokoro:force-run ready to pull Ready to be merged into the codebase labels Apr 6, 2023

kokoro-team removed the kokoro:force-run label Apr 6, 2023

gbaned added ready to pull Ready to be merged into the codebase and removed ready to pull Ready to be merged into the codebase labels Apr 10, 2023

haifeng-jin closed this Apr 12, 2023

google-ml-butler bot removed the ready to pull Ready to be merged into the codebase label Apr 12, 2023

markub3327 deleted the keras-update branch April 13, 2023 18:31

qlzh727 mentioned this pull request Jun 1, 2023

Build against TF2.13 tensorflow/addons#2835

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SpectralNormalization #17648

SpectralNormalization #17648

markub3327 commented Mar 8, 2023

fchollet commented Mar 12, 2023

markub3327 commented Mar 13, 2023 •

edited

Loading

fchollet left a comment

fchollet Mar 13, 2023

markub3327 Mar 13, 2023

fchollet left a comment

fchollet left a comment

markub3327 commented Mar 28, 2023

rchao commented Mar 30, 2023

haifeng-jin commented Mar 30, 2023

haifeng-jin commented Mar 30, 2023

milmor commented Jun 27, 2023 •

edited

Loading

SpectralNormalization #17648

SpectralNormalization #17648

Conversation

markub3327 commented Mar 8, 2023

fchollet commented Mar 12, 2023

markub3327 commented Mar 13, 2023 • edited Loading

fchollet left a comment

Choose a reason for hiding this comment

fchollet Mar 13, 2023

Choose a reason for hiding this comment

markub3327 Mar 13, 2023

Choose a reason for hiding this comment

fchollet left a comment

Choose a reason for hiding this comment

fchollet left a comment

Choose a reason for hiding this comment

markub3327 commented Mar 28, 2023

rchao commented Mar 30, 2023

haifeng-jin commented Mar 30, 2023

haifeng-jin commented Mar 30, 2023

milmor commented Jun 27, 2023 • edited Loading

markub3327 commented Mar 13, 2023 •

edited

Loading

milmor commented Jun 27, 2023 •

edited

Loading