Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Keras imports. #2829

Merged
merged 2 commits into from
Apr 18, 2023
Merged

Fix Keras imports. #2829

merged 2 commits into from
Apr 18, 2023

Conversation

reedwm
Copy link
Member

@reedwm reedwm commented Apr 11, 2023

Description

Brief Description of the PR:

Recently, Keras changed how you must import internal symbols. Now, you must import private symbols from keras.src instead of keras, e.g. you must use from keras.src.utils import tf_utils instead of from keras.utils import tf_utils. See slide 12 of the April Keras community meeting slides for details Because old versions of Keras, including the current stable version, require the old import form, this PR changes every file importing internal Keras symbols to try both import forms: first it tries importing the new version (with keras.src), and if that fails, it tries importing the old version (with keras)

Without this change, TF Addons cannot be imported with the latest keras-nightly pip package.

Fixes # (issue)

Type of change

Checklist:

  • I've properly formatted my code according to the guidelines
    • By running Black + Flake8
    • By running pre-commit hooks
  • This PR addresses an already submitted issue for TensorFlow Addons
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works
  • This PR contains modifications to C++ custom-ops

How Has This Been Tested?

If you're adding a bugfix or new feature please describe the tests that you ran to verify your changes:

  • Ran the official TensorFlow models with the updated TensorFlow Addons package, confirmed it worked correctly.

@bot-of-gabrieldemarmiesse

@hyang0129

You are owner of some files modified in this pull request.
Would you kindly review the changes whenever you have the time to?
Thank you very much.

@reedwm
Copy link
Member Author

reedwm commented Apr 11, 2023

CC @bhack

try:
# New versions of Keras require importing from `keras.src` when
# importing internal symbols.
from keras.src import backend

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this line? Shouldn't from keras import backend work since its public API?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Below, on line 144, backend._current_graph() is called. Since this is an internal symbol, it requires the keras.src import. from keras import backend will only allow public symbols to be accessed on backend, while from keras.src import backend allows all symbols from backend.py to be accessed.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the details. Sounds good.

sampathweb
sampathweb previously approved these changes Apr 11, 2023
@bhack
Copy link
Contributor

bhack commented Apr 11, 2023

Are you using try/except as it is faster instead of checking the version?

@reedwm
Copy link
Member Author

reedwm commented Apr 11, 2023

Are you using try/except as it is faster instead of checking the version?

No, I use try-except because the issue only affects recent keras-nightly versions, and users with older keras-nightly versions need the old import. I don't think there's an easy way to check if the keras-nightly commit is newer than a certain commit.

@bhack
Copy link
Contributor

bhack commented Apr 11, 2023

No, I use try-except because the issue only affects recent keras-nightly versions, and users with older keras-nightly versions need the old import. I don't think there's an easy way to check if the keras-nightly commit is newer than a certain commit.

Is this exported?
https://github.com/keras-team/keras/blob/master/keras/__init__.py#L31-L33

@reedwm
Copy link
Member Author

reedwm commented Apr 11, 2023

I think, but that version only refers to the stable version. The try-except will work on any given commit of Keras, which allows any version of keras-nightly to work. This is a fairly minor use case, so I can switch to a version check if you prefer.

@bhack
Copy link
Contributor

bhack commented Apr 11, 2023

Right, It is that keras-nightly is a quite ambiguous concept.

See my old point (2021) at:

keras-team/tf-keras#86

I am ok with both. I am only worried that in the future users will be surprised by the wrong fallback import if the env is broken.

/cc @seanpmorgan what do you think?

@bhack bhack requested a review from seanpmorgan April 11, 2023 18:30
@reedwm
Copy link
Member Author

reedwm commented Apr 18, 2023

@bhack I switched to checking tf.__version__ instead of using a try-except statement. I'm happy to switch it back if you prefer the try-except statement. The tf.__version__ check means older version of keras-nightly will not be supported, but this is fine as users should expect to have to update the nightly TF and Keras packages if they also update TF-Addons.

The issue is causing various things to break, such as the official models, so it would be great if this could get merged quickly. Since @seanpmorgan hasn't responded yet, can you review this without @seanpmorgan's approval?

@bhack
Copy link
Contributor

bhack commented Apr 18, 2023

No problem, I am re-running two failed jobs, then I'll merge it.

@bhack
Copy link
Contributor

bhack commented Apr 18, 2023

....but this is fine as users should expect to have to update the nightly TF and Keras packages if they also update TF-Addons.

Honestly I think no CI on Keras, TF Add-ons or TF is testing this so for me it is ok as I will not claim any compatibility without a CI coverage.

@reedwm
Copy link
Member Author

reedwm commented Apr 18, 2023

Thanks @bhack! Agreed, without CI coverage we cannot make any compatibility guarantees.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants