Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lazify various calls to all() by using generators instead of comprehensions #1829

Merged
merged 4 commits into from
Sep 12, 2023

Conversation

eumiro
Copy link
Contributor

@eumiro eumiro commented Aug 6, 2023

  • [ ] Closes #xxxx
  • I am familiar with the contributing guidelines
  • Tests added
  • Updates entries in docs/sphinx/source/reference for API changes.
  • Adds description and name entries in the appropriate "what's new" file in docs/sphinx/source/whatsnew for all changes. Includes link to the GitHub Issue with :issue:`num` or this Pull Request with :pull:`num`. Includes contributor name and/or GitHub username (link with :ghuser:`user`).
  • New code is fully documented. Includes numpydoc compliant docstrings, examples, and comments where necessary.
  • Pull request is nearly complete and ready for detailed review.
  • Maintainer: Appropriate GitHub Labels (including remote-data) and Milestone are assigned to the Pull Request and linked Issue.

This simple PR converts several list comprehension expressions into generator expressions. The benefit is that generators are in theory more CPU- and memory-efficient, for example by allowing short-circuiting when passed into a function like all(). None of the cases touched in this PR are likely to make much of a practical difference, but if nothing else, shifting the pvlib style more towards generators is probably a good move in the long run.

Copy link
Member

@kandersolar kandersolar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @eumiro for this PR! And apologies for the delayed review.

I am generally okay with this PR, but a few changes of the changes need to be reverted in order to preserve compatibility pandas Series objects. Details below.

pvlib/tests/test_pvsystem.py Outdated Show resolved Hide resolved
pvlib/modelchain.py Outdated Show resolved Hide resolved
pvlib/modelchain.py Outdated Show resolved Hide resolved
@cwhanse
Copy link
Member

cwhanse commented Aug 30, 2023

@eumiro you'll see test failures in unrelated code. Those are from a scipy issue, and you may ignore them.

Copy link
Member

@kandersolar kandersolar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @eumiro!

@kandersolar kandersolar changed the title Lazify all() Lazify various calls to all() by using generators instead of comprehensions Sep 12, 2023
@kandersolar kandersolar merged commit 29074f5 into pvlib:main Sep 12, 2023
23 of 29 checks passed
@eumiro eumiro deleted the loops branch September 12, 2023 20:30
@adriesse
Copy link
Member

This is another example of a PR where I have to go to the code changes to get any idea of what its purpose is and even then I only half understand it. I recommend putting descriptions in PR's and/or associated issues.

@mikofski
Copy link
Member

A jargon glossary would be helpful. “Lazy” means evaluated at the last possible conceivable instant, in this case because all() will fail if any object in the potential list evaluates as False then it’s unnecessary to evaluate all of the objects in the list or even expand a list into an array. Using a Python generator is “lazy” because it only evaluates one item at a time from the potential list, so that as soon as a False item is evaluated it can return False from the all() or at least that’s the theory. In practice the only advantage may be more terse code with fewer characters. Hard to know unless you run the numbers. There’s also a memory advantage usually too

@adriesse
Copy link
Member

I understand the concepts actually, just think if it's worth the trouble to make the change it should be worth a few sentences from the contributor to describe it. The benefit of a generator for two simple items must surely be very small.

@kandersolar
Copy link
Member

I agree that the benefit of this change is minor at best, but as long as the PR was already submitted I didn't see any harm in merging it. I wouldn't suggest anyone spend time on these kinds of changes in the future though :)

I also agree that a bit of explanation would have been useful here. I did edit the title to be a bit more helpful but did not think to add explanation in the body. I will do that now!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants