use conda.api instead of parallel calls to the conda binary #4775
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
keewis
changed the title
|
the new code should generate the same report as the old code now, so this should be ready for review. The tool says we can bump a few libraries (for example we could bump |
|
LGTM |
dcherian
added a commit
to TomNicholas/xarray
that referenced
this pull request
Jan 18, 2021
* upstream/master: (342 commits) fix decode for scale/ offset list (pydata#4802) Expand user dir paths (~) in open_mfdataset and to_zarr. (pydata#4795) add a version info step to the upstream-dev CI (pydata#4815) fix the ci trigger action (pydata#4805) scatter plot by order of the first appearance of hue (pydata#4723) don't skip the scheduled CI (pydata#4806) coords: retain str dtype (pydata#4759) Fix interval labels with units (pydata#4794) Always force dask arrays to float in missing.interp_func (pydata#4771) Print number of variables in repr (pydata#4762) install conda as a library in the minimum dependency check CI (pydata#4792) Migrate CI from azure pipelines to GitHub Actions (pydata#4730) use conda.api instead of parallel calls to the conda binary (pydata#4775) Speed up missing._get_interpolator (pydata#4776) Remove special case in guess_engines (pydata#4777) improve typing of OrderedSet (pydata#4774) CI: ignore some warnings (pydata#4773) DOC: update hyperlink for xskillscore (pydata#4778) drop support for python 3.6 (pydata#4720) Trigger upstream CI on cron schedule (by default) (pydata#4729) ...
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently, our
min_deps_check.pyscript uses thecondabinary to analyze the package metadata. This is not very efficient becausecondawill download and parserepodata.jsonon each call. To speed up the script, we're currently running 8 of these calls (+ processing the results) in parallel using aThreadPoolExecutor, but that increases the memory consumption (and my old laptop does not have that much memory).condaprovides theconda.api.SubdirData.query_allfunction which will cache the parsedrepodata.jsonfiles between calls. Using that, my laptop can completein about 30 seconds, even though the packages are analyzed sequentially.
The documentation states that that function maybe be changed without warning between minor versions, so we would have to pin
condato ax.yversion to be able to use this.Edit: the documentation does have that warning, but it also says:
so I guess we don't have to pin?
cc @crusaderky
By default, the upstream dev CI is disabled on pull request and push events. You can override this behavior per commit by adding a
[test-upstream]tag to the first line of the commit message.