-
-
Notifications
You must be signed in to change notification settings - Fork 228
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Maintenence: update mybinder environment.yml dependency versions #62
Comments
Yes indeed. That would be good to do. Would you be willing to submit the
PR? I'll be curious (and hopeful) to see if the tests pass or if there
were any conflicting API changes.
…On Tue, Feb 26, 2019 at 5:37 PM Genevieve Buckley ***@***.***> wrote:
I noticed that the mybinder environment.yml file pins dask to version
0.20, but the latest dask release is now up to 1.1.2. It's probably time to
update or unpin some of these dependencies. Should we do that?
Currently:
https://github.com/dask/dask-examples/blob/master/binder/environment.yml
channels:
- conda-forge
dependencies:
- python=3
- bokeh=0.13
- dask=0.20
- dask-ml=0.10.0
- distributed=1.24
- jupyterlab=0.35.1
- nodejs=8.9
- numpy
- pandas
- pyarrow==0.10.0
- scikit-learn=0.20
- matplotlib
- nbserverproxy
- nomkl
- h5py
- xarray
- bottleneck
- py-xgboost
- pip:
- graphviz
- dask_xgboost
- seaborn
- mimesis
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#62>, or mute the thread
<https://github.com/notifications/unsubscribe-auth/AASszOEFeKO-oLAJUPUmowaK0I2tJdyeks5vReFmgaJpZM4bTgQg>
.
|
Sure, I can do that. Do you prefer to have things unpinned if possible, or just increment to a newer release? We'll probably run into this problem again (outdated dependencies) before too long if we keep things pinned, so I'd prefer not to. But I'd understand if you feel pull requests aren't frequent enough on this repo to trigger tests and alert us of any future issues. |
I'd generally prefer that we keep things pinned. Otherwise bugs from
mismatched versions might creep in without us knowing about it and things
will go down and cause confusion. Instead I'd rather that we be explicit
when we change versions and address any bugs that come up when we make such
a change.
…On Tue, Feb 26, 2019 at 6:32 PM Genevieve Buckley ***@***.***> wrote:
Sure, I can do that.
Do you prefer to have things unpinned if possible, or just increment to a
newer release? We'll probably run into this problem again (outdated
dependencies) before too long if we keep things pinned, so I'd prefer not
to. But I'd understand if you feel pull requests aren't frequent enough on
this repo to trigger tests and alert us of any future issues.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#62 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AASszBZs3AIkKVCj46HSjz_4Z7XbNS89ks5vRe5AgaJpZM4bTgQg>
.
|
To update: I think it's better to put this issue on hold until the next dask-ml release is available. I think this because upgrading dask versions has unearthed a few bugs, and the new release should contain some of the fixes we need. Error 1
Error 2
Error 3I found that dask dataframe .std() produces error in cases where there are NaNs in output. Seen in the dask-examples/dataframes.ipynb notebook. Other potential issues aheadI also want to update the version of bokeh. Currently it's at 0.13 and the latest is >= 1.0.0 (a stable version 1.1.0 seems pretty close to release). I anticipate this will mean we'll have to update the examples using bokeh in the notebooks, as there are quite a few changes here. CI build logsMy branch is here: https://github.com/GenevieveBuckley/dask-examples/tree/update-binder-env |
cc @TomAugspurger @jrbourbeau no pressure, but do you all have a timeline
on a dask-ml release?
…On Wed, Feb 27, 2019 at 11:18 PM Genevieve Buckley ***@***.***> wrote:
To update: I think it's better to put this issue on hold until the next
dask-ml release is available.
I think this because upgrading dask versions has unearthed a few bugs, and
the new release should contain some of the fixes we need.
Error 1
AttributeError: module 'dask' has no attribute 'sharedict', seen in the
dask-examples/machine-learning/incremental.ipynb notebook. This is
addressed by the merged PR dask/dask-ml#455
<dask/dask-ml#455>,
Error 2
ValueError: high is out of bounds for int32, `, seen in the
dask-examples/machine-learning.ipynb notebook. I think the open pull
request dask/dask-ml#462 <dask/dask-ml#462> is
likely to resolve this.
Error 3
I found that dask dataframe .std() produces error in cases where there are
NaNs in output. Seen in the dask-examples/dataframes.ipynb notebook.
I raised an issue for it here: dask/dask#4534
<dask/dask#4534>, and the associated pull
request is here: dask/dask#4535 <dask/dask#4535>
We can sidestep it entirely by replacing the dask dataframe std() example
with mean() instead. I've changed that in the example notebook, so we
don't have to wait for this bugfix to be released.
Other potential issues ahead
I also want to update the version of bokeh. Currently it's at 0.13 and the
latest is >= 1.0.0 (a stable version 1.1.0 seems pretty close to release).
I anticipate this will mean we'll have to update the examples using bokeh
in the notebooks, as there are quite a few changes here.
CI build logs
My branch is here:
https://github.com/GenevieveBuckley/dask-examples/tree/update-binder-env
You can take a look at the CI build logs here:
https://travis-ci.com/GenevieveBuckley/dask-examples/branches
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#62 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AASszCTCP2uIF-ahP2r0Ec_sePnLkcpoks5vR4KogaJpZM4bTgQg>
.
|
Also, thanks for driving this and unearthing these bugs @GenevieveBuckley ! It's a great help. |
Whoops forgot to last week. I’ll take a look at doing one today.
… On Feb 28, 2019, at 09:15, Matthew Rocklin ***@***.***> wrote:
cc @TomAugspurger @jrbourbeau no pressure, but do you all have a timeline
on a dask-ml release?
On Wed, Feb 27, 2019 at 11:18 PM Genevieve Buckley ***@***.***>
wrote:
> To update: I think it's better to put this issue on hold until the next
> dask-ml release is available.
>
> I think this because upgrading dask versions has unearthed a few bugs, and
> the new release should contain some of the fixes we need.
> Error 1
>
> AttributeError: module 'dask' has no attribute 'sharedict', seen in the
> dask-examples/machine-learning/incremental.ipynb notebook. This is
> addressed by the merged PR dask/dask-ml#455
> <dask/dask-ml#455>,
> Error 2
>
> ValueError: high is out of bounds for int32, `, seen in the
> dask-examples/machine-learning.ipynb notebook. I think the open pull
> request dask/dask-ml#462 <dask/dask-ml#462> is
> likely to resolve this.
> Error 3
>
> I found that dask dataframe .std() produces error in cases where there are
> NaNs in output. Seen in the dask-examples/dataframes.ipynb notebook.
> I raised an issue for it here: dask/dask#4534
> <dask/dask#4534>, and the associated pull
> request is here: dask/dask#4535 <dask/dask#4535>
> We can sidestep it entirely by replacing the dask dataframe std() example
> with mean() instead. I've changed that in the example notebook, so we
> don't have to wait for this bugfix to be released.
> Other potential issues ahead
>
> I also want to update the version of bokeh. Currently it's at 0.13 and the
> latest is >= 1.0.0 (a stable version 1.1.0 seems pretty close to release).
> I anticipate this will mean we'll have to update the examples using bokeh
> in the notebooks, as there are quite a few changes here.
> CI build logs
>
> My branch is here:
> https://github.com/GenevieveBuckley/dask-examples/tree/update-binder-env
> You can take a look at the CI build logs here:
> https://travis-ci.com/GenevieveBuckley/dask-examples/branches
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <#62 (comment)>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AASszCTCP2uIF-ahP2r0Ec_sePnLkcpoks5vR4KogaJpZM4bTgQg>
> .
>
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.
|
released dask-ml 0.12.0. Binaries should be showing up in
https://anaconda.org/conda-forge/dask-ml over the next hour or two.
On Thu, Feb 28, 2019 at 9:23 AM Tom Augspurger <tom.augspurger88@gmail.com>
wrote:
… Whoops forgot to last week. I’ll take a look at doing one today.
On Feb 28, 2019, at 09:15, Matthew Rocklin ***@***.***>
wrote:
cc @TomAugspurger @jrbourbeau no pressure, but do you all have a timeline
on a dask-ml release?
On Wed, Feb 27, 2019 at 11:18 PM Genevieve Buckley <
***@***.***>
wrote:
> To update: I think it's better to put this issue on hold until the next
> dask-ml release is available.
>
> I think this because upgrading dask versions has unearthed a few bugs,
and
> the new release should contain some of the fixes we need.
> Error 1
>
> AttributeError: module 'dask' has no attribute 'sharedict', seen in the
> dask-examples/machine-learning/incremental.ipynb notebook. This is
> addressed by the merged PR dask/dask-ml#455
> <dask/dask-ml#455>,
> Error 2
>
> ValueError: high is out of bounds for int32, `, seen in the
> dask-examples/machine-learning.ipynb notebook. I think the open pull
> request dask/dask-ml#462 <dask/dask-ml#462> is
> likely to resolve this.
> Error 3
>
> I found that dask dataframe .std() produces error in cases where there
are
> NaNs in output. Seen in the dask-examples/dataframes.ipynb notebook.
> I raised an issue for it here: dask/dask#4534
> <dask/dask#4534>, and the associated pull
> request is here: dask/dask#4535 <dask/dask#4535>
> We can sidestep it entirely by replacing the dask dataframe std() example
> with mean() instead. I've changed that in the example notebook, so we
> don't have to wait for this bugfix to be released.
> Other potential issues ahead
>
> I also want to update the version of bokeh. Currently it's at 0.13 and
the
> latest is >= 1.0.0 (a stable version 1.1.0 seems pretty close to
release).
> I anticipate this will mean we'll have to update the examples using bokeh
> in the notebooks, as there are quite a few changes here.
> CI build logs
>
> My branch is here:
> https://github.com/GenevieveBuckley/dask-examples/tree/update-binder-env
> You can take a look at the CI build logs here:
> https://travis-ci.com/GenevieveBuckley/dask-examples/branches
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <#62 (comment)
>,
> or mute the thread
> <
https://github.com/notifications/unsubscribe-auth/AASszCTCP2uIF-ahP2r0Ec_sePnLkcpoks5vR4KogaJpZM4bTgQg
>
> .
>
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#62 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ABQHIiVkSdWQ6Shk9LrrwsRVK95ZK9qrks5vR_J7gaJpZM4bTgQg>
.
|
If possible I'd like to revert this change. I'd like for us to avoid having the examples work around bugs. I'm more than happy to prioritize bugfixes that block this. (I think that the underlying issue has now been fixed, and a dask/dask release is imminent). |
Totally fine, I can revert that change. I was not expecting everyone to be so quick with bugfixes & new releases! |
I noticed that the mybinder environment.yml file pins dask to version 0.20, but the latest dask release is now up to 1.1.2. It's probably time to update or unpin some of these dependencies. Should we do that?
Currently:
https://github.com/dask/dask-examples/blob/master/binder/environment.yml
The text was updated successfully, but these errors were encountered: