[DOC] upgrade mxnet dependency to 1.5.0 #842

szha · 2019-07-21T00:17:57Z

Description

upgrade mxnet dependency to 1.5.0

Checklist

Essentials

PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
Changes are complete (i.e. I finished coding on this PR)
All changes have test coverage
Code is well-documented

Changes

upgrade mxnet dependency to 1.5.0

codecov · 2019-07-21T00:17:58Z

Codecov Report

❗ No coverage uploaded for pull request head (upgrade_dep@e79e871). Click here to learn what that means.
The diff coverage is n/a.

codecov · 2019-07-21T00:17:59Z

Codecov Report

Merging #842 into master will increase coverage by 0.48%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #842      +/-   ##
==========================================
+ Coverage   89.69%   90.18%   +0.48%     
==========================================
  Files          66       66              
  Lines        6345     6345              
==========================================
+ Hits         5691     5722      +31     
+ Misses        654      623      -31

Impacted Files	Coverage Δ
src/gluonnlp/model/utils.py	`77.58% <ø> (ø)`	⬆️
src/gluonnlp/data/dataset.py	`99.2% <0%> (+1.58%)`	⬆️
src/gluonnlp/data/word_embedding_evaluation.py	`96.96% <0%> (+2.27%)`	⬆️
src/gluonnlp/data/transforms.py	`80.11% <0%> (+2.98%)`	⬆️
src/gluonnlp/data/batchify/batchify.py	`96.42% <0%> (+3.57%)`	⬆️
src/gluonnlp/data/corpora/wikitext.py	`100% <0%> (+5.17%)`	⬆️
src/gluonnlp/model/parameter.py	`100% <0%> (+8%)`	⬆️

mli · 2019-07-22T15:29:45Z

mli · 2019-07-22T15:29:47Z

Job PR-842/3 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-842/3/index.html

leezu · 2019-07-22T17:53:56Z

@eric-haibin-lin there is an issue with float16 support for the bert finetune scripst on mxnet 1.5 mxnet.base.MXNetError: [15:30:30] src/operator/tensor/./ordering_op-inl.h:551: This operation does not support float16

eric-haibin-lin · 2019-07-22T19:05:56Z

There was a fix for AMP which is added to mxnet after 1.5 code freeze. I can help fix the script.

mli · 2019-07-25T04:51:33Z

mli · 2019-07-25T04:51:34Z

Job PR-842/4 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-842/4/index.html

mli · 2019-07-29T09:05:35Z

Found link check problems in job PR-842/5:
(line 19) broken https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/ - 404 Client Error: Not Found for url: https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/
(line 5) broken https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/ - 404 Client Error: Not Found for url: https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/
(line 5) broken https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/ - 404 Client Error: Not Found for url: https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/
(line 21) broken https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/ - 404 Client Error: Not Found for url: https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/
(line 442) broken https://www.aclweb.org/anthology/P02-1040.pdf)[1 - 404 Client Error: Not Found for url: https://www.aclweb.org/anthology/P02-1040.pdf)%5B1
(line 92) broken https://nlp.stanford.edu/pubs/glove.pdf)[2 - 404 Client Error: NOT FOUND for url: https://nlp.stanford.edu/pubs/glove.pdf)%5B2
(line 208) broken https://www.bioinf.jku.at/publications/older/2604.pdf)[3 - 404 Client Error: Not Found for url: https://www.bioinf.jku.at/publications/older/2604.pdf)%5B3

mli · 2019-07-29T09:05:37Z

Job PR-842/5 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-842/5/index.html

leezu · 2019-07-29T09:12:13Z

@szha could you please update cuda on the CI server. The current version is untested for mxnet 1.5 and may be responsible for the out of memory errors
http://ci.mxnet.io/blue/organizations/jenkins/GluonNLP-py3-gpu-integration/detail/PR-848/7/

env/gpu/py3-master.yml

Previously conda would install an outdated version (1.7) time to time http://ci.mxnet.io/blue/organizations/jenkins/GluonNLP-py3-gpu-integration/detail/PR-842/10/pipeline/

Works around an apparent MXNet 1.5 storage manager regression

mli · 2019-07-30T13:56:10Z

Found link check problems in job PR-842/13:
(line 19) broken https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/ - 404 Client Error: Not Found for url: https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/
(line 5) broken https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/ - 404 Client Error: Not Found for url: https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/
(line 5) broken https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/ - 404 Client Error: Not Found for url: https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/
(line 21) broken https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/ - 404 Client Error: Not Found for url: https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/
(line 442) broken https://www.aclweb.org/anthology/P02-1040.pdf)[1 - 404 Client Error: Not Found for url: https://www.aclweb.org/anthology/P02-1040.pdf)%5B1
(line 92) broken https://nlp.stanford.edu/pubs/glove.pdf)[2 - 404 Client Error: NOT FOUND for url: https://nlp.stanford.edu/pubs/glove.pdf)%5B2
(line 208) broken https://www.bioinf.jku.at/publications/older/2604.pdf)[3 - 404 Client Error: Not Found for url: https://www.bioinf.jku.at/publications/older/2604.pdf)%5B3

mli · 2019-07-30T13:56:11Z

Job PR-842/13 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-842/13/index.html

eric-haibin-lin

I'll re-enable the fp16 test in #848

mli · 2019-07-30T16:29:30Z

Found link check problems in job PR-842/15:
(line 19) broken https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/ - 404 Client Error: Not Found for url: https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/
(line 5) broken https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/ - 404 Client Error: Not Found for url: https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/
(line 5) broken https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/ - 404 Client Error: Not Found for url: https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/
(line 21) broken https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/ - 404 Client Error: Not Found for url: https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/
(line 442) broken https://www.aclweb.org/anthology/P02-1040.pdf)[1 - 404 Client Error: Not Found for url: https://www.aclweb.org/anthology/P02-1040.pdf)%5B1
(line 92) broken https://nlp.stanford.edu/pubs/glove.pdf)[2 - 404 Client Error: NOT FOUND for url: https://nlp.stanford.edu/pubs/glove.pdf)%5B2
(line 208) broken https://www.bioinf.jku.at/publications/older/2604.pdf)[3 - 404 Client Error: Not Found for url: https://www.bioinf.jku.at/publications/older/2604.pdf)%5B3

mli · 2019-07-30T16:29:32Z

Job PR-842/15 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-842/15/index.html

Fixes apparent MXNet regression that requires explicit axis keyword

mli · 2019-07-30T18:20:12Z

Found link check problems in job PR-842/16:
(line 19) broken https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/ - 404 Client Error: Not Found for url: https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/
(line 5) broken https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/ - 404 Client Error: Not Found for url: https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/
(line 5) broken https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/ - 404 Client Error: Not Found for url: https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/
(line 21) broken https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/ - 404 Client Error: Not Found for url: https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/
(line 442) broken https://www.aclweb.org/anthology/P02-1040.pdf)[1 - 404 Client Error: Not Found for url: https://www.aclweb.org/anthology/P02-1040.pdf)%5B1
(line 92) broken https://nlp.stanford.edu/pubs/glove.pdf)[2 - 404 Client Error: NOT FOUND for url: https://nlp.stanford.edu/pubs/glove.pdf)%5B2
(line 208) broken https://www.bioinf.jku.at/publications/older/2604.pdf)[3 - 404 Client Error: Not Found for url: https://www.bioinf.jku.at/publications/older/2604.pdf)%5B3

mli · 2019-07-30T18:20:14Z

Job PR-842/16 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-842/16/index.html

szha added 2 commits July 24, 2019 20:57

upgrade mxnet dependency to 1.5.0

562bd8e

fix doc

fbc8fed

szha force-pushed the upgrade_dep branch from c52decc to fbc8fed Compare July 25, 2019 04:15

leezu requested review from eric-haibin-lin and leezu July 29, 2019 08:15

Disable float16 test for scripts/bert/finetune_classifier.py

a22b6e3

leezu approved these changes Jul 29, 2019

View reviewed changes

leezu force-pushed the upgrade_dep branch from d792c55 to 66088f0 Compare July 29, 2019 08:58

leezu force-pushed the upgrade_dep branch from 66088f0 to d792c55 Compare July 29, 2019 09:11

szha commented Jul 30, 2019

View reviewed changes

env/gpu/py3-master.yml Outdated Show resolved Hide resolved

leezu force-pushed the upgrade_dep branch from d792c55 to 58c3871 Compare July 30, 2019 10:08

Use Cuda 10.1 on CI

a22825a

leezu force-pushed the upgrade_dep branch from 58c3871 to b6525db Compare July 30, 2019 10:33

Specify spacy requirement

3c308f8

Previously conda would install an outdated version (1.7) time to time http://ci.mxnet.io/blue/organizations/jenkins/GluonNLP-py3-gpu-integration/detail/PR-842/10/pipeline/

leezu force-pushed the upgrade_dep branch from b6525db to 5667144 Compare July 30, 2019 13:00

Decrease batch_size in word_embeddings evaluation script

7cf1138

Works around an apparent MXNet 1.5 storage manager regression

leezu force-pushed the upgrade_dep branch from 5667144 to 7cf1138 Compare July 30, 2019 13:23

eric-haibin-lin approved these changes Jul 30, 2019

View reviewed changes

leezu force-pushed the upgrade_dep branch from da14621 to fb22fb6 Compare July 30, 2019 15:56

leezu added 2 commits July 30, 2019 17:47

Add axis argument to squeeze()

8cd326f

Fixes apparent MXNet regression that requires explicit axis keyword

Disable test_finetune_chinese_inference test

8f2361f

leezu force-pushed the upgrade_dep branch from fb22fb6 to 8f2361f Compare July 30, 2019 17:47

leezu merged commit 5a5028c into dmlc:master Jul 30, 2019

leezu mentioned this pull request Jul 30, 2019

[CI] Use Cuda 10.1 #854

Closed

5 tasks

szha deleted the upgrade_dep branch July 30, 2019 21:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[DOC] upgrade mxnet dependency to 1.5.0 #842

[DOC] upgrade mxnet dependency to 1.5.0 #842

szha commented Jul 21, 2019

codecov bot commented Jul 21, 2019

codecov bot commented Jul 21, 2019 •

edited

Loading

mli commented Jul 22, 2019

mli commented Jul 22, 2019

leezu commented Jul 22, 2019

eric-haibin-lin commented Jul 22, 2019

mli commented Jul 25, 2019

mli commented Jul 25, 2019

mli commented Jul 29, 2019

mli commented Jul 29, 2019

leezu commented Jul 29, 2019

mli commented Jul 30, 2019

mli commented Jul 30, 2019

eric-haibin-lin left a comment

mli commented Jul 30, 2019

mli commented Jul 30, 2019

mli commented Jul 30, 2019

mli commented Jul 30, 2019

[DOC] upgrade mxnet dependency to 1.5.0 #842

[DOC] upgrade mxnet dependency to 1.5.0 #842

Conversation

szha commented Jul 21, 2019

Description

Checklist

Essentials

Changes

codecov bot commented Jul 21, 2019

Codecov Report

codecov bot commented Jul 21, 2019 • edited Loading

Codecov Report

mli commented Jul 22, 2019

mli commented Jul 22, 2019

leezu commented Jul 22, 2019

eric-haibin-lin commented Jul 22, 2019

mli commented Jul 25, 2019

mli commented Jul 25, 2019

mli commented Jul 29, 2019

mli commented Jul 29, 2019

leezu commented Jul 29, 2019

mli commented Jul 30, 2019

mli commented Jul 30, 2019

eric-haibin-lin left a comment

Choose a reason for hiding this comment

mli commented Jul 30, 2019

mli commented Jul 30, 2019

mli commented Jul 30, 2019

mli commented Jul 30, 2019

codecov bot commented Jul 21, 2019 •

edited

Loading