Skip to content
This repository has been archived by the owner on Jan 15, 2024. It is now read-only.

[API][BUGFIX] remove extra dropout in BERT/RoBERTa #1022

Merged
merged 3 commits into from
Dec 13, 2019

Conversation

eric-haibin-lin
Copy link
Member

@eric-haibin-lin eric-haibin-lin commented Nov 26, 2019

Description

dropout is already applied after the embedding is added with positional embedding

Checklist

Essentials

  • PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
  • Changes are complete (i.e. I finished coding on this PR)
  • All changes have test coverage
  • Code is well-documented

Changes

  • Feature1, tests, (and when applicable, API doc)
  • Feature2, tests, (and when applicable, API doc)

Comments

  • If this change is a backward incompatible change, why must this change be made.
  • Interesting edge cases to note here

cc @dmlc/gluon-nlp-team

@eric-haibin-lin eric-haibin-lin requested a review from a team as a code owner November 26, 2019 06:09
@codecov
Copy link

codecov bot commented Nov 26, 2019

Codecov Report

Merging #1022 into master will increase coverage by 0.22%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1022      +/-   ##
==========================================
+ Coverage   88.05%   88.27%   +0.22%     
==========================================
  Files          67       67              
  Lines        6254     6252       -2     
==========================================
+ Hits         5507     5519      +12     
+ Misses        747      733      -14
Impacted Files Coverage Δ
src/gluonnlp/model/bert.py 92.18% <100%> (-0.07%) ⬇️
src/gluonnlp/data/word_embedding_evaluation.py 96.94% <0%> (+0.76%) ⬆️
src/gluonnlp/data/batchify/batchify.py 95.27% <0%> (+0.78%) ⬆️
src/gluonnlp/data/transforms.py 82.58% <0%> (+1.12%) ⬆️
src/gluonnlp/data/dataset.py 99.2% <0%> (+1.58%) ⬆️
src/gluonnlp/data/corpora/wikitext.py 100% <0%> (+5.17%) ⬆️
src/gluonnlp/model/parameter.py 100% <0%> (+8%) ⬆️

@eric-haibin-lin eric-haibin-lin changed the title remove extra dropout in BERT/RoBERTa [WIP] remove extra dropout in BERT/RoBERTa Nov 26, 2019
@mli
Copy link
Member

mli commented Nov 26, 2019

Job PR-1022/1 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-1022/1/index.html

@eric-haibin-lin eric-haibin-lin changed the title [WIP] remove extra dropout in BERT/RoBERTa [API][BUGFIX] remove extra dropout in BERT/RoBERTa Dec 12, 2019
@mli
Copy link
Member

mli commented Dec 12, 2019

Job PR-1022/2 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-1022/2/index.html

@szha szha added the release focus Progress focus for release label Dec 13, 2019
@eric-haibin-lin eric-haibin-lin added API change bug Something isn't working labels Dec 13, 2019
@eric-haibin-lin eric-haibin-lin merged commit aff2921 into dmlc:master Dec 13, 2019
@eric-haibin-lin eric-haibin-lin deleted the dropout branch February 2, 2020 06:21
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
API change bug Something isn't working release focus Progress focus for release
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants