Skip to content
This repository has been archived by the owner on Jan 15, 2024. It is now read-only.

[Enhancement] Add whole word masking for BERT #770

Merged
merged 12 commits into from
Jun 26, 2019

Conversation

basicv8vc
Copy link
Member

@basicv8vc basicv8vc commented Jun 14, 2019

Description

Add whole word masking for BERT.

referred to bert link

Checklist

Essentials

  • PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
  • Changes are complete (i.e. I finished coding on this PR)
  • All changes have test coverage
  • Code is well-documented

Changes

  • Feature1, tests, (and when applicable, API doc)
  • Feature2, tests, (and when applicable, API doc)

Comments

  • If this change is a backward incompatible change, why must this change be made.
  • Interesting edge cases to note here

@basicv8vc basicv8vc requested a review from szha as a code owner June 14, 2019 08:06
@codecov
Copy link

codecov bot commented Jun 14, 2019

Codecov Report

Merging #770 into master will decrease coverage by 0.08%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff            @@
##           master    #770      +/-   ##
=========================================
- Coverage   90.49%   90.4%   -0.09%     
=========================================
  Files          66      66              
  Lines        6364    6370       +6     
=========================================
  Hits         5759    5759              
- Misses        605     611       +6
Impacted Files Coverage Δ
src/gluonnlp/data/transforms.py 80.51% <100%> (+0.23%) ⬆️
src/gluonnlp/data/dataloader.py 83.62% <0%> (-5.18%) ⬇️

@mli
Copy link
Member

mli commented Jun 14, 2019

Job PR-770/1 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-770/1/index.html

@mli
Copy link
Member

mli commented Jun 14, 2019

Job PR-770/2 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-770/2/index.html

@mli
Copy link
Member

mli commented Jun 15, 2019

Job PR-770/3 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-770/3/index.html

@mli
Copy link
Member

mli commented Jun 15, 2019

Job PR-770/4 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-770/4/index.html

paperplanet
paperplanet approved these changes Jun 16, 2019
@mli
Copy link
Member

mli commented Jun 17, 2019

Job PR-770/5 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-770/5/index.html

Copy link
Member

@eric-haibin-lin eric-haibin-lin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great contribution! Let me know if my comment makes sense or not

scripts/bert/create_pretraining_data.py Show resolved Hide resolved
scripts/bert/create_pretraining_data.py Outdated Show resolved Hide resolved
scripts/bert/create_pretraining_data.py Outdated Show resolved Hide resolved
@mli
Copy link
Member

mli commented Jun 25, 2019

Job PR-770/10 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-770/10/index.html

@mli
Copy link
Member

mli commented Jun 26, 2019

Job PR-770/11 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-770/11/index.html

@mli
Copy link
Member

mli commented Jun 26, 2019

Job PR-770/12 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-770/12/index.html

@eric-haibin-lin eric-haibin-lin merged commit 84738d9 into dmlc:master Jun 26, 2019
@eric-haibin-lin
Copy link
Member

@basicv8vc thanks for the contribution!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants