[SCRIPT][API] Add RoBERTa fine-tuning scripts, add BERTClassifier to API

[eric-haibin-lin]

Description

• add roberta argument options to the finetune script
• move BERTClassifier to the official API

@hhexiy

Checklist

Essentials

• PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
• Changes are complete (i.e. I finished coding on this PR)
• All changes have test coverage
• Code is well-documented

Changes

• Feature1, tests, (and when applicable, API doc)
• Feature2, tests, (and when applicable, API doc)

Comments

• If this change is a backward incompatible change, why must this change be made.
• Interesting edge cases to note here

[codecov]

Codecov Report

Merging #931 into master will increase coverage by 0.8%.
The diff coverage is 36.36%.

@@           Coverage Diff            @@
##           master    #931     +/-   ##
========================================
+ Coverage   88.89%   89.7%   +0.8%     
========================================
  Files          67      67             
  Lines        6360    6408     +48     
========================================
+ Hits         5654    5748     +94     
+ Misses        706     660     -46

Impacted Files	Coverage Δ
src/gluonnlp/model/bert.py	84.95% <19.51%> (-14.51%)	⬇️
src/gluonnlp/data/transforms.py	76.92% <85.71%> (-4.68%)	⬇️
src/gluonnlp/model/parameter.py	92% <0%> (-8%)	⬇️
src/gluonnlp/data/corpora/wikitext.py	94.82% <0%> (-5.18%)	⬇️
src/gluonnlp/data/batchify/batchify.py	93.18% <0%> (-3.41%)	⬇️
src/gluonnlp/data/dataset.py	97.61% <0%> (-1.59%)	⬇️
src/gluonnlp/data/word_embedding_evaluation.py	96.21% <0%> (-0.76%)	⬇️
src/gluonnlp/vocab/subwords.py	86.95% <0%> (+2.17%)	⬆️
src/gluonnlp/model/sequence_sampler.py	91.63% <0%> (+17.07%)	⬆️
... and 1 more

[mli]

Job PR-931/1 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-931/1/index.html

[mli]

Job PR-931/2 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-931/2/index.html

[mli]

Job PR-931/4 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-931/4/index.html

[kaonashi-tyc]

RoBERTa model has a slightly different Classifier structure by default (assuming the fairseq as the official implementation):

https://github.com/pytorch/fairseq/blob/718677ebb044e27aaf1a30640c2f7ab6b8fa8509/fairseq/models/roberta/model.py#L218-L235

Might deserve its own Classifier of sort

[mli]

Job PR-931/9 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-931/9/index.html

[mli]

Job PR-931/10 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-931/10/index.html

[mli]

Job PR-931/11 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-931/11/index.html

[mli]

Job PR-931/12 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-931/12/index.html

[szhengac]

What is this RoBERT? Any reference?

[mli]

Job PR-931/13 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-931/13/index.html

[eric-haibin-lin]

https://arxiv.org/abs/1907.11692

[mli]

Job PR-931/15 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-931/15/index.html

[mli]

Job PR-931/16 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-931/16/index.html

[leezu]

It is not clear that different vocabularies are required/handled for different BERT style models. Let's document that cls_token, sep_token is used if available and otherwise fallback to bos_token, eos_token.

To formally specify the expected attributes ( cls_token, sep_token, etc.) one could (eventually) use Structural subtyping https://mypy.readthedocs.io/en/latest/protocols.html

[kaonashi-tyc]

Nit

[mli]

Job PR-931/17 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-931/17/index.html