Add notebook for text classification with BERT and text classification tokenization caching #382
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
This pull request introduces 3 alerts when merging 0bc3c7a into abaac1b - view on LGTM.com new alerts:
|
|
can you please sign your commits first? The DCO's bot output shows you how it can be done https://github.com/NVIDIA/NeMo/pull/382/checks?check_run_id=454165747 |
|
Okay, I was able to sign off on the commits. |
ekmb
changed the title
|
This pull request introduces 3 alerts when merging 3e2a987 into abaac1b - view on LGTM.com new alerts:
|
|
@ericharper there were some changes with how NLP collection is organized (and I think, for now, we are done with changes there). |
…ded text_classification_with_bert notebooks Signed-off-by: ericharper <complex451@gmail.com> changed name text_classification to text_classification_with_bert Signed-off-by: ericharper <complex451@gmail.com> Delete text_classification_inference.ipynb Signed-off-by: ericharper <complex451@gmail.com> Delete text_classification.ipynb Signed-off-by: ericharper <complex451@gmail.com> added preproc text data layer and dataset and text classification notebook changed name text_classification to text_classification_with_bert Signed-off-by: ericharper <complex451@gmail.com> renamed text_classification_notebook Signed-off-by: ericharper <complex451@gmail.com> renamed text_classification_inference Signed-off-by: ericharper <complex451@gmail.com> add use_cache flag to BertTextClassificationDataset and BertTextClassificationDatalayer and update text_classification_with_bert script and notebooks Signed-off-by: ericharper <complex451@gmail.com> removed preproc_text_data.py Signed-off-by: ericharper <complex451@gmail.com>
|
Rebased to master. |
Signed-off-by: ericharper <complex451@gmail.com>
|
looks fine to me, @ekmb can you please have a look too? @ericharper is this still WIP? |
|
finished on my end, but happy to make more changes if needed |
ekmb
reviewed
Feb 24, 2020
ekmb
changed the title
ekmb
approved these changes
Feb 25, 2020
|
@ericharper Thank you for your contribution! |
dcurran90
pushed a commit
to dcurran90/NeMo
that referenced
this pull request
Oct 15, 2024
* combine lab-troubleshooting.md and output optimization doc Signed-off-by: jaideepr97 <jaideep.r97@gmail.com> * Update README.md Co-authored-by: Kelly Brown <86735520+kelbrown20@users.noreply.github.com> Signed-off-by: Jaideep Rao <jaideep.r97@gmail.com> * Update TROUBLESHOOTING.md Co-authored-by: Kelly Brown <86735520+kelbrown20@users.noreply.github.com> Signed-off-by: Jaideep Rao <jaideep.r97@gmail.com> * Update TROUBLESHOOTING.md Co-authored-by: Kelly Brown <86735520+kelbrown20@users.noreply.github.com> Signed-off-by: Jaideep Rao <jaideep.r97@gmail.com> * Update TROUBLESHOOTING.md Co-authored-by: Kelly Brown <86735520+kelbrown20@users.noreply.github.com> Signed-off-by: Jaideep Rao <jaideep.r97@gmail.com> * Update TROUBLESHOOTING.md Co-authored-by: Kelly Brown <86735520+kelbrown20@users.noreply.github.com> Signed-off-by: Jaideep Rao <jaideep.r97@gmail.com> * Update TROUBLESHOOTING.md Co-authored-by: Kelly Brown <86735520+kelbrown20@users.noreply.github.com> Signed-off-by: Jaideep Rao <jaideep.r97@gmail.com> * Update TROUBLESHOOTING.md Co-authored-by: Kelly Brown <86735520+kelbrown20@users.noreply.github.com> Signed-off-by: Jaideep Rao <jaideep.r97@gmail.com> * address review comments Signed-off-by: jaideepr97 <jaideep.r97@gmail.com> * Update TROUBLESHOOTING.md Co-authored-by: Kelly Brown <86735520+kelbrown20@users.noreply.github.com> Signed-off-by: Jaideep Rao <jaideep.r97@gmail.com> * Update TROUBLESHOOTING.md Co-authored-by: Kelly Brown <86735520+kelbrown20@users.noreply.github.com> Signed-off-by: Jaideep Rao <jaideep.r97@gmail.com> * address review comment Signed-off-by: jaideepr97 <jaideep.r97@gmail.com> * update rouge threshold value in example Signed-off-by: jaideepr97 <jaideep.r97@gmail.com> --------- Signed-off-by: jaideepr97 <jaideep.r97@gmail.com> Signed-off-by: Jaideep Rao <jaideep.r97@gmail.com> Co-authored-by: Kelly Brown <86735520+kelbrown20@users.noreply.github.com>
dcurran90
pushed a commit
to dcurran90/NeMo
that referenced
this pull request
Oct 15, 2024
PR NVIDIA#382 moved the content from lab-troubleshoot.md into TROUBLESHOOTING.md but forgot to delete the lab-troubleshoot.md file. This cleans that up. Signed-off-by: Ben Browning <bbrownin@redhat.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add notebooks detailing text classification with BERT using NeMo.
text_classification_with_bert.ipynb
text_classification_with_bert_inference.ipynb
We also add a script that converts the text classification tokenization to hdf5: preproc_text_data.py
We then modify the text classification dataset to include a preproc version and we modify the text classification data layer to also have a preproc version.
The text classification preproc data layer is similar to bert pretraining preproc data layer
We found it more efficient to cache the tokenization when training. We were able to get better training times and better scaling when using multiple GPUs.
The notebooks also visualize the BERT embeddings before and after fine-tuning by applying T-SNE.
The inference notebooks allow the user to see how well the fine-tuned models are classifying their data.
We also include a script to download the SST-2 dataset for use in the notebook.
We changed the text classification dataset to use the user specified max seq length.