Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is it possible to speed it up without printing a certain line #29

Open
IS5882 opened this issue Sep 11, 2019 · 1 comment
Open

Is it possible to speed it up without printing a certain line #29

IS5882 opened this issue Sep 11, 2019 · 1 comment

Comments

@IS5882
Copy link

IS5882 commented Sep 11, 2019

Thank so much for your code, its helping me in my project.
But I am facing a small issue not sure if we can fix it or not:

I am embedding (using XLNET) 3 inputs each input has 5400 sentences and each sentence has 20 words
so the total is 3 * 5400 * 20

2 hours has passed till now and it still didn't finish embedding the first input (5400 *20)
I think I will run out of memory in Google Colab as well before finishing the embedding part [ofcourse this happened 🙈 it didn't even finish embedding the 1st input]

I notice that with each embedding its printing

"Converting texts to features: 100%|██████████| 20/20 [00:00<00:00, 17637.95it/s]INFO:tensorflow:memory input None
INFO:tensorflow:Use float type <dtype: 'float32'>" 

is it possible to avoid printing that line with each embedding to speed up the process ?

@ashutoshsingh0223
Copy link
Collaborator

Printing that progressbar is not delaying your code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants