Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Got 'Resource temporarily unavailable' using docker #32

Open
ByZ0e opened this issue Sep 27, 2021 · 2 comments
Open

Got 'Resource temporarily unavailable' using docker #32

ByZ0e opened this issue Sep 27, 2021 · 2 comments

Comments

@ByZ0e
Copy link

ByZ0e commented Sep 27, 2021

Hi,
I always got 'runtime/cgo: pthread_create failed: Resource temporarily unavailable' error when using docker.
And the docker process cannot stop itself, I need to use sudo to kill the process, which is very inconvenient.
What's more, I found that saving the code and backup checkpoints needs very large memory space(~GB) which may cause the above error.
Any suggestions for this error? Thanks a lot!

@jayleicn
Copy link
Owner

Hi @zoe-ziyi,

There is no need to backup checkpoints, our code is only intended to backup the source code. The checkpoint files are rather large ~1-2GB. You should probably move the checkpoint directory out of the source code directory to prevent it from being backup-ed. Hope this helps!

Best,
Jie

@ByZ0e
Copy link
Author

ByZ0e commented Sep 30, 2021

Thanks for your reply. In fact, I found the real possible cause of this error is that the user's number of processes (can reach 3W+) exceeds the limit. However, my root user's max process number is unlimited. I set the num_worker of the dataloader to 0, which finally solved the problem. Do you have any better solution, please?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants