Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

High GPU usage #421

Open
cnavarrete opened this issue Jan 17, 2023 · 0 comments
Open

High GPU usage #421

cnavarrete opened this issue Jan 17, 2023 · 0 comments

Comments

@cnavarrete
Copy link

cnavarrete commented Jan 17, 2023

I'm running the docker and I can barely run 16 epochs on my 12GB NVIDIA TITAN X (Pascal) before it crashes with your classic cuda out of memory error.

I ran the training with the following command.

python3 train.py --weights yolov4.weights --data /datasets/anabalon_osses_final_vis/data.yaml --epochs 350 --img-size 640 --batch-size 2 --cfg cfg/yolov4.cfg  --device 1

The error

RuntimeError: CUDA out of memory. Tried to allocate 4.46 GiB (GPU 0; 11.91 GiB total capacity; 943.14 MiB already allocated; 3.48 GiB free; 7.78 GiB reserved in total by PyTorch)

I was seriously taken aback by the high gpu consumption. Anyone knows the cause? I haven't had this problem with yolov3 (ultralycs repo), v5 or v7.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant