Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

内存溢出 #28

Open
102757017 opened this issue Jun 5, 2021 · 6 comments
Open

内存溢出 #28

102757017 opened this issue Jun 5, 2021 · 6 comments

Comments

@102757017
Copy link

我在google colab中运行该代码,程序运行了一半就异常终止了。内存分配有问题?

/usr/local/lib/python3.7/dist-packages/torchvision/transforms/transforms.py:258: UserWarning: Argument interpolation should be of type InterpolationMode instead of int. Please, use InterpolationMode enum.
"Argument interpolation should be of type InterpolationMode instead of int. "
| feature extraction | train | bottle |: 100% 7/7 [01:13<00:00, 10.52s/it]
tcmalloc: large alloc 2013462528 bytes == 0x559a8ac86000 @ 0x7fb9c58f4b6b 0x7fb9c5914379 0x7fb9452dc25e 0x7fb9452dd9d2 0x7fb981fc08e6 0x7fb982422dd9 0x7fb98292d77a 0x7fb9828f8ef9 0x7fb9828af657 0x7fb982753929 0x7fb98242690c 0x7fb982a34209 0x7fb982a3448a 0x7fb982908239 0x7fb9828bc31e 0x7fb98278f4fb 0x7fb993b45823 0x5599bf2bdcc0 0x5599bf2bda50 0x5599bf331be0 0x5599bf2bf30a 0x5599bf32d3b5 0x5599bf32c7ad 0x5599bf2bf3ea 0x5599bf32d3b5 0x5599bf32c4ae 0x5599bf32c1b3 0x5599bf3f6182 0x5599bf3f64fd 0x5599bf3f63a6 0x5599bf3cd723
tcmalloc: large alloc 2013462528 bytes == 0x559b02cb6000 @ 0x7fb9c58f4b6b 0x7fb9c5914379 0x7fb9452dc25e 0x7fb9452dd9d2 0x7fb981fc08e6 0x7fb982422dd9 0x7fb98292d77a 0x7fb9828f8ef9 0x7fb9828af657 0x7fb982753929 0x7fb98242c6a2 0x7fb982a2a882 0x7fb9828b0609 0x7fb982755e7e 0x7fb982177c46 0x7fb982933310 0x7fb98281e5cc 0x7fb983f544c0 0x7fb983f54944 0x7fb98281e5cc 0x7fb993d5dfa3 0x5599bf2bdcc0 0x5599bf2bda50 0x5599bf331be0 0x5599bf32c4ae 0x5599bf2bf3ea 0x5599bf32e32a 0x5599bf2bf30a 0x5599bf32d3b5 0x5599bf32c7ad 0x5599bf2bf3ea
tcmalloc: large alloc 2013462528 bytes == 0x559a62c76000 @ 0x7fb9c58f4b6b 0x7fb9c5914379 0x7fb9452dc25e 0x7fb9452dd9d2 0x7fb981fc08e6 0x7fb982422dd9 0x7fb98292d77a 0x7fb9828f8ef9 0x7fb9828af657 0x7fb982753929 0x7fb98242c6a2 0x7fb982a2a882 0x7fb9828b0609 0x7fb982755e7e 0x7fb982236586 0x7fb9829333ae 0x7fb98281f607 0x7fb983f58bc8 0x7fb983f5904c 0x7fb98281f607 0x7fb993d55532 0x5599bf2bdcc0 0x5599bf2bda50 0x5599bf331be0 0x5599bf32c4ae 0x5599bf2bf3ea 0x5599bf32e32a 0x5599bf2bf30a 0x5599bf32d3b5 0x5599bf32c7ad 0x5599bf2bf3ea
tcmalloc: large alloc 4698079232 bytes == 0x559ba563c000 @ 0x7fb9c58f4b6b 0x7fb9c5914379 0x7fb9452dc25e 0x7fb9452dd9d2 0x7fb981fc08e6 0x7fb982422dd9 0x7fb98292d77a 0x7fb9828f8ef9 0x7fb9828af657 0x7fb982753929 0x7fb98242690c 0x7fb982a34209 0x7fb982a3448a 0x7fb982908239 0x7fb9828bc31e 0x7fb98278f4fb 0x7fb993b45823 0x5599bf2bdcc0 0x5599bf2bda50 0x5599bf331be0 0x5599bf2bf30a 0x5599bf32d3b5 0x5599bf32c7ad 0x5599bf2bf3ea 0x5599bf32d3b5 0x5599bf32c4ae 0x5599bf32c1b3 0x5599bf3f6182 0x5599bf3f64fd 0x5599bf3f63a6 0x5599bf3cd723
tcmalloc: large alloc 4698079232 bytes == 0x559cbd6ac000 @ 0x7fb9c58f4b6b 0x7fb9c5914379 0x7fb9452dc25e 0x7fb9452dd9d2 0x7fb981fc08e6 0x7fb982422dd9 0x7fb98292d77a 0x7fb9828f8ef9 0x7fb9828af657 0x7fb982753929 0x7fb98242c6a2 0x7fb982a2a882 0x7fb9828b0609 0x7fb982755e7e 0x7fb982177c46 0x7fb982933310 0x7fb98281e5cc 0x7fb983f544c0 0x7fb983f54944 0x7fb98281e5cc 0x7fb993d5dfa3 0x5599bf2bdcc0 0x5599bf2bda50 0x5599bf331be0 0x5599bf32c4ae 0x5599bf2bf3ea 0x5599bf32e32a 0x5599bf2bf30a 0x5599bf32d3b5 0x5599bf32c7ad 0x5599bf2bf3ea
^C

@102757017 102757017 changed the title 程序异常终止 内存溢出 Jun 5, 2021
@jackft2
Copy link

jackft2 commented Jun 7, 2021

i met the same problem.
how can we solve this problem

@ksl986735244
Copy link

change the batchsize
修改一下batch大小就行了,batch改小点

@102757017
Copy link
Author

我将main.py中以下代码的batch_size分别改为16,8,4进行了尝试,结果还是产生了内存溢出,将pin_memory设置为False也不起作用。
train_dataset = mvtec.MVTecDataset(args.data_path, class_name=class_name, is_train=True)
train_dataloader = DataLoader(train_dataset, batch_size=32, pin_memory=True)
test_dataset = mvtec.MVTecDataset(args.data_path, class_name=class_name, is_train=False)
test_dataloader = DataLoader(test_dataset, batch_size=32, pin_memory=True)

@Yangly0
Copy link

Yangly0 commented Jul 20, 2021

设置成1吧 我都是设置成1来运行的

@ksl986735244
Copy link

我用的cpu为i7-10750h,内存16G,改小batch_size后内存溢出的问题就消失了

@RichardChangCA
Copy link

换个大点数据集应该直接不能用了

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants