-
-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Memory Error during import_dat #18
Comments
How many images are there in your directory ? |
i tried with 30000-50000 images, seem like the issue is with scipy lapack and the specific error can be reproduced via this managed to find some solutions, the workaround is to switch to lapack driver from 'gesdd' to 'gesvd' in scipy another way i am trying is to use the intel-scipy python modules via pip not to sure about the accuracy or performance after these change but clustimage is no longer reporting the error. |
Great. Nice to hear that you find a solution. However, I am not sure why it states that there is a mismatch in |
Hi,
i am encountering a Memory error during the import_data step. After loading the images it throws a Memory Error. Anyway i can figure out what is the issue.
Traceback (most recent call last):
File "/home/stg/prod/combine_clustimage.py", line 56, in
results = cl.fit_transform(targetdir)
File "/home/stg/.local/lib/python3.10/site-packages/clustimage/clustimage.py", line 352, in fit_transform
_ = self.import_data(X, black_list=black_list)
File "/home/stg/.local/lib/python3.10/site-packages/clustimage/clustimage.py", line 992, in import_data
X = self.preprocessing(Xraw['pathnames'], grayscale=self.params['cv2_imread_colorscale'], dim=self.params['dim'], flatten=flatten)
File "/home/stg/.local/lib/python3.10/site-packages/clustimage/clustimage.py", line 806, in preprocessing
img, imgOK = zip(*imgs)
MemoryError
The text was updated successfully, but these errors were encountered: