-
Notifications
You must be signed in to change notification settings - Fork 28
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Batch size does not decrease encoding time. #20
Comments
I found a few issues in your code that I'm not sure if you have passed the batch parameter correctly. init_settings, encode_settings = geosam.split_settings(settings) This line doesn't seem to be used to encode image in your code. Besides, if you are using parameters from a file, you need to update the batch in settings: settings.update({"batch_size":i})
init_settings, encode_settings = geosam.split_settings(settings)
So, I suggest changing it to: times = {}
img_encoder = ImageEncoder(ckpt)
for i in range(1,5):
start_time = time.time()
img_encoder.batch_size=i
img_encoder.encode_image(image_path, feature_dir)
times[i] = time.time() - start_time Regarding increasing the batch size, the encoding time did not decrease. There are many similar discussions online, and it seems that there is no consensus. https://discuss.pytorch.org/t/increasing-batch-size-didnt-help-reduce-training-time/100960 https://discuss.pytorch.org/t/increasing-batch-size-didnt-help-reduce-training-time/100960 As for SAM, I think this may be because SAM is a large model, and one batch has already occupied all the CUDA cores of your computer. Increasing the batch size in memory will only reduce the number of I/O operations and cannot reduce the model’s running time. In our early tests on A100, increasing the batch size could reduce the running time. You can check if one batch will make your GPU utilization close to 100%. (Note: check GPU utilization not memory usage) In your case, increasing the batch size seems to increase the running time. I guess this may be because our code does not clear the data in the GPU after encoding the image, which causes insufficient GPU memory and a certain amount of I/O with the computer’s memory to prevent memory explosion, thereby reducing the subsequent running speed. We have updated GeoSAM-Image-Encoder to clear memory after each image encoding. We recommend testing it again after reinstalling it using the following code: pip install GeoSAM-Image-Encoder --upgrade
# or
pip install GeoSAM-Image-Encoder==1.0.4 |
Thanks for your feedback. We will test this issue and try to find the problems. |
Hi @4del-Yousefi , My previous guess should be correct based on testing on A100. Even on A100 (6912 cuda cores), when the batch size is increased to 2, the cuda utilization has reached 99%. Increasing the batch size further only puts more data into the GPU, but the GPU cuda cores have been fully utilized and cannot reduce the overall running speed. Therefore, only when the number of cuda cores is extremely large can a significant decrease in runtime be observed when increasing batch size. Below are my test code and running times: ## Download example dataset and sam `vit_l` checkpoint
wget https://github.com/raw/coolzhao/Geo-SAM/main/rasters/beiluhe_google_img_201211_clip.tif
wget https://dl.fbaipublicfiles.com/segment_anything/sam_vit_l_0b3195.pth
wget https://github.com/raw/coolzhao/Geo-SAM/dev/GeoSAM-Image-Encoder/examples/data/setting.json import time
import geosam
from geosam import ImageEncoder
checkpoint_path = './sam_vit_l_0b3195.pth'
image_path = './beiluhe_google_img_201211_clip.tif'
feature_dir = './'
## init ImageEncoder
img_encoder = ImageEncoder(checkpoint_path)
## test
times = {}
for i in range(1,5):
start_time = time.time()
img_encoder.batch_size=i
img_encoder.encode_image(image_path, feature_dir)
times[i] = time.time() - start_time
print(times) {1: 5.970345973968506, |
Hello, Thanks for your update, I understand how it works now. Also just a small question can Light HQ-SAM or any other SAM based models increase the speed or affect how batch size affect the process speed? Thanks, |
Hello,
I am using the plugin and python package to encode and noticed that increasing the batch size doesn't increase the encoding speed.
I have 4070ti and I got the following results.
Batch size: Time in sec
1: 141.44510889053345
2: 279.85611724853516
3: 424.3098211288452
4: 425.72585558891296
Code I used to get the result:
times = {}
for i in range(1,5):
start_time = time.time()
img_encoder = ImageEncoder(ckpt, batch_size=i)
img_encoder.encode_image(image_path,feature_dir)
init_settings, encode_settings = geosam.split_settings(settings)
times[i] = time.time() - start_time
The text was updated successfully, but these errors were encountered: