Can't run with multiple gpu

[volcverse]

Hello, thanks for the great work!

I refer to the example_code/example_chat.py to run the newest InternLM-XComposer-2.5 model using 4 NVIDIA 4090 GPUs. But still meet the OOM problem. It seems that although the weights are divided successfully, the first gpu always runs into OOM when model.chat is called.

Any response will be greatly appreciated!

[Uoops]

same

[hyyuan123]

I refer to the example_code/example_chat.py to run the newest InternLM-XComposer-2.5 model using 4 A800 GPUs. But still meet the OOM problem.

[waltonfuture]

same question

[yhcao6]

Please try to install transformers==4.33.1 with the following command and try again:

pip install transformers==4.33.1

[YerongLi]

Hello, thanks for the great work!

I refer to the example_code/example_chat.py to run the newest InternLM-XComposer-2.5 model using 4 NVIDIA 4090 GPUs. But still meet the OOM problem. It seems that although the weights are divided successfully, the first gpu always runs into OOM when model.chat is called.

Any response will be greatly appreciated!

I found the model cannot take multiple images as inputs, neither can it take a list of images thus the fix is

• Change the image input to a singleton:
  

  InternLM-XComposer/example_code/example_chat.py

  Line 33 in 99a56be

  image = ['./examples/dubai.png']

image = './examples/dubai.png'

• Add <ImageHere> Flag

  InternLM-XComposer/example_code/example_chat.py

  Line 32 in 99a56be

  query = 'Analyze the given image in a detail manner'

query = '<ImageHere>Please describe this image'

[resi1ience]

Please try to install transformers==4.33.1 with the following command and try again:

pip install transformers==4.33.1

Still meet the same problem with transformers 4.33.1. I'm running the video understanding example on the huggingface.
Response will be greatly appreciated