add basic support for internvl2-4b

[MeouSker77]

Description

add basic support for internvl2-4b

1. Why the change?

2. User API changes

test with transformers 4.38.0

A simple example:

pip install lmdeploy timm

import time
import torch

model_path = 'InternVL2-4B'

from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_cod=True)

from ipex_llm.transformers import AutoModelForCausalLM
# use "sym_int8" for better output, use "sym_int4" for better speed
model = AutoModelForCausalLM.from_pretrained(model_path, trust_remote_code=True,
                                             load_in_low_bit="sym_int8", modules_to_not_convert=["vision_model"])
model = model.half().eval()
model = model.to('xpu')

from lmdeploy.vl import load_image
from transformers import CLIPImageProcessor
image_processor = CLIPImageProcessor.from_pretrained(model_path)
# image = load_image('https://github.com/raw/open-mmlab/mmdeploy/main/tests/data/tiger.jpeg')
image = load_image("tiger.jpeg")
pixel_values = image_processor(images=[image], return_tensors='pt').pixel_values
pixel_values = pixel_values.to('xpu')

generation_config = {
    "max_new_tokens": 32,
    "do_sample": False,
}

question = "<image>What's in the picture?"

with torch.inference_mode():
    for i in range(5):
        st = time.time()
        response = model.chat(
            tokenizer=tokenizer,
            pixel_values=pixel_values,
            question=question,
            generation_config=generation_config,
            history=[]
        )
        et = time.time()
        print(response)
        print(et - st)

see this for more examples and usage

for now, lmdeploy usage is not supported

3. Summary of the change

4. How to test?

• N/A
• Unit test: Please manually trigger the PR Validation here by inputting the PR number (e.g., 1234). And paste your action link here once it has been successfully finished.
• Application test
• Document test
• ...

[MeouSker77]

PR validation: https://github.com/intel-analytics/ipex-llm-workflow/actions/runs/10259726207