[Bug] 会话时使用图像沟通报错

[kylyzhh]

📦 部署环境

Vercel

📌 软件版本

1.20.5

💻 系统环境

macOS

🌐 浏览器

Edge

🐛 问题描述

上传图片进行对话的时候返回报错信息如下
{
"error": {
"code": "invalid_image",
"type": "invalid_request_error",
"param": null,
"message": "Invalid image."
},
"endpoint": "https://api.openai.com/v1",
"provider": "openai"
}

📷 复现步骤

上传任意图片进行对话即可触发此报错

🚦 期望结果

希望正常识别图像

📝 补充信息

测试模型为GPT-4所有模型均有此类报错
服务搭建在vercel，根据文档完成了所有配置 https://lobehub.com/zh/docs/self-hosting/server-database/vercel

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

📦 Deployment environment

Vercel

📌 Software version

1.20.5

💻 System environment

macOS

🌐 Browser

Edge

🐛 Problem description

When uploading pictures for conversation, the error message returned is as follows:
{
"error": {
"code": "invalid_image",
"type": "invalid_request_error",
"param": null,
"message": "Invalid image."
},
"endpoint": "https://api.openai.com/v1",
"provider": "openai"
}

📷 Steps to reproduce

Uploading any picture for conversation can trigger this error.

🚦 Expected results

Hope the image can be recognized normally

📝 Supplementary information

The test model is GPT-4 and all models have such error reports.
The service is built on vercel, and all configurations are completed according to the documentation https://lobehub.com/zh/docs/self-hosting/server-database/vercel

[lobehubbot]

👀 @kylyzhh

Thank you for raising an issue. We will investigate into the matter and get back to you as soon as possible.
Please make sure you have given us as much context as possible.
非常感谢您提交 issue。我们会尽快调查此事，并尽快回复您。 请确保您已经提供了尽可能多的背景信息。

[dosubot]

Hey @kylyzhh! I'm here to help you with any bugs, questions, or contributions you have. Let's squash those bugs together! 🤖

I found a similar closed issue that might be relevant to your problem: [Bug] 非 gpt-4-vision-preview 模型传图格式. The suggested solution was to ensure that the image URL is correctly parsed and converted to a base64 format. This involves using utility functions like parseDataUri and imageUrlToBase64 to handle the image data properly.

Here is a code snippet that demonstrates how to handle image URLs:

case 'image_url': {
  const { mimeType, base64, type } = parseDataUri(content.image_url.url);

  if (type === 'base64')
    return {
      source: {
        data: base64 as string,
        media_type: mimeType as Anthropic.ImageBlockParam.Source['media_type'],
        type: 'base64',
      },
      type: 'image',
    };

  if (type === 'url') {
    const { base64, mimeType } = await imageUrlToBase64(content.image_url.url);
    return {
      source: {
        data: base64 as string,
        media_type: mimeType as Anthropic.ImageBlockParam.Source['media_type'],
        type: 'base64',
      },
      type: 'image',
    };
  }

  throw new Error(`Invalid image URL: ${content.image_url.url}`);
}

Ensure that the imageUrlToBase64 function correctly converts the image URL to a base64 string and that the parseDataUri function correctly parses the data URI. If the image URL is not valid or cannot be converted, the function will throw an error ^[1].

Additionally, the specific requirements and limitations for image formats and sizes when using the OpenAI API are as follows:

1. Number of Images: Must be between 1 and 10.
2. Prompt Length: Maximum length of 1000 characters.
3. Image Quality: For DALL-E 3, the quality can be specified as 'hd'.
4. Image Sizes: Must be one of the following:
   • 1792x1024
   • 1024x1024
   • 1024x1792
5. Image Style: For DALL-E 3, the style can be 'vivid' or 'natural' (default is 'vivid') ^[2].

Make sure your images comply with these requirements to avoid the "invalid_image" error.

To continue talking to Dosu, mention @dosu.

[arvinxx]

你的图片链接是啥样的

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

What does your image link look like?

[kylyzhh]

你的图片链接是啥样的