How to set a custom client for a chat agent.

[shenhai-ran]

Hello,

I am inside corporate networking and I need the self signed certificate to access certain websites. In case of OpenAI API, I can do something like

http_client=httpx.Client(verify=<my/certificate>)
client=OpenAI(http_client=http_client)

I tried to read the documents and I only found settings for proxy. I also tried to use .set_model but it doesn't work either.

How to work with Agently in this case?

Thanks

[Maplemx]

The offical framework did not support the method yet but you can customize the OAIClient request plugin to do so.

You can find the plugin file by this path: plugins/request/OAIClient.py

And these's a method named _create_client that using httpx to create a request client. You can add verify information there.

I'll update verify support next version too. Thanks for letting me know this case and help us to improve.

[shenhai-ran]

Hi, thanks for your update...

Just out of curiosity, why do you do you use this way of setting but not expose native, for example, OpenAI client to the user?

[Maplemx]

updated in bc9aa24

[Maplemx]

published in version 3.3.2.0

[Maplemx]

Hi, thanks for your update...

Just out of curiosity, why do you do you use this way of setting but not expose native, for example, OpenAI client to the user?

If we do so, features like agent.input().output().start() won't able to work and developers will have to manage prompt template all by themselves.

Also, we can not promise that developers can switch models without changing codes about how application works but just changing model settings.

We are not only providing model requesting methods but also providing application behaviors management expression with LLMs, that's not the same in many different ways. Don't think it too simple.

[shenhai-ran]

Hi, thanks for your update...
Just out of curiosity, why do you do you use this way of setting but not expose native, for example, OpenAI client to the user?

If we do so, features like agent.input().output().start() won't able to work and developers will have to manage prompt template all by themselves.

Also, we can not promise that developers can switch models without changing codes about how application works but just changing model settings.

We are not only providing model requesting methods but also providing application behaviors management expression with LLMs, that's not the same in many different ways. Don't think it too simple.

I see. When you have multiple LLM clients, the factory mode is indeed more convenient to switch models. In my case I only have one endpoint from the company, so don't have to worried about it....

I have tested agent.set_settings("model.OAIClient.verify", os.environ["<my-ca>"]) and it works. great work!