GPT4All is not straightforward to use offline - blocked by GFW

[flysoulbird]

Bug Report

"The newly installed gpt4all cannot add models, and the error message is 'network error: cannot retrieve http;//gpt4all,io/models/models3,json'."

Example Code

Expected Behavior

I hope to be able to add custom configurations for the local model, which can be separate from the online downloaded model, and not affect each other. Ultimately, this will allow for both local and network downloads to choose the models they need from the model library. Additionally, I hope to be able to manually add or modify the model library configuration to effectively avoid issues with the software not functioning correctly due to network problems.

Your Environment

• Bindings version (e.g. "Version" from pip show gpt4all):gpt4all v3.2.1
• Operating System:Windows 11 Home Chinese Edition 23H2（22631.3880）
• Chat model used (if applicable):

[cosmic-snow]

That URL is basically a redirect to: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-chat/metadata/models3.json. Is this blocked for some reason?

In this file there is metadata for the official list of models. You can also manually download the models and set System Prompt / Prompt Template yourself.

Or you download the file and, on Windows, save it as C:\Users\%USERNAME%\AppData\Roaming\nomic.ai\models.json

Note that it's already possible to download other models and put them next to the ones from the official list. Also be aware of the fact that if there is no problem with downloading models3.json, it'll get overwritten again. So it's not a good idea to mess with that file yourself.

[manyoso]

Could this be the problem?

[3Simplex]

If it is then many of the models provided are also blocked.

[flysoulbird]

That URL is basically a redirect to: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-chat/metadata/models3.json. Is this blocked for some reason?

In this file there is metadata for the official list of models. You can also manually download the models and set System Prompt / Prompt Template yourself.

Or you download the file and, on Windows, save it as C:\Users\%USERNAME%\AppData\Roaming\nomic.ai\models.json

Note that it's already possible to download other models and put them next to the ones from the official list. Also be aware of the fact that if there is no problem with downloading , it'll get overwritten again. So it's not a good idea to mess with that file yourself.models3.json

According to what you said, i have downloaded the https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-chat/metadata/models3.json file and put it in the nomic.ai\models.json directory in C:\Users%USERNAME%\AppData\Roaming\nomic.ai\models.json，However, the error still persists.

[3Simplex]

Please provide a screenshot of the page that comes up when you click these links when using the computer in question.

• models3.json
• Yi-1.5-9B-Chat-16K-GGUF

We need to rule out a firewall problem.

[cosmic-snow]

... However, the error still persists.

I guess you'd need to be offline during startup so it doesn't even try to download the file.

[lesong36]

Please provide a screenshot of the page that comes up when you click these links when using the computer in question.

• models3.json
• Yi-1.5-9B-Chat-16K-GGUF

We need to rule out a firewall problem.

[cebtenzzre]

According to what you said, i have downloaded the main/gpt4all-chat/metadata/models3.json file and put it in the nomic.ai\models.json directory in C:\Users%USERNAME%\AppData\Roaming\nomic.ai\models.json，However, the error still persists.

To be clear, the directory should be %APPDATA%\nomic.ai and the file should be called models.json (renamed from models3.json). For further diagnostics, please start GPT4All from powershell and show the output & "${Env:UserProfile}\gpt4all\bin\chat" (this won't work, sorry).

If you have IPv6 (check with https://ipv6-test.com/), I've heard that Google DNS may be an option in China. Not sure if it would improve anything, but in other (non-US) countries changing to Google DNS (usually the IPv4 one) has helped.

I guess you'd need to be offline during startup so it doesn't even try to download the file.

I believe GPT4All will not display this error if it does successfully load from the cached models.json, even if regardless of whether the network is (un)available.

[cosmic-snow]

I believe GPT4All will not display this error if it does successfully load from the cached models.json, even if the network is unavailable.

Before I suggested that, I did check the case where the network is not available both with the current, valid one and when I renamed it (to ensure what to expect when it's missing). So that definitely works.

I did not look at the code/logic itself, though. If you did, then all the better, and another attempt should be made to do it right, as you described.

[cebtenzzre]

Here's a thought. What if you compare Invoke-WebRequest "http://gpt4all.io/models/models3.json" and Invoke-WebRequest "https://gpt4all.io/models/models3.json" (note the https) in PowerShell? Maybe the problem is the insecure request that leaves everything in plaintext. If it is, only the latter will work.

[3Simplex]

The only way I could manage to replicate your issue was with a "clean install" and a firewall blocking the program. Please make sure you dont have something like avast or norton that is "protecting" your computer from this program accessing the internet.

[flysoulbird]

Could this be the problem?

You are right, my test results are the same as yours. The software introduction said that it could be used offline. Download the models3.json to enable normal use, or can you download the large model and configure it locally?

[lesong36]

I have the same issue with ubuntu, do you know the local path to put the model.json. 获取 Outlook for iOS<https://aka.ms/o0ukef>

…

________________________________ 发件人: flysoulbird ***@***.***> 发送时间: Wednesday, August 21, 2024 9:13:46 AM 收件人: nomic-ai/gpt4all ***@***.***> 抄送: lesong36 ***@***.***>; Comment ***@***.***> 主题: Re: [nomic-ai/gpt4all] can not add any model (Issue #2897) Could this be the problem? test.png (view on web)<https://github.com/user-attachments/assets/3c53b659-478a-4b75-8cf9-b406822a14f5> You are right, my test results are the same as yours. The software introduction said that it could be used offline. Download the models3.json to enable normal use, or can you download the large model and configure it locally? ― Reply to this email directly, view it on GitHub<#2897 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AUPFXUOQ77V325B5QL3UVB3ZSPSUVAVCNFSM6AAAAABMZEJTU6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMBQGAZDCNZWHE>. You are receiving this because you commented.Message ID: ***@***.***>

[flysoulbird]

Please provide a screenshot of the page that comes up when you click these links when using the computer in question.

• models3.json
• Yi-1.5-9B-Chat-16K-GGUF

We need to rule out a firewall problem.

Confirmed, it's the issue with the national firewall. I hope that in the next update, it can achieve the effect of local installation and operation that completely disconnects from the internet. Thank you all.

[cosmic-snow]

I have the same issue with ubuntu, do you know the local path to put the model.json.

On Ubuntu it should be saved as ~/.config/nomic.ai/models.json

Confirmed, it's the issue with the national firewall. I hope that in the next update, it can achieve the effect of local installation and operation that completely disconnects from the internet. Thank you all.

This is still possible if you're able to download models manually. The models.json file is there for convenience, you should try again to put it in the right location.

Even if the application doesn't have that models.json, you can download and save models to use in GPT4All. The wiki has a page on that. You will be in charge of setting a proper System Prompt and Prompt Template, however.

[lesong36]

Thanks, now it works. 获取 Outlook for iOS<https://aka.ms/o0ukef>

…

________________________________ 发件人: cosmic-snow ***@***.***> 发送时间: Wednesday, August 21, 2024 6:52:52 PM 收件人: nomic-ai/gpt4all ***@***.***> 抄送: lesong36 ***@***.***>; Comment ***@***.***> 主题: Re: [nomic-ai/gpt4all] can not add any model (Issue #2897) I have the same issue with ubuntu, do you know the local path to put the model.json. On Ubuntu it should be saved as ~/.config/nomic.ai/models.json ― Reply to this email directly, view it on GitHub<#2897 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AUPFXULFX4PJ2BKYRUUJAETZSRWQJAVCNFSM6AAAAABMZEJTU6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMBRG42TSNJTGU>. You are receiving this because you commented.Message ID: ***@***.***>

[VipCrazyMaster]

In the Mac M1 environment, I solved this error, downloaded this file in the download item https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-chat/metadata/models3.json, renamed it to Models.json, enter this path "Go to a folder" ~/.config/gPT4ALL.IO/, and put it in models.json ~