Bug in Dev Strategy - No Support for LOCAL only models (StarCoder, Llama2, others) for Generative AI models for coding

[richlysakowski]

There is a bug in the Bug in Development Strategy.

You are wedding everyone to big cloud AI platforms, not democratized LOCAL models. None of the supported models run locally. Who is driving strategy? The AI Cloud Monsters who will insource and consume all IT and AI jobs if we let them.

Please address this issue of a LLM for coding that runs 100% LOCALLY.

Description

I also have a very strong and active interest in running local "offline" high-performing AI "democratized" systems that do not use major cloud platforms, and that are not considered "edge" nodes on the global internet. What people call "The Cloud" is really a small set of large cloud platform vendors CONSPIRING to outsource all IT and computing jobs to their own corporations, leaving millions of citizens without meaningful work. If normal citizens don't outsource "The Cloud" back to "The Edge", entire segments of white collar and brown collar society will be marginalized within the next 10 years. Cloud vendors will OWN, i.e., control, all the jobs they insourced to their cloud business. People on "The Edge" are on the margins of society. Local, democractic AI brings power back to the people. The Cloud takes power away from the people. Let's make sure AI stays democratized, offline and online. Remember "Human Lives Matter" (HLM)! Cloud computing is a tool, not the endpoint for society.

[JasonWeill]

@richlysakowski Thanks for your contribution! We have implemented local model support via GPT4All as a resolution to issue #190, as done in #209. Please try it out.

If you find any issues with GPT4All not covered by existing open issues (#348, #226) then we welcome your issues and pull requests.

[richlysakowski]

Thank you. I will check out what you have done. Can this be used to create completely local implementations of popular LLM models, such as StarCoder or CodeLlama?

[JasonWeill]

You can find more about GPT4All here: https://gpt4all.io/

They also have their own GitHub repo, with its own issue queue for bugs and enhancements: https://github.com/nomic-ai/gpt4all/issues