GitHub - haschka/CLI-RAG: Command line tool to Interact with a llama.cpp server. Also implements a basic vector database with cosine similarity search.

Command line client for llama.cpp

Requirements: The tool requires the libraries and headers (-dev packages) of curl, json-c and gnu readline. On debian based systems, i.e. debian, ubuntu etc. an install of these can be achieved using:

sudo apt-get install build-essential libjson-c-dev libcurl-dev libreadline-dev

Further you need a llama.cpp compatible large language and embedding model. For the following instructins we suggest:

Build: in most cases a simple make all should be enough. In case it does not work edit the makefile in order to satisfy your systems libraries cflags.
Conversation Run: 3.1 Start a llama.cpp server:

llama.cpp/bin/llama-server -m Meta-Llama-3.1-8B-Instruct-Q6_K.gguf --host 127.0.0.1

3.2 Connect to your llama.cpp server with the client:

bin/rag-conversation 127.0.0.1 8080 -1

When you type your text finish with Ctrl-d. This allows multiline input on the terminal.

llama.cpp/bin/llama-server -m nomic-embed-text-v1.5.f16.gguf --host localhost --port 8081

4.2 Create a vector database from a text document:

bin/build-vector-db-from-server your-text.txt localhost 8081 2000 your-text.vdb

4.3 Run vector database supported and talk about your text:

bin/rag-with-vdb-cos-client localhost 8080 -1 your-text.vdb 3 localhost 8081

Name		Name	Last commit message	Last commit date
Latest commit History 15 Commits
bin		bin
LICENSE		LICENSE
README.md		README.md
build-vector-db-from-server.c		build-vector-db-from-server.c
curl_helpers.c		curl_helpers.c
curl_helpers.h		curl_helpers.h
embedding-from-server-cli.c		embedding-from-server-cli.c
embedding-from-server.c		embedding-from-server.c
embedding-from-server.h		embedding-from-server.h
load-texts.c		load-texts.c
load-texts.h		load-texts.h
local_resolve.c		local_resolve.c
local_resolve.h		local_resolve.h
makefile		makefile
multirag.c		multirag.c
vector-db.c		vector-db.c
vector-db.h		vector-db.h