This is a Chrome extension that integrates an AI-powered writing tool into your browser, allowing you to interact with a self-hosted LLM model for rewriting text. With this extension, you can rewrite text for various purposes, such as essays, stories, poems, code, and more.
- Highlight any text on a web page and press CTRL+D to send it as a prompt to the server. The generated text will be copied to your clipboard and printed on the console.
- Customize the parameters, such as length, temperature, top_p, and frequency_penalty.
- Choose from different models.
To install this extension, follow these steps:
- Clone or download this repository to your local machine.
- Go to chrome://extensions/ and enable developer mode.
- Click on "Load unpacked" and select the folder where you cloned or downloaded this repository.
- You should see the extension icon on your browser toolbar.
To use this extension, you need to have some dependencies installed. You can easily install it with
pip install ctransformers flask flask-cors flask-apscheduler
for GPU support you will also need:
Nvidia users [CUDA support]
pip install ctransformers[cuda]
AMD users [ROCm support]
CT_HIPBLAS=1 pip install ctransformers --no-binary ctransformers
Apple users [Metal support]
CT_METAL=1 pip install ctransformers --no-binary ctransformers
After that you can start the server with
python server.py -p -model -threads -gpu_layers -batch_size
Options:
[-h], [--help] show this help message and exit
[-m], [-model] model to use
[-p], [-port] port the server will use
[-t], [-threads] threads to use
[-g], [-gpu_layers] gpu layers to use
[-b], [-batch_size] batch size to use
if you dont know what values to use, you can use benchmark.py to see what gets you the best performance for your pc
After starting the server, open your browser and navigate to the options of the extension. Enter the IP address (use "localhost" if local) and port number and save, then set the hotkey on your browser (On Brave, it can be accessed by going to brave://extensions/shortcuts/) After that you can go on any webpage and highlight a text, when you press the hotkey it will send the prompt to the server. This is the default prompt:
### HUMAN:
Rewrite this with proper grammar and more concise writing style: "
Currently, I have tested the application using Llama2-7b-chat-uncensored
This is my first chrome extension, so it may have some bugs or limitations. Some of the known issues are:
- The extension does not have a user interface to display the generated text. It only copies it to the clipboard and prints it on the console.
- The extension does not have a way to cancel or stop the text generation process once it is started.
- The extension does not have a way to save or export the generated text.
If you encounter any other issues or have any suggestions for improvement, please feel free to open an issue or a pull request on this repository.
This project is licensed under the GNU Affero General Public License - see the [LICENSE] file for details.