Adding the following models: Vicuna, Koala, Pythia-Chat-Base-7B, GPT-NeoXT-Chat-Base-20B

[ViktorThink]

Adding support for using Vicuna, Koala, Pythia-Chat-Base-7B, GPT-NeoXT-Chat-Base-20B for mining on netuid 1.

Closing my pervious pull requests 1266 and 1261 since this new pull request includes those models, and it was cleaner to have them all in the same branch.

Vicuna, Pythia and NeoXT has been tested that they work properly on netuid 1. I plan to test Koala today on netuid 1 if there are active validators.

[camfairchild]

Seems good. Not sure about the license implications.

[ifrit98]

Looks good. Thanks for adding tweaks/suggestions @camfairchild

[ifrit98]

@ViktorThink Nice work!

A few Qs:

• Can you provide a complete example for running your Koala miner? Given the instructions and code provided, you cannot instantiate a Koala miner, even after converting the weights.
• Do you use a LLaMA tokenizer? If not, what tokenizer should be used with this model? (unclear)

Command ran:

python neurons/text/prompting/miners/koala/neuron.py --koala.model_name /home/jason/sandbox/PR/llama/llama_weights/koala/7B/model --netuid 1 --logging.debug --wallet.hotkey miner4

Error message (indicating there is no tokenizer found, which makes sense because there is not one mentioned in the readme at all and is not discussed how to move from the final Koala weights to prepare the miner.)

HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/home/jason/sandbox/PR/llama/llama_weights/koala/7B/model'. Use `repo_type` argument if needed.

The error is on line 45:
self.tokenizer = AutoTokenizer.from_pretrained( self.config.koala.model_name, use_fast=False )

https://github.com/opentensor/bittensor/blob/b179b334afc16ed7a8f654fcc1b350abcf559ef8/neurons/text/prompting/miners/koala/neuron.py#L45

[ViktorThink]

@ifrit98 Ah, nice spotting. Yes, it is the same tokenizer as llama. I'll update the documentation later today to show exactly how to do it.

[ViktorThink]

@ifrit98 I had missed to add the last step of the conversion process before starting the miner.

Converting the Koala Weights to HuggingFace Transformers

To run the model on the miner, it has to be converted to Huggingface Transformers format. To do so, use
the following command:

python -m EasyLM.models.llama.convert_easylm_to_hf \
    --load_checkpoint='params::path/to/koala/checkpoint' \
    --tokenizer_path='path/to/llama/tokenizer' \
    --model_size='13b' \  # '7b', '13b', '30b' or '65b'
    --output_dir='path/to/output/huggingface/koala/checkpoint'

Starting Miner

python3 neurons/text/prompting/miners/koala/neuron.py --koala.model_name model_path_or_huggingface_identifier

EasyLM has not released any converted version of the models, only delta, but I saw on Huggingface there are already converted checkpoints that could be loaded directly.

I don't actually have the Llama weights in torch format, so I will look into adding a second option for conversion for those who have Llama in HF format but not in torch.

[ifrit98]

Converted and miner is up and running. Nice work! Closing.