Bug: Does any convert.py supprt llama3?

[zyc1128]

What happened?

Does any convert.py supprt llama3?

Name and Version

Does any convert.py supprt llama3?

What operating system are you seeing the problem on?

No response

Relevant log output

No response

[christianazinn]

You'll want to use convert-hf-to-gguf.py. The Llama3 BPE pretokenizer is supported by default in convert-hf-to-gguf-update.py.

[ego]

How to convert origin Meta-Llama-3 to gguf?

❯ ls models/Meta-Llama-3-8B
checklist.chk       consolidated.00.pth params.json         tokenizer.model

[Galunid]

Convert the .pth weights yourself using https://github.com/huggingface/transformers/blob/main/src/transformers/models/llama/convert_llama_weights_to_hf.py, or download the weights from huggingface. Then you can use convert-hf-to-gguf.py

[ego]

Convert the .pth weights yourself using https://github.com/huggingface/transformers/blob/main/src/transformers/models/llama/convert_llama_weights_to_hf.py, or download the weights from huggingface. Then you can use convert-hf-to-gguf.py

I have tried it:

python src/transformers/models/llama/convert_llama_weights_to_hf.py --input_dir ../llama.cpp/models/Meta-Llama-3-8B --output_dir converted

but it produce error:

.venv/lib/python3.10/site-packages/sentencepiece/__init__.py", line 316, in LoadFromFile
    return _sentencepiece.SentencePieceProcessor_LoadFromFile(self, arg)
RuntimeError: Internal: could not parse ModelProto from ../llama.cpp/models/Meta-Llama-3-8B/tokenizer.model

[Galunid]

See huggingface/transformers#30334, this is not our script and sadly we can't provide support for it.

[ego]

Thanks a lot!

[ArthurZucker]

If you update to the latest version of transformers the scripts supports the conversion