feat(grammar): add llama3.1 schema

[mudler]

Description

This PR allows LocalAI to generate BNF rules to constrain the LLM output to generate valid JSON in the llama3.1 format. It also introduces a series of refactoring as such it is easier now to extend to other schemas to specifically generate a syntax.

The new schema, allows to force the LLM output to this format: <function=example_function_name>{{"example_name": "example_value"}}</function> which is common for Llama 3.1 function calling.

How it works: to enable this behavior, set schema_type to llama3.1, for instance:

function:
  grammar:
    schema_type: llama3.1 # or JSON is supported too (json)

Forces the LLM to return function tool calls always with the llama3.1 format.

It keeps the behavior also with mixed grammar, I've tested with:

context_size: 8192
f16: true
function:
  disable_no_action: true
  grammar:
    #disable: true
    no_mixed_free_string: true
    mixed_mode: true
    schema_type: llama3.1 # or JSON is supported too (json)
  response_regex:
  - <function=(?P<name>\w+)>(?P<arguments>.*)</function>
mmap: true
name: meta-llama-3.1-8b-instruct
parameters:
  model: Meta-Llama-3.1-8B-Instruct.Q4_K_M.gguf
stopwords:
- <|im_end|>
- <dummy32000>
- <|eot_id|>
- <|end_of_text|>
template:
  chat: |
    <|begin_of_text|>{{.Input }}
    <|start_header_id|>assistant<|end_header_id|>
  chat_message: |
    <|start_header_id|>{{if eq .RoleName "assistant"}}assistant{{else if eq .RoleName "system"}}system{{else if eq .RoleName "tool"}}tool{{else if eq .RoleName "user"}}user{{end}}<|end_header_id|>

    {{ if .FunctionCall -}}
    Function call:
    {{ else if eq .RoleName "tool" -}}
    Function response:
    {{ end -}}
    {{ if .Content -}}
    {{.Content -}}
    {{ else if .FunctionCall -}}
    {{ toJson .FunctionCall -}}
    {{ end -}}
    <|eot_id|>
  completion: |
    {{.Input}}
  function: |
    <|start_header_id|>system<|end_header_id|>

    You have access to the following functions:

    {{range .Functions}}
    Use the function '{{.Name}}' to '{{.Description}}'
    {{toJson .Parameters}}
    {{end}}

    Think very carefully before calling functions.
    If a you choose to call a function ONLY reply in the following format with no prefix or suffix:

    <function=example_function_name>{{`{{"example_name": "example_value"}}`}}</function>

    Reminder:
    - If looking for real time information use relevant functions before falling back to searching on internet
    - Function calls MUST follow the specified format, start with <function= and end with </function>
    - Required parameters MUST be specified
    - Only call one function at a time
    - Put the entire function call reply on one line
    <|eot_id|>
    {{.Input }}
    <|start_header_id|>assistant<|end_header_id|>

Notes for Reviewers

Signed commits

• Yes, I signed my commits.

[netlify]

✅ Deploy Preview for localai ready!

Name	Link
🔨 Latest commit	a46d533
🔍 Latest deploy log	https://app.netlify.com/sites/localai/deploys/66a3c7ea469a69000879b506
😎 Deploy Preview	https://deploy-preview-3015--localai.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[dave-gray101]

I think the merging of grammars down makes sense. llama 3.1 compatibility enhancements are a boon as well!