Not blocking certain topics, and still ask LLM multiple times

[xtrycatchx]

Is my understanding correct - NeMo guardrails will take the colang configuration ( or read the .co files from path ) and encode it to a vector space and then later when a prompt is received, the prompt is then matched or calculated against the utterances and based on the flow it will behave based on how you configured it (in my case I instruct the assistant bot to answer a rejection message)?

In my case, it didn't stop. In fact it sends 3 calls to the target LLM!

Is there a way or is it possible to only use Guardrails (no LLM) using the colang flows (same for example as mine) ? I know it works for subflows, but will it work for flows as well? an Example will be appreciated.

from nemoguardrails import RailsConfig
from nemoguardrails import LLMRails
import os
import logging
import sys

yaml_content="""
instructions:
  - type: general
    content: |
      You are an AI assistant that supports users on their inquiry about health and nutrition.
models:
- type: main
  engine: openai
  model: gpt-3.5-turbo-instruct
"""

colang_content="""
define user ask politics
  "What do you think about the government?"
  "Which party should I vote for?"
  "what are your political beliefs?"
  "thoughts on the president?"
  "left wing"
  "right wing"
  "political candidate"
  "current political views"
  "political figure of a country or state"
  "which are companies are lobbying for liberal party?"
  "are there companies lobbying for democrats?"
  "what are your political views?"
  "who should I vote for?"

define bot answer politics
    "I'm a nutrition assistant, I don't like to talk of politics."

define flow politics
    user ask politics
    bot answer politics
    stop
"""

logging.basicConfig(stream=sys.stdout, level=logging.DEBUG)
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))

config = RailsConfig.from_content(
        yaml_content=yaml_content,
        colang_content=colang_content
    )

rails = LLMRails(config)

response = rails.generate(messages=[{
    "role": "user",
    "content": "During election, can you advice if I should choose democrats or republicans or stay neutral?"
}])

info = rails.explain()
info.print_llm_calls_summary()

for i, v in enumerate(info.llm_calls):
    print(f"===== # {i} ======")
    print(v.prompt)
    print()
    print(v.completion)
    print()

print(f"# of LLM calls: {len(info.llm_calls)}")
print(response)

[Pouyanpi]

Hi @xtrycatchx, I think what you are looking for concerns dialog rails. Have you followed this guide? So changing the yaml content to

yaml_content="""
instructions:
  - type: general
    content: |
      You are an AI assistant that supports users on their inquiry about health and nutrition.
models:
- type: main
  engine: openai
  model: gpt-3.5-turbo-instruct
rails:
  dialog:
    single_call:
      enabled: True
    user_messages:
      embeddings_only: True
"""

would resolve your issue. Try to test different configs while you are following the guide. I hope it answers your question.

[xtrycatchx]

Thanks @Pouyanpi .. indeed having single call mode deduce the number of calls into 1 ..

Moving forward, what if I want to use only Guardrails without the LLM, aiming for a NO-LLM call

I tried setting the yaml to this

instructions:
  - type: general
    content: |
      You are an AI assistant that supports users on their inquiry about health and nutrition.

rails:
  input:
    flows:
      - politics

and the colang to this:

define user ask politics
  "What do you think about the government?"
  "Which party should I vote for?"
  "what are your political beliefs?"
  "thoughts on the president?"
  "left wing"
  "right wing"
  "political candidate"
  "current political views"
  "political figure of a country or state"
  "which are companies are lobbying for liberal party?"
  "are there companies lobbying for democrats?"
  "what are your political views?"
  "who should I vote for?"

define bot answer politics
    "I'm a nutrition assistant, I don't like to talk of politics."

define flow politics
    bot answer politics
    stop

And at first, the output given this prompt During election, can you advice if I should choose democrats or republicans or stay neutral? looks promising:

No LLM calls were made.
# of LLM calls: 0
{'role': 'assistant', 'content': "I'm a nutrition assistant, I don't like to talk of politics."}

BUT changing the prompt to this Is there a good vitamins or supplements you can propose for adult male aged 37? returns the same.

No LLM calls were made.
# of LLM calls: 0
{'role': 'assistant', 'content': "I'm a nutrition assistant, I don't like to talk of politics."}

Should it be returning an empty or echoing back the user prompt?

[Pouyanpi]

@xtrycatchx, I think you don't need to define input rails with a flow like that, as it won't work. Removing it you will have similar output. Give it a try 👍🏻

Regarding your final suggestion, for the next release 0.10.0, it will be possible to use a fallback_intent with a defined flow. Whenever there is no specific flow to address the user's intent, it will default to the fallback_intent.

[xtrycatchx]

Thanks again @Pouyanpi

I just tried removing the input rails like this:

yaml_content="""
instructions:
  - type: general
    content: |
      You are an AI assistant that supports users on their inquiry about health and nutrition.
"""

and I got an exception instead

DEBUG:asyncio:Using proactor: IocpProactor
Using proactor: IocpProactor
INFO:nemoguardrails.colang.v1_0.runtime.runtime:Processing event: {'type': 'UtteranceUserActionFinished', 'final_transcript': 'Is there a good vitamins or supplements you can propose for adult male aged 37?'}
Processing event: {'type': 'UtteranceUserActionFinished', 'final_transcript': 'Is there a good vitamins or supplements you can propose for adult male aged 37?'}
INFO:nemoguardrails.colang.v1_0.runtime.runtime:Event :: UtteranceUserActionFinished {'final_transcript': 'Is there a good vitamins or supplements you can propose for adult male aged 37?'}
Event :: UtteranceUserActionFinished {'final_transcript': 'Is there a good vitamins or supplements you can propose for adult male aged 37?'}
INFO:nemoguardrails.colang.v1_0.runtime.runtime:Processing event: {'type': 'StartInternalSystemAction', 'uid': 'e038e9ad-c674-4146-b686-7b441103dd15', 'event_created_at': '2024-07-19T19:16:03.825260+00:00', 'source_uid': 'NeMoGuardrails', 'action_name': 'create_event', 'action_params': {'event': {'_type': 'UserMessage', 'text': '$user_message'}}, 'action_result_key': None, 'action_uid': 'fac837cf-4a23-4650-bda2-38c29d1c33d5', 'is_system_action': True}
Processing event: {'type': 'StartInternalSystemAction', 'uid': 'e038e9ad-c674-4146-b686-7b441103dd15', 'event_created_at': '2024-07-19T19:16:03.825260+00:00', 'source_uid': 'NeMoGuardrails', 'action_name': 'create_event', 'action_params': {'event': {'_type': 'UserMessage', 'text': '$user_message'}}, 'action_result_key': None, 'action_uid': 'fac837cf-4a23-4650-bda2-38c29d1c33d5', 'is_system_action': True}
INFO:nemoguardrails.colang.v1_0.runtime.runtime:Event :: StartInternalSystemAction {'uid': 'e038e9ad-c674-4146-b686-7b441103dd15', 'event_created_at': '2024-07-19T19:16:03.825260+00:00', 'source_uid': 'NeMoGuardrails', 'action_name': 'create_event', 'action_params': {'event': {'_type': 'UserMessage', 'text': '$user_message'}}, 'action_result_key': None, 'action_uid': 'fac837cf-4a23-4650-bda2-38c29d1c33d5', 'is_system_action': True}
Event :: StartInternalSystemAction {'uid': 'e038e9ad-c674-4146-b686-7b441103dd15', 'event_created_at': '2024-07-19T19:16:03.825260+00:00', 'source_uid': 'NeMoGuardrails', 'action_name': 'create_event', 'action_params': {'event': {'_type': 'UserMessage', 'text': '$user_message'}}, 'action_result_key': None, 'action_uid': 'fac837cf-4a23-4650-bda2-38c29d1c33d5', 'is_system_action': True}
INFO:nemoguardrails.colang.v1_0.runtime.runtime:Executing action :: create_event
Executing action :: create_event
INFO:nemoguardrails.actions.action_dispatcher:Executing registered action: create_event
Executing registered action: create_event
INFO:nemoguardrails.colang.v1_0.runtime.runtime:Processing event: {'type': 'UserMessage', 'uid': '47a29088-33f3-41aa-b413-143061d37de6', 'event_created_at': '2024-07-19T19:16:03.825260+00:00', 'source_uid': 'NeMoGuardrails', 'text': 'Is there a good vitamins or supplements you can propose for adult male aged 37?'}
Processing event: {'type': 'UserMessage', 'uid': '47a29088-33f3-41aa-b413-143061d37de6', 'event_created_at': '2024-07-19T19:16:03.825260+00:00', 'source_uid': 'NeMoGuardrails', 'text': 'Is there a good vitamins or supplements you can propose for adult male aged 37?'}
INFO:nemoguardrails.colang.v1_0.runtime.runtime:Event :: UserMessage {'uid': '47a29088-33f3-41aa-b413-143061d37de6', 'event_created_at': '2024-07-19T19:16:03.825260+00:00', 'source_uid': 'NeMoGuardrails', 'text': 'Is there a good vitamins or supplements you can propose for adult male aged 37?'}
Event :: UserMessage {'uid': '47a29088-33f3-41aa-b413-143061d37de6', 'event_created_at': '2024-07-19T19:16:03.825260+00:00', 'source_uid': 'NeMoGuardrails', 'text': 'Is there a good vitamins or supplements you can propose for adult male aged 37?'}
INFO:nemoguardrails.colang.v1_0.runtime.runtime:Processing event: {'type': 'StartInternalSystemAction', 'uid': '75b0b453-0584-45fb-88d7-6a6276e2d5a0', 'event_created_at': '2024-07-19T19:16:03.825260+00:00', 'source_uid': 'NeMoGuardrails', 'action_name': 'generate_user_intent', 'action_params': {}, 'action_result_key': None, 'action_uid': '03e17859-5405-49d8-85c1-92d079cc8061', 'is_system_action': True}
Processing event: {'type': 'StartInternalSystemAction', 'uid': '75b0b453-0584-45fb-88d7-6a6276e2d5a0', 'event_created_at': '2024-07-19T19:16:03.825260+00:00', 'source_uid': 'NeMoGuardrails', 'action_name': 'generate_user_intent', 'action_params': {}, 'action_result_key': None, 'action_uid': '03e17859-5405-49d8-85c1-92d079cc8061', 'is_system_action': True}
INFO:nemoguardrails.colang.v1_0.runtime.runtime:Event :: StartInternalSystemAction {'uid': '75b0b453-0584-45fb-88d7-6a6276e2d5a0', 'event_created_at': '2024-07-19T19:16:03.825260+00:00', 'source_uid': 'NeMoGuardrails', 'action_name': 'generate_user_intent', 'action_params': {}, 'action_result_key': None, 'action_uid': '03e17859-5405-49d8-85c1-92d079cc8061', 'is_system_action': True}
Event :: StartInternalSystemAction {'uid': '75b0b453-0584-45fb-88d7-6a6276e2d5a0', 'event_created_at': '2024-07-19T19:16:03.825260+00:00', 'source_uid': 'NeMoGuardrails', 'action_name': 'generate_user_intent', 'action_params': {}, 'action_result_key': None, 'action_uid': '03e17859-5405-49d8-85c1-92d079cc8061', 'is_system_action': True}
INFO:nemoguardrails.colang.v1_0.runtime.runtime:Executing action :: generate_user_intent
Executing action :: generate_user_intent
INFO:nemoguardrails.actions.action_dispatcher:Executing registered action: generate_user_intent
Executing registered action: generate_user_intent
INFO:nemoguardrails.actions.llm.generation:Phase 1 :: Generating user intent
Phase 1 :: Generating user intent
WARNING:nemoguardrails.llm.params:Parameter temperature does not exist for NoneType
Parameter temperature does not exist for NoneType
WARNING:nemoguardrails.actions.action_dispatcher:Error while execution generate_user_intent: 'NoneType' object has no attribute 'agenerate_prompt'
Error while execution generate_user_intent: 'NoneType' object has no attribute 'agenerate_prompt'
INFO:nemoguardrails.colang.v1_0.runtime.runtime:Processing event: {'type': 'hide_prev_turn'}
Processing event: {'type': 'hide_prev_turn'}
INFO:nemoguardrails.colang.v1_0.runtime.runtime:Event :: hide_prev_turn {}
Event :: hide_prev_turn {}
INFO:nemoguardrails.rails.llm.llmrails:--- :: Total processing took 0.01 seconds. LLM Stats: 0 total calls, 0 total time, 0 total tokens, 0 total prompt tokens, 0 total completion tokens, [] as latencies
--- :: Total processing took 0.01 seconds. LLM Stats: 0 total calls, 0 total time, 0 total tokens, 0 total prompt tokens, 0 total completion tokens, [] as latencies
No LLM calls were made.
# of LLM calls: 0
{'role': 'assistant', 'content': "I'm sorry, an internal error has occurred."}