[bug] Issue with GibberishText package - False Validation

[Kashyapiyer]

Describe the bug
guardrails-ai 0.4.5 is forcing to use Token. Without API token it is not allowing to configure further.

Also gibberish text is falsely reporting validation failure even on correct sentences

Sentence tested : "Hello how is the weather"

To Reproduce
Steps to reproduce the behavior:

1. RAIL spec
2. Runtime arguments (e.g. guard(...))
   Please refer to the code screenshot
   Expected behavior
   A clear and concise description of what you expected to happen.

Library version:
guardrails-ai 0.4.5

Additional context
Add any other context about the problem here.

Please let us know if you need further information. Appreciate your kind support. Thanks

[wylansford]

Hi there,

For issue the first problem regarding the token. Once you install guardrails, you can run
guardrails configure
to get a token. Following the steps there should resolve your issue.

For problem regarding the validator, the validator is an ML based model. It will not always be perfect, so you'll need to adjust the threshold according to the model that is used and the types of inputs you will be using. In this case, if you lower the threshold to .4 it will pass.

Setting the threshold will depend on the system that you are using. If this is a system that involves humans, you may need to lower the threshold to accommodate spelling/grammatical errors. With LLMs, it may be higher. In fact, if you change the input to "How is the weather today?" (using punctuation like an LLM would), then the validator successfully passes.

[Kashyapiyer]

Hi Wylandsford,
Thanks for looking into this issue. I tested both situations after configuring Guardrails. It would be tricky to ensure that always a context text would end in a specific way. Could you suggest the optimum threshold value that could work in most of cases. Appreciate your kind support.
Regards,
Kashyap