TruLens Eval v0.29.0

Breaking Changes

In this release, we re-aligned the groundedness feedback function with other LLM-based feedback functions. It's now faster and easier to define a groundedness feedback function, and can be done with a standard LLM provider rather than importing groundedness on its own. In addition, the custom groundedness aggregation required is now done by default.

Before:

from trulens_eval.feedback.provider.openai import OpenAI
from trulens_eval.feedback import Groundedness

provider = OpenAI() # or any other LLM-based provider
grounded = Groundedness(groundedness_provider=provider)
f_groundedness = (
    Feedback(grounded.groundedness_measure_with_cot_reasons, name = "Groundedness")
    .on(Select.RecordCalls.retrieve.rets.collect())
    .on_output()
    .aggregate(grounded.grounded_statements_aggregator)
)

After:

provider = OpenAI()
f_groundedness = (
    Feedback(provider.groundedness_measure_with_cot_reasons, name = "Groundedness")
    .on(Select.RecordCalls.retrieve.rets.collect())
    .on_output()
)

This change also applies to the NLI-based groundedness feedback function available from the Huggingface provider.

Before:

from trulens_eval.feedback.provider.openai import Huggingface
from trulens_eval.feedback import Groundedness

from trulens_eval.feedback.provider import Huggingface
huggingface_provider = Huggingface()
grounded = Groundedness(groundedness_provider=huggingface_provider)

f_groundedness = (
    Feedback(grounded.groundedness_measure_with_cot_reasons, name = "Groundedness")
    .on(Select.RecordCalls.retrieve.rets.collect())
    .on_output()
    .aggregate(grounded.grounded_statements_aggregator)
)

After:

from trulens_eval.feedback import Feedback
from trulens_eval.feedback.provider.hugs = Huggingface

huggingface_provider = Huggingface()
    
f_groundedness = (
    Feedback(huggingface_provider.groundedness_measure_with_nli, name = "Groundedness")
    .on(Select.RecordCalls.retrieve.rets.collect())
    .on_output()
)

In addition to the change described above, below you can find the full release description.

What's Changed

• update groundedness prompt by @bpmcgough in #1112
• Default names for rag triad utility by @joshreini1 in #1122
• Unify groundedness interface by @joshreini1 in #1135

Bug Fixes

• Fixed bug with trace view initialization when no feedback functions exist by @walnutdust in #1108
• Remove references to running moderation endpoint on AzureOpenAI by @joshreini1 in #1116
• swap rag utility (qs)relevance by @piotrm0 in #1120
• Fix Link in Readme by @timbmg in #1128
• chore: remove unused code cell by @stokedout in #1113
• trurails: update to getattr by @joshreini1 in #1130
• Fix typo in README.md by @eltociear in #1136
• fix rag triad and awaitable calls by @piotrm0 in #1110
• Remove placeholder feedback for asynchronous responses by @arn-tru in #1127
• Stop iteration streams in openai cost tracking by @piotrm0 in #1138

Examples

• Show OSS models (and tracking) in LiteLLM application by @joshreini1 in #1109

New Contributors

• @stokedout made their first contribution in #1113
• @timbmg made their first contribution in #1128
• @bpmcgough made their first contribution in #1112
• @eltociear made their first contribution in #1136

Full Changelog: trulens-eval-0.28.0...trulens-eval-0.29.0