Add example of using LLM as a judge for summarization dataset.

[eladven]

No description provided.

[elronbandel]

Overall looks good. Yet do we really want users to write templates for each task being evaluated? I think a better model is that we will guide them to use something like:
"card=cards.xsum,metrics=[metrics.llm_as_judge]"

[yoavkatz]

Overall looks good. Yet do we really want users to write templates for each task being evaluated? I think a better model is that we will guide them to use something like: "card=cards.xsum,metrics=[metrics.llm_as_judge]"

Yes. I agree. We already have an example of adding an llm metric definition and using it. Here , I think we should use a simple predefined metric. We should show one metric that uses the reference answr and ibe that does not.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.35%. Comparing base (31f7d4b) to head (7445aa5).
Report is 8 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #965      +/-   ##
==========================================
+ Coverage   91.33%   91.35%   +0.01%     
==========================================
  Files         110      112       +2     
  Lines       11704    11794      +90     
==========================================
+ Hits        10690    10774      +84     
- Misses       1014     1020       +6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.