-
Notifications
You must be signed in to change notification settings - Fork 33
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Initial reranker integration in RedisVL #139
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #139 +/- ##
==========================================
+ Coverage 88.43% 88.60% +0.16%
==========================================
Files 26 29 +3
Lines 1565 1667 +102
==========================================
+ Hits 1384 1477 +93
- Misses 181 190 +9 ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
left some suggestions on what the documents need to look like to pass to the rerank call. TLDR:
- If we want, we are free to send
rerank
json docs (ie, list of dicts) but we just need to make sure to passrerank
a list ofrank_fields
that are the keys in the json docs - If we want to instead handle the slicing of json fields on the client side, we can do so and then pass rerank a list of strings and then omit passing
rank_fields
- We should default to
rerank-english-v3.0
Apologies if I'm missing some of the requirements we discussed a few months ago regarding the integration of the new reranker in redis.
Great feedback. I definitely missed the API spec update for v3. My concern with moving to rank_fields was backwards compatibility and the right way to do that. But I think I can updated |
8352efd
to
6388c4b
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
very clean! lgtm 🎉 saw that the tests are not running because of the api_key but if they all run as expected, test_bad_input
addresses the minor comment I left :)
986af5c
to
e5b9deb
Compare
This PR introduces another utility class into RedisVL to improve the querying experience, and result quality, through integration with 3rd party rerankers. We have started with Cohere's reranking API and will also add additional options later on.
This PR introduces another utility class into RedisVL to improve the querying experience, and result quality, through integration with 3rd party rerankers. We have started with Cohere's reranking API and will also add additional options later on.
TODO (with Cohere):
max_chunks_per_doc
)