Add icl_sliding_k_retriever.py and update __init__.py #1305
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
This PR aims to introduce a new Sliding Window Retriever for in-context learning. The goal is to enhance the flexibility and effectiveness of example retrieval in the OpenCompass framework, particularly for scenarios where context relevance varies with the position of test examples.
Modification
This PR introduces two main modifications:
icl_sliding_k_retriever.py
implementing theSlidingWindowRetriever
class.__init__.py
to include the newSlidingWindowRetriever
in the package exports.The
SlidingWindowRetriever
allows for dynamic selection of in-context examples based on a sliding window approach, which can be particularly useful for tasks where the relevance of context changes as we progress through the dataset.BC-breaking (Optional)
This modification does not introduce any changes that break backward compatibility. Existing retriever implementations remain unchanged, and this new retriever can be used alongside them without conflicts.
Use cases (Optional)
The
SlidingWindowRetriever
can be particularly useful in scenarios such as:To use the new retriever, users can specify it in their configuration files like so:
Checklist
Before PR:
After PR: