langchain: Update ContextualCompressionRetriever base_retriever type to RetrieverLike

[shenghann]

Description:
When initializing retrievers with configurable_fields as base retriever, ContextualCompressionRetriever validation fails with the following error:

ValidationError: 1 validation error for ContextualCompressionRetriever
base_retriever
  Can't instantiate abstract class BaseRetriever with abstract method _get_relevant_documents (type=type_error)

Example code:

esearch_retriever = VertexAISearchRetriever(
    project_id=GCP_PROJECT_ID,
    location_id="global",
    data_store_id=SEARCH_ENGINE_ID,
).configurable_fields(
    filter=ConfigurableField(id="vertex_search_filter", name="Vertex Search Filter")
)

# rerank documents with Vertex AI Rank API
reranker = VertexAIRank(
    project_id=GCP_PROJECT_ID,
    location_id=GCP_REGION,
    ranking_config="default_ranking_config",
)

retriever_with_reranker = ContextualCompressionRetriever(
    base_compressor=reranker, base_retriever=esearch_retriever
)

It seems like the issue stems from ContextualCompressionRetriever insisting that base retrievers must be strictly BaseRetriever inherited, and doesn't take into account cases where retrievers need to be chained and can have configurable fields defined.

langchain/libs/langchain/langchain/retrievers/contextual_compression.py

Lines 15 to 22 in 0a1e475

	class ContextualCompressionRetriever(BaseRetriever):
	"""Retriever that wraps a base retriever and compresses the results."""
	base_compressor: BaseDocumentCompressor
	"""Compressor for compressing retrieved documents."""
	base_retriever: BaseRetriever
	"""Base Retriever to use for getting relevant documents."""

This PR proposes that the base_retriever type be set to RetrieverLike, similar to how EnsembleRetriever validates its list of retrievers:

langchain/libs/langchain/langchain/retrievers/ensemble.py

Lines 58 to 75 in 0a1e475

	class EnsembleRetriever(BaseRetriever):
	"""Retriever that ensembles the multiple retrievers.
	It uses a rank fusion.
	Args:
	retrievers: A list of retrievers to ensemble.
	weights: A list of weights corresponding to the retrievers. Defaults to equal
	weighting for all retrievers.
	c: A constant added to the rank, controlling the balance between the importance
	of high-ranked items and the consideration given to lower-ranked items.
	Default is 60.
	id_key: The key in the document's metadata used to determine unique documents.
	If not specified, page_content is used.
	"""
	retrievers: List[RetrieverLike]
	weights: List[float]

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment

Name	Status	Preview	Comments	Updated (UTC)
langchain	⬜️ Ignored (Inspect)	Visit Preview		Jul 12, 2024 5:38pm