Skip to content

Latest commit

 

History

History
18 lines (15 loc) · 2.21 KB

2401.06954.md

File metadata and controls

18 lines (15 loc) · 2.21 KB
Raw

Background

  • Background This paper introduces the challenge where current Retrieval-augmented Generation (RAG) systems might not always align with the preferences of Large Language Models (LLMs) despite strong performance in knowledge-intensive tasks.
  • Existing Work Other studies tried to fine-tune either LLMs or retrievers to make them align better, but these approaches focus mainly on the ranking aspect and do not address the overall "preference gap." Fine-tuning retrievers leads to re-ranking and fine-tuning LLMs are often impractical as many models are only accessible via APIs.

Core Contributions

  • Introduced BGM (Bridging the Gap between retrievers and LLMs) framework
    • Challenge 1: Preference gap The paper highlights a "preference gap" between retrievers and LLMs, especially in ranking and selection. The BGM aims to retain retrievers and LLMs as-is and train a bridge model to convert retrieved information into a more LLM-friendly format.
    • Challenge 2: Lack of effective supervision Since "real" relevance labels for the retrieved content do not exist in the typical RAG system, existing supervised learning methods suffer from sparse supervision inefficiency. The paper proposes a framework combining Supervised Learning (SL) with Reinforcement Learning (RL) to enhance the model's performance in downstream tasks and explore more advanced strategies like relevance-weighting through repetition.

Implementation and Deployment

The BGM framework was evaluated on multiple downstream tasks including Question Answering (QA) and personalized text generation. Experimental outcomes indicate that BGM has improved the performance, surpassing strong retrievers and re-ranking baseline models across various datasets, including public QA, Amazon reviews, and private email conversations. This underscores the importance and potential of the bridging approach in the realm of RAG.

Summary

The paper presents the BGM framework to address the "preference gap" between retrievers and LLMs. Through a seq2seq bridge model and a combined SL and RL training scheme, the framework optimizes the retrieved information to fit LLMs' preferences, improving performance in multiple downstream tasks.