-
Background The paper discusses the interest in leveraging the impressive performance of large language models (LLMs) to construct language agents for completing complex tasks. The authors focus on agents for knowledge-intensive tasks, where agents fulfill users' information-seeking requests by interacting with specific knowledge bases.
-
Existing Work According to the authors, few if any studies of building language agents can meet all the necessary criteria due to their uncontrollable reasoning logic, static model capability, or sparse/missing feedback signals.
- Introduced an Adaptable MOdulaR knowledge agent (AMOR)
-
Challenge 1: Establishing robust reasoning logic and adapting mechanisms to specific environments The AMOR formalizes its reasoning logic using a finite state machine (FSM), solving problems through executions and transitions, which allows users to give direct feedback to each LLM-controlled module. This enables the required process-based supervision mechanism.
-
Challenge 2: Supporting flexible forms of feedback AMOR supports flexible forms of feedback, going beyond binary judgments about the correctness to allow for refinements and detailed feedback pointing out necessary improvements or adjustments.
-
AMOR's training takes place in two stages: the Warm-up, leveraging examples from various public datasets to fine-tune the LLM for generalization across different knowledge environments, and the Adaptation, customizing AMOR to specific domains using process feedback. Experiments demonstrate AMOR's advantages over strong baselines, attributed to its FSM-based reasoning and process feedback mechanisms.
The AMOR framework integrates reasoning logic based on a finite state machine (FSM) and a process feedback mechanism, showcasing how an open-source LLM-based knowledge agent can reason and adapt with human oversight, enhancing the model's capabilities in performing knowledge-intensive tasks.