Implementation for the paper: The Art of SOCRATIC QUESTIONING: Recursive Thinking with Large Language Models. This paper has been accepted by EMNLP 2023.
Schematic comparison of various prompting methods. Each blue rectangle box represents a thought serving as an intermediate reasoning step in the problem-solving process. SOCRATIC QUESTIONING incorporates both a top-down exploration process (in red line) to deconstruct complex problems into smaller sub-questions and a bottom-up backtracking process (in green line) to recursively solve these sub-questions and gather solutions for higher-level problems.
pip install openaiFor the multimodal implementation, please follow the official instructions to install BLIP-2.
Text-Only:
- Prepare the data. Input data should be given as a CSV file. Question ID, Context, Question, Option A, Option B, Option C, Option D, and Ground Truth are split by comma. An example input file is provided:
./socratic_questioning_textOnly/data/example_data.csv - Add your own prompts into the prompts file. Prompts for tasks and datasets presented in the paper have already been written.
./socratic_questioning_textOnly/data/prompt_map.json - Replace the argument of question_type with your dataset name, and replace the OpenAI API with your own.
./socratic_questioning_textOnly/code/socratic_questioning.sh - Run
sh ./socratic_questioning_textOnly/code/socratic_questioning.sh. The output file will be in./socratic_questioning_textOnly/results.
MultiModal:
- Prepare the data. Input data should be given as a JSONL file. The default image store path is:
./socratic_questioning_multimodal/data/imgs. An example input file is provided:./socratic_questioning_multimodal/data/example_data.jsonl - Replace the argument of dataset with your dataset name, and replace the OpenAI API with your own.
./socratic_questioning_multimodal/code/lm_eyes.sh - Run
sh ./socratic_questioning_multimodal/code/lm_eyes.sh. The output file will be in./socratic_questioning_multimodal/result.