-
Background The paper discusses the challenges of achieving optimal text generation with large language models (LLMs), particularly in the context of billion-sized models, even after providing a specific prompt or instruction. Additionally, undesired behaviors such as toxicity or hallucinations can occur in the generated content.
-
Existing Work The existing works fail to address the issue adequately as previous techniques like greedy decoding, nucleus sampling, or beam search mainly focus on past generations without considering future constraint satisfaction while generating text.
-
A formalized approach to text generation
- Challenge 1: How to consider future constraint satisfaction in generation To tackle this challenge, the paper proposes the use of a scoring mechanism evaluated by LLMs to quantify future constraint satisfaction, allowing for better control over the output during the generation process.
-
Challenge 2: Effectiveness of the scoring mechanism The effectiveness of the proposed scoring mechanism is validated by the observable distinct and discernible trends associated with the scoring signals when LLMs use these constraint signals. Future research might explore different signals and enhance their effectiveness, such as improving constraint score evaluation through tuning.
The evaluation results show the effectiveness of the proposed method across three distinct text generation tasks: keyword-constrained generation, toxicity reduction, and factual correctness in question-answering. Compared to other methods (greedy decoding, beam search), there were improvements in speed, fluency, informativeness, and correctness. Human evaluation involved workers from Amazon Mechanical Turk judging model responses, which also confirmed the strength of the proposed decoding method, with better scores achieved in all dimensions, especially correctness.
This work introduces a novel approach to improve the decoding methods for large language models by incorporating future constraint satisfaction. The proposed formal approach and scoring mechanism, benchmarked against LLMs, significantly contribute to the improved quality and control of text generation.