-
Background The paper addresses that Large Language Models (LLMs) have gained extensive attention due to their impressive performance in various tasks. However, the significant computational and memory requirements for LLM inference create challenges for deployment in resource-constrained scenarios. Work in this field is moving towards techniques that improve the efficiency of LLM inference.
-
Existing Work Current work has not addressed the main reasons for inefficient LLM inference, such as the large model size, quadratic complexity attention operation, and autoregressive decoding methods.
- Introduced a comprehensive study
-
Challenge 1: Large model size One major reason for LLMs' inefficient inference is the large model size. The comprehensive study presented in the paper collects and categorizes various techniques from existing literature into optimizations at the data level, model level, and system level to address this challenge.
-
Challenge 2: Quadratic complexity of attention operations The paper analyzes a core issue leading to inefficient LLM inference, which is the quadratic complexity of attention operations that grows with input length. The paper offers a collected comparison of representative methods in different sub-fields, providing quantitative insights and conducting experimental analysis on these methods.
-
The paper includes comparative experiments that provide quantitative analysis on representative methods within key sub-fields, offering practical suggestions and guidance for improving the efficiency of LLM inference. While specific experimental data details are not provided, the classification taxonomy built upon existing literature and the systematic summarization and comparison of methods help readers better understand the current research progress in the area of LLM inference efficiency and point towards directions for future research.
This paper offers an encompassing survey of literature on improving inference efficiency for large language models, proposing a taxonomy that covers data-level, model-level, and system-level optimizations. Additionally, it provides quantified comparisons of key techniques through experiments, pointing out future directions for research.