-
Background The paper introduces Qwen2 series, a new addition to large language models (LLMs) and large multimodal models. The Qwen2 models feature a comprehensive suite of foundational and instruction-tuned language models with a parameter range from 0.5 to 72 billion, encompassing dense models and Mixture-of-Experts models.
-
Existing Work Existing large language models, such as ChatGPT and the Llama series, have generated widespread interest in open weights and high-tech levels, creating significant momentum. Previous models may not have completely overcome optimization and performance competitiveness in various domains, including language understanding, generation, coding, mathematics, and reasoning. While models like Qwen1.5, Mistral, Gemma, etc., are present in the market, there still could be room for improvement.
- Introduced a comprehensive improved series of large language models
-
Challenge 1: Surpass existing open-weight models in multi-task environments The prior large language models faced limitations in performance across various domain tasks. Qwen2, with its models equipped with up to 72 billion dense and mixture-of-experts model parameters, significantly improves performance in tasks involving language understanding, generation, multilingual capabilities, coding, mathematics, and reasoning, surpassing most prior open-weight models, including its predecessor Qwen1.5.
-
Challenge 2: Enhancing multilingual capabilities Multilingual capability is an important performance metric to meet global demands. Qwen2 demonstrates proficiency in approximately 30 languages, including English, Chinese, Spanish, French, German, Arabic, Russian, Korean, Japanese, Thai, Vietnamese, etc., underscoring its versatility and global reach.
-
The paper showcases the performance of the Qwen2 series models across a number of authoritative benchmarks. For instance, the flagship model Qwen2-72B scored impressively in tasks: MMLU (84.2), GPQA (37.9), HumanEval (64.6), GSM8K (89.5), and BBH (82.4). The instruction-tuned variant Qwen2-72B-Instruct also performed prominently in MT-Bench (9.1), Arena-Hard (48.1), and LiveCodeBench (35.7). Moreover, the authors provided a detailed comparison of the Qwen2 series models against other models in their class, showcasing Qwen2's leading edge in multiple aspects.
The Qwen2 series models, as the latest large language models, exhibit excellent performance in multi-task environments such as language understanding, generation, multilingual capabilities, coding, mathematics, and reasoning. The models have also made their weights and resources publicly available in the open-source community, fostering innovation and accessibility. Compared to existing models, Qwen2 shows competitive performance in several benchmarks, especially in terms of multilingualism, showing a wide applicability and global reach.