Ali Qwen2-72B tops the HELM list: performance surpasses Llama3-70B

Recently, Stanford University released the latest results of the HELM MMLU large model evaluation list, which has attracted widespread attention in the industry. With its strict evaluation standards and transparent evaluation process, this list provides a reliable reference for the performance evaluation of large models. The results of the list show that Alibaba's Tongyi Qianwen Qwen2-72B model stands out among many models and has achieved impressive results, demonstrating the significant progress of China's large model technology.

Recently, Stanford University’s large model evaluation list HELM MMLU released the latest results. Percy Liang, director of the Basic Model Research Center at Stanford University, published an article pointing out that Alibaba’s Tongyi Qianwen Qwen2-72B model surpassed Llama3-70B in rankings and became the best-performing open source large model. MMLU (Massive Multitask Language Understanding) is one of the most influential large model evaluation benchmarks in the industry. It covers 57 tasks including basic mathematics, computer science, law, history, etc., and is designed to test world knowledge and problem-solving abilities in large models. However, in actual evaluation, the results of different models often lack consistency and comparability. This is mainly due to the use of non-standard prompt word technology and the failure to uniformly adopt open source evaluation frameworks.

HELM (A holistic framework for evaluating foundation models) proposed by the Center for Research on Foundation Models (CRFM) at Stanford University is committed to creating a transparent and reproducible evaluation method. The HELM framework standardizes and makes transparent the evaluation results of different models on MMLU, solving problems existing in existing MMLU evaluations. For example, use the same prompt words for all participating models, and provide the models with the same 5 examples on each test topic for situational learning. Percy Liang, director of the Fundamental Model Research Center at Stanford University, recently released the latest HELM MMLU list on social platforms. The list shows that Alibaba's Tongyi Qianwen open source model Qwen2-72B ranks fifth, second only to Claude3Opus, GPT-4o, Gemini1.5pro and GPT-4. It is the highest-ranked open source large model and also has the best performance. of chinese large model. Tongyi Qianwen Qwen2 series was open sourced in early June 2024 and contains 5 pre-training and instruction fine-tuning models of different sizes. Up to now, the Qwen series models have been downloaded more than 16 million times, showing their wide recognition and powerful performance in the industry. The latest evaluation results of HELM MMLU not only demonstrate the outstanding performance of Qwen2-72B in multi-task language understanding, but also mark the rise of China's large models in the global AI technology competition. With the continuous advancement of technology, we look forward to seeing more outstanding large models from China show off on the international stage in the future.

The outstanding performance of Qwen2-72B not only reflects Alibaba’s strong technical strength in the field of AI, but also injects strong impetus into the development of large models in China. I believe that in the future, more Chinese large models will show their outstanding strength on the international stage.