Core Insights - Xiaomi's AI team, in collaboration with Peking University, has recently published a paper focusing on MoE (Mixture of Experts) and reinforcement learning, revealing new advancements in large model training [2][8]. Group 1: Research Findings - The paper proposes a novel approach to enhance the stability and efficiency of large model reinforcement learning within the MoE framework [8][10]. - Current reinforcement learning methods face challenges in balancing efficiency and stability, often leading to catastrophic failures during training [14][24]. - The research introduces a method called Rollout Routing Replay (R3), which locks the routing distribution during inference and reuses it during training, ensuring consistency between the two phases [30][31]. Group 2: Experimental Results - Experiments conducted on the Qwen3-30B-A3B model demonstrate that R3 consistently outperforms other methods across various metrics, achieving higher scores in multiple scenarios [41][42]. - The introduction of R3 significantly reduces the occurrence of training crashes, maintaining a stable performance curve even after extended training periods [44][48]. - R3 not only stabilizes the model but also accelerates the optimization process, allowing for quicker identification of effective strategies [50]. Group 3: Team and Contributors - The research team includes notable contributors such as Wenhan Ma, a researcher from Xiaomi's LLM-Core team, and Luo Fuli, who has a strong academic background and has previously worked on significant AI projects [52][59]. - The paper also acknowledges the contributions of Professor Sui Zhifang from Peking University, who has extensive experience in computational linguistics and AI research [62][66].

凤凰网科技讯 5月19日，据小米技术官方透露，近日，计算语言学和自然语言处理领域国际顶级会议 ——第63届国际计算语言学年会（ACL 2025）公布了论文录用结果，小米大模型团队共有10篇研究成 果入选，包括9篇主会长文和1篇findings长文，成果涵盖大模型端侧高效推理、大模型GUI智能体、大 模型基础结构创新等多个领域。 值得注意的是，小米本次入选的10篇论文中，有5篇获得了小米揭榜挂帅科研专项（Xiaomi Open- Competition Research Program）的支持，展示了小米在大模型领域的持续投入和技术积累。 从论文内容来看，小米大模型团队的研究成果聚焦在多个前沿技术方向，如：无长期衰减的新型位置编 码，能够增强模型的上下文感知和外推能力；混合框架，通过定制化KV缓存优化实现长上下文推理； 针对指令扩展过程中的"固定思维模式"问题提出了基于动态提示更新的新方法等。 据了解，ACL是国际计算语言学协会主办的年度学术会议，在计算语言学和自然语言处理领域享有极高 声誉，被中国计算机学会（CCF）列为A类会议。本届ACL将于今年7月27日至8月1日在奥地利维也纳 举行，这也是该会议的第63 ...