以 DeepSeek-R1 等为代表的推理模型（Large Reasoning Models, LRMs），通过生成更长的思维链，在各类复杂任务中取得了更优的表现。但长思维链是推理模型 的 "双刃剑"，虽能提升性能，但 "过度思考" 带来的语义冗余会大幅推高推理成本。 为破解大模型长思维链的效率难题，并且为了更好的端到端加速落地，我们将思考早停与投机采样无缝融合，提出了 SpecExit 方法，利用轻量级草稿模型预测 "退出信号"，在避免额外探测开销的同时将思维链长度缩短 66%，vLLM 上推理端到端加速 2.5 倍。 论文：https://arxiv.org/abs/2509.24248 开源代码：https://github.com/Tencent/AngelSlim 1."思考早停" 的挑战 目前对 LRMs 思维链压缩的相关研究大致可以分为两类，一类是基于训练的方法，另一类是 Training-Free 的方法，它们都有各自的局限性： （1）基于训练的方法，通过标注数据进行有监督微调，或通过强化学习减少思维链长度。尽管压缩效果显著，但往往伴随高昂的训练成本，并导致模型输出分布 被改变，引发模型可靠性 ...

Core Viewpoint - SpecForge is an open-source training framework designed for speculative sampling, specifically tailored for large models, achieving a 2.18x inference acceleration [1][15]. Group 1: SpecForge Overview - SpecForge is developed by the SGLang team in collaboration with Meituan's search recommendation platform and Cloudsway.AI [1]. - The framework is built to address the challenges posed by the increasing size of models, which often leads to lower inference efficiency [4][6]. - SpecForge integrates deeply with the SGLang inference engine, providing a seamless training and inference process for speculative sampling [5][7]. Group 2: Technical Features - The framework incorporates Eagle3, an advanced speculative sampling method that enhances inference speed by training a lightweight draft model to predict token distributions accurately [7]. - SpecForge supports various mainstream models, including complex MoE layers and Transformer variants, ensuring broad applicability [7]. - It features scalable distributed training through Fully Sharded Data Parallel (FSDP) and Tensor Parallelism (TP), optimizing resource utilization on GPU clusters [7][14]. Group 3: Training Modes and Efficiency - SpecForge offers two training modes: Online and Offline, allowing users to choose based on their specific needs and resource availability [10][17]. - The Training-Time Test (TTT) architecture enhances the robustness of the draft model, encapsulating complex processes to simplify implementation for users [9]. - The framework is designed with a focus on memory-efficient training, significantly reducing memory overhead even for trillion-parameter models [7]. Group 4: Experimental Validation - The effectiveness of SpecForge was validated through experiments on datasets like ShareGPT and UltraChat, demonstrating compatibility with the Eagle3 architecture [15]. - The draft models trained using SpecForge achieved a notable 2.18x inference acceleration on the MT-Bench benchmark [15]. Group 5: Future Developments - SpecForge's roadmap includes plans to support additional model architectures and integrate visual-language models (VLM) into the framework [22]. - The team aims to enhance training efficiency through improved parallel strategies and kernel optimizations [22].

清华大学&面壁智能 投稿 量子位 | 公众号 QbitAI 端侧性价比之王，清华大学和面壁智能团队开源新模型—— MiniCP M 4 ，提供 8B、0.5B 两种参数规模， 仅使用同级别开源模型22%的训练开销 ，就达到了同级别最优性能。 MiniCPM4-8B是 开源首个开源的原生稀疏模型，5%的极高稀疏度加持，让长文本、深思考在端侧真正跑起来。 在MMLU、CEval、MATH500、HumanEval等基准测试中，以仅22%的训练开销，性能比肩 Qwen-3-8B，超越Gemma-3-12B。 MiniCPM4-0.5B 在性能上，也展现出以小博大——在MMLU、CEval、BBH、HumanEval等基准测试中，MiniCPM4.0 -0.5B性能超越同级 的Qwen-3-0.6B、Llama 3.2、Gemma3， 并通过 原生QAT技术 实现几乎不掉点的int4量化以及600Token/s的极速推理速度。 在常见端侧芯片，比如Jetson AGX Orin与RTX 4090上，MiniCPM 4可实现长文本处理的5倍常规加速与极限场景下的百倍加速。 请看VCR： 目前团队已公开发布技术报告，该模 ...