持续学习
Search documents
美国AI一骑绝尘,中国平均落后7个月,Epoch AI新报告出炉
3 6 Ke· 2026-01-08 07:53
一张来自Epoch AI图表给出了一个冷静却尖锐的结论:中国AI平均落后7个月。 一张图揭示真相: 自2023年以来,前沿AI全部来自美国! 最近,Epoch AI一份报告指出,中国AI模型的进展平均落后于美国7个月—— 最小差距为4个月,最大差距为14个月。 另一个被人们忽视的事实是,这平均7个月的代差,本质上也是「开源vs闭源」的差距。 仔细观察这张图表,就会发现中美LLM能力的差距,几乎完全贴合闭源和开源之间的整体差距。 7个月的「生死距」 Epoch AI最新图表,依旧衡量的是综合能力指数ECI(模型整体前沿水平)。 实际上,这张图表,隐含了三层重要的信息。 仅看美国AI这条蓝线,便会发现几个显著的特征—— 更新节奏非常密集,从GPT-4到o1,再到GPT-5、Gemini 3 Pro,中间几乎没有长时间停滞。 而且,其能力跃迁不完全依赖参数规模。 就比如,o1系列强大在于推理路径的设计,中间状态建模,以及训练目标的重构。它把「思考过程」纳入了工程对象。 它综合考虑了语言理解与生成、推理与问题分解能力、多任务泛化表现,已公开模型评测与专家校准。 这种差距,被量化为具体的时间。 也就是说,中国AI想要达 ...
IPO首日,智谱创立发起人内部信曝光:明确2026年目标,提及梁文锋
Xin Lang Cai Jing· 2026-01-08 02:37
二,全新的模型架构设计。已经广泛使用近10年的Transformer架构已经显露出一些不足,包括超长上下 文的计算开销、记忆机制、更新机制等。这些都需要探索全新的模型架构,发现新的Scaling范式,通过 芯片-算法协同设计等技术提高计算效率。 三,具有更强泛化能力的RL。当前主流的RLVR范式虽然在数学和代码领域取得了成功,但其依赖人工 构造可验证环境的局限性也日益凸显。今年需要探索更通用的RL范式,支持AI不仅能在人类指令下完 成特定任务,更要能理解并执行跨越数小时甚至数天的长时程任务。 新浪科技讯 1月8日上午消息,智谱AI今日上市,新浪科技了解到,上市当天,清华大学计算机系教 授、智谱创立发起人兼首席科学家唐杰发布内部信,宣布很快将推出新一代模型 GLM-5。并进一步明 确了2026年公司的目标是"成为国际领跑的大模型企业"。 唐杰提及了DeepSeek出现带给自己的警醒道:"文锋2023年创业的时候和我聊过,当时我并没有意识到 他对AGI如此执着,感谢他带给我很多不一样的思考。"唐杰称,"选择对AGI技术的执着追求,不断探 索AGI的上界,同时精准的未来预判成为下一步智谱需要不断改进和升华的地方。" ...
Sebastian Raschka万字年终复盘:2025,属于「推理模型」的一年
机器之心· 2026-01-02 09:30
Core Insights - The AI field continues to evolve rapidly, with significant advancements in reasoning models and algorithms such as RLVR and GRPO, marking 2025 as a pivotal year for large language models (LLMs) [1][4][19] - DeepSeek R1's introduction has shifted the focus from merely stacking parameters to enhancing reasoning capabilities, demonstrating that high-performance models can be developed at a fraction of previously estimated costs [9][10][12] - The importance of collaboration between humans and AI is emphasized, reflecting on the boundaries of this partnership and the evolving role of AI in various tasks [1][4][66] Group 1: Reasoning Models and Algorithms - The year 2025 has been characterized as a "year of reasoning," with RLVR and GRPO algorithms gaining prominence in the development of LLMs [5][19] - DeepSeek R1's release showcased that reasoning behavior can be developed through reinforcement learning, enhancing the accuracy of model outputs [6][19] - The estimated training cost for the DeepSeek R1 model is significantly lower than previous assumptions, around $5.576 million, indicating a shift in cost expectations for advanced model training [10][12] Group 2: Focus Areas in LLM Development - Key focus areas for LLM development have evolved over the years, with 2025 emphasizing RLVR and GRPO, following previous years' focus on RLHF and LoRA techniques [20][22][24] - The trend of "Benchmaxxing" has emerged, highlighting the overemphasis on benchmark scores rather than real-world applicability of LLMs [60][63] - The integration of tools in LLM training has improved performance, allowing models to access external information and reduce hallucination rates [54][56] Group 3: Architectural Trends - The architecture of LLMs is converging towards using mixture of experts (MoE) layers and efficient attention mechanisms, indicating a shift towards more scalable and efficient models [43][53] - Despite advancements, traditional transformer architectures remain prevalent, with ongoing improvements in efficiency and engineering adjustments [43][53] Group 4: Future Directions - Future developments are expected to focus on expanding RLVR applications beyond mathematics and coding, incorporating reasoning evaluation into training signals [25][27] - Continuous learning is anticipated to gain traction, addressing challenges such as catastrophic forgetting while enhancing model adaptability [31][32] - The need for domain-specific data is highlighted as a critical factor for LLMs to establish a foothold in various industries, with proprietary data being a significant concern for companies [85][88]
告别KV Cache枷锁,将长上下文压入权重,持续学习大模型有希望了?
机器之心· 2026-01-02 01:55
人类已经走上了创造 AGI(通用人工智能)的道路,而其中一个关键方面是持续学习,即 AI 能通过与环境互动而不断学习新的知识和能力。 想象一下你生命中的第一次机器学习讲座:你或许记不清教授开口说的第一个单词,但那场讲座留给你的直觉和逻辑,此刻正潜移默化地帮助你理解这篇复杂的 论文。这种能力的本质在于 压缩 。 近日,Astera 研究所、英伟达、斯坦福大学、加州大学伯克利分校、加州大学圣地亚哥分校的一个联合团队提出的 TTT-E2E(端到端测试时训练) 沿着这条 AGI 的必经之路迈出了重要一步。它彻底打破了传统模型在推理时静态不变的局限,让长上下文建模从一种「架构设计」进化为一种「学习问题」。 为此,研究社区已经在探索多种不同的道路,比如开发能够实时更新状态的循环神经网络(RNN),或者试图通过极大的缓存空间来容纳海量历史。然而,真正 的 AGI 或许不应仅仅被动地「存储」信息,而应像人类一样在阅读中「进化」。 该方法可以在测试阶段通过给定上下文的下一个 token 预测持续学习, 将读取的上下文信息压缩至权重参数中 。 编辑|Panda 论文标题:End-to-End Test-Time Training ...
人工智能年度盘点:2025年十大核心趋势及2026年关注焦点
Xin Lang Cai Jing· 2025-12-30 15:15
Group 1: Meta's Acquisition - Meta announced the acquisition of Chinese AI startup Manus for over $2 billion, a significant increase from its previous valuation of $500 million during a funding round in April [1][16] - This acquisition marks a substantial return on investment for its backers, including Benchmark Capital, ZhenFund, and Redpoint Ventures, and continues Meta's trend of acquisitions aimed at restructuring its AI business [1][16] - The effectiveness of this acquisition in revitalizing Meta's AI business remains uncertain [1][16] Group 2: AI Industry Trends - The AI industry continues to attract venture capital and talent, but signs of market fatigue are emerging, including delays in data center construction [2][17] - OpenAI's previous dominance in the AI chatbot market has diminished, with leading companies like OpenAI, Anthropic, and Google now offering comparable models [2][17] - Major clients of AI models, such as Salesforce and Microsoft, are facing sales challenges for their AI-enabled products, raising concerns about an AI bubble [2][17] Group 3: Key Developments in AI - The launch of the DeepSeek model by a Chinese hedge fund in January 2025 created significant industry buzz, claiming to rival top models from OpenAI and others, although its actual training costs were later revealed to be much higher than initially stated [4][19] - Reinforcement learning technology has gained popularity, with major AI labs adopting it to enhance model performance across various applications [6][20] - Over 25 AI application startups have achieved annual revenues of at least $100 million, indicating a shift towards profitability in the sector [7][23] Group 4: Meta's Challenges - 2025 is a challenging year for Meta, with its new Llama 4 model receiving criticism and a significant investment of $14.3 billion in Scale AI yielding limited results [7][23] - Meta's new AI team has struggled to produce successful applications, leading to organizational changes and talent loss [7][23] Group 5: Google's Resurgence - Google has made a strong comeback in the AI space in 2025, releasing several well-received models, including Gemini 3.0, which achieved significant breakthroughs in code generation [8][24] - Despite still trailing behind ChatGPT in user numbers, Google's rapid progress is noteworthy [8][24] Group 6: Financing Trends - The trend of circular financing in the AI industry continues, with companies relying on funding from tech giants like Microsoft and Nvidia to purchase necessary computing resources [9][25] - This financing model has proven effective for AI labs in managing their substantial operational costs [9][25] Group 7: Regulatory Environment - The Trump administration has introduced favorable policies for the AI industry, including prohibiting state-level regulations and expediting data center project approvals [10][26] - These measures have been influenced by significant investments from tech companies to gain favor with the administration [10][26] Group 8: Robotics and AI - Despite substantial investments in robotics startups, the anticipated advancements in practical robots powered by AI have largely failed to materialize [11][27] - The high cost and operational limitations of new robotic products have raised questions about their viability in the market [11][27] Group 9: Research Directions - There is growing skepticism among AI researchers regarding the feasibility of achieving artificial general intelligence (AGI) with current technologies [12][28] - The concept of "continuous learning" is emerging as a new research direction, which could significantly impact the industry if successfully developed [12][28] Group 10: Market Movements - Leading AI companies like OpenAI and Anthropic are signaling intentions to go public in the coming years, driven by the capital-intensive nature of their businesses [13][29] - Successful IPOs could provide individual investors with opportunities to benefit from the AI sector's growth, but potential market corrections pose risks [13][29] Group 11: Industry Dynamics - André Karpathy's recent shift in perspective on AI programming tools highlights the evolving landscape of AI applications in software engineering [14][30] - His endorsement of AI tools suggests a significant transformation in the role of programmers, emphasizing the integration of AI technologies [14][30]
Gemini 3预训练负责人警告:模型战已从算法转向工程化,合成数据成代际跃迁核心,谷歌碾压OpenAI、Meta的秘密武器曝光
3 6 Ke· 2025-12-26 12:21
2025 年底,大模型行业的"年终决战"正式打响,各家纷纷亮出压箱底的杀手锏,就在这场激烈角逐中,Gemini 3 以绝对王者之姿强势突围,一登场就刷 新了行业的认知边界。 11 月 18 日,Gemini 3 直接"横扫"多项权威基准测试,以"世界最强多模态理解""交互最深智能体""推理怪兽"的姿态,强势碾压全球所有同类模型。谷歌 CEO 桑达尔·皮查伊亲自为其站台,直言这是"迄今为止最智能的模型"。消息一出,整个 AI 圈瞬间沸腾,所有人都在追问:Gemini 3 的强悍,到底藏着什 么秘诀? 答案在发布当天就有了初步线索。Google DeepMind 研究与深度学习副总裁 Oriol Vinyals 直接在推特上"剧透":"Gemini 3 这么强,核心秘诀就两点:更 好的预训练,更好的后训练。"这番直白的表态,让"预训练"与"后训练"瞬间成为行业热议的核心话题。 | Description | Description | | Colorded 3-Per | Garried 2.5-Fre. | Claude Sawat 4.5 GPT-5.8 | | | --- | --- | --- | --- ...
Gemini 3预训练负责人警告:模型战已从算法转向工程化!合成数据成代际跃迁核心,谷歌碾压OpenAI、Meta的秘密武器曝光
AI前线· 2025-12-26 10:26
作者 | 高允毅 2025 年底,大模型行业的"年终决战"正式打响,各家纷纷亮出压箱底的杀手锏,就在这场激烈角逐中,Gemini 3 以绝对王者 之姿强势突围,一登场就刷新了行业的认知边界。 11 月 18 日,Gemini 3 直接"横扫"多项权威基准测试,以"世界最强多模态理解""交互最深智能体""推理怪兽"的姿态,强势碾压 全球所有同类模型。谷歌 CEO 桑达尔·皮查伊亲自为其站台,直言这是"迄今为止最智能的模型"。消息一出,整个 AI 圈瞬间沸 腾,所有人都在追问:Gemini 3 的强悍,到底藏着什么秘诀? 答案在发布当天就有了初步线索。Google DeepMind 研究与深度学习副总裁 Oriol Vinyals 直接在推特上"剧透": "Gemini 3 这么强,核心秘诀就两点:更好的预训练,更好的后训练。 "这番直白的表态,让"预训练"与"后训练"瞬间成为行业热议的核心 话题。 作为从强化学习转向表征学习的资深研究者,Sebastian Borgeaud 的预训练功底堪称深厚:从 Transformer 架构,到 BERT、 XLNet,再到 DeepMind 第一篇大语言模型论文 Goph ...
以VLA+MOE架构打造工业具身大脑,赛索德智能斩获千万级天使轮融资
机器人圈· 2025-12-26 10:07
近日, 工业场景具身智能研发商赛索德智能宣布完成数千万元天使轮融资 ,本轮投资由宁波方正 (300998)、扬州金泉(603307)、顺景科技(603007)三家上市企业及创投机构南吉资本联合注 资,资金将用于核心技术迭代与工业化场景落地。 作为一家深耕工业具身智能的创新企业,赛索德智能正构建 "算法定义硬件"的机器人系统新范式。其核心 方向是通过VLA(多模态融合)+MOE(混合专家模型)架构打造工业级具身大脑,专门适配多品种、小 批量、定制化的工厂生产场景,精准填补当前市场中智能装配机器人的应用空白。 硬核团队护航技术落地,跨领域背景筑牢创新根基 赛索德智能的核心团队汇聚了机器人技术、人工智能、工业场景应用等多领域的资深专家,为技术创新与 商业转化提供了坚实支撑。 创始人孙鑫海拥有香港中文大学硕士学位,目前正在攻读清华 -米兰理工管理工程PhD,其研究方向聚焦 多模态融合下的空中交通流量预测与优化。凭借在安徽尼威动力、东方久乐汽车电子等企业的董事任职经 历,他对机器人产业趋势与客户核心需求有着深刻洞察,尤其擅长将前沿技术转化为具备商业价值的产品 方案。 联合创始人兼 CTO周丹弟博士毕业于北京理工大学计算 ...
Dwarkesh最新播客:AI 进展年终总结
3 6 Ke· 2025-12-24 23:15
Core Insights - Dwarkesh's podcast features prominent AI figures Ilya Sutskever and Andrej Karpathy, indicating his significant standing in the AI community [1] - The article summarizes Dwarkesh's views on AI advancements, particularly regarding the timeline for achieving AGI [1] Group 1: AI Development and AGI Timeline - The focus on "mid-training" using reinforcement learning is seen as evidence that AGI is still far off, as it suggests models lack strong generalization capabilities [3][16] - The idea of pre-trained skills is questioned, as human labor's value lies in the ability to flexibly acquire new skills without heavy training costs [4][24] - AI's economic diffusion lag is viewed as an excuse for insufficient capabilities, rather than a natural delay in technology adoption [27][28] Group 2: AI Capabilities and Limitations - AI models currently lack the ability to fully automate even simple tasks, indicating a significant gap in their capabilities compared to human workers [25][30] - The adjustment of standards for AI capabilities is acknowledged as reasonable, reflecting a deeper understanding of intelligence and labor complexity [31] - The scaling laws observed in pre-training do not necessarily apply to reinforcement learning, with some studies suggesting a need for a million-fold increase in computational power to achieve similar advancements [10][33] Group 3: Future of AI and Continuous Learning - Continuous learning is anticipated to be a major driver of model capability enhancement post-AGI, with expectations for preliminary features to emerge within a year [13][40] - Achieving human-level continuous learning may take an additional 5 to 10 years, indicating that breakthroughs will not lead to immediate dominance in the field [14][41] - The potential for an explosion in intelligence once models reach human-level capabilities is highlighted, emphasizing the importance of ongoing learning and adaptation [36] Group 4: Economic Implications and Workforce Integration - The integration of AI labor into enterprises is expected to be easier than hiring human workers, as AI can be replicated without the complexities of human recruitment [29] - The current revenue gap between AI models and human knowledge workers underscores the distance AI still has to cover in terms of capability [30] - The article suggests that if AI models truly reached AGI levels, their economic impact would be profound, with businesses willing to invest significantly in AI labor [29]
假如每十年财产清零,现在最该做什么?
虎嗅APP· 2025-12-12 13:54
以下文章来源于人神共奋 ,作者思想钢印 文章的第一个问题:假如人生每过十年财产清零,你现在最应该做什么? 逢十生日那一天,财产定期清零,这个问题虽然科幻,但并不难回答, 丧失了"钱生钱"的复利效 应,财富的积累就失去了意义。 钱仍然有意义,只是存款没有意义,投资也没有意义,任何不在十年内被花掉的钱,都会失去意义, 类似买那种可以用一辈子的房子,就成为毫无意义的浪费。 当每一个人都不再为"以后"而过度储蓄和吝啬,更重要的事就变成,如何花钱才有意义? 活在当下,不再是一句鸡汤,而是生存的必然选择: 体验消费比实物消费更重要,吃饭泡吧比买名 牌时装重要,旅游比豪车重要。 时间成为最珍贵的东西,加班给十倍工资都没人肯干,让我们充分享受这十年中的每一天吧。 人神共奋 . 财经专栏作家,虎嗅&雪球2020年度十佳作者 本文来自微信公众号: 人神共奋 ,作者:思想钢印,题图来自:AI生成 不要拥有,要体验 还会有人认真地面对个人爱好,真诚地学习乐器、绘画、烹饪、运动等,而不是当成谋生的手段。 不过,这样的世界仍然有贫富差别,每个人赚钱的能力不同,所以很多人还会为下一个十年做准备, 除了消费, 最重要的就是积累知识与培养能力 ...