Workflow
推理模型
icon
Search documents
MIT新论文:2026推理模型过时了,“套娃模型”当立
3 6 Ke· 2026-01-04 10:09
推理模型这就过时了? 当中的扛把子GPT-5被一篇博士生论文打了个措手不及,上下文窗口被甩出两个数量级。 而且新方法面对长文本时的"上下文腐烂"现象也大幅减少,关键是成本还更便宜。 这就是MIT最新论文当中提出的"套娃模型"新范式,被预言将成为今年的主流。 有网友评价说,递归模型不仅是在节省Token,更是在改变交互方式。 "套娃模型"正式名称叫做递归模型,核心流程是将文本存入代码环境,让模型编写程序拆解并递归调用自身处理。 从它的各种指标来看,推理模型,看上去真的是不香了。 代码驱动的递归推理 递归语言模型(RLM)一改将长文本直接作为Prompt输入神经网络的传统做法,转而采用一种"环境化"的处理范式。 其核心逻辑在于将自然语言处理任务重构为交互式编程任务,引入一个外部的Python REPL(读取-求值-输出循环)环境,将超长文本作为一个静态字符 串变量存储在内存中。 在这种架构下,大模型不再一次性编码所有信息,而是作为一个拥有读写权限的Agent,通过生成和执行Python代码来对这个外部变量进行操作。 这种设计从根本上解耦了输入数据的长度与模型自身的上下文窗口大小,允许处理的文本长度仅受限于物理内存 ...
Sebastian Raschka万字年终复盘:2025,属于「推理模型」的一年
机器之心· 2026-01-02 09:30
Core Insights - The AI field continues to evolve rapidly, with significant advancements in reasoning models and algorithms such as RLVR and GRPO, marking 2025 as a pivotal year for large language models (LLMs) [1][4][19] - DeepSeek R1's introduction has shifted the focus from merely stacking parameters to enhancing reasoning capabilities, demonstrating that high-performance models can be developed at a fraction of previously estimated costs [9][10][12] - The importance of collaboration between humans and AI is emphasized, reflecting on the boundaries of this partnership and the evolving role of AI in various tasks [1][4][66] Group 1: Reasoning Models and Algorithms - The year 2025 has been characterized as a "year of reasoning," with RLVR and GRPO algorithms gaining prominence in the development of LLMs [5][19] - DeepSeek R1's release showcased that reasoning behavior can be developed through reinforcement learning, enhancing the accuracy of model outputs [6][19] - The estimated training cost for the DeepSeek R1 model is significantly lower than previous assumptions, around $5.576 million, indicating a shift in cost expectations for advanced model training [10][12] Group 2: Focus Areas in LLM Development - Key focus areas for LLM development have evolved over the years, with 2025 emphasizing RLVR and GRPO, following previous years' focus on RLHF and LoRA techniques [20][22][24] - The trend of "Benchmaxxing" has emerged, highlighting the overemphasis on benchmark scores rather than real-world applicability of LLMs [60][63] - The integration of tools in LLM training has improved performance, allowing models to access external information and reduce hallucination rates [54][56] Group 3: Architectural Trends - The architecture of LLMs is converging towards using mixture of experts (MoE) layers and efficient attention mechanisms, indicating a shift towards more scalable and efficient models [43][53] - Despite advancements, traditional transformer architectures remain prevalent, with ongoing improvements in efficiency and engineering adjustments [43][53] Group 4: Future Directions - Future developments are expected to focus on expanding RLVR applications beyond mathematics and coding, incorporating reasoning evaluation into training signals [25][27] - Continuous learning is anticipated to gain traction, addressing challenges such as catastrophic forgetting while enhancing model adaptability [31][32] - The need for domain-specific data is highlighted as a critical factor for LLMs to establish a foothold in various industries, with proprietary data being a significant concern for companies [85][88]
吴恩达年终总结:2025年或将被铭记为「AI工业时代的黎明」
Hua Er Jie Jian Wen· 2025-12-31 03:10
26日,人工智能领域的知名学者吴恩达(Andrew Ng)在其年度信件与发布的《The Batch》特刊中指出,2025年或将被铭记为AI工业时代的黎明。这一年, 模型性能通过推理能力达到了新高度,基础设施建设成为推动美国GDP增长的关键力量,而顶尖科技公司为争夺人才展开了前所未有的薪酬战。 吴恩达认为,随着技术更紧密地融入日常生活,新的一年将进一步巩固这些变革。 万亿级资本开支与能源挑战 吴恩达表示,2025年,以OpenAI、微软、亚马逊、Meta和Alphabet为首的科技巨头宣布了一系列令人咋舌的基础设施投资计划。 据各方披露,每一吉瓦的数据中心容量建设成本约为500亿美元。OpenAI与其合作伙伴宣布了耗资5000亿美元的"Stargate"项目,并计划最终在全球建设20吉 瓦的容量。 微软在2025年的全球数据中心支出达到800亿美元,并签署了一项为期20年的协议,计划于2028年重启宾夕法尼亚州的三里岛核反应堆,以确保持续的电力供 应。 天价薪酬重塑人才市场 随着AI从学术兴趣转变为革命性技术,顶尖人才的身价已飙升至职业体育明星的水平。 吴恩达表示,Meta在2025年打破了传统的薪酬结构,向来 ...
吴恩达年终总结:2025是AI工业时代的黎明
具身智能之心· 2025-12-31 00:50
Core Insights - 2025 is marked as a pivotal year in the AI industry, characterized by rapid advancements and significant developments in AI technologies and infrastructure [10][14][30] - The competition for AI talent has intensified, with leading companies offering unprecedented salaries to attract top professionals [23][27] - The emergence of reasoning models and programming agents has transformed software development, lowering barriers to entry and enabling more individuals to participate in AI innovation [37][40] Group 1: AI Industry Developments - The year 2025 is described as the dawn of the AI industrial era, with major advancements in AI capabilities and infrastructure [14][30] - AI companies are projected to spend over $300 billion in capital expenditures, primarily on building new data centers to support AI tasks [30][32] - By 2030, the costs associated with building sufficient computing power for AI needs could reach $5.2 trillion, indicating a massive investment trend [30] Group 2: Talent Acquisition and Market Dynamics - AI firms are engaged in a fierce talent war, with salaries reaching levels comparable to professional sports stars, as companies like Meta offer up to hundreds of millions in compensation [23][27] - OpenAI, Meta, and other tech giants are implementing strategies to retain talent, including higher stock compensation and accelerated vesting schedules [27][30] - The influx of capital and talent into the AI sector is contributing to economic growth, with evidence suggesting that the majority of GDP growth in the U.S. in early 2025 is driven by data center and AI investments [30] Group 3: Technological Advancements - The introduction of reasoning models has significantly improved the performance of large language models (LLMs), enhancing their capabilities in various tasks [21][22][24] - Programming agents have become a competitive battleground among AI giants, with advancements allowing them to complete over 80% of programming tasks [31][34] - The development of new benchmarks and evaluation methods for programming agents reflects the evolving landscape of AI capabilities [34]
吴恩达年终总结:2025年或将被铭记为“AI工业时代的黎明”
华尔街见闻· 2025-12-30 12:45
Core Insights - The year 2025 is anticipated to mark the dawn of the AI industrial era, characterized by unprecedented advancements in model performance and infrastructure investments that will significantly contribute to GDP growth in the U.S. [1][2] Group 1: Capital Expenditure and Energy Challenges - Major tech companies, including OpenAI, Microsoft, Amazon, Meta, and Alphabet, have announced substantial infrastructure investment plans, with each gigawatt of data center capacity costing approximately $50 billion. OpenAI's "Stargate" project, in collaboration with partners, involves a $500 billion investment to build 20 gigawatts of capacity globally [3]. - Microsoft is projected to spend $80 billion on global data centers in 2025 and has signed a 20-year agreement to restart the Three Mile Island nuclear reactor in Pennsylvania by 2028 to ensure a stable power supply [3]. - Bain & Co. estimates that to support this scale of construction, AI annual revenue must reach $2 trillion by 2030, exceeding the total profits of major tech companies in 2024 [3]. - Insufficient grid capacity has led to some data centers in Silicon Valley being underutilized, and concerns over debt levels have caused Blue Owl Capital to withdraw from negotiations to finance a $10 billion data center for Oracle and OpenAI [3]. Group 2: Talent Market Transformation - Meta has disrupted traditional compensation structures by offering lucrative packages, including cash bonuses and substantial equity, to researchers from OpenAI, Google, and Anthropic, with some four-year contracts valued at up to $300 million [5]. - Mark Zuckerberg has personally engaged in the talent acquisition battle, successfully recruiting key researchers from OpenAI [5]. - In response, OpenAI has introduced aggressive stock option vesting schedules and retention bonuses of up to $1.5 million for new employees [6]. Group 3: Proliferation of Reasoning Models and Agentic Coding - 2025 is viewed as the year of widespread application of reasoning models, with advancements such as OpenAI's o1 model and DeepSeek-R1 demonstrating enhanced reasoning capabilities through reinforcement learning [8]. - The integration of tools has led to significant improvements in model performance, with OpenAI's o4-mini achieving a 17.7% accuracy rate in a multimodal understanding test, driving the rise of "Agentic Coding" [10]. - By the end of 2025, tools like Claude Code, Google Gemini CLI, and OpenAI Codex are expected to handle complex software development tasks through intelligent workflows [10]. - Despite some limitations in reasoning models identified by research from Apple and Anthropic, the trend of utilizing AI for code generation and cost reduction in development remains strong [11].
吴恩达年终总结:2025年或将被铭记为AI工业时代的黎明
Hua Er Jie Jian Wen· 2025-12-30 10:27
要点提炼: AI工业时代的黎明:2025年标志着AI从"学术探索"正式迈向"工业化基础设施"时代。AI投资成为驱动美国GDP 增长的核心力量,全球年度资本支出突破3000亿美元。 万亿级投入与能源焦虑:科技巨头(如OpenAI、微软、亚马逊)开启"星际之门"等超级数据中心计划,单项投 资动辄数千亿美元。电力供应成为硬约束,科技公司开始通过重启核电站(如三里岛)来保障算力需求。 推理模型与智能体化:以OpenAI o1和DeepSeek-R1为代表的推理模型成为主流,AI具备了"多步思考"能力。 "智能体编码(Agentic Coding)"爆发,AI智能体已能独立处理复杂的软件开发任务,编程效率显著提升。 天价薪酬重塑人才市场:顶尖人才身价比肩体育明星,Meta等巨头甚至开出高达3亿美元的四年期薪酬包。 | The Batch > Weekly Issues > Issue 333 | | --- | | 白 Published 0 Reading tim | | --- | | Dec 26, 2025 16 min read | | Top Stories of 2025! Big AI Poaches ...
吴恩达年终总结:2025是AI工业时代的黎明
机器之心· 2025-12-30 06:57
Core Insights - 2025 is marked as a pivotal year in the AI industry, characterized by intense competition among AI giants, a talent war, and significant advancements in AI infrastructure and capabilities [6][10][13]. Group 1: AI Development and Learning - The rapid advancement in AI has created unprecedented opportunities for software development, with a notable shortage of skilled AI engineers [6][22]. - Structured learning is essential for aspiring AI developers to avoid redundant efforts and to understand existing solutions in the industry [7][8]. - Practical experience is crucial; hands-on project work enhances understanding and sparks new ideas in AI development [8][14]. Group 2: AI Infrastructure and Investment - The AI industry has seen capital expenditures surpassing $300 billion in 2025, primarily for building new data centers to handle AI tasks [26]. - Major companies are planning extensive infrastructure projects, with projected costs reaching up to $5.2 trillion by 2030 to meet anticipated demand for AI capabilities [26][31]. - Companies like OpenAI, Meta, Microsoft, and Amazon are investing heavily in data center capacities, with OpenAI planning to build 20 gigawatts of data center capacity globally [31]. Group 3: Talent Acquisition and Market Dynamics - A fierce competition for top AI talent has led to unprecedented salary offers, with some companies offering compensation packages comparable to professional sports stars [22][26]. - Meta's aggressive recruitment strategy has included significant financial incentives to attract talent from competitors, reflecting the high market value of AI professionals [22][27]. - Despite concerns about an AI bubble, investments in AI infrastructure are contributing to economic growth, particularly in the U.S. [29]. Group 4: Advancements in AI Models - The introduction of reasoning models has significantly improved the performance of large language models (LLMs), enhancing their capabilities in various tasks [20][21]. - AI agents are increasingly capable of automating complex coding tasks, with reports indicating that many companies are now relying on AI-generated code for senior-level tasks [33][39]. - The evolution of programming agents has led to a competitive landscape among AI companies, with advancements in code generation capabilities becoming a focal point [30][39].
蒸馏、GEO、氛围编程 2025年度“AI十大黑话” 能听懂几个?
3 6 Ke· 2025-12-26 09:16
Core Insights - The article discusses the rapid development of AI in 2025, highlighting ten key terms that reflect how AI is reshaping industries and society. Group 1: AI Concepts - Vibe Coding redefines programming by allowing developers to express goals in natural language, with AI generating the necessary code [2] - Reasoning models have emerged as a core focus in AI discussions, enabling complex problem-solving through multi-step reasoning [3] - World Models aim to enhance AI's understanding of real-world causality and physical laws, moving beyond mere language processing [4] Group 2: Infrastructure and Investment - The demand for AI has led to the construction of super data centers, exemplified by OpenAI's $500 billion "Stargate" project, raising concerns about energy consumption and local impacts [5] - The AI sector is experiencing a capital influx, with companies like OpenAI and Anthropic seeing rising valuations, though many are still in the high-investment phase without stable profit models [6] Group 3: AI Challenges and Trends - The term "intelligent agents" is popular in AI marketing, but there is no consensus on what constitutes true intelligent behavior [7] - Distillation technology allows smaller models to learn from larger ones, achieving high performance at lower costs [8] - The concept of "AI garbage" reflects public concern over the quality and authenticity of AI-generated content [9] Group 4: AI in Real-World Applications - Physical intelligence remains a significant challenge for AI, as robots still require human intervention for complex tasks [10] - The shift from traditional SEO to Generative Engine Optimization (GEO) indicates a change in how brands and content creators engage with AI-driven information retrieval [11]
奥特曼的“帝国隐忧”:多线扩张,正在拖慢ChatGPT
创业邦· 2025-12-24 03:25
以下文章来源于腾讯科技 ,作者值得关注的 腾讯科技 . 腾讯新闻旗下腾讯科技官方账号,在这里读懂科技! 来源丨 腾讯科技 (ID:qqtech) 作者丨陆陆 编辑丨 徐青阳 图源丨 midjourney 过去一年,一个令人费解的现象在OpenAI内部蔓延: 即便ChatGPT推出了能在国际数学奥赛摘金、 在顶级编程竞赛夺冠的"最强大脑",但普通用户们似乎并不买账 。 据外媒报道和OpenAI 9月发布的数据显示,多数用户使用ChatGPT可能只是询问相当简单的问题, 根本无需动用那些耗费巨大计算资源、需要"思考"半分钟的推理模型。 这一刺眼的数据,指向了OpenAI在巅峰之下隐藏的深刻危机:一场由CEO山姆·奥特曼亲自推动的战 略扩张正引发严重的深层危机,包括组织架构割裂、多线作战导致资源分散,以及技术路线与用户需 求严重脱节,这正将其王牌产品ChatGPT拖入竞争泥潭。 核心矛盾:前沿研究与大众需求的 "性能过剩"鸿沟 OpenAI的核心矛盾,根植于其研究部门与产品团队日益扩大的目标分歧。 公司内部一个超过千人、相对独立的研究团队,近年来将重心押注在追求"推理模型"和"通用人工智 能"( AGI )这一终极 ...
奥特曼的“帝国隐忧”:多线扩张,正在拖慢ChatGPT
3 6 Ke· 2025-12-23 00:33
据外媒报道和OpenAI 9月发布的数据显示,多数用户使用ChatGPT可能只是询问相当简单的问题,根本无需动用那些耗费巨大计算资源、需 要"思考"半分钟的推理模型。 过去一年,一个令人费解的现象在OpenAI内部蔓延:即便ChatGPT推出了能在国际数学奥赛摘金、在顶级编程竞赛夺冠的"最强大脑",但普 通用户们似乎并不买账。 这一刺眼的数据,指向了OpenAI在巅峰之下隐藏的深刻危机:一场由CEO山姆·奥特曼亲自推动的战略扩张正引发严重的深层危机,包括组织 架构割裂、多线作战导致资源分散,以及技术路线与用户需求严重脱节,这正将其王牌产品ChatGPT拖入竞争泥潭。 01 核心矛盾:前沿研究与大众需求的"性能过剩"鸿沟 公司内部一个超过千人、相对独立的研究团队,近年来将重心押注在追求"推理模型"和"通用人工智能"(AGI)这一终极目标上。这种模型虽 然能在复杂数学和科学问题上表现出色,但其代价是高昂的计算成本和缓慢的响应速度,处理一个问题可能需要数秒甚至数分钟。 然而,这与ChatGPT数亿主流用户的需求严重脱节。AI评估机构LMArena负责人彼得·戈斯特夫所言,"OpenAI 把重心放在'科学、数学基准测 ...