对话嘉宾 | 李建忠、 Lukasz Kaiser 出品 | CSDN（ID：CSDNnews） 今年开年之际，DeepSeek R1 配合前年年末 OpenAI o1 轰炸了整个 AI 圈子，随后强化学习之父 Rich Sutton 荣获图灵奖，又是用一篇论文向大家宣 告了强化学习、经验时代这些词汇将成为 2025 的主题，我们可能都难免这么觉得： 推理模型 的时代已经来了！ 但接下来的一个观点却刷新了我的认知：Transformer 核心发明者之一、OpenAI 科学家 Lukasz Kaiser 就直言，目前的推理模型还处在当年 GPT 都 没出来的机器学习阶段， 未来还需要一个 Transformer 创新级别的推理模型。 而近期，这位定义了大模型核心架构的关键人物，就与奇点智能研究院院长、CSDN 高级副总裁李建忠一道，在 CSDN 的《AI 进化论》栏目中展开了一 场关于 "大模型的第一性思考" 的深度对话。 Lukasz Kaiser 是 AI 领域最具影响力的科学家之一，2017 年他与其他七位谷歌同事（后称"Transformer 八子"）共同撰写了那篇开创性的论文 《Attention I ...

Core Insights - OpenAI is pursuing three main goals: to become a personal AI subscription service, to build large-scale infrastructure, and to achieve a truly useful AGI (Artificial General Intelligence) [2][4][29] - The recent launch of Sora 2 and various investment collaborations, including partnerships with AMD and Nvidia, indicate a strategic shift towards aggressive infrastructure investment [1][29] Group 1: OpenAI's Strategic Goals - OpenAI aims to become a personal AI subscription service, necessitating the construction of vast infrastructure to support this vision [4][29] - The ultimate mission is to create AGI that is genuinely beneficial to humanity, which requires a multifaceted approach beyond traditional business models [4][8] - OpenAI's infrastructure is currently intended for internal use, with future possibilities for external applications remaining uncertain [5][29] Group 2: Sora's Role in AGI Development - Despite skepticism about Sora's relevance to AGI, OpenAI's CEO believes that developing a "truly outstanding world model" through Sora will be crucial for AGI [10][11] - The resources allocated to Sora are relatively small compared to OpenAI's overall computational capacity, emphasizing a balanced approach to innovation and research [13][29] - Sora is seen as a way to engage society with upcoming technological advancements, particularly in video models, which resonate more emotionally than text [16][29] Group 3: Future Interactions and AI Capabilities - OpenAI envisions future interaction interfaces that go beyond basic chat, incorporating real-time video rendering and context-aware hardware [19][21] - The concept of the Turing Test is evolving, with the new benchmark being AI's ability to conduct scientific research, which OpenAI anticipates will happen within two years [21][22] - OpenAI's confidence in its research roadmap and the economic value it can generate has led to a commitment to aggressive infrastructure investments [29][31] Group 4: Leadership and Management Philosophy - OpenAI's CEO acknowledges a preference for an investor role over management, citing challenges in handling organizational dynamics and operational details [41][42] - The transition from an investor to a CEO role has been described as both challenging and rewarding, providing insights into groundbreaking work in AI [41][43] - The future of AI development is closely tied to energy availability, with a call for more efficient energy solutions to support AI advancements [44]

Core Insights - The article emphasizes the shift in focus from pre-training to post-training in large language models (LLMs), highlighting the diminishing returns of scaling laws as model sizes reach hundreds of billions of parameters [2][3][11]. Group 1: Importance of Post-Training - Post-training is recognized as a crucial phase for enhancing the reasoning capabilities of models like OpenAI's series, DeepSeek R1, and Google Gemini, marking it as a necessary step towards advanced intelligence [3][11]. - The article introduces various innovative post-training methods such as Reinforcement Learning from Human Feedback (RLHF), Reinforcement Learning from AI Feedback (RLAIF), and Reinforcement Learning with Verifiable Rewards (RLVR) [2][3][12]. Group 2: Transition from Pre-Training to Post-Training - The evolution from pre-training to instruction fine-tuning is discussed, where foundational models are trained on large datasets to predict the next token, but often lack practical utility in real-world applications [7][8]. - Post-training aims to align model behavior with user expectations, focusing on quality over quantity in the datasets used, which are typically smaller but more refined compared to pre-training datasets [11][24]. Group 3: Supervised Fine-Tuning (SFT) - Supervised Fine-Tuning (SFT) is described as a process that transforms a pre-trained model into one that can follow user instructions effectively, relying on high-quality instruction-answer pairs [21][24]. - The quality of the SFT dataset is critical, as even a small number of low-quality samples can negatively impact the model's performance [25][26]. Group 4: Reinforcement Learning Techniques - Reinforcement Learning (RL) is highlighted as a complex yet effective method for model fine-tuning, with various reward mechanisms such as RLHF, RLAIF, and RLVR being employed to enhance model performance [39][41]. - The article outlines the importance of reward models in RLHF, which are trained using human preference data to guide model outputs [44][46]. Group 5: Evaluation of Post-Training Models - The evaluation of post-training models is multifaceted, requiring a combination of automated and human assessments to capture various quality aspects [57][58]. - Automated evaluations are cost-effective and quick, while human evaluations provide a more subjective quality measure, especially for nuanced tasks [59][60].

Core Viewpoint - Alibaba's CEO, Wu Yongming, delivered a notable presentation at the Yunqi Conference, which led to a significant 9.16% increase in Alibaba's stock price, indicating strong investor sentiment despite a generally cautious market environment [1][3]. Group 1: Company Developments - Wu Yongming highlighted that large models will dominate software as the next-generation operating system, and Alibaba Cloud plans to invest further in AI infrastructure beyond its existing 380 billion yuan commitment over three years [3]. - Alibaba Cloud's Qwen3-Max model has achieved significant advancements, including an increase in pre-training data from 18 terabytes to 36 terabytes, and a focus on scaling laws to enhance model performance [6][10]. - The company has positioned itself as a leader in the AI cloud market, with a reported 35.8% market share, significantly ahead of competitors [16][22]. Group 2: Competitive Landscape - The competition in the AI cloud sector is intensifying, particularly with ByteDance's Volcano Engine, which has captured a 49.2% market share in model-as-a-service (MaaS) [16][18]. - Despite the competitive pressure, Alibaba Cloud has maintained a strong position, with over 53% of Fortune 500 companies using its services for generative AI [16][22]. - The market dynamics are shifting, with a trend towards self-deployment of models on Alibaba Cloud rather than relying solely on API calls, which may not be fully reflected in market share statistics [16][22]. Group 3: Technological Innovations - Alibaba Cloud has made significant strides in AI infrastructure, including the development of a new AI chip that approaches NVIDIA's capabilities and a high-performance network architecture that supports large-scale GPU interconnectivity [25][27]. - The company is focusing on a comprehensive stack for AI infrastructure, which positions it well in the context of increasing domestic demand for AI capabilities [27]. - Innovations in model architecture, such as the introduction of the Qwen3-Next model with a sparse MoE architecture, demonstrate Alibaba's commitment to advancing AI technology [6][10].

2025年，人形机器人行业正站在关键的转型节点，从早期的"主题炒作"逐步迈向"产业趋势投资前期"。华泰证 券近日研报显示，随着特斯拉、Figure等海外龙头及国内企业开启小批量量产， 市场对人形机器人的远期空 间已形成共识，但真正推动行业进入非线性增长的"Scaling Law"（规模法则）时刻 ，仍取决于硬件降本与 机器人大脑智能的双重突破。具身智能作为核心驱动力，正从技术探索走向工程落地，重塑行业技术路线与商 业化路径。 当前人形机器人的核心矛盾并非"能否出货"，而是"能否形成可持续的产业飞轮"。2024年底至2025年初， 国内多家本体企业已完成百台至千台交付，但交付场景多集中于科研、教育、展示等ToG领域 ，采购方更多 用于算法研发，本体企业仍扮演"硬件卖铲人"角色，软件层的智能突破尚未显现。华泰证券指出，初期订单 数并非关键信号，行业真正的转折点在于"机器人大脑的Scaling Law时刻"：即智能随数据量、模型规模增 长呈非线性提升，进而突破场景泛化能力瓶颈。 ▍ 为何Scaling Law时刻还未出现 从产业卡点看，两大难题亟待解决。硬件端主要体现在成本高、方案未定型。以特斯拉Optimus G ...

Group 1 - Baidu's search engine is undergoing a significant transformation towards AI integration, referred to internally as "Big Search," marking the largest change in a decade [1] - The AI-driven agent model is expected to assist users in completing tasks beyond traditional keyword searches, indicating a shift in user interaction [1] - Baidu's Wenku and cloud storage services are also expanding, aiming to create a "one-stop AI creation platform" with a dedicated team of 1,200 [1] Group 2 - The article discusses the evolution of the internet ecosystem, highlighting the complexity of user needs and the competitive landscape dominated by major players like BAT and FANG [2] - The historical context of the internet's development is explored, noting the transition from information-centric models to more integrated social and e-commerce platforms [3] Group 3 - The recommendation engine developed by Baidu is based on user behavior data, aiming to enhance targeted advertising through detailed user profiling [5] - The article critiques the current state of content production, suggesting that the focus on quantity over quality has led to a decline in meaningful engagement [6] Group 4 - The dominance of algorithm-driven content distribution is noted, with implications for user experience and the overall information ecosystem [8] - Baidu's market position is analyzed in light of competition from ByteDance, emphasizing the challenges faced by traditional search models in adapting to new content consumption patterns [8] Group 5 - The article reflects on the missed opportunities for Baidu in the early days of algorithm distribution, suggesting that a more proactive approach could have altered its competitive stance [11] - The potential of AI to revolutionize information access and user interaction is highlighted, with a focus on the implications for Baidu's future strategies [19][20] Group 6 - Baidu's early commitment to AI, including the establishment of a deep learning research institute, is acknowledged, though recent performance in AI competitions has raised questions about its strategic direction [20] - The article emphasizes the importance of application development in AI, suggesting that successful models will depend on practical use cases rather than theoretical frameworks [32]

Core Viewpoint - The article discusses the future of AI and embodied intelligence, emphasizing the need for disruptive innovation to enable generalized action capabilities and the transition from technical feasibility to commercial success [2][4]. Group 1: Embodied Intelligence Development - The concept of embodied intelligence has evolved from simply giving machines a physical body to creating immersive perception processes [6]. - Current challenges in the field include data bottlenecks, which can be addressed through the establishment of training grounds that enhance robustness and generalization capabilities [7]. - The industry is witnessing a surge in the construction of training grounds, which offer benefits such as cost reduction, safety simulation, and unified standards [7]. Group 2: Data Collection and Utilization - Training grounds are described as new data factories in the AI era, crucial for collecting data to train embodied intelligence models [8][10]. - The development paradigm has shifted to a model where data collection occurs post-robot development, emphasizing the importance of large datasets for effective training [10][11]. - The use of synthetic data is highlighted as a viable solution to the challenges of obtaining real-world data, allowing for scalable and controllable training processes [18][19]. Group 3: Future Prospects and Challenges - The industry is exploring various paths for embodied intelligence, including the integration of real-world data and simulation data to enhance model performance [30][31]. - Discussions on the potential of humanoid robots reveal that while they may not be the only form of embodied intelligence, their development is crucial for achieving broader applications [34][35]. - The timeline for the integration of embodied intelligence into daily life is projected to be gradual, with significant advancements expected in the next 5 to 10 years [38]. Group 4: Industry Collaboration and Ecosystem - The need for collaboration across the industry is emphasized, with calls for the establishment of a robust ecosystem to support the development of embodied intelligence [48][49]. - Various stakeholders express the importance of integrating hardware and software capabilities to enhance the overall effectiveness of embodied intelligence solutions [47][49]. - The article concludes with a vision for a future where embodied intelligence significantly transforms industries and daily life, driven by collective efforts from academia and industry [51].

Core Insights - The Scaling Law is being questioned due to perceived diminishing returns in model training, but recent research suggests that small improvements in accuracy can lead to exponential growth in task completion length, which may hold more economic value in real-world applications [1][2][4] Group 1: Research Findings - A recent paper from Cambridge University indicates that while there are diminishing returns in metrics like test loss, the real-world value of large language models (LLMs) often comes from their ability to complete longer tasks [2][4] - The paper highlights that the long-term execution of tasks has been a significant weakness in deep learning, with LLMs struggling to perform complex, lengthy tasks despite improvements in reasoning capabilities [4][6] - The authors propose that the failures in long tasks are primarily due to execution challenges rather than reasoning or planning limitations, emphasizing the need for more focus on execution capabilities in LLM research [6][20] Group 2: Experimental Insights - The study measures LLMs' long-horizon execution capabilities by isolating execution from planning and knowledge retrieval, revealing that larger models can significantly increase the number of successful execution rounds [6][23][25] - The concept of self-conditioning is introduced, where the model's performance deteriorates as it builds on its previous errors, leading to a decline in accuracy over multiple rounds [8][26][30] - The research shows that while increasing model size improves task execution, it does not alleviate the self-conditioning effect, which remains a challenge for LLMs in long-term tasks [27][30] Group 3: Implications for Investment - The findings suggest that the economic value of LLMs may not be accurately reflected in short-task benchmarks, as the ability to complete longer tasks is a more reliable indicator of their potential [18][20] - The paper encourages further investment in scaling models, as the ability to perform longer tasks could justify continued financial commitment despite short-term performance metrics suggesting stagnation [10][18] - The research calls for the design of new benchmarks that better assess the execution depth of models, highlighting a potential area for future investment and development in the AI sector [10][18]

Core Insights - xAI has implemented a sudden internal assessment leading to a significant layoff of its data annotation team, with a reported attrition rate of 33% and over 500 employees terminated [1][11]. Group 1: Layoff Details - The data annotation team, crucial for the development of Grok, has seen its size decrease from 1500 to just over 1000 employees, indicating a nearly one-third reduction [11]. - The layoffs were preceded by a series of one-on-one discussions with employees, creating a sense of panic within the company [5][7]. - The company announced a strategic shift towards hiring specialized data annotators, planning to expand their numbers tenfold, while reducing the focus on general data annotators [11][12]. Group 2: Strategic Shift - This shift from general to specialized data annotation reflects a belief that quality is more important than quantity, aiming to enhance Grok's capabilities in specific fields [12][14]. - The decision may limit the diversity of data available for training, which is essential for the growth of AI systems [12][14]. - The move is seen as a significant gamble on vertical industry AI applications, potentially positioning Grok advantageously if successful [14][15]. Group 3: Management Philosophy - Elon Musk's management style is characterized by a preference for small, high-performing teams, often leading to drastic layoffs to maintain efficiency and performance [22][24]. - This approach has been consistent across Musk's ventures, including Tesla and Twitter, where he has previously enacted similar layoffs to streamline operations [20][24]. - The emphasis on high performance and low tolerance for underachievement is a hallmark of Musk's leadership, which may drive the remaining employees to maximize their potential [22][25].

Jay 发自 凹非寺 量子位 | 公众号 QbitAI 什么情况，帮马斯克训练大模型的人说失业就失业了？ 马斯克裁员式考核 数据标注团队曾是xAI最大的团队，在Grok的开发过程中发挥了关键作用。他们的工作是标记、分类并将原始数据置于特定语境中，从而教 会AI如何更好地理解世界。 自xAI成立以来，数据标注团队的规模一直在持续增长。 与大多数人工智能公司不同，xAI的许多数据标注员都是直接聘请的，而非外包 。通过这种方式，可以让公司对模型训练拥有更多的控制 权，更好的隐私。 但相应的，成本也更高。 今年2月份，xAI披露计划雇用数千人来帮助训练Grok，并在半年内新增了约700名数据标注员。 上周四晚，xAI内部上演了一场突袭测试，还要求员工必须在第二天早上之前完成并提交。 这可不是一次简单的随堂测试—— 截至目前，本次xAI内部测试的淘汰率高达33%，已有 超过500名员工 被通知卷铺盖走人。 然而9月初，Linkedin页面显示，负责管理数据标注团队的十几名经理中， 至少已有9位被解雇 。 这次不太寻常的人事变动，为即将到来的剧烈动荡埋下了种子。 之后一段时间内，xAI开始与数据标注团队的部分员工开展 一 ...