Scaling Law
Search documents
诺奖得主的3个提醒:AI会办事了,世界就变了
3 6 Ke· 2025-12-28 03:44
Core Insights - AI is transitioning from merely providing answers to actively thinking, generating data, and executing tasks, indicating a fundamental shift in its capabilities [1][4][11] Group 1: AI's Evolving Capabilities - AI is developing reasoning abilities, leading to a reduction in hallucinations, which were a significant issue in chatbots [5][11] - The reasoning process involves understanding language contextually rather than converting sentences into logical symbols, allowing for more natural connections between words [6][9] - AI is moving from passive response to active execution, creating a new collaborative relationship between humans and machines [4][11] Group 2: Self-Learning and Data Generation - The next generation of AI will not rely on human-provided data but will generate its own training data through self-learning [15][20] - Hinton cites AlphaZero as an example of AI that learns through self-play, suggesting that AI could excel in fields like mathematics by generating infinite training data [16][17] - This shift represents a fundamental change in training paradigms, moving from external data input to internal self-driven learning [20][21] Group 3: Role Transformation in Collaboration - The concept of "agents" is emerging, where AI can understand tasks, break down processes, and execute them autonomously [23][29] - In fields like healthcare and education, AI is beginning to take over roles traditionally held by humans, enhancing efficiency and accuracy [25][26] - As AI becomes more proactive, the human role shifts from execution to decision-making, requiring a redesign of collaborative frameworks [31][32]
2025AI盘点:10大“暴论”
3 6 Ke· 2025-12-26 00:52
Group 1 - The concept of "Vibe Coding" has emerged, suggesting a new programming approach that emphasizes feeling and embracing exponential growth, leading to a broader trend of "Vibe Everything" in AI [2] - There is a divide in perception regarding "Vibe," with some viewing it as a refreshing product philosophy while others criticize it as a superficial trend that obscures the true essence of AI products [2] - The term "Vibe" reflects a strong narrative appeal, resonating with the desire for transformative change in the AI landscape, indicating its continued relevance in the future [2] Group 2 - The humanoid robot sector is experiencing a valuation surge despite discussions about a potential bubble, with significant capital inflow and a shift towards more conservative funding strategies among companies [6] - The focus on "scene" applications for humanoid robots has intensified, with education and performance being the most viable commercial scenarios, while the pursuit of commercial viability may not be the primary goal for the sector [6] Group 3 - The phrase "Prompt Engineering is Dead" has gained traction, suggesting a shift towards "Context Engineering," which encompasses a broader range of information and tools necessary for AI tasks [8][9] - Context Engineering is seen as a significant advancement, attracting investment and fostering the development of various AI tools, indicating a potential shift in the industry narrative [9] Group 4 - Huang Renxun's assertion that "China will win the AI race" highlights the competitive landscape between China and the U.S., emphasizing China's advantages in developer scale, market size, and infrastructure [12][13] - Huang's comments may also serve as a strategic move to influence U.S. policy regarding AI, aiming to maintain Nvidia's leadership position in the global market [12] Group 5 - Elon Musk and Satya Nadella predict the disappearance of traditional smartphones and apps, suggesting a transition to intelligent agents that could replace conventional software applications [15][16] - The emergence of new devices like the "Doubao phone" indicates a shift in how technology is being approached, with a focus on user interface and system control [16] Group 6 - Sam Altman's response to skepticism about OpenAI reflects a broader divide in opinions regarding the AI bubble, with concerns about the company's ability to deliver on its ambitious revenue projections [19][20] - OpenAI's projected revenue growth and the potential economic implications of AI's impact on employment and inflation are critical factors in assessing the sustainability of the AI market [21] Group 7 - The U.S. faces a potential electricity shortage that could impact AI infrastructure, with projections indicating a significant power gap by 2028 if supply does not keep pace with demand [23][24] - Major tech companies are exploring nuclear energy as a solution to their power needs, highlighting the intersection of AI development and energy infrastructure challenges [24] Group 8 - The debate surrounding the limitations of large language models (LLMs) continues, with experts arguing that scaling may not yield significant advancements and calling for a return to foundational research [27][28] - Despite criticisms, the push for larger models persists, indicating ongoing investment and interest in scaling within the AI community [28] Group 9 - The term "Slop" has been designated as the word of the year, representing the proliferation of low-quality AI-generated content, which poses challenges for content ecosystems [31][32] - The rise of AI-generated adult content is projected to grow significantly, raising questions about the implications for traditional content creation and quality standards [32]
算力芯片行业深度研究报告:算力革命叠浪起,国产 GPU 奋楫笃行
Huachuang Securities· 2025-12-24 05:32
Investment Rating - The report maintains a "Recommended" investment rating for the computing chip industry, particularly focusing on domestic GPU manufacturers [2]. Core Insights - The report emphasizes that the development of large models follows the "Scaling Law," indicating a rigid expansion of computing power demand. This is supported by quantifiable data on AI application deployment and computing consumption, establishing a commercial link where "computing power is production material" [6]. - The GPU industry is characterized by a concentrated market structure, with major players like NVIDIA dominating the landscape. The report highlights the ongoing strategic partnerships between cloud giants and NVIDIA, reinforcing the latter's core position in AI infrastructure [6][7]. - The report analyzes the domestic GPU manufacturers' response to U.S. export restrictions, detailing their technological advancements and market strategies. Companies like Cambricon, Haiguang Information, Moore Threads, and Muxi are highlighted for their efforts to catch up with international standards [6][7]. Summary by Sections 1. GPU's Role in AI - GPUs excel in parallel computing, making them suitable for AI acceleration. The architecture of GPUs allows for simultaneous processing of vast amounts of data, significantly reducing training times for AI models [11][12]. - The GPU industry value chain is primarily concentrated in the midstream, where AI chip demand drives market growth. The report notes that the global GPU market is expected to reach 1,051.54 billion yuan by 2024, with a significant portion attributed to AI computing GPUs [24][29]. 2. Global AI Investment Trends - Major global tech companies are increasing their investments in AI, with NVIDIA maintaining a dominant position. The report cites that NVIDIA holds a 98% market share in the data center GPU segment, underscoring its competitive edge [21][35]. - The report indicates that the AI investment cycle is achieving a closed loop, with companies like Google and Microsoft ramping up their capital expenditures significantly to support AI infrastructure [46][50]. 3. Domestic GPU Development - The report discusses the urgency for domestic GPU manufacturers to achieve self-sufficiency in light of U.S. export controls. Companies are making strides in product development and market entry, with varying degrees of commercial success [6][7]. - The report highlights the financial trajectories of domestic firms, noting that Haiguang Information achieved profitability in 2021, while Cambricon is expected to reach profitability by Q4 2024 [6][7]. 4. Market Projections - The report forecasts that the global GPU market will grow to 3,611.97 billion yuan by 2029, with China's share increasing from 15.6% in 2024 to 37.8% by 2029. AI computing GPUs are projected to be the core growth driver [24][29]. - The report anticipates that the demand for data center GPUs will continue to surge, with a projected market size of 663.92 billion yuan by 2029, reflecting a compound annual growth rate of 70.1% [29][31].
倒反天罡,Gemini Flash表现超越Pro,“帕累托前沿已经反转了”
3 6 Ke· 2025-12-22 10:12
Core Insights - Gemini 3 Flash has outperformed its predecessor Gemini 2.5 Pro and even the flagship Gemini 3 Pro in various performance metrics, achieving a score of 78% in the SWE-Bench Verified test, surpassing the Pro's score of 76.2% [1][5][6] - The Flash version demonstrates significant improvements in programming capabilities and multimodal reasoning, with a score of 99.7% in the AIME 2025 mathematics benchmark when code execution is included [5][6] - Flash's performance in the challenging Humanity's Last Exam test is competitive, scoring 33.7% without tools, closely trailing the Pro's 37.5% [5][6] Performance Metrics - In the SWE-Bench Verified test, Gemini 3 Flash scored 78%, while Gemini 3 Pro scored 76.2% [5][6] - In the AIME 2025 mathematics benchmark, Flash scored 99.7% with code execution, while Pro scored 100% [6] - Flash achieved 33.7% in the Humanity's Last Exam, compared to Pro's 37.5% [5][6] Cost and Efficiency - Gemini 3 Flash has a competitive pricing structure, with input costs at $0.50 per million tokens and output costs at $3.00 per million tokens, which is higher than Gemini 2.5 Flash but justified by its performance [7] - Flash's inference speed is three times that of Gemini 2.5 Pro, with a 30% reduction in token consumption [7] Strategic Insights - Google’s core team views the Pro model as a means to distill the capabilities of Flash, emphasizing that Flash's smaller size and efficiency are crucial for users [11][12] - The development team believes that the traditional scaling law is evolving, with a shift from merely increasing parameters to enhancing inference capabilities [12][14] - The emergence of Flash has sparked discussions about the validity of the "parameter supremacy" theory, suggesting that smaller, more efficient models can outperform larger ones [13][14]
信仰与突围:2026人工智能趋势前瞻
3 6 Ke· 2025-12-22 09:32
谁也无法想到,ChatGPT迎来三周年之际,没有庆祝和纪念,反而是内部发布的一封红色警报,再次敲响了人工智能竞争白热化的战鼓。在受到Gemini 3 惊艳效果的威胁下,Open AI加速推出了GPT 5.2,用更多的资源,在多项指标上实现了反超。但三年下来,各大模型之间的性能差距和范式差异持续缩 小,业界出现不少质疑的声音,认为大模型发展正面临天花板。但也有很多人坚定看好AGI的到来,产业充满了更多的争论和分化。 站在2025的年尾,回顾来时之路,从DeepSeek的火热,到GPT4o 后吉卜力动画的流行,Sora2的与山姆奥特曼同框,再到谷歌Nano Banana生图的各种机器 猫讲解。有时似乎有恍如隔世之感,一项今年的技术,仿佛已是多年前的流行。 展望2026,我们不仅感受到对大模型智能瓶颈和投资回报不确定性的焦虑,看到更多的非共识,也看到大家的坚守和信仰,以及有望在多个方向的突围, 更多的期待和探索正在扑面而来。 信仰 1.Scalling Law驱动向AGI持续进化 自 ChatGPT 横空出世以来,业界主流都相信只要不断增加算力、扩充数据、堆叠参数,机器的智能就会像物理定律一样增长,直至触达 AGI ...
信仰与突围:2026人工智能趋势前瞻
腾讯研究院· 2025-12-22 08:33
Core Insights - The article discusses the competitive landscape of AI, particularly focusing on the advancements and challenges faced by large models like ChatGPT and Gemini 3, highlighting the ongoing debate about the scalability and limitations of AI models [2][3][4]. Group 1: AI Model Development and Scaling - The belief that increasing computational power and data will lead to exponential growth in AI intelligence is being challenged as the performance improvements of large models slow down [3]. - Gary Marcus argues that large models do not truly understand the world but merely fit language correlations, suggesting that future breakthroughs will come from better learning methods rather than just scaling [3][4]. - Despite criticisms, the Scaling Law remains a practical growth path for AI, as evidenced by the successful performance of Gemini 3 and ongoing investments in AI infrastructure in the U.S. [4][5]. Group 2: Data Challenges and Solutions - High-quality data is a critical challenge for the evolution of large models, with the industry exploring systematic methods to expand data sources beyond just internet corpora [5][7]. - The future of data generation will focus on creating scalable, controllable systems that can produce high-quality data through various modalities, including synthetic and reinforcement learning data [7][19]. Group 3: Multi-Modal AI and Its Implications - The emergence of multi-modal models like Google Gemini and OpenAI Sora marks a significant advancement, enabling deeper content understanding and the potential for non-linear leaps in AI intelligence [8][12]. - Multi-modal models can provide a more direct representation of the world, allowing for a more robust world model and the possibility of closing the perception-action loop in AI systems [12][13]. Group 4: Research and Innovation in AI - The article highlights the importance of research-driven approaches in the AI industry, with numerous experimental labs emerging to explore various innovative directions, including safety and multi-modal collaboration [15][16][17]. - Innovations in foundational architectures and learning paradigms are expected to yield breakthroughs in areas such as long-term memory mechanisms and agent-based systems [15][17]. Group 5: AI for Science (AI4S) and Industry Impact - AI for Science is transitioning from model-driven breakthroughs to system engineering, with significant implications for fields like drug development and materials science [24][25]. - The establishment of AI-driven automated research labs signifies a shift towards integrating AI into experimental processes, potentially accelerating scientific discovery [25][28]. Group 6: AI Glasses and Consumer Electronics - The rise of AI glasses is anticipated to reach a critical mass, with projections of significant sales growth, indicating a shift towards a new computing paradigm [46][47]. - The design philosophy of AI glasses focuses on lightweight, user-friendly devices that prioritize functionality over traditional display technologies, potentially transforming user interaction with technology [47][48]. Group 7: AI Safety and Governance - As AI capabilities advance, safety and ethical considerations are becoming increasingly important, with a growing emphasis on establishing safety protocols and governance structures within AI development [50][53]. - The establishment of AI safety committees and the allocation of computational resources for safety research are becoming essential components of responsible AI deployment [54][55].
倒反天罡!Gemini Flash表现超越Pro,“帕累托前沿已经反转了”
量子位· 2025-12-22 08:01
Core Insights - Gemini 3 Flash outperforms its predecessor Gemini 2.5 Pro and even the flagship Gemini 3 Pro in various benchmarks, achieving a score of 78% in the SWE-Bench Verified test, surpassing Gemini 3 Pro's score of 76.2% [1][6][9] - The performance of Gemini 3 Flash in the AIME 2025 mathematics competition benchmark is notable, scoring 99.7% with code execution capabilities, indicating its advanced mathematical reasoning skills [7][8] - The article emphasizes a shift in perception regarding flagship models, suggesting that smaller, optimized models like Flash can outperform larger models, challenging the traditional belief that larger models are inherently better [19][20] Benchmark Performance - In the Humanity's Last Exam, Flash scored 33.7% without tools, closely trailing Pro's 37.5% [7][8] - Flash's performance in various benchmarks includes: - 90.4% in GPQA Diamond for scientific knowledge [8] - 95.2% in AIME 2025 for mathematics without tools [8] - 81.2% in MMMU-Pro for multimodal understanding [8] - Flash's speed is three times that of Gemini 2.5 Pro, with a 30% reduction in token consumption, making it cost-effective at $0.50 per million tokens for input and $3.00 for output [9] Strategic Insights - Google’s team indicates that the Pro model's role is to "distill" the capabilities of Flash, focusing on optimizing performance and cost [10][12][13] - The evolution of scaling laws is discussed, with a shift from merely increasing parameters to enhancing reasoning capabilities through advanced training techniques [15][16] - The article highlights the importance of post-training as a significant area for future development, suggesting that there is still substantial room for improvement in open-ended tasks [17][18] Paradigm Shift - The emergence of Flash has sparked discussions about the validity of the "parameter supremacy" theory, as it demonstrates that smaller, more efficient models can achieve superior performance [19][21] - The integration of advanced reinforcement learning techniques in Flash is cited as a key factor in its success, proving that increasing model size is not the only path to enhancing capabilities [20][22] - The article concludes with a call to reconsider the blind admiration for flagship models, advocating for a more nuanced understanding of model performance [23]
MiniMax海螺视频团队首次开源:Tokenizer也具备明确的Scaling Law
量子位· 2025-12-22 04:41
一水 发自 凹非寺 量子位 | 公众号 QbitAI MiniMax海螺视频团队不藏了! 首次开源 就揭晓了一个困扰行业已久的问题的答案—— 为什么往第一阶段的视觉分词器里砸再多算力,也无法提升第二阶段的生成效果? 翻译成大白话就是,虽然图像/视频生成模型的参数越做越大、算力越堆越猛,但用户实际体验下来总有一种微妙的感受——这些庞大的投入 与产出似乎不成正比,模型离完全真正可用总是差一段距离。 So why?问题,大概率就出在 视觉分词器(Tokenizer) 这个东西身上了。 当算力不再是答案时,真正需要被重新审视的,其实是生成模型的"起点"。 在当前主流的两阶段生成框架中 (分词器+生成模型) ,业界已经在视觉分词器的预训练上投入了大量算力与数据,但一个尴尬的事实是: 这些成本,几乎没有线性地转化为生成质量的提升 。 而MiniMax海螺视频团队,不止挑战了这一现实——用实验证明"Tokenizer的scaling能够提升模型性能"。 更关键的是,还带来了一款 开箱即用、专为"下一代生成模型"打造的可扩展视觉分词器预训练框架——Visual Tokenizer Pre-training (以下简称VTP) ...
Scaling Law没死,Gemini核心大佬爆料,谷歌已有颠覆性密钥
3 6 Ke· 2025-12-22 01:05
谷歌又要有重大突破了? 最近,Google DeepMind的Gemini预训练负责人Sebastian Borgeaud在采访中给出重磅爆料—— Google DeepMind的Gemini预训练负责人Sebastian Borgeaud在最近的访谈中表示,预计在未来一年内,针对提升长上下文处理效率以及进一步扩展模型上 下文长度的预训练技术,将会有重大创新。 未来一年,大模型预训练领域将在「长上下文处理效率」和「上下文长度扩展」两大方向迎来重大技术创新。 同时,Google Gemini三巨头——Jeff Dean、OriolVinyalsML和Noam Shazeer罕见同台了,他们的对谈中,跟Sebastian的内容展现出了惊人的一致。 众多高瞻远瞩、闪烁着智慧光芒的思想让人深思。 难怪,谷歌依然是那个巨人。 谷歌大佬激动预言 已破解大模型核心秘密 另外他还透露说,最近他们在注意力机制方面取得了一些非常有趣的发现,这可能在未来几个月内重塑他们的研究方向。 对此,他表示非常兴奋。 而且他提出了振聋发聩的一句话:Scaling Law并未消亡,只是正在演变! Sebastian Borgeaud是Gemin ...
自变量王潜:具身智能是物理世界的独立基础模型|MEET2026
量子位· 2025-12-21 05:45
Core Viewpoint - The embodiment intelligence model is considered an independent foundational model parallel to language and multimodal models, specifically designed for the physical world [6][12][61] Group 1: Differences Between Physical and Virtual Worlds - The fundamental differences between the physical and virtual worlds are recognized, with the physical world characterized by continuity, randomness, and processes related to force, contact, and timing [2][10] - Existing models based on language and visual paradigms are structurally misaligned with the complexities of the physical world [3][21] Group 2: Need for a Separate Foundational Model - A separate foundational model is necessary due to the significant randomness in the physical world, which existing models struggle to accurately represent [10][17] - The current reliance on multimodal models for embodiment intelligence is seen as inadequate, necessitating a complete rethinking of model architecture and training methods [9][21] Group 3: Future of Multimodal Models - Shifting perspectives on embodiment intelligence will lead to new insights in model architecture and data utilization [24][30] - The learning processes in the physical world differ fundamentally from those in the virtual world, suggesting that future multimodal models must adapt to these differences [25][28] Group 4: Scaling Laws and Data Utilization - The concept of Scaling Law is crucial in the development of large models, particularly in robotics, where data sourcing remains a significant challenge [47][49] - A phased approach to training and data collection is recommended, emphasizing the importance of real-world data for effective learning [52][53] Group 5: Hardware and AI Integration - A new learning paradigm necessitates the redesign of hardware in the physical world, advocating for AI to define hardware rather than the other way around [54][55] - The potential for embodiment intelligence to drive exponential growth in resources and capabilities is highlighted, drawing parallels to historical industrial advancements [60][61]