AI界「双神会」来了！一场NeurIPS 2025炉边谈话，AI教父Hinton和Jeff Dean同台，亲口爆料了AI革命「那些年」，还有更多鲜为人知的轶 事。 NeurIPS 2025那场轰动一时的访谈，如今终于放出来了！ AI教父Hinton和DeepMind首席科学家Jeff Dean，两位AI圈关键人物，曾经合作多年的老友聚在一起。 现场，Hinton直接抛出了一个尖锐的问题—— 谷歌是否后悔发表Transformer论文？ Jeff Dean给出了干脆的回应，「不后悔！因为它对世界产生了巨大的影响」。 不仅如此，Hinton还公开透露，自己关于Scaling的顿悟，源于Ilya的一场演讲。 在近1小时的对话中，两位大佬回顾了从ML早期突破，到当今塑造该领域的挑战、机遇等等。 他们还分享了，一些非常精彩的轶事—— 从卧室运行AlexNet的两块GPU，到谷歌大脑（Google Brain）的早期岁月。 AI教父Scaling顿悟，来自Ilya 对话的开场，先从一个有趣的共同点开始： 两位Geoff和Jeff都对「反向传播」（backpropagation）着迷。 这一概念的论文虽在1986年于Nat ...

2025 年 12 月初，圣地亚哥 NeurIPS 大会。 Geoffrey Hinton（神经网络奠基人、2024年诺贝尔物理学奖得主）与Jeff Dean（Google首席科学家、 Gemini模型联合负责人、TPU架构师）的炉边对谈，成为这场大会的重要时刻。 对话聚焦一个关键问题： 现代 AI 为什么能从实验室走向数十亿用户？ 从 AlexNet 在学生卧室的两块 GPU 上训练，到 Google 在餐巾纸上算出TPU需求；从学术圈的小众实 验，到支撑全球亿级应用的基础设施。 这是一次对 AI 工业化进程的系统性复盘。 他们给出的答案是：现代 AI 的突破从来不是单点奇迹，而是算法、硬件、工程同时成熟后的系统性涌 现。强算法必须与强基础设施结合，才能真正走向规模化。 看清这条路径，你就能理解AI为什么是今天这个样子。 第一节｜AI的突破，起于一块GPU板 Geoffrey Hinton 说，现代 AI 真正的转折，不在某篇论文里，而是在他学生 Alex 的卧室里：两块 NVIDIA GPU 板，插在父母家电脑上，训练图像识别模型。电费，还是家里人掏的。 那是 2012年 ，ImageNet 比赛。 别人 ...

Core Insights - Waymo has released a deep blog detailing its AI strategy centered around its foundational model, emphasizing the use of distillation methods to create high-efficiency models for onboard operations [1][2] - Jeff Dean highlighted the significance of knowledge distillation, comparing it to the creation of the Gemini Flash model, which showcases the importance of distillation in AI model efficiency [1][2] Historical Context of Rejected Papers - Many foundational technologies in AI, such as optimizers for large models and computer vision techniques, were initially rejected by top conferences, showcasing a historical pattern of oversight in recognizing groundbreaking innovations [6] - Notable figures in AI, including Geoffrey Hinton and Yann LeCun, have faced rejection for their pioneering work, which was later recognized as transformative [6] Case Studies of Rejected Innovations - LSTM, a milestone for sequence data processing, was rejected by NIPS in 1996 but later became crucial in speech recognition and machine translation, highlighting the delayed recognition of its value [7][10] - SIFT, a dominant algorithm in computer vision, faced rejection from ICCV and CVPR due to its perceived complexity, yet proved to be vital in real-world image processing [11][13] - Dropout, a key regularization method for deep neural networks, was initially rejected for its radical approach but later became essential in training deep networks effectively [17][19] - Word2Vec, despite being rejected at ICLR, became a cornerstone in NLP due to its efficiency and practical application, eventually receiving recognition for its impact [20][24] - YOLO transformed object detection by prioritizing speed over precision, facing rejection for its perceived shortcomings but later becoming a widely adopted framework in the industry [28][30] Reflection on Peer Review Limitations - The peer review system often struggles to recognize disruptive innovations, leading to a systematic cognitive lag in evaluating groundbreaking research [40][41] - The tendency to equate mathematical complexity with research contribution can hinder the acceptance of simpler yet effective methods [41] - Historical examples illustrate that the true measure of a research's impact is not determined by initial peer review outcomes but by its long-term relevance and problem-solving capabilities [43][47]

Core Insights - The article highlights the significance of the "Test of Time Award" received by the paper "Faster R-CNN," co-authored by renowned researchers, marking its impact on the field of computer vision since its publication in 2015 [1][5][25] - The presentation by He Kaiming at NeurIPS 2025 summarizes the evolution of visual object detection over the past 30 years, showcasing key milestones and influential works that have shaped the field [6][31] Historical Development - The early attempts at face detection in the 1990s relied on handcrafted features and statistical methods, which were limited in adaptability and speed [12] - The introduction of AlexNet in 2012 demonstrated the superior feature extraction capabilities of deep learning, paving the way for its application in object detection [15] - The R-CNN model, proposed in 2014, revolutionized object detection by integrating CNNs for feature extraction and classification, although it initially faced computational challenges [17][18] Technological Advancements - The development of Faster R-CNN in 2015 addressed the speed bottleneck by introducing the Region Proposal Network (RPN), allowing for end-to-end real-time detection [25] - Subsequent innovations, such as YOLO and SSD in 2016, further enhanced detection speed by enabling direct output of object locations and categories [32] - The introduction of Mask R-CNN in 2017 added instance segmentation capabilities, while DETR in 2020 redefined detection using Transformer architecture [32][34] Future Directions - The article concludes with reflections on the ongoing exploration in computer vision, emphasizing the need for innovative models to replace outdated components as bottlenecks arise [35][36]

Core Insights - The discussion highlights the transformative impact of artificial intelligence (AI) and the role of NVIDIA in driving this technological revolution, emphasizing the importance of GPUs in various applications from gaming to modern data centers [2][10]. Group 1: AI and Technological Competition - The conversation underscores that the world is in a significant technological race, particularly in AI, where the first to reach advanced capabilities will gain substantial advantages [11][12]. - Historical context is provided, indicating that the U.S. has always been in a technological competition since the Industrial Revolution, with AI being the latest frontier [12][13]. Group 2: Energy and Manufacturing - The importance of energy growth and domestic manufacturing is emphasized as critical for national security and economic prosperity, with a call for revitalizing U.S. manufacturing capabilities [8][9]. - The discussion points out that without energy growth, industrial growth and job creation would be severely hindered, linking energy policies directly to advancements in AI and technology [9][10]. Group 3: AI Development and Safety - Concerns about the risks associated with AI are acknowledged, particularly regarding its potential military applications and ethical implications [19][20]. - The conversation suggests that AI's development will be gradual rather than sudden, with a focus on enhancing safety and reliability in AI systems [14][15]. Group 4: Future of AI and Knowledge Generation - The potential for AI to generate a significant portion of knowledge in the future is discussed, with predictions that AI could produce up to 90% of knowledge within a few years [41][42]. - The necessity for continuous verification of AI-generated information is highlighted, stressing the importance of ensuring accuracy and reliability in AI outputs [41][42]. Group 5: Cybersecurity and Collaboration - The dialogue emphasizes the collaborative nature of cybersecurity, where companies share information and best practices to combat threats collectively [23][24]. - The need for a unified approach to cybersecurity in the face of evolving threats is reiterated, suggesting that cooperation is essential for effective defense [23][24].

公众号记得加星标⭐️，第一时间看推送不会错过。 近日，英伟达首席执行官黄仁勋做客乔·罗根（Joe Rogan）的播客节目，深入探讨了人工智能革命、英伟达的起源，以及GPU如何驱动从游戏到现 代数据中心等方方面面的应用。在这次内容丰富的对话中，黄仁勋谈到了人工智能的真正风险与回报、全球"人工智能竞赛"，以及能源和制造业为 何是未来创新的核心。 他还分享了自己作为移民的成长经历，以及将英伟达打造成为一家改变世界的公司的早期奋斗历程。如果您想了解未来几十年计算、国家安全和日 常生活的发展方向，那么这期节目绝对不容错过。 黄仁勋：你知道，他魅力的一部分——或者说，他才华的一部分——是的，他直言不讳。 以下为对话原文： 黄仁勋：嘿，乔。 乔·罗根：很高兴再次见到你。我们刚才还在聊——那是我们第一次交谈吗？还是——我们第一次交谈是在SpaceX？ 黄仁勋：SpaceX。 乔·罗根：SpaceX。第一次你把那块疯狂的人工智能芯片交给Elon Musk的时候。 黄仁勋：是吗？ DGX Spark。 乔·罗根：是的。 黄仁勋：哦，那是—— 乔·罗根：那是一个重要的时刻。 黄仁勋：那是一个意义重大的时刻。 乔·罗根：当时在场感觉 ...

Core Viewpoint - The current AI boom is fundamentally misdirected, with an overemphasis on scaling and computational power rather than genuine research and innovation [1][2]. Group 1: Scaling and Its Limits - The era of scaling through increased computational power is coming to an end, as the industry faces diminishing returns on investment in data and computation [3][5]. - High-quality training data is becoming scarce, leading to a plateau in performance improvements from current scaling methods [3][5]. - Existing models lack true intelligence and generalization capabilities, indicating a fundamental flaw in the underlying architecture [6][8]. Group 2: Generalization Challenges - Current AI models excel in benchmark tests but fail in real-world applications, revealing significant weaknesses in their generalization abilities [6][8]. - The focus on narrow optimization for specific tasks leads to models that perform well in limited contexts but struggle with broader applications [7][8]. - Understanding reliable generalization mechanisms is crucial for addressing various AI challenges, including alignment and value learning [8]. Group 3: SSI's Research Focus - Safe Superintelligence Inc. (SSI) aims to prioritize research over product development, challenging the industry's default assumptions about resource allocation [9][10]. - SSI's structure is designed to eliminate distractions from research, focusing solely on validating theories related to generalization [10]. - Historical precedents show that significant breakthroughs in AI do not require massive computational resources but rather insightful approaches [10]. Group 4: AGI and Its Misconceptions - The concept of Artificial General Intelligence (AGI) may be overestimated, as human intelligence operates differently from the proposed models [12]. - Human learning involves mastering foundational skills before acquiring complex abilities, contrasting with the notion of a universally capable AI [12]. - This understanding influences deployment strategies, suggesting that AI should be viewed as a system capable of continuous learning rather than a fully formed entity at launch [12]. Group 5: Future Predictions - Systems with improved generalization capabilities are expected to emerge within 5 to 20 years, reflecting uncertainty about the path forward rather than doubt about solutions [13]. - As AI capabilities become more apparent, industry behaviors will shift, leading to increased collaboration on safety and deeper government involvement [13]. - The alignment goal should encompass all sentient AI, not just humans, based on the premise of shared understanding across species [13]. Group 6: Research Aesthetics - The pursuit of research is driven by a sense of aesthetic and simplicity, with promising directions often appearing elegant and inspired by biological intelligence [14][15]. - A strong belief in the validity of certain research paths is essential for overcoming challenges and failures in the development process [15]. - The shift away from reliance on scaling as a substitute for belief in research direction emphasizes the need for genuine innovation and insight [15].

Core Insights - The interview with Fei-Fei Li highlights the emergence of "world models" as the next frontier in AI over the next decade, emphasizing the importance of spatial intelligence in AI development [1][28]. Group 1: Historical Context of AI - Two decades ago, AI was in a "winter" phase, with limited public interest and funding, often referred to as "machine learning" [10][14]. - Fei-Fei Li entered the AI field during this period, focusing on visual intelligence and the need for large datasets to train models effectively [11][20]. - The creation of ImageNet, which involved collecting 15 million images across 22,000 categories, marked a pivotal moment in AI, leading to the rise of deep learning [23][24]. Group 2: The Concept of World Models - "World models" are defined as systems that can generate an infinite 3D world based on input, allowing for reasoning and interaction [37]. - The Marble platform exemplifies this concept, significantly reducing production time in various industries, including film and gaming, by allowing creators to generate navigable worlds from simple descriptions [40][43]. - The integration of spatial intelligence into AI is seen as crucial for enhancing both robotic capabilities and human understanding [39][32]. Group 3: Challenges in Robotics - The primary challenge in robotics lies in data acquisition, as robots require extensive real-world interaction data, which is difficult to obtain [44][45]. - Unlike language models that operate on text, robots must navigate and interact within a 3D environment, complicating their training [45]. - The historical context of autonomous vehicles illustrates the complexities involved in developing effective robotic systems [46]. Group 4: Fei-Fei Li's Career and Vision - Fei-Fei Li's career trajectory reflects a commitment to addressing significant problems in AI, transitioning from academia to industry and now to entrepreneurship with World Labs [47]. - Her focus on collaboration and team dynamics underscores the importance of human roles in the evolving landscape of AI [47]. - Li emphasizes that every individual has a vital role in the future of AI, regardless of their profession [47].

Core Insights - The AI industry is transitioning from a phase of rapid investment and growth to a more competitive and cost-conscious environment, as evidenced by layoffs and restructuring among major players like Meta, OpenAI, and AWS [1][2]. Group 1: Historical Context of AI Development - Google was founded with AI as a core principle, influenced by co-founder Larry Page's background in machine learning [5][9]. - The term "Artificial Intelligence" was first coined in 1956, but the field faced significant setbacks due to limitations in computing power and data, leading to two major "AI winters" [8]. - Larry Page's vision for Google included the belief that AI would be the ultimate version of their search engine, aiming to understand everything on the web [9][10]. Group 2: Key Innovations and Breakthroughs - Google's early AI efforts included the development of the PHIL language model, which significantly improved search functionalities and contributed to the company's revenue through AdSense [14][15][16]. - The introduction of neural networks and deep learning at Google was catalyzed by the arrival of key figures like Geoff Hinton, who advocated for the potential of deep learning [19][21]. - The "cat paper," which demonstrated a deep learning model's ability to recognize images without supervision, marked a significant milestone for Google Brain and had profound implications for YouTube's content understanding [30][34]. Group 3: Competitive Landscape and Strategic Moves - The success of AlexNet in 2012 revolutionized deep learning and established GPU as the core hardware for AI, leading to a surge in interest and investment in AI talent [35][39]. - Google acquired DNN Research, further solidifying its leadership in deep learning, while Facebook established its own AI lab, FAIR, to compete in the space [41][43]. - The acquisition of DeepMind by Google in 2014 expanded its AI capabilities but also led to internal conflicts between DeepMind and Google Brain [56][57]. Group 4: Emergence of OpenAI and Market Dynamics - OpenAI was founded in 2015 with a mission to promote and develop friendly AI, attracting talent from Google and other tech giants [66][68]. - The launch of ChatGPT in late 2022 marked a pivotal moment in the AI landscape, rapidly gaining users and prompting a competitive response from Google [97][99]. - Google's response included the rushed launch of Bard, which faced criticism and highlighted the challenges of adapting to disruptive innovations [102][103]. Group 5: Future Directions and Challenges - Google is now focusing on the Gemini project, aiming to unify its AI efforts and leverage its extensive resources to compete effectively in the evolving AI landscape [105][106]. - The competitive dynamics in the AI industry are shifting, with emerging players in China and the ongoing evolution of established companies like OpenAI and Meta [109][110].

Core Insights - Yoshua Bengio has become the first scholar globally to surpass one million citations on Google Scholar, marking a significant milestone in AI academic influence [3][5][6] - Geoffrey Hinton follows closely with approximately 970,000 citations, positioning him as the second-highest cited scholar [5][6] - The citation growth of AI papers has surged, reflecting the current AI era's prominence [19][30] Citation Rankings - Yoshua Bengio ranks first globally in total citations, with a significant increase in citations post-2018 when he received the Turing Award [6][9][38] - Geoffrey Hinton ranks second, with a notable citation count of 972,944, showcasing his enduring impact in the field [5][8] - Yann LeCun, another Turing Award winner, has over 430,000 citations, but remains lower than both Bengio and Hinton [13][18] AI Research Growth - The total number of AI papers has nearly tripled from approximately 88,000 in 2010 to over 240,000 in 2022, indicating a massive increase in research output [30] - By 2023, AI papers constituted 41.8% of all computer science papers, up from 21.6% in 2013, highlighting AI's growing dominance in the field [31][32] - The foundational works of AI pioneers have become standard references in subsequent research, contributing to their citation growth [22][33] Key Contributions - The introduction of AlexNet in 2012 is considered a pivotal moment that significantly advanced deep learning methodologies [20] - The development of the Transformer model in 2017 and subsequent innovations like BERT have further accelerated research and citations in AI [24][27] - The increasing number of AI-related submissions to top conferences reflects the field's rapid evolution and the growing interest in AI research [36]