生成式AI
Search documents
哈工大孟维康:让注意力有 “棱角”|Attention
3 6 Ke· 2025-10-20 07:58
Core Insights - The article discusses the evolution and challenges of Linear Attention in the context of Vision Transformers, highlighting the need for improved efficiency and performance in AI models [1][2][3]. Group 1: Linear Attention Challenges - Linear Attention faces two main issues: the distribution of attention weights becomes too flat, reducing model sharpness, and the use of non-negative kernel functions leads to the loss of negative interaction information [2][9]. - The traditional Self-Attention mechanism has high computational costs and energy consumption, making it difficult for smaller teams and companies to compete [1][2]. Group 2: PolaFormer Innovation - PolaFormer introduces a dual-stream architecture that separates positive and negative interactions, allowing for independent processing of these relationships [4][6][10]. - The model employs a learnable channel-wise power function to enhance the sharpness of attention distributions, aiming to recover the expressiveness of Softmax Attention while maintaining efficiency [6][10][20]. Group 3: Experimental Validation - Extensive experiments demonstrate that PolaFormer effectively replaces Self-Attention in Vision Transformer frameworks, showing significant performance improvements across various tasks such as object detection, semantic segmentation, and long sequence benchmarks [7][31]. - The model's design allows it to maintain stable performance across different input types, including short texts and long sequences, without losing global information [9][29]. Group 4: Future Applications and Implications - PolaFormer is expected to enhance applications in long-sequence and high-resolution scenarios, such as video processing and large language models, by providing a more efficient solution without compromising performance [31][32]. - The research emphasizes the importance of co-designing algorithms with hardware to address deployment challenges, particularly in resource-constrained environments [30][31].
万条推文“怒轰”、估值下跌, OpenAI被误导性“突破”反噬!陶哲轩:有实力,但方向错了?
AI前线· 2025-10-20 05:23
整理 | 华卫 "搬起自己的 GPT 石头砸了自己的脚。"这是 Meta 首席 AI 科学家 Yann LeCun 对 OpenAI 研究员们的最新评价。 事件起因是,此前这些研究员因 GPT-5 的一项新数学"突破"而高调庆祝,但在受到整个 AI 社区质疑后又迅速撤回了该说法。连谷歌 DeepMind 首席执 行官 Demis Hassabis 也对此提出批评,称其沟通存在疏漏。 GPT-5"突破" 被证明是一个错误 取得"突破"的消息,最早是由前微软副总裁、现 OpenAI 研究科学家 Sebastien Bubeck 放出。他在 X 上称,两位研究人员在周末借助 GPT-5 找到了 10 个埃尔德什问题(Erdős problems)的答案。埃尔德什问题是匈牙利数学家 Paul Erdős 提出的一系列数学问题的统称,其中既包含未解决的难题,也有 已解决的问题,著名案例包括 "不同距离问题"(Distinct Distances Problem)与 "偏差问题"(Discrepancy Problem)。这类问题以难度高著称,常成为 学界深入研究的对象,部分问题甚至设有现金奖励,鼓励研究者攻克。 10 ...
拉斯·特维德:未来5年最具前景的5大投资主题
首席商业评论· 2025-10-20 04:21
Group 1 - The core investment themes for the next five years include technology, metals and mining, passion investments, ASEAN and Chinese markets, and biotechnology [9][30][40] - The rapid growth of AI technology is expected to drive significant profits in the future, with effective compute power increasing by 100,000 times from 2019 to 2023 [13][19] - The emergence of generative AI is anticipated to create strong business moats for companies that effectively utilize it, contrasting with the commoditization of large language models [20][19] Group 2 - The metals and mining sector is projected to face a potential shortage, particularly in uranium, silver, and platinum, with uranium prices expected to rise by 225% if they return to historical peaks [31][30] - Passion investments, such as prime real estate and limited edition assets, are expected to see increased demand as wealth grows, despite their supply remaining fixed [33] - The ASEAN and Chinese markets are highlighted for their potential growth, with China showing significant innovation capabilities and a favorable investment environment [36][38] Group 3 - The biotechnology sector is currently undervalued, with an average P/E ratio of 10-11, and is expected to benefit from AI advancements that lower R&D costs and accelerate product development [40][42] - The future of work is projected to be heavily influenced by AI, with estimates suggesting that 80% of jobs could be performed by intelligent robots by 2050 [29][22] - The development of physical AI, including robotics and autonomous vehicles, is expected to create a significant market by 2027-2028, with China positioned to play a crucial role [24][28]
你是否患上了AI冒名顶替综合征?
3 6 Ke· 2025-10-19 23:13
神译局是36氪旗下编译团队,关注科技、商业、职场、生活等领域,重点介绍国外的新技术、新观点、新风向。 编者按:越来越多的人正被一种悄然滋生的怀疑所困扰:因为比不上AI,因此质疑自己的能力。本文来自编译,希望对您有所启发。 过去两年职场中发生了一件耐人寻味的事。那些曾以敏捷、聪慧和机智自豪的人,如今正默默怀疑自己是否太迟钝、太笨拙、太传统,无法应对日常工作挑 战。 并非他们突然工作能力下降,而是新加入的"同事"永不疲倦、永不呆滞地盯着屏幕,几乎能在八秒内对任何问题给出精炼答案。生成式人工智能已成为办公 室里干劲十足的实习生,以令人不安的速度批量生产备忘录、演示文稿,甚至连冷笑话都信手拈来。然而面对这份免费劳动力,许多人非但没有享受其便 利,反而陷入自卑情绪。 欢迎来到AI冒名顶替综合征(AI Impostor Syndrome):当你发现自己无法与机器匹敌时,那种潜移默化的怀疑便悄然滋生。过去的冒名顶替综合征以(非 理性地)自认优越的同类为参照,如今的标杆却变成了日常使用的生成式AI。我们既依赖又钦佩它,却因此感到自身价值被削弱。 1. 何为AI冒名顶替综合征? 传统冒名顶替症源于人类间的才能质疑,而AI冒名顶 ...
AI资本开支太狂热了?高盛:这才到哪呢
美股IPO· 2025-10-19 22:59
Core Viewpoint - Despite record nominal investments in AI infrastructure, the current investment level is not excessive compared to historical technology cycles, with AI investment in the U.S. accounting for less than 1% of GDP, while peaks in past cycles like railroads and IT reached 2-5% [1][7]. Group 1: AI Investment Sustainability - Recent capital expenditures in the AI sector have raised concerns about sustainability; however, Goldman Sachs' latest report indicates that the current scale of AI investment is not overheated and remains sustainable [3]. - Since mid-2023, AI infrastructure investments have accelerated, with U.S. companies projected to generate an additional $300 billion in AI-related infrastructure revenue by 2025 [5]. - AI-related spending has seen an annualized growth of $277 billion compared to 2022 [5]. Group 2: Productivity and Computational Demand - The report highlights two main reasons supporting continued AI capital expenditure: significant productivity gains and increasing computational demand [6]. - Goldman Sachs estimates that the widespread application of generative AI will enhance U.S. labor productivity by 15% over the next decade, with AI applications potentially delivering a 25-30% average productivity increase [6]. - The demand for computational power is growing at an annual rate of 400%, outpacing the cost decline of computational resources at 40% per year, indicating sustained investment motivation in AI infrastructure [6]. Group 3: Economic Impact of AI - Goldman Sachs projects that productivity improvements from generative AI could create a present value of $20 trillion for the U.S. economy, with $8 trillion flowing as capital gains to U.S. companies [7]. - Even under pessimistic or optimistic scenarios, the projected economic impact ranges from $5 trillion to $19 trillion, significantly exceeding current and future AI investment totals [7].
商业管理者如何用好AI技术?这场会议这么说
Guo Ji Jin Rong Bao· 2025-10-19 22:17
"未来企业的竞争本质是'AI 赋能能力'与'人类独特价值'的协同进化能力竞争。管理者需跳出 "AI替 代人类" 的二元思维,建立'AI 作为认知延伸'的新范式,在数据洪流中坚守人类独有的价值判断、伦理 感知与创新直觉,实现技术赋能与人文理性的辩证统一。"上海国家会计学院刘凤委教授分享道。 他进一步表示,互联网与生成式 AI 的本质差异在于前者聚焦信息传播,而后者核心是知识创造, 这一特质将对公司组织产生影响,其重塑了商业决策逻辑,深度改变了组织协同与运行控制方式,对企 业经营与管理甚至带来革命性变革。伴随技术革新的风险与挑战也不容忽视:一是对企业知识型员工的 职业冲击;二是AI训练数据质量不足可能导致知识创造与运用出现偏差甚至 "幻觉";三是 AI 在缺乏伦 理情境下仅能进行事实判断,难以处理复杂价值判断进而影响决策最优性;四是责任认定层面存在模糊 地带。 刘凤委认为可采取的改进路径包括:让AI专注于任务边界明确的知识型工作以降低出错概率,提 升AI训练数据质量,发挥AI在事实判断领域的优势而由人类主导复杂逻辑下的价值决策。同时,推动 管理者能力重构与组织能力升级 —— 管理者需平衡 "算法思维" 与 "人类 ...
腾讯研究院AI速递 20251020
腾讯研究院· 2025-10-19 16:01
Group 1: Nvidia and TSMC Collaboration - Nvidia and TSMC unveiled the first Blackwell chip wafer produced in the U.S., marking a significant milestone in domestic chip manufacturing [1] - The TSMC Arizona factory has a total investment of $165 billion and will produce advanced chips using 2nm, 3nm, and 4nm processes [1] - The Blackwell chip features 208 billion transistors and achieves a connection speed of 10TB/s between its two sub-chips through NV-HBI [1] Group 2: Anthropic's Agent Skills - Anthropic launched the Agent Skills feature, allowing users to load prompts and code packages as needed, enhancing the capabilities of AI [2] - Skills can be used across Claude apps, Claude Code, and API platforms, with a focus on minimal necessary information loading [2] - The official presets include nine skills for various document formats, and users can upload custom skills [2] Group 3: New 3D World Model by Fei-Fei Li - Fei-Fei Li's World Labs introduced a real-time generative world model, RTFM, which can render persistent 3D worlds using a single H100 GPU [3] - RTFM employs a self-regressive diffusion Transformer architecture to learn from large-scale video data without explicit 3D representations [3] - The model maintains spatial memory for persistent world geometry through pose-aware frames and context scheduling technology [3] Group 4: Manus 1.5 Update - Manus released version 1.5, introducing a built-in browser that allows AI to interact with web pages, test functions, and fix bugs [4] - A new Library file management system enables collaborative editing within the same Agent session, reducing average task completion time significantly [4] - The system allows for no-code music web application construction through natural language, supporting real-time updates [4] Group 5: Windows 11 Major Update - Windows 11's major update features "Hey Copilot" for voice activation and Copilot Vision for screen understanding, enhancing user interaction [5][6] - Copilot Actions can perform operations on local files, while Copilot Connectors integrate with OneDrive, Outlook, and Google services [5][6] - Manus AI operations are integrated into the file explorer, allowing for automatic website generation and video editing functionalities [6] Group 6: Baidu's PaddleOCR-VL Model - Baidu open-sourced the PaddleOCR-VL model, achieving a score of 92.6 on the OmniDocBench V1.5 leaderboard with only 0.9 billion parameters [7] - The model supports 109 languages and excels in text recognition, formula recognition, table understanding, and reading order prediction [7] - It utilizes a two-stage architecture combining dynamic resolution visual encoding and a language model, achieving high inference speed on A100 [7] Group 7: AI in Fusion Energy Development - Google DeepMind collaborates with CFS to accelerate the development of the SPARC fusion device using AI [8] - The partnership focuses on creating precise plasma simulation systems and optimizing fusion energy output [8] - The TORAX simulator is a key tool for CFS, enabling extensive virtual experiments and real-time control strategy exploration [8] Group 8: Harvard Study on AI's Impact on Employment - A Harvard study tracking 62 million workers found a significant decline in entry-level positions in companies using AI, primarily through slowed hiring [9] - The impact of AI is most pronounced among graduates from mid-tier universities, while top-tier and bottom-tier institutions are less affected [9] - The wholesale and retail sectors face the highest risk for entry-level jobs, with a trend towards skill polarization [9] Group 9: Concerns Over AI-Generated Content - Reddit co-founder Ohanian warned that much of the internet is "dead," overwhelmed by AI-generated content [10] - Reports indicate that automated traffic could reach 51% by 2024, with AI-generated articles surpassing human-written ones [10] - Research suggests that training models on AI-generated data may lead to a decline in model performance [10] Group 10: Andrej Karpathy on AGI Development - AI expert Andrej Karpathy expressed skepticism about the current state of AI agents, predicting that AGI is still a decade away [11] - He criticized the noise in reinforcement learning and the limitations of pre-training methods [11] - Karpathy anticipates that AGI will contribute modestly to GDP growth, emphasizing the importance of education in the AI era [11]
人工智能到底是不是泡沫?回答业内最大问题的一个实用框架
3 6 Ke· 2025-10-19 10:16
Core Viewpoint - The article argues that the current state of artificial intelligence (AI) is not a bubble, but there are potential danger signals that need to be monitored through a framework of five indicators [1][2][6]. Group 1: Definition and Historical Context of Bubbles - Bubbles are not just financial phenomena but also cultural products, often associated with greed and folly [7]. - Historical examples of bubbles include the South Sea Bubble, the Roaring Twenties stock market, and the 2008 housing market crash, each characterized by overvaluation and subsequent collapse [9][10]. - The article defines a bubble as a situation where stock values drop by 50% from their peak and remain low for at least five years [10][13]. Group 2: Current AI Investment Landscape - Since the launch of ChatGPT, capital expenditures by large-scale cloud service providers have more than doubled, raising questions about the sustainability of such spending [14][16]. - Morgan Stanley predicts that AI infrastructure spending will reach $3 trillion by 2029, indicating significant investment momentum [17]. Group 3: Five Indicators Framework - The five indicators to assess the AI landscape are: 1. Economic Pressure: Evaluates whether current investment levels are distorting the economy [18]. 2. Industry Pressure: Assesses if industry revenues align with capital expenditures [30]. 3. Revenue Growth: Measures the speed of revenue growth relative to investment [35]. 4. Valuation Heat: Analyzes how high valuations are compared to historical standards [39]. 5. Quality of Capital: Examines the source and structure of funding supporting the industry [46]. Group 4: Economic Pressure - Current AI investment is at approximately 0.9% of U.S. GDP, projected to rise to 1.6% by 2030, indicating it is currently in the green zone but may soon enter the yellow zone [23][27]. Group 5: Industry Pressure - The capital expenditure to revenue ratio for generative AI is currently six times, indicating significant pressure, but this is not yet a warning sign as demand for AI services remains high [33]. Group 6: Revenue Growth - Generative AI revenue is expected to grow significantly, with estimates suggesting it could reach $1 trillion by 2028, indicating strong growth potential [38]. Group 7: Valuation Heat - Current market valuations are not as extreme as during the internet bubble, with the Nasdaq's P/E ratio around 32, which is lower than the peak of 72 during the internet boom [42][44]. Group 8: Quality of Capital - The quality of capital in the AI sector appears stable, with major companies generating substantial cash flow to support investments, although there are concerns about future funding gaps [49][51]. Group 9: Conclusion - The analysis suggests that generative AI is in a demand-driven, capital-intensive growth phase rather than a bubble, but vigilance is required as certain indicators may signal a shift towards instability in the future [52][54].
AI资本开支太狂热了?高盛:这才到哪呢
Hua Er Jie Jian Wen· 2025-10-19 08:12
Core Insights - The current scale of AI investment is sustainable and not overheated, indicating a robust macro story for AI infrastructure development [1][4] - AI-related investments account for less than 1% of the US GDP, significantly lower than historical peaks in other technology cycles [4] - The productivity gains from AI are projected to generate $8 trillion in capital income for US companies, far exceeding current and foreseeable AI investment totals [1][4] Investment Trends - Since mid-2023, there has been a significant acceleration in AI infrastructure investment, with an estimated $300 billion in revenue growth for US companies in AI-related infrastructure by 2025 [2] - AI-related spending has seen an annualized growth of $277 billion compared to 2022 [2] - Major investment agreements have been announced by OpenAI, including a $300 billion partnership with Oracle and a $100 billion investment from Nvidia [2] Supporting Factors for AI Capital Expenditure - Productivity improvements are expected to be substantial, with a projected 15% increase in US labor productivity due to the widespread application of generative AI over the next decade [3] - The demand for computing power is increasing rapidly, with AI model sizes growing at an annual rate of 400%, outpacing the 40% annual decline in computing costs [3] - The growth rates for training queries and cutting-edge models are 350% and 125% respectively, indicating sustained demand for AI infrastructure investment [3] Historical Context of AI Investment - Although nominal AI infrastructure investment has reached new highs, it remains modest compared to historical technology cycles, where peaks accounted for 2-5% of GDP [4] - The estimated present value of productivity gains from generative AI is $20 trillion, with $8 trillion expected to flow as capital gains to US companies [4] - Even under pessimistic or optimistic scenarios, the projected economic benefits from AI significantly exceed current and future investment totals [4]
英伟达(NVDA.US)的又一场“阳谋”
智通财经网· 2025-10-19 05:49
Core Insights - The performance advancements in data centers over the past two decades have primarily relied on the evolution of computing chips, but the advent of generative AI has redefined the entire computing power framework, emphasizing the importance of network efficiency in large model training [1][10] - NVIDIA's Spectrum-X Ethernet switch and related technologies have been adopted by major tech giants Meta and Oracle, marking a significant step towards AI-optimized Ethernet solutions [1][9] Group 1: Spectrum-X Features - Spectrum-X is designed to address the unique challenges of AI workloads, focusing on ensuring performance under extreme conditions rather than average performance [2] - Key improvements of Spectrum-X include: - Lossless Ethernet capabilities achieved through RoCE technology, PFC, and DDP, ensuring end-to-end lossless transmission [2][5] - Adaptive routing and packet scheduling to manage large "elephant flows" and prevent network congestion [5][7] - Advanced congestion control with in-band telemetry for real-time network status reporting, achieving 95% data throughput compared to 60% for traditional Ethernet [7][8] - Performance isolation and security features, including shared buffer architecture and encryption mechanisms, providing a level of security akin to private clusters [8][9] Group 2: Industry Impact - The introduction of Spectrum-X represents a strategic shift in the Ethernet networking industry, effectively integrating multiple components into a cohesive ecosystem that challenges traditional network vendors [11][12] - Companies like Broadcom and Marvell, which have historically dominated the high-end Ethernet chip market, may face challenges as Spectrum-X's capabilities threaten their value proposition [13] - Traditional network equipment suppliers such as Cisco and Arista Networks may also be impacted, as NVIDIA's integrated approach reduces reliance on their optimization solutions in AI-centric environments [14][15] Group 3: Competitive Landscape - The launch of Spectrum-X could significantly alter the competitive dynamics within the Ethernet networking sector, compelling companies to either integrate into NVIDIA's AI network framework or risk marginalization [12][13] - Startups focused on interconnect solutions may find their market space constrained as large cloud providers adopt Spectrum-X architecture, which centralizes control and reduces compatibility with independent solutions [16][17] - NVIDIA's Quantum InfiniBand remains the leading high-performance network standard, emphasizing the contrast between its closed ecosystem and the open standards being pursued by the Ultra Ethernet Consortium [19][21]