作者 | 钱红中 编辑 | 自动驾驶之心 原文链接： https://zhuanlan.zhihu.com/p/12088531309 点击下方 卡片 ，关注" 自动驾驶之心 "公众号 戳我-> 领取 自动驾驶近30个 方向 学习 路线 >>自动驾驶前沿信息获取 → 自动驾驶之心知识星球 本文只做学术分享，如有侵权，联系删文 Q：端到端模型通常大致分为几种？ 分为两种，一种是完全黑盒OneNet，模型直接优化Planner；另一种是模块化端到端，即模块级联或者并联，通过感知模块，预测模块以及规划模块之间feat- level/query-level的交互，减少分段式自动驾驶模型的误差累积。 Q：[UniAD] 整个框架分为4部分，输入multi-view camera imgs，Backbone模块提取BEV feat，Perception模块完成对于scene-level的感知包括对于agents+ego以及map， Prediction模块基于时序交互以及agents-scene的交互完成对于agents+ego的multi-mode轨迹预测，Planner模块基于预测的轨迹以及BEV feat完成路径的 ...

Core Viewpoint - The article discusses the intersection of simulation technology, real data, and world models in the context of embodied intelligence, highlighting the ongoing debate about the importance of simulation versus real data and the potential breakthroughs in world modeling [3][11]. Group 1: Roundtable Discussion - The roundtable focuses on the "data dilemma" in embodied intelligence, featuring four young scientists who explore the boundaries between simulation and real interaction, as well as the technological advancements in world models like Genie [3][11]. - Sergey Levine's assertion that real data is irreplaceable is examined, questioning whether this is a strategic choice or an inevitable path in AI evolution [11]. Group 2: Key Participants - Li Hongyang, an assistant professor at the University of Hong Kong, leads the OpenDriveLab and has made significant contributions to end-to-end autonomous driving solutions, including the award-winning UniAD [4]. - Zhao Hao, an assistant professor at Tsinghua University, specializes in computer vision related to robotics and has co-founded over ten startups since 2009 [5]. - Gu Jiayuan, an assistant professor at ShanghaiTech University, focuses on generalizable robotic decision-making models and has received multiple awards for his research [6][7]. - Mu Yao, an assistant professor at Shanghai Jiao Tong University, has published extensively in top conferences and has received numerous academic honors [7].

Core Viewpoint - The article discusses the advancements and potential of Horizon Robotics' HSD (Highway Smart Driving) system, emphasizing its transition from testing to near mass production, and its implications for the smart automotive industry in China [1][4][36]. Group 1: Technological Advancements - Horizon Robotics has launched its self-developed computing hardware, the J6P, and the UniAD software, marking significant milestones in autonomous driving technology [2][4]. - The HSD system features a one-stage end-to-end architecture, which improves the driving experience by integrating various functionalities without relying on traditional rule-based systems [19][36]. - Key innovations include dense modal information processing, joint optimization of lateral and longitudinal control, and a robust safety verification module to ensure safe vehicle operation [20][21][24]. Group 2: Performance and User Experience - The latest version of HSD demonstrates improved speed control and smoother driving experiences, particularly in complex scenarios like traffic lights and merging [6][10]. - The system's ability to navigate through obstacles and make real-time decisions showcases its advanced cognitive capabilities, allowing it to handle various driving conditions effectively [11][36]. - Despite its advancements, some issues were noted, such as misidentifying stationary vehicles, indicating areas for further refinement [13][36]. Group 3: Industry Implications - Horizon Robotics aims to establish a comprehensive solution for the automotive industry, moving away from isolated implementations for specific manufacturers [41]. - The company's strategy focuses on a gradual approach to autonomous driving, with a clear timeline for achieving hands-off and eyes-off driving capabilities [37][39]. - The competitive landscape suggests that companies failing to keep pace with these advancements may risk obsolescence in the rapidly evolving smart automotive sector [37][39].

Core Viewpoint - The article discusses the current development status of end-to-end autonomous driving algorithms, comparing them with traditional algorithms and highlighting their advantages and limitations [1][3][53]. Summary by Sections Traditional vs. End-to-End Algorithms - Traditional autonomous driving algorithms follow a pipeline of perception, prediction, and planning, where each module has distinct inputs and outputs [3]. - End-to-end algorithms take raw sensor data as input and directly output path points, simplifying the process and reducing error accumulation [3][5]. - Traditional algorithms are easier to debug and have some level of interpretability, but they suffer from cumulative error issues due to the inability to ensure complete accuracy in perception and prediction modules [3][5]. Limitations of End-to-End Algorithms - End-to-end algorithms face challenges such as limited ability to handle corner cases, as they rely heavily on data-driven methods [7][8]. - The use of imitation learning in these algorithms can lead to difficulties in learning optimal ground truth and handling exceptional cases [53]. - Current end-to-end paradigms include imitation learning (behavior cloning and inverse reinforcement learning) and reinforcement learning, with evaluation methods categorized into open-loop and closed-loop [8]. Current Implementations - The ST-P3 algorithm is highlighted as an early work focusing on end-to-end autonomous driving, utilizing a framework that includes perception, prediction, and planning modules [10][11]. - Innovations in the ST-P3 algorithm include a perception module that uses a self-centered cumulative alignment technique and a prediction module that employs a dual-path prediction mechanism [11][13]. - The planning phase of ST-P3 optimizes predicted trajectories by incorporating traffic light information [14][15]. Advanced Techniques - The UniAD system employs a full Transformer framework for end-to-end autonomous driving, integrating multiple tasks to enhance performance [23][25]. - The TrackFormer framework focuses on the collaborative updating of track queries and detect queries to improve prediction accuracy [26]. - The VAD (Vectorized Autonomous Driving) method introduces vectorized representations for better structural information and faster computation in trajectory planning [32][33]. Future Directions - The article suggests that end-to-end algorithms still primarily rely on imitation learning frameworks, which have inherent limitations that need further exploration [53]. - The introduction of more constraints and multi-modal planning methods aims to address trajectory prediction instability and improve model performance [49][52].

SenseTime Group Inc. 2024 Interim Results Presentation TO CREATE A BETTER AI-EMPOWERED FUTURE THROUGH INNOVATION 2024.08.27 —— Disclaimer The information in this presentation has been prepared by representatives of SenseTime Group Inc. (the "Company", and together with its subsidiaries, the "Group") for use in presentations by the Group for information purposes. No part of this presentation should form the basis of, or be relied on in connection with, any contract or commitment or investment decision whatso ...

Financial Data and Key Metrics Changes - Group revenue for the first half of 2024 reached RMB 1,740 million, representing a 21.4% increase year-on-year [6] - Generative AI revenue surged to RMB 1,050 million, accounting for 60% of total group revenue, up from 21% last year [12] - EBITDA loss reduced by 26.5% and overall loss decreased by 21.2% in the first half of 2024 [8][42] - Gross profit margin remained at 44%, consistent with the previous year [42] Business Line Data and Key Metrics Changes - Generative AI revenue increased by 256% year-on-year, becoming the primary driver of revenue growth [39] - Sensors revenue doubled to RMB 1,168 million, accounting for 10% of group revenue [12] - Traditional AI revenue was RMB 520 million, contributing 30% of group revenue, indicating a decline [40] Market Data and Key Metrics Changes - Overseas market revenue grew by 40% year-on-year, now accounting for 18% of total revenue [13][41] - The Chinese intelligent computing services market is projected to grow at a CAGR of over 50% for the next five years, reaching nearly RMB 200 billion by 2028 [21] Company Strategy and Development Direction - The company is focused on generative AI, leveraging deep synergies between large models and infrastructure to enhance model capabilities and reduce costs [7][39] - The strategic pivot towards generative AI has been more successful than anticipated, with significant growth in various sectors including intelligent hardware, electric vehicles, and finance [10][12] - The company aims to expand its operational computing power to 25,000 petabytes by the end of the year [19] Management Comments on Operating Environment and Future Outlook - Management expressed optimism about the generative AI market, highlighting its rapid growth and the need for companies to invest in large models [9][76] - The competitive landscape is described as fierce, with significant investments required to maintain competitiveness [8][39] - Management emphasized the importance of balancing long-term growth with short-term investments [8] Other Important Information - The company has deployed over 50,000 GPUs, with total computing power exceeding 20,000 petabytes, positioning it as a key player in the AI infrastructure market [18] - The SESNOVA large model series has shown significant improvements, with version 5.5 released in July 2024, enhancing capabilities and real-time interaction [24][25] Q&A Session Summary Question: What are the potential applications for edge AI in collaboration with smartphone manufacturers? - Management is optimistic about edge AI prospects, emphasizing the growth of the user base and the potential for new application models beyond smartphones, including IoT devices [52][54] Question: How is the company planning to scale computing power resources? - The company is focusing on improving operational efficiency while expanding computing power, adopting a strategic approach to maintain competitiveness [58][59] Question: What are the core capabilities of the next generation large model? - Management discussed the importance of reasoning and high-order data in enhancing model capabilities, emphasizing the need for better data and model architecture [63][66] Question: Which products or services predominantly contribute to the increase in generative AI revenue? - The company is focusing on the commercialization of its technological expertise in AI infrastructure and large models, which has led to significant growth in generative AI revenue [72][76] Question: What is the current progress in commercializing end-to-end algorithms in the autonomous driving sector? - The company is dedicated to a pure visual technology path for autonomous driving, leveraging its computational resources to support automakers in developing advanced driving technologies [81][84]