作者 | Abel chen 编辑 | 自动驾驶之心 原文链接： https://zhuanlan.zhihu.com/p/1948681769332240910 点击下方 卡片 ，关注" 自动驾驶之心 "公众号 戳我-> 领取 自动驾驶近30个 方向 学习 路线 本文只做学术分享，如有侵权，联系删文 1. GRPO是on policy还是off policy？为什么？ 简短答案： GRPO 最初设计和常用实现是 on-policy（在线/近端策略式） ；但它可以被扩展为 off-policy，已有工作专门研究这种扩展及其优缺点。 为什么是 on-policy（解释） 为什么有人说可以 off-policy（扩展） 最近有工作把 GRPO 的思想推广到 off-policy 场景（比如用来自别的策略 / 旧批次的数据来估计优势并做修正），并且报告了在样本效率、稳定性等方面的潜在好 处与权衡。也就是说，虽然 GRPO 本质上是基于 on-policy 的 surrogate objective，但数学上和工程上可以设计重要性采样、批内归一化或裁剪等技巧把它改成 off- policy 版本。 实践建议（简要） ...

Core Viewpoint - The article discusses the evolution and differentiation between AI Agents and Agentic AI, highlighting their respective roles in automating tasks and collaborating on complex objectives, with a focus on the advancements since the introduction of ChatGPT in November 2022 [2][10][57]. Group 1: Evolution of AI Technology - The development of AI technology has progressed from early expert systems like MYCIN to modern AI Agents and Agentic AI, marking a significant paradigm shift in capabilities [10][11]. - ChatGPT's release in November 2022 is identified as a pivotal moment that catalyzed the evolution of AI Agents, transitioning from passive responders to more autonomous systems capable of executing multi-step tasks [12][24]. - The introduction of frameworks like AutoGPT and BabyAGI in 2023 signifies the formal establishment of AI Agents, which integrate LLMs with external tools to perform complex tasks [12][24]. Group 2: Characteristics of AI Agents - AI Agents are defined as modular systems driven by LLMs and LIMs, designed for task automation, filling the gap where generative AI lacks execution capabilities [13][16]. - Three core features distinguish AI Agents from traditional automation scripts: autonomy, task-specificity, and reactivity [16][17]. - The integration of tools allows AI Agents to overcome limitations of static knowledge and hallucination issues, enabling them to perform real-time data retrieval and processing [19][20]. Group 3: Agentic AI and Multi-Agent Collaboration - Agentic AI represents a shift towards multi-agent collaboration, where multiple AI Agents work together to achieve complex goals, enhancing system-level intelligence [24][27]. - The architecture of Agentic AI includes dynamic task decomposition and shared memory, facilitating efficient collaboration among specialized agents [33][36]. - Real-world applications of Agentic AI demonstrate its advantages in various fields, such as healthcare and agriculture, where multiple agents coordinate to optimize processes [37][38]. Group 4: Challenges and Future Directions - Both AI Agents and Agentic AI face challenges, including causal reasoning deficits and coordination issues among multiple agents [48][50]. - Proposed solutions include enhancing retrieval-augmented generation (RAG), implementing causal modeling, and establishing shared memory architectures to improve collaboration and decision-making [49][53]. - The future roadmap emphasizes the need for deeper causal reasoning, transparency in decision-making, and ethical governance to ensure the responsible deployment of AI technologies [56][59].

Core Viewpoint - NIO has successfully raised over 10 billion USD in funding, indicating strong support from long-term investors despite ongoing losses, which positions the company strategically for future growth and profitability [5][6][7][10]. Financing Details - NIO announced the issuance of 181.8 million Class A ordinary shares, including American Depositary Shares (ADS), with underwriting by top international investment banks [11]. - The financing was completed on the same day as the announcement, raising over 1 billion USD, approximately 71.2 billion RMB [12]. - The public offering price for ADS was set at 5.57 USD (about 39.7 RMB) per share, while the ordinary shares were priced at 43.36 HKD [14]. Use of Funds - The raised funds will primarily be allocated to core technology development for smart electric vehicles, including advanced driver assistance systems, smart cockpit, and next-generation electric drive systems [17]. - Additional funds will be used to develop new technology platforms and models, expand the charging and battery swap network, and optimize the financial situation to strengthen the balance sheet for long-term strategic investments [18][20]. Financial Health - As of mid-2023, NIO's total assets were 100.046 billion RMB, with total liabilities of 93.43 billion RMB, resulting in a debt-to-asset ratio of 93.4%, significantly higher than the industry average of 60%-80% [23][24]. - NIO's current liabilities amounted to 62.282 billion RMB, exceeding current assets of 52.508 billion RMB, indicating a declining short-term solvency [25][27]. - The cash reserves stood at 27.2 billion RMB, a slight increase from the previous quarter but still insufficient to cover payables [26]. Operational Performance - In Q2, NIO delivered 72,056 vehicles, marking a year-on-year increase of 25.6% and a quarter-on-quarter increase of 71.2%, achieving a historical high for the same period [36]. - The gross margin improved to 10.0% in Q2, up from 7.6% in Q1, although the net loss was still significant at 4.995 billion RMB [37]. - NIO aims to achieve profitability by Q4 2023, contingent on sustained sales growth and improved gross margins [32][34]. Investor Sentiment - NIO's ability to secure funding despite continuous losses reflects investor confidence in its long-term vision and potential in the electric vehicle market [48][49]. - The company has conducted at least 18 financing rounds since its inception, raising nearly 100 billion RMB in total, showcasing its strong fundraising capabilities compared to peers [43][45].

非常开心这周有机会和星球的小伙伴线上探讨一下，峰哥和柱哥通过这次交流也学习到很多。和星友们交流了很 多，有涉及VLA、端到端方向选择，也有小伙伴在数据闭环这个方向工作的，也有关于具身还是智驾的选择问 题。 通过这次交流，我们更实际了解到大家存在的困惑，更需要什么方向的帮助，未来也会根据大家的反馈进一步优 化咱们的星球，后续的星友面对面环节计划邀请一些学术界和工业界某一技术方向的大佬和大家交流，也欢迎更 多的小伙伴参与进来！ 我们把第一次交流的部分问题分享给大家，如果您也对这些问题感兴趣，欢迎加入『自动驾驶之心知识星球』， 和星主面对面！ 『自动驾驶之心知识星球』目前集视频 + 图文 + 学习路线 + 问答 + 求职交流为一体，是一个综合类的自驾社 区，已经超过4000人了。 我们期望未来2年内做到近万人的规模。给大家打造一个交流+技术分享的聚集地，是 许多初学者和进阶的同学经常逛的地方。 社区内部还经常为大家解答各类实用问题：端到端如何入门？自动驾驶多模态大模型如何学习？自动驾驶VLA 的学习路线。数据闭环4D标注的工程实践。快速解答，方便大家应用到项目中。 更有料的是：星球内部为大家梳理了近40+技术路线， ...

Core Viewpoint - The article emphasizes the importance of creating an engaging learning environment in the field of autonomous driving, aiming to bridge the gap between industry and academia while providing resources for students and professionals [1][15]. Group 1: Community and Resources - The "Autonomous Driving Heart Knowledge Planet" has created a comprehensive community that integrates video, text, learning paths, Q&A, and job exchange, currently hosting over 4,000 members with a goal to reach nearly 10,000 in the next two years [4][15]. - The community offers nearly 40 technical routes, catering to various needs such as consulting industry applications and the latest VLA benchmarks, significantly reducing search time for users [4][16]. - Members can access a wealth of resources, including academic content, industry roundtables, open-source code solutions, and timely job information [1][4]. Group 2: Learning and Development - The community provides structured learning paths for beginners and advanced learners, covering topics like end-to-end autonomous driving, multi-modal large models, and practical engineering practices [4][16]. - Regular discussions with industry and academic leaders are held to explore trends in autonomous driving technology and production challenges [5][15]. - The platform includes a variety of learning materials, including video tutorials on topics such as sensor calibration, SLAM, and decision-making algorithms [8][16]. Group 3: Job Opportunities and Networking - The community facilitates job opportunities by connecting members with positions in leading autonomous driving companies, offering resume forwarding services [9][23]. - Members can engage in discussions about career choices and research directions, receiving guidance from experienced professionals in the field [88][91]. - The platform aims to foster networking among peers and industry leaders, enhancing collaboration and knowledge sharing [23][96].

Core Insights - The article discusses the transition from static large language models (LLMs) to self-evolving agents capable of continuous learning and adaptation in dynamic environments, paving the way towards artificial superintelligence (ASI) [3][4][46] - It emphasizes the need for a structured framework to understand and design self-evolving agents, focusing on three fundamental questions: what to evolve, when to evolve, and how to evolve [6][46] Group 1: What to Evolve - Self-evolving agents can improve various components such as models, memory, tools, and architecture over time to enhance performance and adaptability [19][20] - The evolution of these components is crucial for the agent's ability to handle complex tasks and environments effectively [19][20] Group 2: When to Evolve - The article categorizes self-evolution into two time modes: intra-test-time self-evolution, which occurs during task execution, and inter-test-time self-evolution, which happens between tasks [22][23] - Intra-test-time self-evolution allows agents to adapt in real-time to specific challenges, while inter-test-time self-evolution leverages accumulated experiences for future performance improvements [22][23] Group 3: How to Evolve - Self-evolution emphasizes a continuous learning process where agents learn from real-world interactions, seek feedback, and adjust strategies dynamically [26][27] - Various methodologies for self-evolution include reward-based evolution, imitation learning, and population-based approaches, each with distinct feedback types and data sources [29][30] Group 4: Applications and Evaluation - Self-evolving agents have significant potential in various fields, including programming, education, and healthcare, where continuous adaptation is essential [6][34] - Evaluating self-evolving agents presents unique challenges, requiring metrics that capture adaptability, knowledge retention, and long-term generalization capabilities [34][36] Group 5: Future Directions - The article highlights the importance of addressing challenges such as catastrophic forgetting, knowledge transfer, and ensuring safety and controllability in self-evolving agents [40][43] - Future research should focus on developing scalable architectures, dynamic evaluation methods, and personalized agents that can adapt to individual user preferences [38][44]

自动驾驶之心世界模型技术交流群成立了，欢迎大家加入一起世界模型相关的内容。感兴趣的同学欢迎添 加小助理微信进群：AIDriver005， 备注：昵称+世界模型加群。 ...

Core Viewpoint - The article discusses the significance of world modeling in the field of artificial intelligence and robotics, emphasizing the need for a structured approach to 3D and 4D world modeling to enhance autonomous driving and robotics applications [5][7][13]. Group 1: Introduction to World Modeling - World modeling is a foundational task in AI and robotics, aimed at enabling agents to understand, represent, and predict their dynamic environments [5][7]. - Recent advancements in generative modeling techniques have primarily focused on 2D data, while the real-world scenarios are inherently 3D and dynamic, necessitating the use of native 3D and 4D representations [5][6][9]. Group 2: Importance of Native 3D and 4D Representations - Native 3D and 4D signals encode metric geometry, visibility, and motion information, making them essential for actionable modeling in safety-critical scenarios [9][10]. - These representations provide the necessary constraints for generating visually realistic frames while adhering to geometric laws and causal relationships [9][10]. Group 3: Research Contributions - The review provides precise definitions of "world models" and "3D/4D world modeling," offering clarity and a unified terminology for the research community [13][14]. - A hierarchical classification system is proposed, categorizing existing methods based on representation modalities such as VideoGen, OccGen, and LiDARGen [13][14]. - The review encompasses datasets and evaluation protocols specifically suited for 3D/4D scenarios, supporting comprehensive benchmarking for current and future world modeling methods [13][14]. Group 4: Methodology and Classification - The article outlines a structured classification of world modeling methods based on representation modalities, detailing the advantages and limitations of each approach [16][42]. - It distinguishes between generative and predictive world models, highlighting their dual capabilities to imagine diverse and controllable worlds and predict reasonable future evolutions under specific conditions [27][28]. Group 5: Applications and Future Directions - The review discusses practical applications of 3D/4D world models in autonomous driving, robotics, and simulation environments, emphasizing their growing importance in both academia and industry [16][18][55]. - It identifies key challenges and potential future research directions, aiming to pave the way for continuous innovation in the field [16][18].

Core Viewpoint - The article discusses the significance and application of Diffusion Models in various fields, particularly in autonomous driving, emphasizing their ability to denoise and generate data effectively [1][2][11]. Summary by Sections Introduction to Diffusion Models - Diffusion Models are generative models that focus on denoising, learning the distribution of data through a forward diffusion process and a reverse generation process [2][4]. - The concept is illustrated through the analogy of ink dispersing in water, where the model aims to recover the original data from noise [2]. Applications in Autonomous Driving - In the field of autonomous driving, Diffusion Models are utilized for data generation, scene prediction, perception enhancement, and path planning [11]. - They can handle both continuous and discrete noise, making them versatile for various decision-making tasks [11]. Course Offering - The article promotes a new course on end-to-end and VLA (Vision-Language Alignment) algorithms in autonomous driving, developed in collaboration with top industry experts [14][17]. - The course aims to address the challenges faced by learners in keeping up with rapid technological advancements and fragmented knowledge in the field [15][18]. Course Structure - The course is structured into several chapters, covering topics such as the history of end-to-end algorithms, background knowledge on VLA, and detailed discussions on various methodologies including one-stage and two-stage end-to-end approaches [22][23][24]. - Special emphasis is placed on the integration of Diffusion Models in multi-modal trajectory prediction, highlighting their growing importance in the industry [28]. Learning Outcomes - Participants are expected to achieve a level of understanding equivalent to one year of experience as an end-to-end autonomous driving algorithm engineer, mastering key frameworks and technologies [38][39]. - The course includes practical components to ensure a comprehensive learning experience, bridging theory and application [19][36].

Group 1 - The article introduces a significant learning package for the new academic season, including a 299 yuan discount card that offers a 30% discount on all platform courses for one year [3][5]. - Various course benefits are highlighted, such as a 1000 yuan purchase giving access to two selected courses, and discounts on specific classes and hardware [3][6]. - The focus is on cutting-edge autonomous driving technologies for 2025, particularly end-to-end (E2E) and VLA (Vision-Language Alignment) autonomous driving systems [5][6]. Group 2 - End-to-end autonomous driving is emphasized as a core algorithm for mass production, with a notable mention of the competition sparked by the UniAD paper winning the CVPR Best Paper award [6][7]. - The article discusses the rapid evolution of technology in the field, indicating that previous learning materials may no longer be suitable for current industry standards [7]. - The challenges faced by beginners in understanding fragmented knowledge and the lack of high-quality documentation in end-to-end autonomous driving research are addressed [7][8]. Group 3 - The article outlines specific courses aimed at addressing the complexities of autonomous driving, including a small class on 4D annotation algorithms, which are crucial for training data generation [11][12]. - The importance of automated 4D annotation in enhancing the efficiency of data loops and improving the generalization and safety of autonomous driving systems is highlighted [11]. - The introduction of a multi-modal large model and practical courses in autonomous driving is noted, reflecting the growing demand for skilled professionals in this area [15][16]. Group 4 - The article features expert instructors for the courses, including Jason, a leading algorithm expert in the industry, and Mark, a specialist in 4D annotation algorithms [8][12]. - The curriculum is designed to provide a comprehensive learning experience, addressing real-world challenges and preparing students for job opportunities in the autonomous driving sector [23][29]. - The article emphasizes the importance of community engagement and support through dedicated VIP groups for course participants, facilitating discussions and problem-solving [29].