作者 | 糯盐@知乎 来源 | 知乎 原文链接： https://zhuanlan.zhihu.com/p/1890430442580194282 点击下方 卡片 ，关注" 自动驾驶之心 "公众号 戳我-> 领取 自动驾驶近30个 方向 学习 路线 >>自动驾驶前沿信息获取 → 自动驾驶之心知识星球 本文只做学术分享，文章已获转载授权。 终于还是没有忍住，非要来掰扯Lidar这个找死话题 在FSD没入华之前，这个争论不会终结，现在FSD在国内初试牛刀，有了些客观事实，天平开始倒向纯视觉。 Lidar技术可以回溯到1960年代，当时主要用于军事航空测绘，例如绘制地图或侦测军事目标。Lidar用于自动驾驶是2004开始的DARPA(美国国防部高级研究 计划) Challenges项目，这个项目主要推动自动驾驶技术的发展，举办了多次公开赛事。2005年第二届比赛中，斯坦福胜出的"Stanley"用了5个工业上应用的 Lidar。2007年的城市赛中，卡耐基梅陇的"Boss"使用了64线的旋转式Velodyne HDL-64E激光雷达，它成了这次比赛的技术标杆，一战成名。 2009年Google启动了它的自动驾驶研究 ...

Core Viewpoint - The company has evolved from a small workshop into a platform with significant technical depth and breadth, indicating a growing demand in the industry for embodied intelligence and related technologies [1]. Group 1: Team and Operations - The team has spent over two years developing four key IPs: Embodied Intelligence, Autonomous Driving, 3D Vision, and Large Model Tech, with a total online following of nearly 360,000 across various platforms [1]. - The company is currently hiring for full-time and part-time positions in operations and sales to support its expanding business lines [2]. Group 2: Job Responsibilities and Requirements - The operations role includes managing course progress, enhancing platform engagement, and developing content related to the AI industry [4]. - The sales role involves creating promotional materials for online and hardware products and liaising with hardware manufacturers and academic/enterprise clients [5][6]. - Candidates for both roles are expected to have strong execution, communication skills, and a background in computer science, AI, or robotics, with familiarity in social media operations being a plus [12]. Group 3: Growth Opportunities - The company offers exposure to top-tier operational teams, providing opportunities to learn operational techniques and sales strategies, leading to rapid personal growth [7]. - Employees will engage with cutting-edge content in fields like autonomous driving and embodied intelligence, broadening their industry perspective [8]. - There are opportunities for further academic pursuits, such as research and doctoral studies, which can enhance personal development [9].

Core Insights - The article discusses the author's experience in job searching and interviews, highlighting the challenges and changes in the job market, particularly in the computer vision (CV) and deep learning sectors [4][6][8]. Job Search Experience - The author experienced a high volume of interviews, averaging six per day over a month, with some days reaching eight interviews, indicating a competitive job market [4][5]. - The author transitioned from a role in a delivery company focused on CV to seeking opportunities in more stable and specialized areas, reflecting a shift in personal career focus [6][8]. Market Trends - There has been a significant increase in job opportunities compared to previous years, with many large and mid-sized companies actively hiring [8]. - The demand for traditional CV roles has diminished, with a notable shift towards large models, multi-modal applications, and end-to-end models in the autonomous driving sector [8][10]. Interview Preparation - The author prepared for interviews by reviewing popular coding problems, particularly from LeetCode, indicating a trend where companies now require candidates to demonstrate coding skills more rigorously than in the past [9][10]. - The author noted that many interview questions were derived from the "Hot100" list of coding problems, emphasizing the importance of algorithmic knowledge in technical interviews [11]. Career Transition - After several interviews, the author received offers from companies like Kuaishou, Xiaomi, and Weibo, but faced challenges in securing positions at larger firms like Alibaba and Baidu [10]. - Ultimately, the author accepted a position at a foreign company, which was described as a significantly better work environment compared to previous domestic companies, highlighting the differences in corporate culture [10][12]. Technical Skills and Trends - The author observed a shift in technical skills required in the job market, with a growing emphasis on large models and multi-modal technologies, suggesting that professionals in the field need to adapt to these changes to remain competitive [13].

Group 1 - The article highlights a surge in IPO activities within the autonomous driving sector, indicating a significant shift in the industry landscape with new players entering the market [1][2] - Key events include the acquisition of Shenzhen Zhuoyu Technology by China First Automobile Works, Wayve's partnership with NVIDIA for a $500 million investment, and multiple companies filing for IPOs or completing strategic investments [1] - The article discusses the intense competition in the autonomous driving field, suggesting that many companies are pivoting towards embodied AI as a response to market saturation [1][2] Group 2 - The article emphasizes the importance of comprehensive skill sets for professionals remaining in the autonomous driving industry, as the market is expected to undergo significant restructuring [2] - It mentions the creation of a community platform, "Autonomous Driving Heart Knowledge Planet," aimed at providing resources and networking opportunities for individuals interested in the field [3][19] - The community offers a variety of learning resources, including video tutorials, technical discussions, and job placement assistance, catering to both beginners and experienced professionals [4][11][22] Group 3 - The community has gathered over 4,000 members and aims to expand to nearly 10,000 within two years, focusing on knowledge sharing and technical collaboration [3][19] - It provides structured learning paths and resources for various topics in autonomous driving, including end-to-end learning, multi-sensor fusion, and real-time applications [19][39] - The platform also facilitates discussions on industry trends, job opportunities, and technical challenges, fostering a collaborative environment for knowledge exchange [20][91]

Group 1 - The article announces the recruitment of 10 outstanding partners for the autonomous driving sector, focusing on course development, paper guidance, and hardware research [2] - The main areas of expertise sought include large models, multimodal models, diffusion models, end-to-end systems, embodied interaction, joint prediction, SLAM, 3D object detection, world models, closed-loop simulation, and model deployment and quantization [3] - Candidates are preferred from universities ranked within the QS200, holding a master's degree or higher, with priority given to those with significant conference contributions [4] Group 2 - The compensation package includes resource sharing for job seeking, doctoral studies, and overseas study recommendations, along with substantial cash incentives and opportunities for entrepreneurial project collaboration [5] - Interested parties are encouraged to add WeChat for consultation, specifying "organization/company + autonomous driving cooperation inquiry" [6]

Group 1 - The article highlights the importance of continuous learning in the field of autonomous driving, emphasizing the need for professionals to stay updated with the latest technologies and trends [6] - It mentions a variety of advanced topics and learning routes available, including VLA, world models, closed-loop simulation, and diffusion models, indicating a comprehensive curriculum for learners [6] - The platform offers opportunities for direct interaction with industry leaders and academic experts, facilitating knowledge exchange and networking [6] Group 2 - The article outlines various promotional offers for new users, including discounts on courses and membership renewals, aimed at attracting more participants to the learning community [4][3] - It lists seven premium courses available, covering essential topics such as trajectory prediction, camera calibration, and 3D point cloud detection, catering to both beginners and advanced learners [6] - The content emphasizes the significance of face-to-face discussions with top authors and experts in the field, enhancing the learning experience through direct engagement [6]

Core Insights - The article discusses the evolution of end-to-end autonomous driving technology, highlighting the shift from horizontal to vertical end-to-end systems as a new industry focus [2][3] - It emphasizes the importance of vertical end-to-end control for achieving human-like driving efficiency, particularly in speed and braking control [4][16] Group 1: Importance of Vertical End-to-End Control - Vertical end-to-end control is essential for achieving smooth acceleration and deceleration, which is a key differentiator between novice and experienced drivers [3][4] - The article defines "defensive deceleration" as the ability to adjust speed based on necessity and prediction, balancing safety and efficiency [4][12] - Current autonomous systems often prioritize navigation efficiency over vertical control, making it challenging to implement effective speed adjustments [15][16] Group 2: Challenges in Achieving Vertical End-to-End Control - Many autonomous driving systems have successfully implemented horizontal end-to-end control, but vertical control remains a significant challenge [13][16] - The noise in human driving data complicates the learning process for autonomous systems, making it difficult to distinguish meaningful speed control from random fluctuations [16][17] - Solutions to improve vertical control include data cleaning, causal reasoning, and reinforcement learning, which are being explored by leading autonomous driving teams [17]

Core Viewpoint - The article discusses the challenges of end-to-end autonomous driving, particularly focusing on the limitations of imitation learning and the introduction of DriveDPO, a safety-oriented policy learning framework that enhances driving safety and reliability [1][7][28]. Summary by Sections Imitation Learning Challenges - Imitation learning can lead to unsafe driving behaviors despite generating trajectories that appear human-like, as it does not account for the safety implications of certain maneuvers [5][11]. - The symmetric loss functions commonly used in imitation learning fail to differentiate between safe and unsafe deviations from human trajectories, leading to potential risks [5][11]. DriveDPO Framework - DriveDPO integrates human imitation signals and rule-based safety scores into a unified strategy distribution for direct policy optimization, addressing the shortcomings of both imitation learning and score-based methods [8][12]. - The framework employs an iterative Direct Preference Optimization (DPO) approach to prioritize trajectories that are both human-like and safe, enhancing the model's responsiveness to safety preferences [8][19]. Experimental Results - Extensive experiments on the NAVSIM benchmark dataset demonstrated that DriveDPO achieved a PDMS (Policy Decision Metric Score) of 90.0, outperforming previous methods by 1.9 and 2.0 points respectively [8][22]. - Qualitative results indicate significant improvements in safety and compliance in complex driving scenarios, showcasing the potential of DriveDPO for safety-critical applications [12][28]. Contributions - The article identifies key challenges in current imitation learning and score-based methods, proposing DriveDPO as a solution that combines unified strategy distillation with safety-oriented DPO for effective policy optimization [12][28]. - The framework's ability to suppress unsafe behaviors while enhancing overall driving performance highlights its potential for deployment in autonomous driving systems [12][28].

Core Viewpoint - The article discusses the ongoing debate in the field of embodied intelligence regarding the reliance on simulation efficiency versus real-world data, and the potential of world models to redefine the landscape of data utilization in this domain [4][8]. Group 1: Understanding Sim-to-Real Gap - The "Sim-to-Real gap" refers to the discrepancies between simulated environments and real-world scenarios, primarily due to incomplete simulations that fail to accurately replicate visual and physical details [8]. - Research indicates that the gap exists because simulation models do not fully capture the complexities of the real world, leading to limited generalization capabilities and a focus on specific scenarios [8][11]. - Solutions to bridge this gap involve optimizing data, including designing virtual and real data ratios and leveraging AIGC to generate diverse datasets that balance volume and authenticity [11][12]. Group 2: Data Utilization in Embodied Intelligence - There is a consensus among experts that while real data is ideal for training, the current landscape necessitates a reliance on simulation data due to the scarcity of high-quality real-world datasets in the embodied intelligence field [20][21]. - Simulation data plays a crucial role in foundational model iteration and testing, as it allows for safe and efficient algorithm testing before deploying on real machines [21][24]. - The potential of simulation in scaling reinforcement learning is highlighted, as well-constructed simulators can facilitate large-scale parallel training, enabling models to learn from scenarios that are difficult to capture in real life [24][26]. Group 3: World Models and Future Directions - The article emphasizes the significance of world models in future research, particularly in areas like autonomous driving and embodied intelligence, showcasing their potential in general visual understanding and long-term planning [30][32]. - Challenges remain in automating the generation of simulation data and ensuring the diversity and generalization of actions within simulations, which are critical for advancing the field [28][29]. - The introduction of new modalities, such as force and touch, into world models is suggested as a promising direction for future research, despite current limitations in computational resources [30][31]. Group 4: Reaction to Boston Dynamics Technology - Experts acknowledge the advanced capabilities of Boston Dynamics robots, particularly their smooth execution of complex tasks that require sophisticated motion control [33][37]. - The discussion highlights the importance of hardware and data in the field of embodied intelligence, with Boston Dynamics' approach serving as a benchmark for future developments [37][39]. - The consensus is that the seamless performance of these robots is attributed not only to hardware differences but also to superior motion control techniques that could inform future research in embodied intelligence [39][41].

Core Insights - The article introduces WorldSplat, a novel feedforward framework that integrates generative methods with explicit 3D reconstruction for 4D driving scene synthesis, addressing the challenges of generating controllable and realistic driving scene videos [5][36]. Background Review - Generating controllable and realistic driving scene videos is a core challenge in autonomous driving and computer vision, crucial for scalable training and closed-loop evaluation [5]. - Existing generative models have made progress in high-fidelity, user-customized video generation, reducing reliance on expensive real data, while urban scene reconstruction methods have optimized 3D representation and consistency for new view synthesis [5][6]. - Despite advancements, generative and reconstruction methods face challenges in creating unknown environments and synthesizing new views, with existing video generation models often lacking 3D consistency and controllability [5][6]. WorldSplat Framework - WorldSplat combines generative diffusion with explicit 3D reconstruction, constructing dynamic 4D Gaussian representations that can render new views along any user-defined camera trajectory without scene-by-scene optimization [6][10]. - The framework consists of three key modules: a 4D perception latent diffusion model for multimodal latent variable generation, a latent Gaussian decoder for feedforward 4D Gaussian prediction and real-time trajectory rendering, and an enhanced diffusion model for video quality optimization [10][12]. Algorithm Details - The 4D perception latent diffusion model generates multimodal latent variables containing RGB, depth, and dynamic target information based on user-defined control conditions [14][15]. - The latent Gaussian decoder predicts pixel-aligned 3D Gaussian distributions, separating static backgrounds from dynamic targets to create a unified 4D representation [20][21]. - The enhanced diffusion model optimizes the rendered RGB video based on both the original input and the rendered video, enriching spatial details and enhancing temporal coherence [24][27]. Experimental Results - Extensive experiments demonstrate that WorldSplat achieves state-of-the-art performance in generating high-fidelity, temporally consistent free-view videos, significantly benefiting downstream driving tasks [12][36]. - Comparative results show that WorldSplat outperforms existing generative and reconstruction techniques in terms of realism and new view quality [31][32]. Conclusion - The proposed WorldSplat framework effectively integrates generative and reconstruction methods, enabling the generation of explicit 4D Gaussian distributions optimized for high-fidelity, temporally and spatially consistent multi-trajectory driving videos [36].