一般对应闭环仿真或场景重建的算法岗位，日常工作聚焦在： 一般公司需要 5-20 人的算法团队支撑闭环仿真中重建的优化。除此之外，在和其他小伙伴的交流过程中，我们发现 云端的数据生产 也有需求，比如BEV视角下 的静态路面重建，即2DGS重建，可以应用到静态真值生产中，比如RoGS。最近小米的ParkGaussian把GS应用到 泊车场景中 。综合来说，每个方向都需要至少10 人左右的算法团队规模支撑最基本的功能需求。 点击下方 卡片 ，关注" 自动驾驶之心 "公众号 戳我-> 领取 自动驾驶近30个 方向 学习 路线 上周有企业做招聘的朋友和柱哥聊了聊，26年需要在重建方向投入一些HC，借这个机会，和大家盘一下GS在自驾业内的职位和需求。 对于重建来说，主要是用于测试的闭环仿真。闭环仿真的用途比较明确，对于一个离线的clip数据，用3DGS重建动静态元素，验证新模型在重建clip上的效果，是 否可以预测合理的新轨迹，沿用新的轨迹能否正常行驶。 添加助理领取优惠！ 讲师介绍 Chris：QS20 硕士，现任某Tier1厂算法专家，目前从事端到端仿真、多模态大模型、世界模型等前沿算法的预研和量产，参与过全球TOP ...

Core Viewpoint - The article discusses the advancements and applications of 3D Generative Systems (3DGS) in the context of autonomous driving, emphasizing the importance of scene reconstruction and generation technologies for creating realistic driving environments [1][3]. Group 1: Scene Reconstruction Work - The publication of StreetGaussian at ECCV2024 marks a significant step in the wave of autonomous driving scene reconstruction [2]. - A large-scale vehicle asset reconstruction dataset named 3DRealCar has been released [2]. - The Balanced3DGS algorithm accelerates 3DGS training by nearly eight times [2]. - The Hierarchy UGP paper is set to be presented at ICCV2025, focusing on autonomous driving scene reconstruction [2]. - StyledStreets introduces a multi-style scene generation algorithm with spatiotemporal consistency for autonomous driving [2]. Group 2: Importance of Scene Reconstruction - Traditional vehicle testing heavily relies on real-world tests, which often fail to replicate many corner cases, and there is a significant domain gap in conventional simulation environments [3]. - The high-fidelity scene reconstruction and editing capabilities of 3DGS make it possible to address these challenges [3]. - The development trajectory of 3DGS is clear: static reconstruction → dynamic reconstruction → hybrid reconstruction → feed-forward GS, with applications extending beyond autonomous driving to 3D fields, embodied intelligence, and the gaming industry [3]. Group 3: 3DGS Learning Path - A comprehensive learning roadmap for 3DGS has been developed, covering point cloud processing, deep learning theories, real-time rendering, and practical coding [5]. - The course titled "3DGS Theory and Algorithm Practical Tutorial" aims to provide a structured approach to mastering the 3DGS technology stack [5]. Group 4: Course Structure - The course consists of six chapters, starting with foundational knowledge in computer graphics and progressing through principles, algorithms, and specific applications in autonomous driving [10][11][12][13][14]. - Each chapter includes practical assignments and discussions on important research directions and industry applications [13][14][15]. Group 5: Target Audience and Outcomes - The course is designed for individuals with a background in computer graphics, visual reconstruction, and programming, aiming to equip them with comprehensive knowledge and skills in 3DGS [19]. - Participants will gain insights into industry demands, pain points, and opportunities for further engagement with academic and industrial peers [15][19].

Core Viewpoint - The article discusses the essence of world models and digital twins in the context of autonomous driving, emphasizing their role in training perception models in virtual environments and applying them to real-world scenarios [5][6]. Group 1: World Models - World models are defined as the ultimate goal of modeling the physical world, focusing on "spatiotemporal cognition" and requiring vast amounts of video data for training [7]. - The development of world models is shifting from simple visual dynamics simulation to creating immersive interactive environments that reflect real-world complexities [8]. - The core consensus among researchers is that the primary purpose of world models is to understand dynamic environments and predict future scenarios [7][9]. Group 2: Applications in Autonomous Driving - In autonomous driving, world models must provide real-time perception of road conditions and accurately predict their evolution, focusing on immediate environmental awareness and complex trend forecasting [11]. - Key features of effective world models include physical consistency, multiscale spatiotemporal modeling, causal reasoning capabilities, and the ability to generate interactive environments [11]. - Various companies are implementing world models, such as NIO's NWM world model for simulation training, Xiaomi's ORION framework for integrating simulation tools, and Wayve's GAIA-1 for generative world modeling [17]. Group 3: Digital Twins - Digital twins are defined as virtual representations of physical systems that allow for low-cost, high-efficiency research on key technologies and solutions in autonomous driving [19]. - The role of digital twins extends beyond mere observation; they participate in iterative processes to enhance real-world applications [19]. - Digital twins facilitate the modeling of physical world elements in virtual spaces, enabling further work on perception models and system iterations [20][21]. Group 4: Related Technologies - Technologies such as 3D occupancy grids and point clouds are utilized to predict spatial occupancy and enhance scene understanding in autonomous driving [22]. - The integration of multimodal inputs, including visual and LiDAR data, is crucial for improving depth estimation and overall perception accuracy [92]. - The article highlights the importance of self-supervised learning techniques in enhancing the efficiency of 3D scene reconstruction and semantic labeling in autonomous driving applications [90][91].

Core Insights - The article highlights the advancements in 3D Gaussian Splatting (3DGS) technology, particularly its application in autonomous driving, and emphasizes the need for structured learning pathways in this rapidly evolving field [2][4]. Group 1: 3DGS Technology and Developments - Tesla's introduction of 3D Gaussian Splatting at ICCV has garnered significant attention, indicating a shift towards feed-forward GS algorithms in the industry [2]. - The rapid iteration of 3DGS technology includes static reconstruction (3DGS), dynamic reconstruction (4DGS), and surface reconstruction (2DGS), showcasing the need for effective learning resources [4]. Group 2: Course Offering - A comprehensive course titled "3DGS Theory and Algorithm Practical Tutorial" has been developed to provide a structured learning roadmap for newcomers, covering essential theories and practical applications [4]. - The course is designed to help participants understand point cloud processing, deep learning, real-time rendering, and coding practices, with a focus on hands-on experience [4]. Group 3: Course Structure - The course consists of six chapters, starting with foundational knowledge in computer graphics and progressing to advanced topics such as feed-forward 3DGS and its applications in autonomous driving [8][9][10][11][12]. - Each chapter includes practical assignments and discussions to enhance understanding and application of the concepts learned [8][9][10][11][12]. Group 4: Target Audience and Prerequisites - The course is aimed at individuals with a background in computer graphics, visual reconstruction, and programming, particularly those interested in pursuing careers in the 3DGS field [17]. - Participants are expected to have a foundational understanding of probability, linear algebra, and programming languages such as Python and PyTorch [17].

Core Insights - The article discusses the rising importance of 3DGS (3D Geometry Synthesis) technology in various fields, particularly in autonomous driving, healthcare, virtual reality, and gaming [2][4] - A comprehensive learning roadmap for 3DGS has been developed to address the industry's need for effective training in scene reconstruction and world modeling [4][6] Course Overview - The course titled "3DGS Theory and Algorithm Practical Tutorial" aims to provide a detailed understanding of 3DGS algorithms, covering both theoretical foundations and practical applications [6][10] - The course is designed in six chapters, starting from basic knowledge to advanced research directions in 3DGS [10][11] Chapter Summaries - **Chapter 1: Background Knowledge** Introduces foundational concepts in computer graphics, including implicit and explicit representations of 3D space, rendering pipelines, and tools like SuperSplat and COLMAP [10][11] - **Chapter 2: Principles and Algorithms** Focuses on the core principles of 3DGS, including dynamic and surface reconstruction, and introduces the 3DGRUT framework for practical learning [11][12] - **Chapter 3: 3DGS in Autonomous Driving** Highlights key works in the field, such as Street Gaussian and OmniRe, and utilizes DriveStudio for practical applications [12][13] - **Chapter 4: Important Research Directions** Discusses significant research areas like COLMAP extensions and depth estimation, emphasizing their relevance to both industry and academia [13][14] - **Chapter 5: Feed-Forward 3DGS** Explores the development and principles of feed-forward 3DGS, including recent algorithms like AnySplat and WorldSplat [14][15] - **Chapter 6: Q&A Discussion** Provides a platform for participants to discuss industry pain points and job demands related to 3DGS [15] Target Audience and Learning Outcomes - The course is aimed at individuals with a background in computer graphics, visual reconstruction, and programming, particularly those interested in pursuing careers in the 3DGS field [19][17] - Participants will gain comprehensive knowledge of 3DGS theory, algorithm development frameworks, and opportunities for networking with industry professionals [19][17]

Core Insights - The article discusses the advancements in autonomous driving technology, particularly focusing on the development and implementation of VLA (Vehicle-Language Architecture) by Xiaopeng Motors, highlighting its significance in the industry [14]. Group 1: Company Developments - Xiaopeng Motors has announced the launch of VLA 2.0, which represents a significant step in the evolution of autonomous driving technology, transitioning from perception-based systems to more integrated approaches [14]. - The article reflects on a year of research and development in VLA, indicating a shift in focus from traditional perception methods to VLA, which aims to enhance the vehicle's decision-making capabilities [14]. Group 2: Industry Trends - The article notes a growing trend in the industry towards end-to-end autonomous driving solutions, with VLA being positioned as a potential game-changer in how vehicles interact with their environment [14]. - There is a discussion on the competitive landscape, particularly the debate between world models and VLA routes, suggesting that the industry is at a crossroads in terms of technological direction [14]. Group 3: Research and Academic Contributions - The article mentions recent academic contributions, such as the paper from The Chinese University of Hong Kong (Shenzhen) and Didi, which proposes a new method for dynamic driving scene reconstruction, indicating ongoing research efforts in the field [14].

Core Viewpoint - Neural Rendering (NERF/3DGS) is revolutionizing 3D reconstruction technology, significantly enhancing the realism of images used in autonomous driving and embodied intelligence simulations, addressing the limitations of traditional computer graphics rendering [3][4]. Group 1: Background and Technology - NERF and 3DGS utilize neural networks to express spatial data, excelling in new perspective synthesis, which is crucial for sensor simulation in autonomous driving and embodied intelligence [3]. - The integration of NERF and 3DGS into existing simulation frameworks is proposed as a more efficient approach than developing new frameworks from scratch, allowing for real-time rendering while leveraging existing 3D digital assets and algorithm interfaces [3][4]. Group 2: Implementation in Simulation Software - NVIDIA's Isaac Sim has incorporated neural rendering technology, enabling the insertion of 3DGS models into simulation environments, allowing for both static backgrounds and dynamic interactive objects [4][5]. - The process of importing 3DGS models into Isaac Sim involves generating USDZ models and ensuring they possess physical properties for interaction within the simulation [5][8]. Group 3: Model Interaction and Physics - To achieve realistic interactions, imported models must have physical attributes added, such as collision properties, to ensure they interact correctly with other objects in the simulation [8][14]. - The integration of dynamic objects, such as a LEGO bulldozer, into the simulation environment demonstrates the capability of 3DGS models to interact with both static and dynamic elements [11][15]. Group 4: Performance and Future Considerations - The performance metrics indicate that even with a high workload, the simulation maintains a good frame rate and low memory usage, showcasing the efficiency of the neural rendering technology [17]. - Future challenges include improving light and shadow interactions between 3DGS models, providing accurate ground truth information for algorithms, and enhancing computational efficiency for larger scenes [18][19].

Core Viewpoint - 3D reconstruction is a critical intersection of computer vision and graphics, serving as the digital foundation for cutting-edge applications such as virtual reality, augmented reality, autonomous driving, and digital twins. Recent advancements in new perspective synthesis technologies, represented by Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS), have significantly improved reconstruction quality, speed, and dynamic adaptability [5][6]. Group 1: Introduction and Demand - The resurgence of interest in 3D reconstruction is driven by new application demands across various fields, including city-scale digital twins requiring kilometer-level coverage and centimeter-level accuracy, autonomous driving simulations needing dynamic traffic flow and real-time semantics, and AR/VR social applications demanding over 90 FPS and photo-realistic quality [6]. - Traditional reconstruction pipelines are inadequate for these new requirements, prompting the integration of geometry, texture, and lighting through differentiable rendering techniques [6]. Group 2: Traditional Multi-View Geometry Reconstruction - The traditional multi-view geometry approach (SfM to MVS) has inherent limitations in quality, efficiency, and adaptability to dynamic scenes, which have been addressed through iterative advancements in NeRF and 3DGS technologies [7]. - A comprehensive comparison of various methods highlights the evolution and future challenges in the field of 3D reconstruction [7]. Group 3: NeRF and Its Innovations - NeRF models scenes as continuous 5D functions, enabling advanced rendering techniques that have evolved significantly from 2020 to 2024, addressing issues such as data requirements, texture limitations, lighting sensitivity, and dynamic scene handling [13][15]. - Various methods have been developed to enhance quality and efficiency, including Mip-NeRF, NeRF-W, and InstantNGP, each contributing to improved rendering speeds and reduced memory usage [17][18]. Group 4: 3DGS and Its Advancements - 3DGS represents scenes as collections of 3D Gaussians, allowing for efficient rendering and high-quality output. Recent methods have focused on optimizing rendering quality and efficiency, achieving significant improvements in memory usage and frame rates [22][26]. - The comparison of 3DGS with other methods shows its superiority in rendering speed and dynamic scene reconstruction capabilities [31]. Group 5: Future Trends and Conclusion - The next five years are expected to see advancements in hybrid representations, real-time processing on mobile devices, generative reconstruction techniques, and multi-modal fusion for robust reconstruction [33]. - The ultimate goal is to enable real-time 3D reconstruction accessible to everyone, marking a shift towards ubiquitous computing [34].

Core Viewpoint - The article discusses the current state and future potential of end-to-end (E2E) autonomous driving systems, emphasizing the need for a shift from modular to E2E approaches in the industry, while acknowledging the challenges and limitations that still exist in achieving maturity in this technology [3][5]. Group 1: End-to-End Autonomous Driving - The concept of end-to-end systems involves directly processing raw sensor data to output control signals for vehicles, representing a significant shift from traditional modular approaches [3][4]. - E2E systems are seen as a way to provide a comprehensive representation of the information affecting vehicle behavior, which is crucial for handling the open-set scenarios of autonomous driving [4]. - The industry is currently divided, with some companies focusing on Vehicle Language Architecture (VLA) and others on traditional methods, but there is a consensus that E2E systems are the future [2][5]. Group 2: Industry Trends and Challenges - There is a growing recognition that autonomous driving is transitioning from rule-based to knowledge-driven systems, which necessitates a deeper understanding of E2E methodologies [5]. - Despite the high potential of E2E systems, there are still significant challenges to overcome before they can fully replace traditional planning and control methods [5]. - The article suggests that companies should allow more time for E2E systems to mature rather than rushing to implement them without adequate understanding [5]. Group 3: Community and Learning Resources - The "Autonomous Driving Heart Knowledge Planet" community aims to provide a platform for sharing knowledge and resources related to autonomous driving, including technical routes and job opportunities [8][18]. - The community has gathered over 4,000 members and aims to expand to nearly 10,000 within two years, offering a space for both beginners and advanced learners to engage with industry experts [8][18]. - Various learning resources, including video tutorials and technical discussions, are available to help members navigate the complexities of autonomous driving technologies [12][18].

Core Viewpoint - The article discusses the development and implementation of the Street Gaussians algorithm for dynamic scene representation in autonomous driving, highlighting its efficiency in training and rendering compared to previous methods [2][3]. Group 1: Background and Challenges - Previous methods faced challenges such as slow training and rendering speeds, as well as inaccuracies in vehicle pose tracking [3]. - Street Gaussians aims to generate realistic images for view synthesis in dynamic urban street scenes by modeling them as a combination of foreground moving vehicles and static backgrounds [3][4]. Group 2: Technical Implementation - The background model is represented as a set of points in world coordinates, each assigned a 3D Gaussian to represent geometry and color, with parameters optimized to avoid invalid values [8]. - The object model for moving vehicles includes a set of optimizable tracking poses and point clouds, with similar Gaussian attributes to the background model but defined in local coordinates [11]. - A 4D spherical harmonic model is introduced to encode temporal information into the appearance of moving vehicles without high storage costs [12]. Group 3: Initialization and Data Handling - Street Gaussians utilizes aggregated LiDAR point clouds for initialization, addressing the limitations of traditional SfM point clouds in urban environments [17]. - For objects with fewer than 2,000 LiDAR points, random sampling is employed to ensure sufficient data for model initialization [17]. Group 4: Course and Learning Opportunities - The article promotes a specialized course on 3D Gaussian Splatting (3DGS), covering various subfields and practical applications in autonomous driving, aimed at enhancing understanding and implementation skills [26][35].