3D高斯Splatting(GS)技术

Search documents
基于移动设备采集的3DGS实现个性化Real-to-Sim-to-Real导航
具身智能之心· 2025-09-25 00:04
Group 1 - The core issue of embodied AI is the sim-to-real dilemma, where high fidelity in simulation conflicts with cost, leading to challenges in transferring successful strategies from simulation to real-world applications [2] - The potential of 3D Gaussian Splatting (GS) technology has been underexplored, with recent advancements enabling high-fidelity 3D representations from standard devices, addressing the gap between low-cost real scene reconstruction and embodied navigation [3][4] - The proposed method, EmbodiedSplat, consists of a four-stage pipeline that captures real scenes using low-cost mobile devices and reconstructs them in high fidelity for effective training and deployment [4][6] Group 2 - The first stage involves capturing RGB-D data using an iPhone 13 Pro Max and the Polycam app, which simplifies the process and reduces operational barriers [11] - The second stage focuses on mesh reconstruction, utilizing DN-Splatter for 3D GS training, ensuring geometric consistency and minimizing reconstruction errors [11][12] - The third stage includes simulation training with a composite reward function to balance success, path efficiency, and collision avoidance, employing a two-layer LSTM for decision-making [10][13] Group 3 - The fourth stage is real-world deployment, where the Stretch robot connects to a remote server for strategy inference, allowing real-time navigation based on observed images [14][17] - Experiments validate the zero-shot performance of pre-trained strategies in new environments, revealing that scene scale significantly impacts performance [20][22] - Fine-tuning pre-trained strategies leads to substantial performance improvements across various environments, demonstrating the effectiveness of personalized adjustments [25][28] Group 4 - The study highlights the limited success of zero-shot transfer from simulation to real-world scenarios, with significant performance gaps observed [32] - Fine-tuning enhances the transferability of strategies, with success rates increasing significantly after adjustments [32][35] - The necessity of large-scale pre-training is emphasized, as it provides foundational knowledge that aids in adapting to new environments and overcoming real-world challenges [35][44]