Workflow
4D重建
icon
Search documents
SIGGRAPH Asia 2025|30FPS普通相机恢复200FPS细节,4D重建方案来了
机器之心· 2025-12-14 04:53
Core Viewpoint - The article discusses advancements in 4D reconstruction technology, specifically focusing on a new method that combines asynchronous capture with a video diffusion model to enhance the quality of high-speed dynamic scene reconstruction using low-cost hardware [3][10]. Group 1: Hardware Innovation - The asynchronous capture method allows multiple cameras to work in a "relay" fashion, overcoming the speed limitations of individual cameras. This method introduces a slight delay in the activation of different cameras, effectively doubling the frame rate from 25 FPS to 100 FPS or even reaching 200 FPS by organizing the cameras into groups [5][6][8]. Group 2: Software Innovation - A video diffusion model is employed to address the "sparse view" problem that arises from asynchronous capture, which can lead to visual artifacts in the initial 4D reconstruction. This model is trained to repair these artifacts and enhance video quality by utilizing the spatio-temporal context provided by the input video [9][10][13]. Group 3: Overall Process - The method integrates hardware capture with AI algorithms in an iterative optimization framework. The process includes initial reconstruction using asynchronous capture, generating pseudo ground truth videos, enhancing these videos with the diffusion model, and optimizing the 4D Gaussian model based on the enhanced output [14][15][17]. Group 4: Method Effectiveness - The proposed method outperforms several state-of-the-art techniques in key metrics such as Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), and Learned Perceptual Image Patch Similarity (LPIPS) across two public datasets, demonstrating its effectiveness in producing high-quality 4D reconstructions [19][21]. Group 5: Real-World Validation - A multi-view capture system consisting of 12 cameras operating at 25 FPS was established to validate the method in real-world scenarios. The experiments confirmed that the approach could robustly reconstruct high-quality, temporally consistent 4D content even in complex asynchronous capture environments [22].
ICCV高分论文|可灵ReCamMaster在海外爆火,带你从全新角度看好莱坞大片
机器之心· 2025-07-23 10:36
Core Viewpoint - The article introduces ReCamMaster, a video generation model that allows users to reframe existing videos along new camera trajectories, addressing common issues faced by video creators such as equipment limitations and shaky footage [2][17]. Group 1: ReCamMaster Overview - ReCamMaster enables users to upload any video and specify a new camera path for re-framing, thus enhancing the quality of video production [2]. - The model has significant applications in fields such as 4D reconstruction, video stabilization, autonomous driving, and embodied intelligence [3][17]. Group 2: Innovation and Methodology - The primary innovation of ReCamMaster lies in its new video conditioning paradigm, which combines condition video and target video in a time dimension after patchifying, resulting in substantial performance improvements over previous methods [11][17]. - The model achieves near-product-level performance in re-framing single videos, demonstrating the potential of video generation models in this area [13][17]. Group 3: MultiCamVideo Dataset - The MultiCamVideo dataset, created using Unreal Engine 5, consists of 13,600 dynamic scenes captured by 10 cameras along different trajectories, totaling 136,000 videos and 112,000 unique camera paths [13]. - The dataset features 66 different characters, 93 types of actions, and 37 high-quality 3D environments, providing a rich resource for research in camera-controlled video generation and 4D reconstruction [13][17]. Group 4: Experimental Results - ReCamMaster has shown significant performance improvements compared to baseline methods in experimental comparisons [15][17].