Core Viewpoint - The article discusses the introduction of the Puffin unified multimodal model, which integrates the understanding of camera parameters and the generation of corresponding perspective images, addressing previous limitations in multimodal models [2][12]. Research Motivation - The ability to understand scenes from any perspective and hypothesize about the environment beyond the field of view allows for the mental recreation of a real-world with free viewpoints [8]. - Cameras serve as crucial interfaces for machines to interact with the physical world and achieve spatial intelligence [9]. Model Design - The Puffin model combines language regression and diffusion-based generation capabilities, enabling understanding and creation of scenes from any angle [12]. - A geometric-aligned visual encoder is introduced to maintain geometric fidelity while ensuring strong semantic understanding, addressing performance bottlenecks in existing models [14]. Thinking with Camera Concept - The concept of "thinking with camera" allows for the decoupling of camera parameters in a geometric context, establishing connections between spatial visual cues and professional photography terminology [20][21]. - The model incorporates spatially constrained visual cues and professional photography terms to bridge the gap between low/mid-level camera geometry and high-level multimodal reasoning [22][23]. Shared Thinking Chain - A shared thinking chain mechanism is introduced to unify the reasoning processes between controllable image generation and understanding tasks, enhancing the model's ability to generate accurate spatial structures [28]. Puffin-4M Dataset - The Puffin-4M dataset consists of approximately 4 million image-language-camera triples, addressing the scarcity of multimodal datasets in the spatial intelligence domain [29][30]. Experimental Results - Puffin demonstrates superior performance in camera understanding tasks, achieving significant improvements in accuracy compared to existing methods [36][38]. - The model's robustness is evident in various scene configurations, showcasing its capability for controllable image generation [41]. Applications - Puffin can assist in the insertion of virtual 3D objects into natural scene images through precise camera parameter predictions [43]. - The model can be flexibly extended to various cross-perspective tasks, including spatial imagination and world exploration, maintaining spatial consistency in generated results [44]. Future Plans - The team aims to enhance Puffin's cross-perspective capabilities and expand its application to video generation and understanding centered around camera parameters, promoting broader use in dynamic and immersive scenarios [45].