理想TOP2·2025-12-15 12:02Core Viewpoint - The article discusses the advancements in the development of MindGPT, a multimodal cognitive model designed to enhance human-machine interaction in smart vehicles, emphasizing its capabilities in perception, understanding, and interaction [2][20][39]. Group 1: Technology and Model Architecture - MindGPT is built on a self-developed TaskFormer structure, which has been recognized for its performance in industry evaluations [2][35]. - The model incorporates multimodal perception capabilities, allowing it to process audio and visual data simultaneously, enhancing user interaction through features like voice recognition and gesture control [29][30]. - The architecture supports a complete agent capability, integrating perception, planning, memory, tools, and action [35][36]. Group 2: Training and Performance - The training strategy focuses on 15 key areas relevant to in-car scenarios, utilizing self-supervised learning and reinforcement learning from human feedback (RLHF) to cover over 110 domains and 1,000 specialized capabilities [3][35]. - The training platform, Li-PTM, achieves training speeds that are significantly faster than industry standards, with SFT phase speeds over three times better than the best open-source capabilities [46][47]. - The model's inference engine, LisaRT-LLM, has been optimized for performance, achieving a throughput increase of over 1.3 times compared to previous models under high concurrency [5][53]. Group 3: User Interaction and Experience - MindGPT aims to create a natural interaction experience by allowing users to communicate with the vehicle using simple commands and gestures, reducing the complexity of user input [10][32]. - The system is designed to understand and remember user preferences, providing personalized interactions based on historical conversations [36][39]. - The integration of advanced AI technologies aims to enhance emotional connections between users and their vehicles, creating a more immersive experience [14][18].