谢赛宁也玩MC？开源全新世界模型生成多人一致的游戏视角

Core Insights - The article discusses the significant role of video games in advancing AI technology, particularly in training AI to understand physical interactions and world models [1] - The research team led by Xie Saining is exploring new directions in world models using the game "Minecraft" [3] Group 1: Video World Model Development - The Solaris model is the first multiplayer video world model that generates consistent first-person views for multiple players [5] - The research team identified that existing video world models only handle single-player perspectives, which do not accurately reflect real-world interactions [7] - SolarisEngine, a custom-built data collection system, supports coordinated multi-agent interactions and visual capture in games like "Minecraft" [7][14] Group 2: Data Collection and Model Training - The team collected a dataset of 12.6 million frames from 9,240 task rounds, focusing on various tasks such as building, combat, and exploration [16] - The dataset is the first of its kind with action annotations suitable for training world models [17] - The model utilizes a combination of flow matching and diffusion forcing to predict future observations based on players' historical actions [19] Group 3: Model Architecture and Improvements - The model architecture includes enhancements such as expanded action space and multi-player self-attention layers to facilitate information exchange among players [20][22] - The improvements allow the model to generalize to any number of players, although it has currently been trained with data from two players [20] Group 4: Evaluation Metrics and Results - The Solaris Eval dataset was created to test five collaborative abilities, including movement, positioning, consistency, memory, and building capabilities [24][28] - The results indicate that the Solaris model outperforms previous methods in visual quality and quantitative metrics across various evaluation categories [27][29]