李飞飞的答案：大模型之后，Agent向何处去？

Core Viewpoint - The article discusses the emergence of Agent AI, highlighting its potential to revolutionize various fields through a new cognitive architecture that integrates perception, cognition, action, learning, and memory [4][9][10]. Summary by Sections Introduction to Agent AI - 2025 is anticipated to be the year of Agent AI, with increasing interest in concepts like AI Agents and Agentic AI [4]. - A significant paper led by Fei-Fei Li titled "Agent AI: Surveying the Horizons of Multimodal Interaction" has sparked widespread discussion in the industry [4][6]. Framework of Agent AI - The paper establishes a clear framework for Agent AI, integrating various technologies into a unified perspective [6][7]. - It outlines five core modules: Environment and Perception, Cognition, Action, Learning, and Memory, which together form a dynamic cognitive loop [10][12][14][16][17]. Core Modules Explained - Environment and Perception: Agents actively perceive information from their surroundings, incorporating task planning and skill observation [12]. - Cognition: Acts as the processing center, utilizing large language models (LLMs) and visual language models (VLMs) for reasoning and strategy formulation [14]. - Action: Converts cognitive decisions into executable commands, affecting the environment [15]. - Learning: Emphasizes continuous learning through various mechanisms, allowing agents to adapt based on feedback [16]. - Memory: Features a structured system for long-term knowledge retention, enabling agents to leverage past experiences [17]. Role of Large Models - The development of Agent AI is driven by the maturity of foundation models, particularly LLMs and VLMs, which provide agents with extensive knowledge and planning capabilities [20]. - The paper addresses the challenge of "hallucination" in models, emphasizing the importance of environmental interaction to mitigate this issue [21][22]. Application Potential - The paper explores Agent AI's applications in three key areas: - Gaming: Agent AI can create dynamic NPCs that interact meaningfully with players, enhancing immersion [24][25]. - Robotics: Robots can execute complex tasks based on natural language commands, improving user interaction [27]. - Healthcare: Agent AI can assist in preliminary diagnostics and patient monitoring, increasing efficiency in healthcare delivery [29][31]. Conclusion - The paper recognizes that Agent AI is still in its early stages, facing challenges in integrating multiple modalities and creating general agents for diverse applications [32]. - It proposes new evaluation benchmarks to guide the development and measure progress in the field [32].