Core Viewpoint - The article discusses the evaluation of the PI0-FAST-DROID model in real-world scenarios, highlighting its potential and limitations in robotic operations, particularly in handling new objects and tasks without extensive prior training [4][10][77]. Evaluation Method - The evaluation utilized the π₀-FAST-DROID model, specifically fine-tuned for the DROID robot platform, which includes a Franka Panda robot equipped with cameras [5][10]. - The assessment involved over 300 trials across various tasks, focusing on the model's ability to perform in diverse environments, particularly in a kitchen setting [10][11]. Findings - The model demonstrated a strong prior assumption of reasonable behavior, often producing intelligent actions, but these were not always sufficient to complete tasks [11]. - Prompt engineering was crucial, as variations in task descriptions significantly affected success rates, indicating the need for clear and structured prompts [12][59]. - The model exhibited impressive visual-language understanding and could mimic continuous actions across different scenarios [13][28]. Performance in Complex Scenarios - The model showed robust performance in recognizing and manipulating transparent objects, which is a significant challenge for traditional methods [20][27]. - It maintained focus on tasks despite human movement in the background, suggesting effective prioritization of relevant visual inputs [25]. Limitations - The model faced challenges with semantic ambiguity and often froze during tasks, particularly when it encountered unfamiliar commands or objects [39][42]. - It lacked memory, which hindered its ability to perform multi-step tasks effectively, leading to premature task completion or freezing [43][32]. - The model struggled with precise spatial reasoning, particularly in estimating distances and heights, which resulted in failures during object manipulation tasks [48][50]. Task-Specific Performance - The model's performance varied across different task categories, with notable success in simple tasks but significant challenges in complex operations like pouring liquids and interacting with household appliances [89][91][100]. - For instance, it achieved a 73.3% progress rate in pouring toy items but only 20% when dealing with real liquids, indicating limitations in physical capabilities [90]. Conclusion - The evaluation indicates that while the PI0 model shows promise as a generalist policy in robotic applications, it still requires significant improvements in instruction adherence, fine manipulation, and handling partial observability [77][88].

点击下方 卡片 ，关注" 具身智能 之心 "公众号 作者丨 Jie Wang等 编辑丨具身智能之心 本文只做学术分享，如有侵权，联系删文 >> 点击进入→ 具身智能之心 技术交流群 更多干货，欢迎加入国内首个具身智能全栈学习社区 ： 具身智能之心知识星球 (戳我) ， 这里包含所有你想要的。 blog：https://penn-pal-lab.github.io/Pi0-Experiment-in-the-Wild/ 这是 GRASP Lab 的一篇在复杂真实场景中（in the wild）评估 PI0-FAST-DROID 的工作，这样可以更直观的帮助理解 PI0 这类通用 policy 的目前性能和边界，以 及探索未来可以解决的方向。 当然现在还有更新一代的 PI0.5 方案（但是还没有开源）。 相关资料 ： Droid 数据集 ：https://droid-dataset.github.io/ 引言： 机器人操作领域，一直以来都缺少能够"开箱即用"地处理新物体、新位置和新任务的预训练模型 。机器人专家们往往曾经历过令人沮丧的过程：为了获取一个 机器人 policy，不得不进行繁琐的工程设计和数据收集，结 ...