Workflow
机器之心
icon
Search documents
大模型智能体如何突破规模化应用瓶颈,核心在于Agentic ROI
机器之心· 2025-05-30 04:16
Core Viewpoint - The main barrier to the usability of large language model agents (LLM Agents) is not the capability of the models but rather the "Agentic ROI" which has not reached a practical threshold for widespread application [1][3][4]. Group 1: Agentic ROI Concept - Agentic ROI (Agentic Return on Investment) is a key metric that measures the ratio of "information yield" to "usage cost" for LLM Agents in real-world scenarios [4]. - Usability is achieved only when the quality of information exceeds a certain threshold and the ratio of time and cost saved by the agent is sufficiently high [4][5]. Group 2: Current Application Landscape - Most LLM Agents are currently applied in high human task time cost scenarios, such as research and programming, where human labor is intensive, thus allowing for significant efficiency improvements [7]. - In everyday applications with high user demand, such as e-commerce and personal assistants, the tasks are simpler, leading to lower marginal value from LLM Agents, which may introduce additional interaction costs and delays, resulting in low Agentic ROI [7]. Group 3: Development Trajectory - The development path of LLM Agents is characterized by a "zigzag" model of first scaling up to enhance information quality, followed by scaling down to reduce time and cost while maintaining quality [9]. - The evolution of foundational models, such as the OpenAI series, illustrates this zigzag trend, with significant performance improvements in larger models and the introduction of smaller models that maintain performance while reducing inference costs and delays [9]. Group 4: Scaling Up Information Quality - Pre-training scaling involves expanding model size, data volume, and computational resources to enhance foundational capabilities in language understanding and reasoning [11]. - Post-training scaling, including supervised fine-tuning and reinforcement learning, aligns the agent's performance with human needs and values, relying on extensive interaction data for continuous learning [12]. - Test-time scaling focuses on building a world model that supports multimodal interactions and can handle complex tasks while reflecting real-world uncertainties [13]. Group 5: Ensuring Robustness and Security - Ensuring the robustness and security of LLM Agents is crucial for enhancing information quality, preventing exploitation of reward mechanisms, and safeguarding against data contamination and feedback manipulation [16]. Group 6: Scaling Down to Reduce Time and Cost - Introducing memory mechanisms allows agents to skip redundant calculations, leveraging past knowledge to enhance processing speed [18]. - Model compression techniques can significantly reduce computational resources and inference delays without compromising performance [18]. - Optimizing reasoning strategies and infrastructure can further enhance the efficiency and responsiveness of LLM Agents [18]. Group 7: Cost Management - Reducing interaction time by enabling agents to proactively understand user intent can lower cognitive burdens and improve user experience [19]. - Managing operational costs effectively is essential, especially in large-scale deployments, by optimizing context management and controlling inference complexity [19]. - Agentic ROI serves as a framework for evaluating the real usability of LLM Agents, shifting focus from mere model performance to practical benefits and comprehensive efficiency [19].
多模态扩散模型开始爆发,这次是高速可控还能学习推理的LaViDa
机器之心· 2025-05-30 04:16
Core Viewpoint - The article introduces LaViDa, a large vision-language diffusion model that combines the advantages of diffusion models with the ability to process both visual and textual information effectively [1][5]. Group 1: Model Overview - LaViDa is a vision-language model that inherits the high speed and controllability of diffusion language models, achieving impressive performance in experiments [1][5]. - Unlike autoregressive large language models (LLMs), diffusion models treat text generation as a diffusion process over discrete tokens, allowing for better handling of tasks requiring bidirectional context [2][3][4]. Group 2: Technical Architecture - LaViDa consists of a visual encoder and a diffusion language model, connected through a multi-layer perceptron (MLP) projection network [10]. - The visual encoder processes multiple views of an input image, generating a total of 3645 embeddings, which are then reduced to 980 through average pooling for training efficiency [12][13]. Group 3: Training Methodology - The training process involves a two-stage approach: pre-training to align visual embeddings with the diffusion language model's latent space, followed by end-to-end fine-tuning for instruction adherence [19]. - A third training phase using distilled samples was conducted to enhance the reasoning capabilities of LaViDa, resulting in a model named LaViDa-Reason [25]. Group 4: Experimental Performance - LaViDa demonstrates competitive performance across various visual-language tasks, achieving the highest score of 43.3 on the MMMU benchmark and excelling in reasoning tasks [20][22]. - In scientific tasks, LaViDa scored 81.4 and 80.2 on ScienceQA, showcasing its strong capabilities in complex reasoning [23]. Group 5: Text Completion and Flexibility - LaViDa provides strong controllability for text generation, particularly in text completion tasks, allowing for flexible token replacement based on masked inputs [28][30]. - The model can dynamically adjust the number of tokens generated, successfully completing tasks that require specific constraints, unlike autoregressive models [31][32]. Group 6: Speed and Quality Trade-offs - LaViDa allows users to balance speed and quality by adjusting the number of diffusion steps, demonstrating flexibility in performance based on application needs [33][35]. - Performance evaluations indicate that LaViDa can outperform autoregressive baselines in speed and quality under certain configurations, highlighting its adaptability [35].
美团开放AI代码工具,零代码实现全栈能力,项目负责人揭秘架构细节
机器之心· 2025-05-30 04:16
Core Viewpoint - Meituan has developed a free AI no-code tool called NoCode, enabling users without programming experience to create applications through natural language and dialogue, significantly lowering development barriers and enhancing creativity [2][3][4]. Group 1: Product Features - NoCode allows users to generate code, preview results in real-time, make localized modifications, and deploy applications with a single click [12][10]. - The tool is designed to assist small and medium-sized businesses in IT and digital upgrades, showcasing various applications from websites to data analysis tools [4][10]. - Users can create applications by simply describing their ideas in natural language, with NoCode interpreting and converting these into functional capabilities [12][10]. Group 2: Technical Architecture - NoCode operates on a multi-layer architecture, including infrastructure, runtime sandbox, and agent application layers, utilizing various AI models for collaboration [24][25]. - The tool employs a specialized 7B parameter model to enhance code generation speed and efficiency, achieving a generation rate of 2000 tokens per second without compromising accuracy [27][28]. - Continuous optimization and iteration are integral to NoCode's development, with frequent updates and improvements based on user feedback and internal testing [44][48]. Group 3: User Experience and Efficiency - The implementation of NoCode has led to significant efficiency improvements within Meituan, with reports indicating that AI-generated code accounts for 27% of the code submitted in Q1 2023, with expectations for further increases [40][41]. - Non-technical users have found the tool to be three times more utilized than technical users, indicating its accessibility and effectiveness in various roles, including product managers and data analysts [21][39]. - The tool has enabled rapid prototyping and development cycles, allowing users to create functional applications in a fraction of the time previously required [36][39]. Group 4: Future Directions - Meituan plans to enhance NoCode's stability and user experience while exploring the development of a more professional IDE called "Dev Mode" to cater to advanced user needs [48][50]. - The company aims to democratize AI technology, making it more accessible for users across different skill levels, and fostering a collaborative environment between non-technical and technical users [22][46].
ICML 2025 Spotlight | 谁导致了多智能体系统的失败?首个「自动化失败归因」研究出炉
机器之心· 2025-05-30 03:28
问题来了:到底是哪个 Agent 出了错?又是在对话流程的哪一环节?调试这样的多智能体系统如同大海捞针,需要翻阅大量复杂日志,极其耗时。 这并非虚构。在多智能体 LLM 系统中,失败常见但难以诊断。随着这类系统愈加普及,我们急需新方法快速定位错误。正因如此,ICML 2025 的一篇 Spotlight 论 文提出了「自动化失败归因(Automated Failure Attribution)」的新研究方向,目标是让 AI 自动回答:是谁、在哪一步导致了失败。 该工作由 Penn State、Duke、UW、Goolge DeepMind 等机构的多位研究人员合作完成。 论文标题:Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems 背景挑战 LLM 驱动的多智能体系统在诸多领域展现出巨大潜力,从自动化助手协同办公到多 Agent 合作完成 Web 复杂操作等。然而,这些系统 脆弱性 也逐渐显现:多个 Agent 之间的误解、信息传递错误或决策不当,都可能导致 ...
谷歌之后,英伟达入局扩散大语言模型,Fast-dLLM推理速度狂飙27.6倍
机器之心· 2025-05-30 03:28
Core Viewpoint - The article discusses the breakthrough in inference speed for diffusion models through the introduction of Fast-dLLM, which utilizes a training-free acceleration approach, enhancing the practical application of large language models (LLMs) [2][20]. Group 1: Core Technology - Fast-dLLM employs a Block-Wise KV Cache mechanism, achieving over 90% activation reuse, significantly improving computational efficiency for long sequence inference [6][12]. - The Confidence-Aware Parallel Decoding method allows for parallel decoding while maintaining token dependency, ensuring coherent generation by filtering tokens based on confidence levels [9][13]. - The dual cache strategy enables simultaneous caching of prefix and suffix attention activations, reducing redundant calculations and enhancing performance [12]. Group 2: Performance Breakthrough - Fast-dLLM achieves a 27.6 times end-to-end acceleration for long text generation tasks, reducing single-step latency from 0.26 seconds to 0.09 seconds, and overall time from 266 seconds to 12 seconds [18]. - The accuracy loss in mainstream benchmark tests is kept under 2%, demonstrating the model's effectiveness in maintaining quality while improving speed [19]. Group 3: Application Value - Fast-dLLM's zero-training cost feature makes it an ideal tool for inference optimization, allowing for quick integration into existing systems without altering model architecture or training processes [20]. - The model shows compatibility with various existing models like LLaDA and Dream, achieving significant throughput improvements while maintaining competitive accuracy [21]. Group 4: Summary and Outlook - Fast-dLLM represents a significant advancement in inference efficiency for diffusion models while ensuring stable generation quality, paving the way for broader applications in real-time interaction and long text generation [23].
用Veo 3+Suno做了个AI Rapper,吊打音乐节上的流量明星
机器之心· 2025-05-29 11:38
Core Viewpoint - The article discusses the advancements in AI-generated music and video content, highlighting the capabilities of tools like Google Flow Veo3 and Suno 4.5 in creating realistic performances that challenge traditional music production methods [1][2][3]. Group 1: AI Music Generation - The AI model Suno has evolved significantly, now at version 4.5, and is referred to as the "ChatGPT of the music industry" [12]. - A notable example of AI music generation is the work of a blogger who created songs combining Cantonese lyrics with traditional poetry and rock elements, achieving over a million plays on various platforms [10]. - The article compares two AI tools: Suno, which specializes in music generation but has some limitations in naturalness, and Doubao, which offers a broader range of functionalities including clearer pronunciation of complex words [16][17]. Group 2: AI Video Generation - Google Flow is introduced as a comprehensive AI film production platform that allows users to create complete scenes or short films based on text prompts or images [20]. - The article emphasizes the importance of prompt engineering in generating high-quality video content, showcasing a detailed prompt for creating a hip-hop concert scene [22]. - By using Flow, users can create seamless and engaging concert videos by extending short clips and combining them with music, demonstrating the potential for AI to revolutionize video production in the music industry [25][27].
Linear-MoE:线性注意力遇上混合专家的开源实践
机器之心· 2025-05-29 11:38
Core Insights - The article highlights the rise of Linear-MoE architecture, which effectively combines linear sequence modeling and Mixture-of-Experts (MoE) for enhanced performance in large language models [1][10]. Group 1: Linear Sequence Modeling - Significant advancements in linear sequence modeling have been achieved over the past two years, characterized by linear time complexity in training and constant memory usage during inference [5]. - The main categories of linear sequence modeling include Linear Attention, State Space Models (SSM), and Linear RNN, with notable works such as Lightning Attention, GLA, Mamba2, and RWKV [5]. Group 2: Mixture-of-Experts (MoE) - MoE has become a standard in the industry, with various models like GPT-4, Gemini, and domestic models such as DeepSeek and Qwen all adopting MoE architectures [8]. - The importance of MoE in enhancing model capabilities is emphasized, although the article does not delve deeply into this aspect [8]. Group 3: Linear-MoE Architecture - Linear-MoE offers a complete system from modeling to training, allowing flexible combinations of linear sequence modeling layers and MoE layers, while also being compatible with traditional Softmax Attention Transformer layers [10]. - Key features include a modular architecture with support for various linear modeling methods and multiple MoE implementations, ensuring stability and scalability through the Megatron-Core framework [10]. Group 4: Performance and Future Prospects - Large-scale experiments validate the superiority of Linear-MoE, demonstrating faster inference speeds (2-5 times quicker than traditional architectures) and over 50% reduction in memory usage [12][13]. - The open-source nature of Linear-MoE fills a technical gap and provides reproducible training solutions, with future exploration planned for applications in long-context understanding and Vision-Language model architectures [13].
135 个项目、七大趋势、三大赛道:撕开大模型开源生态真相,你会怎么卷?
机器之心· 2025-05-29 07:10
Core Viewpoint - The article emphasizes the importance of understanding trends in the rapidly evolving AI landscape, particularly in the context of open-source projects and their development trajectories [2][6]. Group 1: Overview of Open-Source Landscape - Ant Group's open-source team released a comprehensive "2025 Large Model Open-Source Development Ecosystem Panorama," detailing 135 core projects across 19 technical domains, highlighting the significant role of open-source in the large model wave [2][6]. - The three dominant technical tracks identified are model training frameworks, efficient inference engines, and low-code application development frameworks [2][6]. Group 2: Project Rankings and Trends - The top 20 projects in the 2025 OpenRank ranking include notable names like PyTorch, vLLM, and Dify, showcasing their community engagement and technical impact [3][6]. - A comparison of OpenRank indicators from 2024 shows significant year-on-year growth in the three leading technical tracks, indicating a shift in focus towards more practical applications [6][14]. Group 3: Market Dynamics and Project Viability - The article discusses the "hackathon phenomenon," where many projects gain rapid attention but also face high turnover rates, leading to a challenging environment for sustainability [8][10]. - AI coding projects are thriving, with OpenRank trends showing consistent upward movement, contrasting with the decline of AI search projects [11][26]. Group 4: Future Trends and Predictions - Seven key trends have emerged from tracking the activity and community feedback of 135 core projects, with a notable shift towards low-code platforms and user-centric applications [17][20]. - The article predicts that by 2025, low-code platforms will dominate, reflecting a transition from developer-focused tools to more accessible solutions for end-users [21][26]. Group 5: Technical Innovations and Challenges - The article highlights the advancements in model training and inference, particularly the emergence of tools like vLLM and SGLang, which are reshaping the deployment landscape [34][36]. - It also points out the ongoing need for new protocols to facilitate agent collaboration, indicating a significant area for future innovation within the open-source community [25][26].
中国团队让AI拥有「视觉想象力」,像人类一样脑补画面来思考
机器之心· 2025-05-29 07:10
在人类的认知过程中,视觉思维(Visual Thinking)扮演着不可替代的核心角色,这一现象贯穿于各个专业领域和日常生活的方方面面。 生物化学家在探索新的治疗途径时,会在脑海中构建蛋白质的三维立体结构,通过视觉化的分子间相互作用来理解复杂的生化过程;法医分析师在破解疑难 案件时,需要在心中重建犯罪现场的空间布局,通过视觉推理来验证证据之间的逻辑连接;建筑师在设计创新建筑时,会在脑海中不断勾勒和修正建筑草 图,通过视觉想象来优化空间配置和光照效果;篮球运动员在制定战术策略时,需要在脑海中构想队友的跑位路线、防守阵型的变化以及关键时刻的战术配 合,通过视觉化的场景想象来设计最佳的进攻方案;在日常决策中, 一般人也会通过「脑补」各种可能的场景图像来辅助判断和选择,用脑海中自发生成 的图像作为认知媒介。 这种视觉思维能力的独特之处在于,它能够创造概念间的独特组合和新颖连接,帮助我们发现仅通过纯文本推理无法获得的洞察和创意。而在现代认知科学 中,这种「深思熟虑」往往需要多模态的思维过程来支撑。 如今,AI 也迈出了这一步:上海交通大学、上海创智学院、复旦大学和 Generative AI Research Lab( ...
RSS 2025|从说明书学习复杂机器人操作任务:NUS邵林团队提出全新机器人装配技能学习框架Manual2Skill
机器之心· 2025-05-29 04:53
Core Viewpoint - The article discusses the development of Manual2Skill, an innovative framework that utilizes Vision-Language Models (VLMs) to enable robots to autonomously understand and execute complex furniture assembly tasks based on visual manuals, bridging the gap between abstract instructions and physical execution [3][35]. Summary by Sections Research Background - Furniture assembly is a complex long-term task requiring robots to understand part relationships, estimate poses, and generate feasible actions. Existing methods often rely on imitation or reinforcement learning, which require large datasets and computational resources, limiting their applicability in real-world scenarios [6][35]. Manual2Skill Framework - Manual2Skill consists of three core phases: 1. **Hierarchical Assembly Diagram Generation**: Converts human-readable manuals into executable task plans using VLMs to generate a hierarchical assembly diagram that encodes the relationships between furniture parts [10][14]. 2. **Step-by-Step Pose Estimation**: Predicts the 6D poses of all parts involved in each assembly step, allowing for precise physical alignment. This method improves learning of basic connection methods across different furniture shapes [12][13]. 3. **Robot Assembly Action Generation and Execution**: Translates predicted poses into real-world robot actions, employing heuristic grasping strategies and robust motion planning algorithms for part manipulation [18][35]. Experimental Results and Analysis - The framework was tested on various IKEA furniture in both simulation and real environments, demonstrating robustness and effectiveness. The hierarchical assembly diagram generation method showed superior performance compared to baseline methods, especially for simple to medium complexity furniture [20][29][35]. Conclusion and Outlook - Manual2Skill represents a new paradigm in robotic learning, allowing robots to learn complex operational skills from human-designed manuals, significantly reducing the cost and complexity of skill acquisition. The framework captures the underlying structure and logic of operations, enabling effective generalization across different configurations and conditions [35].