Core Insights - The article discusses the evolution of Agentic AI from pipeline-based systems to model-native paradigms, emphasizing the internalization of reasoning, memory, and action capabilities within the models themselves [1][44]. - It highlights the role of reinforcement learning (RL) as a driving force in transforming static models into adaptive, goal-oriented entities capable of learning from interactions with their environment [1][44]. Background - The rapid advancement of generative AI has primarily focused on reactive outputs, lacking long-term reasoning and environmental interaction. The shift towards Agentic AI emphasizes three core capabilities: planning, tool usage, and memory [3]. - Early systems relied on pipeline paradigms where these capabilities were externally orchestrated, leading to passive models that struggled in unexpected scenarios. The new model-native paradigm integrates these capabilities directly into the model parameters, allowing for proactive decision-making [3][6]. Reinforcement Learning for LLMs - The scarcity of programmatic data and vulnerability to out-of-distribution scenarios necessitate the use of result-driven RL to internalize planning and other capabilities, moving away from prompt-induced behaviors [6][7]. - RL offers advantages over supervised fine-tuning (SFT) by enabling dynamic exploration and relative value learning, transforming models from passive imitators to active explorers [8][9]. Unified Paradigm and Algorithm Evolution - Early RLHF methods excelled in single-turn alignment but struggled with long-term, multi-turn, and sparse rewards. Newer result-driven RL methods like GRPO and DAPO enhance training stability and efficiency [12]. - The evolution of algorithms involves leveraging foundational models to provide priors while refining capabilities through interaction and rewards in task environments [12]. Core Capabilities: Planning - The pipeline paradigm views planning as automated reasoning and action sequence search, which is limited in flexibility and stability under complex tasks [14][15]. - The model-native paradigm integrates planning capabilities directly into model parameters, enhancing flexibility and robustness in open environments [15][18]. Core Capabilities: Tool Usage - Early systems embedded models in fixed nodes, lacking flexibility. The model-native transition internalizes decision-making regarding tool usage, forming a multi-objective decision problem [21][22]. - Challenges remain in credit assignment and environmental noise, which can destabilize training. Modular training approaches aim to isolate execution noise and improve sample efficiency [22]. Core Capabilities: Memory - Memory capabilities have evolved from external modules to integral components of task execution, emphasizing action-oriented evidence governance [27][30]. - Short-term memory utilizes techniques like sliding windows and retrieval-augmented generation (RAG), while long-term memory focuses on external libraries and parameter-based internalization [30]. Future Directions - The trajectory of Agentic AI indicates a shift towards deeper integration between models and their environments, moving from systems designed to use intelligence to those that grow intelligence through experience and collaboration [44].