Continual Learning in AI Agents - The industry recognizes the gap between AI agents and human learning capabilities, emphasizing the need for agents to continually learn and improve over time [1] - The industry is exploring different methods for AI systems to learn, including weight updates and learning in context using large language models (LLMs) [2] - Reflection over trajectories is emerging as a key theme, allowing agents to update memories, core instructions, and learn new skills [3][4][5] Skill Learning and Implementation - Skill learning involves reflecting over trajectories to learn skills, exemplified by the skill creator skill adapted from Anthropic [8][9] - Deep agent CLI allows specifying environment variables for logging traces, which is useful for reflection [10][11] - The industry is using Langsmith Fetch to grab recent threads from deep agents for reflection and persistent skill creation [12][13] - A practical example demonstrates how an agent can read a JSON file, reflect on its contents, and create a new deep agent skill, showcasing the utility of continual learning [15][16][17] Benefits and Future Directions - Skill learning enables agents to encapsulate standard operating procedures, such as grabbing Langsmith traces, for repeated use [19][20] - Continual learning loop involves agents reflecting on past trajectories to learn facts, memories, skills, and improve instructions [21][22]

Memory Updating Mechanisms for Agents - Explicit memory updating involves directly instructing the agent to remember specific information, similar to how cloud code functions [2][5][6][29] - Implicit memory updating occurs through the agent learning from natural interactions with users, revealing preferences without explicit instructions [7][19][29] Deep Agent CLI and Memory Management - Deep agents have a configuration home directory with an `agent MD` file that stores global memory, similar to Claude's `cloud MD` [3][4][6] - The `agent MD` files are automatically loaded into the system prompt of deep agents, ensuring consistent memory access [6] - Deep agent CLI allows adding information to global memory using natural language commands, updating the `agent MD` file [5] Implicit Memory Updating and Reflection - Agents can reflect on past interactions (sessions or trajectories) to generate higher-level insights and update their memory [8][9][10][28] - Reflection involves summarizing session logs (diaries) and using these summaries to refine and update the agent's memory [11][12] - Accessing session logs is crucial for implicit memory updating; Langsmith can be used to store and manage deep agent traces [13][14][15] Practical Implementation and Workflow - A utility can be used to programmatically access threads and traces from Langsmith projects [21] - The deep agent can be instructed to read interaction threads, identify user preferences, and update global memory accordingly [24][25] - Reflecting on historical threads allows the agent to distill implicit preferences and add them to its global memory, improving future interactions [26][27][28]

Hey, I'm Tanish from Langchain. Today I'm going to go through how to use Open Router's new broadcast feature with Langchain to send traces to Langmith. The cool thing about broadcast is it stores destination information, which in this case is Langmith server side.So the only thing that you need to worry about in your code is your open router API key. Let's go through how to set this up. So let me walk you through a quick code snippet.This uses lang chain's init chat model in order to initialize a model and ...

Evals are an important step for having confidence in the LLM application or agent that you're putting out into the real world is performing as you would expect it to and they're hard to do right. Eugene Yan is one of the people in the business who I think is the sharpest around evals and I saw this tweet from him over the weekend. After repeating myself for the nth time on how to build product evals, I figured I should write it down.It's just three basic steps. One, labeling a small data set. two, aligning ...

AI Workflow Automation & Observability - N8N is an AI workflow builder that allows users to string together nodes into AI agents and set up external triggers for automated execution [1] - Langsmith is an AI observability and evaluation product designed to monitor the performance of AI applications [2] Integration & Setup - Connecting N8N to Langsmith requires generating a Langsmith API key and setting it in the N8N deployment environment [3][8] - Additional environment variables can be set to enable tracing to Langsmith, specify the trace destination, and define the project name [4] Monitoring & Debugging - Langsmith traces provide visibility into the workflow, including requests to OpenAI, model usage, latency, and token consumption [6] - Langsmith offers a monitoring view to track app usage, latency spikes, error rates, and LLM usage/spending [7]

Agent Architecture & Functionality - The Langchain deep research agent is highly configurable and open source, allowing for customization to specific use cases [1] - The agent operates in three main phases: scoping the problem, research, and report writing [3] - The research phase utilizes a supervisor to delegate tasks to sub-agents for in-depth research on specific subtopics [4] - Sub-agents use a tool calling loop, which can be configured with default or custom tools (like MCP servers) for searching flights, hotels, etc [17][18] - A compression step is used by sub-agents to synthesize research findings into comprehensive mini-reports before returning to the supervisor, mitigating context window overload [21][23] - The supervisor analyzes findings from sub-agents to either complete research or continue with follow-up questions [25] - Final report generation is done in a single shot using all condensed research findings [5][27] Implementation & Configuration - The agent is built on Langraph and can be run locally by cloning the Open Deep Research repository [29] - Configuration involves setting API keys for models (default OpenAI) and search tools (default Tavily) [30] - Langraph Studio can be used for iterating and testing the agent with different configurations [32] - The agent is highly configurable, allowing users to choose between default or model provider native search tools, connect to MCP servers, and select models for different steps [33][34] Application & Output - The agent can be used for complex research tasks, such as planning a trip, by iteratively calling tools and searching the web [2] - The agent provides a final report with an overview, flight options, transportation options, accommodation options with booking links, a sample itinerary, and a list of sources [36] - Open Agent Platform provides a UI to configure and try out the research agent without cloning the code [37]

Agentic Platform Engineering Implementation - Outshift by Cisco is redefining platform engineering with agentic platform engineering, aiming to automate workflows and allow platform engineers to focus on high-value work [1][2] - The company built Jarvis, a genai-powered multi-agentic system, to automate tasks and improve efficiency [2] - Langraph, combined with Langsmith's observability tools, enables debugging agentic applications and improving reasoning capabilities at scale [6] Efficiency Improvement - Implementing Jarvis allows developers to self-service through genai-powered automation, eliminating manual toil [4] - CI/CD pipeline setup time reduced from a week to less than an hour [7] - Resource provisioning time (e.g., S3 buckets, EC2 instances, access keys) reduced from half a day to nearly instantaneous [7] - The company has eliminated unnecessary back and forth between developers and SREs [7] Workflow Transformation - The company shifted from traditional automation to agentic reasoning-based workflows by adopting Langraph [5] - Developers interact with Jarvis for platform-related questions and configurations, retrieving information autonomously [8] - The new system allows the company to handle a higher volume of requests with the same team while reducing burnout [8] Technology Evaluation - Langraph's tight integration with Langsmith, especially for debugging and evaluations, is a significant advantage [9] - The company found Langraph to be superior compared to other agentic solutions or custom-built alternatives [9]