Langsmith Platform Updates - Langchain is launching new features for Langsmith, a platform for agent engineering, focusing on tracing, evaluation, and observability to improve agent reliability [1] - Langsmith introduces "Insights," a feature designed to automatically identify trends in user interactions and agent behavior from millions of daily traces, helping users understand how their agents are being used and where they are making mistakes [1] - Insights is inspired by Anthropic's work on understanding conversation topics, but adapted for Langsmith's broader range of agent payloads [5][6] Evaluation and Testing - Langsmith emphasizes the importance of methodical testing, including online evaluations, to move beyond simple "vibe testing" and add rigor to agent development [1][33] - Langsmith introduces "thread evals," which allow users to evaluate agent performance across entire user interactions or conversations, providing a more comprehensive view than single-turn evaluations [16][17] - Online evals measure agent performance in real-time using production data, complementing offline evals that are based on known examples [24] - The company argues against the idea that offline evals are obsolete, highlighting their continued usefulness for regression testing and ensuring agents perform well on known interaction types [30][31] Use Cases and Applications - Insights can help product managers understand which product features are most frequently used with an agent, informing product roadmap prioritization [2][12] - Insights can assist AI engineers in identifying and categorizing agent failure modes, such as incorrect tool usage or errors, enabling targeted improvements [3][13] - Thread evals are particularly useful for evaluating user sentiment across an entire conversation or tracking the trajectory of tool calls within a conversation [21] Future Development - Langsmith plans to introduce agent and thread-level metrics into its dashboards, providing greater visibility into agent performance and cost [26] - The company aims to enable more flows with automation rules over threads, such as spot-checking threads with negative user feedback [27]

Core Functionality - Deep agent CLI is introduced as an open-source coding tool built on the deep agents package, designed for writing, editing, and understanding code [1] - A key feature is its built-in memory, enabling it to learn and save memory profiles as different agents for access and portability across projects [1] - The CLI can be augmented with web search capabilities using a Tavilla API key [3] - It operates in manual accept mode for potentially dangerous actions like writing to files or running bash commands, with an option for auto-accept mode [4] Memory System - The CLI leverages a memory system where each agent has an associated memory that can be accessed and edited [9][10] - Agents can write to and update their memory, allowing them to accumulate context over time and reference it in the future [15][16] - Users can create specific agents with specialized knowledge and store their memory for later use in different contexts [10][11] - The memory is stored in a dedicated directory, allowing the agent to retain information across different sessions and locations [12][13] Usage and Application - Deep agent CLI can be used for both coding and non-coding tasks, allowing agents to evolve and learn alongside the user [16] - The tool is installed via `pip install deep agent CLI` and requires either an OpenAI or Anthropic API key [2][17] - The CLI allows users to interact with the agent, ask questions, and instruct it to perform tasks such as adding content to a readme file [6][7]

Product Overview - Langchain introduces a no-code agent builder, aiming to empower non-technical users to create agents easily [2][4] - The agent builder is built upon the "deep agents" architecture, simplifying agent creation to a configuration of tools and prompts [5][11] - The platform supports both chat-based interaction and autonomous background operation via triggers [27] Key Features and Technologies - Deep agents architecture utilizes sub-agents for handling long-running or context-intensive tasks, improving efficiency [5][35] - The platform incorporates a natural language interface for agent creation, abstracting away the complexities of prompt engineering [14][50] - Human-in-the-loop controls, such as interrupts, allow users to review and approve actions before execution, balancing autonomy with oversight [39][40] User Experience and Iteration - The platform provides a chat interface for testing and iterating on agents, allowing users to understand agent behavior and refine instructions [17][18] - An agent inbox facilitates the management of agent conversations and interrupted actions, mirroring a familiar email experience [41][42] - The platform allows users to iterate on agents by updating the agent over time [17] Integration and Deployment - Agents built in the agent builder are compatible with Langraph, enabling seamless transition to production deployments [45] - The platform currently hosts deep agents in the cloud, with plans to allow users to bring their own deep agents and graph architectures [46][47] Future Development and Feedback - Langchain seeks user feedback on optimizing agent improvement workflows, exploring various methods such as chatbot agents, canvas experiences, and thumbs up/down feedback [56][57] - The company is interested in user input on desired tools and triggers, as well as the experience for core platform teams to add new modules [55]

Product Overview - Langsmith Agent Builder is a new no-code agent experience that allows users to build and describe agents in natural language, incorporating memory for adaptation and learning [1] - The core of building an agent lies in crafting a detailed prompt, with Langsmith Agent Builder providing tools like metaprompting to assist in this process [2][3] - Agents can remember interactions and update prompts in a memory bank, enabling continuous learning and adaptation [4] Key Features - Metaprompting helps users refine their initial ideas into detailed prompts through follow-up questions [3][6] - Agents can connect to internal tools or APIs with proper Oauth and credentials [8] - A test chat feature allows users to debug agents by pausing execution before tools are used [10] - Memory enables agents to update instructions based on interactions, allowing them to learn and adapt over time [11][12][14] - The agent inbox allows users to monitor agents, receive notifications, and provide assistance when needed [18] Agent Management - Users can monitor agent threads, categorized as idle, busy, interrupted, or errored, to manage agent performance [16] - The agent builder is not a visual workflow builder but focuses on agent autonomy and memory [19] Example Use Case - A daily briefer agent can be created to read a user's calendar and send a report via Slack at a specified time [5][7] - The agent can be instructed to perform specific tasks, such as writing a poem, and will update its memory to incorporate these instructions [11][12][13]

We’re releasing a new LangChain Academy course, LangGraph Essentials, where you can learn the basics of LangGraph in less than an hour. LangGraph is a low-level orchestration framework designed specifically for building AI agents. It provides a durable runtime for agents with graph-based execution.LangGraph allows you to create flexible, agentic workflows with its modular components. It allows you to control execution, manage state, allow for human intervention when needed, and scale reliably. LangGraph add ...

LangChain Essentials Course Highlights - LangChain releases a new LangChain Essentials course for learning the basics of LangChain in an hour [1] - The course focuses on building agents using the `create_agent` abstraction [2] - The pre-built agent utilizes a ReAct-style architecture for reasoning and acting with tools [3] Agent Architecture and Scalability - The agent is built on LangGraph to balance flexibility with pre-built abstraction benefits [4] - The agent is designed to be scalable, resilient to failures, and allows for human intervention [3] - The agent can dynamically select prompts and models, with optional middleware for customization [4] Course Content - The course covers features of the `create_agent` abstraction through building increasingly sophisticated agents [5] - The course utilizes LangChain building blocks including messages, tools, and models [5]

Hey, I'm Tanishi from Langchain. I'm excited to share a new feature that we're launching called multi-turn evaluations. These are used to run online evaluations over end to end user interactions.If you're already using evals and linksmith, these complement those and should be used when your evaluator needs the context of an entire thread or an entire user conversation rather than just the next message. So to walk through a quick example, let's say you have a chat app like a customer support agent. This is a ...

Langchain Evolution & Strategy - Langchain started as an open-source package and has evolved into Typescript packages, Langchain, and Langraph [1][2] - The industry focus has shifted from easy prototyping to production-ready solutions, leading to the launch of Langraph [7] - Langchain 1.0 is built on top of Langraph, combining ease of use with production-ready runtime [16] Langraph Features & Benefits - Langraph was launched to provide more controllability and customization for users transitioning to production [8][9] - Langraph includes utilities like durable execution environments, error recovery from checkpoints, and streaming capabilities [13][14] - Langraph allows for deterministic steps and workflows, making it suitable for complex applications [39] Langchain 1.0 & Create Agent Abstraction - Langchain 1.0 aims to be the easiest way to get started with generative AI, specifically building agents [17] - The create agent abstraction simplifies agent creation with a few lines of code, leveraging a battle-tested pattern [18][19] - Middleware allows developers to add custom logic at any point in the agent loop, enabling extensibility [23] Models & Content Blocks - Dynamic model middleware enables dynamic selection of models based on context, allowing builders to stay on the bleeding edge [27][29] - Content blocks are introduced as a standard representation for message content, addressing the issue of varying formats across model providers [31][32] Langchain vs Langraph - Langchain is recommended for getting started due to its ease of use, while Langraph is suitable for extremely custom workflows [36][37] - Langraph is ideal for workflows that require deterministic components and agentic components [37]

Heat. [Applause] [Music] Heat. Heat. [Applause] [Music] Heat. Heat.Heat. [Music]. ...

Hi there, this is Bogattor from Lang Chain and today I'm really excited to introduce the new insights agent in Langmith. Let's say you've just shipped your first agent to production. You're tracing it with Langmith and you're starting to see an uptick in traffic. You're excited and you're really curious to understand how end users are engaging with your agent.What questions are they asking. What tools is your agent using. What subpar responses is your agent returning.And more. To try and answer these questi ...