Workflow
Evaluations
icon
Search documents
AWS just made some MASSIVE Announcements (AgentCore)
Matthew Berman· 2025-12-04 01:21
This is the report out of MIT that absolutely shook the AI industry. It said 95% of AI pilots inside the enterprise fail. And this report went viral.It was everywhere just a couple months ago. Fast forward just a few months later and today AWS is hosting reinvent their annual massive conference and they just made two announcements to their agentic platform that really look to solve the issues that enterprises are having deploying agentic systems into production. Those two big problems how to trust AI and ho ...
Getting Started with LangSmith (3/6): Datasets & Evaluations
LangChain· 2025-06-25 01:05
Resources & Tools - Eli5 代码库位于 GitHub:https://github.com/xuro-langchain/eli5 [1] - LangSmith 提供免费试用:https://smith.langchain.com/ [1] - LangSmith 文档地址:https://docs.smith.langchain.com/ [1] LangChain Platform - LangSmith 平台详情:https://www.langchain.com/langsmith/?utm_medium=social&utm_source=youtube&utm_campaign=q2-2025_onboarding-videos_co [1]
No Code LangSmith Evaluations
LangChain· 2025-06-18 15:10
LangChain Agent Evaluation - LangChain 降低了 Agent 评估的门槛,使得非开发者也能轻松进行 [1] - Langraph Studio 新增了快速评估 Langraph Agent 的功能 [3] - 用户可以在 Langraph Studio 中选择数据集并启动评估实验 [3][4] - 评估结果可在 Langsmith 中查看,包括模型输出和评估分数 [5] Evaluation Importance and Accessibility - 评估对于构建有效的 Agent 至关重要 [7] - 传统评估对开发者有较高要求,需要掌握 SDK、Piest 和 Evaluate API 等 [7] - LangChain 旨在提供一种无需代码的方式,让任何人都能评估 Langraph Agent [8] - 非技术用户可以基于直觉评估模型选择和提示词等 [9] Configuration and Customization - 用户可以在 Studio 界面中轻松切换 graph 配置,并以此为基础启动评估 [9] - 开发者可以预先设置包含输入主题和参考输出的数据集 [10] - 可以将评估器(Evaluator)绑定到数据集,并自定义评估标准和评分规则 [11][12][13] - 用户可以在 Studio 中修改 graph 配置(如模型、提示词),并启动新的评估实验 [15][16][17] - Studio 提供了无代码配置方式,方便快速迭代 [18]