Hi, I'm Robert. I'm the co-founder and CTO at Wordware. And at Wordware, I've personally helped hundreds of teams build reliable AI agents.I'm here to share a few of the insights that we got, especially when it comes to tools. Um, really agentic MCPs, giving your tools time to think. Before I worked on uh LLMs and agents, I used to work on self-driving cars, and really, you know, building high reliable systems is in my blood.So, uh, yeah, here we go. The promise of agents are automated systems that can take ...

Amazon Q Developer Overview - Amazon Q Developer is an AI coding assistant designed to integrate into various stages of the Software Development Life Cycle (SDLC) [2][5] - It supports multiple IDEs (VS Code, IntelliJ, Eclipse), CLI, and GitHub, aiming for a seamless experience across different developer preferences [5][15][22] - Users can start using Amazon Q Developer in the command line or IDE without requiring an AWS account [5] Key Features and Use Cases - Amazon Q Developer can assist in the planning phase by providing best practices and suggesting robust, production-ready approaches [30] - It can generate code, unit tests, and documentation, streamlining the development process [16][19] - The tool facilitates debugging by analyzing logs and identifying root causes of errors within the AWS console [26] - Amazon Q Developer supports feature development through context-aware feature building and bug fixing [18] - It offers security scans and monitoring within GitHub, enabling code review and automated fixes in pull requests [23] SDLC Integration - Amazon Q Developer aids in deploying applications to cloud services by generating SAM scripts [25] - It helps maintain and modernize applications by providing insights into application performance and potential issues [4][26] Best Practices and Considerations - The industry emphasizes the importance of pre-planning, including infrastructure considerations, to avoid production issues [29][30] - Prompt engineering is highlighted as a crucial skill for developers to effectively utilize generative AI and obtain desired outputs [31]

Hi, I'm Philip and I'm the CEO at Wordware. Today I want to talk to you about what sucks about chatbased interfaces, how documents can actually solve those issues and how do they lead to um background agents that do tasks for you in the or in the background. So firstly, let's start with what are the problems with chatbased systems.When I interact with um cla or open AI, it all seems very affirmal. um I end up often creating workflows for myself using projects or just copy pasting stuff and in that way when ...

Agent Evaluation Challenges - Building agents is difficult, requiring iteration at the prompt, model, and tool call definition levels [2][3] - Systematically tracking the performance of new prompts versus previous ones is challenging [4] - Including product managers or other team members in the iterative evaluation process is difficult [5] - Identifying bottlenecks in applications and pinpointing specific sub-agents or tool calls that create poor responses is hard [6] Evaluation Components - Agent evaluation should include evaluating at the tool call level, considering whether the right tool was called and if the correct arguments were passed [7][11] - Trajectory evaluation is important to determine if tool calls are executed in the correct order across a series of steps [7][20] - Multi-turn conversation evaluation is necessary to assess consistency in tone and context retention across multiple interactions [8][22][23] - Improving evaluation prompts is crucial, as the evals used to identify failure cases are essential for improving the agent [8][27] Arise Product Features - Arise offers a product for tracing and evaluating agent performance, allowing teams to ask questions about application performance and suggest improvements [12][13] - The product provides a high-level view of different paths an agent can take, helping to pinpoint performance bottlenecks [14][15] - Users can drill down into specific traces to evaluate tool call correctness and argument alignment [17][18]

Challenges & Initial State - ST Blitz faced near failure due to practically zero revenue and flat growth after 7 years of development and millions in investment [1] - Investors initially doubted browser-based technology and web containers, viewing them as a dead end [3] Turnaround & Growth - ST Blitz experienced explosive growth, evidenced by 20 million errors in just two months [1] - 18 months later, the company anticipates surpassing 100 million AR (likely Annual Recurring Revenue) for product B [2] - The success was not overnight, but the result of 7 years of development [2] AI Integration Strategy - Simply adding a chatbot interface is often ineffective due to user fatigue and underwhelming results [4] - Attempting to AI-power everything leads to direct competition with industry giants like OpenAI and Anthropic, which is difficult for smaller teams with limited resources [5] - Waiting for perfect AI is not practical; companies must build tooling iteratively [6] - The biggest mistake is neglecting user feedback; companies should prioritize user interaction to avoid developing unnecessary features [6][7] Keys to Success - Identify the company's unique competitive advantage, not just product features, such as domain expertise, data distribution, or infrastructure [7][8] - Determine how AI can amplify this competitive advantage, focusing on what becomes possible when AI meets the company's unique capabilities [8][9] - Create a new category by reinventing the user flow, rather than fitting into existing workflows [10] - ST Blitz's secret was not unique AI (as everyone has access), but their unique competitive advantage [10] - Reinventing the user flow and betting everything on it was crucial for ST Blitz [11]