How BlackRock Built Production AI Agents for Global Asset Management

Company Overview & Strategy - BlackRock manages over $11 trillion in assets, aiming to enhance financial well-being through its Aladdin platform [3] - Aladdin is a proprietary technology platform unifying the investment management process for both public and private markets, serving institutional and retail investors [3] - BlackRock aims to increase productivity, drive alpha generation, and personalize user experience through AI, particularly with the Aladdin Copilot initiative [5][6] Aladdin Copilot & Architecture - Aladdin Copilot is embedded across 100 front-end applications, proactively surfacing relevant content and enhancing productivity [6][7] - The architecture supporting Aladdin Copilot uses a plug-in registry, allowing 50-60 Aladdin engineering teams to integrate their functionalities [9][10] - The system uses Langchain for orchestration, incorporating input and output guardrails for responsible AI moderation, including off-topic, toxic content, and PII handling [16][24] - GPT4 function calling is used for planning and action within the orchestration node, iterating until a final answer is obtained [19] Evaluation & Testing - BlackRock emphasizes evaluation-driven development, similar to test-driven development in traditional coding, to ensure the system's reliability [25] - The company tests every system prompt and intended behavior by generating synthetic data and building evaluation pipelines [26][27] - End-to-end testing capabilities are provided to developers, allowing them to configure testing scenarios, including application context, user settings, and multi-turn scenarios [30][31] - Ground truth data is collected for every plug-in to guarantee Aladdin Copilot's performance and routing in various scenarios [33]