Challenges in Building AI Applications at BlackRock - BlackRock faces challenges in prompt engineering, requiring significant time investment from domain experts to iterate, version, and compare prompts effectively [10] - BlackRock encounters difficulties in selecting appropriate LLM strategies (e.g., RAG, chain-of-thought) due to instrument complexity and document size variations, impacting data extraction [11] - BlackRock experiences deployment challenges, including determining suitable cluster types (GPU-based inference vs burstable) and managing cost controls for AI applications [12][14] BlackRock's Solution: Sandbox and App Factory - BlackRock developed a framework with a "sandbox" for domain experts to build and refine extraction templates, accelerating the app development process [15][17] - BlackRock's "sandbox" provides greater configuration capabilities beyond prompt engineering, including QC checks, validations, constraints, and interfield dependencies [19][20] - BlackRock's "app factory" is a cloud-native operator that takes a definition from the sandbox and spins out an app, streamlining deployment [15] Key Takeaways for Building AI Apps at Scale - BlackRock emphasizes investing heavily in prompt engineering skills for domain experts, particularly in the financial space, due to the complexity of financial documents [26] - BlackRock highlights the importance of educating the firm on LLM strategies and how to choose the right approach for specific use cases [27] - BlackRock stresses the need to evaluate the ROI of AI app development versus off-the-shelf products, considering the potential cost [27] - BlackRock underscores the importance of human-in-the-loop design, especially in regulated environments, to ensure compliance and accuracy [28]