Transforming search and discovery using LLMs — Tejaswi & Vinesh, Instacart

Search & Discovery Challenges in Grocery E-commerce - Instacart faces challenges with overly broad queries (e.g., "snacks") and very specific, infrequent queries (e.g., "unsweetened plant-based yogurt") due to limited engagement data [6][7] - Instacart aims to improve new item discovery, similar to the experience of browsing a grocery store aisle, but struggles due to lack of engagement data [8][9][10] - Existing models improve recall, but maintaining precision, especially in the long tail of queries, remains a challenge [8] LLM-Powered Query Understanding - Instacart utilizes LLMs to enhance query understanding, specifically focusing on query to category classification and query rewrites [10][11][12] - For query to category classification, LLMs, when augmented with top converting categories as context, significantly improved precision by 18 percentage points and recall by 70 percentage points for tail queries [13][21] - For query rewrites, LLMs generate precise rewrites (substitute, broader, synonymous), leading to a large drop in queries with no results [23][24][25][26] - Instacart pre-computes outputs for head and torso queries and caches them to minimize latency, while using existing or distilled models for the long tail [27][28] LLM-Driven Discovery-Oriented Content - Instacart uses LLMs to generate complementary and substitute items in search results, enhancing product discovery and user engagement [31][34] - Augmenting LLM prompts with Instacart's domain knowledge (e.g., top converting categories, query annotations, subsequent user queries) significantly improves the relevance and effectiveness of generated content [39][40][41] - Instacart serves discovery-oriented content by pre-computing and storing content metadata and product recommendations, enabling fast retrieval [42][43] Key Takeaways & Future Directions - Combining LLMs with Instacart's domain knowledge is crucial for achieving topline wins [47] - Evaluating content and query predictions is more important and difficult than initially anticipated [47][48] - Consolidating multiple query understanding models into a single LLM or SLM can improve consistency and simplify system management [28]