All right, I need everybody to take a deep breath here because um I'm about to stress you out. And right, this is just one little microcosm of the entire tech industry, but if you look around at all the different categories of software right now, the same exact thing is happening. And I haven't even mentioned the horde of startups, wellunded startups, uh that are getting funded in every single one of these spaces as well.And among all of this chaos, we have companies that are essentially collapsing in month ...

Generative Media Platform & Market Overview - File.ai 将自身定义为一个生成式媒体平台，专注于视频、音频和图像的生成 [1] - 生成式媒体正在改变社交媒体、广告、营销、时尚、电影、游戏和电子商务等行业，最终将影响所有内容 [10] - 广告行业预计将成为首批大规模受到生成式媒体影响的行业之一，行业规模预计将会扩大 [13] AI Model Development & Trends - 边缘计算的创作边际成本正在接近于零，但故事叙述和创造力仍然至关重要 [8][9] - 视频模型的使用率正在快速增长，从10月初的几乎为零增长到2月份的18%，并且持续增长，目前约为30% [25][26] - 视频模型预计将比图像生成市场大 100 到 250 倍，因为视频模型计算密集程度是图像的 20 倍，互动性是图像的 5 倍，并且将影响更多行业 [27] - 视频生成技术将朝着更快、更便宜的方向发展，最终实现实时视频生成，这将对用户互动方式产生重大影响，模糊游戏和电影之间的界限 [31] - 图像模型也在不断改进，例如 Flux context 和 GPT4o 引入了新的编辑功能和更好的文本渲染功能，为行业开辟了更多用例 [34] Applications of Generative Media - 个性化广告是生成式媒体的一个重要应用方向，可以针对不同的人口统计群体快速生成大量不同版本的广告，或者根据用户的浏览行为动态生成广告 [15] - 电子商务是生成式媒体的另一个重要应用领域，特别是虚拟试穿技术，许多零售商和初创公司都在采用这项技术 [21][22] - AI 正在帮助创建互动和个性化的体验，例如 A24 电影《内战》的互动广告活动，用户可以将自己的自拍照放在时代广场的玩具士兵上 [18][19]

Search & Discovery Challenges in Grocery E-commerce - Instacart faces challenges with overly broad queries (e.g., "snacks") and very specific, infrequent queries (e.g., "unsweetened plant-based yogurt") due to limited engagement data [6][7] - Instacart aims to improve new item discovery, similar to the experience of browsing a grocery store aisle, but struggles due to lack of engagement data [8][9][10] - Existing models improve recall, but maintaining precision, especially in the long tail of queries, remains a challenge [8] LLM-Powered Query Understanding - Instacart utilizes LLMs to enhance query understanding, specifically focusing on query to category classification and query rewrites [10][11][12] - For query to category classification, LLMs, when augmented with top converting categories as context, significantly improved precision by 18 percentage points and recall by 70 percentage points for tail queries [13][21] - For query rewrites, LLMs generate precise rewrites (substitute, broader, synonymous), leading to a large drop in queries with no results [23][24][25][26] - Instacart pre-computes outputs for head and torso queries and caches them to minimize latency, while using existing or distilled models for the long tail [27][28] LLM-Driven Discovery-Oriented Content - Instacart uses LLMs to generate complementary and substitute items in search results, enhancing product discovery and user engagement [31][34] - Augmenting LLM prompts with Instacart's domain knowledge (e.g., top converting categories, query annotations, subsequent user queries) significantly improves the relevance and effectiveness of generated content [39][40][41] - Instacart serves discovery-oriented content by pre-computing and storing content metadata and product recommendations, enabling fast retrieval [42][43] Key Takeaways & Future Directions - Combining LLMs with Instacart's domain knowledge is crucial for achieving topline wins [47] - Evaluating content and query predictions is more important and difficult than initially anticipated [47][48] - Consolidating multiple query understanding models into a single LLM or SLM can improve consistency and simplify system management [28]

Foundation Model Strategy - Netflix is leveraging foundation models for personalized recommendations [1] - The strategy is based on work by Yesu Feng, a staff research scientist/engineer at Netflix, focused on generative foundation models [1] - Prior to Netflix, Feng worked on feed and marketplace optimization at LinkedIn and Uber, respectively [1] Industry Focus - The application of foundation models aims to improve personalized recommendations [1] - The discussion took place at the AI Engineer World's Fair in San Francisco [1]

Model Building and Training - LinkedIn leverages large language models (LLMs) for personalization and ranking tasks, aiming to use one model for all tasks [2][3] - The process involves converting user information into prompts, a method called "promptification" [8] - LinkedIn builds a large foundation model, Blue XL, with 150 billion parameters, then distills it to smaller, more efficient models like a 3B model for production [12] - Distillation from a large model is more effective than training a small model from scratch [14] - Increasing data, model size (up to 8x22B), and context length can improve model performance, but longer contexts may require model adjustments [17][18][19] Model Performance and Generalization - The model improves performance for cold start users, showing a growing gap compared to production models as interactions decrease [21] - The model demonstrates generalization to new domains, performing on par with or better than task-specific production models in out-of-domain tasks [23] Model Serving and Optimization - LinkedIn focuses on model specification, pruning, and quantization to improve throughput and reduce latency for production [26] - Gradual pruning and distillation are more effective than aggressive pruning, minimizing information loss [29][30] - Mixed precision, including FP8 for activations and model parameters but FP32 for the LM head, is crucial for maintaining prediction precision [31][32] - Sparsifying attention scores can reduce latency by allowing multiple item recommendations without each item attending to each other [34][35] - LinkedIn achieved a 7x reduction in latency and a 30x increase in throughput per GPU through these optimization techniques [36]

[Music] Yeah. Hi everyone. Um, thanks for joining the talk today.Um, we're super excited to be here and shares some of the learnings we um, we have from integrating the LM into Pinterest search. My name is Khan and today I'll be presenting with Mukunda and we are both machine learning engineers from search relevance team at Pinterest. So start with a brief introduction to Pinterest.Um Pinterest is a visual discovery platform where piners can come to find inspiration to create a life they love. And there are ...

Large Language Models Evolution - Scaling laws 表明，增加计算量、数据和参数可以提高 Transformer 模型的性能，并推广到其他领域 [2][3] - 随着模型规模的扩大，性能持续提高，并在中等数学难题的解决率上有所体现，尤其是在提示模型展示思维链时 [5][7] - 通过强化学习和人类反馈，模型能够更好地遵循指令，从而实现聊天机器人等应用 [10][11] Inference Time Optimization - 通过生成多个响应并进行多数投票（自洽性），可以在推理时提高性能 [15] - 顺序修改之前的响应，特别是在可以验证答案的领域（如数学和编程），可以显著提高性能 [16][17] - 在可以验证答案的领域，推理时间计算的扩展可以转化为智能 [19] Reinforcement Learning for Autonomous Coding - 强化学习是下一个扩展前沿，特别是在可以自动验证输出的领域 [24] - 经验时代将通过强化学习构建超级智能系统，尤其是在具有自动验证的领域 [25] - 自动编码是一个扩展强化学习的绝佳领域，因为它具有验证输出的能力 [30][31] Challenges in Scaling Reinforcement Learning - 扩展强化学习比扩展 LLM 更具挑战性，因为它需要多个模型副本以及训练和推理循环 [29] - 在强化学习中，奖励模型的奖励函数设计是一个挑战 [29][30] Reflection's Mission - Reflection 致力于构建超级智能，并以自主编码作为根本问题 [33] - Reflection 团队由在 LLM 和强化学习领域有开创性工作的 35 位先驱组成 [33]

Industry Trend - Recommendation and search systems have been significantly impacted by advances in language modeling, evolving from Word2vec to GRUs, Transformers, and BERT [1] - The emergence of large language models (LLMs) is driving innovation in model architecture, scalable system designs, and customer experiences within recommendation and search systems [1] - The industry is exploring real-world implementations and measurable outcomes of LLMs in recommendation and search systems [1] Technological Advancement - LLM-driven techniques are expected to shape the future of content discovery and intelligent search [1] - Amazon is building recommendation systems and AI-powered products using ML/AI [1]

uh it feels tangible and real but it's sort of underelling the job that each of you does. Code is sort of 10 to 20% of the value that you bring. The other 80 to 90% is in structured communication and this is going to be different for everyone but a process typically looks something like you talk to users in order to understand their challenges.You distill these stories down and then ideulate about how to solve these problems. What what is the goal that you want to achieve. You plan ways to achieve those goa ...

Core Concept - The industry emphasizes that specifications are a universal concept applicable across various fields, including programming, product management, and lawmaking [1] - The industry views prompt engineering as a form of specification writing, aligning AI models with intentions and values [1] Benefits of Specifications - Specifications enable faster and safer product development and deployment [2] - Specifications allow for broader contributions from various roles, blurring the lines between traditional roles like PM, lawmaker, engineer, marketer, and programmer [2]