让 Anthropic 破防的「蒸馏」风波，美国 AI 大牛泼冷水：中国 AI 成功不靠走捷径

Core Viewpoint - Anthropic has accused three Chinese AI labs of "distilling" its Claude model, sparking widespread discussion. Nathan Lambert, a prominent researcher in RLHF, suggests that the situation is not as severe as perceived but is also not straightforward [1][22][23]. Summary by Sections Distillation and Accusations - Distillation refers to the process where a weaker model learns from the outputs of a stronger model to quickly gain similar capabilities. Anthropic claims that the three companies used approximately 24,000 fake accounts to generate over 16 million dialogues with Claude, violating service terms and regional access restrictions [4][25]. - Anthropic's infrastructure, termed the "Hydra cluster," allegedly managed thousands of accounts to obscure detection algorithms, allowing for the mixing of distillation traffic with regular user requests [5][26]. Differentiation Among Companies - Lambert emphasizes the need to differentiate between the three companies, as their actions and motivations vary significantly. DeepSeek's distillation efforts are minimal, with only 150,000 interactions, focusing on producing chain-of-thought training data rather than direct answers [6][27]. - In contrast, Moonshot and MiniMax have significantly higher interaction volumes, with 3.4 million and 13 million respectively, targeting advanced capabilities like agentic behavior and tool usage [8][28]. Limitations of Distillation - Lambert raises concerns about the limitations of distillation, questioning its effectiveness in achieving top-tier model capabilities. He argues that while distillation can mimic outputs, true model strength relies on reinforcement learning (RL), which involves exploration and self-generated solutions [29][40]. - The differences in data distribution between models can lead to ineffective or even detrimental results when directly feeding outputs from one model to another, indicating that distillation requires substantial engineering efforts to be effective [31][41]. Anthropic's Position and Double Standards - Lambert suggests that Anthropic's public naming of the Chinese companies may not primarily stem from technical defense motives but rather from geopolitical pressures, as the U.S. Department of Defense recently threatened Anthropic regarding its operational permissions [33]. - The article highlights a perceived double standard, noting that Anthropic has engaged in distillation practices itself, including controversial methods to gather training data, raising questions about its credibility in accusing others [34][39]. Conclusion on Distillation's Role - While distillation is acknowledged as a useful technique, Lambert asserts that it is not as powerful as many believe. The true innovation in AI development relies on reinforcement learning rather than distillation alone, and achieving top-tier performance requires more than just shortcuts [40][41].