AI也需要"记笔记"：Karpathy从Claude 1.6万字提示词中看到的未来

Core Viewpoint - The article discusses the significance of system prompts in large language models (LLMs), particularly focusing on Claude's extensive system prompt and the potential for a new learning paradigm termed "system prompt learning" proposed by Karpathy [6][12]. Group 1: System Prompts Overview - Claude's system prompt consists of 16,739 words, significantly longer than OpenAI's ChatGPT o4-mini, which has only 2,218 words, representing just 13% of Claude's prompt [2][3]. - System prompts serve as an initial instruction manual for LLMs, guiding their roles, rules, and response styles [4]. - The content of Claude's system prompt includes tool definitions, user preferences, and guidelines for various tasks, indicating a structured approach to AI interactions [8]. Group 2: Current Learning Paradigms - The existing learning paradigms for LLMs include pretraining, which provides broad knowledge through large datasets, and finetuning, which adjusts model behavior through parameter updates [9]. - Unlike LLMs, humans often learn by summarizing experiences and strategies, akin to "note-taking," rather than solely relying on parameter updates [10]. Group 3: System Prompt Learning - Karpathy suggests that LLMs should adopt a "system prompt learning" mechanism, allowing them to store strategies and knowledge in an explicit format, enhancing efficiency and scalability [10][12]. - This new learning paradigm could lead to more effective data utilization and improved generalization capabilities for LLMs [19]. Group 4: Practical Implications - Clear and detailed instructions in system prompts lead to more accurate AI responses, emphasizing the importance of structured communication [13][14]. - The article highlights that "prompt engineering" is an extension of everyday communication skills, making it accessible for ordinary users [16].