学习

Search documents
盘一盘,2017年Transformer之后,LLM领域的重要论文
机器之心· 2025-06-29 04:23
Core Insights - The article discusses Andrej Karpathy's concept of "Software 3.0," where natural language becomes the new programming interface, and AI models execute specific tasks [1][2]. - It emphasizes the transformative impact of this shift on developers, users, and software design paradigms, indicating a new computational framework is being constructed [2]. Development of LLMs - The evolution of Large Language Models (LLMs) has accelerated since the introduction of the Transformer architecture in 2017, leading to significant advancements in the GPT series and multimodal capabilities [3][5]. - Key foundational papers that established today's AI capabilities are reviewed, highlighting the transition from traditional programming to natural language interaction [5][6]. Foundational Theories - The paper "Attention Is All You Need" (2017) introduced the Transformer architecture, which relies solely on self-attention mechanisms, revolutionizing natural language processing and computer vision [10][11]. - "Language Models are Few-Shot Learners" (2020) demonstrated the capabilities of GPT-3, establishing the "large model + large data" scaling law as a pathway to more general artificial intelligence [13][18]. - "Deep Reinforcement Learning from Human Preferences" (2017) laid the groundwork for reinforcement learning from human feedback (RLHF), crucial for aligning AI outputs with human values [15][18]. Milestone Breakthroughs - The "GPT-4 Technical Report" (2023) details a large-scale, multimodal language model that exhibits human-level performance across various benchmarks, emphasizing the importance of AI safety and alignment [26][27]. - The release of LLaMA models (2023) demonstrated that smaller models trained on extensive datasets could outperform larger models, promoting a new approach to model efficiency [27][30]. Emerging Techniques - The "Chain-of-Thought Prompting" technique enhances reasoning in LLMs by guiding them to articulate their thought processes before arriving at conclusions [32][33]. - "Direct Preference Optimization" (2023) simplifies the alignment process of language models by directly utilizing human preference data, making it a widely adopted method in the industry [34][35]. Important Optimizations - The "PagedAttention" mechanism improves memory management for LLMs, significantly enhancing throughput and reducing memory usage during inference [51][52]. - The "Mistral 7B" model showcases how smaller models can achieve high performance through innovative architecture, influencing the development of efficient AI applications [55][56].
锲而不舍落实中央八项规定精神丨重庆渝中、甘肃平凉等地坚持问题导向 推动学习教育走深走实
Yang Guang Wang· 2025-06-29 02:28
开林企业管理集团有限公司总裁助理郭战:为我们节省了大量时间和精力。这个服务确实让我们感 觉很周到,现在企业办事确实也更加方便了。 央广网北京6月29日消息 据中央广播电视总台中国之声《新闻和报纸摘要》报道,在深入贯彻中央 八项规定精神学习教育中,重庆渝中、甘肃平凉坚持问题导向,结合工作实际,动真碰硬解决问题。 重庆渝中区楼宇企业密集,针对之前搭建的企业服务平台,对企业一些需求难以第一时间解决企业 办事慢、多头跑等问题,渝中区探索推广一站式对企服务模式,为企业提供帮办、代办服务。 重庆市渝中区委组织部副部长王静:突出深学细悟、笃行实干,建立问题分类管理、分级领办、挂 单销号机制,切实把学习教育成效转化为推动高质量发展、高效能治理、高品质生活的实效。 甘肃平凉聚焦政务办理程序繁杂、材料冗杂等导致群众来回跑路的问题,联动多部门打通堵点,升 级一网通办、一窗受理,为企业和群众办事提效、减负。 甘肃省平凉市委组织部副部长陈涛:坚持立查立行改、集中整治改、为民解难改,整改整治作风顽 瘴痼疾,确保作风建设常态化长效化,以实际行动坚定拥护"两个确立"、坚决做到"两个维护"。 目前,重庆渝中区在学习教育过程中,重点梳理企业实际 ...
市委常委会暨市委理论学习中心组召开会议
Nan Jing Ri Bao· 2025-06-29 01:12
Group 1 - The meeting emphasized the importance of the United Front Work as a core responsibility of the Party Committee, highlighting the need to deeply understand Xi Jinping's thoughts on the United Front Work in the new era [1] - The meeting discussed the necessity to implement the responsibility system for United Front Work, focusing on key tasks such as industrial and technological innovation, reform breakthroughs, private economy development, and optimizing the business environment [1] - The meeting underscored the unique advantages of the United Front in terms of intellectual resources, talent, and broad connections, aiming to promote high-quality development of the United Front Work in the new era [1] Group 2 - The meeting reviewed the work report of the Municipal Commission for Discipline Inspection and Supervision for the first half of the year, stressing the ongoing severe and complex situation of the anti-corruption struggle [2] - The meeting highlighted the need for a combination of strict management and care, addressing issues of inaction and misconduct while also supporting responsible officials through mechanisms for error correction and clarification [2] - The meeting discussed the implementation of the Central Eight Regulations and emphasized the importance of integrating strict requirements into educational efforts, focusing on addressing prominent issues in work style [2] Group 3 - The meeting also reviewed the work related to the compilation of the "14th Five-Year Plan" and discussed other matters [3]
赵一德在深入贯彻中央八项规定精神学习教育年轻干部代表座谈会上强调涵养优良作风 脚踏实地干事 在推动高质量发展现代化建设中展现更大作为
Shan Xi Ri Bao· 2025-06-28 23:33
Core Points - The meeting emphasized the importance of young cadres in learning and education, highlighting the need to understand the significance of the Central Committee's deployment and the long-term effectiveness of the Eight Regulations [1][2] - Zhao Yide stressed the necessity for young cadres to enhance their political qualities and to draw nourishment from Xi Jinping's thoughts, improving their political judgment, comprehension, and execution [2] Group 1 - Young cadres are the focus of the learning education, and they must recognize the serious nature of the "Four Winds" issues and implement educational tasks thoroughly [1] - The process of refining work style should be seen as a way to strengthen party character and enhance political awareness [2] - Young cadres should actively engage in key tasks and reforms, demonstrating a spirit of responsibility and problem-solving [2] Group 2 - The meeting called for a strategic focus on the cultivation of young cadres, emphasizing the need for strict education, management, and support [2] - It was highlighted that young cadres should maintain high moral standards and be vigilant against corruption, emphasizing the importance of family education and integrity [2]
2025年如何从小白进阶成为AI/ML专家:助你拿下offer的修炼路线图
3 6 Ke· 2025-06-28 23:05
Core Insights - The article outlines an eight-step roadmap for efficiently advancing in AI/ML by focusing on essential skills and avoiding common pitfalls [1]. Group 1: Step-by-Step Learning Path - **Step 1: Master Python Core Libraries** Proficiency in Python is essential for AI/ML, including data cleaning, model building, and result visualization [2]. Key content includes Python basics, advanced AI programming techniques, and libraries like scikit-learn, NumPy, Matplotlib, Seaborn, and Pandas [4]. Recommended resources include CS50 Python course and "Python Data Science Handbook" [4]. Suggested learning period is 3-4 weeks [4]. - **Step 2: Solidify Mathematical Foundations** A strong grasp of linear algebra, probability, and calculus is crucial for understanding models [5]. Key content includes matrix operations, Bayesian thinking, and optimization techniques [5]. Recommended resources include "Linear Algebra" by 3Blue1Brown and MIT's Probability Introduction [5]. Suggested learning period is 4-6 weeks [5]. - **Step 3: Understand Machine Learning Basics** This step is pivotal for transitioning from beginner to competent AI/ML engineer [6]. Key content includes supervised vs. unsupervised learning, reinforcement learning, and deep learning [6]. Recommended resources include Google's Machine Learning Crash Course and "Machine Learning" by Andrew Ng [8]. Suggested learning period is 6-8 weeks [8]. - **Step 4: Hands-On Project Experience** Practical experience through real AI/ML applications is essential for job readiness [9]. Key content includes practical guides and project development [9]. Suggested learning period is ongoing [9]. - **Step 5: Learn MLOps** Understanding MLOps is vital for deploying and maintaining models in real-world scenarios [10]. Key content includes foundational concepts and best practices for model deployment [10]. Suggested learning period is 3-4 weeks [10]. - **Step 6: Specialize in a Domain** After building a foundation, focusing on a specific area like NLP or computer vision enhances employability [11]. Suggested learning period is ongoing [11]. - **Step 7: Stay Updated** Continuous learning is necessary to keep skills relevant in the fast-evolving AI field [12]. Key resources include ArXiv for research papers and notable figures in the field [12]. Suggested learning period is ongoing [12]. - **Step 8: Prepare for Interviews** Comprehensive preparation for interviews is crucial, including explaining model principles and system design [13]. Recommended resources include machine learning interview guides [13]. Suggested learning period is 4-6 weeks [13]. Conclusion - The article emphasizes a structured approach to mastering AI/ML, enabling individuals to transition from novices to job-ready professionals efficiently [1].
减了会议,多了调研(锲而不舍落实中央八项规定精神·一线见闻)
Ren Min Ri Bao· 2025-06-28 21:49
"这哪行?暴雨来了,肯定得涝!"姜旺对张青云说,"必须抓紧按照排水渠的标准,挖深、拓宽、延 长。" "村里一时拿不出这么多钱……"张青云说。 "得要这么深,才不怕暴雨。"跳进排水渠,姜旺用身体作尺,量了量渠深。 "放心!这渠深1米、宽80厘米。"一旁的黄狮村党支部书记张青云说。 抢夏收、忙夏播、抓防汛,这段时间,河南省南阳市西峡县五里桥镇党委书记姜旺奔忙在田间地 头,"现在会议少了,有更多时间抓具体工作了。" 深入贯彻中央八项规定精神学习教育开展以来,西峡县积极推进减会议、减报表、优调研,让干部腾出 更多时间和精力,到一线办实事。 以精简会议为例,西峡县推行"多会合一",将同类型的会议套开。以往,县农业农村局、水利局、交通 运输局、住房和城乡建设局都要召开防汛会议,今年全县安全生产、抗旱、防汛3个会议套开。今年以 来,西峡县召开各类会议数量同比下降35%,会议时长缩减50%以上。 减了会议,多出来的时间用在哪里?西峡县要求干部——"不能只在空调房里听汇报,必须直插基层一 线,去解决实际问题。" 来到黄狮村的猕猴桃园,姜旺发现,路旁只有一条灌溉用的水沟。 刚在这处检查完,又有村民找过来:"修渠把路给弄断了,我们 ...
市政府召开党组(扩大)会议:在一体推进学查改上再加力再深化,确保学习教育取得实效
Chang Jiang Ri Bao· 2025-06-28 07:44
会议要求,市政府党组要切实扛牢主体责任,各级领导干部要坚持以身作则、以上率 下,带头入脑入心学、全面彻底查、注重实效改,全力营造风清气正、干事创业的良好政治 生态。 编辑:胡之澜 6月27日,市政府召开党组(扩大)会议,传达学习中央第三指导组指导督导湖北见面 会精神、省委党的建设工作领导小组会议暨深入贯彻中央八项规定精神学习教育推进会精 神,部署政府系统贯彻落实措施。市委副书记、市长、市政府党组书记盛阅春主持会议并讲 话。 会议指出,在全党开展深入贯彻中央八项规定精神学习教育,是今年党建工作的重点任 务。全市政府系统要切实把思想和行动统一到党中央决策部署和省委、市委工作要求上来, 进一步增强抓好学习教育的政治自觉、思想自觉、行动自觉,把接受指导督导作为践行"两 个维护"的政治检验、纵深推进全面从严治党的有力抓手,全力支持配合中央指导组开展工 作,确保学习教育取得实效。 会议强调,要在一体推进学查改上再加力再深化,对中央指导组指出的问题照单全收、 立行立改,确保学有质量、查有力度、改有成效。要以推动解决突出问题为重点,结合巡视 巡察、整治形式主义为基层减负、整治群众身边不正之风和腐败问题等重点工作,明确查摆 问 ...
从后训练回到预训练,LLM+RL 的潜力兑现有有机会走更远吗?
机器之心· 2025-06-28 05:22
都是 NPT,用 RL 做预训练的潜力更大吗?为什么强化学习里很少有预训练模型?最流行的 RL 范式有何理论缺陷? 已有成效 的后训练 RL 实现存在什么问题? 2. 硅谷 AI Leaders 近期「暴论」大盘点! 1.从后训练回到预训练,LLM+RL 的潜力兑现有有机会走更远吗? 未来订阅 ChatGPT 就送人形机器人?AGI 为什么可能永远无法实现?为什么 AI 比程序员更显性价比?行业大模型真的没必要 吗?做好研究不如写好推文?OpenAI 和 Nvidia 的「AI 工厂」有何区别? 本期完整版通讯含 2 项专题解读 + 29 项 AI & Robotics 赛道要事速递,其中技术方面 11 项,国内方面 9 项,国外方面 9 项。 本期通讯总计 23143 字,可免费试读至 9% 机器之心PRO · 会员通讯 Week 26 --- 本周为您解读 ② 个值得细品的 AI & Robotics 业内要事 --- ① LLM 预训练对监督数据的需求趋于无穷,且需要覆盖尽可能所有遇到的问题,同时要求监督信号必须准确无 误,从而保证模型正确性。 ② 两项要求在现实中均难以实现,原因在于高质量人类标注数据 ...
OpenAI 4 名王牌研究员“叛变”,Meta 上亿美元的签约奖金终于花出去了
AI前线· 2025-06-28 05:13
整理 | 华卫 近日,据外媒报道,Meta 平台公司已招募四名前 OpenAI 研究人员加入其新成立的超级智能实验 室。 消息称,此次招聘对象包括 2022 年加入 ChatGPT 开发团队的特拉皮特·班萨尔(Trapit Bansal)。 据悉,他在启动 OpenAI 强化学习项目中发挥了关键作用。强化学习作为一种 AI 训练方法,适用于 构建推理模型。 另外三名已加入 Meta 的 OpenAI 研究人员分别是卢卡斯·拜尔(Lucas Beyer)、亚历山大·科列斯尼 科夫(Alexander Kolesnikov)和翟晓华(Xiaohua Zhai)。据了解,这三人曾于去年底协助建立 OpenAI 苏黎世办公室,此前他们在谷歌母公司 Alphabet 旗下的 DeepMind 机器学习实验室工作。 此次招聘发生在 Meta 首次披露组建超级智能研究团队的数周后。该实验室将负责开发能在广泛任务 中超越人类表现的 AI 模型。据悉,Meta 成立该部门的背景是其内部开发的大型语言模型 Llama 4 Behemoth 面临性能问题——该模型于今年早些时候预览,但因性能担忧已推迟发布。 上周,OpenAI 透 ...
量化指增迎超额盛宴!半鞅、蒙玺、龙旗、橡木、量盈等知名量化私募最新研判来袭!
私募排排网· 2025-06-28 02:37
今年来市场风格呈现出明显的大小盘分化,随着市场情绪的修复和市场活跃度的提升,大盘股表现相对较弱,而小盘股则受益于风险偏好提升、 流动性充沛等,表现尤为突出,量化策略的超额收益显著累积。 私募排排网数据显示,截至2025年5月底,有业绩显示的574只量化指增产品,近1年超额收益均值高达24.48%,其中正超额产品539只,正超额 占比高达93.91%。分三级策略来看,47只其他指增产品表现较为领先,近1年超额均值高达34.74%。(可参考: 最新量化多头超额榜揭晓!今 通、量创投资等领衔!进化论、龙旗、幻方等上榜! ) 本文首发于公众号"私募排排网"。 (点击↑↑ 上图查看详情 ) 半鞅私募基金 : 今年指数增强产品整体超额收益表现尤为突出,表观上来看,这得益于市场成交活跃度高、股票间的分化程度增加,这种市场 环境为量化管理人提供了丰富的交易机会,从而更容易获取超额收益。 从深层原因来看,则是在 固收类资产收益整体下行的背景下,权益市场因其相对较高的潜在回报和一定的"托底"效应,吸引了更多投资者的目 光,新增资金持续流入。 与此同时,特朗普上任后带来的市场不确定性增加,进一步激发了市场的波动性和交易活跃度。 最 ...