Workflow
Qwen3系列模型
icon
Search documents
刚刚,Thinking Machines Lab博客提出在策略蒸馏,Qwen被cue 38次
3 6 Ke· 2025-10-28 02:00
Core Insights - Thinking Machines Lab (TML) has introduced a new training method called on-policy distillation, which combines reinforcement learning (RL) error correlation with supervised fine-tuning (SFT) reward density, achieving superior performance at a lower cost [1][17]. Group 1: Methodology and Applications - On-policy distillation is effective for small models, enhancing their domain performance and continuous learning capabilities [1][17]. - The method is inspired by the Qwen team’s research and heavily utilizes the Qwen3 series models during experiments [3][34]. - The training process consists of three stages: pre-training, mid-training, and post-training, focusing on general capabilities, domain knowledge, and target behavior respectively [6][7]. Group 2: Advantages of On-Policy Distillation - Small models trained with on-policy distillation often outperform larger general models in specialized fields due to benefits like local deployment, easier continuous training, and reduced inference costs [7][17]. - The method provides dense reward signals, allowing for more efficient learning compared to traditional RL, which offers sparse feedback [9][18]. Group 3: Performance and Cost Efficiency - TML's experiments show that on-policy distillation can achieve performance comparable to RL at a fraction of the cost, with reported costs being only one-tenth of traditional RL methods [34][41]. - The method has demonstrated significant computational efficiency, requiring 7-10 times fewer gradient steps to achieve similar performance levels as RL [58]. Group 4: Continuous Learning and Personalization - On-policy distillation is positioned as a promising tool for continuous learning, allowing models to update without degrading previously learned behaviors [66][70]. - The approach can effectively personalize models, enabling them to adapt to specific tasks while retaining core capabilities [42][53].
刚刚,Thinking Machines Lab博客提出在策略蒸馏,Qwen被cue 38次
机器之心· 2025-10-28 00:41
Core Viewpoint - Thinking Machines Lab (TML) has introduced a new training method called on-policy distillation, which combines reinforcement learning (RL) error correlation with supervised fine-tuning (SFT) reward density, achieving superior performance at a lower cost compared to other methods [1][2][27]. Group 1: Methodology and Advantages - On-policy distillation allows small models to exhibit strong domain performance and continuous learning capabilities [1][2]. - The training process is divided into three stages: pre-training for general capabilities, mid-training for domain knowledge, and post-training for guiding target behaviors [6][7]. - On-policy training samples trajectories from the student model itself, providing direct feedback to avoid errors, while off-policy training relies on external sources [8][9][12]. Group 2: Comparison with Other Methods - On-policy distillation combines the advantages of on-policy training's reliability and the dense reward signals from SFT, making it a cost-effective alternative to traditional RL methods [28][92]. - In experiments, on-policy distillation achieved a score of 74.4% on the AIME'24 benchmark with significantly lower computational costs compared to RL, which required 17,920 GPU hours for a score of 67.6% [47][46]. Group 3: Applications and Future Directions - The method has been successfully applied to train models for mathematical reasoning and to develop assistant models with domain knowledge and instruction-following capabilities [26][27]. - TML aims to continue exploring new applications of on-policy distillation, improving teacher supervision methods, and enhancing data efficiency and continuous learning [92][93].
英伟达千亿豪赌OpenAI;混沌HDDI商业智能体亮相云栖;红杉揭秘95%企业AI应用失败真相 | 混沌AI一周焦点
混沌学园· 2025-09-28 11:58
Core Insights - The article discusses the introduction of the HDDI, an AI-driven consulting tool by Hundun, aimed at transforming business strategy decision-making and making professional consulting services more accessible to small and medium enterprises [2][3]. Group 1: HDDI Features and Functionality - HDDI integrates Hundun's unique innovation theory framework and a decade's worth of case studies, functioning like a real consulting advisor [3]. - It shifts the business service model from a one-time project basis to a subscription-based partnership, providing continuous strategic support [3]. - The tool can help decision-makers identify core issues through guided conversations and generate comprehensive analysis reports within minutes, including feasibility assessments and implementation paths [6]. Group 2: AI Trends and Market Dynamics - Sequoia Capital's research indicates a "productivity paradox" with only 5% of companies deriving significant value from generative AI, while 95% see minimal benefits due to static tools that fail to integrate deeply into business processes [8]. - The AI landscape is witnessing a shift where AI is replacing entry-level jobs, emphasizing the importance of experienced employees' tacit knowledge as a competitive advantage [8]. - The article highlights the need for entrepreneurs to develop AI agents that can learn and integrate into backend processes, moving towards a business outcome-based pricing model [8]. Group 3: Major Industry Developments - Nvidia's strategic partnership with OpenAI involves an investment of up to $100 billion to build AI data centers, marking a significant advancement in AI infrastructure [17][23]. - The launch of the Dimensity 9500 chip by MediaTek represents a breakthrough in edge AI capabilities, with a 111% performance increase and a 56% reduction in power consumption [19][24]. - The article emphasizes the competitive landscape where large companies are integrating AI into their core products, creating new opportunities for startups to provide specialized AI solutions [20].
数字经济双周报(202507第2期)-20250801
Yin He Zheng Quan· 2025-08-01 10:37
Group 1: US AI Action Plan - The US AI Action Plan aims to establish global leadership in AI, focusing on "innovation-driven" and "deregulation" strategies to enhance market vitality and reduce development barriers[5] - Key policies include accelerating AI innovation, building AI infrastructure, and leading global AI order, with over 90 specific administrative orders outlined[6] - The plan emphasizes the importance of ensuring American workers benefit from AI advancements, creating high-paying jobs through infrastructure development[5] Group 2: Risks and Challenges for China - The US views China as its strongest competitor in AI, leading to risks such as deepening technology blockades and increased supply chain vulnerabilities, particularly in AI chip markets where Nvidia holds a 66% market share in China[9] - China's AI development may face fragmentation in industry standards and open-source barriers as the US promotes a "full-stack AI package" to expand its technological influence globally[13] - The US's focus on AI infrastructure and energy competition may create a technological gap between the US and China, impacting China's AI capabilities[16] Group 3: Global AI Governance and Cooperation - China has proposed the "Global AI Governance Action Plan," advocating for an inclusive and sustainable global AI governance system, emphasizing cooperation among developing countries[19] - The plan includes 13 key tasks, such as technology innovation and data governance, aiming to establish a unified international rule-making framework[20] - Local policies in China are accelerating the development of regional data industry systems, with Jiangxi province targeting a 20% annual growth in data markets by 2027[21] Group 4: AI Infrastructure Investments - Major US companies, including Google and Meta, are investing significantly in AI infrastructure, with Google planning to invest $25 billion in data centers and AI facilities across 13 states[33] - Trump's administration announced a $90 billion investment plan for AI and energy infrastructure, focusing on new data centers and power generation facilities[31] - The National Science Foundation (NSF) is collaborating with Voltage Park to provide 1 million hours of high-end GPU cloud computing resources for AI research[35]
数字经济双周报(202507第2期)-20250731
Yin He Zheng Quan· 2025-07-31 10:00
Group 1: US AI Action Plan - The US AI Action Plan aims to establish global leadership in AI, focusing on "innovation-driven" and "deregulation" strategies to enhance market vitality and reduce development barriers[5] - Key policies include accelerating AI innovation, building AI infrastructure, and leading global AI order, with over 90 specific administrative orders outlined[6] - The plan emphasizes the importance of ensuring American workers benefit from AI advancements, creating high-paying jobs through infrastructure development[5] Group 2: Risks and Challenges for China - China faces risks of deepening technology blockades, with Nvidia holding a 66% market share in China's AI chip market, indicating reliance on US technology[9] - The US aims to export a "full-stack AI package" to allies, potentially sidelining Chinese technologies and creating a fragmented global AI ecosystem[13] - Infrastructure gaps in AI capabilities may widen, as the US accelerates data center and energy infrastructure development to meet AI demands[16] Group 3: Global AI Governance and Cooperation - China released the "Global AI Governance Action Plan," advocating for an inclusive and sustainable global AI governance framework, emphasizing cooperation among developing countries[19] - The plan includes 13 key tasks, such as technology innovation and data governance, aiming to unify international rules and enhance participation from the Global South[20] - Local policies in China are rapidly emerging to build regional data industry systems, with Jiangxi aiming for a 20% annual growth in data industries by 2027[21] Group 4: AI Infrastructure Investments - Major US companies, including Google and Meta, are investing significantly in AI infrastructure, with Google planning to invest $25 billion in data centers and AI facilities[33] - Trump's administration announced a $90 billion investment plan for AI and energy infrastructure, focusing on new data centers and power generation facilities[31] - The establishment of the National AI Research Resource (NAIRR) aims to provide open AI research resources, enhancing collaboration in scientific fields[35]
整个HuggingFace榜,已经被中国AI模型一统江湖了。
数字生命卡兹克· 2025-07-31 01:06
Core Viewpoint - The article highlights a significant shift in the AI landscape, where domestic models in China are rapidly being open-sourced while overseas models are increasing in price and becoming less accessible [3][4][54]. Group 1: Open-source Models - Numerous Chinese companies have been actively open-sourcing their AI models, including MiniMax, Kimi, Qwen, and others [1]. - The top ten models on Hugging Face are all Chinese open-source models, with notable mentions such as Zhiyu GLM-4.5 at the top and Qwen holding five positions [8][9]. - The article emphasizes the rapid development and release of various models over a short period, showcasing the strength of domestic open-source efforts [11][12]. Group 2: Recent Model Releases - Tencent released the Hunyuan A13B model on June 27, featuring 80 billion total parameters and 13 billion active parameters [17][18]. - Baidu's ERNIE 4.5 was officially open-sourced on June 30, offering both pure LLM and multimodal capabilities [20]. - Alibaba's Tongyi launched the first CoT audio model, ThinkSound, on July 1, aimed at video dubbing [21]. - Zhiyu introduced the GLM-4.1V-Thinking model on July 2, which received positive evaluations for its performance [23]. - Kunlun Wanwei released the Skywork-Reward-V2 series on July 4, comprising eight reward models with parameters ranging from 600 million to 8 billion [25][26]. - The MOSS-TTSD model was open-sourced by Qiu Xipeng's team on July 5, trained on a million hours of audio [27]. - Ant Group's KAG-Thinker model, focused on interactive reasoning, was released on July 8 [32]. - The Intern-S1 model, a multimodal model, was launched by the Shanghai AI Lab on July 26 [41]. - Qwen's series of models, including Qwen3-235B and Qwen3-Coder, were released throughout July, achieving high rankings on the Hugging Face leaderboard [37][38][39]. Group 3: Industry Impact - The article reflects on the transformation of the AI landscape over the past two years, noting that China has moved from being a follower to a leader in open-source AI models [11][56]. - The ongoing trend of open-sourcing in China contrasts sharply with the increasing restrictions and pricing of models from overseas companies [54][55]. - The author concludes that this period marks the beginning of a new era for domestic AI models and the Chinese open-source community [56].
估值超100亿,传宇树科技完成C轮融资,腾讯阿里吉利联投;AI智能体对话存在低俗擦边内容,涉事APP被依法约谈丨AI周报
创业邦· 2025-06-22 23:45
Group 1 - The article highlights significant AI investment and financing events from June 14 to June 20, showcasing the dynamic landscape of the global AI market [1] - Yushu Technology, a prominent robotics company, completed a Series C financing round with a valuation exceeding 10 billion RMB, led by major investors including China Mobile and Tencent [3] - MiniMax, an AI unicorn, is considering an IPO in Hong Kong and has recently raised $600 million in Series A funding, with a post-money valuation of $2.5 billion [7][10] Group 2 - The article reports that the AI sector in China saw a total financing amount of 9.8 billion RMB this week, with the highest financing coming from PAXINI, a developer of three-dimensional intelligent tactile sensors [58][59] - The global AI financing events totaled 13.6 billion RMB this week, with a decrease of 6 events compared to the previous week [51][68] - The report indicates that the majority of AI financing events were concentrated in Shanghai, Guangdong, Jiangsu, and Zhejiang, with Shanghai leading with 5 disclosed events [55] Group 3 - The article mentions that the demand for humanoid robots in China has surged, with job postings in this sector increasing by 409% year-on-year in the first five months of 2025 [26] - The first invasive brain-machine interface clinical trial in China has been successfully conducted, marking a significant advancement in this technology [31][32] - The article discusses the launch of various AI products, including a new AI programming mode by Tencent and an AI podcast feature by Doubao [12][17]
跨境电商去年出口超2万亿,高息高返购车贷被叫停 | 财经日日评
吴晓波频道· 2025-06-17 17:02
Trade Agreements - The US and UK have reached a preliminary trade agreement covering various sectors including steel, automobiles, ethanol, beef, and aerospace, marking the first such agreement since Trump's administration began imposing tariffs [1] - The agreement allows for increased market access for US agricultural products in the UK, particularly beef and ethanol, with the US setting an annual quota of 100,000 vehicles for UK imports, subject to a 10% tariff [1][2] - While the agreement is seen as a foundation for expanding trade, it is limited in scope and does not cover several key industries such as pharmaceuticals and steel [1] Cross-Border E-commerce - China's cross-border e-commerce exports reached approximately 2.15 trillion yuan in 2024, a year-on-year increase of 16.9%, while imports were about 555.25 billion yuan, growing by 4.1% [3] - The export of consumer goods constitutes 97.5% of cross-border e-commerce, with major products including apparel, digital devices, and home goods, primarily exported to the US, UK, and Germany [3] - The growth of cross-border e-commerce is supported by favorable policies, contributing significantly to overall foreign trade despite rising trade protectionism [4] Online Literature - The user base for Chinese online literature has reached 575 million, with a significant portion of readers aged 26 to 45, and nearly 25% being from the "post-2000" generation [5] - The revenue from online literature is estimated at around 44 billion yuan in 2024, with a total of over 33 million works published [5] - Despite the large user base, the industry faces challenges such as content homogenization and competition from short video formats, leading to a slowdown in user growth [6] Automotive Financing - The "high interest, high return" car loan model has been halted in several regions, with banks suspending such business practices due to regulatory changes [7] - This model involved dealerships receiving high commissions from banks to subsidize car prices, but it has led to various issues, including disputes over early loan repayments [8] - The cessation of this model may indicate a shift towards more sustainable lending practices, as banks need to focus on genuine consumer demand rather than artificially created demand [8] AI and Technology - Alibaba has launched a new version of its Qwen3 model optimized for Apple's MLX framework, which is expected to enhance AI deployment across Apple devices [9] - This move signals a strengthening partnership between Alibaba and Apple, as the models will be available in various precision versions to cater to different devices [9][10] - The growing influence of Qwen in the open-source model space may help revitalize Apple's presence in the Chinese market [10] Pension Funds - As of the first quarter of 2025, China's enterprise annuity fund has accumulated a scale of 3.73 trillion yuan, with a three-year cumulative return rate of 7.46% [11] - The shift to reporting cumulative returns aligns with a long-term investment strategy, reflecting the ability of pension funds to achieve consistent returns, particularly in stock investments [12][13] - The favorable environment for pension fund management, free from the pressures faced by traditional fund companies, allows for a focus on long-term value investment [13] Stock Market Trends - The stock market experienced slight declines, with the Shanghai Composite Index closing at 3387.4 points, down 0.04%, amid a lack of strong market momentum [16] - Despite fluctuations, certain sectors such as oil and gas remain active, while consumer sectors like gaming and beauty faced declines [16] - Upcoming policy announcements from the Lujiazui Forum may create short-term market disturbances but are unlikely to change the overall market trend [17]
中东局势突发升级!A股保持韧性,创新药被爆吹后套人了?
Sou Hu Cai Jing· 2025-06-17 09:28
Group 1 - A-share market is supported by the upcoming Lujiazui Forum, with financial sectors showing strength, indicating potential policy speculation [1] - Tensions in the Middle East are escalating, with Israeli Prime Minister Netanyahu stating that actions against Iran will continue, and Trump leaving the G7 summit early due to the situation, heightening market anxiety [1][3] - Despite initial strength in gold and oil prices, the market remains skeptical about further deterioration in the Middle East, leading to a reduction in price increases [3] Group 2 - The market is experiencing conflicting messages regarding potential talks and military actions, particularly affecting international oil trading strategies [5] - A notable trend in A-shares is the "end-of-month effect," where high-flying stocks often see profit-taking as the month closes, suggesting caution for investors [7] - The stock of Brain Rejuvenation Technology (RGC.US) surged by 283%, reaching a market cap of over $29.6 billion, significantly impacting related sectors in A-shares [8] Group 3 - New tax refund policies for overseas travelers in Dalian and Hubei are set to take effect in July 2025, which could influence retail sectors [10] - Micron has announced plans to halt production of DDR4/LPDDR4 chips, indicating a supply-demand imbalance in the memory market, which may affect related stocks [12] - The solid-state battery sector is gaining traction, with several companies experiencing significant stock price increases due to strong demand and positive market sentiment [18][17] Group 4 - The coal, public utilities, oil and petrochemicals, transportation, and retail sectors are leading in performance, while pharmaceuticals, beauty care, media, textiles, and light manufacturing are lagging [20] - The human brain engineering sector saw a rise of 3.34%, while the innovative drug sector experienced a decline of 2.81%, reflecting market volatility [21]
港股收盘(06.17) | 恒指收跌0.34% 脑机接口概念爆发 医药、新消费显著调整
智通财经网· 2025-06-17 08:44
Market Overview - The Hong Kong stock market indices experienced fluctuations, with the Hang Seng Index closing down 0.34% at 23,980.30 points and a total trading volume of HKD 20.21 billion [1] - The Hang Seng China Enterprises Index fell by 0.4% to 8,694.67 points, while the Hang Seng Tech Index decreased by 0.15% to 5,291.85 points [1] Investment Opportunities - According to Shenwan Hongyuan, investment opportunities in Hong Kong stocks are expected to expand in the second half of 2025, focusing on growth sectors such as internet technology and pharmaceuticals [1] - New consumption stocks still hold alpha advantages in the medium term, but face short-term challenges regarding cost-effectiveness [1] Blue-Chip Stocks Performance - Alibaba (09988) rose by 1.68% to HKD 114.8, contributing 31.36 points to the Hang Seng Index [2] - Other notable blue-chip performances include Sands China (01928) up 5.24%, Sunny Optical Technology (02382) up 2.73%, while Chow Tai Fook (01929) fell by 7.29% and CSPC Pharmaceutical Group (01093) dropped by 6.4% [2] Sector Highlights Brain-Computer Interface Sector - The brain-computer interface sector saw significant gains, with Nanjing Panda Electronics (00553) surging 38.02% [3] - The sector's growth is supported by recent advancements in clinical trials for invasive brain-computer interfaces in China [3][4] Gaming Sector - The gaming sector benefited from Macau's new visa policies for citizens of Saudi Arabia and other Gulf countries, with Sands China and MGM China showing strong gains [4][5] Apple-Related Stocks - Apple-related stocks collectively rose, with notable increases in companies like GoerTek (01415) and Sunny Optical Technology (02382) following Alibaba's announcement of new AI models optimized for Apple devices [6] Pharmaceuticals and New Consumption - The pharmaceutical and new consumption sectors faced significant adjustments, with companies like Green Leaf Pharmaceutical (02186) and Lepu Medical Technology (02157) experiencing declines [6] Gold Stocks - Gold stocks generally declined, with Tongguan Gold (00340) down 12.68% as international gold prices fell below USD 3,400 per ounce [7] Notable Stock Movements - Chow Tai Fook (01929) saw a sharp decline of 7.29% after announcing a proposed issuance of HKD 8.8 billion convertible bonds [8] - Maimai (02556) rose by 5.41% after being recognized as a top AI SaaS company for seven consecutive years [9] - New City Development (01030) reached a new high, with a rating upgrade from "Neutral" to "Buy" by Bank of America [10] - Weimeng Group (02013) increased by 2.82% following the announcement of new features for WeChat public accounts [11]