线性规划
Search documents
DeepSeek悄悄开源LPLB:用线性规划解决MoE负载不均
3 6 Ke· 2025-11-20 23:53
Core Insights - DeepSeek has launched a new code repository called LPLB on GitHub, which aims to address the bottlenecks of correctness and throughput in model training [1][4] - The project currently has limited visibility, with fewer than 200 stars on GitHub, indicating a need for more attention [1] Project Overview - LPLB stands for Linear-Programming-Based Load Balancer, designed to optimize load balancing in machine learning models [3] - The project is still in the early research phase, with performance improvements under evaluation [7] Mechanism of LPLB - LPLB implements dynamic load balancing through three main steps: dynamic reordering of experts, constructing replicas, and solving optimal token allocation for each batch [4] - The mechanism utilizes a built-in linear programming solver and NVIDIA's cuSolverDx and cuBLASDx libraries for efficient linear algebra operations [4][10] Comparison with EPLB - LPLB extends the capabilities of EPLB (Expert Parallel Load Balancer) by focusing on dynamic fluctuations in load, while EPLB primarily addresses static imbalances [8] Key Features - LPLB introduces redundant experts and edge capacity definitions to facilitate token redistribution and minimize load imbalance among experts [9] - The communication optimization leverages NVLINK and NVSHMEM to reduce overhead compared to traditional methods [10] Limitations - Current limitations include ignoring nonlinear computation costs and potential delays in solving optimization problems, particularly for small batch sizes [11][12] - In extreme load imbalance scenarios, LPLB may not perform as well as EPLB due to its allocation strategy [12] Typical Topologies - LPLB allows for various topological configurations, such as Cube, Hypercube, and Torus, to define the distribution of expert replicas [13] Conclusion - The LPLB library aims to solve the "bottleneck effect" in large model training, where the training speed is limited by the slowest GPU [14] - It innovatively employs linear programming for real-time optimal allocation and utilizes NVSHMEM technology to overcome communication bottlenecks, making it a valuable resource for developers working on MoE architecture training acceleration [14]
DeepSeek悄悄开源LPLB:用线性规划解决MoE负载不均
机器之心· 2025-11-20 15:13
机器之心报道 编辑:Panda 没有发推文,也没有公众号更新,少有的几个技术博主分享的推文也关注不多。截至目前,该项目的 star 数量也还没超过 200。 但仔细一看,这个项目却似乎并不简单,值得更多关注。X 网友 gm8xx8 评论认为这表明 DeepSeek 正在解决正确性和吞吐量瓶颈问题,为下一版模型发布做准 备。 昨天,DeepSeek 在 GitHub 上线了一个新的代码库: LPLB 。 项目地址:https://github.com/deepseek-ai/LPLB 项目简介 LPLB,全称 Linear-Programm i ng-Based Load Balancer ,即基于线性规划的负载均衡器。 顾名思义,LPLB 是一个并行负载均衡器,它利用线性规划(Linear Programming)算法来优化 MoE(混合专家)模型中的专家并行工作负载分配。 具体来说,LPLB 通过以下三个步骤实现动态负载均衡: 3. 求解最优分配 : 针对每个批次(Batch)的数据,求解最优的 Token 分配方案。 1. 动态重排序 : 基于工作负载统计信息对专家进行重排序(Reordering)。 2 ...
一个运行了80年的算法,我们现在才真正理解它?
机器之心· 2025-10-19 03:48
来自 Quanta Magazine 作者: Steve Nadis 机器之心编译 从你网购的包裹如何以最快速度送达,到航空公司如何规划数千架飞机的航线以节省燃料,背后都有一个近 80 岁「高龄」的数学方法在默默 工作。它被誉为优化领域的基石,高效又令人信赖。然而,一个奇怪的事实是:几十年来,没有人能从理论上完美解释它为何如此高效。现 在,这个谜题的最后一块拼图,终于被找到了。 1939 年,当时还是加州大学伯克利分校一年级研究生的 乔治·丹齐格(George Dantzig)在一次统计学课上迟到了。他从黑板上抄下了两个问题,以为是家庭作 业。他后来回忆说,他发现这次的作业「比平时难得多」,并为自己多花了好几天才完成而向教授道歉。 几周后,他的教授告诉他,他成功解决了统计学领域两个尚待解决的 著名 问题。 丹齐格 的这项成果为他的博士论文奠定了基础,并在几十年后成为了电影《心灵捕手》的灵感来源。 乔治 · 丹齐格( George Dantzig , 1914—2005 ),美国著名数学家, 1947 年提出了单纯形法,被称为线性规划之父。 丹齐格 在 1946 年,也就是二战刚结束后不久,获得了博士学位,并很 ...
展望未来:炼油与石化行业战略转型已成必选项
麦肯锡· 2025-08-26 10:06
Core Viewpoint - The refining and chemical industries are facing significant challenges due to slowing demand growth, the rise of electric vehicles, and ongoing capacity expansion, leading to a projected decline in refining margins by about 5% to 30% by 2030 [3][4]. Recent Trends and Market Outlook - The refining market is expected to see a notable decline in profit margins, primarily driven by demand slowdown and capacity expansion disrupting supply-demand balance [3]. - The chemical market is also under pressure, with rapid capacity expansion, especially in China, outpacing demand growth, leading to overcapacity and compressed profit margins [3]. Challenges for Asian Refining and Chemical Industries - The evolving market dynamics are reshaping the competitive landscape, necessitating adaptation from companies [4]. - Uncertainties in carbon neutrality policies complicate long-term planning for refining and chemical companies, potentially leading to the exit of outdated capacities and cancellation of planned projects [4]. - Geopolitical tensions and fluctuating trade policies are adding further challenges, with tariffs on key raw materials increasing production costs by approximately 7% [4]. Strategic Pathways for Survival - Companies are focusing on cost reduction, capacity optimization, and digital transformation to navigate the challenges in the refining and petrochemical sectors [5]. - Operational transformation is essential for survival, with companies leveraging various strategies to enhance operations and profitability [5][6]. Production and Optimization - Linear programming (LP) models can provide insights to capture high-value opportunities with minimal investment, potentially increasing capacity by up to 5% [7]. - Advanced process control (APC) is being deployed to optimize operations and improve product yields, with potential cost reductions of $0.3 per barrel [8]. Efficient Maintenance - Effective maintenance strategies can significantly reduce costs and downtime, with potential savings of 5-15% on turnaround costs [10]. - Predictive maintenance is being utilized to monitor equipment health and reduce unplanned downtime [10]. Capital Expenditure (CAPEX) Optimization - Optimizing CAPEX is crucial for addressing tightening capital constraints and ensuring maximum returns while minimizing costs and risks [11]. - Structured methodologies like risk threat prioritization (RTP) are being employed to ensure rigorous evaluation of capital projects, leading to CAPEX reductions of 10-20% [11][13]. Sales Optimization - Optimizing commercial performance is vital for maintaining profitability, with effective sales strategies leveraging data-driven analysis to accelerate revenue growth [14]. - Dynamic pricing models based on customer willingness to pay are being adopted to maximize revenue and profit [15]. Conclusion - The Asian refining and petrochemical industries are entering a period of structural upheaval, with traditional advantages becoming less reliable [16]. - Future winners will be those companies that can adapt quickly to market changes, deeply integrate digital technologies, and optimize costs and product portfolios [16].
杉数科技申请基于线性规划的热能交易市场调控方法专利,优化热能交易市场运作机制
Sou Hu Cai Jing· 2025-05-16 02:45
Core Insights - The article discusses the patent application by multiple subsidiaries of Shanshu Technology, focusing on a method for regulating the thermal energy trading market based on linear programming [1][2][3] Group 1: Patent Application - Shanshu Technology has applied for a patent titled "Method, Device, and Electronic Equipment for Regulating Thermal Energy Trading Market Based on Linear Programming," with publication number CN119990663A, filed on February 2025 [1] - The patent aims to optimize the operation mechanism of the thermal energy trading market, enhancing overall market efficiency and stability [1] Group 2: Company Overview - Shanshu Technology (Beijing) Co., Ltd. was established in 2016, with a registered capital of 13.003 million RMB, and has invested in 10 companies and participated in 70 bidding projects [2] - Shanghai Shanshu Network Technology Co., Ltd., also founded in 2016, has a registered capital of 10 million RMB and has participated in 8 bidding projects [2] - Guangzhou Shanshu Technology Co., Ltd. was established in 2023 with a registered capital of 1 million RMB, focusing on software and IT services [2] - Shanshu Technology (Nanjing) Co., Ltd., founded in 2023, has a registered capital of 3 million RMB and specializes in professional technical services [2][3] - Fifth Paradigm (Shenzhen) Technology Co., Ltd., established in 2018, has a registered capital of 1 million RMB and focuses on software and IT services [3] - Shanshu Technology (Suzhou) Co., Ltd., founded in 2016, has a registered capital of 1 million RMB and provides professional technical services [3]