梦晨 发自 凹非寺 量子位 | 公众号 QbitAI 可以说它是让深度学习从小规模实验，走向大规模实用化和可靠性的关键技术之一。 | TITLE | CITED BY | YEAR | | --- | --- | --- | | Batch normalization: Accelerating deep network training by reducing internal covariate shift | 61928 | 2015 | | S loffe, C Szegedy | % 公众号·量子位 | | | International conference on machine learning 448-456 | | | 深度学习界的传奇论文，终于等来了它的"封神"时刻！ 刚刚， ICML 2025 会议上，2015年发表的 Batch Normalization （批次归一化，简称BatchNorm）论文荣获 时间检验奖 。 这篇如今引用量 超过6万次 的开创性工作，是深度学习发展史上一个里程碑式的突破，极大地推动了深层神经网络的训练和应用。 当时谷歌研究员 Sergey Ioffe 和 C ...

Core Insights - The Batch Normalization paper, published in 2015, has been awarded the Time-Tested Award at ICML 2025, highlighting its significant impact on deep learning [1] - With over 60,000 citations, this work is considered a milestone in the development of deep learning, facilitating the training and application of deep neural networks [2][4] - Batch Normalization is a key technology that enabled deep learning to transition from small-scale experiments to large-scale practical applications [3] Group 1 - In 2015, deep learning faced challenges in training deep neural networks, which were often unstable and sensitive to parameter initialization [5][6][7] - Researchers Sergey Ioffe and Christian Szegedy identified the issue of Internal Covariate Shift, where the distribution of data within the network changes during training, complicating the training process [8][11] - Their solution involved normalizing the data at each layer, similar to input layer normalization, which significantly improved training speed and stability [12] Group 2 - The original paper demonstrated that using Batch Normalization allowed advanced image classification models to achieve the same accuracy with only 1/14 of the training steps [13] - Batch Normalization not only accelerated training but also introduced a regularization effect, enhancing the model's generalization ability [14][15] - Following its introduction, Batch Normalization became foundational for many mainstream convolutional neural networks, such as ResNet and DenseNet [18] Group 3 - In 2018, a paper from MIT challenged the core theory of Batch Normalization, showing that even with introduced noise, models with Batch Normalization still trained faster than those without it [21][23] - This research revealed that Batch Normalization smooths the Optimization Landscape, making gradient behavior more predictable and stable [24] - It was suggested that Batch Normalization acts as an unsupervised learning technique, allowing networks to adapt to the data's inherent structure early in training [25] Group 4 - Recent studies have provided deeper insights into Batch Normalization from a geometric perspective [29] - Both authors, Ioffe and Szegedy, have continued their careers in AI, with Szegedy joining xAI and Ioffe following suit [30][32] - Szegedy has since transitioned to a new role at Morph Labs, focusing on achieving "verifiable superintelligence" [34]