Workflow
深度监督网络)
icon
Search documents
被拒稿11年后翻盘获时间检验奖,DSN作者谢赛宁:拒稿≠学术死刑
量子位· 2025-05-06 04:24
Core Viewpoint - The article discusses the recognition of the paper "Deeply-Supervised Nets" (DSN) by AISTATS 2025 with a Time-Tested Award, highlighting its long-term impact on the field of deep learning and computer vision despite initial rejection by NeurIPS ten years ago [1][5][21]. Group 1: Paper Background and Development - The paper DSN was submitted in September 2014 and aimed to address issues in deep learning related to hidden layer feature learning and classification performance [2][12]. - The concept of intermediate layer supervision proposed in DSN has been further developed in subsequent works by the author, Saining Xie, such as REPA and U-REPA, showcasing the evolution from single model optimization to cross-model knowledge transfer [3][4]. Group 2: Technical Contributions - DSN addresses three major pain points of traditional Convolutional Neural Networks (CNNs): gradient vanishing, feature robustness, and training efficiency [14][15]. - The introduction of auxiliary classifiers in hidden layers enhances gradient signals, improves the discriminative power of shallow features, and accelerates training convergence, with empirical results showing a 30% faster convergence for ResNet-50 on the CIFAR-10 dataset and a 2.1% increase in Top-1 accuracy [15][17]. Group 3: Recognition and Impact - The paper has been cited over 3000 times on Google Scholar, indicating its significant influence in the field [18]. - The Time-Tested Award from AISTATS recognizes the paper as a seminal work that has laid the foundation for subsequent research, similar to the impact of GANs and Seq2Seq models in their respective areas [22][23]. Group 4: Personal Reflections and Insights - Saining Xie reflects on the initial rejection from NeurIPS, emphasizing the importance of perseverance in academic careers and the value of a strong support system [25][26]. - The article encourages researchers to view rejections as opportunities for improvement, citing examples of other significant works that faced initial rejection but later gained recognition [30][31].