Workflow
生成式视角重塑监督学习!标签不只是答案,更是学习指南 | ICML 2025
量子位·2025-06-24 13:36

Core Viewpoint - A new paradigm in supervised learning called Predictive Consistency Learning (PCL) is introduced, which redefines the role of labels as auxiliary references rather than just standard answers for comparison [1][5]. Group 1: Training Process Overview - PCL aims to capture complex label representations by progressively decomposing label information, allowing the model to predict complete labels with partial label hints [5][6]. - The training process involves mapping noisy labels back to true labels, with noise levels controlled by time steps, ensuring predictions remain consistent across different noise levels [7][8]. Group 2: Noise Process - The noise process for discrete labels is modeled using a categorical distribution, while continuous labels follow a Gaussian diffusion model, introducing noise progressively [9][11]. - In cases where labels are too complex, PCL introduces Gaussian noise directly into the latent embedding space, aligning with the continuous label noise process [11]. Group 3: Testing Process Overview - After training, the model can efficiently predict by sampling from a random noise distribution, achieving results that surpass traditional supervised learning even without label hints [14][28]. - A multi-step inference strategy is employed to refine predictions, where previous predictions are perturbed with noise to serve as hints for subsequent predictions [14][28]. Group 4: Information Theory Perspective - PCL proposes a structured learning process that gradually captures information, allowing the model to learn from noisy labels while minimizing dependency on them [15][18]. - The model's goal is to minimize noise condition dependence, ensuring predictions remain consistent across varying noise levels [19]. Group 5: Experimental Results - PCL demonstrates significant improvements in prediction accuracy across various tasks, including image segmentation, graph-based predictions, and language modeling, compared to traditional supervised learning [20][25][30]. - In image segmentation, PCL outperforms traditional methods in single-step predictions and continues to improve with additional prediction steps [22][28]. - The results indicate that while more inference steps can enhance detail capture, they also risk error accumulation, necessitating a balance in the number of steps [26][28].