Core Viewpoint - The article emphasizes that the internet is the key technology driving the advancement of artificial intelligence, rather than focusing solely on model architectures like Transformers [1][5][55]. Group 1: Importance of the Internet - The internet provides a rich and diverse data source that is essential for training AI models, enabling scalable deployment and natural learning pathways [1][5][54]. - Without the internet, even advanced models like Transformers would lack the necessary data to perform effectively, highlighting the critical role of data quality and quantity [28][30]. Group 2: Critique of Current Research Focus - The article critiques the current emphasis on optimizing model architectures and manual dataset creation, arguing that these approaches are unlikely to yield significant improvements in model capabilities [1][19][55]. - It suggests that researchers should shift their focus from deep learning optimizations to exploring new methods of data consumption, particularly leveraging the internet [16][17]. Group 3: Data Paradigms - The article outlines two main paradigms in data consumption: the compute-bound era and the data-bound era, indicating a shift in focus from algorithmic improvements to data availability [11][13]. - It argues that the internet's vast array of sequence data is perfectly suited for next-token prediction, which is a fundamental aspect of many AI models [17][22]. Group 4: Role of Reinforcement Learning - While reinforcement learning (RL) is seen as a necessary condition for achieving advanced AI, the article points out the challenges in obtaining high-quality reward signals for RL applications [55][61]. - The article posits that the internet serves as a complementary resource for next-token prediction, which is crucial for RL to thrive [55][56]. Group 5: Future Directions - The article calls for a reevaluation of how AI research is conducted, suggesting that a collaborative approach between product development and research could lead to more meaningful advancements in AI [35][54]. - It emphasizes the need for diverse and economically viable data sources to support the development of robust AI systems, indicating that user engagement is vital for data contribution [51][54].