Rich Sutton: synthetic data is "just a big mistake"

Industry Trends - Foundation model laboratories focus heavily on synthetic data generation to transcend the limitations of the existing human internet data sources [1] - Industry experts debate whether synthetic data generation within the Large Language Model scaling paradigm serves as a viable general method leveraging computation [1] Strategic Methodology - Artificial intelligence research in Alberta explores the big world perspective hypothesis suggesting the world is infinitely vast for learning systems [2][3] - Autonomous systems require the removal of humans from the loop to enable continuous learning from direct experience across infinitely many tasks [3]