Generalization performance
Search documents
X @Avi Chawla
Avi Chawla· 2025-12-03 19:06
RT Avi Chawla (@_avichawla)Bias-variance tradeoff has a missing detail!Not many ML engineers know about it.Consider fitting a polynomial regression model on some dummy dataset, say, y=sin(x) + noise.As shown in the first plot in the image, as we increase the degree (m):- The training loss will go down to zero.- The test (or validation) loss will decrease and then increase.But notice what happens as we continue to increase the degree (m):↳ Test loss decreases again (shown in the second plot)This is called th ...