Underfitting and overfitting, revisited

Diagnose a model by comparing training and validation error:

• Both high, close together → underfitting (high bias). The model can't even fit the training data.
• Training low, validation much higher → overfitting (high variance). The model memorised the training data.
• Both low, close together → a good fit.

Learning curves plot error against the number of training examples. For an underfitting model, the two curves quickly meet at a high error — more data won't help. For an overfitting model, there is a big gap that narrows as data grows — more data will help.

Validation curves plot error against a complexity hyperparameter (degree, depth, λ). Training error falls steadily; validation error forms a U. The bottom of the U is the best setting.