Lesson 6 of 11 8 min · 131 words
Model complexity
How capacity moves a model between underfitting and overfitting.
Model complexity (or capacity) is how rich a family of functions the model can represent. It rises with the polynomial degree, tree depth, number of features, number of layers and neurons, and falls with stronger regularisation.
As complexity grows, training error keeps falling. Test error first falls (less underfitting), reaches a minimum, then rises again (more overfitting). This U-shaped test-error curve is the classic picture of model selection.
The best complexity depends on the amount of data: with more data, you can afford a more complex model before overfitting starts.
Formula to remember
Lasso objective
L1 penalty sets some weights exactly to zero.
Quick check · +5 XP