Regularized objective
Data fit plus a complexity penalty Ω.
← Machine Learning learning path0%
09
Supervised learning
Overfitting and Regularization
Revisiting the bias–variance trade-off with the tools to control it.
Not started yet
Start Quiz mastery0%
What you’ll take away
- Compare training and validation error to diagnose bias vs. variance.
- Fix bias with more capacity; fix variance with more data or regularisation.
- L2 shrinks all weights smoothly; L1 zeroes many out.
- Choose λ by cross-validation on a logarithmic grid.
Major concepts (syllabus)
Bias–variance trade-off, underfitting, overfitting, L1/L2 regularization, model complexity
Lessons
- Underfitting and overfitting, revisitedDiagnosing both with training and validation curves.7 min
- The bias–variance trade-off in practiceLearning curves and what to change for each problem.9 min
- Model complexityDegree, depth, number of features and parameters.7 min
- L2 regularizationShrinking weights smoothly; always gives a unique solution.8 min
- L1 regularizationSparse weights and built-in feature selection.8 min
- Choosing λTuning the penalty strength with cross-validation.8 min
Key formulas
Ridge closed form
λI keeps the matrix invertible.
Key terms
- Regularization
- Penalising model complexity to improve generalisation.
Practice this topic5 questions · earn XP and build your streak