Linear model
Weighted sum of features plus intercept.
← Machine Learning learning path0%
06
Supervised learning
Linear Regression
The first complete model: a linear function fit by least squares, in closed form or by gradient descent.
Not started yet
Start Quiz mastery0%
What you’ll take away
- Linear regression predicts a weighted sum of features plus a bias.
- Least squares has a closed form (normal equation) and a convex loss for gradient descent.
- Residual plots reveal non-linearity, outliers and changing variance.
- RMSE punishes big errors, MAE is robust, R² is the share of variance explained.
Major concepts (syllabus)
Linear models, least squares, closed-form solution, gradient descent, prediction error, Univariate vs. multivariate regression, Overfitting, underfitting, and regularization (Ridge/Lasso intro). Evaluation metrics: RMSE, MAE, R², F1 score
Lessons
- Linear modelsPredicting a target as a weighted sum of features plus a bias.8 min
- Univariate vs. multivariate regressionOne input feature versus many.8 min
- Least squaresChoosing weights that minimise the sum of squared residuals.8 min
- The closed-form solutionThe normal equation and when it is practical.10 min
- Gradient descent for regressionTraining the same model iteratively when data is large.9 min
- Prediction errorResiduals and what their patterns tell you.7 min
- Overfitting and regularization in regressionPolynomial features, and Ridge/Lasso to tame them.9 min
- Metrics: RMSE, MAE and R²Error in target units, robust error and explained variance.9 min
- A first look at the F1 scoreThe classification metric you will meet properly in Model Evaluation.6 min
Key formulas
Least squares loss
Sum of squared residuals.
Normal equation
Closed-form least-squares solution.
MSE gradient
Used for gradient-descent training.
RMSE
Error in the target's units; punishes big errors.
MAE
Average absolute error; robust to outliers.
R²
Fraction of variance explained; 1 is perfect.
Key terms
- Linear regression
- Predicting a continuous target as a weighted sum of features plus a bias.
- Normal equation
- Closed-form least squares solution w = (XᵀX)⁻¹Xᵀy.
- R²
- Fraction of the target's variance explained by the model.
- Residual
- Difference between the true and predicted value.
- RMSE
- Root mean squared error, in the target's units.
Practice this topic6 questions · earn XP and build your streak