From regression to classification

In classification the target is a category, encoded for binary problems as $y\in \{0,1\}$. You could fit ordinary linear regression to 0/1 targets and threshold at 0.5 — but it works poorly.

The fitted line produces values below 0 and above 1, which can't be probabilities. Worse, a few points far from the boundary pull the line around and shift the threshold, even though they are already easy to classify. Squared error also punishes predictions that are 'too correct' (e.g. 1.8 for a positive).

The fix: keep a linear score $z={\mathbf{w}}^{\top }\mathbf{x}+b$, but squash it into $(0,1)$ with an S-shaped function and train it with a loss suited to probabilities. That is logistic regression.