Maximum-margin classification

When two classes are linearly separable, infinitely many lines separate them. A support vector machine picks the one with the maximum margin — the widest gap between the boundary and the nearest points of either class.

With labels $y_i\in \{-1,+1\}$ and boundary ${\mathbf{w}}^{\top }\mathbf{x}+b=0$, we scale so the closest points satisfy $y_i({\mathbf{w}}^{\top }{\mathbf{x}}_i+b)=1$. The margin width is then $2/\Vert \mathbf{w}\Vert$, and maximising it means:

This is a convex quadratic program with a unique solution. The intuition: a wide margin leaves room for noise, so new points near the boundary are still likely to be classified correctly — better generalisation.