Support vectors

Support vectors are the training points that lie exactly on the margin (or, with soft margins, inside it or on the wrong side). They are the only points that determine the boundary.

In the dual formulation, the solution is $\mathbf{w}={\sum }_i{\alpha }_i{y}_i{\mathbf{x}}_i$, and ${\alpha }_i>0$ only for support vectors. Every other point has ${\alpha }_i=0$: you could delete it and the boundary would not move.

This makes SVMs memory-efficient at prediction time (only support vectors are kept) and robust to points far from the boundary. It also gives a rough generalisation hint: fewer support vectors relative to $n$ usually means a simpler boundary.