Distance measures for k-NN

k-NN depends entirely on its notion of 'close'. Common choices:

• Euclidean (L2): $\sqrt{{\sum }_j(x_j-z_j{)}^2}$ — the default.\n- Manhattan (L1): ${\sum }_j|x_j-z_j|$ — less sensitive to one large coordinate difference.\n- Minkowski: $({\sum }_j|x_j-z_j{|}^p{)}^{1/p}$ — generalises both.\n- Cosine distance: $1-\cos \theta$ — for text and embeddings, where direction matters more than length.\n- Hamming: count of differing positions — for categorical or binary features.

Scaling is essential. If income ranges up to 100,000 and age up to 100, Euclidean distance is almost entirely income. Standardise (z-score) or min-max scale every feature first.

In high dimensions, distances between all points become similar (the curse of dimensionality), and 'nearest' loses meaning. Reduce dimensions with PCA or feature selection before using k-NN.