Inner products

The inner (dot) product of two vectors is ${\mathbf{u}}^{\top }\mathbf{v}={\sum }_i{u}_i{v}_i$. It equals $\Vert \mathbf{u}\Vert \Vert \mathbf{v}\Vert \cos \theta$, where $\theta$ is the angle between them — so it measures how much two vectors point the same way.

Positive dot product: angle under 90°, vectors broadly agree. Zero: perpendicular (orthogonal), unrelated directions. Negative: they point in opposing directions. Cosine similarity is the cosine above; it ignores length and is used to compare documents, word embeddings and user profiles.

Dot products are everywhere in ML: a neuron computes ${\mathbf{w}}^{\top }\mathbf{x}$, a linear model predicts ${\mathbf{w}}^{\top }\mathbf{x}+b$, and attention in transformers scores tokens by dot products of query and key vectors.