Convolutions

A convolution slides a small window — the filter or kernel — over an image and, at each position, multiplies overlapping values and sums them. The result is a feature map that responds to edges, textures and patterns.

Lecture example: the 4×4 image with rows $[1,2,3,4],[5,6,7,8],[9,10,11,12],[13,14,15,16]$ and the 2×2 kernel $\left[\begin{array}{cc}1& 0\\ 0& -1\end{array}\right]$. The first output is $(1\times 1)+(0\times 2)+(0\times 5)+(-1\times 6)=-5$, and every position gives $-5$, so the output is a 3×3 matrix of $-5$s.

Variants: standard 2D convolution for images; 1D for time series, audio and text; 3D for video and volumetric medical scans; dilated convolution (gaps between filter elements to enlarge the receptive field, e.g. DeepLab); transposed convolution for learned upsampling in generators and segmentation decoders; grouped convolution (ResNeXt, AlexNet); 1×1 pointwise convolution for channel mixing and bottlenecks; and depthwise separable convolution, one filter per channel followed by a pointwise step.

A Gaussian kernel is a fixed filter for smoothing: $G(x,y)=\frac{1}{2\pi {\sigma }^2}{e}^{-(x^2+y^2)/2{\sigma }^2}$. With $\sigma =1$ the centre weight is largest and weights fall off symmetrically, emphasising nearby pixels.