Linear separability

Two classes are linearly separable if a single line (hyperplane) can put all of one class on one side and all of the other on the other side. A single perceptron can only learn linearly separable functions.

AND and OR are separable: one line separates (1,1) from the rest for AND. XOR is not: the points (0,1) and (1,0) are positive, (0,0) and (1,1) negative, sitting on opposite corners of a square. No line separates them.

Minsky and Papert's 1969 book highlighted this limitation, and interest in neural networks faded for over a decade (an 'AI winter').

The fix is a hidden layer: XOR = (x₁ OR x₂) AND NOT (x₁ AND x₂). Two hidden neurons compute OR and NAND; an output neuron ANDs them. Hidden layers transform the data into a space where it becomes linearly separable.