Entropy

Entropy measures how mixed the classes in a node are:

where $p_k$ is the fraction of examples in class $k$. A pure node (all one class) has $H=0$. For two classes, a 50/50 node has the maximum $H=1$ bit. With $K$ classes, the maximum is ${\log }_{2}K$.

An alternative impurity measure is the Gini impurity, $G=1-{\sum }_k{p}_k^2$ (used by CART). It behaves very similarly and is slightly faster to compute.