ROC curves and AUC

A ROC curve plots the true positive rate $\text{TPR}=TP/(TP+FN)$ against the false positive rate $\text{FPR}=FP/(FP+TN)$ for every possible threshold. Threshold 1 gives the point (0, 0); threshold 0 gives (1, 1).

A random classifier lies on the diagonal. A perfect one goes straight up to (0, 1) and then across. The closer the curve hugs the top-left corner, the better.

The area under the curve (AUC) summarises it in one number: 1 is perfect, 0.5 is random guessing. It has a neat meaning — the probability that a random positive example is scored higher than a random negative one. It is threshold-independent and insensitive to class balance.