Bayes' theorem

Updating a prior belief with evidence to get a posterior.

Bayes' theorem reverses a conditional probability:

P (H ∣ E) = \frac{P ( E ∣ H ) P ( H )}{P ( E )}

$P (H)$ is the prior — belief in hypothesis $H$ before seeing evidence. $P (E ∣ H)$ is the likelihood — how probable the evidence is if $H$ is true. $P (H ∣ E)$ is the posterior — the updated belief. $P (E)$ normalises, usually via total probability.

Bayes' theorem underlies Naïve Bayes classifiers, Bayesian inference (putting priors on model parameters), and the interpretation of regularisation as a prior: L2 regularisation corresponds to a Gaussian prior on the weights, L1 to a Laplace prior.

Formula to remember

Lp norm

∥ x ∥_{p} = (j \sum ∣ x_{j} ∣^{p})^{1/ p}

p = 1 Manhattan, p = 2 Euclidean, p → ∞ max-norm.

Quick check · +5 XP

The dot product of (1, 2, 3) and (4, 0, −1) is…

Your notes

Saved in this browser as you type.