Naïve Bayes

Naïve Bayes applies Bayes' theorem with a strong simplifying assumption: features are conditionally independent given the class. Then the likelihood factorises:

Training just means counting: estimate each class prior $P(y)$ and each per-feature likelihood $P(x_j\mid y)$ from frequencies. That makes it extremely fast and good with small data and many features.

• Gaussian NB: continuous features, each modelled as a normal distribution per class.
• Multinomial NB: word counts — the classic spam filter.
• Bernoulli NB: binary present/absent features.

The independence assumption is almost always false ('free' and 'money' co-occur in spam), yet the class ranking is often still right — Naïve Bayes classifies well even when its probabilities are poorly calibrated.