Lesson 7 of 13 7 min · 122 words
Hellinger distance
A symmetric, bounded divergence based on square roots of densities.
The Hellinger distance compares the square roots of the densities:
It is symmetric, satisfies the triangle inequality, and lies in : 0 for identical distributions, 1 for distributions with disjoint support. The squared Hellinger distance is an f-divergence with (up to a constant).
Taking square roots softens the effect of large probability ratios, so Hellinger is more stable than KL where one density is near zero. It is related to TV by .
Formula to remember
Total variation
Largest difference in probability assigned to any event.
Quick check · +5 XP