JS divergence

The Jensen–Shannon divergence compares both distributions to their average M:

Worked example with $P=[0.4,0.6]$, $Q=[0.5,0.5]$: $M=[0.45,0.55]$. Then $D_{KL}(P\Vert M)\approx 0.005$ and $D_{KL}(Q\Vert M)\approx 0.0045$, so $D_{JS}=\frac{1}{2}(0.005+0.0045)\approx 0.00475$.

JS is symmetric and bounded: for two non-overlapping distributions it equals $\log 2$ instead of blowing up, so it is more stable than KL — but a constant value also gives no useful gradient.