Expectation and variance

The expectation (mean) is the probability-weighted average: $\mathbb{E}[X]={\sum }_{x}xp(x)$ for discrete and $\int xp(x)dx$ for continuous variables. It is linear: $\mathbb{E}[aX+bY]=a\mathbb{E}[X]+b\mathbb{E}[Y]$, even if $X$ and $Y$ are dependent.

The variance measures spread around the mean: $\mathrm{Var}(X)=\mathbb{E}[(X-\mu {)}^2]=\mathbb{E}[X^2]-{\mu }^2$. Its square root is the standard deviation $\sigma$, in the same units as $X$. Scaling: $\mathrm{Var}(aX+b)=a^2\mathrm{Var}(X)$.

Expectation shows up as risk (expected loss); variance shows up in the bias–variance trade-off and in why averaging many models (bagging) or many gradient samples (mini-batches) gives more stable results — averaging $n$ independent copies divides the variance by $n$.