Probability basics

A sample space $\Omega$ is the set of all possible outcomes of an experiment (for a die: $\{1,\dots ,6\}$). An event is a subset of outcomes ("an even number"). A probability $P$ assigns each event a number in $[0,1]$.

The axioms: $P(A)\ge 0$; $P(\Omega )=1$; and for disjoint events, $P(A\cup B)=P(A)+P(B)$. From these follow the complement rule $P(A^c)=1-P(A)$ and the general addition rule $P(A\cup B)=P(A)+P(B)-P(A\cap B)$.

Events are independent if $P(A\cap B)=P(A)P(B)$ — knowing one happened tells you nothing about the other. Independence is a strong assumption that many models make (for example, that training examples are drawn i.i.d.).