Parameters

Parameters $\mathbf{\theta }$ pick one specific function out of the hypothesis class. For a line $f(x)=wx+b$, the pair $(w,b)$ is the parameter vector; each choice is a different line.

Learning becomes a search in parameter space: find the $\mathbf{\theta }$ that makes the loss smallest. For linear models that space has $d+1$ dimensions; for modern networks it has millions or billions.

Parameters are initialised (often randomly for networks, at zero for linear models) and then updated by the optimiser. Good initialisation matters for deep networks: weights that are too large or too small make gradients explode or vanish.