Parameters vs. hyperparameters

Parameters are the numbers the learning algorithm adjusts during training: the weights $\mathbf{w}$ and bias $b$ of a linear model, or the millions of weights in a neural network. You never set them by hand — the optimiser finds them by minimising the loss.

Hyperparameters are choices made *before* training that control how learning happens: the learning rate $\eta$, the number of layers, the regularisation strength $\lambda$, the $k$ in k-nearest neighbours, the maximum depth of a tree, the batch size, the number of epochs.

Hyperparameters are tuned by trying several values and comparing performance on a validation set (or with cross-validation) — never on the test set. Common strategies are grid search, random search and Bayesian optimisation.