Datasets, features and labels

A dataset is a table. Each row is one example (also called a sample, instance or observation). Each column is a feature (attribute, variable) describing that example. One special column — the thing we want to predict — is the label or target.

We usually write the features as a matrix $X\in {\mathbb{R}}^{n\times d}$ with $n$ examples and $d$ features, and the labels as a vector $\mathbf{y}\in {\mathbb{R}}^n$. Example $i$ is the row ${\mathbf{x}}_i$ with label $y_i$.

Features can be numerical (age, income), categorical (city, colour) or ordinal (low / medium / high). Categorical features are usually turned into numbers with one-hot encoding. Images, audio and text are also features — just very many of them (every pixel or every token).

Feature engineering — creating useful features such as ratios, dates split into weekday and month, or log-transformed amounts — often matters more than the choice of algorithm for tabular data.