Decision boundaries

A decision boundary is the set of inputs where the classifier is undecided — where the predicted class flips. Points on one side are labelled positive, on the other negative.

For logistic regression with threshold 0.5, the boundary is where $\sigma (z)=0.5$, i.e. ${\mathbf{w}}^{\top }\mathbf{x}+b=0$. That is a line in 2-D, a plane in 3-D, a hyperplane in general. $\mathbf{w}$ is perpendicular to it, and the distance from the boundary relates to the model's confidence.

To get curved boundaries, add non-linear features (squares, products) — a linear boundary in the expanded feature space is curved in the original. Or use inherently non-linear models: kernel SVMs, trees, k-NN, neural networks.