Neuron
Weighted sum through an activation φ.
← Machine Learning learning path0%
15
Neural & generative models
Introductory Neural Networks
From a single neuron to multilayer networks trained with backpropagation, plus convolutional networks.
Not started yet
Start Quiz mastery0%
What you’ll take away
- A neuron is a weighted sum through a non-linear activation.
- Hidden layers make non-linearly separable problems (like XOR) learnable.
- Backpropagation applies the chain rule backwards to get every gradient efficiently.
- CNNs use convolution, ReLU and pooling to learn image features.
Major concepts (syllabus)
Artificial neurons; perceptron; linear separability; multilayer perceptrons; hidden layers; activation functions; sigmoid; tanh; ReLU; forward propagation; loss functions; backpropagation; computational graphs; gradient-based learning
Lessons
- Artificial neuronsA weighted sum followed by an activation.7 min
- The perceptronThe first trainable neuron and its update rule.8 min
- Linear separabilityWhat a single perceptron can and cannot learn (XOR).7 min
- Multilayer perceptrons and hidden layersStacking layers to learn non-linear functions.9 min
- Activation functions: sigmoid, tanh, ReLUTheir shapes, ranges and gradients.9 min
- Forward propagationComputing outputs layer by layer.8 min
- Loss functions for networksMSE for regression, cross-entropy for classification.7 min
- Computational graphsBreaking a computation into simple nodes.8 min
- BackpropagationThe chain rule applied backwards through the graph.12 min
- Gradient-based learningPutting forward pass, backprop and an optimiser together.7 min
- Convolutions SlidesSliding a small learnable filter over an image to extract features.10 min
- Padding SlidesAdding pixels around an image to control output size and keep edge information.6 min
- Pooling SlidesDownsampling feature maps to cut computation, reduce overfitting and add translation invariance.9 min
- Key components of a CNN SlidesConvolution, ReLU, pooling, fully connected layers, softmax and the loss.10 min
Key formulas
Perceptron update
Adjust weights only on mistakes.
Sigmoid / tanh / ReLU
Ranges (0,1), (−1,1) and [0,∞).
Layer forward pass
Each layer transforms the previous one.
Backpropagation
Errors flow backwards via the chain rule.
2D convolution
Slides
Slide kernel K over image I and sum products.
Convolution output size
Slides
Input n, padding p, kernel k, stride s.
2D Gaussian kernel
Slides
Smoothing filter; largest weight at the centre.
L2 pooling
Slides
L2 norm of each pooling region R.
Key terms
- Activation function
- Non-linear function applied to a neuron's weighted sum, such as ReLU, sigmoid or tanh.
- Adam
- Optimiser that adapts each parameter's step size using running averages of gradients and squared gradients.
- Backpropagation
- Computing gradients of the loss for every weight by applying the chain rule backwards through the network.
- Convolution
- Sliding a small learnable filter over an input and summing element-wise products to build a feature map.
- Dropout
- Regularisation that randomly switches off neurons during training.
- Padding
- Adding border pixels around an input so convolution keeps size and edge information.
- Perceptron
- Single neuron with a step activation, trained by updating weights on mistakes.
- Pooling
- Downsampling a feature map by taking the max or average over small regions.
- ReLU
- max(0, z): the default hidden-layer activation.
Practice this topic8 questions · earn XP and build your streak