GAN game (general)
Slides
D maximises the discrepancy; G minimises it.
← Machine Learning learning path0%
16
Neural & generative models
Advanced Topics
Learning to generate data: GANs as a two-player game over a divergence, and Boltzmann machines.
Not started yet
Start Quiz mastery0%
What you’ll take away
- Generative models learn the data distribution and can sample from it.
- GANs pit a generator against a discriminator in a min–max game.
- The divergence the GAN implicitly minimises (f-divergence vs IPM) shapes its stability.
- Boltzmann machines are energy-based; RBMs make them trainable.
Major concepts (syllabus)
Generative Models, Generative Adversarial Networks (GANs), Boltzmann Machine, Restricted Boltzmann Machine
Lessons
- Generative modelingLearning a model of the data distribution so you can sample new data.8 min
- GANs: generator and discriminator SlidesA generator turns noise into fake samples; a discriminator tries to tell real from fake.10 min
- The GAN min–max game SlidesD maximises a discrepancy between real and generated data; G minimises it.10 min
- The distribution-level view: a two-sample test SlidesDeciding whether real and generated samples come from the same distribution.10 min
- Classes of GANs SlidesF-divergence GANs versus IPM-based GANs, and why the choice matters.8 min
- Problems with basic GANs SlidesLack of synchronisation, mode collapse and unstable training.8 min
- Boltzmann machinesEnergy-based models with stochastic binary units fully connected to each other.9 min
- Restricted Boltzmann machinesVisible and hidden layers with no within-layer connections, trained with contrastive divergence.9 min
Key formulas
Generative model goal
Slides
Make the generated distribution match the data.
Vanilla GAN objective
The original JS-based formulation.
Two-sample test
Slides
Distribution-level view of the discriminator.
Boltzmann distribution
Lower energy means higher probability; Z normalises.
RBM energy
Visible–hidden interactions only.
Key terms
- Boltzmann machine
- Energy-based network of stochastic binary units whose configurations follow the Boltzmann distribution.
- Discriminator
- GAN network that tries to tell real samples from generated ones.
- GAN
- Generative adversarial network: a generator and a discriminator trained against each other.
- Generator
- GAN network that turns random noise into synthetic samples.
- Mode collapse
- GAN failure where the generator produces only a few kinds of output.
- RBM
- Restricted Boltzmann machine: visible and hidden layers with no within-layer connections.
Practice this topic7 questions · earn XP and build your streak