Definition
A forger tries to paint fake banknotes, and a bank inspector tries to catch them. Each time the inspector spots a flaw, the forger improves; each time the forger improves, the inspector looks harder. Generative adversarial networks pit two neural networks against each other in exactly this game, until the fakes look real.
A generative adversarial network (GAN), introduced by Ian Goodfellow and colleagues in 2014, has two networks. The generator G takes random noise z and produces a fake sample G(z). The discriminator D takes a sample and outputs the probability that it is real. D is trained to tell real from fake; G is trained to make D call its fakes real. This is a two-player minimax game. When training succeeds, the generator's samples follow the same distribution as the real data and the discriminator can do no better than guessing, outputting about 0.5. GANs produced the first strikingly realistic synthetic faces (StyleGAN, 2019).
In simple words
Picture a student who writes essays and a teacher who must say whether each essay was written by the student or copied from a famous author. The student keeps practising until the teacher cannot tell the difference. In a GAN, the student is the generator and the teacher is the discriminator, and both learn at the same time.
Overview
- A GAN trains a generator and a discriminator against each other.
- The generator turns random noise into fake samples; the discriminator scores how real a sample looks.
- At balance, fakes match the real data and the discriminator outputs about 0.5.
- Ian Goodfellow and colleagues introduced GANs in 2014.
- Training is unstable and can suffer mode collapse.
- Deepfakes made with generative models raise serious ethical and legal issues.
Why it matters
GANs showed that neural networks could create convincing images, and they are used for image-to-image translation (sketch to photo, day to night), super-resolution, data augmentation and synthetic data. They also explain the risks of deepfakes. Diffusion models have now overtaken GANs for most image generation, but the adversarial idea remains important.
Key terms
- Generator
- The network that turns random noise into fake samples.
- Discriminator
- The network that scores how likely a sample is to be real.
- Latent noise (z)
- The random input vector the generator transforms; different z give different samples.
- Adversarial training
- Training two models against each other in a game.
- Mode collapse
- The generator produces only a few kinds of output, ignoring the variety in real data.
- Deepfake
- Realistic synthetic media of a real person, often made without consent.
Types
There are many GAN variants. These types are the most important.
1. Original (vanilla) GAN
A generator and discriminator, usually dense networks, trained with the minimax loss.
- How it works: The discriminator maximises log D(x) + log(1 − D(G(z))); the generator is usually trained to maximise log D(G(z)).
- Example: A tiny GAN learning to produce numbers from N(4, 1.25), reaching N(4.01, 1.25).
- When to use it: Use it to learn how adversarial training works.
2. DCGAN
Deep convolutional GAN (Radford et al., 2015) using convolutions in both networks.
- How it works: Transposed convolutions upsample noise into images; batch normalisation and careful design made training stable.
- Example: Generating 64 × 64 images of bedrooms and faces.
- When to use it: Use it as the basic architecture for image GANs. Its design rules still guide image GANs today.
3. Conditional GAN
A GAN that also receives a label or input image to control what it generates.
- How it works: Both networks see the condition, so the generator learns to produce samples of the requested kind.
- Example: Generating a chosen digit, or turning sketches into photos with Pix2Pix (2017).
- When to use it: Use it when you need control over the output.
4. CycleGAN
A GAN for translating between two image domains without paired examples (Zhu et al., 2017).
- How it works: Two generators map A to B and B to A, and a cycle-consistency loss requires A → B → A to return the original.
- Example: Turning summer landscapes into winter ones, or horses into zebras.
- When to use it: Use it for style and domain translation when paired data is unavailable.
5. StyleGAN and super-resolution GANs
High-quality GANs for realistic faces (StyleGAN, 2019) and for adding detail to low-resolution images (SRGAN, 2017).
- How it works: StyleGAN controls coarse and fine features at different layers; SRGAN combines adversarial and perceptual losses.
- Example: Photorealistic faces of people who do not exist; upscaling old video frames.
- When to use it: Use them for high-quality image synthesis and enhancement, with clear labels on synthetic content.
Explained in detail
The adversarial game
Training alternates between the two players. First, show the discriminator a batch of real samples labelled 1 and a batch of the generator's fakes labelled 0, and update D to classify them better. Then generate a fresh batch of fakes and update G so that D's output on them moves towards 1, passing gradients back through D into G without changing D. In practice the generator usually maximises log D(G(z)) rather than minimising log(1 − D(G(z))), which gives stronger gradients early on when the fakes are poor. The ideal end point is a balance: the generator's distribution matches the real one and D outputs 0.5 everywhere, because it truly cannot tell.
Famous GAN variants
| Variant | Idea | Use |
|---|---|---|
| DCGAN (2015) | Convolutional generator and discriminator | Stable image generation |
| Conditional GAN | Both networks also receive a label or image | Generate a chosen digit or class |
| Pix2Pix and CycleGAN | Translate one image domain to another | Maps to satellite photos, horses to zebras |
| SRGAN | Generate high-resolution detail | Upscaling photos |
| StyleGAN (2019) | Controls style at different scales | Photorealistic faces |
Why GANs are hard to train, and responsible use
GAN training is a balancing act. If the discriminator becomes too strong, the generator receives almost no useful gradient; if it is too weak, the generator learns nothing. The two can oscillate instead of settling, and mode collapse can make the generator produce the same few outputs. Evaluating GANs is also hard, because there is no single loss that says how good the samples are; researchers use measures such as the Fréchet Inception Distance. Because GANs and their successors can create realistic faces and voices, they raise serious concerns about deepfakes, fraud and harassment. Many countries, including India under its IT rules, require platforms to act against impersonation and non-consensual synthetic media, and responsible developers label synthetic content and add watermarks.
The code below first shows the discriminator's view: against fakes drawn from N(0, 1) it is 96 per cent accurate, but as the fakes approach the real N(4, 1.25) its accuracy falls to 82, 60 and finally 48 per cent, no better than guessing. It then trains a tiny GAN in which the generator starts by producing N(0, 1) and within 4,000 steps produces N(4.01, 1.25), with the discriminator's average score for fakes at 0.49.
How it works
- Collect real samples and decide on the generator's noise input.
- Update the discriminator on a batch of real and fake samples.
- Update the generator so the discriminator scores its fakes as real.
- Repeat, watching both losses and looking at samples regularly.
- Evaluate sample quality and variety, and label synthetic outputs.
The maths
Suppose the discriminator gives a real photo D(x) = 0.9 and a fake D(G(z)) = 0.2. Its loss is −ln 0.9 − ln(1 − 0.2) ≈ 0.105 + 0.223 = 0.33, fairly low: it is doing well. The generator's loss is −ln 0.2 ≈ 1.61, high: it is being caught. After more training, if both scores move to 0.5, the discriminator's loss becomes −ln 0.5 − ln 0.5 ≈ 1.39 and the generator's −ln 0.5 ≈ 0.69. That is the balance point, where the discriminator can only guess.
In code
import numpy as np
from sklearn.linear_model import LogisticRegression
rng = np.random.default_rng(1)
sig = lambda s: 1 / (1 + np.exp(-np.clip(s, -30, 30)))
# Part 1: a discriminator judging better and better fakes. Real data come from N(4, 1.25)
real = rng.normal(4, 1.25, 2000)
for mean, sd in [(0, 1), (2, 1), (3.5, 1.1), (4, 1.25)]:
fake = rng.normal(mean, sd, 2000)
x = np.concatenate([real, fake])
y = np.r_[np.ones(2000), np.zeros(2000)]
feats = np.column_stack([x, x ** 2])
idx = rng.permutation(4000)
acc = LogisticRegression(max_iter=1000).fit(feats[idx[:3000]], y[idx[:3000]]).score(feats[idx[3000:]], y[idx[3000:]])
print(f"fakes from N({mean}, {sd}): discriminator accuracy {acc:.2f}")
# Part 2: a tiny GAN. Generator G(z) = a*z + b; discriminator D(x) = sigmoid(w1*x + w2*x^2 + c) on x / 4
a, b, w1, w2, c = 1.0, 0.0, 0.0, 0.0, 0.0
for step in range(4001):
for _ in range(3): # three discriminator updates
xr = rng.normal(4, 1.25, 128) / 4
xf = (a * rng.normal(0, 1, 128) + b) / 4
dr, df = sig(w1 * xr + w2 * xr ** 2 + c), sig(w1 * xf + w2 * xf ** 2 + c)
w1 -= 0.5 * (np.mean(-(1 - dr) * xr) + np.mean(df * xf))
w2 -= 0.5 * (np.mean(-(1 - dr) * xr ** 2) + np.mean(df * xf ** 2))
c -= 0.5 * (np.mean(-(1 - dr)) + np.mean(df))
z = rng.normal(0, 1, 128) # one generator update: try to fool D
xf = (a * z + b) / 4
df = sig(w1 * xf + w2 * xf ** 2 + c)
k = -(1 - df) * (w1 + 2 * w2 * xf) / 4
a -= 0.05 * np.mean(k * z)
b -= 0.05 * np.mean(k)
if step in (0, 500, 1000, 2000, 4000):
print(f"step {step:>4}: generator makes N({b:.2f}, {abs(a):.2f}); real data is N(4, 1.25)")
print(f"Discriminator's average score for fakes at the end: {df.mean():.2f} (0.5 means it cannot tell)")Output
fakes from N(0, 1): discriminator accuracy 0.96 fakes from N(2, 1): discriminator accuracy 0.82 fakes from N(3.5, 1.1): discriminator accuracy 0.60 fakes from N(4, 1.25): discriminator accuracy 0.48 step 0: generator makes N(0.00, 1.00); real data is N(4, 1.25) step 500: generator makes N(4.02, 1.30); real data is N(4, 1.25) step 1000: generator makes N(4.03, 1.06); real data is N(4, 1.25) step 2000: generator makes N(4.02, 1.27); real data is N(4, 1.25) step 4000: generator makes N(4.01, 1.25); real data is N(4, 1.25) Discriminator's average score for fakes at the end: 0.49 (0.5 means it cannot tell)
Try it yourself
One round of GAN training
Put the steps in order.
- 1Generate fresh fakes and score them with the discriminator
- 2Show the discriminator real samples and fakes
- 3Update the discriminator to tell them apart
- 4Update the generator so its fakes score as real
- 5Sample random noise and generate fakes
Responsible or harmful use?
Judge each use of generative models.
Restoring old family photos with consent
Cloning a friend's voice to trick their bank
Creating synthetic medical images, clearly labelled, to train a diagnosis model
Making a fake video of a politician saying something they never said
Common mistakes
- Letting the discriminator become far stronger than the generator.
- Judging a GAN only by its loss values instead of looking at the samples.
- Publishing synthetic media of real people without consent or labels.
Check yourself
Summary
- A GAN trains a generator and a discriminator against each other.
- At balance the fakes match the real distribution and the discriminator outputs about 0.5.
- GANs are hard to train and can be misused for deepfakes; use consent and labels.
Finished reading?
Sources: Goodfellow et al. (2014), Generative Adversarial Networks · Google for Developers: Generative Adversarial Networks course