Skip to content
ExamSpine

GAN training instability and how to fix it

Balancing two networks that learn against each other is hard. Know the three classic failures and what engineers do about each.

Challenge 3HL only

Failure 1: mode collapse

The generator finds a few images that fool the discriminator and keeps making only those.

  • Every customer asking for a living room gets nearly the same living room.
  • It happens because the generator is rewarded for fooling the discriminator, not for variety. If one output works, nothing pushes it to make others until the discriminator catches up, and then it jumps to another narrow set.

Key idea: Mode collapse is a loss of diversity, not of quality: each image can look excellent while the set of images is poor.

Failure 2: vanishing gradients

If the discriminator becomes too good too fast, it rejects every generated image with near-total confidence. The loss then barely changes when the generator changes slightly, so the gradient the generator learns from becomes almost zero. The generator stops improving.

Failure 3: oscillation and non-convergence

Each network keeps changing the problem the other is solving. Instead of settling at an equilibrium, the two can circle: the generator moves to fool the current discriminator, the discriminator adapts, the generator moves back. Image quality goes up and down during training, and the loss values do not tell you clearly whether things are getting better.

GANs are also very sensitive to hyperparameters such as learning rates and batch size: a setting that works for one dataset can fail completely on another.

Mitigations

TechniqueWhat it doesMainly helps with
Wasserstein loss (WGAN), with a gradient penaltyReplaces "real or fake?" with a smooth score of how far apart the two distributions areVanishing gradients, oscillation
Spectral normalisationLimits how sharply the discriminator can change its outputDiscriminator overpowering the generator
Balancing updatesDifferent learning rates for each network, or more discriminator steps per generator stepOscillation
Minibatch discriminationLets the discriminator see a whole batch, so a batch of near-identical images looks fakeMode collapse
Label smoothing, adding noise to inputsStops the discriminator becoming overconfidentVanishing gradients
Progressive growingTrains at low resolution first, then adds layers for higher resolutionsStability at high resolution
Monitoring and checkpointsTrack a quality score such as FID and sample images; keep the best saved modelChoosing a good stopping point

Key idea: No single fix guarantees stability; in practice teams combine several and watch the samples, not just the loss.

Key terms

Mode collapse
A GAN failure where the generator makes only a few kinds of output: high quality, low variety.
Vanishing gradient
When the gradient a network learns from becomes almost zero, so its weights stop changing. In GANs, caused by an overconfident discriminator.
Wasserstein loss
A GAN loss that scores how far apart the real and generated distributions are, giving smoother gradients than "real or fake".
Spectral normalisation
A limit on how sharply a discriminator's output can change, which keeps GAN training more stable.
Hyperparameter
A setting chosen before training rather than learned, such as the learning rate or batch size.
FID (Fréchet Inception Distance)
A score comparing features of many real and many generated images. Lower means the generated set is closer to the real one.
Checkpoint
A saved copy of a model's weights during training, so the best version can be kept if training later gets worse.

See all 65 terms in the glossary

Exam-style questions

In the style of Paper 1 Section B. Write your answer first, then open the marking points and mark yourself honestly. The questions use our own company, Hearth & Oak: an online furniture shop that wants to generate room scenes around its products.

Worked example · Explain · AO2 · HL

[4 marks]

Hearth & Oak's GAN produces beautiful sofas, but testers notice that almost all of them are the same shape and colour. Identify this problem and explain one technique that could reduce it.

How to approach it

Name it first (one mark for the word). Then the cause in terms of what each network is rewarded for. Then one technique, explained by how it changes those rewards.

0 words

Marking points and model answer

A model answer

This is mode collapse. The generator has found a few sofa images that the discriminator accepts as real, and because its only goal is to fool the discriminator, nothing rewards it for making other shapes or colours. Minibatch discrimination lets the discriminator compare the images within a batch rather than judging each alone. A batch of near-identical sofas then looks suspicious, because real batches are varied, so the generator is pushed to spread its output across more kinds of sofa.

For each point, does your answer make it? Be strict: a marker would be.

  • Point 1: Identifies mode collapse. (1 mark)
  • Point 2: Explains the cause: the generator found outputs that fool the discriminator and is not rewarded for variety. (1 mark)
  • Point 3: A valid technique, for example minibatch discrimination, Wasserstein loss, or unrolled updates. (1 mark)
  • Point 4: How the technique helps, for example the discriminator can spot a batch of near-identical images as fake, so the generator is pushed to vary. (1 mark)

Earns nothing

  • "Train it for longer" with no explanation.

Mark every point to see your total.

Question 1 · Explain · AO1 · HL

[3 marks]

Explain why a discriminator that is too strong can stop a GAN from learning.

0 words

Marking points and model answer

A model answer

If the discriminator becomes too good, it labels every generated image as fake with near certainty. Small changes to the generator then make almost no difference to the loss, so the gradient passed back to the generator is close to zero. With almost no signal to learn from, the generator's weights barely change and it stops getting better.

For each point, does your answer make it? Be strict: a marker would be.

  • Point 1: A strong discriminator rejects all generated images with high confidence. (1 mark)
  • Point 2: The loss then hardly changes as the generator changes, so the gradient is close to zero (vanishing gradient). (1 mark)
  • Point 3: With almost no gradient, the generator's weights barely update and it stops improving. (1 mark)

Mark every point to see your total.

Question 2 · Outline · AO1 · HL

[4 marks]

Outline two reasons why the loss values of a GAN are a poor guide to whether training is going well.

0 words

Marking points and model answer

A model answer

First, each network is judged against the other, and the other keeps changing, so a lower generator loss might only mean the discriminator has become weaker rather than the images better. Second, the losses measure how well each network does in the game, not how varied the images are, so the generator can collapse to a few outputs while the losses look fine. Teams therefore look at sample images and a score such as FID.

For each point, does your answer make it? Be strict: a marker would be.

  • Point 1: A first reason, for example each network's loss depends on the other, which keeps changing / the losses oscillate / they do not measure diversity. (1 mark)
  • Point 2: Development of that reason, for example a falling generator loss may only mean the discriminator got worse, not that images improved. (1 mark)
  • Point 3: A different second reason. (1 mark)
  • Point 4: Development of that reason, for example mode collapse can happen while the losses look fine, so sample images / FID are needed. (1 mark)

Mark every point to see your total.

Question 3 · Evaluate · AO3 · HL

[4 marks]

Hearth & Oak's engineers have limited time. Evaluate replacing the standard GAN loss with a Wasserstein loss.

0 words

Marking points and model answer

A model answer

A Wasserstein loss measures how far apart the real and generated distributions are with a smooth score, so the generator still gets a useful gradient even when the critic is strong. Its value also tracks image quality more closely, which would help Hearth & Oak's small team see whether training is improving. However, it needs a gradient penalty, which adds computation to each step, and typically several critic updates per generator update, so training is slower and still needs some tuning. For a team with little time to babysit training runs, it is worth it: fewer failed runs saves more time than the slower steps cost.

For each point, does your answer make it? Be strict: a marker would be.

  • Point 1: Benefit: gives a smooth, useful gradient even when the discriminator (critic) is strong, reducing vanishing gradients. (1 mark)
  • Point 2: Benefit: the loss value tracks image quality better, and training is less prone to collapse/oscillation. (1 mark)
  • Point 3: Cost: needs a constraint on the critic (gradient penalty or clipping), extra computation per step / more critic steps, slower training, still needs tuning. (1 mark)
  • Point 4: A justified judgement for a small team with limited time. (1 mark)

Mark every point to see your total.

Glossary

65 key terms, in plain English

Practice questions

Paper 1 Section B style, with full answers

All case study topics

  1. Challenge 1

    SL and HL

    How diffusion models make an image

    Noise, denoising, the network that predicts the noise, and how a text prompt steers it.

  2. Challenge 2

    SL and HL

    Training data: ethics, ownership and bias

    Consent, copyright and attribution for training images, and how curation choices shape what the model makes.

  3. Challenge 3

    HL only

    GANs: a generator against a discriminator

    How two networks train against each other, and why that training is unstable.

  4. Challenge 4

    HL only

    Hybrid models and the trade-offs between them

    Systems that combine diffusion, GANs and autoencoders, judged on quality, speed, diversity and complexity.

Written by Spine in our own words. Spine is not affiliated with or endorsed by the International Baccalaureate. Every question, set of marking points and model answer here is our own, set in our own made-up company. Facts about the case study itself come from public guides and are marked “reported”. Last checked 2026-10-23.