Skip to content
ExamSpine

Hybrid generative models: VAEs, flows and combinations

Real systems are rarely one model. They combine parts so that each covers another's weakness, at the price of a more complex system.

Challenge 4HL only

The trade-off a hybrid tries to beat

Each family of models is good at two of three things:

FamilyHigh qualityVaried outputFast to generate
GANYesOften not (mode collapse)Yes
Variational autoencoder (VAE)Often blurryYesYes
DiffusionYesYesNo (many steps)

Key idea: A hybrid model combines families so the system gets closer to all three at once. Researchers call this the generative learning trilemma.

Two other families, broadly

  • A variational autoencoder (VAE) has an encoder that compresses an image into a small set of numbers describing a probability distribution in latent space, and a decoder that rebuilds an image from a point sampled from it. It is trained to rebuild images well while keeping the latent space smooth and organised. Generation is one fast pass, but images tend to be blurry.
  • A flow-based model is a chain of transformations that can be run exactly backwards (invertible). It maps simple noise to images and images back to noise, and can calculate exactly how likely an image is. The invertibility limits the architecture, so classic flows need very large models for good images. A newer idea, flow matching, trains a network to move noise to images along straight paths and is used in some recent image generators.

You need a broad understanding of these two: what they are, and their main strength and weakness.

Common hybrid designs

  1. Latent diffusion (VAE + diffusion + text encoder). A VAE-style autoencoder compresses each image to a latent grid many times smaller. Diffusion runs on that grid, and the decoder turns the result back into pixels. Every step becomes much cheaper, which is why most popular text-to-image systems work this way. Cost: fine detail depends on the decoder, which can blur small text or patterns.
  2. Adversarial distillation (diffusion + GAN). A fast "student" model is trained to match a slow diffusion "teacher" in 1 to 4 steps, with a discriminator judging its output to keep it sharp. Cost: a complex training process, and often less variety than the teacher.
  3. GAN refiner or upscaler. Diffusion makes a good image at modest resolution; a GAN-based super-resolution model enlarges it and adds crisp texture in one fast pass. Cost: the upscaler can invent detail that was not there.
  4. Diffusion with a GAN discriminator during training. Adding an adversarial loss lets the denoiser take much larger steps without blurring.

How to evaluate a hybrid

Use the same headings every time, and always for the company in the question:

  • Quality of images (sharpness, correctness of the product).
  • Diversity across many generations.
  • Speed per image and throughput per GPU.
  • Training stability and how much expert tuning it needs.
  • Complexity: more components means more to build, test, update and debug, and more places for something to fail.
  • Cost: training and running.

Key idea: A hybrid is better only if what it gains matters more to this company than the complexity it adds. A small team may be wiser to use one well-supported model than to maintain three.

Key terms

Hybrid model
A generative system that combines different kinds of model, such as an autoencoder with diffusion, or diffusion with a GAN.
Variational autoencoder (VAE)
An encoder that maps images to a smooth latent distribution and a decoder that rebuilds them. Fast, but images can be blurry.
Flow-based model
A generative model built from exactly invertible transformations, mapping noise to images and back, with exact likelihoods.
Latent diffusion
Running diffusion on a compressed latent version of the image made by an autoencoder, then decoding it, so each step is much cheaper.
Distillation
Training a smaller or faster "student" model to copy a larger "teacher", for example a diffusion model that needs 1 to 4 steps instead of 50.
Super-resolution (upscaling)
Enlarging an image while adding plausible detail, often with a GAN-based model.
Generative learning trilemma
The observation that GANs, VAEs and diffusion models each achieve only two of high quality, variety and fast generation.
Latent space
A compressed space of numbers that represents images; nearby points give similar images.

See all 65 terms in the glossary

Exam-style questions

In the style of Paper 1 Section B. Write your answer first, then open the marking points and mark yourself honestly. The questions use our own company, Hearth & Oak: an online furniture shop that wants to generate room scenes around its products.

Worked example · Explain · AO1 · HL

[4 marks]

Explain why most text-to-image diffusion systems run the diffusion process in a latent space rather than on the pixels of the image.

How to approach it

This is a "why" question: the answer is about cost per step times number of steps. Mention the price (detail depends on the decoder) for the last mark.

0 words

Marking points and model answer

A model answer

A variational autoencoder first compresses the image into a latent grid many times smaller than the pixel image, and a decoder turns latents back into pixels. Running diffusion on the latent means each denoising step processes far fewer values, so it needs much less computation and memory. Because generation repeats that step many times, the saving is multiplied, making both training and generation much faster and cheaper. The latent keeps the important structure of the image, so quality stays high, although very fine detail now depends on how good the decoder is.

For each point, does your answer make it? Be strict: a marker would be.

  • Point 1: An autoencoder (VAE) compresses the image into a much smaller latent representation, and decodes it back afterwards. (1 mark)
  • Point 2: Each denoising step on the smaller latent needs far less computation and memory. (1 mark)
  • Point 3: Since diffusion needs many steps, the saving is multiplied, making generation (and training) much faster/cheaper. (1 mark)
  • Point 4: The latent keeps the meaningful structure, so quality stays high; any fine detail is restored by the decoder (with a possible loss of tiny details). (1 mark)

Earns nothing

  • "It is more efficient" with no explanation of why.

Mark every point to see your total.

Question 1 · Outline · AO1 · HL

[2 marks]

Outline one strength and one weakness of a variational autoencoder (VAE) as an image generator.

0 words

Marking points and model answer

A model answer

A VAE generates an image in one fast pass of its decoder and trains stably, but its images tend to be blurrier and less detailed than those from GANs or diffusion models.

For each point, does your answer make it? Be strict: a marker would be.

  • Point 1: Strength: fast single-pass generation / smooth, organised latent space / stable training / varied output. (1 mark)
  • Point 2: Weakness: images tend to be blurry / less sharp than GANs or diffusion. (1 mark)

Mark every point to see your total.

Question 2 · Discuss · AO3 · HL

[6 marks]

Hearth & Oak is considering adding a GAN-based upscaler after its diffusion model, so it can generate at a lower resolution and enlarge the result. Discuss this design.

0 words

Marking points and model answer

A model answer

Generating at a lower resolution would cut the time and GPU cost of each diffusion run a lot, and a GAN upscaler adds sharp texture in one quick pass, so customers would get crisp images faster. The danger for Hearth & Oak is that an upscaler invents plausible detail. A fabric weave or wood grain that the upscaler made up could make a sofa look unlike the one the customer will receive, which breaks the trust the brand depends on. It is also one more model to train, test and keep updated. A good compromise is to upscale only the room around the product and composite the real studio photo of the product on top. With that safeguard the design is worth adopting; without it, the risk of misleading customers outweighs the speed.

For each point, does your answer make it? Be strict: a marker would be.

  • Point 1: Benefit: diffusion at lower resolution is much faster/cheaper per image. (1 mark)
  • Point 2: Benefit: the GAN upscaler is a single fast pass and adds sharp texture. (1 mark)
  • Point 3: Risk: the upscaler can invent (hallucinate) detail, so the product may not look like the real one (misleading customers). (1 mark)
  • Point 4: Risk: an extra component to train, test and maintain; two places for errors; GAN upscalers can add artefacts. (1 mark)
  • Point 5: A mitigation, for example paste the real product photo in, or only upscale the background. (1 mark)
  • Point 6: A justified conclusion for this company. (1 mark)

Mark every point to see your total.

Question 3 · Distinguish · AO1 · HL

[2 marks]

Distinguish between a diffusion model and a flow-based model.

0 words

Marking points and model answer

A model answer

A diffusion model is trained to reverse a noising process approximately, by removing a little predicted noise at each of many steps, whereas a classic flow-based model is built from exactly invertible transformations, so it can turn noise into images and images back into noise and calculate exactly how likely an image is.

For each point, does your answer make it? Be strict: a marker would be.

  • Point 1: Diffusion learns to remove noise step by step; the noising process is not exactly reversible, so it learns to approximate the reverse. (1 mark)
  • Point 2: A (classic) flow-based model is a chain of exactly invertible transformations, so it can map in both directions and compute exact likelihoods. (1 mark)

Mark every point to see your total.

Glossary

65 key terms, in plain English

Practice questions

Paper 1 Section B style, with full answers

All case study topics

  1. Challenge 1

    SL and HL

    How diffusion models make an image

    Noise, denoising, the network that predicts the noise, and how a text prompt steers it.

  2. Challenge 2

    SL and HL

    Training data: ethics, ownership and bias

    Consent, copyright and attribution for training images, and how curation choices shape what the model makes.

  3. Challenge 3

    HL only

    GANs: a generator against a discriminator

    How two networks train against each other, and why that training is unstable.

  4. Challenge 4

    HL only

    Hybrid models and the trade-offs between them

    Systems that combine diffusion, GANs and autoencoders, judged on quality, speed, diversity and complexity.

Written by Spine in our own words. Spine is not affiliated with or endorsed by the International Baccalaureate. Every question, set of marking points and model answer here is our own, set in our own made-up company. Facts about the case study itself come from public guides and are marked “reported”. Last checked 2026-10-23.