Skip to content
ExamSpine

Dataset curation and bias in generated images

A model can only make what its data taught it. Curation decides who and what appears, how often, and in what roles.

Challenge 2SL and HL

How bias gets in

Key idea: A generative model reproduces the patterns in its training data, including the unfair ones, and often exaggerates them. Common sources:

  • Representation bias. Some groups, places or styles appear far more often online, so the model makes them by default. Ask for "a family living room" and you may get one kind of home, one kind of family.
  • Caption bias. Captions written by people carry their assumptions: which jobs get labelled with which genders, which neighbourhoods get called "nice".
  • Collection bias. Scraping favours content in some languages and from wealthier countries with more photos online.
  • Filtering side effects. Automatic filters that remove "unsafe" or "low quality" images can remove some groups more than others.

Why generators can amplify bias

If 70% of the living-room photos in the data are large and bright, a perfectly balanced generator would make that kind of room 70% of the time. Real generators often lean further towards the most common case, and a high guidance scale pushes harder still towards what the prompt "usually" means.

There is also a feedback loop: generated images are posted online, scraped into the next model's training data, and the same defaults become even more common.

Curation choices and their trade-offs

ChoiceWhat it helpsWhat it costs
Deduplicate the dataLess memorisation, less over-weighting of popular imagesProcessing time
Filter for qualitySharper, cleaner outputMay remove under-represented groups or styles
Rebalance or reweightFairer variety of people and placesNeeds labels, which raise their own privacy questions
Rewrite captionsBetter prompt following, less loaded languageExpensive; the captioning model has its own bias
Use licensed or own dataClear consent and provenanceSmaller and possibly narrower

Key idea: Curation is a set of value judgements, not a neutral clean-up, so a company should write down what it chose and why. Documents such as a datasheet for the dataset and a model card record this.

Measuring bias in the output

  • Generate many images for neutral prompts ("a person relaxing at home") and count what appears.
  • Compare against a target the company has chosen and can defend (for example its real customer base or the population it sells to).
  • Have human reviewers from different backgrounds check results.
  • Re-test after every model update, because changes can bring bias back.

Key terms

Bias (in AI output)
A systematic unfairness or skew in what a model produces, usually learned from unbalanced training data.
Representation bias
Some groups, places or styles appearing much more or less often in the data than in the world the model serves.
Dataset curation
Choosing, filtering, balancing and documenting the data a model is trained on.
Deduplication
Removing repeated or near-identical images from a dataset, which reduces memorisation and over-weighting.
Rebalancing / reweighting
Making under-represented examples count more during training, or adding more of them, so the model learns a fairer spread.
Feedback loop
When generated images are posted online and scraped into later training data, reinforcing the same defaults and biases.
Model card / datasheet
A document recording how a model or dataset was made, what it is for, and its known limits and biases.
Bias audit
A structured test of a model's outputs, for example generating many images for neutral prompts and counting who and what appears.

See all 65 terms in the glossary

Exam-style questions

In the style of Paper 1 Section B. Write your answer first, then open the marking points and mark yourself honestly. The questions use our own company, Hearth & Oak: an online furniture shop that wants to generate room scenes around its products.

Worked example · Explain · AO2

[4 marks]

Almost every room Hearth & Oak's model generates is large, bright and minimalist, even when the customer asks for "a cosy family home". Explain how the training data could have caused this, and suggest one way to reduce it.

How to approach it

Cause first (what is in the data), then mechanism (the model copies the most common pattern), then a fix and how you would know it worked. Keep it about rooms: the company context earns the AO2 marks.

0 words

Marking points and model answer

A model answer

Most interior photos online are taken by professionals to sell homes or furniture, so they show large, bright, tidy rooms. If the model was trained mostly on such photos, it learned that this is what "a room" looks like and makes that by default, even pulling a "cosy family home" towards it. Hearth & Oak could add and upweight photos of smaller, lived-in homes, including its own customers' photos shared with permission, and then generate a few hundred images for prompts like "a cosy family home" to check that the variety has improved.

For each point, does your answer make it? Be strict: a marker would be.

  • Point 1: The training images over-represent this kind of room (for example professional interior and estate-agent photos dominate online). (1 mark)
  • Point 2: The model learns the most common pattern as the default and reproduces / exaggerates it. (1 mark)
  • Point 3: A valid curation fix: add or upweight images of other kinds of homes, or rebalance the dataset. (1 mark)
  • Point 4: Why it helps or how to check: the model then sees more variety; measure by generating many images for neutral prompts and counting. (1 mark)

Also accepted

  • Caption bias (photos of cosy homes rarely captioned that way) is a valid cause for point 1.

Earns nothing

  • "The AI is biased" with no cause.

Mark every point to see your total.

Question 1 · State · AO1

[2 marks]

State two sources of bias in the training data of an image generation model.

0 words

Marking points and model answer

A model answer

Some groups and places appear far more often than others in images collected from the web, and the captions attached to images carry the assumptions of the people who wrote them.

For each point, does your answer make it? Be strict: a marker would be.

  • Point 1: One source, for example over- or under-representation of groups, places or styles. (1 mark)
  • Point 2: A different source, for example biased captions/labels, collection mainly from some languages or countries, or filters that remove some groups more. (1 mark)

Earns nothing

  • "Bad data."

Mark every point to see your total.

Question 2 · Explain · AO1

[2 marks]

Explain why removing duplicate images from a training dataset is good practice.

0 words

Marking points and model answer

A model answer

Images that appear many times are much more likely to be memorised and copied in the output, and duplicates make popular images count more than others, skewing what the model learns as normal.

For each point, does your answer make it? Be strict: a marker would be.

  • Point 1: Duplicated images are more likely to be memorised and reproduced. (1 mark)
  • Point 2: Duplicates over-weight popular images, skewing the model towards them (bias) / waste training time. (1 mark)

Mark every point to see your total.

Question 3 · Discuss · AO3

[6 marks]

Hearth & Oak plans to filter its training data by removing every image an automatic quality model scores as "low quality". Discuss this plan.

0 words

Marking points and model answer

A model answer

Filtering out low-quality images should make the model's rooms sharper and better lit, which matters for a shop whose images must show products accurately, and it saves training time on blurry or irrelevant pictures. However, the quality model was itself trained on someone's idea of a "good" photo. It may score phone photos of small, cluttered or older homes as low quality far more often than professional shots of large ones. Removing them would make Hearth & Oak's generator even worse at showing the homes many customers actually live in. Hearth & Oak should still filter, but first check a sample of what the filter removes, broken down by kind of room, and lower the threshold or keep a set of real-home photos if one kind is being removed much more than others.

For each point, does your answer make it? Be strict: a marker would be.

  • Point 1: Benefit: sharper, cleaner training images, so better-looking output. (1 mark)
  • Point 2: Benefit: less training time / fewer junk or irrelevant images. (1 mark)
  • Point 3: Risk: the quality model has its own bias, so it may score some homes, places or groups as "low quality" more often. (1 mark)
  • Point 4: Risk: removing them makes the generator even less able to show those homes, increasing representation bias. (1 mark)
  • Point 5: A mitigation: check what the filter removes by category, set different thresholds, or have humans review a sample. (1 mark)
  • Point 6: A justified conclusion for this company. (1 mark)

Earns nothing

  • A conclusion with no reasoning.

Mark every point to see your total.

Glossary

65 key terms, in plain English

Practice questions

Paper 1 Section B style, with full answers

All case study topics

  1. Challenge 1

    SL and HL

    How diffusion models make an image

    Noise, denoising, the network that predicts the noise, and how a text prompt steers it.

  2. Challenge 2

    SL and HL

    Training data: ethics, ownership and bias

    Consent, copyright and attribution for training images, and how curation choices shape what the model makes.

  3. Challenge 3

    HL only

    GANs: a generator against a discriminator

    How two networks train against each other, and why that training is unstable.

  4. Challenge 4

    HL only

    Hybrid models and the trade-offs between them

    Systems that combine diffusion, GANs and autoencoders, judged on quality, speed, diversity and complexity.

Written by Spine in our own words. Spine is not affiliated with or endorsed by the International Baccalaureate. Every question, set of marking points and model answer here is our own, set in our own made-up company. Facts about the case study itself come from public guides and are marked “reported”. Last checked 2026-10-23.