B7 · Data Literacy and Visualisation

Sample and Population

A population is the whole group you want to learn about. A sample is the smaller group you actually collect data from. A sample is representative when it contains the same mix of people as the population, so that its results can be trusted to describe the whole group.

8 min Beginner Jump to the activity

What you need: Nothing. This page starts from the beginning, with no programming and no formulas.

Definition

In 1936 a magazine asked over two million people how they would vote, and predicted the wrong winner by a landslide. A smaller poll of fifty thousand got it right. How?

A population is the whole group you want to learn about. A sample is the smaller group you actually collect data from. A sample is representative when it contains the same mix of people as the population, so that its results can be trusted to describe the whole group.

In simple words

To know whether a pot of dal needs more salt, you do not eat the whole pot. You stir it well and taste one spoonful. The spoonful is the sample; the pot is the population. Stirring is what makes the spoonful representative. Taste only from the top without stirring and you may get a very different answer.

Why it matters

Almost all surveys, polls and studies use samples, because asking everyone is too slow and costly. AI training data is also a sample of the world. A sample that leaves people out produces conclusions, and AI systems, that fail those people.

Key terms

Population
The entire group you want to understand.
Sample
The part of the population you actually collect data from.
Representative sample
A sample with the same mix of people as the population.
Random sample
A sample in which every member of the population has an equal chance of being chosen.
Sampling bias
A systematic difference between the sample and the population.
Margin of error
How far a sample's result may differ from the true population value, because of chance.

Explained in detail

The poll that failed

In 1936 the Literary Digest, a popular American magazine, mailed questionnaires to about ten million people and received around 2.4 million replies. It predicted that Alf Landon would defeat President Franklin Roosevelt easily. Roosevelt won one of the largest victories in American history. The magazine had drawn its names mainly from telephone directories and car registration lists. In the middle of the Great Depression, people who owned telephones and cars were wealthier than average, and wealthier voters favoured Landon. Many people also did not reply, and those who did were not typical. Meanwhile George Gallup, using a much smaller but carefully chosen sample, predicted Roosevelt's win. Size did not save a biased sample.

How to choose a fair sample

MethodHow it worksStrength
Simple randomDraw names by lottery from the full listEveryone has an equal chance
StratifiedSplit the population into groups, then sample from each in proportionEvery group is properly represented
SystematicChoose every tenth name on a listSimple and spread out
ConvenienceAsk whoever is easiest to reachQuick, but often biased

How big should a sample be?

Bigger samples reduce the effect of chance, but only up to a point, and only if they are representative. Surprisingly, the size needed depends far more on how precise you want the answer to be than on how big the population is. National opinion polls in large countries often use a few thousand people, and a carefully chosen sample of about a thousand can usually estimate a percentage to within about three percentage points either way. This allowance for chance is called the margin of error. It does not cover bias: a biased sample of a million is still wrong.

Samples and AI

An AI model's training data is a sample of the world it will be used in. If a speech model's sample contains mostly city voices, or a face model's sample mostly lighter skin, it will perform worse for those left out, exactly as the 1936 poll misjudged the voters it missed. Asking "is this sample representative of the people the system will serve?" is one of the most important checks in building fair AI.

How it works

  1. Define the population exactly: whom do you want to learn about?
  2. Get a list of the population if possible.
  3. Choose people randomly, or randomly within important groups.
  4. Make the sample big enough for the precision you need.
  5. Compare the sample's mix with the population and note any gaps.

Try it yourself

The poll with two million answers

1936, United States: who will win the presidential election?

Representative or biased?

Would each sample give a fair picture of the whole school?

  • Ten students drawn by lottery from every class

  • Your own friends

  • Every tenth name on the full school register

  • Students who reply to an optional online poll

  • Only the students in the cricket team

Common mistakes

  • Believing a bigger sample is always better, even if it is biased.
  • Asking only people who are easy to reach.
  • Treating an open online poll as the view of the public.

Check yourself

Question 1

1. What is a population in statistics?

Question 2

2. Why did the 1936 magazine poll fail despite 2.4 million replies?

Question 3

3. What makes a random sample fair?

Summary

  • A sample is a part used to learn about the whole population.
  • Representative samples, often chosen randomly, give trustworthy results.
  • Size cannot fix bias, and AI training data is a sample too.

Finished reading?

Sources: Pew Research Center: Writing survey questions · AI4K12: Five Big Ideas in AI