B7 · Data Literacy and Visualisation

Designing Good Survey Questions

A good survey question is clear, neutral and answerable. It asks about one thing at a time, uses simple words, does not push people towards an answer, and offers choices that cover every possible response fairly.

8 min Beginner Jump to the activity

What you need: Nothing. This page starts from the beginning, with no programming and no formulas.

Definition

"Don't you agree that our school canteen needs healthier food?" What is wrong with this question?

A good survey question is clear, neutral and answerable. It asks about one thing at a time, uses simple words, does not push people towards an answer, and offers choices that cover every possible response fairly.

In simple words

Asking a question is like holding a measuring tape. If the tape is stretched or bent, every measurement is wrong, however carefully you read it. A badly worded question is a bent tape: the answers measure the question, not people's real views.

Why it matters

Surveys guide decisions in schools, businesses and governments, and survey answers become training data for many AI systems. Small changes in wording can change results dramatically, so careful questions are the foundation of trustworthy data.

Key terms

Leading question
A question worded to push people towards a particular answer.
Double-barrelled question
A question that asks about two things at once.
Closed question
A question with fixed answer choices, such as yes or no.
Open question
A question people answer in their own words.
Response options
The answer choices offered in a closed question.
Pilot test
Trying a survey on a few people first to find confusing questions.

Explained in detail

Five common faults

FaultBad exampleBetter
Leading"Don't you agree the canteen needs healthier food?""How would you rate the healthiness of canteen food?"
Two things at once"Is the canteen food tasty and cheap?"Ask two separate questions
Unclear words"Do you often eat out?""How many times did you eat outside home last week?"
Missing options"How do you come to school? Bus / Car"Add walk, cycle, auto and "other"
Sensitive and unnecessary"What is your family's income?" in a canteen surveyLeave it out unless truly needed

Wording changes answers

Survey researchers have shown again and again that wording matters. In a famous series of experiments in the United States, more people agreed that the government should "not allow" certain speeches than agreed it should "forbid" them, although the two mean the same thing. People also tend to agree with whatever statement is put to them, especially if they are unsure, so "Do you agree that…" questions inflate agreement. Balanced wording that gives both sides, such as "Some people think X, others think Y. Which is closer to your view?", produces more honest answers.

Open or closed?

Closed questions are quick to answer and easy to count, which makes them good for large surveys. Open questions capture things you did not think of and the reasons behind answers, but they take longer to read and group. A good survey often uses mostly closed questions with one or two open ones, such as "Is there anything else you would like to tell us?"

Order, length and testing

  • Start with easy, interesting questions; put sensitive ones later.
  • Keep it short. People rush or give up when surveys are long.
  • Earlier questions can influence later ones, so think about the order.
  • Pilot test with five people and ask what was confusing.
  • Tell people why you are asking and keep answers private.

Before sending a survey to everyone, try it on three or four people and watch them answer. If someone hesitates, asks what a word means, or gives an answer you did not expect, the question needs rewriting. This small test, called a pilot, catches most problems while they are still cheap to fix. Keep surveys short, too: people who are bored start choosing answers at random.

How it works

  1. Write down exactly what you need to find out.
  2. Write one question per piece of information.
  3. Check each for leading words, two-in-one, unclear terms and missing options.
  4. Remove questions you do not truly need.
  5. Pilot test and revise.

Try it yourself

Fixing a biased canteen survey

The student council drafts a survey about the canteen.

Good question or faulty?

Spot the problem in each question.

  • "How many hours did you sleep last night?"

  • "Are the teachers friendly and the classes interesting?"

  • "Isn't the new library wonderful?"

  • "Which subject do you find hardest?" with all subjects listed and "none"

  • "Do you use the internet a lot?"

Common mistakes

  • Writing questions that reveal what answer you hope for.
  • Leaving out answer options people need.
  • Skipping the pilot test.

Check yourself

Question 1

1. What is wrong with "Don't you agree homework is too long?"

Question 2

2. "Is the bus clean and on time?" is a problem because…

Question 3

3. Why run a pilot test?

Summary

  • Good questions are clear, neutral, single and answerable.
  • Leading words, two-in-one questions and missing options bias results.
  • Keep surveys short and always pilot test them.

Finished reading?

Sources: Pew Research Center: Writing survey questions · DataBasic.io: tools and activities for data literacy