B2 · How Artificial Intelligence Works: Data and Learning

Predictions and Confidence

A prediction is the answer a model gives for a new case. Confidence is how sure the model is about that answer, often shown as a percentage. High confidence does not guarantee the answer is right.

8 min Beginner Jump to the activity

What you need: Nothing. This page starts from the beginning, with no programming and no formulas.

Definition

The model says "67 per cent dog". What does that mean?

A prediction is the answer a model gives for a new case. Confidence is how sure the model is about that answer, often shown as a percentage. High confidence does not guarantee the answer is right.

In simple words

Our animal model looks at the three most similar examples. If all three are dogs, it says dog, 100 per cent sure. If two are dogs and one is a cat, it says dog, 67 per cent sure. Like a weather forecast of 70 per cent chance of rain, it tells you how much to rely on the answer.

Why it matters

Confidence helps decide when to trust AI and when a person should check. A hospital might let AI handle cases it is very sure about and send uncertain ones to a doctor. But a model can be confidently wrong, especially on cases unlike its training data.

Key terms

Prediction
The answer a model gives for a new case.
Confidence
How sure the model is of its prediction, usually shown as a percentage.
Probability
A number between 0 and 1, or 0 and 100 per cent, that says how likely something is.
Threshold
The level of confidence at which a system takes action.
Calibration
How well a model's confidence matches how often it is actually right.

Explained in detail

A model gives two things

When you show a trained model a new case, it gives back a prediction and, usually, a measure of how sure it is. A photo app does not only say "cat". Inside, it has worked out something like: cat 91 per cent, dog 7 per cent, rabbit 2 per cent. It reports the highest one as its answer. Seeing the whole list is often more useful than seeing only the winner, because a model that says "cat 51 per cent, dog 49 per cent" is telling you it cannot really decide.

Predicting a category or a number

Predictions come in two main kinds. Sometimes the answer is a category: spam or not spam, healthy leaf or diseased leaf. Sometimes it is a number: tomorrow's temperature, the price of a house, the minutes until a delivery arrives. For a number, confidence is often shown as a range, such as "arriving in 20 to 25 minutes". A narrow range means the model is fairly sure; a wide one means it is not.

What a percentage should mean

Weather forecasters give a good example of honest confidence. When a careful forecaster says there is a 70 per cent chance of rain, it should rain on about seven out of every ten such days. If it does, the forecaster is well calibrated: the stated confidence matches reality. The same test applies to a model. Collect all the cases where it said it was 90 per cent sure. If it was right in about nine out of ten, its confidence can be trusted. If it was right in only six out of ten, it is overconfident, and its percentages mean less than they appear to.

Confidence fails on the unfamiliar

Confidence is worked out from the examples the model trained on. It measures how closely the new case resembles what the model has seen before, not whether the model is right. Show a fruit classifier a cricket ball and it may answer "apple, 96 per cent", because among the things it knows, an apple is by far the closest match. It has no way to say "this is outside everything I learned". High confidence on an unfamiliar input is one of the most common ways AI systems mislead people.

Thresholds: deciding when to act

People who build a system choose a threshold, the confidence above which it acts by itself. The right threshold depends on what a mistake costs. A photo app can label a picture "beach" at 60 per cent, because a wrong label harms nobody. A system that flags possible disease in a scan may alert a doctor at only 20 per cent, because missing a real case is far worse than a false alarm. A system that blocks a bank payment may need 95 per cent, and below that it asks the customer to confirm.

If the model is…A sensible response
Very confident, and the case is ordinaryAccept the prediction, with occasional spot checks
Unsure, with two answers close togetherSend the case to a person
Very confident, but the case looks unusualBe suspicious and check

How it works

ConfidenceWhat to do
Very high (above 90%)Usually accept, but still check a sample
Medium (60 to 90%)Use with care; check important cases
Low (below 60%)Ask a person to decide

Worked example

A model that tells apples from oranges looks at four picture cards. Card 1, a clear photo of a red apple: 'apple, 98 per cent', and it is right. Card 2, a blurry orange in dim light: 'orange, 55 per cent', right but unsure. Card 3, a red cricket ball: 'apple, 97 per cent', and it is wrong. The model only knows apples and oranges, so a round red ball looks just like its apples. Card 4, a green apple: 'apple, 72 per cent', and it is right.

When should a person check? Card 2, because the confidence is low. Card 3, because the picture is unlike anything in the training data, so even 97 per cent means nothing here. And any card where a mistake would be costly, whatever the confidence. Card 1 and card 4 can usually be accepted, with a few spot checks now and then. The lesson: low confidence means ask a person, but high confidence is not proof, especially for unusual inputs.

Try it yourself

Read the confidence bars

Pick each new animal and read how sure the model is. Find one where it is only 67 per cent sure, and work out why.

Accept the answer or ask a person?

An apple-or-orange model gives each card an answer and a confidence. Decide what to do.

  • A red cricket ball: 'apple, 97 per cent'

  • A clear red apple: 'apple, 98 per cent'

  • A blurry orange: 'orange, 55 per cent'

  • A medical scan: 'healthy, 91 per cent'

  • A green apple: 'apple, 72 per cent'

A new case: the pet photo sorter

Think it through on paper.

  1. A model sorts pet photos into cat or dog. It says 'cat, 99 per cent' for a photo of a rabbit. Explain why it can be so sure and still wrong.
  2. Write two rules for when a person should check this sorter's answers.
  3. Name one change to the training examples that would help.

Think about it

  • Why does a high confidence number not protect you from unusual pictures?

Common mistakes

  • Thinking 100 per cent confidence means certainly right.
  • Ignoring low confidence. That is the model asking for help.
  • Thinking confidence and accuracy are the same. Accuracy is measured on a test set.

Check yourself

Question 1

1. Two of the three nearest examples are cats. The model says…

Question 2

2. What should happen when confidence is low?

Question 3

3. Can a model be confidently wrong?

Summary

  • A prediction is the model's answer; confidence is how sure it is.
  • Use confidence to decide when a person should check.
  • High confidence is not proof: models can be confidently wrong.

Finished reading?