Definition
The model says "67 per cent dog". What does that mean?
A prediction is the answer a model gives for a new case. Confidence is how sure the model is about that answer, often shown as a percentage. High confidence does not guarantee the answer is right.
In simple words
Our animal model looks at the three most similar examples. If all three are dogs, it says dog, 100 per cent sure. If two are dogs and one is a cat, it says dog, 67 per cent sure. Like a weather forecast of 70 per cent chance of rain, it tells you how much to rely on the answer.
Why it matters
Confidence helps decide when to trust AI and when a person should check. A hospital might let AI handle cases it is very sure about and send uncertain ones to a doctor. But a model can be confidently wrong, especially on cases unlike its training data.
Key terms
- Prediction
- The answer a model gives for a new case.
- Confidence
- How sure the model is of its prediction, usually shown as a percentage.
- Probability
- A number between 0 and 1, or 0 and 100 per cent, that says how likely something is.
- Threshold
- The level of confidence at which a system takes action.
- Calibration
- How well a model's confidence matches how often it is actually right.
Explained in detail
A model gives two things
When you show a trained model a new case, it gives back a prediction and, usually, a measure of how sure it is. A photo app does not only say "cat". Inside, it has worked out something like: cat 91 per cent, dog 7 per cent, rabbit 2 per cent. It reports the highest one as its answer. Seeing the whole list is often more useful than seeing only the winner, because a model that says "cat 51 per cent, dog 49 per cent" is telling you it cannot really decide.
Predicting a category or a number
Predictions come in two main kinds. Sometimes the answer is a category: spam or not spam, healthy leaf or diseased leaf. Sometimes it is a number: tomorrow's temperature, the price of a house, the minutes until a delivery arrives. For a number, confidence is often shown as a range, such as "arriving in 20 to 25 minutes". A narrow range means the model is fairly sure; a wide one means it is not.
What a percentage should mean
Weather forecasters give a good example of honest confidence. When a careful forecaster says there is a 70 per cent chance of rain, it should rain on about seven out of every ten such days. If it does, the forecaster is well calibrated: the stated confidence matches reality. The same test applies to a model. Collect all the cases where it said it was 90 per cent sure. If it was right in about nine out of ten, its confidence can be trusted. If it was right in only six out of ten, it is overconfident, and its percentages mean less than they appear to.
Confidence fails on the unfamiliar
Confidence is worked out from the examples the model trained on. It measures how closely the new case resembles what the model has seen before, not whether the model is right. Show a fruit classifier a cricket ball and it may answer "apple, 96 per cent", because among the things it knows, an apple is by far the closest match. It has no way to say "this is outside everything I learned". High confidence on an unfamiliar input is one of the most common ways AI systems mislead people.
Thresholds: deciding when to act
People who build a system choose a threshold, the confidence above which it acts by itself. The right threshold depends on what a mistake costs. A photo app can label a picture "beach" at 60 per cent, because a wrong label harms nobody. A system that flags possible disease in a scan may alert a doctor at only 20 per cent, because missing a real case is far worse than a false alarm. A system that blocks a bank payment may need 95 per cent, and below that it asks the customer to confirm.
| If the model is… | A sensible response |
|---|---|
| Very confident, and the case is ordinary | Accept the prediction, with occasional spot checks |
| Unsure, with two answers close together | Send the case to a person |
| Very confident, but the case looks unusual | Be suspicious and check |
How it works
| Confidence | What to do |
|---|---|
| Very high (above 90%) | Usually accept, but still check a sample |
| Medium (60 to 90%) | Use with care; check important cases |
| Low (below 60%) | Ask a person to decide |
Worked example
A model that tells apples from oranges looks at four picture cards. Card 1, a clear photo of a red apple: 'apple, 98 per cent', and it is right. Card 2, a blurry orange in dim light: 'orange, 55 per cent', right but unsure. Card 3, a red cricket ball: 'apple, 97 per cent', and it is wrong. The model only knows apples and oranges, so a round red ball looks just like its apples. Card 4, a green apple: 'apple, 72 per cent', and it is right.
When should a person check? Card 2, because the confidence is low. Card 3, because the picture is unlike anything in the training data, so even 97 per cent means nothing here. And any card where a mistake would be costly, whatever the confidence. Card 1 and card 4 can usually be accepted, with a few spot checks now and then. The lesson: low confidence means ask a person, but high confidence is not proof, especially for unusual inputs.
Try it yourself
Read the confidence bars
Pick each new animal and read how sure the model is. Find one where it is only 67 per cent sure, and work out why.
Accept the answer or ask a person?
An apple-or-orange model gives each card an answer and a confidence. Decide what to do.
A red cricket ball: 'apple, 97 per cent'
A clear red apple: 'apple, 98 per cent'
A blurry orange: 'orange, 55 per cent'
A medical scan: 'healthy, 91 per cent'
A green apple: 'apple, 72 per cent'
A new case: the pet photo sorter
Think it through on paper.
- A model sorts pet photos into cat or dog. It says 'cat, 99 per cent' for a photo of a rabbit. Explain why it can be so sure and still wrong.
- Write two rules for when a person should check this sorter's answers.
- Name one change to the training examples that would help.
Think about it
- Why does a high confidence number not protect you from unusual pictures?
Common mistakes
- Thinking 100 per cent confidence means certainly right.
- Ignoring low confidence. That is the model asking for help.
- Thinking confidence and accuracy are the same. Accuracy is measured on a test set.
Check yourself
Summary
- A prediction is the model's answer; confidence is how sure it is.
- Use confidence to decide when a person should check.
- High confidence is not proof: models can be confidently wrong.
Finished reading?