Definition
A disease-screening AI is "95 per cent accurate". Sounds great. But if only 5 in 100 people have the disease, an AI that simply says "healthy" to everyone is also 95 per cent accurate, and misses every sick person.
Evaluation is the stage of the AI project cycle where the team tests how well the model works, using data it has never seen. It counts the different kinds of right and wrong answers, often in a table called a confusion matrix, and uses measures such as accuracy, and the numbers of false alarms and missed cases, to decide whether the model is good enough and fair enough to use.
In simple words
A teacher does not test students only on questions they have already practised; that would just test memory. A fair exam uses new questions. Evaluation is the model's fair exam, and the marks show not just how many it got right, but which kinds of mistakes it made.
Why it matters
A single accuracy number can hide serious problems. Different mistakes have different costs: missing a disease is far worse than a false alarm. Careful evaluation protects people from models that look good on paper but fail where it matters.
Key terms
- Evaluation
- Testing how well a model works on new data.
- Accuracy
- The share of all predictions that were correct.
- Confusion matrix
- A table counting correct and incorrect predictions of each kind.
- False positive
- A false alarm: the model says yes, but the true answer is no.
- False negative
- A miss: the model says no, but the true answer is yes.
- Test set
- Examples kept aside, never used in training, for a fair check.
Explained in detail
Why test on new data
A model can memorise its training examples without learning the real pattern, just as a student can memorise answers without understanding. So before training, a portion of the data, often around a fifth, is set aside as a test set. After training, the model makes predictions for the test set, and its answers are compared with the true labels. Only this tells you how it will perform on the new cases it will meet in real life.
The four outcomes
For a yes-or-no question, such as "does this leaf have the disease?", every prediction falls into one of four boxes. True positive: the model said yes and it was yes. True negative: it said no and it was no. False positive: it said yes but it was no, a false alarm. False negative: it said no but it was yes, a miss. Laying these four counts out in a two-by-two table gives the confusion matrix.
| Truly diseased | Truly healthy | |
|---|---|---|
| Model says diseased | 18 true positives | 7 false positives (false alarms) |
| Model says healthy | 2 false negatives (misses) | 73 true negatives |
From this table of 100 test leaves: correct predictions are 18 plus 73, which is 91, so accuracy is 91 per cent. Of the 20 truly diseased leaves, the model caught 18 and missed 2. Of the 25 leaves it flagged, 7 were false alarms. Each number tells a different part of the story.
Which mistake matters more?
| System | Worse mistake | Why |
|---|---|---|
| Cancer screening | Missing a case (false negative) | A missed patient goes untreated; a false alarm leads to a further test |
| Spam filter | Blocking a real email (false positive) | A lost job offer is worse than seeing one spam |
| Fraud alert | Depends on the cost of each | Missed fraud loses money; too many false alarms annoy customers |
Evaluate for every group
Finally, look at the results separately for different groups: regions, ages, genders, languages, camera types. A model that is 91 per cent accurate overall might be 96 per cent accurate for one group and 70 per cent for another. That gap would be hidden by the single overall number. Checking groups separately is one of the most important ways to catch unfairness before a model is deployed.
How it works
- Set aside a test set before training, and never train on it.
- Run the model on the test set and compare with the true answers.
- Fill in the four boxes: true and false positives and negatives.
- Work out accuracy, and count false alarms and misses.
- Decide which mistake matters more, and check results for each group.
Try it yourself
Mark the model's test
Ten pictures the model has never seen. How many did it get right, and what kinds of mistakes did it make?
The 95 per cent model that helped nobody
A team tests a disease-screening model on 100 people, 5 of whom are ill.
Common mistakes
- Testing the model on its training data.
- Reporting only accuracy when one outcome is rare.
- Not checking results separately for each group.
Check yourself
Summary
- Evaluation tests a model on new data it has never seen.
- The confusion matrix shows true and false positives and negatives.
- Accuracy alone can mislead; weigh the cost of each mistake and check every group.
Finished reading?
Sources: Google Machine Learning Crash Course: Classification · CBSE Artificial Intelligence curriculum resources