Definition
A confusion matrix is a table that compares a classifier's predictions with the true labels. For two classes it has four counts: true positives, false positives, true negatives and false negatives.
In simple words
Think of a fire alarm. It rings during a fire: true positive. It rings when someone burns toast: false positive. It stays silent on a normal day: true negative. It stays silent during a fire: false negative, the dangerous one. A single accuracy number can't show you this, and this table can.
Overview
- A confusion matrix counts a classifier's correct and incorrect predictions for each class.
- For two classes it has four cells: true positives, false positives, true negatives and false negatives.
- Rows usually show actual classes and columns predicted classes (scikit-learn's convention).
- Accuracy, precision, recall, specificity and F1 are all computed from these counts.
- Changing the decision threshold moves predictions between cells.
- Multi-class matrices show which classes are confused with each other.
Why it matters
Different errors have different costs. Missing a cancer is far worse than a false alarm, and blocking a real customer's payment has a different cost from missing a fraud. All the common metrics are calculated from these four numbers.
Key terms
- True positive (TP)
- Predicted positive, actually positive.
- False positive (FP)
- Predicted positive, actually negative: a false alarm, or Type I error.
- False negative (FN)
- Predicted negative, actually positive: a miss, or Type II error.
- True negative (TN)
- Predicted negative, actually negative.
- Accuracy
- (TP + TN) divided by all predictions.
- Class imbalance
- When one class is much rarer than the other, making accuracy misleading.
Types
Each cell of the confusion matrix is a type of outcome with its own real-world cost.
1. True positive (TP)
A positive case correctly predicted as positive.
- How it works: Counted when the actual class is positive and the model's prediction is also positive.
- Example: A diseased patient correctly flagged by a screening model.
- When to use it: Maximise these when catching positives matters, and report them alongside false negatives as recall. Each one is a success the model can claim.
2. False positive (FP), a Type I error
A negative case wrongly predicted as positive.
- How it works: Counted when the actual class is negative but the model says positive; lowering the threshold usually increases these.
- Example: A genuine payment blocked as fraud, annoying the customer.
- When to use it: Watch these when false alarms are costly, such as blocking payments or sending people for unnecessary tests.
3. True negative (TN)
A negative case correctly predicted as negative.
- How it works: Counted when both the actual class and the prediction are negative; with rare positives this cell is usually huge.
- Example: A normal email correctly left in the inbox.
- When to use it: Use it to compute specificity, TN ÷ (TN + FP), remembering it can make accuracy look misleadingly high.
4. False negative (FN), a Type II error
A positive case wrongly predicted as negative.
- How it works: Counted when the actual class is positive but the model says negative; raising the threshold usually increases these.
- Example: A cancer case missed by a screening model.
- When to use it: Minimise these when misses are dangerous, such as disease, fraud or safety faults.
5. Multi-class confusion matrix
A k × k table for k classes showing every actual-versus-predicted combination.
- How it works: The diagonal holds correct predictions; off-diagonal cells show which pairs of classes are confused; per-class precision and recall come from rows and columns.
- Example: A digit classifier often confusing 4 with 9 and 3 with 8.
- When to use it: Use it to find specific weaknesses and decide where more data or features are needed.
Explained in detail
Reading the table
A confusion matrix lays out every prediction against the truth. By the convention scikit-learn uses, rows are the actual classes and columns are the predicted classes, so the diagonal holds correct predictions and everything off the diagonal is an error. Other tools sometimes swap rows and columns, so always read the labels. The name comes from the fact that the table shows exactly which classes the model confuses with which, which is far more informative than a single accuracy number.
A worked example
| Predicted fraud | Predicted genuine | |
|---|---|---|
| Actually fraud (50) | TP = 40 | FN = 10 |
| Actually genuine (9,950) | FP = 200 | TN = 9,750 |
Accuracy is (40 + 9,750) / 10,000 = 97.9 per cent, which sounds excellent. But a useless model that calls every transaction genuine would score 99.5 per cent, higher still, while catching no fraud at all. The matrix shows what matters: 40 of 50 frauds were caught, 10 were missed, and 200 genuine customers were wrongly flagged. Whether that is good depends on the cost of each error, which only the business can judge.
The cost of each error
- Cancer screening: a false negative may delay treatment, so misses are usually far worse than false alarms.
- Spam filtering: a false positive can hide an important email, so false alarms are worse.
- Fraud: misses lose money and false alarms annoy customers and cost staff time; both are weighed.
- Changing the decision threshold moves errors between FP and FN; it cannot remove both.
Beyond two classes
For several classes the matrix becomes larger, one row and one column per class. A digit recogniser's matrix might show that 4s are often mistaken for 9s and 3s for 8s, which tells you exactly where to collect more examples or improve features. In scikit-learn, confusion_matrix(y_true, y_pred) computes the table and ConfusionMatrixDisplay draws it as a coloured grid. Computing the matrix separately for different groups of people is also one of the simplest and most important fairness checks.
Every other classification metric, accuracy, precision, recall, specificity and F1, is just a different ratio of these four counts, so learning to read the matrix first makes all of them easy.
How it works
| Predicted: Yes | Predicted: No | |
|---|---|---|
| Actually: Yes | True Positive (TP) | False Negative (FN): missed it |
| Actually: No | False Positive (FP): false alarm | True Negative (TN) |
The maths
100 patients: 10 have the disease. The model catches 8, and flags 5 healthy people wrongly.
TP = 8 FN = 2 FP = 5 TN = 85 Accuracy = (8 + 85) / 100 = 93%
93% sounds fine, but the model still missed 2 of 10 patients. A model that says "healthy" to everyone would score 90% and catch nobody. That's why accuracy alone can't be trusted on imbalanced data.
In code
from sklearn.metrics import accuracy_score, confusion_matrix
actual = [1] * 10 + [0] * 90
predicted = [1] * 8 + [0] * 2 + [1] * 5 + [0] * 85
tn, fp, fn, tp = confusion_matrix(actual, predicted).ravel()
print("TP:", tp, " FN:", fn, " FP:", fp, " TN:", tn)
print("Accuracy:", accuracy_score(actual, predicted))
everyone_healthy = [0] * 100
print("Always says healthy:", accuracy_score(actual, everyone_healthy))Output
TP: 8 FN: 2 FP: 5 TN: 85 Accuracy: 0.93 Always says healthy: 0.9
Common mistakes
- Judging a model on accuracy alone, particularly on imbalanced classes.
- Mixing up false positives and false negatives. Think about what each costs in your case.
Check yourself
Summary
- Four counts: TP, FP, TN and FN.
- All common classification metrics come from them.
- Accuracy can mislead when classes are imbalanced.
Finished reading?
Sources: Google Machine Learning Crash Course