Definition
Conditional probability is the probability of an event A, given that another event B has happened. It is written P(A|B) and read "the probability of A given B". Two events are independent if knowing one doesn't change the chance of the other.
In simple words
The chance a random person is carrying an umbrella may be 10%. The chance given that it's raining is far higher. New information changes the probability, because it shrinks the world you're looking at: now you count only rainy days.
Overview
- Conditional probability P(A | B) is the probability of A given that B is known to have happened.
- It is computed as P(A and B) ÷ P(B), restricting attention to cases where B is true.
- P(A | B) and P(B | A) are usually different, and confusing them is a classic error.
- Two-way tables and tree diagrams make conditional probabilities easy to read.
- If P(A | B) = P(A), the events are independent.
- Precision and recall are conditional probabilities: P(actual positive | predicted positive) and P(predicted positive | actual positive).
Why it matters
Nearly every prediction is conditional: the chance of spam given these words, or the next word given the sentence so far. Classifiers estimate P(label | features) directly. Naive Bayes and Bayesian networks are built entirely on this idea.
Key terms
- Conditional probability
- P(A|B), the probability of A given that B has happened.
- Joint probability
- P(A and B), the probability that both happen.
- Marginal probability
- The probability of one event on its own, ignoring others.
- Independence
- A and B are independent if P(A|B) = P(A).
- Contingency table
- A table of counts for combinations of two categorical variables.
- Base rate
- How common something is in the whole population before any evidence.
Types
Conditional probability appears in several standard forms. Each type is a tool you will reuse often.
1. Conditional probability from a table
Reading P(A | B) from counts in a two-way table.
- How it works: Take the row or column for B, then divide the count where A also holds by that row's or column's total.
- Example: Of 200 students, 80 play cricket and 60 of those passed maths: P(pass | cricket) = 60 ÷ 80 = 0.75.
- When to use it: Use it for survey data, confusion matrices and any count table.
2. Multiplication (chain) rule
Finding the probability of several events happening in sequence.
- How it works: P(A and B and C) = P(A) × P(B | A) × P(C | A and B); each factor uses everything that has already happened.
- Example: The chance that three randomly chosen students from 10 girls and 10 boys are all girls is 10/20 × 9/19 × 8/18 ≈ 0.105.
- When to use it: Use it for sequences of dependent events and to understand how language models multiply next-word probabilities.
3. Law of total probability
Finding an overall probability by adding it up across separate cases.
- How it works: If cases B₁, B₂, … cover every possibility without overlap, P(A) = Σ P(A | Bᵢ) × P(Bᵢ).
- Example: If 60% of a factory's bolts come from machine X with 2% defects and 40% from Y with 5%, P(defect) = 0.6 × 0.02 + 0.4 × 0.05 = 0.032.
- When to use it: Use it to combine information from several sources or groups, and as the denominator of Bayes' theorem.
4. Tree diagrams
Diagrams with branches for each stage, labelled with conditional probabilities.
- How it works: Multiply along a path to get the probability of that sequence, and add the paths that lead to the same outcome.
- Example: A disease screening tree: first branch sick or healthy, second branch test positive or negative.
- When to use it: Use them to set up multi-step problems clearly before calculating.
5. Conditional independence
Two events that become independent once a third piece of information is known.
- How it works: P(A and B | C) = P(A | C) × P(B | C); naive Bayes assumes every word in an email is conditionally independent given the class.
- Example: Two children's colds are related in general, but independent once you know whether a flu outbreak is happening at their school.
- When to use it: Use it to simplify models with many variables, while remembering it is an assumption that may be only roughly true.
Explained in detail
Restricting the world
Conditioning means narrowing attention to the cases where B is true, then asking how often A happens among them. Formally, P(A|B) = P(A and B) / P(B). Suppose 1,000 emails arrive; 200 contain the word "prize", and 150 of those are spam. Then P(spam | "prize") = 150 / 200 = 0.75. Among all emails, perhaps only 30 per cent are spam, so seeing "prize" raises the probability from 0.30 to 0.75. This shift, from what is true in general to what is true given the evidence, is the basic operation of every classifier.
A contingency table
| Spam | Not spam | Total | |
|---|---|---|---|
| Contains "prize" | 150 | 50 | 200 |
| No "prize" | 150 | 650 | 800 |
| Total | 300 | 700 | 1,000 |
From this table you can read every kind of probability. Joint: P(spam and "prize") = 150/1,000 = 0.15. Marginal: P(spam) = 300/1,000 = 0.30. Conditional: P(spam | "prize") = 150/200 = 0.75, but P("prize" | spam) = 150/300 = 0.50. Notice that these last two are different questions with different answers.
The most common mistake
Confusing P(A|B) with P(B|A) is so common it has a name: the prosecutor's fallacy, after courtroom errors. "The chance of this DNA match if the accused is innocent is one in a million" is P(match | innocent). It is not the chance that the accused is innocent given the match, P(innocent | match), which also depends on how many people could have left the sample. In medicine, a test's sensitivity, P(positive | disease), is often mistaken for P(disease | positive), which can be far lower when the disease is rare. Bayes' theorem, on the next page, is the tool for converting one into the other.
Independence
- Two coin tosses are independent: the first result tells you nothing about the second.
- Drawing cards without replacement is not independent: after an ace is drawn, fewer aces remain.
- The naive Bayes classifier assumes words in an email are independent given the class; this is false, yet the model often works well.
- Checking independence in data: compare P(A|B) with P(A); in the table, 0.75 versus 0.30 shows strong dependence.
Conditioning also explains why more data per group matters: estimates of P(A|B) for a rare B rest on few examples and can swing wildly.
How it works
Out of 100 emails, 30 are spam. The word "free" appears in 25 emails, and 20 of those are spam.
| Has "free" | No "free" | Total | |
|---|---|---|---|
| Spam | 20 | 10 | 30 |
| Not spam | 5 | 65 | 70 |
| Total | 25 | 75 | 100 |
P(spam) is 30/100 = 0.30. But P(spam | free) looks only at the 25 emails with "free", so it is 20/25 = 0.80. Seeing that one word moved the chance from 30% to 80%.
The maths
- The chance of A once B is known
- The chance that both happen
P(spam ∩ free) = 20/100 = 0.20 P(free) = 25/100 = 0.25 P(spam | free) = 0.20 / 0.25 = 0.80
In code
emails = [("spam", True)] * 20 + [("spam", False)] * 10 + [("safe", True)] * 5 + [("safe", False)] * 65
with_free = [label for label, has_free in emails if has_free]
print("P(spam):", sum(label == "spam" for label, _ in emails) / len(emails))
print("P(spam | free):", with_free.count("spam") / len(with_free))Output
P(spam): 0.3 P(spam | free): 0.8
Common mistakes
- Confusing P(A|B) with P(B|A).
- Dividing by the total. Divide by the count of the condition, because the condition shrinks the world.
Check yourself
Summary
- P(A|B) is the chance of A once you know B.
- Formula: P(A and B) divided by P(B).
- Independent means knowing one doesn't change the other.
Finished reading?
Sources: Deisenroth, Faisal & Ong, Mathematics for Machine Learning