I3 · Mathematics for AI

Correlation

Correlation measures how strongly two variables move together, on a scale from −1 to +1. Pearson's correlation measures straight-line relationships. Spearman's correlation applies the same idea to ranks, so it catches any steady rising or falling relationship.

10 min Intermediate · Sublevel Beginner

What you need: The earlier topics of this module.

Definition

Correlation measures how strongly two variables move together, on a scale from −1 to +1. Pearson's correlation measures straight-line relationships. Spearman's correlation applies the same idea to ranks, so it catches any steady rising or falling relationship.

In simple words

Hours studied and marks usually rise together, so that's a positive correlation. Price and sales often move oppositely, a negative one. Shoe size and marks don't move together at all: near zero.

Overview

  • Correlation measures how strongly two variables move together, from −1 to +1.
  • Positive correlation: both rise together; negative: one rises as the other falls; zero: no linear link.
  • Correlation is not causation: ice-cream sales and drownings both rise in summer because of heat.
  • A scatter plot should always be checked, because one outlier can create or hide a correlation.
  • Correlation can be zero even when a strong curved relationship exists.
  • In machine learning, correlation helps find useful features and redundant ones.

Why it matters

In EDA, correlation shows which features may help and which are duplicates. Two features with 0.99 correlation carry nearly the same information, a problem called multicollinearity, which can make linear models unstable.

Key terms

Correlation coefficient
A number from −1 to +1 describing how strongly two variables move together.
Pearson's r
Correlation measuring straight-line relationships between two numeric variables.
Spearman's rho
Correlation computed on ranks, capturing any consistently rising or falling relationship.
Covariance
The average product of deviations of two variables; correlation is covariance rescaled to −1…+1.
Confounder
A third variable that affects both variables and creates a misleading correlation.
Spurious correlation
A correlation that is coincidental or caused by something else, not a real link.

Types

There are several types of correlation coefficient. They differ in what kind of relationship they detect and what data they need.

1. Pearson correlation (r)

Measures the strength of a straight-line relationship between two numeric variables.

  • How it works: It is the covariance divided by the product of the standard deviations; r² gives the share of variance one variable explains in the other.
  • Example: Hours studied and marks with r = 0.8 show a strong positive linear link; r² = 0.64.
  • When to use it: Use it for numeric data with a roughly linear pattern and no big outliers.

2. Spearman rank correlation (ρ)

Measures how well the relationship is described by any steadily increasing or decreasing curve.

  • How it works: Each variable is replaced by its ranks, and Pearson's formula is applied to the ranks.
  • Example: Experience and salary rise together but not in a straight line; Spearman captures this better than Pearson.
  • When to use it: Use it for ordinal data such as rankings and ratings, for curved monotonic patterns and when outliers are present.

3. Kendall's tau (τ)

A rank correlation based on counting agreeing and disagreeing pairs.

  • How it works: For every pair of observations it checks whether both variables order them the same way; τ = (concordant − discordant) ÷ total pairs.
  • Example: Comparing how two judges ranked ten dance teams.
  • When to use it: Use it for small samples and ranking data, where it is often more reliable than Spearman.

4. Point-biserial and phi correlation

Correlations involving yes/no variables: point-biserial for one binary and one numeric, phi for two binaries.

  • How it works: Both are Pearson's formula applied with the binary variable coded as 0 and 1.
  • Example: Correlation between passed (0/1) and attendance percentage; between smoker (0/1) and disease (0/1).
  • When to use it: Use them to check how strongly a yes/no feature relates to an outcome.

5. Spurious and partial correlation

Spurious correlation is a link caused by chance or a hidden third variable; partial correlation measures the link after removing a third variable's effect.

  • How it works: Partial correlation correlates the residuals of each variable after regressing both on the control variable.
  • Example: Shoe size correlates with reading ability among children only because age drives both; controlling for age removes the link.
  • When to use it: Use these ideas to question surprising correlations before drawing conclusions or building features on them.

Explained in detail

What the number means

A correlation of +1 means that as one variable rises, the other rises in a perfect straight line; −1 means a perfect falling line; 0 means no straight-line relationship. In real data, values such as 0.3 are weak, 0.5 moderate and above 0.7 strong, though these labels vary by field. Pearson's r, developed by Karl Pearson in the 1890s from ideas of Francis Galton, is the covariance of the two variables divided by the product of their standard deviations. Dividing by the standard deviations is what makes r independent of units: the correlation between height in centimetres and weight in kilograms is the same as between height in inches and weight in pounds.

Always look at the scatter plot

In 1973 the statistician Francis Anscombe published four small data sets, now called Anscombe's quartet, that have almost identical means, variances and a correlation of about 0.82, yet look completely different when plotted: one is a straight trend, one a curve, one a straight line with one outlier, and one has all points stacked vertically except a single far-away point. The lesson is that a single number can hide very different stories. A strong curved relationship, such as performance rising and then falling with hours of practice, can have a Pearson correlation near zero.

Correlation is not causation

Observed correlationReal explanation
Ice-cream sales and drowning incidents rise togetherHot weather drives both (a confounder)
Students with more books at home score higherFamily income and education affect both
Cities with more hospitals have more deathsLarger cities have more of both
A country's cheese consumption tracks some unrelated statisticPure coincidence across a few years

Correlation in AI work

  • Feature exploration: a correlation matrix, usually drawn as a heatmap, shows which features relate to the target and to each other.
  • Multicollinearity: two features with correlation near ±1 carry nearly the same information, which can make linear models unstable.
  • Leakage: a feature suspiciously correlated with the target, such as "refund issued" when predicting complaints, may be a consequence of the target rather than a cause.
  • Fairness: features correlated with protected attributes, such as postcode with religion or caste, can act as proxies.

pandas computes Pearson correlation with df.corr() and Spearman with df.corr(method="spearman"). Use Spearman when data is ranked, skewed or has outliers, because ranks blunt the effect of extreme values.

How it works

Value of rMeaning
+1A perfect rising straight line
+0.7Strong and rising
0No straight-line relationship
−0.7Strong and falling
−1A perfect falling straight line

The maths

Pearson correlation
The correlation, from −1 to +1
Covariance: do they move together?

x = [1, 2, 3], y = [2, 4, 7]. The means are 2 and 4.33.

x − x̄ = −1, 0, 1           y − ȳ = −2.33, −0.33, 2.67
Σ(x−x̄)(y−ȳ) = 2.33 + 0 + 2.67 = 5
Σ(x−x̄)² = 2       Σ(y−ȳ)² = 5.44 + 0.11 + 7.11 = 12.67
r = 5 / (√2 × √12.67) = 5 / 5.03 ≈ 0.993

In code

correlation.py
import numpy as np
import pandas as pd

x = np.array([1, 2, 3])
y = np.array([2, 4, 7])
print("Pearson r:", round(np.corrcoef(x, y)[0, 1], 3))

curve_x = np.array([-3, -2, -1, 0, 1, 2, 3])
print("r for y = x squared:", round(np.corrcoef(curve_x, curve_x ** 2)[0, 1], 3))

df = pd.DataFrame({"a": [1, 2, 3, 4, 5], "b": [1, 8, 27, 64, 125]})
print("Pearson:", round(df.corr(method="pearson").loc["a", "b"], 3))
print("Spearman:", round(df.corr(method="spearman").loc["a", "b"], 3))

Output

Pearson r: 0.993
r for y = x squared: 0.0
Pearson: 0.943
Spearman: 1.0

Common mistakes

  • Claiming that one thing causes another because they correlate.
  • Trusting r without a plot. Very different datasets can share the same r.
  • Using Pearson on data with outliers. One extreme point can swing it. Consider Spearman.

Check yourself

Question 1

1. r = −0.9 means:

Question 2

2. Students who sleep more score higher. What can you conclude?

Summary

  • Correlation runs from −1 to +1. The sign is direction, and the size is strength.
  • Pearson is for straight lines, and Spearman for any steady trend.
  • Correlation is not causation, and always plot the data.

Finished reading?

Sources: Python statistics, correlation and linear regression