Definition
A coaching institute says you are in the "98th percentile". Does that mean you scored 98 marks? Not at all.
A percentile is the value below which a given percentage of the data falls: the 90th percentile is the value that 90 per cent of observations are at or below. Quartiles are the 25th, 50th and 75th percentiles, called Q1, the median (Q2) and Q3. The interquartile range, IQR = Q3 − Q1, measures the spread of the middle half, and values more than 1.5 × IQR beyond the quartiles are commonly flagged as outliers.
In simple words
Line up 100 students by marks. Your percentile is roughly how many are standing behind you. In the 98th percentile, 98 out of 100 scored at or below you, whatever the actual marks were. Quartiles simply cut the line into four equal groups.
Overview
- A percentile is the value below which a given percentage of the data falls.
- Quartiles split sorted data into four equal parts: Q1 (25th), Q2 (median) and Q3 (75th percentile).
- The five-number summary is minimum, Q1, median, Q3 and maximum, drawn as a box plot.
- An outlier is a value far from the rest, often flagged by the 1.5 × IQR rule.
- Percentile ranks are common in exams: a 90th percentile score beat 90% of candidates.
- Different software uses slightly different interpolation rules, so quartiles can differ in the decimals.
Why it matters
Percentiles describe position without being fooled by extremes, which makes them ideal for skewed data such as incomes and response times. Engineers judge systems by the 95th or 99th percentile of delay, and data scientists use the IQR rule to find suspicious values before training a model.
Key terms
- Percentile
- The value below which a given percentage of observations falls.
- Percentile rank
- The percentage of observations at or below a particular score.
- Quartiles
- Q1, Q2 and Q3: the values cutting sorted data into four equal parts.
- Interquartile range (IQR)
- Q3 − Q1, the range of the middle 50 per cent of the data.
- Box plot
- A chart showing the median, quartiles, whiskers and outliers.
- Tail latency
- The slowest responses, often reported as the 95th or 99th percentile of response time.
Types
Position measures and outlier rules come in several types. These are the ones you will use.
1. Percentiles
Values that divide sorted data into 100 equal parts.
- How it works: The p-th percentile has p% of values below it; np.percentile(x, 90) computes it with interpolation between neighbouring values.
- Example: If the 90th percentile of delivery time is 45 minutes, nine in ten orders arrive within 45 minutes.
- When to use it: Use them for service targets, growth charts for children and exam percentile ranks.
2. Quartiles and the five-number summary
Three cut points (Q1, Q2, Q3) plus the minimum and maximum.
- How it works: Sort the data; Q2 is the median, Q1 the median of the lower half and Q3 of the upper half; describe() in pandas shows them.
- Example: Marks with Q1 = 52, median = 64 and Q3 = 78 show the middle half of students scored between 52 and 78.
- When to use it: Use them to describe skewed data robustly and to draw box plots for comparing groups.
3. IQR (Tukey) outlier rule
Flags values below Q1 − 1.5 × IQR or above Q3 + 1.5 × IQR.
- How it works: John Tukey's rule uses quartiles, which outliers do not distort; 3 × IQR marks extreme outliers.
- Example: With Q1 = 20 and Q3 = 30 (IQR 10), values below 5 or above 45 are flagged.
- When to use it: Use it as the default outlier check for one variable, especially when data is skewed.
4. Z-score rule
Flags values more than about 3 standard deviations from the mean.
- How it works: z = (x − mean) ÷ std; for normal data only about 0.3% of values have |z| > 3, but outliers inflate the mean and std themselves.
- Example: A height with z = 4.1 in a sample of adults is suspicious and worth checking.
- When to use it: Use it for roughly normal data with many points; prefer the IQR rule or a robust z-score using the median otherwise.
5. Percentile capping (winsorising)
Replacing values beyond chosen percentiles with those percentile values.
- How it works: Values above the 99th percentile are set to the 99th percentile, and below the 1st to the 1st, keeping every row.
- Example: Capping a few ₹10 lakh orders at the 99th percentile of ₹40,000 before training a sales model.
- When to use it: Use it when genuine extreme values would distort a model but the rows themselves are valid.
Explained in detail
Percentile versus percentage
A percentage describes your own result: 72 out of 100 is 72 per cent. A percentile describes your position relative to everyone else. If a paper was very hard and the highest mark was 60, a score of 55 might be the 99th percentile. Many national entrance examinations in India, including JEE Main, report a percentile so that results from different sessions with papers of different difficulty can be compared fairly. A percentile of 98 means 98 per cent of candidates in that session scored at or below you.
Finding quartiles
Take the sorted data 3, 5, 7, 8, 9, 11, 13, 15, 40. The median, Q2, is the middle value, 9. Q1 is the median of the lower half, 3, 5, 7, 8, which is 6. Q3 is the median of the upper half, 11, 13, 15, 40, which is 14. So the IQR is 14 − 6 = 8. Different textbooks and software use slightly different methods for quartiles, especially for small data sets, so NumPy, Excel and a calculator may disagree a little. For large data the differences become negligible.
The 1.5 × IQR rule
| Step | Calculation | Result for the data above |
|---|---|---|
| Lower fence | Q1 − 1.5 × IQR | 6 − 12 = −6 |
| Upper fence | Q3 + 1.5 × IQR | 14 + 12 = 26 |
| Flag | Values outside the fences | 40 is an outlier |
This rule, from the statistician John Tukey's 1977 book Exploratory Data Analysis, is the basis of the box plot: a box from Q1 to Q3 with a line at the median, whiskers to the most extreme points inside the fences, and individual dots for outliers. Unlike the z-score rule, it does not assume the data is bell-shaped, and the outlier itself does not distort the quartiles used to judge it.
Outliers: error or discovery?
- A flagged value is a question, not a verdict. Check the source before removing anything.
- Errors: a height of 1,750 cm is a typing slip for 175; fix or remove it.
- Real extremes: a festival-day sales spike is genuine and the model should learn about it.
- Discoveries: in fraud detection or medical screening, the outliers are exactly what you are looking for.
- Options other than deleting: cap extreme values at a percentile (winsorising), transform with a logarithm, or use models and losses that are robust to outliers.
How it works
- Sort the data.
- Find the median, then the medians of the lower and upper halves to get Q1 and Q3.
- Compute IQR = Q3 − Q1 and the fences Q1 − 1.5 × IQR and Q3 + 1.5 × IQR.
- Investigate every value outside the fences before deciding what to do.
The maths
- The first and third quartiles
In code
import numpy as np
data = np.array([3, 5, 7, 8, 9, 11, 13, 15, 40])
q1, q2, q3 = np.percentile(data, [25, 50, 75])
iqr = q3 - q1
low, high = q1 - 1.5 * iqr, q3 + 1.5 * iqr
print("Q1, median, Q3:", q1, q2, q3)
print("IQR:", iqr)
print("Fences:", low, high)
print("Outliers:", data[(data < low) | (data > high)])
# Percentile rank: what share of scores are at or below 55?
scores = np.array([22, 31, 38, 40, 44, 47, 51, 55, 58, 60])
print("Percentile rank of 55:", np.mean(scores <= 55) * 100)Output
Q1, median, Q3: 7.0 9.0 13.0 IQR: 6.0 Fences: -2.0 22.0 Outliers: [40] Percentile rank of 55: 80.0
Try it yourself
The 99th percentile delay
A team runs an AI chatbot. The average reply takes 0.8 seconds. Users still complain.
Outlier: fix, keep or investigate?
Each value was flagged by the 1.5 × IQR rule.
One transaction far larger than a customer's usual pattern
Sales on Diwali three times the usual
A student's age recorded as 154
A temperature of −999 °C from a sensor
Common mistakes
- Confusing a percentile with a percentage score.
- Deleting every flagged outlier without investigating.
- Using the z-score rule on heavily skewed data, where the IQR rule is safer.
Check yourself
Summary
- A percentile is a position: the share of data at or below a value.
- Quartiles split data into four; IQR = Q3 − Q1 measures the middle half.
- The 1.5 × IQR rule flags outliers, which must be investigated, not blindly deleted.
Finished reading?
Sources: Khan Academy: Statistics and probability · NumPy documentation: percentile