Definition
You want to subtract each subject's average from every student's mark. The averages form 5 numbers; the marks form a 40 × 5 table. NumPy lines them up automatically. How?
Array operations apply arithmetic, comparisons and mathematical functions element by element. Broadcasting is NumPy's rule for combining arrays of different shapes: dimensions are compared from the right, and a dimension of size 1, or a missing one, is stretched to match the other array, without copying data. Matrix multiplication uses the separate @ operator.
In simple words
If you tell a class "everyone add 5 marks for the bonus question", you do not need 40 separate instructions; one number is applied to everyone. Broadcasting does the same: a single number, row or column is spread across the whole table.
Overview
- Arithmetic on arrays happens element by element: a + b adds matching positions.
- Vectorised code replaces loops, making programs shorter and much faster.
- Broadcasting lets arrays of different shapes work together by stretching dimensions of size 1.
- Shapes are compared from the right; each pair must be equal or one of them must be 1.
- Universal functions (ufuncs) such as np.sqrt, np.exp and np.log apply to every element.
- Matrix multiplication uses @ or np.dot, not *, which multiplies element-wise.
Why it matters
Broadcasting makes common data operations, centring, scaling, normalising images and adding biases in neural networks, one short line each. Understanding its rule also explains the shape errors you will meet.
Key terms
- Broadcasting
- Stretching arrays of compatible shapes so element-wise operations work.
- Compatible dimensions
- Two sizes that are equal, or where one is 1.
- Aggregation
- Reducing an array to a summary, such as sum, mean or max, optionally along an axis.
- keepdims
- An option that keeps reduced axes as size 1, so the result broadcasts back easily.
- @ operator
- Matrix multiplication, as opposed to * which is element-wise.
- Normalisation
- Rescaling values, for example (x − mean) / std for each column.
Types
Array operations fall into a few types depending on what they combine and how. These are the operations used in nearly every data and AI program.
1. Element-wise arithmetic
+, -, *, /, ** and comparisons applied to matching elements of same-shaped arrays.
- How it works: NumPy loops in C over every position and returns a new array of the same shape.
- Example: total = maths + science + english adds three mark arrays for every student at once.
- When to use it: Use it whenever the same calculation applies to every item.
2. Scalar broadcasting
Combining an array with a single number.
- How it works: The number is treated as if it filled an array of the same shape.
- Example: prices * 1.18 adds 18% GST to every price; pixels / 255 scales an image to 0–1.
- When to use it: Use it for unit changes, scaling and adding constants.
3. Array broadcasting
Combining arrays of different but compatible shapes, such as a table and one row.
- How it works: A (40, 5) table minus a (5,) row of column means subtracts each column's mean from every row; a (40, 1) column broadcasts across columns.
- Example: X - X.mean(axis=0) centres every feature; (X - X.mean(0)) / X.std(0) standardises.
- When to use it: Use it for normalising features and adding biases in neural networks; check shapes when it gives unexpected results.
4. Universal and aggregate functions
Ufuncs that transform every element (np.sqrt, np.exp, np.log, np.abs, np.round) and aggregates that reduce arrays (sum, mean, min, max, argmax).
- How it works: Ufuncs keep the shape; aggregates collapse an axis, chosen with axis=0 or axis=1.
- Example: marks.mean(axis=0) gives each subject's average; np.argmax(probs) gives the predicted class.
- When to use it: Use them for feature transforms and summaries without writing loops.
5. Matrix (linear-algebra) operations
Dot products and matrix multiplication with @, plus np.linalg functions.
- How it works: A (m, n) @ (n, p) gives (m, p): each result is a row-by-column sum of products; np.linalg provides inverse, solve and eig.
- Example: X @ w computes predictions for every row of a linear model at once.
- When to use it: Use it for linear models, neural-network layers and transformations.
Explained in detail
The broadcasting rule
NumPy compares the shapes of two arrays from the rightmost dimension leftwards. Two dimensions are compatible if they are equal or if one of them is 1. Missing dimensions on the left are treated as 1. Where a size is 1, the array is stretched along that axis. So a (40, 5) array and a (5,) array are compatible: the (5,) is treated as (1, 5) and stretched down 40 rows. A (40, 5) array and a (40,) array are not compatible, because the rightmost sizes, 5 and 40, differ; reshaping the second to (40, 1) makes it work, stretching it across 5 columns. The stretching is virtual, so no memory is wasted.
Examples
| Shapes | Compatible? | Result | Typical use |
|---|---|---|---|
| (40, 5) and scalar | Yes | (40, 5) | Adding a bonus to every mark |
| (40, 5) and (5,) | Yes | (40, 5) | Subtracting each subject's mean |
| (40, 5) and (40, 1) | Yes | (40, 5) | Dividing each row by its total |
| (40, 5) and (40,) | No | Error | Reshape to (40, 1) first |
| (480, 640, 3) and (3,) | Yes | (480, 640, 3) | Normalising each colour channel of an image |
Standardising a whole table in two lines
To standardise every column of a feature matrix X of shape (n, d), compute mean = X.mean(axis=0) and std = X.std(axis=0), each of shape (d,), then write Z = (X − mean) / std. Broadcasting stretches the means and standard deviations down every row. This is exactly what scikit-learn's StandardScaler does inside. In a neural-network layer, the expression X @ W + b relies on the same rule: X @ W has shape (batch, units) and the bias b has shape (units,), so b is added to every example in the batch.
Aggregations and other operations
- sum, mean, std, min, max, argmax and cumsum all accept axis=; argmax gives the position of the largest value, which is how a classifier's predicted class is read from its scores.
- keepdims=True keeps the reduced axis as size 1: X.sum(axis=1, keepdims=True) has shape (n, 1), ready to divide X row by row.
- np.where(condition, a, b) chooses element by element, such as np.where(marks >= 40, "pass", "fail").
- np.clip(x, 0, 100) limits values to a range.
- * is element-wise; @ is matrix multiplication; confusing them is a classic bug.
How it works
- Write down both shapes.
- Align them from the right and check each pair is equal or 1.
- If not, reshape one array, often adding an axis of size 1.
- Use keepdims=True on aggregations you will broadcast back.
- Check the result's shape is what you intended.
In code
import numpy as np
X = np.array([[60., 1200.], [75., 900.], [90., 1500.], [45., 600.]]) # marks, steps per day
mean, std = X.mean(axis=0), X.std(axis=0)
Z = (X - mean) / std
print("Means:", mean, " Std:", std.round(1))
print("Standardised:\n", Z.round(2))
print("New column means:", Z.mean(axis=0).round(6))
row_share = X / X.sum(axis=1, keepdims=True)
print("Row shares shape:", row_share.shape)
scores = np.array([[2.0, 0.5, 1.0], [0.1, 3.2, 0.3]])
print("Predicted class per row:", scores.argmax(axis=1))
try:
X + np.array([1, 2, 3, 4])
except ValueError as err:
print("Error:", str(err).split(":")[0])
print("Fixed with reshape:", (X + np.array([1, 2, 3, 4]).reshape(4, 1)).shape)Output
Means: [ 67.5 1050. ] Std: [ 16.8 335.4] Standardised: [[-0.45 0.45] [ 0.45 -0.45] [ 1.34 1.34] [-1.34 -1.34]] New column means: [0. 0.] Row shares shape: (4, 2) Predicted class per row: [0 1] Error: operands could not be broadcast together with shapes (4,2) (4,) Fixed with reshape: (4, 2)
Try it yourself
Normalising the photo
A vision model expects each colour channel of a (224, 224, 3) image to have mean 0 and standard deviation 1.
Will these shapes broadcast?
Decide for each pair.
(4, 5) and (5, 4)
(10, 3) and (10, 1)
(10, 3) and (10,)
(10, 3) and (3,)
Common mistakes
- Using * when matrix multiplication @ was intended.
- Forgetting keepdims and getting a shape mismatch when dividing rows.
- Broadcasting by accident and getting a large, wrong result without an error.
Check yourself
Summary
- Operations are element-wise; @ is matrix multiplication.
- Broadcasting aligns shapes from the right and stretches sizes of 1.
- Use axis and keepdims in aggregations, and always check result shapes.
Finished reading?
Sources: NumPy: Broadcasting · Python for Data Analysis, 3rd edition (Wes McKinney)