I3 · Mathematics for AI

Matrix Multiplication

Matrix multiplication combines two matrices into one: each entry of the result is the dot product of a row from the first and a column from the second. The transpose flips a matrix over its diagonal, turning rows into columns. The identity matrix has 1s on the diagonal and 0s elsewhere, and leaves anything it multiplies unchanged.

10 min Intermediate · Sublevel Beginner

What you need: The earlier topics of this module.

Definition

Matrix multiplication combines two matrices into one: each entry of the result is the dot product of a row from the first and a column from the second. The transpose flips a matrix over its diagonal, turning rows into columns. The identity matrix has 1s on the diagonal and 0s elsewhere, and leaves anything it multiplies unchanged.

In simple words

A shop has a table of how many items each customer bought, and a table of prices. Matrix multiplication gives each customer's bill in one step. Each bill is a row of quantities dotted with a column of prices.

Overview

  • Matrices can be added, scaled, multiplied, transposed and, if square and non-singular, inverted.
  • Matrix multiplication needs inner dimensions to match: (m × n) @ (n × p) gives (m × p).
  • Order matters: AB is usually not equal to BA.
  • A neural-network layer is a matrix multiplication followed by an activation function.
  • The determinant tells whether a matrix can be inverted and how it scales area or volume.
  • GPUs are fast at AI because they perform huge numbers of matrix multiplications in parallel.

Why it matters

A layer in a neural network is a matrix multiplication followed by a non-linear function. GPUs are valuable because they do matrix multiplication extremely fast. Writing data as matrices also replaces slow loops with one line of code.

Key terms

Matrix multiplication
Combining matrices where each result entry is a row-by-column dot product.
Transpose
Flipping a matrix so rows become columns, written Aᵀ.
Identity matrix
A square matrix with 1s on the diagonal and 0s elsewhere; multiplying by it changes nothing.
Inverse
A matrix A⁻¹ that undoes A, so A × A⁻¹ = I; not every matrix has one.
Element-wise product
Multiplying matching entries of two same-shaped matrices.
Linear transformation
A function, represented by a matrix, that rotates, scales or shears vectors.

Types

Matrix operations fall into several types. Each does a different job in data processing and machine learning.

1. Element-wise operations

Adding, subtracting or multiplying matrices of the same shape position by position.

  • How it works: A + B and A * B (the Hadamard product) work entry by entry; scalars and compatible shapes broadcast.
  • Example: Adding the marks of two terms; applying a mask that zeros out some pixels.
  • When to use it: Use them for combining tables, masking and applying the same change to every entry.

2. Matrix multiplication

Combining rows of the first matrix with columns of the second by dot products.

  • How it works: Entry (i, j) of AB is the dot product of row i of A and column j of B; in NumPy write A @ B.
  • Example: Data of 100 rows × 4 features @ weights of 4 × 3 gives 100 rows × 3 outputs, as in a layer with 3 neurons.
  • When to use it: Use it for linear models, neural-network layers, chaining transformations and computing many predictions at once.

3. Transpose

Flipping a matrix over its diagonal, so rows become columns.

  • How it works: Aᵀ has entry (j, i) equal to A's entry (i, j); (AB)ᵀ = BᵀAᵀ.
  • Example: Turning a 3 × 2 table into 2 × 3; computing XᵀX in the normal equation of linear regression.
  • When to use it: Use it to match shapes for multiplication and in formulas such as covariance and least squares.

4. Inverse and solving linear systems

A⁻¹ undoes A, so A⁻¹A = I; solving Ax = b finds x.

  • How it works: Only square matrices with non-zero determinant have inverses; np.linalg.solve(A, b) is faster and more accurate than computing the inverse.
  • Example: Solving 2x + y = 5 and x − y = 1 gives x = 2, y = 1.
  • When to use it: Use it for systems of equations and the exact least-squares solution of linear regression.

5. Determinant and decompositions

The determinant is a single number describing a square matrix; decompositions split a matrix into simpler factors.

  • How it works: det(A) = 0 means A squashes space and has no inverse; eigen-decomposition and singular value decomposition (SVD) reveal a matrix's main directions.
  • Example: For [[2, 1], [1, 2]], det = 4 − 1 = 3; SVD compresses an image by keeping its largest singular values.
  • When to use it: Use decompositions for PCA, recommender systems, compression and LoRA fine-tuning.

Explained in detail

The shape rule

Matrix multiplication A × B is defined only when the number of columns of A equals the number of rows of B. An (m × n) matrix times an (n × p) matrix gives an (m × p) matrix: the inner sizes must match and disappear, the outer sizes remain. Each entry of the result is the dot product of a row of A with a column of B. For example, a (2 × 3) matrix times a (3 × 4) matrix gives a (2 × 4) result, but (3 × 4) times (2 × 3) is undefined. In NumPy, A @ B performs matrix multiplication and raises a clear error when shapes do not fit.

A neural network layer is a matrix multiplication

Suppose a batch of 32 examples, each with 4 features, forms an input matrix X of shape (32, 4). A layer with 10 neurons has a weight matrix W of shape (4, 10), one column of weights per neuron. Then X @ W has shape (32, 10): for every example, the output of every neuron, computed as 320 dot products in one operation. Adding a bias vector of length 10 and applying an activation completes the layer. Stacking layers means multiplying again and again. This is why matrix multiplication is the single most important operation in deep learning, and why GPUs, which perform it extremely fast, transformed the field after 2012.

Key properties

PropertyTrue or false?Note
A × B = B × AUsually falseOrder matters; B × A may not even be defined
(A × B) × C = A × (B × C)TrueGrouping can change speed but not the answer
A × I = ATrueThe identity matrix acts like the number 1
(A × B)ᵀ = Bᵀ × AᵀTrueThe order reverses
A * B (element-wise) = A @ BFalseIn NumPy, * is element-wise; @ is matrix multiplication

Transpose, identity and inverse

  • The transpose often fixes shapes: if a and b are both (n, 1) column vectors, aᵀ @ b is their (1 × 1) dot product.
  • A matrix can represent a transformation of space: a 2 × 2 matrix can rotate, stretch or reflect every point in a plane, which is how image editors and game graphics move objects.
  • Solving linear regression exactly uses an inverse: w = (XᵀX)⁻¹Xᵀy, the normal equation. In practice libraries use more stable methods, because inverses are slow and numerically fragile.

How it works

The shape rule

You can multiply A × B only when A's columns equal B's rows. The inner numbers must match, and the outer numbers give the result's shape: (m × n) × (n × p) = (m × p).

PropertyTrue?
A(BC) = (AB)CYes, order of grouping doesn't matter
A(B + C) = AB + ACYes
AB = BANo. Usually false, and this catches many people.
(AB)ᵀ = BᵀAᵀYes, and the order reverses

The maths

Matrix multiplication
The result's entry at row i, column j
Row i of A
Column j of B
Transpose
Identity matrix
A = [1 2]    B = [5 6]
    [3 4]        [7 8]

AB row 1: [1×5 + 2×7,  1×6 + 2×8] = [19, 22]
AB row 2: [3×5 + 4×7,  3×6 + 4×8] = [43, 50]

BA row 1: [5×1 + 6×3,  5×2 + 6×4] = [23, 34]
BA row 2: [7×1 + 8×3,  7×2 + 8×4] = [31, 46]

AB and BA are different, which shows why the order matters.

In code

matrices.py
import numpy as np

A = np.array([[1, 2], [3, 4]])
B = np.array([[5, 6], [7, 8]])

print("A x B =\n", A @ B)
print("B x A =\n", B @ A)
print("Transpose of A =\n", A.T)
print("A x identity =\n", A @ np.eye(2, dtype=int))

quantities = np.array([[2, 1], [0, 3]])   # customers x items
prices = np.array([10, 40])               # price per item
print("Bills:", quantities @ prices)

Output

A x B =
 [[19 22]
 [43 50]]
B x A =
 [[23 34]
 [31 46]]
Transpose of A =
 [[1 3]
 [2 4]]
A x identity =
 [[1 2]
 [3 4]]
Bills: [ 60 120]

Common mistakes

  • Using * for matrix multiplication in NumPy. * multiplies element by element. Use @.
  • Assuming AB = BA.
  • Ignoring the shape rule. Most runtime errors in deep learning are shape mismatches.

Check yourself

Question 1

1. A is 3×4 and B is 4×2. What is the shape of AB?

Question 2

2. A is 2×3 and B is 2×3. Can you compute AB?

Question 3

3. Why are GPUs so useful for AI?

Summary

  • Each entry of AB is a row of A dotted with a column of B.
  • Inner shapes must match: (m×n)(n×p) = (m×p).
  • Order matters: AB is usually not BA.

Finished reading?

Sources: Deisenroth, Faisal & Ong, Mathematics for Machine Learning