I2 · Working with Data: NumPy, pandas, SQL and Visualisation

Introduction to NumPy

NumPy is Python's core library for numerical computing. Its central object, the ndarray, is a grid of values that all share one data type, stored in a single block of memory. Operations on whole arrays run in fast compiled code, an approach called vectorisation, so you rarely need Python loops over the numbers.

10 min Intermediate · Sublevel Beginner Jump to the activity

What you need: The earlier topics of this module.

Definition

Add 1 to each of a million numbers with a Python loop and it takes a noticeable moment. With NumPy it is about a hundred times faster, and the line of code is shorter too.

NumPy is Python's core library for numerical computing. Its central object, the ndarray, is a grid of values that all share one data type, stored in a single block of memory. Operations on whole arrays run in fast compiled code, an approach called vectorisation, so you rarely need Python loops over the numbers.

In simple words

A Python list is like a bag of assorted items: flexible, but you must handle each one separately. A NumPy array is like a tray of identical boxes in neat rows, which a machine can process all at once.

Overview

  • NumPy (Numerical Python) provides the ndarray, a fast grid of numbers that all share one data type.
  • Operations run on whole arrays at once (vectorisation) in compiled C code, often 10 to 100 times faster than Python loops.
  • It was released in 2006 by Travis Oliphant and is imported by convention as import numpy as np.
  • pandas, scikit-learn, matplotlib, OpenCV and PyTorch all build on NumPy-style arrays.
  • An image, a table of features and a batch of sound clips are all just arrays of numbers.
  • Key attributes are shape, ndim, size and dtype.

Why it matters

NumPy is the foundation of the Python data stack. pandas, scikit-learn, Matplotlib, OpenCV and PyTorch either build on NumPy arrays or work in the same style, so understanding arrays is essential for every later module.

Key terms

ndarray
NumPy's n-dimensional array: a grid of values of one type.
dtype
The data type shared by every element, such as int64 or float64.
Vectorisation
Applying an operation to a whole array at once instead of looping in Python.
Element-wise operation
An operation applied to each pair of matching elements, such as a + b.
Universal function (ufunc)
A fast NumPy function that works element by element, such as np.sqrt.
Array creation functions
np.zeros, np.ones, np.arange and np.linspace, for making arrays quickly.

Types

NumPy arrays are described by how many dimensions they have and how they are created. These are the main types of array you will meet.

1. 1-D array (vector)

A single row of numbers with one axis.

  • How it works: Created with np.array([1, 2, 3]) or np.arange(10); its shape is (n,) and items are reached with one index.
  • Example: Daily temperatures for a week: np.array([31, 33, 32, 35, 36, 34, 30]).
  • When to use it: Use it for one feature measured many times, a time series or one row of data.

2. 2-D array (matrix)

A table of numbers with rows and columns.

  • How it works: Its shape is (rows, columns); a[r, c] reaches one cell, a[r] a row and a[:, c] a column; axis=0 works down columns, axis=1 across rows.
  • Example: A marks table of 40 students × 5 subjects has shape (40, 5).
  • When to use it: Use it for data sets (samples × features), greyscale images and weight matrices of neural networks.

3. N-D array (tensor)

An array with three or more axes.

  • How it works: Each extra axis adds a dimension, such as colour channels or a batch of images; reshape and transpose rearrange axes.
  • Example: A colour photo is (height, width, 3); a batch of 32 such photos is (32, 224, 224, 3).
  • When to use it: Use it for colour images, video and the batches fed to deep-learning models.

4. Arrays from creation functions

Arrays built by NumPy functions rather than from lists: zeros, ones, full, arange, linspace, eye and random.

  • How it works: np.zeros((3, 4)) fills with 0; np.linspace(0, 1, 5) gives 5 evenly spaced values; rng = np.random.default_rng(0); rng.normal(size=100) draws random numbers reproducibly.
  • Example: np.eye(3) is the 3 × 3 identity matrix; np.arange(0, 1, 0.25) is [0, 0.25, 0.5, 0.75].
  • When to use it: Use them to set up empty results, test data, grids for charts and random weights.

5. Arrays by data type (dtype)

Arrays of integers (int64, uint8), floats (float32, float64), booleans or strings.

  • How it works: Every element has the same dtype; astype() converts; smaller dtypes use less memory, e.g. uint8 holds 0–255 in one byte.
  • Example: Images are usually uint8; deep-learning weights are often float32; a mask like marks >= 33 is a boolean array.
  • When to use it: Choose dtypes deliberately for large data, and watch for overflow when small integer types are added together.

Explained in detail

Why NumPy is fast

A Python list stores references to separate objects scattered around memory, and each number carries extra information about its type. Adding 1 to every item means a Python loop that checks types and creates new objects a million times. A NumPy array stores raw numbers side by side in one block, all of the same type, so a single compiled loop written in C can process them, using the processor's ability to handle several numbers per instruction. The result is typically 10 to 100 times faster and uses far less memory. NumPy was created by Travis Oliphant in 2005 by merging two earlier array libraries, Numeric and Numarray, and a 2020 paper in Nature described it as the foundation of scientific computing in Python.

Creating arrays

FunctionExampleResult
np.arraynp.array([72, 85, 64])array([72, 85, 64])
np.zeros / np.onesnp.zeros((2, 3))A 2 × 3 array of 0.0
np.arangenp.arange(0, 10, 2)array([0, 2, 4, 6, 8])
np.linspacenp.linspace(0, 1, 5)Five evenly spaced values from 0 to 1
Randomnp.random.default_rng(0).normal(size=3)Three reproducible random values

Vectorised thinking

Instead of writing a loop, describe the operation for the whole array. To convert marks out of 80 to percentages, write marks / 80 * 100. To find which pass, write marks >= 40, which gives an array of True and False values. To count them, write (marks >= 40).sum(), because True counts as 1. To square every element, write marks ** 2 or np.square(marks). Functions such as np.sqrt, np.exp and np.log apply to every element at once. Learning to think this way, whole arrays rather than single numbers, is the most important skill in this chapter, and it carries straight into pandas and deep-learning libraries.

Data types matter

  • All elements share one dtype; mixing ints and floats gives a float array, and mixing numbers with text gives an array of strings.
  • Integer arrays have fixed sizes, such as int64; values beyond the limit wrap around silently, unlike Python's unlimited int.
  • Images are usually uint8 arrays with values 0 to 255; convert to float before arithmetic that might exceed 255.
  • arr.dtype shows the type and arr.astype(float) converts it.

How it works

  1. Import with the standard alias: import numpy as np.
  2. Create arrays from lists or with creation functions.
  3. Check .shape, .dtype and .ndim before operating.
  4. Express calculations on whole arrays rather than loops.
  5. Use aggregate functions such as .sum(), .mean() and .max() for summaries.

In code

numpy_intro.py
import time
import numpy as np

marks = np.array([72, 38, 85, 64, 91, 40])
print("As a fraction of 100:", marks / 100)
print("Passed:", marks >= 40, "count =", (marks >= 40).sum())
print("Mean, max:", marks.mean().round(1), marks.max())
print("dtype and shape:", marks.dtype, marks.shape)

numbers = list(range(1_000_000))
start = time.perf_counter()
loop_result = [n + 1 for n in numbers]
loop_time = time.perf_counter() - start

arr = np.arange(1_000_000)
start = time.perf_counter()
vector_result = arr + 1
vector_time = time.perf_counter() - start
print("Same answer:", loop_result[-1] == vector_result[-1])
print("NumPy was faster:", vector_time < loop_time)

Output

As a fraction of 100: [0.72 0.38 0.85 0.64 0.91 0.4 ]
Passed: [ True False  True  True  True  True] count = 5
Mean, max: 65.0 91
dtype and shape: int64 (6,)
Same answer: True
NumPy was faster: True

Try it yourself

A million marks

A state exam board must convert a million marks out of 80 into percentages.

Match the NumPy call to its result

What does each produce?

  • np.zeros(3)
  • np.arange(4)
  • np.linspace(0, 1, 3)
  • np.array([1, 2]) * 3

Common mistakes

  • Looping over arrays in Python when a vectorised expression exists.
  • Expecting list behaviour: array * 2 doubles values, while list * 2 repeats the list.
  • Ignoring dtype and getting silent overflow or truncation.

Check yourself

Question 1

1. What does vectorisation mean?

Question 2

2. What is np.array([1, 2, 3]) * 2?

Question 3

3. Why are NumPy arrays faster than lists for numbers?

Summary

  • NumPy's ndarray holds same-type values in one block of memory.
  • Vectorised, element-wise operations replace loops and run much faster.
  • Check shape and dtype; NumPy underpins the whole data and AI stack.

Finished reading?

Sources: NumPy: the absolute basics for beginners · Python for Data Analysis, 3rd edition (Wes McKinney)