I6 · Neural Networks and Deep Learning Essentials

Convolutional Neural Networks

A convolutional neural network (CNN) processes grid-like data such as images using convolutional layers, in which small learnable filters slide across the input, computing a dot product at each position to produce feature maps. Because the same filter is used everywhere, CNNs share weights and detect a pattern wherever it appears. Pooling layers shrink feature maps, and stacking layers builds features from edges to whole objects.

12 min Intermediate · Sublevel Intermediate Jump to the activity

What you need: The earlier topics of this module.

Definition

A 224 × 224 colour photo has about 150,000 numbers. A dense layer of 1,000 neurons would need 150 million weights just to look at it, and would have to learn separately that a cat in the corner is the same as a cat in the middle. Convolutional networks solve both problems.

A convolutional neural network (CNN) processes grid-like data such as images using convolutional layers, in which small learnable filters slide across the input, computing a dot product at each position to produce feature maps. Because the same filter is used everywhere, CNNs share weights and detect a pattern wherever it appears. Pooling layers shrink feature maps, and stacking layers builds features from edges to whole objects.

In simple words

Imagine scanning a photo with a small magnifying window that looks for one thing, such as a vertical edge. Slide it everywhere and mark where it finds edges. Use many windows for many patterns, then look for patterns of patterns. That is a CNN.

Overview

  • A convolutional neural network (CNN) learns small filters that slide across an image to detect patterns.
  • Early layers detect edges and textures; deeper layers detect parts and whole objects.
  • Weight sharing means the same filter is used everywhere, so CNNs need far fewer parameters than dense networks.
  • Pooling layers shrink feature maps and make detection tolerant of small shifts.
  • Yann LeCun's LeNet read handwritten cheque digits in the 1990s; AlexNet won ImageNet in 2012.
  • CNNs also process audio spectrograms, medical scans and satellite images.

Why it matters

CNNs launched the deep-learning era in computer vision and still power image classification, medical scan analysis, face detection, self-driving perception and quality inspection in factories.

Key terms

Convolution
Sliding a small filter across an input and computing a weighted sum at each position.
Filter (kernel)
A small grid of learnable weights, such as 3 × 3, that detects one pattern.
Feature map
The output of one filter across the whole input, showing where its pattern appears.
Stride and padding
How far the filter moves each step, and extra border added so outputs keep their size.
Pooling
Shrinking a feature map, often by taking the maximum in each 2 × 2 block.
Weight sharing
Using the same filter weights at every position, which cuts parameters and gives translation invariance.

Types

CNNs are built from several types of layer and come in several famous architecture types.

1. Convolutional layers

Layers that apply learned filters (kernels) across the input.

  • How it works: Each filter, often 3 × 3, computes a weighted sum at every position; stride sets the step and padding keeps the border.
  • Example: 64 filters of size 3 × 3 × 3 on a colour image produce 64 feature maps with 1,792 parameters.
  • When to use it: Use them as the core of any network that processes images or other grids.

2. Pooling layers

Layers that downsample feature maps by taking the maximum or average in small windows.

  • How it works: 2 × 2 max pooling with stride 2 halves the height and width, keeping the strongest response in each window.
  • Example: A 32 × 32 feature map becomes 16 × 16; global average pooling turns each map into one number before the classifier.
  • When to use it: Use them to reduce computation and add tolerance to small shifts.

3. Classic CNN architectures

LeNet-5 (1998), AlexNet (2012), VGG (2014) and GoogLeNet/Inception (2014).

  • How it works: They stack convolution and pooling blocks, then dense layers; AlexNet used ReLU, dropout and GPUs; VGG used only 3 × 3 filters; Inception combined several filter sizes in parallel.
  • Example: AlexNet cut ImageNet top-5 error to about 15%, far below the next entry's 26%.
  • When to use it: Study them to see how design ideas developed; use modern architectures in practice.

4. Residual networks (ResNet) and efficient CNNs

ResNet (2015) adds skip connections; MobileNet and EfficientNet are designed for speed and small size.

  • How it works: Skip connections add a block's input to its output, letting gradients flow through 100+ layers; depthwise separable convolutions cut computation for phones.
  • Example: ResNet-50 as a standard backbone; MobileNet running on a phone to identify crop diseases.
  • When to use it: Use them as pre-trained backbones for image classification and detection.

5. Task-specific CNNs

CNN designs for detection (YOLO, Faster R-CNN) and segmentation (U-Net).

  • How it works: Detection heads predict boxes and classes; U-Net's encoder-decoder with skip connections outputs a label for every pixel.
  • Example: YOLO detecting vehicles in traffic video; U-Net outlining organs in CT scans.
  • When to use it: Use them when you need locations or outlines, not just one label per image.

Explained in detail

A convolution step by step

Take a 3 × 3 filter designed to detect vertical edges: its left column is −1, middle 0, right +1. Place it on the top-left 3 × 3 patch of a grayscale image, multiply each pixel by the matching weight and add the nine products: the result is large when the right side of the patch is brighter than the left, that is, at a vertical edge. Slide the filter one pixel right and repeat, then across every row. The grid of results is a feature map. In a trained CNN, nobody designs filters by hand; the network learns the values of dozens or hundreds of filters per layer through backpropagation.

Why convolution works for images

PropertyMeaningBenefit
Local connectionsEach output looks at a small neighbourhoodMatches how image patterns are local: edges, textures
Weight sharingThe same filter is used at every positionFar fewer parameters; a 3 × 3 filter has 9 weights however large the image
Translation invarianceA pattern is detected wherever it appearsA cat is recognised in any part of the photo
HierarchyStacked layers see larger regionsEdges, then textures, then parts, then objects

A typical architecture

A classic CNN repeats a block of convolution, ReLU and pooling several times, with the number of filters increasing as the feature maps shrink: for example 32 filters at full size, 64 at half size, 128 at a quarter. The final feature maps are flattened or averaged and passed to dense layers that output class probabilities. Yann LeCun's LeNet read handwritten postal codes and bank cheques in the 1990s. In 2012, AlexNet, by Alex Krizhevsky, Ilya Sutskever and Geoffrey Hinton, cut the error on the ImageNet competition of 1.2 million images in 1,000 categories from about 26 per cent to about 15 per cent, using GPUs and ReLU, which convinced the field that deep learning worked. Later designs such as VGG, ResNet and EfficientNet went deeper and more efficient.

  • Output size of a convolution: (input − filter + 2 × padding) ÷ stride + 1.
  • Max pooling with a 2 × 2 window and stride 2 halves the width and height.
  • CNNs also work on audio spectrograms, time series and other grid-like data.

How it works

  1. Scale pixel values, for example to 0 to 1.
  2. Stack blocks of convolution, ReLU and pooling, increasing filters as size shrinks.
  3. Flatten or globally average the final feature maps.
  4. Add a dense output layer with softmax for classification.
  5. Train with augmentation, or start from a pre-trained CNN.

In code

convolution.py
import numpy as np

image = np.zeros((6, 6))
image[:, 3:] = 1.0  # left half dark, right half bright: a vertical edge
vertical_edge = np.array([[-1, 0, 1], [-1, 0, 1], [-1, 0, 1]])

def convolve(img, kernel):
    k = kernel.shape[0]
    out = np.zeros((img.shape[0] - k + 1, img.shape[1] - k + 1))
    for r in range(out.shape[0]):
        for c in range(out.shape[1]):
            out[r, c] = np.sum(img[r:r + k, c:c + k] * kernel)
    return out

def max_pool(fmap, size=2):
    h, w = fmap.shape[0] // size, fmap.shape[1] // size
    return fmap[:h * size, :w * size].reshape(h, size, w, size).max(axis=(1, 3))

feature_map = convolve(image, vertical_edge)
print("Feature map (strong values mark the edge):\n", feature_map)
print("After 2x2 max pooling:\n", max_pool(feature_map))
print("Weights in a 3x3 filter:", vertical_edge.size, " vs a dense layer on a 224x224x3 image to 1,000 neurons:", 224 * 224 * 3 * 1000)

Output

Feature map (strong values mark the edge):
 [[0. 3. 3. 0.]
 [0. 3. 3. 0.]
 [0. 3. 3. 0.]
 [0. 3. 3. 0.]]
After 2x2 max pooling:
 [[3. 3.]
 [3. 3.]]
Weights in a 3x3 filter: 9  vs a dense layer on a 224x224x3 image to 1,000 neurons: 150528000

Try it yourself

Spotting disease on X-rays

A hospital trains a CNN to flag chest X-rays that may show tuberculosis.

Match the CNN part to its job

What does each part do?

  • Filter
  • Feature map
  • Max pooling
  • Dense output with softmax

Common mistakes

  • Using dense layers on raw images instead of convolutions.
  • Training a large CNN from scratch on a few hundred images.
  • Ignoring that CNNs can learn spurious cues, such as hospital-specific markings on X-rays.

Check yourself

Question 1

1. What does weight sharing in a CNN mean?

Question 2

2. A 2 × 2 max pool with stride 2 on a 32 × 32 feature map gives…

Question 3

3. Which 2012 network convinced the field that deep learning worked for vision?

Summary

  • CNNs slide small learnable filters across images to produce feature maps.
  • Local connections and weight sharing cut parameters and detect patterns anywhere.
  • Stacked convolution and pooling build features from edges to objects; AlexNet in 2012 started the deep-learning era.

Read it. Now try it yourself.

Open the interactive activity

Finished reading?

Sources: Stanford CS231n: Deep Learning for Computer Vision · Dive into Deep Learning (Zhang, Lipton, Li, Smola)