Intermediate · module I6 · Sublevel Intermediate
Neural Networks and Deep Learning Essentials
By the end: Explain how a neural network learns, train a small network, and describe what CNNs, RNNs and Transformers are used for
- Covers
- How neural networks work and learn, and the main deep learning architectures, with hands-on simulators
- Contents
- 4 chapters, 19 topics
- Time
- About 4 hours
- Level
- Intermediate
- Your progress
- 0 / 19
This module builds on Machine Learning Foundations. We suggest looking at it first, but the choice is yours.
Chapter 1 From Neuron to Network
- 1.110 min Read Activity
Biological and Artificial Neurons
What the brain inspired, and what a computer neuron actually computes
Interactive: one neuron, three dials
y = f(w·x + b)
- 1.210 min Read Activity
The Perceptron
Weights, bias and a threshold, and the XOR problem
Interactive: separate two groups with a line
- 1.310 min Read
Activation Functions
Sigmoid, tanh, ReLU and softmax, and why they are needed
Interactive: compare activation curves
ReLU(x) = max(0, x)
- 1.412 min Read Activity
Layers and Network Architecture
Input, hidden and output layers, and depth
Interactive: build a network layer by layer
Chapter 2 How Networks Learn
- 2.110 min Read
Forward Propagation and Loss
Producing a prediction and measuring its error
Interactive: trace numbers through a network
- 2.212 min Read
Backpropagation and Gradient Descent
Sending the error backwards to adjust every weight
Interactive: watch weights update
- 2.310 min Read Activity
Epochs, Batches and Learning Rate
The settings that control training
Interactive: train with different learning rates
- 2.410 min Read Activity
- 2.510 min Read Activity
Optimisers: SGD, Momentum and Adam
How different update rules speed up and steady training
Code: race three optimisers downhill
m ← βm + (1 − β)g
Chapter 3 Deep Learning Architectures
- 3.112 min Read Activity
Convolutional Neural Networks
How networks find edges, shapes and objects in images
Interactive: apply image kernels
- 3.210 min Read Activity
Recurrent Neural Networks and LSTMs
Networks with memory for sequences
Interactive: predict the next letter
- 3.312 min Read Activity
Attention and Transformers
How a model decides which words matter, the idea behind every LLM
Interactive: attention heat map
- 3.412 min Read Activity
Transfer Learning and Pre-Trained Models
Reusing a trained model for a new task
Code: classify images with a pre-trained model
- 3.510 min Read Activity
Autoencoders
Networks that compress and rebuild their input, for denoising and anomaly detection
Code: compress handwritten digits
- 3.610 min Read Activity
Generative Adversarial Networks
A generator and a discriminator competing to create realistic data
Code: a discriminator judging fakes
Chapter 4 Building Networks
- 4.112 min Read Activity
Training a Network Without Code
Exploring networks in a visual simulator
Lab: neural network playground
- 4.220 min Read Activity
Your First Network in Keras
Building, training and evaluating a small network
Code: a digit recogniser
- 4.36 min Read Activity
- 4.415 min Read Activity