Definition
Five lines of Keras can define, train and evaluate a neural network that reads handwritten digits with about 98 per cent accuracy. Each line hides exactly the ideas you have learned in this module.
Keras is a high-level Python library for building neural networks, which runs on TensorFlow, PyTorch or JAX. A typical workflow has five steps: prepare and scale the data, define the model as a stack of layers, compile it with an optimiser, a loss and metrics, fit it to the training data for a number of epochs, and evaluate it on test data before using predict on new inputs.
In simple words
Keras is like building with standard bricks. You choose the bricks (layers), stack them in order, say how you will judge the building (loss and metrics) and how you will improve it (optimiser), and Keras does the heavy lifting of training.
Overview
- Keras is a high-level deep-learning library, created by François Chollet in 2015 and built into TensorFlow.
- A model is built by stacking layers, compiled with a loss and optimiser, then trained with fit().
- evaluate() measures test performance and predict() makes predictions.
- Callbacks such as EarlyStopping and ModelCheckpoint control training.
- Keras 3 can run on TensorFlow, PyTorch or JAX back ends.
- This site's browser cannot run Keras, so the course shows the same steps in NumPy and explains the Keras equivalents.
Why it matters
Keras is one of the most widely used deep-learning libraries in education and industry. Writing a first model in it connects the theory of layers, activations, losses and optimisers to working code you can extend to images, text and more.
Key terms
- Sequential model
- A model built as a simple stack of layers, each feeding the next.
- Dense layer
- keras.layers.Dense(units, activation): a fully connected layer.
- compile
- Sets the optimiser, loss function and metrics before training.
- fit
- Trains the model for a number of epochs with a given batch size and validation split.
- evaluate and predict
- Measure performance on test data, and produce outputs for new inputs.
- History
- The object fit returns, holding loss and metrics per epoch for plotting.
Types
Keras offers three types of model-building API and several types of layer for a first network.
1. Sequential API
Building a model as a simple stack of layers, one after another.
- How it works: keras.Sequential([layers.Input((784,)), layers.Dense(128, activation="relu"), layers.Dense(10, activation="softmax")]).
- Example: A digit classifier with one hidden layer of 128 neurons.
- When to use it: Use it for most first networks, where data flows straight through. Call model.summary() to check every layer's output shape and parameter count before training.
2. Functional API
Building models as graphs of layers, with branches, multiple inputs or outputs.
- How it works: Each layer is called on the output of another; keras.Model(inputs, outputs) wraps the graph.
- Example: A model taking both a product photo and its description to predict the category.
- When to use it: Use it for non-linear topologies, shared layers and residual connections.
3. Model subclassing
Writing a Python class that defines layers in __init__ and the forward pass in call().
- How it works: It gives full control, including loops and conditions in the forward pass.
- Example: Research models with custom behaviour at each step.
- When to use it: Use it when the other APIs cannot express your model.
4. Core layers for a first network
Input, Dense, Dropout, Flatten, Conv2D and MaxPooling2D.
- How it works: Dense layers learn weighted sums; Dropout regularises; Flatten turns images into vectors; Conv2D and pooling handle images.
- Example: A small CNN for 28 × 28 Fashion-MNIST images reaching about 90% test accuracy.
- When to use it: Use them to build and experiment with your first models.
5. Compile, fit and callbacks
The training set-up: choosing a loss, optimiser and metrics, then training with callbacks.
- How it works: model.compile(optimizer="adam", loss="sparse_categorical_crossentropy", metrics=["accuracy"]); model.fit(X, y, epochs=20, validation_split=0.2, callbacks=[EarlyStopping(patience=3)]).
- Example: Training stops automatically when validation loss stops improving.
- When to use it: Use this standard recipe for every Keras model. Change only one thing at a time, such as the learning rate or the number of neurons, and compare validation curves after each change.
Explained in detail
The Keras code, line by line
Below is the standard first Keras program for the MNIST data set of 70,000 handwritten digit images, each 28 × 28 pixels. You can run it in Google Colab, where TensorFlow and Keras are already installed. The table explains every line in terms you already know; the runnable example on this page then builds the same kind of network by hand with NumPy so you can see what Keras does inside.
| Keras line | What it does |
|---|---|
| (x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data() | Loads 60,000 training and 10,000 test images |
| x_train = x_train.reshape(-1, 784) / 255.0 | Flattens each image to 784 numbers and scales them to 0 to 1 |
| model = keras.Sequential([keras.Input(shape=(784,)), keras.layers.Dense(128, activation="relu"), keras.layers.Dropout(0.2), keras.layers.Dense(10, activation="softmax")]) | 784 inputs, a hidden layer of 128 ReLU neurons, dropout, and 10 softmax outputs: 101,770 parameters |
| model.compile(optimizer="adam", loss="sparse_categorical_crossentropy", metrics=["accuracy"]) | Adam optimiser; cross-entropy loss for integer labels; report accuracy |
| history = model.fit(x_train, y_train, epochs=5, batch_size=32, validation_split=0.1) | Trains for 5 epochs of 1,688 batches each, holding back 10 per cent for validation |
| model.evaluate(x_test, y_test) | Reports test loss and accuracy, typically around 0.97 to 0.98 |
| model.predict(x_test[:1]).argmax() | Gives the most probable digit for one new image |
What fit does inside
For each epoch, fit shuffles the data, splits it into batches of 32, runs the forward pass, computes the loss, uses automatic differentiation to backpropagate gradients through every layer and lets Adam update all 101,770 weights. At the end of the epoch it computes validation loss and accuracy and prints a progress line. The returned History object holds those numbers so you can plot learning curves. Adding callbacks such as EarlyStopping(patience=3, restore_best_weights=True) gives you early stopping in one line.
- To improve further, replace the dense layers with Conv2D and MaxPooling2D layers on the 28 × 28 images: accuracy rises to about 99 per cent.
- model.summary() prints each layer's output shape and parameter count, a quick check that the architecture is what you intended.
- model.save("digits.keras") stores the trained model so it can be loaded later or converted for phones and browsers.
The NumPy example on this page uses the smaller 8 × 8 digits that come with scikit-learn, so it runs in seconds in a browser. It follows exactly the same steps: a 64-input network with one hidden layer of 32 ReLU neurons and a softmax output, trained in mini-batches with cross-entropy loss.
How it works
- Load the data and scale inputs.
- Define a Sequential model with an Input, hidden Dense layers and a suitable output layer.
- Compile with an optimiser, a loss that fits the task and metrics.
- Fit with a validation split and, ideally, early stopping.
- Evaluate on the test set, inspect mistakes and save the model.
In code
import numpy as np
from sklearn.datasets import load_digits
from sklearn.model_selection import train_test_split
# The same steps Keras runs, written out with NumPy on 8x8 digit images
X, y = load_digits(return_X_y=True)
X = X / 16.0 # scale inputs to 0-1
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.25, random_state=0)
Y_tr = np.eye(10)[y_tr] # one-hot labels
rng = np.random.default_rng(0)
W1, b1 = rng.normal(0, 0.1, (64, 32)), np.zeros(32) # Dense(32, relu)
W2, b2 = rng.normal(0, 0.1, (32, 10)), np.zeros(10) # Dense(10, softmax)
print("Parameters:", W1.size + b1.size + W2.size + b2.size)
def forward(X):
h = np.maximum(0, X @ W1 + b1)
z = h @ W2 + b2
p = np.exp(z - z.max(axis=1, keepdims=True))
return h, p / p.sum(axis=1, keepdims=True)
lr, batch = 0.1, 32
for epoch in range(1, 21): # fit(epochs=20, batch_size=32)
order = rng.permutation(len(X_tr))
for s in range(0, len(X_tr), batch):
i = order[s:s + batch]
h, p = forward(X_tr[i])
dz = (p - Y_tr[i]) / len(i) # gradient of cross-entropy with softmax
dh = (dz @ W2.T) * (h > 0)
W2 -= lr * h.T @ dz; b2 -= lr * dz.sum(axis=0)
W1 -= lr * X_tr[i].T @ dh; b1 -= lr * dh.sum(axis=0)
if epoch % 5 == 0:
_, p = forward(X_tr)
loss = -np.mean(np.log(p[np.arange(len(y_tr)), y_tr]))
print(f"epoch {epoch:2d} loss {loss:.3f}")
_, p_test = forward(X_te) # evaluate
print("Test accuracy:", round(float((p_test.argmax(axis=1) == y_te).mean()), 3))
print("Prediction for first test image:", p_test[0].argmax(), " true label:", y_te[0])Output
Parameters: 2410 epoch 5 loss 0.470 epoch 10 loss 0.265 epoch 15 loss 0.246 epoch 20 loss 0.118 Test accuracy: 0.956 Prediction for first test image: 2 true label: 2
Try it yourself
The Keras workflow
Put the steps in order.
- 1Evaluate on the test set
- 2Predict on new inputs and save the model
- 3Fit on training data with a validation split
- 4Define the Sequential model and its layers
- 5Load and scale the data
- 6Compile with optimiser, loss and metrics
Match the Keras argument to its meaning
What does each one set?
- activation="softmax"
- optimizer="adam"
- epochs=5
- validation_split=0.1
- Dropout(0.2)
Common mistakes
- Forgetting to scale pixel values, so training is slow or unstable.
- Using categorical_crossentropy with integer labels instead of sparse_categorical_crossentropy.
- Reporting validation accuracy as the final result instead of test accuracy.
Check yourself
Summary
- The Keras workflow is prepare, define, compile, fit, evaluate and predict.
- Each call maps to ideas from this module: layers, activations, losses, optimisers, epochs and regularisation.
- Inside fit, batches flow forward, gradients flow back and the optimiser updates every weight.
Finished reading?
Sources: Keras documentation: getting started · Dive into Deep Learning (Zhang, Lipton, Li, Smola)