Definition
Can you teach a computer to tell cats from dogs in two minutes?
Training a model means showing it many labelled examples so that it can find a pattern linking the features to the labels. A model is the result of training: the learned pattern, ready to make predictions about new cases.
In simple words
In the activity, every animal has two clues: its size and how long its snout is. You choose which labelled examples the model learns from. When it meets a new animal, it looks for the three examples most like it and goes with the majority. The examples you choose are all it knows.
Why it matters
Training is where a model gets all of its knowledge. If the training examples are too few, one-sided or wrongly labelled, the model will be wrong in exactly those ways. Whoever chooses the training data shapes the AI.
Key terms
- Training
- The process of adjusting a model, example by example, until its predictions are mostly right.
- Model
- The result of training: a learned pattern, stored as numbers, that turns an input into a prediction.
- Parameter
- One of the adjustable numbers inside a model. Training sets their values.
- Training set
- The labelled examples used to train the model.
- Epoch
- One complete pass through all the training examples.
- Error
- The gap between the model's prediction and the correct answer.
Explained in detail
What a model is
A model is not a list of stored examples and it is not a set of rules written by a person. It is a kind of adjustable machine. Picture a radio with thousands of tuning knobs. The setting of the knobs decides what comes out for any input. Before training, the knobs are set at random and the output is nonsense. Training is the process of turning the knobs, little by little, until the outputs match the right answers. The knobs are called parameters. A small model may have a few dozen; the largest language models have hundreds of billions.
The training loop
- Show the model one training example, without its label.
- The model makes a prediction using its current settings.
- Compare the prediction with the label. The difference is the error.
- Adjust the parameters slightly in the direction that would have made the error smaller.
- Move to the next example and repeat.
One pass through every training example is called an epoch. Models usually need many epochs. In the first, almost every prediction is wrong. With each pass the errors shrink, quickly at first and then more slowly, until further training brings little improvement. The adjustments are worked out by mathematics, automatically; no person decides how to turn each knob.
What training needs
| Ingredient | Why it matters |
|---|---|
| Labelled examples | The model can only learn patterns that the examples contain |
| A type of model | A simple one for small tables, a deep network for images and language |
| Computing power | Each adjustment is arithmetic, repeated millions or billions of times |
| Time | Seconds for a small model, weeks for the largest |
The cat-and-dog model in the lab on this page trains in an instant, because it has a handful of examples. Training a large language model is a different scale: thousands of specialised chips running for weeks, using as much electricity as a small town. That cost is one reason only a few organisations build the very largest models.
What the model keeps
After training, the examples can be put away. What remains is the model: the final settings of its parameters. It does not hold the photos it learned from, just as a cricketer does not carry around every ball ever bowled in practice. The practice shaped the skill, and the skill is what stays. This also explains a limit. The model knows only what the examples could teach. If you trained it with no small dogs, it has no knowledge of small dogs, and no amount of extra training on the same examples will add it.
How it works
- Choose the task: tell cats from dogs.
- Choose the features: size and snout length.
- Collect labelled examples of both kinds.
- Let the model learn from them.
- Test it on animals it has never seen.
Try it yourself
Train a model in the browser
Add labelled examples, then test the model on new animals. Try a few examples first, then more.
Common mistakes
- Training with examples of only one kind.
- Using too few examples and expecting perfect results.
- Thinking the model understands animals. It only compares numbers for size and snout.
Check yourself
Summary
- Training means learning a pattern from labelled examples.
- The trained model is the learned pattern, used on new cases.
- The training examples decide what the model knows and gets wrong.
Finished reading?
Sources: Google Machine Learning Crash Course