Definition
What clues tell an apple from an orange, even with your eyes closed?
In machine learning, an example is one item the model learns from. Its label is the correct answer, such as 'apple' or 'orange'. Its features are the measurable clues the model uses to decide, such as colour, weight or how smooth the skin is.
In simple words
Think of a table. Each row is one fruit (an example). The last column says what it really is (the label). The other columns are clues: colour, weight, skin (features). Good features make the job easy: skin texture separates apples from oranges well. Bad features don't help: both can weigh the same.
Why it matters
Choosing good features is one of the most important decisions in building a model. If the clues don't separate the answers, no amount of training will help.
Key terms
- Example
- One item in the data, such as one fruit, one email or one patient.
- Feature
- A measurable property of an example that may help to predict the answer.
- Label
- The correct answer for an example, which the model must learn to predict.
- Class
- One of the possible labels, such as "apple" or "orange".
- Annotator
- A person who adds labels to data so that a model can learn from it.
- Dataset
- The whole collection of examples, usually arranged as a table.
Explained in detail
Reading a dataset
Most datasets for machine learning can be pictured as a table. Each row is one example. Each column except the last is a feature, a property that has been measured or recorded. The last column is the label, the answer we want the model to predict. For a fruit sorter, a row might read: colour orange, weight 150 grams, skin rough, label "orange". The model studies many such rows and looks for a link between the feature columns and the label column.
What makes a good feature
A feature helps only if it differs between the classes. To separate apples from oranges, skin texture is excellent, because apples are smooth and oranges are rough. Weight is poor, because the two fruits weigh about the same. The day of the week on which the fruit was photographed is useless. A good feature is relevant to the answer, can be measured reliably, and is available at the moment the prediction is needed.
| Task | Useful features | Useless feature |
|---|---|---|
| Is this email spam? | Words used, links, whether the sender is known | The colour of your phone |
| Will it rain tomorrow? | Humidity, air pressure, cloud cover, wind | The name of the forecaster |
| What will this house sell for? | Area, location, number of rooms, age | The owner's favourite food |
Who chooses the features?
For tables of data, people usually choose the features, using their knowledge of the problem. A doctor knows that age and blood pressure matter for heart disease. Choosing and preparing features well is a skilled part of the work. For images, sound and text, deep learning takes a different route. The model is given the raw pixels or words and discovers useful features by itself during training. This is one reason deep learning works so well on photos and language, where it is very hard for a person to say which features matter.
Where labels come from
Labels are the expensive part. Sometimes they come free from records: a bank already knows which past loans were repaid. Sometimes users supply them, as when you press "report spam". Very often people are paid to add them. Annotators look at photos and draw boxes around every pedestrian, or listen to recordings and type the words. Millions of people around the world do this work, and medical labels need qualified doctors, which makes them costly.
Labels can be wrong. A tired annotator marks a wolf as a dog. Two doctors disagree about a scan. A model cannot know that a label is mistaken; it simply learns from it. Careful projects have each example labelled by more than one person and check the cases where they disagree.
How it works
| Colour | Weight (g) | Skin | Label |
|---|---|---|---|
| Red | 150 | Smooth | Apple |
| Orange | 160 | Bumpy | Orange |
| Green | 140 | Smooth | Apple |
| Orange | 150 | Bumpy | Orange |
Here weight is a poor feature, because apples and oranges weigh about the same. Skin is an excellent feature: smooth means apple, bumpy means orange.
Try it yourself
Pull one example apart
An apple and an orange: what does the model actually see?
Label or feature?
A model predicts whether a student will enjoy a book. Is each item a feature or the label?
Number of pages
Type of book: adventure, mystery, science
Did the student enjoy it? Yes or no
Has pictures or not
Choose the clues that separate apples from oranges
Would each feature help a model tell them apart?
Skin texture: smooth or bumpy
Price
Smell
The day it was bought
Weight
Common mistakes
- Mixing up the label and the features. The label is the answer; features are the clues.
- Using features that have nothing to do with the answer.
- Using a feature that secretly contains the answer, which makes the model look better than it is.
Check yourself
Summary
- An example is one item; its label is the right answer.
- Features are measurable clues the model uses.
- Good features separate the answers; poor ones don't help.
Finished reading?
Sources: Google Machine Learning Crash Course