Definition
Nobody wrote a rule for every spam email. So how does your inbox catch them?
Machine Learning (ML) is the part of AI in which a computer learns patterns from data, instead of being given step-by-step rules. Tom Mitchell's definition (1997) is the standard one: a program learns from experience E, for tasks T and performance measure P, if its performance at T, measured by P, improves with E.
In simple words
Think of teaching a child what a mango is. You don't give a list of rules about colour and shape. You point at mangoes and say "mango". After enough examples, the child recognises a new mango. ML works the same way: show many examples with the right answers, and the computer works out the pattern by itself.
Overview
- Machine learning (ML) is the field of AI in which computers learn patterns from data instead of following hand-written rules.
- Arthur Samuel used the term in 1959 for a checkers program that improved by playing against itself.
- Tom Mitchell's definition: a program learns from experience E on task T if its performance P improves with E.
- A model is trained on examples and then used to make predictions on new data.
- ML suits problems where rules are hard to write but data is plentiful, such as recognising speech or spotting fraud.
- Its quality depends on the data: biased or poor data gives biased or poor models.
Why it matters
Some problems are too hard to write as rules. Try writing rules that recognise any handwritten 7, or any fraudulent payment. ML solves these by learning from examples, and it keeps improving as more data arrives. It powers recommendations, fraud checks, voice typing, translation and chatbots.
Key terms
- Machine learning
- Building systems that improve at a task by learning patterns from data rather than following hand-written rules.
- Model
- The learned function that maps inputs to outputs.
- Training
- The process of adjusting a model's parameters to fit examples.
- Inference
- Using a trained model to make predictions on new data.
- Generalisation
- Performing well on new, unseen examples, not just the training data.
- Parameter
- A number inside the model, such as a weight, that training adjusts.
Types
Machine-learning problems are grouped into a few task types by what the model must produce. These are the main ones.
1. Classification
Predicting which category an example belongs to.
- How it works: The model learns boundaries between labelled classes and outputs a class, often with a probability.
- Example: Spam or not spam; which digit is in an image; whether a loan applicant will repay.
- When to use it: Use it whenever the answer is one of a fixed set of labels, from medical diagnosis to sorting customer complaints.
2. Regression
Predicting a number.
- How it works: The model learns how a continuous target changes with the features and outputs a value.
- Example: House prices, tomorrow's temperature, delivery time or crop yield.
- When to use it: Use it when the answer is a quantity and errors are measured in its units, such as rupees or minutes.
3. Clustering
Grouping similar examples without labels.
- How it works: Algorithms such as k-means or DBSCAN place examples close in feature space into the same group.
- Example: Customer segments for marketing, or grouping news articles by topic.
- When to use it: Use it to explore data and discover structure when no labels exist. Clusters still need people to name and check them.
4. Recommendation and ranking
Ordering items by how relevant or appealing they are to a user.
- How it works: Models learn from past behaviour and item features to score and sort items.
- Example: Videos on a streaming app, products on a shopping site, results in a search engine.
- When to use it: Use it when users must choose from many options and personal relevance matters.
5. Generation
Creating new content, such as text, images, audio or code.
- How it works: Generative models learn the patterns of training data and sample new examples that follow them.
- Example: Chatbots writing answers, diffusion models drawing images, translation systems producing sentences.
- When to use it: Use it for creative and language tasks, with checks for errors, bias and misuse.
Explained in detail
Rules written by people versus rules learned from data
In traditional programming, a person writes the rules and the computer applies them to data to produce answers. In machine learning, the person supplies data and the answers, and the computer works out the rules. For a spam filter, writing rules by hand, block "lottery", block "urgent transfer", soon becomes an endless and fragile list, because spammers change their wording. A learning system studies thousands of labelled emails and discovers which combinations of words, senders and links tend to signal spam, then keeps improving as new examples arrive.
Mitchell's definition, unpacked
| Part | Meaning | Spam example |
|---|---|---|
| Task T | What the program should do | Classify emails as spam or not |
| Experience E | The data it learns from | Thousands of emails labelled by users |
| Performance P | How success is measured | Share of emails classified correctly, and how few real emails are blocked |
Tom Mitchell's 1997 definition says a program learns if its performance on T, measured by P, improves with experience E. It is useful because it forces you to state all three clearly before building anything. Arthur Samuel, who built a self-improving draughts program at IBM in the 1950s, is credited with popularising the term "machine learning" in 1959.
When machine learning is the right tool
- The rules are too complex to write down, such as recognising faces or speech.
- The patterns change over time, such as fraud tactics or shopping trends.
- There is plenty of relevant, labelled or structured data.
- Some mistakes are acceptable, or a human can check the important ones.
Machine learning is the wrong tool when a simple rule already works, when there is little or poor data, when every decision must be perfectly explainable by law, or when a single error could be catastrophic and cannot be checked. Many real systems combine both approaches: learned models make predictions, and hand-written rules add safety limits.
Training and inference
- Collect examples with inputs and the correct outputs.
- Training: the algorithm adjusts the model's parameters to reduce its errors on those examples.
- Evaluation: test the model on examples it never saw.
- Inference: use the trained model on new inputs, often millions of times, quickly and cheaply.
How it works
Programming vs machine learning
| You give the computer | You get back | |
|---|---|---|
| Traditional programming | Rules + data | Answers |
| Machine learning | Data + answers | Rules (a model) |
Mitchell's definition, on a spam filter
- Task T: decide whether an email is spam.
- Experience E: thousands of emails that people have already marked.
- Performance P: the share of new emails it gets right.
The steps of learning
- Collect examples, each with its features (what you know) and label (the right answer).
- Choose a model: a formula with adjustable numbers called parameters.
- Train: the model predicts, measures how wrong it was, and adjusts its parameters. Repeat many times.
- Test on examples it has never seen. That is the only fair measure.
- Use it on new data. This step is called inference.
The three main kinds
| Kind | Learns from | Example |
|---|---|---|
| Supervised | Examples with right answers | Predicting house prices |
| Unsupervised | Examples with no answers | Grouping similar customers |
| Reinforcement | Rewards for its actions | Learning to play a game |
The maths
- The model's parameters, the numbers it can adjust
- The best parameters found
- The model's prediction for example i
- The right answer for example i
- The loss: how wrong one prediction is
- The number of examples
In words: find the parameters that make the average error as small as possible. Suppose a model predicts 3 house prices (in lakhs) as 50, 60 and 80, and the right answers are 52, 60 and 75. Using squared error as the loss:
Average loss = [ (50−52)² + (60−60)² + (80−75)² ] / 3
= [ 4 + 0 + 25 ] / 3
= 9.67Training adjusts the parameters so this number falls. That's all "learning" means here.
In code
A complete ML program with scikit-learn. The model is never told the rule behind the numbers. It works it out from the examples.
from sklearn.linear_model import LinearRegression
# Experience: house size in sq ft, and price in lakhs
sizes = [[500], [800], [1000], [1200], [1500]]
prices = [25, 40, 50, 60, 75]
model = LinearRegression()
model.fit(sizes, prices) # learning happens here
# The true intercept is 0; adding 0.0 stops a tiny rounding error from printing as -0.0.
print("Learned rule: price =", round(model.coef_[0], 3), "x size +", round(model.intercept_, 3) + 0.0)
print("A 1100 sq ft house:", round(model.predict([[1100]])[0], 1), "lakhs")Output
Learned rule: price = 0.05 x size + 0.0 A 1100 sq ft house: 55.0 lakhs
Common mistakes
- Testing a model on the data it trained on. It may have just memorised the answers.
- Thinking more data always fixes everything. Wrong or biased data teaches wrong patterns.
- Using ML when a simple rule would do. If you can write the rule in five lines, write the rule.
Check yourself
Summary
- ML learns patterns from data, instead of following hand-written rules.
- Mitchell: performance at a task, measured by P, improves with experience E.
- Training means adjusting parameters to reduce the average loss.
- Three main kinds: supervised, unsupervised and reinforcement.
Read it. Now try it yourself.
Open the interactive activityFinished reading?
Sources: Mitchell, Machine Learning (1997) · Google Machine Learning Crash Course