I5 · Machine Learning Foundations

What Is Machine Learning?

Machine Learning (ML) is the part of AI in which a computer learns patterns from data, instead of being given step-by-step rules. Tom Mitchell's definition (1997) is the standard one: a program learns from experience E, for tasks T and performance measure P, if its performance at T, measured by P, improves with E.

12 min Intermediate · Sublevel Intermediate

What you need: The earlier topics of this module.

Definition

Nobody wrote a rule for every spam email. So how does your inbox catch them?

Machine Learning (ML) is the part of AI in which a computer learns patterns from data, instead of being given step-by-step rules. Tom Mitchell's definition (1997) is the standard one: a program learns from experience E, for tasks T and performance measure P, if its performance at T, measured by P, improves with E.

In simple words

Think of teaching a child what a mango is. You don't give a list of rules about colour and shape. You point at mangoes and say "mango". After enough examples, the child recognises a new mango. ML works the same way: show many examples with the right answers, and the computer works out the pattern by itself.

Overview

  • Machine learning (ML) is the field of AI in which computers learn patterns from data instead of following hand-written rules.
  • Arthur Samuel used the term in 1959 for a checkers program that improved by playing against itself.
  • Tom Mitchell's definition: a program learns from experience E on task T if its performance P improves with E.
  • A model is trained on examples and then used to make predictions on new data.
  • ML suits problems where rules are hard to write but data is plentiful, such as recognising speech or spotting fraud.
  • Its quality depends on the data: biased or poor data gives biased or poor models.

Why it matters

Some problems are too hard to write as rules. Try writing rules that recognise any handwritten 7, or any fraudulent payment. ML solves these by learning from examples, and it keeps improving as more data arrives. It powers recommendations, fraud checks, voice typing, translation and chatbots.

Key terms

Machine learning
Building systems that improve at a task by learning patterns from data rather than following hand-written rules.
Model
The learned function that maps inputs to outputs.
Training
The process of adjusting a model's parameters to fit examples.
Inference
Using a trained model to make predictions on new data.
Generalisation
Performing well on new, unseen examples, not just the training data.
Parameter
A number inside the model, such as a weight, that training adjusts.

Types

Machine-learning problems are grouped into a few task types by what the model must produce. These are the main ones.

1. Classification

Predicting which category an example belongs to.

  • How it works: The model learns boundaries between labelled classes and outputs a class, often with a probability.
  • Example: Spam or not spam; which digit is in an image; whether a loan applicant will repay.
  • When to use it: Use it whenever the answer is one of a fixed set of labels, from medical diagnosis to sorting customer complaints.

2. Regression

Predicting a number.

  • How it works: The model learns how a continuous target changes with the features and outputs a value.
  • Example: House prices, tomorrow's temperature, delivery time or crop yield.
  • When to use it: Use it when the answer is a quantity and errors are measured in its units, such as rupees or minutes.

3. Clustering

Grouping similar examples without labels.

  • How it works: Algorithms such as k-means or DBSCAN place examples close in feature space into the same group.
  • Example: Customer segments for marketing, or grouping news articles by topic.
  • When to use it: Use it to explore data and discover structure when no labels exist. Clusters still need people to name and check them.

4. Recommendation and ranking

Ordering items by how relevant or appealing they are to a user.

  • How it works: Models learn from past behaviour and item features to score and sort items.
  • Example: Videos on a streaming app, products on a shopping site, results in a search engine.
  • When to use it: Use it when users must choose from many options and personal relevance matters.

5. Generation

Creating new content, such as text, images, audio or code.

  • How it works: Generative models learn the patterns of training data and sample new examples that follow them.
  • Example: Chatbots writing answers, diffusion models drawing images, translation systems producing sentences.
  • When to use it: Use it for creative and language tasks, with checks for errors, bias and misuse.

Explained in detail

Rules written by people versus rules learned from data

In traditional programming, a person writes the rules and the computer applies them to data to produce answers. In machine learning, the person supplies data and the answers, and the computer works out the rules. For a spam filter, writing rules by hand, block "lottery", block "urgent transfer", soon becomes an endless and fragile list, because spammers change their wording. A learning system studies thousands of labelled emails and discovers which combinations of words, senders and links tend to signal spam, then keeps improving as new examples arrive.

Mitchell's definition, unpacked

PartMeaningSpam example
Task TWhat the program should doClassify emails as spam or not
Experience EThe data it learns fromThousands of emails labelled by users
Performance PHow success is measuredShare of emails classified correctly, and how few real emails are blocked

Tom Mitchell's 1997 definition says a program learns if its performance on T, measured by P, improves with experience E. It is useful because it forces you to state all three clearly before building anything. Arthur Samuel, who built a self-improving draughts program at IBM in the 1950s, is credited with popularising the term "machine learning" in 1959.

When machine learning is the right tool

  • The rules are too complex to write down, such as recognising faces or speech.
  • The patterns change over time, such as fraud tactics or shopping trends.
  • There is plenty of relevant, labelled or structured data.
  • Some mistakes are acceptable, or a human can check the important ones.

Machine learning is the wrong tool when a simple rule already works, when there is little or poor data, when every decision must be perfectly explainable by law, or when a single error could be catastrophic and cannot be checked. Many real systems combine both approaches: learned models make predictions, and hand-written rules add safety limits.

Training and inference

  1. Collect examples with inputs and the correct outputs.
  2. Training: the algorithm adjusts the model's parameters to reduce its errors on those examples.
  3. Evaluation: test the model on examples it never saw.
  4. Inference: use the trained model on new inputs, often millions of times, quickly and cheaply.

How it works

Programming vs machine learning

You give the computerYou get back
Traditional programmingRules + dataAnswers
Machine learningData + answersRules (a model)

Mitchell's definition, on a spam filter

  • Task T: decide whether an email is spam.
  • Experience E: thousands of emails that people have already marked.
  • Performance P: the share of new emails it gets right.

The steps of learning

  1. Collect examples, each with its features (what you know) and label (the right answer).
  2. Choose a model: a formula with adjustable numbers called parameters.
  3. Train: the model predicts, measures how wrong it was, and adjusts its parameters. Repeat many times.
  4. Test on examples it has never seen. That is the only fair measure.
  5. Use it on new data. This step is called inference.

The three main kinds

KindLearns fromExample
SupervisedExamples with right answersPredicting house prices
UnsupervisedExamples with no answersGrouping similar customers
ReinforcementRewards for its actionsLearning to play a game

The maths

What every learning algorithm does
The model's parameters, the numbers it can adjust
The best parameters found
The model's prediction for example i
The right answer for example i
The loss: how wrong one prediction is
The number of examples

In words: find the parameters that make the average error as small as possible. Suppose a model predicts 3 house prices (in lakhs) as 50, 60 and 80, and the right answers are 52, 60 and 75. Using squared error as the loss:

Average loss = [ (50−52)² + (60−60)² + (80−75)² ] / 3
             = [ 4 + 0 + 25 ] / 3
             = 9.67

Training adjusts the parameters so this number falls. That's all "learning" means here.

In code

A complete ML program with scikit-learn. The model is never told the rule behind the numbers. It works it out from the examples.

first_model.py
from sklearn.linear_model import LinearRegression

# Experience: house size in sq ft, and price in lakhs
sizes = [[500], [800], [1000], [1200], [1500]]
prices = [25, 40, 50, 60, 75]

model = LinearRegression()
model.fit(sizes, prices)          # learning happens here

# The true intercept is 0; adding 0.0 stops a tiny rounding error from printing as -0.0.
print("Learned rule: price =", round(model.coef_[0], 3), "x size +", round(model.intercept_, 3) + 0.0)
print("A 1100 sq ft house:", round(model.predict([[1100]])[0], 1), "lakhs")

Output

Learned rule: price = 0.05 x size + 0.0
A 1100 sq ft house: 55.0 lakhs

Common mistakes

  • Testing a model on the data it trained on. It may have just memorised the answers.
  • Thinking more data always fixes everything. Wrong or biased data teaches wrong patterns.
  • Using ML when a simple rule would do. If you can write the rule in five lines, write the rule.

Check yourself

Question 1

1. In traditional programming you supply rules and data. In ML you supply:

Question 2

2. For a spam filter, what is the experience E in Mitchell's definition?

Question 3

3. Grouping customers with no labels is which kind of ML?

Summary

  • ML learns patterns from data, instead of following hand-written rules.
  • Mitchell: performance at a task, measured by P, improves with experience E.
  • Training means adjusting parameters to reduce the average loss.
  • Three main kinds: supervised, unsupervised and reinforcement.

Read it. Now try it yourself.

Open the interactive activity

Finished reading?

Sources: Mitchell, Machine Learning (1997) · Google Machine Learning Crash Course