I5 · Machine Learning Foundations

Machine Learning Terminology

These are the words used throughout ML. A sample is one row of data. A feature is one input column, and the label is the output to predict. A hypothesis is the function a model has learned. Loss measures error on one example, while cost averages it across the data. A batch is a group of samples, and an epoch is one full pass over the training data.

8 min Intermediate · Sublevel Intermediate

What you need: The earlier topics of this module.

Definition

These are the words used throughout ML. A sample is one row of data. A feature is one input column, and the label is the output to predict. A hypothesis is the function a model has learned. Loss measures error on one example, while cost averages it across the data. A batch is a group of samples, and an epoch is one full pass over the training data.

In simple words

You have 1,000 flats in a spreadsheet. Each flat is a sample, area and floor are features, and price is the label. If training looks at 100 flats at a time, that's a batch of 100. It takes 10 batches to see all flats once, and that one full round is an epoch.

Overview

  • A data set is a collection of examples, also called samples, instances or rows.
  • Features (inputs, X) describe each example; the label or target (y) is what we want to predict.
  • A model is a function with parameters learned from data; hyperparameters are settings chosen before training.
  • Training fits the model; inference (prediction) applies it to new examples.
  • A loss function measures error during training; metrics measure performance for people.
  • Generalisation is how well a model performs on data it has never seen.

Why it matters

Every library, paper and interview uses these words. Mixing them up leads to wrong settings, such as confusing batch size with epochs.

Key terms

Sample
One example or row of data, also called an instance or observation.
Feature
One input variable, a column such as area or age.
Label or target
The output the model should predict.
Hyperparameter
A setting chosen before training, such as tree depth or learning rate.
Loss and cost
Loss measures error on one example; cost or objective averages it over the data.
Feature matrix X and target vector y
The standard way of storing inputs (rows × features) and outputs.

Types

ML vocabulary falls into a few groups. These types of term cover most of what you will read.

1. Data terms

Words for the parts of a data set: example, feature, label, feature vector and data split.

  • How it works: Each row is an example; each column a feature; the feature vector is one row's inputs; labels exist only in supervised learning.
  • Example: In a house-price data set, area, rooms and location are features and price is the label.
  • When to use it: Use these words to describe any data set before choosing a model.

2. Model terms

Words for what is learned: model, parameter (weight, bias), hyperparameter and architecture.

  • How it works: Parameters are fitted by training; hyperparameters such as k in k-NN or the learning rate are set by the practitioner and tuned on validation data.
  • Example: In y = 2.5x + 10, the slope 2.5 and intercept 10 are parameters.
  • When to use it: Use them to say clearly what a model learned and what you chose.

3. Training terms

Words for the learning process: loss, cost, gradient, epoch, batch, iteration and convergence.

  • How it works: An epoch is one pass over the training data, split into batches; each iteration updates parameters to reduce the loss until it converges.
  • Example: Training for 20 epochs with batches of 32 on 3,200 examples means 100 iterations per epoch.
  • When to use it: Use them to read training logs and describe how a model was trained.

4. Evaluation terms

Words for judging models: training, validation and test sets, metric, baseline, overfitting and underfitting.

  • How it works: A model is compared with a simple baseline using a metric on held-out data; overfitting means good training but poor test results.
  • Example: A model with 99% training accuracy but 70% test accuracy is overfitting.
  • When to use it: Use them to report results honestly and compare models fairly.

5. Deployment terms

Words for using models in the real world: inference, latency, drift, monitoring and retraining.

  • How it works: After deployment, data drift (changes in inputs) and concept drift (changes in the input-output relationship) can reduce accuracy, so models are monitored and retrained.
  • Example: A fraud model trained before the pandemic may drift as shopping moves online.
  • When to use it: Use them when planning how a model will be served, watched and updated.

Explained in detail

The shape of a data set

Almost every supervised learning problem is stored the same way. A feature matrix X has one row per sample and one column per feature, and a target vector y holds the label for each row. For a house-price data set of 500 houses with 4 features, X has shape (500, 4) and y has shape (500,). Libraries such as scikit-learn expect exactly this layout: model.fit(X, y) learns from it, and model.predict(X_new) returns one prediction per row of X_new. Learning to picture your data in this shape makes the rest of the vocabulary fall into place.

Parameters versus hyperparameters

ParametersHyperparameters
Who sets themThe training algorithm, from dataYou, before training
ExamplesWeights in linear regression; split thresholds in a tree; a network's weightsLearning rate; maximum tree depth; number of neighbours k; number of layers
How you chooseTraining finds themCompare options on validation data

More words you will meet

  • Hypothesis: the particular function a model has learned; the hypothesis space is all functions it could learn.
  • Classification predicts a category, such as spam or not; regression predicts a number, such as price.
  • Ground truth: the correct labels, often created by people; errors here limit any model.
  • Baseline: a very simple method, such as always predicting the most common class, that any real model must beat.
  • Epoch: one full pass of the training data, used in iterative training.
  • Feature engineering: creating more useful features from raw data, such as age from date of birth.

Loss, cost and metrics

Loss and metrics both measure how wrong a model is, but they serve different purposes. The loss is what training minimises; it must be smooth enough for gradient descent, like squared error or log loss. A metric is what humans use to judge the model; it should match the real goal, like accuracy, recall or average error in rupees. Sometimes they are the same; often they differ. A fraud model may be trained on log loss but judged on how many frauds it catches for each false alarm. Keeping the two ideas separate avoids a lot of confusion when reading results.

How it works

TermAlso calledSymbol
SampleExample, row, instancex⁽ⁱ⁾
FeatureInput, attribute, column, predictorxⱼ
LabelTarget, output, yy
PredictionOutput of the modelŷ
WeightParameter, coefficientw or θ
Learning rateStep sizeη or α

Generalisation

The real goal of ML is generalisation: doing well on data the model has never seen. Doing well on training data alone means little, because a model could just memorise it.

The maths

Number of updates in one epoch
Loss and cost
Loss on one example
Cost: the average over the data
1,000 samples, batch size 32:
Updates per epoch = ⌈1000 / 32⌉ = 32
Training for 10 epochs = 320 updates

Common mistakes

  • Mixing up batch and epoch. A batch is a group, and an epoch is a full pass.
  • Thinking lower training loss always means a better model. Check validation data.

Check yourself

Question 1

1. 2,000 samples and batch size 100. How many updates per epoch?

Question 2

2. What's the true goal of ML?

Summary

  • Sample: row. Feature: input. Label: answer.
  • Batch: a group. Epoch: one full pass.
  • Generalisation is the real goal.

Finished reading?

Sources: Google Machine Learning Glossary