Definition
Before training anything clever, ask: how well would a lazy guess do? For air pollution, "tomorrow will be like today" is surprisingly hard to beat. If your fancy model cannot beat it, it is not useful yet.
A baseline is the simplest reasonable method for a task, used as a reference point. Common baselines are predicting the average of the training data, predicting the most common class, a persistence forecast that repeats the latest value, or a simple rule from domain knowledge. A first machine-learning model, such as linear or logistic regression on a few features, is then compared against it on the same properly split data. Every later improvement is judged by how much it beats the baseline, not by its score in isolation.
In simple words
Before claiming your new shoes make you run faster, time yourself in your old shoes. The baseline is the time in the old shoes; only the difference tells you whether the new shoes help.
Overview
- A baseline is the simplest reasonable model or rule that your project must beat.
- Without a baseline you cannot tell whether a complex model is actually helping.
- Baselines are quick to build, easy to explain and reveal data problems early.
- Always split data into training and test sets before building any model.
- Use the same metric and test set for the baseline and later models.
- A surprisingly strong baseline sometimes turns out to be the best solution.
Why it matters
Without a baseline, a score such as "mean error of 20" means nothing. Baselines catch bugs, show whether machine learning is worth the effort and set realistic expectations for stakeholders.
Key terms
- Baseline
- The simplest sensible method, used to judge whether a model adds value.
- Persistence forecast
- Predicting that the next value equals the latest one.
- Time-based split
- Training on earlier dates and testing on later ones, so the model never sees the future.
- Mean absolute error (MAE)
- The average size of prediction errors, in the target's units.
- Skill score
- How much a model reduces error relative to the baseline, such as 1 − MAE_model ÷ MAE_baseline.
- Reproducibility
- Fixed random seeds, saved data splits and code so results can be repeated.
Types
There are several types of baseline. Projects usually try two or three, from simplest upward.
1. Majority-class and mean baselines
Always predicting the most common label, or the average value.
- How it works: scikit-learn's DummyClassifier and DummyRegressor compute these automatically.
- Example: If 85% of emails are not spam, predicting "not spam" always gives 85% accuracy.
- When to use it: Use it first to learn the minimum score any useful model must exceed. A model that cannot beat it has learned nothing useful.
2. Rule-based (heuristic) baselines
Simple hand-written rules based on common sense or domain knowledge.
- How it works: Write if-else rules or simple formulas using one or two key features.
- Example: Predicting next week's sales as the average of the last four weeks.
- When to use it: Use it when experts already have a rule of thumb your model should beat.
3. Simple linear models
Linear or logistic regression on the main features.
- How it works: Fit with default settings after basic scaling and encoding; inspect coefficients to understand the data.
- Example: A logistic regression predicting pass or fail from attendance and homework scores with 78% accuracy.
- When to use it: Use it as the standard ML baseline; it is fast, stable and interpretable.
4. Simple tree baselines
A shallow decision tree or a default random forest.
- How it works: Train with default or limited depth to capture non-linear patterns without tuning.
- Example: A depth-3 tree revealing that crop yield depends mostly on rainfall above a threshold.
- When to use it: Use it to check whether non-linear models are likely to help before investing in tuning.
5. Pre-trained and zero-shot baselines
Using an existing model or LLM without any training on your data.
- How it works: Apply a pre-trained classifier or prompt an LLM with clear instructions, then score it on your test set.
- Example: Asking an LLM to label customer reviews as positive or negative and getting 88% accuracy.
- When to use it: Use it for text, image and audio projects, where pre-trained models are often strong.
Explained in detail
Splitting time-series data correctly
With data that runs through time, a random split lets the model peek at the future: if 15 January is in the test set but 14 and 16 January are in training, the model effectively sees the answer. Instead, split by date: for example, train on the first two years and test on the most recent year, which mimics real use, where the model only knows the past. If you tune settings, hold out a validation period between training and test, or use rolling-origin evaluation that repeatedly trains on all data up to a date and tests on the following weeks. Keep the test period untouched until the end, and record the exact dates of each split.
Baselines for our project
| Baseline | Prediction for tomorrow | What it tells us |
|---|---|---|
| Training mean | The average PM2.5 over the training years | Any useful model must beat this easily |
| Seasonal mean | The average for this month in past years | Captures the winter peak |
| Persistence | Today's PM2.5 | Usually the hardest simple baseline for pollution |
| Linear regression | Weighted sum of today's PM2.5, the 3-day average, wind and humidity | Our first real model |
Evaluate all of them on the same test period with the same metrics. For regression, report mean absolute error, which is easy to explain ("on average we are 18 µg/m³ off"), and also root mean squared error, which punishes big misses. Because the decision is an alert, also convert predictions to "Poor or worse" yes or no and report recall and precision for those days. If linear regression barely beats persistence, that is a valuable finding: it tells you that most of the predictable signal is simply persistence, and the next page explores what extra information might help. Save the baseline results in a table that every later experiment adds a row to.
Baselines also protect you from fooling yourself. In our example, persistence already catches 96 per cent of Poor-or-worse days, because winter pollution stays high for weeks. That means the real value of a model lies in the difficult days, when the level changes sharply, and it tells us to judge improvements on those days, not only on the yearly average. Writing this insight down early keeps the project honest and focused on what users need.
How it works
- Split the data by date into training, validation and test periods.
- Compute simple baselines: mean, seasonal mean and persistence.
- Train a simple model such as linear regression on a few features.
- Evaluate all on the test period with MAE, RMSE and alert recall and precision.
- Record results in an experiment table with seeds, dates and settings.
In code
import numpy as np
import pandas as pd
from sklearn.linear_model import LinearRegression
def make_city_data(days=3 * 365, seed=3):
rng = np.random.default_rng(seed)
d = pd.date_range("2022-01-01", periods=days, freq="D")
winter = np.cos((d.dayofyear - 15) / 365 * 2 * np.pi) # +1 in mid-January, -1 in mid-July
wind = np.clip(rng.normal(3 - 1.0 * winter, 1.2), 0.3, None) # calmer air in winter
humidity = np.clip(rng.normal(60 + 15 * winter, 10), 20, 100)
festival = ((d.month == 11) & (d.day <= 3)).astype(int)
pm = np.zeros(days); pm[0] = 120
for t in range(1, days):
level = 85 + 60 * winter[t] - 14 * (wind[t] - 3) + 0.3 * (humidity[t] - 60) + 150 * festival[t]
pm[t] = 0.45 * pm[t - 1] + 0.55 * level + rng.normal(0, 10)
return pd.DataFrame({"pm25": np.clip(pm, 10, None), "wind": wind, "humidity": humidity, "festival": festival}, index=d)
df = make_city_data()
df["target"] = df["pm25"].shift(-1) # tomorrow's value
df["pm25_3day"] = df["pm25"].rolling(3).mean()
df["season"] = np.cos((df.index.dayofyear - 15) / 365 * 2 * np.pi)
df = df.dropna()
train, test = df[df.index < "2024-01-01"], df[df.index >= "2024-01-01"] # split by date
def scores(name, pred):
mae = np.mean(np.abs(pred - test["target"]))
bad, alert = test["target"] > 90, pred > 90 # "Poor or worse"
recall = (alert & bad).sum() / bad.sum()
false_alarms = int((alert & ~bad).sum())
print(f"{name:<18} MAE={mae:5.1f} alert recall={recall:.2f} false alarms={false_alarms}")
return mae
print("Poor-or-worse days in the 2024 test year:", int((test["target"] > 90).sum()))
scores("training mean", np.full(len(test), train["target"].mean()))
base = scores("persistence", test["pm25"].values)
features = ["pm25", "pm25_3day", "wind", "humidity", "season"]
lin = LinearRegression().fit(train[features], train["target"])
mae = scores("linear regression", lin.predict(test[features]))
print(f"Skill vs persistence: {1 - mae / base:.0%} lower error")Output
Poor-or-worse days in the 2024 test year: 170 training mean MAE= 51.2 alert recall=0.00 false alarms=0 persistence MAE= 11.7 alert recall=0.96 false alarms=6 linear regression MAE= 10.7 alert recall=0.96 false alarms=6 Skill vs persistence: 8% lower error
Try it yourself
Beating "tomorrow equals today"
The team runs its first comparison on the 2024 test year.
Correct split or leakage?
Judge each way of splitting the time-series data.
Shuffle all days and split 80/20 at random
Use today's measured wind and tomorrow's wind forecast
Use tomorrow's measured wind as a feature
Train on 2022–2023, test on 2024
Common mistakes
- Reporting a model's score without any baseline.
- Splitting time-series data at random.
- Testing many ideas on the test set until one looks good.
Check yourself
Summary
- A baseline is the simplest sensible method; every model must beat it on the same test data.
- For time series, split by date and keep the test period untouched.
- Report MAE, RMSE and alert recall and precision in an experiment table.
Finished reading?
Sources: Google: Rules of Machine Learning · scikit-learn: model evaluation