A1 · Supervised Machine Learning in Depth

Machine Learning Pipelines

A machine learning pipeline chains preprocessing steps and a final estimator into a single object with fit and predict methods. Fitting the pipeline fits each step in order on the training data; predicting applies the already-fitted steps to new data. Pipelines can be cross-validated, tuned and saved as one unit.

12 min Advanced Jump to the activity

What you need: The earlier topics of this module, and its prerequisites.

Definition

How do you make preprocessing and modelling one object you can fit, tune and deploy?

A machine learning pipeline chains preprocessing steps and a final estimator into a single object with fit and predict methods. Fitting the pipeline fits each step in order on the training data; predicting applies the already-fitted steps to new data. Pipelines can be cross-validated, tuned and saved as one unit.

In simple words

Without a pipeline you must remember to scale the training data, then scale the test data with the same numbers, then call the model, every time. A pipeline does all of that for you, in the right order, and makes it impossible to accidentally fit the scaler on test data.

Why it matters

Pipelines prevent leakage, make experiments reproducible, let grid search tune preprocessing and model together, and turn the deployed model into a single file that cannot drift out of step with its preprocessing.

Key terms

Pipeline
A chain of preprocessing steps and a final model treated as one estimator.
Transformer
In scikit-learn, an object with fit and transform, such as StandardScaler.
Estimator
An object with fit and predict, such as LogisticRegression.
ColumnTransformer
Applies different preprocessing to different columns and joins the results.
Parameter grid
Hyperparameter values to search, addressed as step__parameter in a pipeline.
Serialisation
Saving a fitted pipeline to a file, for example with joblib, so it can be loaded for prediction.

Explained in detail

Why pipelines matter

A pipeline bundles every step from raw columns to prediction into one object. This has three big benefits. First, it prevents leakage: when you pass a pipeline to cross_val_score or GridSearchCV, every step, including imputation and scaling, is refitted on each training fold, so validation folds never influence preprocessing. Second, it guarantees consistency: the exact same transformations are applied in training and in production, avoiding the classic bug where a deployed model receives unscaled inputs. Third, it makes code shorter and easier to tune, because hyperparameters of any step can be searched together, such as the imputation strategy and the regularisation strength.

Building blocks in scikit-learn

ToolPurpose
Pipeline / make_pipelineChain steps in order; the last step is the model
ColumnTransformerRoute numeric and categorical columns to different preprocessing
FunctionTransformerWrap a simple function, such as np.log1p, as a step
GridSearchCV / RandomizedSearchCVTune step parameters such as model__C or prep__num__imputer__strategy
joblib.dump / joblib.loadSave and reload the whole fitted pipeline

In production, the saved pipeline accepts raw rows in the same format as training data and returns predictions, so the serving code stays tiny. Record the library versions used to train it, because a pipeline saved with one version of scikit-learn may not load correctly in another. Custom feature engineering can be added as your own transformer class with fit and transform methods, which keeps even complex logic inside the leak-proof structure. Inspect a pipeline with get_feature_names_out to see exactly which columns the model receives, and test it with a few hand-made rows, including unusual ones such as unseen categories and missing values, before deployment.

A typical tabular pipeline looks like this: a ColumnTransformer sends numeric columns through a median imputer and a StandardScaler, sends categorical columns through a most-frequent imputer and a OneHotEncoder with handle_unknown set to ignore, and passes the combined matrix to a gradient-boosting or logistic-regression model. The whole object is tuned with GridSearchCV over parameters such as model__learning_rate and prep__num__imputer__strategy, refitted on all training data and saved as a single file. In production a FastAPI endpoint loads that file once and calls predict on each incoming request, so no preprocessing code is duplicated. Because training and serving share the same object, a whole class of bugs, where the live system prepares data slightly differently from the notebook, disappears.

How it works

  • Pipeline([("scale", StandardScaler()), ("model", SVC())]) names each step.
  • Tune parameters of any step with the name__parameter syntax, such as model__C.
  • Combine with ColumnTransformer for mixed data, and save with joblib.

In code

A pipeline cross-validated as one unit, then tuned through its step names.

pipeline.py
from sklearn.datasets import load_wine
from sklearn.model_selection import GridSearchCV, cross_val_score
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC

X, y = load_wine(return_X_y=True)
pipe = Pipeline([("scale", StandardScaler()), ("model", SVC())])

print("Without scaling:", round(cross_val_score(SVC(), X, y, cv=5).mean(), 3))
print("Pipeline with scaling:", round(cross_val_score(pipe, X, y, cv=5).mean(), 3))

search = GridSearchCV(pipe, {"model__C": [0.1, 1, 10], "model__gamma": ["scale", 0.01]}, cv=5)
search.fit(X, y)
print("Best parameters:", search.best_params_)
print("Best cross-validated accuracy:", round(search.best_score_, 3))
print("Steps:", list(search.best_estimator_.named_steps))

Output

Without scaling: 0.663
Pipeline with scaling: 0.983
Best parameters: {'model__C': 10, 'model__gamma': 'scale'}
Best cross-validated accuracy: 0.989
Steps: ['scale', 'model']

Try it yourself

What does pipe.fit(X_train) do?

Put the internal steps of fitting a two-step pipeline in order.

  1. 1The fitted model predicts on the scaled test data
  2. 2The scaler learns the mean and standard deviation of X_train
  3. 3The scaler transforms X_train
  4. 4Later, pipe.predict(X_test) scales X_test with the stored numbers
  5. 5The model is fitted on the scaled training data

Common mistakes

  • Scaling outside the pipeline and then cross-validating the pipeline.
  • Forgetting the step__parameter naming when tuning.
  • Saving only the model, not the pipeline, so production forgets the preprocessing.

Check yourself

Question 1

1. How do you tune C of a step named 'model'?

Question 2

2. What does pipe.predict do with the scaler?

Question 3

3. Why deploy the whole pipeline?

Summary

  • A pipeline chains preprocessing and a model into one estimator.
  • It prevents leakage in cross-validation and tuning.
  • Tune with step__parameter, and deploy the whole pipeline.

Read it. Now try it yourself.

Open the interactive activity

Finished reading?

Sources: scikit-learn: Pipelines and composite estimators