I5 · Machine Learning Foundations

The scikit-learn Workflow

scikit-learn is Python's standard library for classical machine learning. Every model is an estimator with the same interface: fit(X, y) learns from training data, predict(X) makes predictions, and score(X, y) gives a default metric. Transformers such as scalers use fit and transform, and a Pipeline chains preprocessing and a model into one estimator.

12 min Intermediate · Sublevel Intermediate Jump to the activity

What you need: The earlier topics of this module.

Definition

Linear regression, decision trees, k-NN and naive Bayes are very different algorithms. In scikit-learn, you train every one of them with the same two words: fit and predict.

scikit-learn is Python's standard library for classical machine learning. Every model is an estimator with the same interface: fit(X, y) learns from training data, predict(X) makes predictions, and score(X, y) gives a default metric. Transformers such as scalers use fit and transform, and a Pipeline chains preprocessing and a model into one estimator.

In simple words

Think of a universal remote: every appliance responds to the same buttons. In scikit-learn, every model responds to fit and predict, so swapping one algorithm for another means changing one line.

Overview

  • scikit-learn is Python's main machine-learning library for classical algorithms, started in 2007.
  • Every model follows the same API: create it, fit(X, y), then predict(X) or transform(X).
  • Pipelines chain preprocessing and models so they are fitted together without leakage.
  • ColumnTransformer applies different preprocessing to numeric and categorical columns.
  • model_selection provides splitting, cross-validation and hyperparameter search.
  • Trained pipelines can be saved with joblib and loaded in an app.

Why it matters

A consistent workflow lets you compare many models quickly and correctly, avoid leakage by bundling preprocessing with the model, and save a finished pipeline for use in an app. It is the everyday toolkit of data scientists.

Key terms

Estimator
Any scikit-learn object that learns from data with fit().
Transformer
An estimator that changes data with transform(), such as StandardScaler.
Pipeline
A chain of transformers ending in a model, treated as one estimator.
ColumnTransformer
Applies different preprocessing to different columns.
random_state
A seed that makes random steps repeatable.
Model persistence
Saving a trained pipeline to a file, for example with joblib, and loading it later.

Types

scikit-learn objects fall into a few types that all share the same interface. Knowing them makes the whole library predictable.

1. Estimators (models)

Objects that learn from data with fit, such as LinearRegression, RandomForestClassifier and KMeans.

  • How it works: Hyperparameters are set when the object is created; learned attributes end with an underscore, such as coef_ and feature_importances_.
  • Example: model = LogisticRegression(C=1.0).fit(X_train, y_train).
  • When to use it: Use them for every learning task; swapping one estimator for another needs no other code changes.

2. Transformers (preprocessors)

Objects with fit and transform that change data, such as StandardScaler, OneHotEncoder, SimpleImputer and PCA.

  • How it works: fit learns settings (means, categories) from training data; transform applies them to any data.
  • Example: scaler.fit(X_train) then scaler.transform(X_test) uses the training mean and standard deviation.
  • When to use it: Use them for all preprocessing, always fitted on training data only.

3. Pipelines and column transformers

Containers that chain steps (Pipeline) or route columns to different steps (ColumnTransformer).

  • How it works: make_pipeline(preprocess, model) behaves like one estimator, so cross-validation and tuning refit everything correctly in each fold.
  • Example: A pipeline that imputes, scales numbers, one-hot encodes cities and then fits gradient boosting.
  • When to use it: Use them in every real project to prevent leakage and make deployment simple.

4. Model-selection tools

Functions and classes for splitting and searching: train_test_split, cross_val_score, KFold, GridSearchCV and RandomizedSearchCV.

  • How it works: They work with any estimator or pipeline and accept scoring names such as "f1" or "neg_mean_absolute_error".
  • Example: GridSearchCV(pipeline, param_grid, cv=5) tuning a forest's depth and leaf size.
  • When to use it: Use them to compare and tune models fairly.

5. Metrics and inspection tools

Functions that score and explain models: accuracy_score, classification_report, mean_absolute_error, roc_auc_score, permutation_importance and partial dependence.

  • How it works: Metrics compare predictions with true values; inspection tools show which features matter and how.
  • Example: permutation_importance revealing that distance and preparation time drive delivery predictions.
  • When to use it: Use them to report results and explain models to users.

Explained in detail

A short history

scikit-learn began in 2007 as a Google Summer of Code project by David Cournapeau and was developed into a full library from 2010 by researchers at INRIA in France, led by Gaël Varoquaux, Alexandre Gramfort, Vincent Michel and Fabian Pedregosa. Its design paper in 2011 emphasised a consistent interface, sensible defaults and excellent documentation. It is now one of the most cited machine-learning tools in science and is used throughout industry for tabular data.

The standard workflow

Stepscikit-learn tool
Load data into X and ypandas, or sklearn.datasets
Splittrain_test_split(..., stratify=y, random_state=0)
PreprocessStandardScaler, OneHotEncoder, SimpleImputer in a ColumnTransformer
Choose a modelLogisticRegression, RandomForestClassifier, KNeighborsClassifier…
BundlePipeline or make_pipeline
Evaluate and tunecross_val_score, GridSearchCV
Final checkscore on the test set, classification_report, confusion_matrix
Savejoblib.dump(pipeline, "model.joblib")

Why pipelines matter

Without a pipeline it is easy to fit a scaler on all the data, leak information, and then forget to apply exactly the same scaling to new data in the app. A pipeline fixes both problems. During cross-validation, scikit-learn refits every step on each training fold automatically. After training, the saved pipeline contains the fitted scaler, encoder and model together, so pipeline.predict(new_rows) applies identical preprocessing every time. When data has numeric and categorical columns, a ColumnTransformer sends each group to the right preprocessing before the model.

Good habits

  • Set random_state everywhere randomness appears, so results can be reproduced.
  • Start with a DummyClassifier or DummyRegressor baseline.
  • Compare several models with the same cross-validation folds.
  • Read each estimator's documentation for its defaults; for example, LogisticRegression applies regularisation by default.
  • Only load saved models from trusted sources: loading a pickled file can run arbitrary code.

scikit-learn deliberately focuses on classical machine learning for tabular data, the rows-and-columns data found in spreadsheets and databases, which is still the most common kind of data in business. It does not try to be a deep-learning library; for images, audio and long text you will use PyTorch or TensorFlow in later modules. But even there, the ideas you practise here carry over directly: separate the data honestly, bundle preprocessing with the model, compare against baselines with the same splits and save exactly what you evaluated. Many deep-learning tools even copy scikit-learn's fit and predict naming for that reason.

How it works

  1. Split the data and set aside the test set.
  2. Build a ColumnTransformer for numeric and categorical columns.
  3. Put it in a Pipeline with a model.
  4. Cross-validate several models and tune the best.
  5. Score once on the test set and save the pipeline.

In code

sklearn_workflow.py
import numpy as np
import pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.dummy import DummyClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler

rng = np.random.default_rng(1)
n = 400
df = pd.DataFrame({
    "monthly_bill": rng.normal(600, 200, n).round(),
    "complaints": rng.poisson(1, n),
    "plan": rng.choice(["basic", "standard", "premium"], n),
})
risk = 0.004 * (df["monthly_bill"] - 600) + 0.8 * df["complaints"] - 1 + (df["plan"] == "basic") * 0.7
df["left"] = (rng.random(n) < 1 / (1 + np.exp(-risk))).astype(int)

X, y = df.drop(columns="left"), df["left"]
prep = ColumnTransformer([
    ("num", StandardScaler(), ["monthly_bill", "complaints"]),
    ("cat", OneHotEncoder(handle_unknown="ignore"), ["plan"]),
])
models = {
    "baseline": DummyClassifier(strategy="most_frequent"),
    "logistic": LogisticRegression(),
    "forest": RandomForestClassifier(n_estimators=200, random_state=0),
}
for name, model in models.items():
    pipe = Pipeline([("prep", prep), ("model", model)])
    scores = cross_val_score(pipe, X, y, cv=5)
    print(f"{name:>8}: {scores.mean():.3f} +/- {scores.std():.3f}")

Output

baseline: 0.497 +/- 0.005
logistic: 0.745 +/- 0.055
  forest: 0.655 +/- 0.054

Try it yourself

Swapping models in one line

A data analyst at a telecom company wants to predict which customers will leave.

The scikit-learn workflow

Put the steps in order.

  1. 1Save the fitted pipeline
  2. 2Score once on the test set
  3. 3Build a Pipeline of preprocessing and model
  4. 4Split into training and test sets
  5. 5Cross-validate and tune on the training set

Common mistakes

  • Preprocessing outside the pipeline and leaking test information.
  • Comparing models on different random splits.
  • Loading saved models from untrusted files.

Check yourself

Question 1

1. Which method trains any scikit-learn model?

Question 2

2. What is the main benefit of a Pipeline?

Question 3

3. Why set random_state?

Summary

  • Every scikit-learn model shares fit, predict and score.
  • Pipelines and ColumnTransformers bundle preprocessing with the model and prevent leakage.
  • Compare models fairly with cross-validation, test once and save the pipeline.

Read it. Now try it yourself.

Open the interactive activity

Finished reading?

Sources: scikit-learn: Pipelines and composite estimators · scikit-learn User Guide