Definition
Linear regression, decision trees, k-NN and naive Bayes are very different algorithms. In scikit-learn, you train every one of them with the same two words: fit and predict.
scikit-learn is Python's standard library for classical machine learning. Every model is an estimator with the same interface: fit(X, y) learns from training data, predict(X) makes predictions, and score(X, y) gives a default metric. Transformers such as scalers use fit and transform, and a Pipeline chains preprocessing and a model into one estimator.
In simple words
Think of a universal remote: every appliance responds to the same buttons. In scikit-learn, every model responds to fit and predict, so swapping one algorithm for another means changing one line.
Overview
- scikit-learn is Python's main machine-learning library for classical algorithms, started in 2007.
- Every model follows the same API: create it, fit(X, y), then predict(X) or transform(X).
- Pipelines chain preprocessing and models so they are fitted together without leakage.
- ColumnTransformer applies different preprocessing to numeric and categorical columns.
- model_selection provides splitting, cross-validation and hyperparameter search.
- Trained pipelines can be saved with joblib and loaded in an app.
Why it matters
A consistent workflow lets you compare many models quickly and correctly, avoid leakage by bundling preprocessing with the model, and save a finished pipeline for use in an app. It is the everyday toolkit of data scientists.
Key terms
- Estimator
- Any scikit-learn object that learns from data with fit().
- Transformer
- An estimator that changes data with transform(), such as StandardScaler.
- Pipeline
- A chain of transformers ending in a model, treated as one estimator.
- ColumnTransformer
- Applies different preprocessing to different columns.
- random_state
- A seed that makes random steps repeatable.
- Model persistence
- Saving a trained pipeline to a file, for example with joblib, and loading it later.
Types
scikit-learn objects fall into a few types that all share the same interface. Knowing them makes the whole library predictable.
1. Estimators (models)
Objects that learn from data with fit, such as LinearRegression, RandomForestClassifier and KMeans.
- How it works: Hyperparameters are set when the object is created; learned attributes end with an underscore, such as coef_ and feature_importances_.
- Example: model = LogisticRegression(C=1.0).fit(X_train, y_train).
- When to use it: Use them for every learning task; swapping one estimator for another needs no other code changes.
2. Transformers (preprocessors)
Objects with fit and transform that change data, such as StandardScaler, OneHotEncoder, SimpleImputer and PCA.
- How it works: fit learns settings (means, categories) from training data; transform applies them to any data.
- Example: scaler.fit(X_train) then scaler.transform(X_test) uses the training mean and standard deviation.
- When to use it: Use them for all preprocessing, always fitted on training data only.
3. Pipelines and column transformers
Containers that chain steps (Pipeline) or route columns to different steps (ColumnTransformer).
- How it works: make_pipeline(preprocess, model) behaves like one estimator, so cross-validation and tuning refit everything correctly in each fold.
- Example: A pipeline that imputes, scales numbers, one-hot encodes cities and then fits gradient boosting.
- When to use it: Use them in every real project to prevent leakage and make deployment simple.
4. Model-selection tools
Functions and classes for splitting and searching: train_test_split, cross_val_score, KFold, GridSearchCV and RandomizedSearchCV.
- How it works: They work with any estimator or pipeline and accept scoring names such as "f1" or "neg_mean_absolute_error".
- Example: GridSearchCV(pipeline, param_grid, cv=5) tuning a forest's depth and leaf size.
- When to use it: Use them to compare and tune models fairly.
5. Metrics and inspection tools
Functions that score and explain models: accuracy_score, classification_report, mean_absolute_error, roc_auc_score, permutation_importance and partial dependence.
- How it works: Metrics compare predictions with true values; inspection tools show which features matter and how.
- Example: permutation_importance revealing that distance and preparation time drive delivery predictions.
- When to use it: Use them to report results and explain models to users.
Explained in detail
A short history
scikit-learn began in 2007 as a Google Summer of Code project by David Cournapeau and was developed into a full library from 2010 by researchers at INRIA in France, led by Gaël Varoquaux, Alexandre Gramfort, Vincent Michel and Fabian Pedregosa. Its design paper in 2011 emphasised a consistent interface, sensible defaults and excellent documentation. It is now one of the most cited machine-learning tools in science and is used throughout industry for tabular data.
The standard workflow
| Step | scikit-learn tool |
|---|---|
| Load data into X and y | pandas, or sklearn.datasets |
| Split | train_test_split(..., stratify=y, random_state=0) |
| Preprocess | StandardScaler, OneHotEncoder, SimpleImputer in a ColumnTransformer |
| Choose a model | LogisticRegression, RandomForestClassifier, KNeighborsClassifier… |
| Bundle | Pipeline or make_pipeline |
| Evaluate and tune | cross_val_score, GridSearchCV |
| Final check | score on the test set, classification_report, confusion_matrix |
| Save | joblib.dump(pipeline, "model.joblib") |
Why pipelines matter
Without a pipeline it is easy to fit a scaler on all the data, leak information, and then forget to apply exactly the same scaling to new data in the app. A pipeline fixes both problems. During cross-validation, scikit-learn refits every step on each training fold automatically. After training, the saved pipeline contains the fitted scaler, encoder and model together, so pipeline.predict(new_rows) applies identical preprocessing every time. When data has numeric and categorical columns, a ColumnTransformer sends each group to the right preprocessing before the model.
Good habits
- Set random_state everywhere randomness appears, so results can be reproduced.
- Start with a DummyClassifier or DummyRegressor baseline.
- Compare several models with the same cross-validation folds.
- Read each estimator's documentation for its defaults; for example, LogisticRegression applies regularisation by default.
- Only load saved models from trusted sources: loading a pickled file can run arbitrary code.
scikit-learn deliberately focuses on classical machine learning for tabular data, the rows-and-columns data found in spreadsheets and databases, which is still the most common kind of data in business. It does not try to be a deep-learning library; for images, audio and long text you will use PyTorch or TensorFlow in later modules. But even there, the ideas you practise here carry over directly: separate the data honestly, bundle preprocessing with the model, compare against baselines with the same splits and save exactly what you evaluated. Many deep-learning tools even copy scikit-learn's fit and predict naming for that reason.
How it works
- Split the data and set aside the test set.
- Build a ColumnTransformer for numeric and categorical columns.
- Put it in a Pipeline with a model.
- Cross-validate several models and tune the best.
- Score once on the test set and save the pipeline.
In code
import numpy as np
import pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.dummy import DummyClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler
rng = np.random.default_rng(1)
n = 400
df = pd.DataFrame({
"monthly_bill": rng.normal(600, 200, n).round(),
"complaints": rng.poisson(1, n),
"plan": rng.choice(["basic", "standard", "premium"], n),
})
risk = 0.004 * (df["monthly_bill"] - 600) + 0.8 * df["complaints"] - 1 + (df["plan"] == "basic") * 0.7
df["left"] = (rng.random(n) < 1 / (1 + np.exp(-risk))).astype(int)
X, y = df.drop(columns="left"), df["left"]
prep = ColumnTransformer([
("num", StandardScaler(), ["monthly_bill", "complaints"]),
("cat", OneHotEncoder(handle_unknown="ignore"), ["plan"]),
])
models = {
"baseline": DummyClassifier(strategy="most_frequent"),
"logistic": LogisticRegression(),
"forest": RandomForestClassifier(n_estimators=200, random_state=0),
}
for name, model in models.items():
pipe = Pipeline([("prep", prep), ("model", model)])
scores = cross_val_score(pipe, X, y, cv=5)
print(f"{name:>8}: {scores.mean():.3f} +/- {scores.std():.3f}")Output
baseline: 0.497 +/- 0.005 logistic: 0.745 +/- 0.055 forest: 0.655 +/- 0.054
Try it yourself
Swapping models in one line
A data analyst at a telecom company wants to predict which customers will leave.
The scikit-learn workflow
Put the steps in order.
- 1Save the fitted pipeline
- 2Score once on the test set
- 3Build a Pipeline of preprocessing and model
- 4Split into training and test sets
- 5Cross-validate and tune on the training set
Common mistakes
- Preprocessing outside the pipeline and leaking test information.
- Comparing models on different random splits.
- Loading saved models from untrusted files.
Check yourself
Summary
- Every scikit-learn model shares fit, predict and score.
- Pipelines and ColumnTransformers bundle preprocessing with the model and prevent leakage.
- Compare models fairly with cross-validation, test once and save the pipeline.
Read it. Now try it yourself.
Open the interactive activityFinished reading?
Sources: scikit-learn: Pipelines and composite estimators · scikit-learn User Guide