A1 · Supervised Machine Learning in Depth

XGBoost, LightGBM and CatBoost

XGBoost, LightGBM and CatBoost are optimised gradient-boosting libraries. XGBoost adds second-order (Newton) updates and explicit regularisation of tree complexity. LightGBM bins features into histograms and grows trees leaf-wise for speed on large data. CatBoost uses ordered boosting and built-in target statistics for categorical features. scikit-learn's HistGradientBoosting implements the histogram approach.

15 min Advanced Jump to the activity

What you need: The earlier topics of this module, and its prerequisites.

Definition

Why do these three libraries win so many data competitions?

XGBoost, LightGBM and CatBoost are optimised gradient-boosting libraries. XGBoost adds second-order (Newton) updates and explicit regularisation of tree complexity. LightGBM bins features into histograms and grows trees leaf-wise for speed on large data. CatBoost uses ordered boosting and built-in target statistics for categorical features. scikit-learn's HistGradientBoosting implements the histogram approach.

In simple words

They all do gradient boosting; they differ in engineering. Histograms turn a million unique values into 256 buckets, so finding splits is fast. Built-in handling of missing values and categories saves preprocessing. Early stopping on a validation set picks the number of trees automatically.

Why it matters

For structured business data, a tuned gradient-boosting library is usually the model to beat. Knowing their differences helps you choose quickly and tune the parameters that matter.

Key terms

XGBoost
A gradient-boosting library (2014–2016) with regularised trees, second-order gradients and parallel training.
LightGBM
Microsoft's library (2017) using histogram binning and leaf-wise growth for speed on large data.
CatBoost
Yandex's library (2017) with ordered target statistics for categorical features and symmetric trees.
Histogram binning
Grouping each feature's values into, say, 255 bins so split search is much faster.
Leaf-wise growth
Splitting the leaf with the largest gain anywhere in the tree, rather than level by level.
Monotonic constraint
Forcing a prediction to only increase (or decrease) with a feature, such as risk with age.

Explained in detail

What these libraries add

All three implement gradient-boosted decision trees but add engineering that made boosting fast and accurate enough to dominate tabular machine learning. XGBoost, created by Tianqi Chen and described in a 2016 paper, uses both the gradient and the second derivative of the loss to choose splits, adds L1 and L2 penalties on leaf values, handles missing values by learning a default direction at each split, and parallelises split finding. LightGBM speeds things up further with histogram binning, leaf-wise growth that reaches lower loss with fewer splits, and techniques that sample rows with small gradients and bundle sparse features. CatBoost focuses on categorical data: it encodes categories with target statistics computed in a random order so a row never uses its own label, which avoids a subtle leakage, and builds balanced symmetric trees that predict very quickly.

Choosing and tuning

LibraryStrong whenKey parameters
XGBoostGeneral tabular problems; mature ecosystem and GPU supporteta, max_depth, subsample, colsample_bytree, lambda, n_estimators
LightGBMVery large data sets; fastest traininglearning_rate, num_leaves, min_data_in_leaf, feature_fraction
CatBoostMany categorical features; good results with default settingslearning_rate, depth, l2_leaf_reg, iterations

A practical tuning recipe: set a moderate learning rate such as 0.05, a large maximum number of trees and early stopping on a validation set; tune tree size (max_depth around 4 to 8, or num_leaves around 31 to 255 in LightGBM) and minimum leaf size; then add row and column subsampling around 0.7 to 0.9 and some L2 regularisation; finally, lower the learning rate and retrain for a small extra gain. Watch for LightGBM's leaf-wise trees overfitting small data sets: limit num_leaves and increase min_data_in_leaf. All three support monotonic constraints, useful in credit scoring where regulators expect risk to rise with missed payments, and all integrate with SHAP for explaining predictions.

  • On typical tabular data with fewer than a few million rows, these libraries usually outperform neural networks while training far faster.
  • Differences between the three are often small once tuned; data quality and features matter more than the library.

Typical scenarios include credit risk, fraud detection, insurance pricing, click-through prediction for advertising, demand forecasting with lag features and ranking search results, where LightGBM and XGBoost provide learning-to-rank objectives.

How it works

LibraryStrengthReach for it when
XGBoostMature, regularised, widely deployedA solid default with strong tooling
LightGBMVery fast on large data, leaf-wise growthMillions of rows, many features
CatBoostNative categorical handling, robust defaultsMany high-cardinality categories
HistGradientBoostingIn scikit-learn, histogram-basedYou want no extra dependency

The parameters that matter most: learning rate, number of trees (with early stopping), tree size (max_depth or num_leaves), minimum samples per leaf, and subsampling of rows and columns.

The maths

XGBoost's regularised objective at round m
First and second derivatives of the loss
Number of leaves
Leaf weights
Optimal leaf weight

In code

Histogram gradient boosting with early stopping and native missing-value handling. The same pattern works with xgboost.XGBClassifier or lightgbm.LGBMClassifier on your own machine.

hist_gradient_boosting.py
import numpy as np
from sklearn.datasets import make_classification
from sklearn.ensemble import GradientBoostingClassifier, HistGradientBoostingClassifier
from sklearn.model_selection import train_test_split

X, y = make_classification(n_samples=6000, n_features=30, n_informative=12, random_state=0)
rng = np.random.default_rng(0)
X[rng.random(X.shape) < 0.05] = np.nan            # 5 per cent missing values
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.25, random_state=0)

hgb = HistGradientBoostingClassifier(learning_rate=0.1, max_iter=500, early_stopping=True,
                                     validation_fraction=0.15, n_iter_no_change=20, random_state=0)
hgb.fit(X_tr, y_tr)                               # missing values are handled natively
print("Trees used after early stopping:", hgb.n_iter_)
print("Histogram boosting test accuracy:", round(hgb.score(X_te, y_te), 3))

X_filled = np.nan_to_num(X_tr, nan=0.0)          # classic GradientBoosting needs no NaNs
gb = GradientBoostingClassifier(n_estimators=hgb.n_iter_, random_state=0).fit(X_filled, y_tr)
print("Classic boosting test accuracy:", round(gb.score(np.nan_to_num(X_te, nan=0.0), y_te), 3))

Output

Trees used after early stopping: 189
Histogram boosting test accuracy: 0.916
Classic boosting test accuracy: 0.872

Try it yourself

Which library when?

Match each situation to the library you'd try first.

  • 50 million rows and speed matters most
  • Dozens of categorical columns with thousands of values each
  • No extra libraries allowed in production
  • A well-tested default with broad deployment support

Common mistakes

  • Fixing the number of trees by hand instead of early stopping.
  • One-hot encoding hundreds of categories for CatBoost, which handles them natively.
  • Tuning everything at once. Start with learning rate, tree size and early stopping.

Check yourself

Question 1

1. What makes LightGBM fast?

Question 2

2. What does XGBoost use beyond plain gradient boosting?

Question 3

3. How should the number of trees be chosen?

Summary

  • XGBoost, LightGBM and CatBoost are engineered gradient boosting.
  • Histograms, missing-value handling and early stopping make them fast and practical.
  • Tune learning rate, tree size and subsampling; let early stopping pick the trees.

Finished reading?

Sources: XGBoost documentation · LightGBM documentation · CatBoost documentation