Definition
You have measured a 20-point gap in how often qualified women are approved. Now what? You can fix the data, change how the model learns, or adjust its decisions afterwards, and each choice has costs. Reducing bias is engineering plus judgement.
Bias mitigation methods fall into three stages. Pre-processing changes the training data: collecting more data from under-represented groups, re-weighting or re-sampling examples, fixing biased labels or removing proxy features. In-processing changes how the model is trained, for example by adding a fairness constraint or penalty to the objective. Post-processing changes decisions after the model scores, for example by choosing group-specific thresholds to equalise error rates. Beyond technical fixes, reducing bias needs diverse teams, involvement of affected communities, human review and the option not to deploy.
In simple words
If a recipe keeps coming out too salty, you can buy different ingredients (fix the data), change how you cook (fix the training) or add water at the end (fix the output). Sometimes you need all three, and sometimes you should not serve the dish at all.
Overview
- Bias can be reduced at three stages: before training (data), during training (model) and after training (outputs).
- No single method removes all bias; teams combine methods and measure the effect.
- Removing the sensitive attribute alone rarely works because other features act as proxies.
- Fixes often trade a little overall accuracy for fairer results.
- Process changes, diverse teams, community input and audits, matter as much as algorithms.
- Bias must be re-measured after deployment because data and populations change.
Why it matters
Measuring bias without acting on it changes nothing for the people harmed. Knowing the options and their trade-offs lets teams choose fixes that work, explain them and check that they did not create new problems.
Key terms
- Pre-processing
- Mitigating bias by changing the training data.
- Re-weighting
- Giving examples from under-represented groups more weight during training.
- In-processing
- Adding fairness constraints or penalties to the training objective.
- Post-processing
- Adjusting decisions after scoring, such as group-specific thresholds.
- Fairness-accuracy trade-off
- Improving a fairness metric may reduce overall accuracy slightly, or sometimes not at all.
- Human-in-the-loop
- People review uncertain or high-stakes decisions before they take effect.
Types
Bias reduction methods come in several types, grouped by the stage at which they act.
1. Better data collection
Gathering more representative and accurately labelled data.
- How it works: Collect from under-represented groups, review labelling guidelines and use diverse annotators.
- Example: Recording speech samples in 22 scheduled Indian languages for a voice assistant.
- When to use it: Use it first whenever possible; better data fixes problems at the root. Plan it with the communities affected, pay contributors fairly and record who is still missing.
2. Pre-processing: reweighting and resampling
Adjusting the training data so groups and labels are balanced.
- How it works: Give higher weights to under-represented group-label combinations, oversample them or remove biased features and proxies.
- Example: Weighting women applicants' successful examples more heavily in a hiring dataset.
- When to use it: Use it when you can change the training data but not the learning algorithm.
3. In-processing: fairness constraints
Building fairness into the training objective.
- How it works: Add a penalty for unfairness or a constraint such as equal opportunity; methods include Fairlearn's ExponentiatedGradient and adversarial debiasing.
- Example: Training a credit model with a constraint that true positive rates differ by under 5 points between groups.
- When to use it: Use it when you control training and need a specific fairness guarantee.
4. Post-processing: threshold adjustment
Changing how predictions are turned into decisions after the model is trained.
- How it works: Choose decision thresholds per group, or calibrate scores, so that a chosen fairness metric is met.
- Example: Using a slightly lower threshold for a group whose qualified members were being missed.
- When to use it: Use it when the model cannot be retrained; check that group-specific rules are legal in your setting.
5. Process and governance measures
Organisational practices that catch and prevent bias.
- How it works: Diverse teams, impact assessments, external audits, user feedback channels and regular re-testing after launch.
- Example: A city council publishing an annual audit of its benefits-eligibility algorithm.
- When to use it: Use them for every system that affects people, alongside technical fixes. Give someone clear ownership of acting on what audits find.
Explained in detail
Three points of intervention
| Stage | Methods | Pros and cons |
|---|---|---|
| Data (pre-processing) | Collect more data, re-weight, re-sample, relabel, remove proxies | Fixes root causes; collecting good data takes time |
| Training (in-processing) | Fairness constraints, adversarial debiasing, fairness-aware loss | Targets a chosen metric directly; more complex to build |
| Decisions (post-processing) | Group-specific thresholds, reject-option review | Simple and transparent; may need group data at decision time, which can raise legal questions |
The best fixes usually start with the data. If a speech model fails rural women, collecting recordings from rural women, with consent and payment, helps far more than any algorithmic trick. If labels reflect past discrimination, such as historical hiring decisions, consider a better label, such as actual job performance, or human review of a sample. Removing proxy features can help, but check that the model has not simply found other proxies. Re-weighting is a simple and often effective step: if group B makes up 20 per cent of the data but you want it to count equally, give each of its examples four times the weight of a group A example, though check that the model is not then overfitting to a small group.
Checking the fix
After any mitigation, re-run the full evaluation: overall accuracy, every fairness metric you care about, performance for each group and intersection, and qualitative review of examples. Fixes can move the problem rather than solve it; equalising selection rates might raise false approvals in one group, harming borrowers who are then over-indebted. Often the accuracy cost is small, and sometimes accuracy improves because the model now works better for groups it had neglected. Record what you changed and why in the model card. Finally, remember that some problems cannot be fixed by tuning: if a task such as predicting "criminality" from faces has no valid scientific basis, the responsible choice is not to build it.
Who does the work also matters. Teams that include people from the communities a system serves notice problems earlier, ask better questions about labels and proxies, and design more useful fixes. Paying community members for their time, sharing results with them and giving them a voice in deployment decisions turns bias mitigation from a technical exercise into a partnership. Organisations can also set up external review boards or invite independent auditors, so that the people who built a model are not the only ones judging whether it is fair enough to use.
How it works
- Diagnose the cause: missing data, biased labels, proxies or model behaviour.
- Fix the data first where possible: collect, re-weight or relabel.
- Add in-training or post-processing methods if gaps remain.
- Re-evaluate all metrics and groups, and look for new harms.
- Add human review for uncertain cases, document the changes and monitor after launch.
In code
import numpy as np
from sklearn.linear_model import LogisticRegression
rng = np.random.default_rng(1)
def make(n_a, n_b):
X_a = rng.normal(0, 1, (n_a, 2)); y_a = (X_a[:, 0] + 0.3 * X_a[:, 1] > 0).astype(int)
X_b = rng.normal(0, 1, (n_b, 2)); y_b = (X_b[:, 1] + 0.3 * X_b[:, 0] > 0).astype(int) # B's pattern differs
X = np.vstack([X_a, X_b]); y = np.concatenate([y_a, y_b])
g = np.array(["A"] * n_a + ["B"] * n_b)
flag = (g == "B").astype(float)[:, None]
return X, np.hstack([X, flag, X * flag]), y, g # plain features, and features with group terms
X_tr, F_tr, y_tr, g_tr = make(900, 100) # group B is under-represented in training
X_te, F_te, y_te, g_te = make(500, 500) # a fresh, balanced test set
def report(name, model, X):
pred = model.predict(X)
acc = {grp: (pred[g_te == grp] == y_te[g_te == grp]).mean() for grp in ["A", "B"]}
print(f"{name:<26} test accuracy A={acc['A']:.2f} B={acc['B']:.2f} gap={acc['A'] - acc['B']:.2f}")
plain = LogisticRegression().fit(X_tr, y_tr)
report("plain model", plain, X_te)
weights = np.where(g_tr == "B", 900 / 100, 1.0) # re-weight so both groups count equally
fairer = LogisticRegression().fit(F_tr, y_tr, sample_weight=weights)
report("re-weighted + group terms", fairer, F_te)Output
plain model test accuracy A=0.97 B=0.71 gap=0.26 re-weighted + group terms test accuracy A=0.99 B=0.98 gap=0.01
Try it yourself
Fixing a speech model for rural women
A health helpline's speech recogniser makes twice as many errors for rural women as for urban men.
Which stage is the fix?
Classify each mitigation.
Send borderline cases to a human reviewer
Add a fairness penalty to the loss function
Give examples from a small group more weight
Collect more recordings from under-represented speakers
Use different approval thresholds to equalise error rates
Common mistakes
- Applying a mitigation without re-checking all groups and metrics.
- Assuming fairness always costs a lot of accuracy.
- Trying to "debias" a task that should not be automated at all.
Check yourself
Summary
- Mitigate bias in the data, in training or in decisions after scoring.
- Better data for neglected groups is often the most effective fix; re-weighting is simple.
- Re-evaluate everything, add human review, document changes, and do not build tasks that cannot be fair.
Finished reading?
Sources: Fairlearn: fairness assessment and mitigation toolkit · Barocas, Hardt and Narayanan, Fairness and Machine Learning