B6 · Responsible and Safe Use of AI

Making AI Fairer

Making AI fairer means reducing unfair differences in how a system treats groups of people. It combines better data, testing results for every group, clear documentation, human review of important decisions, a way for people to complain, and teams that include the people affected.

8 min Beginner Jump to the activity

What you need: Nothing. This page starts from the beginning, with no programming and no formulas.

Definition

If a model is unfair, can it be made fair? And who decides what "fair" means?

Making AI fairer means reducing unfair differences in how a system treats groups of people. It combines better data, testing results for every group, clear documentation, human review of important decisions, a way for people to complain, and teams that include the people affected.

In simple words

A cricket umpire who keeps making mistakes against one team is not fixed by a single speech. You review the videos, count the decisions for each side, give the umpire better training and a second opinion from a television replay, and let captains challenge calls. Fairer AI is built the same way: by measuring, correcting and checking, again and again.

Why it matters

Bias is not a reason to give up on AI, and it is not something that disappears by itself. It is a problem with known remedies. Knowing them lets you ask the right questions of any system that makes decisions about people.

Key terms

Fairness metric
A measurement that compares how a system performs for different groups.
Representative data
Data that contains every group the system will serve, in sensible numbers.
Model card
A short document describing what a model does, how it was tested and where it should not be used.
Bias audit
A check, often by an independent party, of whether a system treats groups fairly.
Human review
A person checking an automated decision before it takes effect, or when it is challenged.
Redress
A way for a person who was treated unfairly to have the decision corrected.

Explained in detail

Step 1: fix the data

Most bias begins with data that leaves people out. The first remedy is to collect data from every group the system will serve: villages as well as cities, older and younger people, every major language and accent, every skin tone. Where a group is small, teams can gather more examples on purpose, or give the existing ones more weight in training. Labels need the same care: checked by more than one person, and reviewed where they come from past decisions that may have been unfair.

Step 2: measure by group

A fair system cannot be built without measuring fairness. Teams report results separately for each group: how often the system is right, how often it wrongly refuses someone, how often it wrongly approves. There are several ways to define fairness, and they can conflict. One asks that every group receive the same share of approvals. Another asks that qualified people in every group have the same chance of being approved. Another asks that the system's errors fall equally. Mathematicians have shown that, in most real situations, these cannot all be met at once. So choosing which kind of fairness matters is a human decision, which should be made openly and with the people affected.

Step 3: document and disclose

In 2018 and 2019 researchers proposed two simple documents that are now widely used. A datasheet for a dataset records where the data came from, who is in it, who is missing and what it should not be used for. A model card records what a model is for, how it was tested, how it performs for different groups and its known limits. These documents let others spot problems before a system is used on them.

Step 4: keep people in charge

  • Important decisions about people, such as loans, jobs and benefits, should be reviewed by a person who can overrule the system.
  • People should be told when an automated system was used, and given the main reasons for a decision.
  • There must be a simple way to complain and have a decision corrected.
  • Systems should be checked again after launch, because the world and the data change.

Step 5: include the people affected

Teams notice problems that affect people like themselves. A team made up only of young engineers from big cities may never think to test a system on a farmer speaking a regional dialect. Including people from the communities a system will serve, as designers, testers or advisers, catches problems early. Some laws now require outside checks: since 2023, New York City has required employers who use automated hiring tools to have them audited for bias each year and to publish a summary.

How it works

ProblemRemedy
A group is missing from the dataCollect more examples from that group
Labels carry past unfairnessReview and correct the labels; rethink the goal
Overall accuracy hides a failureReport results for every group
Nobody can see how it was builtPublish a datasheet and a model card
People cannot challenge a decisionAdd human review and a complaints route

Try it yourself

Fixing an unfair voice assistant

A state helpline's voice assistant understands city callers well and rural callers badly.

Put the steps for a fairer system in order

What should a team do first?

  1. 1Test results separately for each group
  2. 2Train the model
  3. 3Collect data that represents all of them
  4. 4Keep checking after launch
  5. 5List every group the system will affect
  6. 6Launch with human review and a complaints route

Common mistakes

  • Thinking fairness can be added at the end, just before launch.
  • Assuming there is one correct definition of fairness that a computer can calculate.
  • Testing once and never again.

Check yourself

Question 1

1. What is the first thing a team should do to find unfairness?

Question 2

2. What is a model card?

Question 3

3. Why must choosing a definition of fairness be a human decision?

Summary

  • Fairer AI needs representative data, careful labels and results measured for every group.
  • Documentation, human review and a way to complain keep systems accountable.
  • Choosing what fairness means is a human decision, and checking never stops.

Finished reading?

Sources: OECD AI Principles · UNESCO: Recommendation on the Ethics of Artificial Intelligence