Definition
Your camera draws a square around every face. Does it know who those people are?
Face detection finds where faces are in an image, without knowing whose they are. Face recognition goes further and works out identity, either by checking that a face matches one saved person (verification) or by searching a collection of many faces for a match (identification). Recognition uses personal information, so it needs consent and strict rules.
In simple words
A guard at a wedding can do three different things. Count how many guests have arrived: that is detection. Check that your face matches the photo on your own invitation: that is verification. Look at every person and search the whole guest list for their name: that is identification, and it is the one that needs the most care.
Why it matters
Your face is not a password you can change. If face data is misused or a system wrongly matches you, the harm is hard to undo. Understanding the difference between finding a face and naming a person helps you judge which uses are reasonable.
Key terms
- Face detection
- Finding where faces are in an image, without knowing whose they are.
- Face recognition
- Working out or confirming a person's identity from their face.
- Faceprint
- The list of numbers a model produces to describe one face.
- Verification
- Checking one face against one saved faceprint: "is this the same person?"
- Identification
- Searching many saved faceprints to find who a face belongs to.
- Biometric data
- Data about the body that identifies a person, such as a face, fingerprint or iris.
Explained in detail
How cameras learned to find faces
In 2001 two researchers, Paul Viola and Michael Jones, published a method that could find faces in an image fast enough to run on an ordinary camera. It looked for simple patterns of light and dark, such as the eyes being darker than the cheeks below them, and rejected most of the image very quickly. Within a few years digital cameras were drawing boxes around faces and focusing on them. That is face detection: it answers "is there a face here?" and nothing more.
From a face to a list of numbers
Recognition needs more. A deep network is trained on millions of face photos so that it turns any face into a list of perhaps a hundred or more numbers, sometimes called a faceprint. The network is trained so that two photos of the same person give lists that are close together, and photos of different people give lists that are far apart, whatever the lighting or expression. Recognition is then a matter of measuring how close two lists are. The system does not store your photo for matching; it stores the numbers.
Why one-to-many is much riskier
Unlocking your phone compares your face with one saved faceprint. Suppose the system wrongly accepts a stranger once in a hundred thousand tries; you would almost never see an error. Now search a collection of ten million faces for one person in a crowd. The same tiny error rate, repeated ten million times, produces about a hundred false matches for every search. The technology is identical. The scale changes everything. This is why searching large collections of faces needs far stricter rules than unlocking a phone.
When a wrong match has consequences
In 2020 a man in the United States was arrested at his home in front of his family and held for more than a day after a face recognition system wrongly matched his driving-licence photo to blurry security footage of a shoplifter. He was innocent. Several similar cases have since come to light, and most of the people wrongly arrested were Black, in line with research showing higher error rates for darker-skinned faces. In each case, officers treated a computer's suggestion as proof.
How the rules are developing
- The European Union's AI Act of 2024 sharply restricts the use of live face identification by police in public places, and bans systems that claim to read emotions in workplaces and schools.
- Several cities, starting with San Francisco in 2019, have banned their own agencies from using face recognition.
- In India, face matching at airports through DigiYatra is voluntary, and the data protection law of 2023 requires consent for using personal data.
A password you cannot change
If a password leaks, you choose a new one. If a collection of faceprints leaks, the people in it cannot change their faces. Face data also allows tracking without your knowledge, since a camera can capture your face from a distance without your taking any action. For these reasons, treat requests for your face data with care. Ask who will hold it, for how long, for what purpose, and whether you can refuse and still use the service.
How it works
| Question | Needs your saved face? | Example | |
|---|---|---|---|
| Detection | Is there a face, and where? | No | Camera focus squares, counting people in a queue |
| Verification | Is this the person they claim to be? | Yes, one saved face | Face unlock on your own phone |
| Identification | Who is this, out of many people? | Yes, a collection of many faces | Searching a crowd for one person |
How recognition works
- Detect: find the face in the image.
- Measure: a model turns the face into a list of numbers that describes it.
- Compare: the list is compared with saved lists.
- Decide: if two lists are close enough, the system reports a match.
"Close enough" is a setting chosen by people. Set it loosely and the system wrongly matches strangers. Set it strictly and it fails to recognise the right person in poor light. No setting removes both kinds of error.
| Use | Reasonable? | Why |
|---|---|---|
| Unlocking your own phone | Usually yes | You chose it, and the data stays on your device |
| Faster airport entry that you signed up for | Yes, with consent and another option | It is voluntary, and you can use the normal queue |
| A shop tracking every visitor without telling them | No | No consent and no clear need |
| Marking attendance of children by face | Needs great care | Children's data, and a simpler method may do the job |
Try it yourself
Finding a face, checking a face, naming a face
Follow one photo through three levels of face technology.
Should face recognition be used here?
Think about consent, need and what happens if the system is wrong.
An app that names strangers from a photo taken on the street
A mall that secretly records which shops each visitor enters
A camera that only counts how many people enter a station
Opening a bank account by video, after you agree to a face check
Unlocking your own phone with your face
Common mistakes
- Thinking a square around a face means the camera knows who it is. That is only detection.
- Thinking face recognition is equally accurate for everyone.
- Uploading photos of other people to face-search tools without their permission.
Check yourself
Summary
- Detection finds faces; recognition attaches an identity to them.
- Verification compares with one saved face; identification searches many.
- Face data is personal and permanent, so consent, accuracy for every group and a clear need all matter.
Finished reading?
Sources: Gender Shades (MIT Media Lab): accuracy of face analysis across groups · AI4K12: Five Big Ideas in AI