Facial Recognition
Facial recognition is the automated comparison of a face against one or more stored facial references. Two very different applications share the name, and conflating them causes most of the confusion in public debate: one-to-one verification, which confirms a person matches a specific reference they have presented, and one-to-many identification, which searches a database to determine who someone is.
| One-to-one (verification) | Does this face match this specific reference? — used in identity verification and device sign-in |
| One-to-many (identification) | Who is this face, among a database? — used in law enforcement and surveillance |
| What is compared | A mathematical template derived from the image, not the image itself |
| Accuracy measures | False match rate and false non-match rate, traded against each other by threshold |
| Independent benchmark | NIST’s face recognition evaluation program |
| Known issue | Demographic differentials in accuracy, varying substantially between algorithms |
| Regulated by | Illinois BIPA and similar U.S. state laws, the GDPR, and the EU AI Act |
| Primary attack | Presentation attacks and injected synthetic media |
The distinction that matters
These two applications have different accuracy profiles, different risks and, increasingly, different legal treatment. Treating them as one technology is the source of most bad arguments in both directions.
| One-to-one verification | One-to-many identification | |
|---|---|---|
| The question | Is this the person on this document? | Who is this person? |
| Reference | One image the subject supplied | A database the subject may know nothing about |
| Consent | Inherent — the subject initiated it | Frequently absent |
| Error consequence | A failed check, retried | A misidentification, potentially with legal consequences |
| Accuracy at scale | Stable | Degrades as the database grows — more candidates, more chances of a false match |
| EU AI Act treatment | Permitted, with obligations | Real-time remote use in public spaces heavily restricted |
The scale effect is the technical heart of it. A false match rate acceptable for one comparison becomes a large absolute number of false matches when searching millions of records. This is why a system that performs well in verification can perform poorly in identification without anything being wrong with the algorithm.
How it works
The image is not stored or compared directly. Feature extraction produces a template — a numerical representation of facial geometry — and matching compares templates and returns a similarity score. That score is tested against a threshold, and the threshold is a policy decision rather than a technical one: raising it rejects more impostors and more genuine users; lowering it does the reverse.
On accuracy, the honest position is that modern algorithms are very good and that aggregate accuracy figures conceal important variation. NIST evaluations have found meaningful demographic differentials in false match rates, varying substantially between algorithms — which means the differential is a property of a specific implementation, not of the technology as a category, and it is testable. Buying without asking for that testing is the mistake.
Why it matters for identity verification
In an identity verification flow, facial recognition does one specific job: it binds a document to the person presenting it. The document check establishes that the credential is genuine. The face comparison establishes that the person holding it is the person it was issued to. Neither is sufficient alone — a genuine document proves someone exists, and a live face proves someone is present.
This is one-to-one verification with an explicit reference the subject supplied, which is a materially different activity from database surveillance and is worth saying plainly rather than letting the terms blur.
The comparison is also only as good as the assurance that a real person is in front of the camera. A photograph, a screen replay, a mask or an injected video stream will all match a template if nothing checks for them, which is why liveness detection is not an optional addition but the thing that makes the match mean anything. Biometrics in an identity document verification workflow is the binding step, and identity verification that reads a document without it has verified a credential and nobody in particular.
What facial recognition can’t do
It does not prove a live person is present. That is liveness detection’s job, and without it a printed photo can match.
It does not establish identity on its own. A match confirms similarity to a reference. Whether the reference is authoritative is a separate question — a match against a fraudulent document is a successful match.
Accuracy is not uniform. Performance varies by algorithm, by image quality, and across demographic groups. Independent testing is the only way to know where a given system sits.
A face cannot be reissued. Templates should be protected accordingly, and stored in a form that cannot be reused elsewhere if exposed.
Legality varies sharply. Illinois BIPA, comparable state laws, the GDPR and the EU AI Act impose different obligations, with the strictest aimed at one-to-many use rather than consented verification.
Frequently asked questions
What is the difference between facial recognition and facial verification?
Facial verification is a one-to-one comparison against a specific reference the subject has presented — typically the photograph on their identity document. Facial recognition in the broader sense often means one-to-many identification, searching a database to determine who someone is. They have different accuracy profiles, different consent positions and different legal treatment.
Is a face image stored during verification?
In a well-designed system the comparison uses a mathematical template rather than the image, and retention should be limited to what the purpose and any applicable record-keeping rules require. Templates are biometric data and should be protected as such, since a compromised face cannot be reissued.
Is facial recognition accurate?
Modern algorithms perform very well in one-to-one verification with reasonable image quality. Accuracy degrades in one-to-many search as the database grows, and NIST evaluations have found demographic differentials that vary substantially between algorithms — which makes independent testing of the specific system the relevant question rather than the technology in general.
Can facial recognition be fooled?
Yes, if nothing verifies that a real person is present. Photographs, screen replays, masks and injected video streams can all produce a match against a stored template. Liveness detection and injection attack detection are what close this, and their absence is a more common weakness than matching accuracy.
Related reading
- Liveness detection — what establishes a real person is present, without which a match proves little
- Biometric authentication — the wider category, and why authentication and identification differ
- Facial biometrics explained — how templates are built and compared in practice
- False rejection rate — what accuracy figures mean and how they are commonly misread
