How Facebook Recognizes Faces
Believe it or not, facial-recognition technology has improved rapidly in the past few years, with Facebook now at the crest of the trend. Initially, point-and-shoot cameras paved the way, using basic face-shape recognition, without assigning identity, to make autofocus more accurate.
The next step was making the recognition more detailed, and assigning identity to faces. Computer makers began building face-tracking and face-identifying software into webcams, and now even game consoles are adopting the technology to let users interact with games using biometric data from their faces (first with Microsoft's Kinect for the Xbox 360 and possibly with Nintendo's upcoming Wii U).
Clearly the most exciting--and scary--part about Facebook’s facial-recognition technology, however, is that it’s supposed to learn what your face looks like as you add and tag more photos of yourself, even though it doesn’t work well now.
“Where we see a face, a computer only sees numbers," notes Andrew Ng of Stanford University, who has worked on artificial intelligence and machine-learning problems throughout his career. "It assigns a value to every pixel, and it's the computer's task to decide that the values add up to 'This is my good friend Joe.'" There's a disparity between the process that’s instant and natural for our eyes and brain, and the process that takes a computer hundreds of thousands of calculations to decide. Facebook, however omnipresent it may seem, isn't omniscient--yet.
To get a better grasp of how Facebook recognizes your face, it’s worth looking at companies like Viewdle, which developed a mobile facial-recognition app that can use Facebook’s databases to identify the names of your friends as you snap a picture of them. Marisol MacGregor, head of product management for Viewdle, says that machine learning is the key to making facial recognition more accurate. "The next time the phone is held up to you, we're using all your database photos [to identify your face],” MacGregor explains. This means that the more photos you have from different angles and in different lighting, the more capable an algorithm becomes at identifying who you are.
However, despite the promises of facial-recognition technology, Facebook and others undoubtedly have a long way to go to create accurate and workable programs.
A big hurdle in biometric tech is that much of it thus far has in some way failed to detect a wide range of facial variation. Companies that adopted facial-recognition technology in the early days learned the hard way that capturing and reading faces means that a computer has to digitally quantify something as socially sensitive and complex as the shape and color of a person's face. The Nikon Coolpix S630 asked Asian users if they blinked after the photo was taken, and HP's MediaSmart computer and Microsoft's Kinect both failed at detecting people with dark skin in low lighting.
Although Facebook doesn’t yet deliver when it comes to consistently suggesting the names of people in uploaded photos, that might not be because the feature doesn’t work--instead, the feature may be erring on the side of caution. Users have hundreds (sometimes thousands) of accurately tagged photos that Facebook can access, but the success of the feature depends on Facebook's refraining from giving embarrassing or insensitive results.
Should You Worry?
Facebook's facial-recognition technology, as it currently exists, is pretty harmless. After all, not only is Facebook using it for benign purposes--tagging photos--but it's just not very accurate, as we’ve shown.
But that doesn't mean you shouldn't be worried about the future implications of such technology. Although facial-recognition technology on its own is harmless, you should realize what, exactly, Facebook is doing: It's building a huge database of people's faces.
Many people may say, "So what? If you don't want Facebook to know what your face looks like, don't use Facebook."
Well, it isn't as simple as that. Facebook can use its ever-growing database as a means of improving its facial-recognition technology, and whether you are on Facebook or not, you will be affected in the future by the site's advances in this technology, as long as you (however accidentally) show up in photos that other people have snapped.
Correction: Marisol MacGregor's title was changed from "marketing director" to "Head of Product Management".