A Face in the Crowd: Say goodbye to anonymity

The following is a script from "A Face in the Crowd" which aired on May 19, 2013. Lesley Stahl is the correspondent. Shachar Bar-On, producer.

Over the last 10 years, the ability of computers to identify faces has gotten 100 times better, a million times faster, and exponentially cheaper.

Yet facial recognition technology is still a work in progress. While investigators in the Boston marathon bombing had multiple images of both suspects, the technology did not come up with a match. They were not identified by their faces, but by their fingerprints! Authorities won't say what went wrong. One possibility is that government data banks - through which the photos would've been searched - are not big enough.

As we discovered, the FBI is working on expanding its database. Businesses are tapping facial recognition to sell us stuff and computer scientists are upgrading the technology.

[Lesley Stahl: So, here it comes! Oh my.]

This may look like a high school science project, but this is Carnegie Mellon's CyLab, a world-class research center.

[Lesley Stahl: Look at that!]

Marios Savvides and his students outfitted this ordinary toy drone with their new advanced facial recognition software... that locks in on a face from a distance, and then identifies it.

[Drone: Hello Lesley, nice to see you.

Lesley Stahl: It got it.]

The students are taking surveillance technology to the next level. They can now turn a blurry face into a clear one; a flat image into a 3D model.

[Lesley Stahl: Oh my goodness.]

Their technology can take a masked face and by focusing only on the eyebrows search a catalog of faces, come up with several people with very similar eyebrows and eventually find the identity of the person.

Marios Savvides: So Utzav is going to take a normal photo of you.

The software maps a face using dots like electronic measles and creates something as unique as a fingerprint: a faceprint.

Lesley Stahl: This is your facial recognition technology working right now to find me?

Utzav: Yes.

For this demonstration, they had added my picture ahead of time to the university's database.

Marios Savvides: That's the top match.

[Samsung Lady: To use face recognition, use the color-coded button on your remote.]

Facial recognition is already in some of our home appliances like TVs. In our mobile devices, PINs and passwords are giving way to faceprints. And the technology can single us out in real-time as we go about our daily business, often without us ever knowing.

Joseph Atick: What's unique about face recognition is the fact that you can do it surreptitiously, from a distance, and continually.

Lesley Stahl: It can happen-- we don't even know.

Joseph Atick: That's the point.

Joseph Atick was one of the first scientists to develop facial recognition software. Twenty years ago he was just about to give up on it when --

Joseph Atick: I opened up the door to my lab. And what I heard in a metallic voice: "I see Joseph."

Lesley Stahl: The computer said I see Joseph cause it took your picture.

Joseph Atick: It detected my presence in the room, it found my face and then it recognized that "this was Joseph." And so I started screaming and invited other people in the lab to come in and see, and the computer started alternating from "I see Joseph" to "I see Paul" to...

But Atick fears he helped create a monster and it's headed to the mall.

In "Minority Report," Tom Cruise is bombarded by ads recognizing him and telling him what to buy. That's still science fiction, but companies are racing to develop digital billboards for shopping malls that without your being aware of it scan your face to tell your gender and age. We found this promotional video by Intel online showing how this would work.

[Video: Is the viewer a teenage girl? Then change the content to highlight a back-to-school shoe promotion a few stores down. Is it a senior male? Then why not tell him about the golf club sale at the sporting goods store?]

And now mannequins! A few national chains are installing them with facial recognition as a way to covertly profile their customers. As for identifying us as individuals - well, several companies are working on it: like Hitachi of Japan, as seen in this online sales video:

[Video: The system can automatically detect a face from either surveillance footage or a regular camera and search for it.]

Joseph Atick: Big Brother is no longer big government; Big Brother is big business.

And big business is free to do this kind of surveillance; while government has all kinds of restrictions.

Lesley Stahl: So there are rules for law enforcement, government, military. But no rules for commerce?

Joseph Atick: Commerce. No rules for commercial companies.

There are in Europe, where laws require companies get your consent before they collect your faceprint, but not in the U.S. where regulation is lagging far behind the technology. Meanwhile, some of the biggest companies online are busily building banks of faceprints. If you've been tagged on Facebook, chances are they have your faceprint on file. Google and Apple also make faceprints.

Joseph Atick: My identity, my faceprint should be recognized as my property. My face is as important as my financial records, as my health records. It's very private to me.

Lesley Stahl: What do you mean, "our faces are private"? We're out in the street.

Joseph Atick: Absolutely.

Lesley Stahl: We're walking around. Closed-circuit cameras all over the place. Are they really private?

Joseph Atick: Our faces are private in the sense that my face does not walk around with a tag saying, "I'm Joseph Atick," in the street.

But marketers are working not just on linking our faces on the street to our names, but to our online profiles with our personal data and shopping history.

Lesley Stahl: We used to worry about privacy on the web, now we have to worry about privacy just walking around.

Joseph Atick: The link is between the online and offline persona is becoming possible and that's--

Lesley Stahl: Because of our faces.

Joseph Atick: Yes, because of our faces, exactly.

With security cameras ever present, some people are already thinking up countermeasures.

Joseph Atick: Artists, very clever artists have now begun to create new forms of anonymity by creating patterns that would interfere with face recognition algorithms. So they can go down the street and this system cannot recognize them.

Lesley Stahl: We'll all wear masks! The veil will come HERE.

Joseph Atick: The veil might come here.

Short of wearing a burka, we may all one day become Tom Cruise at the mall, because marketers who track us as we shop online and send us ads, want to do that as we shop in the real world. We found a company that's figured out how to do that.

David McMullen: A customer would just walk into an establishment like this just like normal.

David McMullen is the CEO of redpepper, a Nashville marketing firm developing an app called FaceDeals. As we walk into a bar, this camera identifies me using facial recognition.

David McMullen: And this will actually be the moment when-- I got a deal. How did you--

Lesley Stahl: I got-- I don't know. Something just came up on my phone here. Oh, my, look at that. "Welcome, Lesley. Get a free Diet Coke with a purchase of a Caesar salad." My, my.

My cellphone knew I liked Diet Coke, because in the three seconds it took to walk in, the camera at the door matched my faceprint to my Facebook profile, where redpepper mined my shopping history and Facebook "likes" to send me the perfect deal. They did that only after I "opted in" or explicitly gave them permission. But if you're queasy about trading your privacy for a Diet Coke, McMullen says we've already given up our privacy: cameras in stores, our phones with GPS locators and our credit cards all know where we are when we shop.

David McMullen: All these things are tracking us. What benefit do we get from it? What control do we have over it? Not much.

Lesley Stahl: So they know we're in the store anyway.

David McMullen: That's right.

Lesley Stahl: And they're not offering us anything.

David McMullen: That's correct.

Companies tracking us by our faces may seem a little like spying. Well, since so many of us have one of these - we may soon be able to spy on each other.

Alessandro Acquisti: The ability of remaining anonymous is shrinking. And the places where we can be anonymous are getting fewer and fewer.

Alessandro Acquisti is a professor at Carnegie Mellon who does research on how technology impacts privacy. He says that smart phones may make "facial searches" as common as Google searches and he did an experiment to show how easy it could be. He took photos of random students on his campus. He then ran the pictures through a facial recognition program he downloaded for free that sifted through Facebook profiles and other websites. And he was able not only to identify many of them instantly, he also got their personal data, including in some cases, their social security numbers.

Lesley Stahl: In order for this to work, does the person you're trying to identify have to be on one of these social networks?

Alessandro Acquisti: You must have, somewhere on the Internet, a face with your name on it.

Lesley Stahl: Well, let's say someone doesn't have a Facebook account, but his or her daughter or son does, and they've got your picture. So are they now automatically in the mix?

Alessandro Acquisti: It's funny because one of the participants, before doing the experiment, told us, "You're not going to find me because I'm very careful about my photos online." And we found him. Because someone else had uploaded a photo of him.

But if an academic can easily mine our data with facial recognition, what about the government? Well, the government has a problem because to be effective, facial recognition requires a good database. Facebook for instance has one with billions and billions of photos. The government not nearly that many, and so the FBI is now assembling on these rows of servers the largest biometric database on Earth, costing over a billion dollars. Showing the system for the first time publicly, FBI Assistant Director David Cuthbertson demonstrated how police detectives might use it, when it's fully up and running next year.

David Cuthbertson: This would be the person-- the photograph of the person they are trying to identify.

He used a picture of a deceased criminal.

David Cuthbertson: And so we're submitting the photograph into the system. And it's looking through 12.8 million mug shots in the current system. The FBI has been collecting photographs along with arrest fingerprints for a number of years. This is the first time that anyone's been able to search against those using facial recognition technology.

You've seen this on cop shows, but actually it hasn't been possible to do on a national scale in real time until now.

Lesley Stahl: Will you have a picture of every single American?

David Cuthbertson: No. Absolutely not. Just people who've been arrested.

But why doesn't the FBI just download pictures from Facebook or LinkedIn, since there's no law saying they can't.

David Cuthbertson: There's maybe no legal barrier, but no legal authorization.

Lesley Stahl: You couldn't just do it 'cause you wanted to?

David Cuthbertson: No, ma'am. I would have lawyers lining up outside my door.

Lesley Stahl: So why are so many privacy experts up in arms over what you're doing if you're so restricted by rules and regulations and codes?

David Cuthbertson: I think we get lumped into other factors, other uses of facial recognition whether they be commercial, social media. We're all kind of in this thing together.

You can't forget that it begins with all the information we feed so freely and perpetually onto the Internet: "likes," purchases, searches, not to mention our faces.

Alessandro Acquisti: Often we are not even aware of how much data we are actually revealing or it is being gathered about us or, in fact, how it would be used. The idea that you can start from a face and predict social security numbers from that face seemed quite alien and surprising. But now we know that it can be done.

Lesley Stahl: So there's no place to hide, absolutely no place to hide.

Alessandro Acquisti: It's those places are shrinking.