Instant Face ID: CRM Will Never Be The Same

In a development that will have a huge impact on retail CRM and IT security operations, a series of Carnegie-Mellon University experiments has established startling recent improvements in facial recognition. The biometric technology improvements would be meaningless, though, were it not for a social-media-fueled avalanche of tagged images and personal information.

The combination makes absolutely practical a science-fiction-like scenario of customers being identified as they walk into a store by virtue of their face alone. Cameras at POS could match faces with names from a payment card, thereby enabling the customer to be subsequently identified and tracked without a loyalty card. The CMU experiments suggest an even more powerful privacy-smashing scenario, with consumers walking into a store identified by any site that has ever posted their pictures.

Imagine a loyalty program with 90 percent participation, requiring no effort—nor, for that matter, intent or consent—from any customers. An absence of laws will make the ethics of retail senior executives the only boundary. (Uh-oh.)

Alessandro Acquisti, a CMU associate professor of IT and public policy, made the presentation at this week's Black Hat USA conference. Acquisti's team wanted to see how far facial recognition had advanced, so they tried matching images of people against the tons of publicly available online images. Last year, for example, according to Facebook itself, 2.5 billion photos were uploaded to Facebook's site alone—each month. That figure has sharply increased this year, and it's only one site.

In CMU's first experiment, the team wanted to see how truly anonymous people were on U.S. dating Web sites. "We downloaded primary profile photos for Facebook profiles from a North American city using a search engine's API (i.e., without even logging on Facebook itself)," Acquisti said. "One out of 10 dating site's pseudonymous members was identified. We constrained ourselves to using only a single Facebook (primary profile) photo and only considering the top match returned by the recognizer. Because an 'attacker' can use more photos, and test more matches, the ratio of identifiable individuals would dramatically increase."

The second experiment was closer to the retail scenario. They asked students to have their picture taken and to then fill out a survey. One-third of all random students were correctly identified in "less than three seconds," Acquisti said.

The third experiment was the CRM privacy coup de grâce. That's where the CMU team shot pictures of subjects and then used public databases to identify the people and their birth dates, activities and Social Security numbers.The professor sees this as the beginning of a period where "faces will be conduits between online and offline data" and an "age of augmented reality, in which online and offline data are blended in real time, [and] may force us to reconsider our notions of privacy." Even that old retail privacy standby—seeking opt-in—won't be a help here, Acquisti argues, "since most data is already publicly available. Facebook sets primary profile photos to be visible to all by default, and members to sign up to the network with their real identities."

Said Acquisti: "I am not sure this is the kind of world I'd like to live in."

The retail potential here is virtually limitless. Any time someone walks into your chain, the video can identify that person as a unique individual. Every product examined, every perfume sniffed and any book that is flipped through can be captured. In the meantime, the system is using public databases to identify the customer, to add a name—and much more—to the file. Once identified, it can do a system lookup to try and match the name with an existing CRM file.

This system could work up extensive records, even for a customer who has no loyalty card and pays for everything with cash.

That's all on the CRM side. This social-media-images-facial-recognition intersection also has vast potential for loss prevention and security. What if security footage could identify—within the three seconds that the CMU study found—almost anyone filmed? What if any prospective employees could be linked to a criminal record before they fill out their application?

Here's one for the store general manager: What if the system could flag store management whenever the camera identifies someone who works for a direct rival entering the store? How about some data sharing? What if a security third-party company decided to offer data analysis on the side? Perhaps stores could be sold data about customers shopping at competitors, so they can flag those customers for extra nice treatment?

Mark Rasch, StorefrontBacktalk's legal columnist and the former head of the U.S. Justice Department's technology crimes group, said the viability of these experiments is quite real given that Facebook is just one part of the images publicly out there.

"Even if you don't have a Facebook page, if someone has posted a picture of you—say, your high school yearbook picture—and attached your name to it, you can be searched. Other picture databases, like Flickr and others, may provide additional information, particularly if they are made public. Note that we are not talking about any private databases like DMV records, credit reports or internal company databases. Just what can be found on the Web."

The next step will be to marry the publicly discovered data with private sources, including the databases in your own chain. "Once I have linked your picture to your name, I can then search for things like your address, your telephone number, your date of birth, criminal history, lawsuits, bankruptcy filings, divorce actions, lawsuits filed and a host of other public databases. Knowing your date of birth and location, I can make educated guesses about your Social Security number (at least the first five digits), and compile a detailed profile of you. If I then link this information to proprietary databases, I can know a great deal about your purchasing habits, Web surfing habits, economics, etc. I can also link this to the car you drove to the mall in, the license plate number and other information. All of this information about potential customers can be displayed instantly."Even more intriguing is that there are no current laws regulating any of this. "Indeed, the law would likely look at all of the information as public—that is, information that someone has exposed to the public. Most laws deal with what information retailers are permitted to collect and what they can do with it, but they don't deal with information like video. Indeed, with the exception of peeping tom laws, very few laws restrict the collection of images of shoppers. In fact, while the law might prohibit you from audio recording a conversation, it would likely not prohibit you from collecting a video of that conversation and hiring lip readers, which is what several British tabloids did at the Royal Wedding," Rasch said.

Clearly, this raises a wide range of ethical and moral issues. But those are typically not IT matters. IT needs to be able to use technology to serve the chain, while others set limits. Hopefully, such limits will be voluntarily set soon. But until then, the amount of data-crunching and data-retention could soon rise rather sharply. As for me, the next time I shop, I'm wearing a Halloween mask. It will be one of those rare times when wearing a gorilla mask will be less scary than not wearing one.

Suggested Articles

Costco changes up its menu items, and Alibaba and Guess partner for a physical store.

Janey Whiteside, Walmart's new chief customer officer, is well acquainted with the importance of customer service in modern retail.

Whole Foods will offer deals on Amazon's Prime Day, and tariffs against China are causing pricing hikes.