The Latest PCI Compliance Stats Disappointing For Level 3s

The latest PCI compliance stats—released by Visa this month—are a mixed bag, with Level 1s plateauing at about 15 major chains still non-compliant. But at the small and midsize merchant level, the numbers are so unimpressive that Visa has given up specifying the numbers. Not a good sign.

We now have three years of data to examine—2007 through 2009—so, to the extent that Visa has used the same categories during that time, we can add a bit of context to this information.

Small merchant compliance is a big deal because they account for roughly one-third of all Visa card transactions. But about the best Visa can report for this segment right now is that its rate of compliance is "moderate."

Compliance for Level 3 merchants—primarily E-tailers with between 20,000 and 1 million Visa transactions annually—is stagnant at a very low level. Visa reported that this group of roughly 2,500 merchants was 54 percent compliant at the end of 2007. Fair enough. There are more Level 3 merchants; they are not always big enough to show up on acquirers' radar; and Visa's Compliance Acceleration Program (CAP) focused on their larger L1 and L2 brethren. Sadly, the data for 2008 showed almost no movement by L3 merchants, and now Visa has stopped showing their numbers altogether. It says only that compliance in this segment is "moderate."

(Related story: New PCI Details: Changes For Network Segmentation, One-Way PAN Hashing, End-To-End Encryption)

We have no idea what "moderate" means. Is it more than or less than 50 percent or 70 percent or any percent? What we do know is that Visa did not use words like "high" or even "really good," which it could have. We're wondering if this new language (which first appeared, we believe, in the September 2009 Visa report) is a tacit admission that there hasn't been much progress. Maybe it's just too hard to track. In the absence of any kind of numbers since 2008, we have to rate L3 compliance as an industry Fail.

The PCI compliance situation for the smaller merchant universe—the millions of Level 4 merchants—is even murkier. Visa didn't even attempt to track compliance for these merchants who, by the way, account for roughly one-third of all Visa transactions annually.

There is no data for 2007 or 2008 and, as of 2009, Visa says only that compliance in this segment is—wait for it—"moderate." Except this time we get a footnote stating "Level 4 compliance is moderate among standalone terminal merchants, but lower among merchants using integrated payment applications." So now we have "moderate" and "lower than moderate." Perhaps "lower than moderate" is somewhere below "moderate" and slightly above "let's not go there." We can only grade this result as another Fail.

As for the major retailers, Visa classifies merchants having more than 6 million Visa transactions a year as Level 1. These retailers account for half of all Visa transactions annually. At the end of 2007, only 77 percent of L1 merchants were PCI compliant. Two years later, the rate shot up to 96 percent while the number of merchants actually increased slightly (from 326 to 360).

We'd love to know which are the 15 or so L1 merchants that are not compliant. We'd love to know which are the 15 or so L1 merchants that are not compliant (4 percent of 360, for those of you who are not former math teachers), and the bad guys would like to know this information, too—that is if they don't already.

For analysis purposes, the lack of specifics makes meaningful conclusions impossible. It's accepted fact—especially with Visa—that there's a huge difference between a retailer being truly compliant and being certified as compliant.

What's the difference? The certified chain hasn't been breached yet. Yes, the compliance certification is only good until it's actually needed.

Quick quiz: How many PCI-certified retailers in the last few years did not quickly lose their compliance shortly after a breach? They had to. How else could Visa keep saying that no PCI-compliant merchant has ever been breached if it didn't quickly cancel the certification after a breach?

Still, a rate of 96 percent PCI compliance is pretty good, so for L1 retailers we give the industry a Pass.

The story with other large merchants is similarly positive. These are the Level 2 merchants with between 1 million and 6 million Visa transactions annually. Here, PCI compliance increased even faster—rising from a dismal 62 percent at the end of 2007 to a whopping 94 percent by the end of 2009. Once again, 94 percent is an "A" in anybody's book: hence, our Pass assessment. We have to credit Visa's CAP in 2008 with stimulating compliance. This program offered a series of carrots and sticks aimed particularly at L1 and L2 merchants to encourage them to validate their PCI compliance. The results speak for themselves: L1 merchant compliance increased to 91 percent and L2 was at 87 percent by the end of 2008, each higher than the 2007 figures of 77 percent and 62 percent, respectively.

In the prohibited data realm, the numbers are somewhat better, but the conclusions are ambiguous. At the end of 2007, Visa reported that 99 percent of Level 1 and Level 2 merchants told Visa they did not store prohibited data. At the end of 2008, Visa reported the identical 99 percent for the same groups. At the end of 2009, the percentages for both Level 1 and Level 2 were bumped up to 100 percent.

First, everyone is going to raise their eyebrows at any report claiming 100 percent of anything. Second, Visa has introduced a subtle wording change. In the 2007 and 2008 reports (with 99 percent), it reported the retailers "confirmed that they do not store prohibited data." In 2009, Visa said "validated not storing prohibited data." Validated by whom? And how? In some cases, the retailer reported it directly; other times, confirmation came from an assessor.

If asked the question, "Are you still retaining stuff that you're not allowed to retain?," who's going to reply, "Yep, we sure are." (It's like walking into an IRS audit and being asked, "Is there any significant source of revenue you're not reporting?" and replying, "OK. You got me. Yep. May I go now?")

But that scenario still assumes the person answering the compliance question even knows the answer.But that scenario still assumes the person answering the compliance question even knows the answer. What if he or she is confused about what constitutes prohibited data? Much more likely, what if that person believes the retailer is not storing such data but many of his or her colleagues are secretly doing it? (Marketing. Always blame marketing. When seeking out an ethics-challenged department, marketing is universally a prime choice.)

To be fair, it doesn't even have to be ethics-challenged. Other departments—and even some people in E-Commerce or IT—may simply not know the rules. At a large chain, data retention questions are simply impossible to answer honestly. But they are easy to reply to apathetically, because the answer being sought is so clear.

The 2009 Visa report brought some new "prohibited data retention" data points into the mix. For the first time, Visa added columns for Level 3 and Level 4 merchants. But the included data wasn't especially helpful ("Not applicable" and "To Be Determined," respectively). Visa also added VisaNet Processor (direct connection) and Agent (downstream). Rather than the 100 percent or 99 percent for Level 1s and Level 2s, though, VisaNet and Agent got the vague "high," with no definition. (We assume "high" is better than the undefined "moderate." It at least sounds better.)

So, does this mean that VisaNet and Agent are, to some unspecified extent (but presumably a lot higher than one percent—given Visa's willingness to report 99 percent for years), indeed retaining prohibited data? That's comforting.

Regardless, the key story here is merchant compliance and the failure so far to get the millions of merchants that account for a third of all Visa card transactions to be PCI compliant. What is good for large merchants should also be good for smaller merchants, especially because these smaller merchants self-assess (versus needing an outside assessment by a QSA) and they can use a self-assessment questionnaire (SAQ) that can be a short as 11 questions. Smaller merchants have been victimized disproportionately, and the consequences of a major breach and fine may mean bankruptcy.

A year ago, when StorefrontBacktalk PCI Columnist Walt Conway first started tracking these numbers, he figured the brands would turn their attention to small merchants in 2009. And Visa did indeed issue a series of mandates. But these requirements don't seem to be having much of an impact based on the numbers Visa itself reports.

It will take a concerted effort on the part of the brands to make progress, and their only leverage is interchange. Incentive interchange rates successfully paved the way for broad adoption of electronic terminals at the point of sale back in the 1980s and 1990s. Maybe that approach can be used again to incent merchants to become PCI compliant and stay compliant. We're not sure how it would work, but clever minds should be able to devise a program with appropriate incentives and penalties, like was done with CAP.

The only alternative is to have more mandates and acquirer reporting. Unfortunately, these requirements have had only "moderate" success so far. And yet it is hard to see widespread compliance coming about any other way.

One answer may be the acquirer-provided secure processing products that are announced weekly. We need to get to "plug-and-play PCI" that will work for the franchisee, college athletic department, golf course, theater and hardware store that is not compliant today and needs to be. Even with this approach, an interchange incentive from the brands sure would help.