Re-Thinking PCI Assessor Selection: Does Quality Matter?

Columnist David Taylor is the Founder of the PCI Knowledge Base and former E-Commerce and Security analyst with Gartner.

Back in high school, I had a Latin teacher who used to make her students recite in front of the room. We all dreaded this because nobody ever got it right. No matter how much we studied, she always found something wrong with our recitations. So, after a while, most of us stopped trying.

That’s sort of how merchants feel after they spend hundreds of thousands (even millions) of dollars to achieve PCI compliance. Then, if they have a breach, suddenly they are told that they weren’t really compliant after all. It’s not surprising some merchants view this as a “no win” situation.

  • Are We Ever Really PCI Compliant?
    In the case of the recent Network Solutions breach, after the company stated that they had been found to be PCI compliant, Bob Russo, head of the PCI Security Standards Council, said, via E-mail: “Until a forensics investigation is completed, an organization cannot comment accurately on its compliance status.”

    As one PCI manager told me recently: “Why should we bother doing PCI ‘right?’ We’ve always stayed away from the low-ball QSAs that would just rubber stamp our compliance. We paid premium prices to get the best assessment. But, if we get breached, it probably won’t matter. The card brands will just bring in an ‘A List’ forensic team and all they have to find is one little thing (out of 200+ controls) and suddenly we’re not compliant, regardless of how much money, time and effort we put in. We should just hire an ‘easy grader’ QSA and get compliant at minimal cost as fast as possible.”

    Now, I’m a pretty cynical guy, but this sentiment made me rethink some of the advice I’ve been giving to companies seeking to hire QSAs.

  • Selecting a QSA – Is Price All That Matters?
    Ever since MasterCard launched a new “gold rush” for QSAs by mandating their use for both Level 1 and Level 2 merchants, we’ve been receiving more and more questions about how to choose a QSA. This comment was enough to make me re-think a sample QSA Request for Proposal (RFP) that we developed and placed in our Document Archive.

    We’ve been telling merchants and service providers to reject "easy graders" who guarantee they will get merchants PCI compliant for a specific fee and in a specific timeframe. Our RFP template includes advice to merchants that tells them to select a QSA that has specific experience in their industry, who understand their business processes and have worked with their specific application platforms. The goal is to find the vulnerabilities before they are exploited to cause a breach.

    Our QSA RFP template goes into great detail about how to evaluate the candidate QSA’s expertise in the testing of controls and his/her experience in designing or evaluating the effectiveness of compensating controls. Maybe we’re wrong. Maybe all that really matters is price.

    Maybe what merchants should look for is “flexibility” when it comes to the evaluation of such areas as store-level security and application security. These controls can be expensive to test properly, involving many personal visits to randomly-selected stores and detailed application code reviews. However, if you choose “wisely,” you can find an assessor who will complete these assessments in a few hours by reviewing a couple of IT architecture diagrams and some screen shots sent via E-mail.

  • Don’t Do It For Compliance – Do It For Your Job
    So what is the downside if you decide to “go cheap” and just hire whatever QSA your acquirer “encourages” you to use, or whoever gives you the best price, or who guarantees compliance? One answer: Your job and your reputation. If you (or a committee that includes you) hire a QSA company primarily based on price and/or “flexibility,” the risk of a breach will be higher, and the risk that the breach will compromise a larger amount of confidential data is also higher.

    But the more immediate problem is that if you use a bargain basement QSA, your Report on Compliance (ROC) is more likely to be bounced by your acquirer or the PCI SSC’s Quality Assurance team. That will lead to more work for you and, more importantly, could be an embarrassment that could cost you the respect of upper management, even if they are the ones who told you to cut the QSA budget.

  • Do it Right Because You Care
    As much as people like to complain about the PCI compliance assessment process, I would estimate that 80 percent of the people I talk to for our research personally care about security. That is why, in the long run, it’s better to hire a QSA company that does a good job, has great references, knows your industry, understands business risk as well as compliance, and won’t just “give in” if somebody challenges them.

    Yes, achieving compliance may not be terribly meaningful beyond avoiding fines. But it’s a way to improve the security of the business and the process helps ensure actions are taken. So, despite the cynicism about the “no win scenario,” we, and our companies, are winners if we simply reduce risk as best we can, given the constraints under which we operate in our business.

  • The Bottom Line
    We started out seriously cynical and ended up just plain serious. I am very interested in what companies are doing as they reverse the mindset from increased self-assessment to the use of QSAs. I especially would like to talk to anyone doing a QSA assessment or who would like to help us improve our QSA selection RFP template. So, please visit the PCI Knowledge Base, and our “Contact us” page, or if you want to have a personal discussion about assessment selection, just send me an E-Mail at [email protected].