Vulnerability Scan Often Says "Pass" Even If You've Flunked

Peter Spier, CISSP, CRISC, CISM, PMP, QSA, PA-QSA is Manager of Professional Services at Fortrex Technologies.

Take a look at your most recent PCI vulnerability scan. Do you see any identified vulnerabilities that are assigned a score of 4.0 or higher? Are you still passing? Is the "Exceptions, False Positives or Compensating Controls" column blank? If so, your approved scanning vendor (ASV) is not doing its job, and this will prevent you from passing your next PCI assessment.

Since PCI 2.0's implementation earlier this year ASVs have had QSAs closely reviewing their findings and comparing those results against the ASV Program Guide.

The problem here is beyond having the PCI assessment not pass. Retailers will now have to go back to their ASVs, explain the problems and get both sides to work together to fix it. (Yes, the ASVs are going to be thrilled.)

What is worse: That is exactly what the PCI Council's selected Validation Labs are intended to test for, and solve, on an annual basis to prevent just such an occurrence.

The requirement forcing this change: the testing procedure of DSS v2.0 requirement 11.2.2.b, which instructs QSAs to "review the results of each quarterly scan to ensure that they satisfy the ASV Program Guide requirements (for example, no vulnerabilities rated higher than a 4.0 by the CVSS and no automatic failures)."

The ASV Program Guide has an exceptions column for a reason. When no one fills it out, then the retailer's ability to be compliant won't happen. Why would ASVs do this? The answer is simple—and it's hardly a defense: The column is commonly overlooked. This fact raises a more frightening question: Why would ASVs, who focus on minute details for a living, not adhere to a written requirement? The answer: No one knows, including—perhaps—not even the ASV. In short, retailers will now have to go back to their ASVs.

Another aspect is the change in how security issues are ranked. The adoption of the CVSS and its open framework-based calculation methodology permits both ASVs and merchants/service providers to consider multiple vulnerability criteria and effectively determine the probable impact to affected systems. It is this very capability that establishes the foundation upon which DSS v2.0 risk ranking requirements are derived.

The shift from the PCI Security Scanning Procedures' 1 to 5 severity rating method to the ASV Program Guide's CVSS reflects a maturation of DSS-endorsed vulnerability management processes.

The ASV Program Guide gives four clear exceptions in its CVSS guidance:

  • The vulnerability is not included in the National Vulnerability Database (NVD). Such problems may be independently scored by the ASV using the CVSS scoring system, in which ratings of 4.0 and higher remain a failure.
  • The vulnerability is purely denial of service (DoS) related. In that case, it may be passed despite its CVSS score.
  • The identified vulnerabilities violate PCI DSS despite being labeled low severity.
  • The ASV disagrees with the CVSS score noted in the NVD. The retailer may determine a passing status.

In each case, however, ASVs are directed that they must "document these exceptions under Exceptions, False Positives or Compensating Control, as noted in Appendix B: ASV Scan Report Executive Summary."

As a result, ASV-documented exception details help provide valuable insights into applied vulnerability management processes for scan customers. With such intelligence merchants/service providers are better empowered to consider the accuracy of independently scored ASV vulnerabilities as they relate to impacted system components and affected services. Further, the information provides useful reference to merchants/service providers that, as of June 30, are required to assess and appropriately rank vulnerability risks wherein overlapping occurrences are frequently identified in the course of internal scanning processes.

If you notice that the exceptions column is incomplete, consider requesting either corrected and certified reports or a letter of explanation and evidence of the process having been corrected in subsequent scans. Not doing so will impede the timely completion of your PCI assessment and threaten the accuracy of attestation.

If your assessment has already been completed and your QSA did not identify such errors, you should ask the QSA why, because it cannot safely be relied upon that he or she won't again miss the violation in the next assessment. Indeed, this is the very sort of error that, if made, constitutes the potential for Security Standards Council (SSC) initiated QSA remediation. As such, QSAs should want to be aware of and correct the issue.

And if your ASV responds that it has completed annual SSC-required Validated Lab certification without this issue having been raised, it does not prove that the ASV's procedure is correct. Rather, and far worse, it would only illustrate that the Validated Labs are not uniformly enforcing the ASV Program Guide in whole. Without ASV understanding and willing correction, it may prove very appropriate to offer the PCI Council your feedback via the ASV Feedback Form.

As a retailer, it is your responsibility to manage your compliance. In other words, if your QSA passes you and you know that you should not have passed, it's your fault and it will be your neck. With ASVs, it's no different; and it should be no different.

Disagree? Would love to hear from you, either with a comment below or you can zap me an E-mail.