Comment: Get Your Money's Worth from PCI Pen Testing

Paying for a penetration test without establishing the right objectives wastes both time and money, says Hollis
Paying for a penetration test without establishing the right objectives wastes both time and money, says Hollis

By now, we all know about the cost and complexity of Payment Card Industry (PCI) Data Security Standards (DSS) compliance. Business professionals everywhere have been struggling in the current market climate to meet PCI program implementation challenges with little or no qualified staff or financial resources. Every line item activity must be unequivocally confirmed as being required, and its purchase must guarantee that it will bring the organization one step closer to compliance.

Given that network and application security penetration testing are such large-ticket items, you would think that CIOs would scrutinize them all-the-more closely to ascertain their scope and applicability to compliance objectives. If you thought so, then you’d be wrong.

In general, security penetration testing is often seen as a service best delivered by the ‘experts’. But testing companies, while often very good at their craft, do not automatically understand your specific business drivers and objectives (such as PCI). They believe that their job is to answer the question: ‘Can someone get unauthorized access to my system?’ While the answer to this question may be very important, it has little or no relevance to achieving PCI compliance objectives, and a CIO paying for a penetration test without establishing the right objectives wastes both time and money.

The Right Objectives

The purpose of conducting a security penetration test of a network or application that processes, stores or transmits card holder data is to discover if its possible to get unauthorized access to this data. This is not only the right objective, it’s the only objective. The question that should drive all your PCI penetration testing is: Can someone get unauthorized access to the cardholder data on my system? Nothing else matters.

This objective must be made clear to the testing company from the outset and stated in all associated contract documentation for two reasons. The first is to ensure the security integrity of the card data environment (CDE) to meet the intent of the PCI DSS, and the second is to be able to produce evidence that you did.

In the world of PCI compliance, if you can’t produce evidence associated with implementing the control, then you can’t verify compliance. Hence, if the report from the security penetration test you’ve commissioned makes no mention of testing the systems for unauthorized access to cardholder data, then you have no actual proof that this was done.

It’s a mistake that countless CIOs make, and one that’s easily corrected. Never assume that the security testing company understands your PCI objectives or burden of proof. Spell it out for them. It will set the right tone for the engagement and result in a bigger return on your investment.

The Right Scope

Do not even consider conducting PCI-required security penetration testing without having your CDE documented. A current and comprehensive network diagram depicting all components, operating systems and applications in your CDE is essential in meeting your PCI testing requirements. The diagram should also show all third-party party supplier, virtual private networks (VPN) and remote connections, in addition to any wireless subnets to the CDE.

It’s a good idea to check the final diagram against your PCI asset register to ensure completeness and accuracy. The completed diagram should serve as your scope and be transmitted to the testing company to clearly mark the territory to be tested. Be sure that this map is also included in the report so that you have evidence of what exactly was tested.

Finally, make sure that your company’s card data security policies are included in the testing scope: specifically your “need to know” principle, your access control mechanisms, password policies and end-user acceptability agreements. In fact, as you go through the two hundred and some odd PCI DSS controls, you should be asking yourself: ‘What else can I include in the testing scope?’

The Right Methodology

Your security penetration test report must detail the methodology that was used for the tests performed. Ensure the methodology corresponds to the DSS. Network testing should include port scanning, TCP fingerprinting and email routing, for example, and web application testing should include Open Source Web Application Security Project (OWASP) issues, such as buffer overflow attempts, cross site scripting and SQL injections.

If possible, have the testing company address each of the stated controls in the report. Once again, this will provide the evidence to prove that the network or application was in fact tested for a PCI-stated vulnerability.

The Right Qualifications

This is something that very few CIOs even consider. The assumption is that if you hire an external professional company to conduct the tests, the testers will possess the appropriate qualifications. This is usually not a problem. The problem is once again a question of evidence. Can you prove that the resources you used to conduct the testing were qualified to do so? You are required to provide this evidence to meet the testing control.

Ensure that your testing company includes the name(s), professional training, certifications and experience of the resources who conducted the test in their report.

The Right Report

The objective, scope and methodology all come together in the report. A good report addresses the results of testing all the network and procedural controls associated with the CDE.

For example, the report should state the start and end dates and locations associated with the testing, if the testing activity was identified in firewall, IDS and file integrity monitoring logs, and cite the last test conducted or last significant change to the network. These activities are required by the PCI DSS and should all be addressed in any test conducted for compliance.

A good report provides evidence that the controls are in place and correctly tested. It should provide one-stop shopping for your QSA or internal auditor to validate your compliance program. The more evidence your testing report can provide, the greater the value you’ll get for your compliance money.

Worthless or priceless? The choice is yours.


Richard Hollis is the chief executive officer for Orthus Ltd., a European information security risk management consulting firm specializing in information risk management services. As a certified information security manager (CISM), certified protection professional (CPP) and a Payment Card Industry (PCI) qualified security assessor (QSA), Hollis possesses extensive hands-on skills and experience in designing, implementing, managing and auditing information security programs.

Over the course of his career, Hollis has served as director of security for Phillips, Paris, and deputy director of security for the US Embassy Moscow Reconstruction Project, as well as a variety of sensitive security positions within the US government and military. In addition to his work with Orthus, he serves on several security technology company boards and security industry advisory councils.

What’s Hot on Infosecurity Magazine?