April 23, 2008
One thing that has surprised us at Q-CERT is how often our constituents ask us to perform penetration tests on their networks. Penetration testing, sometimes called pen testing, seems to have become the "gold standard" for information security. "If your experts can't manage to hack into my infrastructure," the thinking seems to be, "then I can be sure the bad guys can't either."
It is true that identifying vulnerabilities in systems is a beneficial exercise. Nonetheless, we believe that penetration testing should be approached from a broader perspective. It needs to be integrated into an overall framework for improving an organization's cybersecurity. Pen testing is not a "silver bullet"—it is simply one tool in the information security arsenal.
It is from this point of view that Q-CERT decided to develop a methodology to provide a more comprehensive, reliable, and repeatable approach to assessing the security of a system. In this methodology, pen testing is just one data point—an important data point, but not the only one.
Before going into detail about the methodology, it is important to understand the objectives of the methodology, which can be outlined as follows:
Once those ground rules were set, we produced the System-Security Assessment Process (S-SAP), which is an assessment of a system's technology; the processes used to administrator, operate, and support the system; and the people managing it. This is known as the people-process-technology (PPT) triangle. The idea of assessing against the PPT triangle is to ensure that the assessment is aligned with the business objectives of the system and that all the false positives observed in the testing are quickly eliminated. The last thing a competent business needs is to be given a long list of vulnerabilities that do not apply because there are compensating controls in place.
The S-SAP methodology is based on four stages. The first two stages set the scene; the third carries out the assessment; and the last one analyzes all the data and produces a meaningful report. The following are the four stages:
This stage focuses on setting up confidentiality agreements, scoping the work, and making sure that the security objectives of the organization are understood. There is little point in providing an assessment report that does not help to maintain the system's value in fulfilling a business process. By fully defining and agreeing on the testing scope, both parties have a much clearer understanding of what will be covered and to what depth. "No surprises" is the rule.
This stage is key to ensuring that an assessment team works effectively, in a coordinated way. It also permits both parties to understand what the schedule of tasks is, how resources will be deployed, and what risks need to be managed. Of course you can't effectively plan an assessment without knowing the system in depth, so system documentation is obtained, analyzed, and fed into the planning project. Once this stage is complete, you are set to start the actual assessment.
This stage is where the action takes place, and it involves assessment of the processes and the people involved in these processes, and technical assessment of the system. Using ISO27002 controls, it is relatively simple to provide an assessment of the people and processes in place, although technically assessing the system is definitely more complicated. For this assessment, S-SAP splits the task into network discovery, configuration assessment, and internal/external vulnerability assessments. These assessments are based on the work produced by the Open Source Security Testing Methodology Manual (OSSTMM) and the Open Web Application Security Project (OWASP), which provide great coverage of the most common assessment areas. But how do we handle technology that is unknown and not assessable by our tools? By relying on the vendor-supplied security guides and tools specifically designed for specific systems (e.g., Oracle DB, SAP module) we fill that missing gap. S-SAP provides details of commonly used vulnerability assessment tools for each area of testing, and the use of the readily available standards means that testing is consistent and follows best practice.
By now, you will have amassed a lot of data about your system—in practice, you end up with far too much data. The analysis phase is a key step in S-SAP, as it takes all the data and tries to make sense of it and prioritize it. Remember those security objectives we obtained early on in the project? In this stage, the results from the assessment are measured against these objectives, which should provide a clear compliance matrix of whether the system is meeting the organizational requirements. Each identified issue is graded with an impact rating of high, medium, or low, and a recommendation on how to fix it is provided. After completion of the report and possibly a presentation (S-SAP includes report and presentation templates), we can close the project, knowing that we really have analyzed the system in the best manner possible.
There you have it: a consistent and comprehensive way of assessing the security of your critical systems. In the coming weeks, we will be posting materials on this site that describe S-SAP in more detail. In the meantime, if you are a member of Q-CERT's CyberSecurity Network (CSN), or are just generally interested in using S-SAP for your assessments, contact us.