Security in Obscurity

Understanding the difference between a penetration test and a vulnerability assessment

While both pen tests and vulnerability assessments can be encompassed in a vulnerability management program each serves a different purpose with potentially very different security objectives, December 2018

Vulnerability assessments and penetration tests are two very common vulnerability management techniques used to better secure an environment. While both can be encompassed in a vulnerability management program each serves a different purpose with potentially very different security objectives.

What is a Penetration test

Lets start by understanding what a penetration test or "pen" test is. Penetration testing is when a system or network is "hacked" using whatever means necessary. You are basically trying to use many of the same techniques that malicious actors would use in the real-world. The purpose is to identify any holes in the system's security as well to validate that vulnerabilities can actually be compromised in a real-world attack [1]. Penetration testing can be done internally leveraging various exploitation and reconnaissance tools as well as outsourced to security consultants to hack the network. Penetration testing is typically done is a way to preserve the system so nothing is damaged. If the process is outsourced, agreements are made to ensure weaknesses in the system are not divulged to any third parties.

Different types of Pen Tests

Penetration tests typically fall into a few general categorizes. The type of pen test you choose to run will greatly depend upon the security objectives you are looking to accomplish with the exercise.

White box: During a white box pen test the tester will have full knowledge of the environment or systems that they are reviewing.

Grey box: In a grey box test some but not all knowledge of the systems to be tested will be known.

Black box: A black box penetration test means the tester is going into the engagement without any knowledge of the environment they will be testing.

Common uses

1. You have a limited number of systems
Penetration tests are practical against a small set of systems on a network. They do not attempt to exploit all vulnerabilities in a network but rather try to exploit a specific vulnerability or confirm a network can be compromised [2]. The larger the network the more diverse the vulnerabilities will be. This will make a penetration test less successful because less will be tested and questions will still remain on how vulnerable a system is.

2. Confirmation is needed that a system can really be compromised
The best way to validate that an identified vulnerability can actually be compromised is to perform penetration test [2]. The tester will select exploits known to compromise the vulnerability in the system and get definitive results on whether or not the system has been hacked.

3. There are not strong time constraints
Penetration testing is not just scanning a system of identify vulnerabilities but rather the validation that those vulnerabilities can be compromised. This requires more time than a vulnerability assessment and a common scenario would be one to three weeks [3].

What is a Vulnerability Assessment

Compared to a penetration test, a vulnerability scan will serve a much different security objective. A vulnerability assessment will scan a network in search of unpatched and known security vulnerabilities. They highlight an network's security liabilities on their systems. Additionally they can help security stakeholders determine the risk associated with each vulnerability.[2] It should be noted that a vulnerability assessment only report vulnerabilities and does not perform any type of remediation or validation of whether the system that has been identified as vulnerable can actually be compromised. Tooling is heavily leveraged in performing vulnerability scanning but due to the limited scope and nature of the activity the results from a vulnerability scan can be gathered significantly faster than a typical penetration test.

Common uses

1. Time constraints
As mentioned above, vulnerability assessments are automated scans of systems on a network. Once configured to scan a IP range the scan will run until it is finished providing results of possible vulnerabilities that can be exploited. Scans can take hours instead of weeks in the case of penetration tests and can give security stakeholders a quicker picture of the agency's overall environment and where exploits could take place.

2. Validation on mitigation attempts
Vulnerability assessments can provide the agency great feedback on steps they have taken to try to mitigate any risks [2]. Once the baseline vulnerability assessment has been completed and the police department has a report detailing systems that may be exploitable, mitigation tasks will be initiated such as patching vulnerable systems. Subsequent assessments performed on the network will allow the agency to see and validate progress man in secure the environment.

3. Activity Trends (see improvement over time)
Closely in line with validation, vulnerability assessments will give the police department feedback of improvement to improving security on the network over time. This trending can provide valuable insight into the agency's change control and remediation processes [2].

Considerations for outsourcing the activities

There are definitely certain areas that warrant consideration as to outsourcing pen testing and/or vulnerability assessment activities. In house experts and resources such as tooling to perform the activities will weight heavily on your decision-making process. For example, successfully penetration tests by outsourced professionals can offer a structured methodology as well as creativity in performing their attacks rather than a just running a fixed set of exploits in a program to generate a report [4]. Since vulnerability scanning is reliant more on tooling that security expertise, the exercise is more common that penetration tests to be done in house. However, companies that provide vulnerability assessments as a service can operate everything that the process entails. This ranges from updating any software required for the scans to run, as well as, monitoring for updates on new vulnerabilities to automatically be included for future scans. Outsourced vulnerability assessments also have the advantage of getting more timely results as new vulnerabilities become know with a turnaround time of a few hours [5].

---

In summary, vulnerability assessments scan a network to identify known vulnerabilities on systems while penetration testing provides validation for whether vulnerabilities. Vulnerability assessments are more apt to time sensitive scenarios where knowledge that a system could be vulnerable is more important so that remediation can take place. Penetration testing takes a more in depth response to provide definitive answers on if the vulnerability is exploitable.

Note: Parts of this thought were previously used in content I submitted while in Grad School during my Vulnerability Assessment course.

---

Resources Consulted

[1] Schifreen, Robert. (2006). Defeating the Hacker: A Non-Technical Guide to Computer Security. West Sussex, England: Wiley. Buy on Amazon

[2] Snedaker, Susan & , et al.. (2007). The best damn it security management book period. Buy on Amazon

[3] FAQ: Penetration Testing. (n.d.). Article Link

[4] White, Jarred. (2012, October 22). Don't Be Fooled! There's No Such Thing as an Automated Penetration Test. Article Link

[5] Korzeniowski, Paul. (2004, May 10). Pros and cons of outsourcing vulnerability assessments. Article Link