| 7 min read
Offensive security testing comes in different approaches. The prominent names are breach and attack simulation, penetration testing and red teaming. Their common goal is to determine the strengths and weaknesses of security defenses in systems by assessments based on attacks. But they differ in their execution. At least as they are conceived most commonly. Mainly, there are differences in whether they involve manual tests, assess the attack surface continuously, attack with the organizations' security teams' awareness of the contract, include security at more than the IT system level, and have only specific crown jewels in their scope. At Fluid Attacks we see the shortcomings in the way most products and providers conduct some of these approaches and raise our own way, based on what we perceive to be the present and future of cybersecurity.
In this blog post, we talk about the differences between breach and attack simulation, penetration testing and red teaming, and explain how we break out of the mold in these last two methods. Along the way, we link to other posts where we handle these approaches separately in detail.
The goal they have in common
Organizations worldwide are beginning to understand that they need to secure their digital assets preventively against the looming threat of cybercrime. Under the present circumstances, they recognize it is not enough to scan or review their systems for known weaknesses and vulnerabilities. They need also to prove that such systems do not let actual attacks be successful. A safe way to test and verify this is found in offensive security testing approaches.
We talk about three such approaches here: breach and attack simulation (BAS), penetration testing and red teaming. Development teams, and organizations in general, leverage them to test their systems' detection, resistance and response against attacks that today's threat actors may conduct. As we will repeat below, in the case of red teaming, security testing may include physical facilities and employees as its targets.
Breach and attack simulation vs. penetration testing
Breach and attack simulation solutions have been introduced to the market of security testing more recently than penetration testing solutions. In some instances, they are pitted against each other with the purpose of answering which one is best. Still, because they share the goal we mentioned above, there's confusion around the differences between the two.
Most BAS descriptions agree that this approach tests the effectiveness of the security controls of organizations' systems through attack simulations. Especially, it evaluates whether controls are configured correctly and can aptly detect, resist and respond to the latest threats and techniques. (To learn which controls are paid the most attention, read our blog post about BAS.)
Penetration testing is a quite more familiar term for some. We describe it as the simulation of attacks that a genuine threat actor may conduct against systems. It usually involves looking for vulnerabilities and generating exploits to bypass the system defenses. The targets are often the same controls that are tested with BAS solutions.
To differentiate between the two, some sources say BAS answers the question of whether operational security controls work properly, whereas pen testing assesses whether attackers can breach the system. But these do not seem like two different things. To explain what such sources may be referring to, it's important to understand that not all BAS solutions resemble the standard penetration testing.
Some BAS solutions are tools that limit their tests to internal network security. They install agents on virtual machines, personal computers and servers, where they only check for issues that match a database of known vulnerabilities. They do not test organizations' perimeter defenses. Neither do more advanced BAS tools that work generating malicious traffic inside the internal network, simulating the logical steps of known techniques. These tools cannot answer whether external attackers can get in.
Other BAS solutions can launch Internet-based attacks that test perimeter defenses. They reproduce attacks present in knowledge bases like the MITRE ATT&CK® (adversarial tactics, techniques and common knowledge) framework to check whether the defenses can be bypassed. Their results point to controls in governance frameworks (e.g., those in NIST 800-53) mapped to such knowledge bases. These, like other kinds of BAS tools, return likely attack paths and actionable mitigation insights.
Now let us compare this last kind to pentesting as they are commonly conceived:
Their differences are in the execution. One such difference is most BAS solutions rely solely on tools, whereas penetration testing is always done manually. Indeed, pen testing is performed by humans, who can creatively launch exploits according to the organizations' context and imitate the likely behavior of criminals. (So, throughout this writing we are always referring to "manual penetration testing.") Despite the seemingly legit claims by some vendors that their tools leverage this methodology, we have argued in a previous blog post that there is no such thing as automated penetration testing.
Now, while automation offers reports faster than humans and its implementation can decrease manual effort, tool accuracy is a concern. Relying only on automated scans would yield a great number of false positives. Moreover because some time must be allowed between the announcement and understanding of an attack technique and its later addition to the tools' repertoire, there tend to be significant rates of false negatives.
Manual work helps accuracy and detecting the issues that represent the most exposure to attacks. (To learn more, read our 2022 report.) Ethical hackers, or pen testers, use the same tools and techniques a malicious threat actor would. Contrary to tools, they can get started as soon as cybersecurity researchers and response teams, among other entities (e.g., the US Computer Emergency Readiness Team), announce the threats. This prevents causing a time window during which the current threats may hit the systems.
Another difference is that BAS solutions always test systems continuously, whereas most pentesting offerings assess them just regularly. Please notice we say "most": A model that addresses the need of continuous penetration tests does exist and is called penetration testing as a service (PTaaS). In regards to cybersecurity spend, it is the general opinion that having humans perform security testing is more costly than relying on automation. But given what was explained above, it may actually be more costly in the long run to stick to automation only and avoid continuous manual assessments.
At Fluid Attacks, we acknowledge the limitations in the way penetration testing solutions are conducted in the industry. Our understanding is that, as threats evolve constantly and rapidly, security testing should support prevention effectively. It can do so by providing accurate and updated statuses of the security of systems. For this reason, we offer a penetration testing solution along with automated security testing in a single service offering, Continuous Hacking. We combine automation and manual work to test systems continuously. Our vulnerability scanner runs permanently, and at the same time we assign a team of ethical hackers to probe systems. This methodology allows organizations to make sure that every change in their systems can withstand the relentless efforts of attackers to get in.
Now that we have settled the differences between BAS and pentesting, as they are offered by most other vendors, let's move on to see the differences between penetration testing and red teaming.
Red teaming vs. pentesting
Red teaming is another familiar term that refers to the simulation of real-world attacks. In this approach, security specialists attack under the modality of what are known as "red team exercises." These are essentially operations where they, the red team, advance offensively toward specific objectives. The target organization must detect the attacks and respond. In such exercises, different teams may be created: blue team, white team and purple team.
Basically, the blue team is a group of defensive security specialists (e.g., incident response consultants) in the target organization that manages the security controls to address the intrusions of (in this context) the red team. The white team is the group that calls and approves the initiation and interruption of attacks. And the purple team is responsible for analyzing the interactions between the red and blue teams and suggesting fixes, among other things.
The above suggests a couple of differences to other offensive security approaches in terms of execution. For instance: In red teaming, none, or some, of the defenders know the attackers are contracted by the target organization, whereas in penetration testing all the defenders are fully aware of it.
We have included this in a previous blog post as one of our suggested management policies for conducting offensive security tests. In short, red teams should act just like malicious threat actors, who do not notify when, how and where they will attack, what techniques they use, what information they have accessed, etc.
Further, as could be suspected from our definition of red team exercises, in red teaming, the attackers often have only specific crown jewels in their scope, whereas in penetration testing the objective often is to find all the vulnerabilities and weaknesses in a system.
Another difference, which is one we mentioned earlier, is that red teaming may target an organization's employees and physical facilities, whereas pentesting solutions often attack only IT systems. This means that a red team would engage in activities such as social engineering (e.g., attempting to influence employees to take security risks) and trying to penetrate the company's facilities (e.g., to install devices).
At Fluid Attacks, we offer our red teaming solution in which we agree with our clients on which targets to attack. We do this, again, combining automated tools and human ingenuity to evaluate targets continuously.
Breach and attack simulation vs. red teaming
It could be easily guessed that the differences between breach and attack simulation and red teaming mirror those between the former and penetration testing. That is, BAS solutions normally rely on automation and test continuously, whereas red teaming solutions rely on ethical hackers and normally test only regularly.
There is yet another issue to address. Some vendors claim that their BAS tools have red team and blue team features. That is, they emulate today's adversarial threat actors and provide insight on mitigation. And thus, they help purple teaming tasks. But all this seems like an effort to liken tool actions to what happens in a red teaming exercise. In the end, as long as the organization is run by humans, who panic at the moment of discovering a breach, are influenced by social factors, make mistakes, etc. these tools could not accurately test blue team operations. We mentioned above how tools are limited also in their red team operations. In short, continuous manual intervention is always necessary. Combining it with automated scans helps getting an accurate idea, and fast, about the security status of organizations.
BAS vs. red teaming vs. pentesting in summary
BAS vs. red teaming vs. pentesting in summary
Leverage Fluid Attacks' offensive security testing
At Fluid Attacks we conduct offensive security testing continuously, combining automation and manual work. Take your time to learn more about our penetration testing and red teaming solutions.
You can also check out our 21-day free trial of automated security testing. You can upgrade at any time to include offensive security testing.
Recommended blog posts
You might be interested in the following related posts.
