Penetration testing is an evaluation technique in which skilled security professionals simulate real-world cyberattacks against a system, with its owner's permission, to determine how exposed it really is. Unlike a scan, penetration testing does not stop at listing suspicious findings: it tries to exploit them within an agreed scope and measures what an attacker could actually reach.
In Fluid Attacks' benchmark, one application seeded with 1,201 vulnerabilities across 105 CWE categories was assessed by a pentester, by Fluid Attacks' own scanner and by 36 third-party AppSec tools. The pentester reported 1,076 of those vulnerabilities —89.6% recall, with no false positives. The highest-scoring automated tool was Fluid Attacks' own scanner at 22.7% recall; the best third-party tool reached 21.6%, and the other 34 averaged 1.7%.
This page covers what penetration testing is and is not, why it still finds what tools miss, how Fluid Attacks runs it, the stages a test follows, the types and tools involved, and how often penetration testing should be done.
What is penetration testing?
Manual penetration testing is a security assessment in which a pentester attacks a target of evaluation under agreed rules, exploits the weaknesses found, and documents the impact. Its output is not a list of alerts but evidence: a reproducible path from an entry point to a business consequence.
Two boundaries matter. It is not vulnerability scanning, which automates detection but stops before exploitation. And it is not red teaming, which tests an organization's detection and response capability rather than the security of a specific system.
The technique is also called manual penetration testing, and that qualifier is not redundant. Tools support the work at every stage, but the decisions that produce a finding —which hypothesis to test, which weaknesses to chain, when a result proves impact— are made by a person. "Automated penetration testing" describes scanning with a more ambitious name.
Because pentesting is legal and authorized, the professional performing it is called a pentester, an ethical hacker or a security analyst. Certifications such as OSCP or OSEE signal training, but the condition for the role is demonstrable skill at bypassing defenses within a time budget and a defined scope.
Why is penetration testing important?
Penetration testing matters because it is still where the highest-risk findings come from. Automated detection has improved, and it covers breadth no human can match, but depth and proof of impact remain manual work.
Pentesters report most of the critical findings
In its State of Attacks 2026 report, Fluid Attacks analyzed security testing data collected on client systems between January 1 and December 31, 2025. Pentesters reported 87% of the critical-severity findings across those systems.
The same report shows why counting findings is misleading. Automated tools identified 88.4% of the total vulnerability count, but only 55.8% of the total risk exposure. Measured in CVSSF units, a tool finding averaged 15.5 and a manual finding 93.2 —six times as much risk per issue.
Tools alone leave most of the risk undetected
The benchmark above measured both approaches against the same application. The pentester's recall was 89.6% by vulnerability count and 98.9% by CVSSF, meaning almost all of the application's risk exposure was found. Combined, every automated tool tested would still not reach 60% CVSSF recall.
That gap is not a failure of the scanners: recognizing known patterns at speed is what they are built for. What they cannot do is reason about an application's rules —which is where penetration testing earns its cost.
Compliance sets a floor, not a target
Several standards set a fixed cadence. PCI DSS v4.0.1 requires internal and external penetration testing at least once every 12 months, and tests of segmentation controls every 12 months —every six months at service providers. That six-month cycle covers segmentation controls, not the full penetration test, a distinction most summaries get wrong. Under the FTC Safeguards Rule implementing the Gramm-Leach-Bliley Act, a financial institution without effective continuous monitoring runs annual penetration testing and vulnerability assessments every six months. Those intervals define a minimum, not a security posture. A system that changes weekly is not described by a test run once a year.
Penetration testing, vulnerability assessment and vulnerability scanning are not alternatives
These three terms are often compared as if an organization had to choose one. They answer different questions: vulnerability assessment is the "what" —the systematic evaluation of a system's security— while penetration testing and scanning are two "how"s that feed it.
Attribute | Penetration testing | Vulnerability scanning |
Executed by | A pentester, supported by tools | A tool, on a schedule or trigger |
Typical findings | Business logic flaws, access control bypasses, chained weaknesses | Known vulnerable patterns, outdated components, exposed secrets |
Exploitation | Yes, within the agreed scope | No |
False positives | Near zero; findings are verified before reporting | Common; results need triage |
Coverage | Depth over breadth | Breadth over depth |
Time to first result | Days | Minutes to hours |
Both methods feed the same vulnerability assessment; the comparison is between techniques, not between a technique and a goal.
Framing them as rivals leads to the wrong purchase. An organization that runs only automated scans gets breadth without proof, and one that commissions penetration testing once a year gets proof without coverage. The pairing is what produces a usable vulnerability assessment.
Penetration testing at Fluid Attacks
Fluid Attacks runs Pentesting as a Service (PTaaS) continuously rather than as a point-in-time engagement, alongside Secure Code Review and Reverse Engineering, and combines them with AI and in-house scanners on a single platform. The company has been a CVE Numbering Authority since 2021 —CVE-2026-6093, a SQL injection in Corteza, is one of the records our team assigned— and a CREST-approved penetration testing service provider since 2025.
What we assess, and under what rules
Scope is defined per target of evaluation. Clients register repositories, applications and cloud infrastructure as targets, and penetration testing follows what is registered, not a fixed asset list agreed months earlier. Preproduction and production environments can both be in scope.
Rules of engagement set the limits. Our pentesters operate in safe mode by default, so that attacks demonstrate impact without halting the client's operations, and it changes only when the client asks. They also fix what counts as proof: a finding is reported with the steps to reproduce it, not with a severity label alone.
What it looks like in practice
Technique | What it inspects | Evidence delivered |
PTaaS | Running applications, APIs, networks, cloud infrastructure | Exploit path, video proof of concept, CVSSF-scored impact |
Secure Code Review | Source code, including business logic a scanner cannot interpret | Vulnerable code location, compliant code example |
Reverse Engineering | Compiled binaries and mobile packages | Deconstructed behavior, hardcoded secrets, weak controls |
Reattack | The fix a developer submitted | Verified open or closed status for the original finding |
Each manual technique reports into the same platform as the automated findings, so remediation is tracked in one place.
▶️ Video: we explain how Fluid Attacks combines AI, scanners and pentesters to test software continuously.
Manual and automated results, measured on the same target
Our benchmark is the clearest comparison we can publish, because both approaches ran against an identical application.
Metric | Pentester | Fluid Attacks' scanner | Best third-party tool |
True positives (of 1,201) | 1,076 | 273 | 260 |
Recall by count | 89.6% | 22.7% | 21.6% |
Recall by CVSSF | 98.9% | 8.8% | Not reported |
Precision | 100% | 99% | 72% |
F1 score | 94.5% | 37.0% | Lower than 37.0% |
Figures from Fluid Attacks' benchmark of 36 third-party AppSec tools against one application containing 1,201 vulnerabilities.
On the same 1,201-vulnerability application, the pentester reached 89.6% recall with no false positives.
The limit of that number: 89.6% is not 100%
The pentester's result on that target was 1,076 of 1,201. The benchmark counts 73 of the remainder as false negatives, because the pentester deliberately skipped sections the scanner had already covered; the full gap of 125 is what separates the result from total coverage. Either figure is the argument against selling penetration testing as complete coverage.
It is also the argument against the opposite claim. In 2025, automated tools found 88.4% of all vulnerabilities by count and carried 55.8% of the total risk exposure; pentesters found the remaining 11.6% of vulnerabilities and carried 44.2% of the risk. Neither method is sufficient alone, and any vendor presenting one as a replacement for the other is describing a product, not a result.
Every fix gets re-tested
Finding a vulnerability is half the work. Fluid Attacks' documentation is explicit that "you should reassess the code after a fix attempt", because a change can fail to close the original issue or introduce a new one. Clients request reattacks from the platform, automated or manual, and for a finding reported by the pentesting team it is a pentester who confirms whether it is actually closed. Without that step, a penetration testing program measures how many findings were marked resolved, not how many actually were.
How much access and knowledge do pentesters get?
Two independent variables are agreed before penetration testing starts: how much information the tester receives about the target, and how much the organization's own staff knows about the exercise. Neither is a type of penetration testing, though both are often described as one.
Opaque or black box testing gives the pentester no source code and no internal documentation, so the approach mirrors an external threat actor. Black box testing takes longer and usually surfaces fewer issues, because time goes into discovering what the system is before probing how it fails.
Semi-opaque or gray box testing shares some internal detail with the penetration testing team —architecture notes, test credentials, an API specification— while withholding the source code. It is the practical middle ground for an authenticated surface.
Transparent or white box testing shares the source code and supporting context. White box testing produces the best results, because the analyst can review code and attack the running system at once, and can confirm from the source whether a weakness is genuinely reachable.
On the knowledge side, targeted testing has both parties share every move, so defenses can be adjusted and re-verified during the engagement.
Blind testing gives the tester only the company name.
Double-blind testing keeps part of the security team unaware that an exercise was commissioned —the setting we recommend for red teaming operations, since it measures detection as well as exposure.
Penetration testing stages
There is no single universal sequence, but recognized methodologies converge. NIST SP 800-115 (2008) groups penetration testing into four phases —planning, discovery, attack and reporting— with a feedback loop running from attack back to discovery, because a successful exploit reveals new surface to examine. The Penetration Testing Execution Standard expands the same work into seven, adding threat modeling and post-exploitation as explicit steps. Picking one of them is not optional under PCI DSS: Requirement 11.4.1 obliges the entity to document a methodology built on industry-accepted approaches.
NIST SP 800-115 models penetration testing as four phases, with the attack phase looping back into discovery. Source: NIST SP 800-115 (2008).
Planning and reconnaissance
The organization and the tester agree on scope, applicable standards and what information will be shared. These terms form the rules of engagement. Threat modeling belongs here: deciding which assets an attacker would want, and which paths are worth the time available.
Reconnaissance follows. Passive reconnaissance gathers intelligence from public sources without touching the target; active reconnaissance interacts with it to identify technologies and entry points.
Discovery: scanning and vulnerability assessment
The pentester maps the attack surface with tooling —port scanners for exposed services, Static Application Security Testing when source code is in scope, Dynamic Application Security Testing (DAST) against the running system— and then works through what automation cannot interpret: business logic, authorization boundaries, and combinations of weaknesses that are harmless in isolation.
This is the stage where manual work and automated scanning overlap most, and where their outputs diverge. The scan produces candidates; the analyst decides which ones are worth attacking.
Gaining access through exploitation
This is where penetration testing separates from scanning. The tester builds or adapts an exploit, confirms the vulnerability is reachable, and measures what it yields: data exposure, privilege escalation, lateral movement to adjacent systems.
Blind SQL injection illustrates why this demands judgment. Fluid Attacks' catalog describes it as an attack in which "an attacker sends queries to the server that ask the database true or false questions" —the application returns nothing useful, so the finding is confirmed through behavior rather than output.
Jonathan Armas, security analyst at Fluid Attacks, describes that decision when a filter blocks the usual route:
"From an ORDER BY keyword, we need to use an error-based SQLi or a time-based one".
A scanner reports a suspicious parameter; choosing which variant proves exploitability is the tester's call.
Each weakness entry links the finding to its impact, its remediation and the security requirements it breaks.
Post-exploitation and maintaining access
Once inside, the analyst establishes how long a real attacker could stay and how far they could move. That may involve a backdoor, slow data extraction, or tampering with logs —all simulated within the agreed scope, to document persistence rather than to achieve it. Post-exploitation turns a confirmed vulnerability into a statement about blast radius.
Analysis and reporting
The report is the deliverable. OWASP's Web Security Testing Guide puts it plainly: "a good report helps your client understand your findings". It recommends an introduction covering scope, limitations and timeline; a non-technical executive summary; detailed findings with risk ratings; and reproducible test artifacts.
A penetration testing report should tell the owner what was assessed, what was found and how, what each finding exposes, and what to do about it. Video proofs of concept make the impact hard to dispute, and a clear document lets a development team start fixing the day it arrives.
What are the types of penetration testing?
Types of penetration testing are usually classified by what is being attacked. The categories below follow the target of evaluation, which is the classification we use because it maps to how organizations actually scope an engagement.
Application pen tests
Web and mobile applications are the most common targets. Beyond the OWASP Top 10, penetration testing verifies security requirements that scanners cannot evaluate —for example, restricting access to critical processes, which requires that functions executing critical business processes be available only to authenticated users with the right privileges.
That is a logic check, not a pattern match. A tester holding a low-privilege account tries to invoke an administrative function directly, and the result is either a verified authorization flaw or a control that holds.
Network pen tests
External network penetration testing probes whether the perimeter can be breached, using techniques such as password spraying and credential stuffing. Internal testing establishes what damage is possible from inside —man-in-the-middle attacks, segmentation failures, weak access controls— and is usually the more revealing of the two.
IoT and hardware pen tests
Smart devices are assessed on logical access restrictions, protection of stored and transmitted data, default configuration, credential handling and update mechanisms. A device shipping with a shared default password and no signed-update path fails before any protocol analysis starts.
Personnel pen tests
Social engineering targets people, not systems. Phishing is the most common vector: the tester impersonates a trusted party to get an employee to disclose information or run something they should not. Results are reported as process gaps, never individual failures.
Types of penetration testing tools
Tools do not replace the analyst, but no penetration testing engagement runs without them. Fluid Attacks' documentation states the relationship directly: "Expert security analysts leverage automation enabled by tools to maximize their security testing speed".
Network sniffers: monitor traffic to map which devices communicate, over which protocols and ports. Example: Wireshark.
Port scanners: identify open ports and infer the services and operating systems behind them. Example: Nmap.
Password crackers: recover credentials through brute force, decryption or cached material. Penetration testing uses them to measure password policy in practice.
Vulnerability scanners: run SAST, SCA, DAST and cloud posture checks to flag known issues at speed.
Web proxies: intercept and modify traffic between browser and server to probe request handling.
Attack frameworks: assemble and deliver payloads once a weakness has been confirmed.
The list is a resource inventory, not a methodology. Scope, analysis and validation stay with the person running the penetration testing engagement.
How often should penetration testing be done?
As often as the system changes. Point-in-time penetration testing describes a system on the day it ran, and every deployment after that is untested surface.
Continuous penetration testing addresses that gap in three ways: scope adjusts as new assets appear instead of being frozen in the rules of engagement, the scoping and staffing work a point-in-time engagement repeats each time shrinks to occasional adjustments, and remediation support stays available past the short window a one-off engagement offers. Fluid Attacks' analysis of continuous versus point-in-time testing develops the trade-offs in detail.
One qualifier: continuous penetration testing is not a universal obligation. The right cadence depends on how fast the system changes, what data it handles, and which standards apply. A static internal tool and a payment platform shipping daily do not need the same answer, and an annual test that satisfies an auditor can still leave eleven months of changes unexamined.
Conclusions
Penetration testing earns its place because it produces evidence rather than alerts. The data supports that: 87% of critical-severity findings on Fluid Attacks' client systems in 2025 came from pentesters, and on a controlled target a single analyst found 98.9% of the risk exposure against 8.8% for the best scanner. That is the case for penetration testing, stated with its base.
The reverse is also true. The same pentester left 125 of 1,201 vulnerabilities unreported on that target, and automated tools covered breadth no human could. Penetration testing is a depth instrument, and treating it as full coverage misreads what the numbers say.
The useful question is not which method wins, but how a penetration testing program combines them —and whether it treats a finding as closed only after someone verified the fix.
Frequently asked questions
What is the difference between penetration testing and vulnerability scanning?
A scanner detects; a pentester exploits. Scanning reports potentially vulnerable patterns without confirming they are reachable, while penetration testing proves exploitability within an agreed scope and measures the impact. Scanning covers more surface, faster, with false positives to triage.
Can penetration testing be automated?
No. Tools automate reconnaissance, scanning and part of the attack phase, but the judgment that defines a pentest —choosing which hypothesis to test, chaining weaknesses, deciding when impact is proven— is human. Products marketed as automated pentesting are performing automated scanning.
How often do security standards require penetration testing?
The common floor is once every 12 months, plus a retest after any significant change. That is what PCI DSS v4.0.1 requires for internal and external testing; its six-month rule applies to segmentation controls at service providers, not to the whole test. The FTC Safeguards Rule under GLBA requires an annual test from institutions that lack effective continuous monitoring.
What should a penetration testing report contain?
Scope and limitations, a non-technical executive summary, detailed findings with risk ratings, and reproducible artifacts. OWASP's Web Security Testing Guide sets out this structure, and adds that the report should serve both management and the engineers who will fix the issues.
Does Fluid Attacks do penetration testing?
Yes. Fluid Attacks delivers it continuously through PTaaS, combined with AI, in-house scanners, secure code review and reverse engineering. In its benchmark against 36 third-party AppSec tools, a Fluid Attacks pentester reported 1,076 of 1,201 seeded vulnerabilities with no false positives.
Test your systems the way an attacker would
Our pentesters attack your applications, networks and cloud infrastructure continuously, and verify every fix before a finding is marked closed. Start free trial · Contact us
Get started with Fluid Attacks' PTaaS right now
