Penetration Testing Methodology: Scope, Run and Act on a Test

A penetration test is an authorized, time-boxed attempt by skilled testers to find and prove exploitable weaknesses in your systems before an attacker does. The methodology has five working phases (scoping, reconnaissance, discovery and exploitation, post-exploitation, reporting) and one that buyers often skip: the retest. The value of a test is decided less by the testers' skill than by how well it is scoped and how quickly the findings get fixed.

Key takeaways

• Scope by question, not by asset list. "Can a user of one tenant read another tenant's data?" produces better testing than "test app.example.com".
• Gray box testing, with an account per role and architecture notes, gives the most findings per day of budget for almost every company.
• A scanner report is not a penetration test. Ask any vendor what share of the work is manual.
• Severity needs business context. A CVSS score tells you how bad a flaw is in general; your data and exposure tell you how bad it is for you.
• Budget the retest from the start. A finding is closed when a tester confirms the fix, not when a ticket moves to Done.
• Run tests annually and after significant change, and keep continuous scanning in between.

Penetration test types and when each one fits

Different targets need different testers and different preparation. Most companies need the first two; the rest depend on what you run.

Two related services are often confused with a penetration test:

• Red teaming is goal-based and stealthy. It tests whether your detection and response team notices an attack, over weeks, with few people told in advance. It makes sense after you have a monitoring function worth testing.
• Bug bounty programs pay external researchers per valid finding, continuously. They complement testing on mature, internet-facing products, and they generate noise you need staff to triage.

Black box, gray box and white box

The difference is how much the testers know at the start.

• Black box: no credentials, no documentation. Realistic for an anonymous outsider, expensive per finding, because days go into discovery an insider could skip.
• Gray box: test accounts for each role, API documentation and a short architecture walkthrough. The default choice. Most serious flaws in modern applications are authorization flaws, and those can only be tested with accounts.
• White box: source code and configuration access. The most thorough, best paired with a secure code review for high-risk components such as payment or identity flows.

The methodologies testers follow

Good vendors work to a public methodology and say which one. The main references:

• PTES (Penetration Testing Execution Standard): the phase model most engagements follow, from pre-engagement interactions to reporting.
• OWASP Web Security Testing Guide: the checklist for web applications, organized by area (authentication, authorization, session management, input handling, business logic).
• OWASP API Security Top 10 and the Mobile Application Security Testing Guide: the equivalents for APIs and mobile apps.
• NIST SP 800-115: the US government's technical guide to security testing and assessment, useful as a planning reference.
• OSSTMM: a measurement-focused methodology used by some firms for operational security testing.

A methodology does not make a test good, but its absence is a warning sign. Ask to see the test plan before the engagement starts.

The five phases, from the buyer's side

Each phase has work for you as well as for the testers. Most disappointing tests fail on the buyer's side of these steps.

Phase 1: Scoping and rules of engagement

Scoping decides what gets tested, how deep, and what the test is meant to prove. Write it down as a document both sides sign. A useful scope covers:

engagement:
  questions:
    - "Can a user in one organization read or change another organization's data?"
    - "Can a free-tier account reach paid or admin functionality?"
    - "Can a leaked API key reach anything beyond its own tenant?"
  in_scope:
    - app.example.com (web app, all roles)
    - api.example.com/v2 (public API)
    - AWS account 'prod' (read-only role for configuration review)
  out_of_scope:
    - third-party SaaS (payment provider, email provider)
    - denial-of-service and load testing
    - social engineering of staff
  access_provided:
    roles: [org_admin, member, read_only, billing]
    tenants: 2 separate test organizations
    docs: [OpenAPI spec, architecture diagram]
  environment: staging, production-like data volume, no real customer data
  window: 2026-11-03 to 2026-11-14, testing 09:00 to 19:00 IST
  contacts:
    technical: on-call engineer, phone and chat
    emergency_stop: engineering lead

The rules of engagement sit alongside the scope and cover what a production incident looks like if testing causes one:

• Authorization in writing, signed by someone entitled to authorize testing of every in-scope system. For cloud and SaaS, check the provider's testing policy; AWS, Azure and GCP publish theirs.
• Testing windows and whether production is in scope at all. Testing against staging is safer, but only if staging runs the same code and configuration.
• A stop condition: the testers stop and call you if they find live customer data, critical exposure, or evidence that someone else has already been in.
• Data handling: how screenshots, extracted samples and credentials are stored during the test and destroyed after it.
• Source IP addresses the testers will use, so your team can tell the test from a real attack and does not block it halfway through.

Phase 2: Reconnaissance

Testers map the attack surface: subdomains, exposed services, API endpoints, client-side code, public repositories, leaked credentials and anything published about your stack. For you, this phase is mostly a mirror. The first useful output of many tests is a forgotten admin panel, an old staging host still resolving, or a storage bucket nobody remembers creating.

Keep your own asset inventory and compare it with what the testers found. Every surprise belongs in the report.

Phase 3: Discovery and exploitation

This is where manual testing earns its cost. Scanners find known patterns; testers find logic. In modern applications the highest-severity findings are usually:

• Broken object level authorization: changing an ID in a request returns another customer's record.
• Broken function level authorization: a normal user can call an admin endpoint because the UI hid the button but the API never checked the role.
• Tenant isolation failures: a query, cache key or file path that does not include the tenant.
• Authentication flaws: weak session handling, password reset tokens that do not expire, missing MFA on privileged paths.
• Server-side request forgery: a URL field (webhooks, image imports, PDF rendering) that lets the server fetch internal addresses, including cloud metadata endpoints.
• Injection in its many forms, still present wherever input reaches an interpreter.

For your side: keep the on-call engineer reachable, watch error rates and alerts, and do not "fix live" during the test without telling the testers, or they will report a confusing mix of fixed and unfixed behavior.

Phase 4: Post-exploitation and impact

Once a weakness is confirmed, testers measure how far it reaches, within the rules of engagement. Can a stolen application credential read the database? Does a compromised container hold cloud credentials that reach other services? Does one developer laptop give access to production?

This phase turns "a vulnerability" into "this is what an attacker would get", which is what your leadership needs to decide on priority. It is also where cloud configuration testing pays off: the path from a small bug to account-wide access usually runs through over-permissioned roles. Our guide to cloud IAM across AWS, Azure and GCP covers how to cut those paths.

Phase 5: Reporting

A good report is written for two audiences. The executive summary states risk in business terms. The findings section gives engineers what they need to fix each issue without a follow-up call.

A finding worth paying for looks like this:

### F-03: Any organization member can read invoices of other organizations (High)
 
Affected: GET /api/v2/invoices/{invoice_id}
Role used: member of Test Org A
Impact: Exposes invoice line items, billing contact and address for every
customer. Invoice IDs are sequential, so all invoices can be enumerated.
 
Evidence:
1. As a member of Test Org A, request an invoice belonging to Test Org B.
2. The API returns 200 with the full invoice body (request and response attached).
 
Root cause: The handler loads the invoice by ID without checking that
invoice.organization_id matches the caller's organization.
 
Fix: Scope the query by the caller's organization, return 404 for
cross-tenant IDs, and add a regression test per role. Consider
non-sequential identifiers as defense in depth, not as the fix.
 
CVSS 4.0 base score: provided by tester, adjusted for business context below.

Read the report for what it does not say. A good one lists the areas tested and found secure. Without that list, you cannot tell "secure" from "not looked at".

Rating severity with business context

CVSS (the Common Vulnerability Scoring System, now at version 4.0) gives a consistent base score for how severe a flaw is in general. It does not know that the affected endpoint holds health records, or that it is only reachable from an internal network. Adjust it:

Then set remediation times you will actually meet. A common pattern: critical within days, high within a few weeks, medium within the quarter, low tracked with normal work. Write the targets into your security policy so the next test can check you kept them.

Fixing findings and retesting

Most of the value of a test is in the week after the report. A workflow that holds up:

1. Triage within two working days. Confirm each finding, assign an owner, agree on the fix.
2. Fix root causes, not instances. If one endpoint skipped the tenant check, search for every handler that loads by ID. Authorization flaws come in families.
3. Add a regression test per finding so the same flaw cannot return quietly. For authorization, a test per role per endpoint is the durable fix.
4. Retest. Ask the same testers to verify each fix. Most vendors include one retest round if you ask during scoping.
5. Feed the lessons back into code review checklists and CI checks. Our guide to adding security to a CI/CD pipeline shows where those checks fit.

If a test found something that suggests a real attacker may already have been in, stop treating it as a report and start treating it as an incident. The incident response playbook covers the first hours.

How often to test, and what auditors expect

• Annually at minimum, for each customer-facing application and your cloud environment.
• After significant change: a new product surface, a new authentication system, a migration, a major infrastructure rework.
• Before high-stakes moments: an enterprise deal with a security review, a fundraise with technical due diligence, a compliance audit.

Compliance frameworks set the floor. PCI DSS requires internal and external penetration testing at least every twelve months and after significant changes. SOC 2 does not name penetration testing as a mandatory control, but auditors routinely ask for it as evidence that your controls work; see our SOC 2 guide for startups. ISO 27001 expects technical vulnerability management, and an annual test is the usual way to show it.

Between tests, keep continuous coverage: dependency and container scanning in CI, cloud configuration monitoring, and external attack surface scans. A yearly test on top of nothing finds the same easy issues every year.

Choosing a penetration testing vendor

Questions that separate real testing from rebadged scanning:

• What share of the effort is manual, and who exactly will test? Ask for the testers' experience with your stack, not the firm's logo list.
• Which methodology do you follow, and can I see a sample report? A redacted sample shows the depth of findings and the quality of fix advice.
• How do you test authorization? If the answer does not mention multiple roles and multiple tenants, keep looking.
• What is included after the report? A walkthrough call and a retest round should be standard.
• How do you handle data you access? Storage, retention and destruction should be written down.

Price by scope and days, not by number of IPs or pages. A quote that does not ask about roles, tenants and business logic is a quote for a scan.

What goes wrong in production

The failures we see repeatedly are not exotic:

• Testing the wrong environment. Staging runs an older build or a different identity configuration, and the report describes a system you no longer run.
• Accounts that do not cover the roles. Testers get one admin account, so tenant isolation and role checks, the most likely critical findings, go untested.
• Blocking the testers halfway. A WAF or an alert-happy engineer bans the testers' IP addresses on day two, and the rest of the budget is spent on workarounds. Allowlist the source addresses for the window, then review what your monitoring caught afterwards: that is useful detection data.
• Reports that sit unread. Findings go into a backlog with no owner and no date, and next year's test finds them again.
• Fixing the instance, not the pattern. One endpoint gets a tenant check; twelve similar ones do not.
• Treating a clean report as proof of security. A short, clean report from a narrow scope proves very little. Read the scope before reading the result.

Frequently Asked Questions

What are the phases of a penetration test?

Most methodologies use five: scoping and rules of engagement, reconnaissance, vulnerability discovery and exploitation, post-exploitation to measure impact, and reporting. A good engagement adds a sixth: a retest that confirms each finding is actually fixed.

How often should a company run a penetration test?

At least once a year, and after any significant change such as a new product surface, a major architecture change or a cloud migration. PCI DSS requires annual internal and external tests plus tests after significant changes; most SOC 2 and ISO 27001 auditors expect a similar cadence.

What is the difference between a vulnerability scan and a penetration test?

A scanner matches what it sees against known issues and reports possibilities, many of them false positives. A penetration tester confirms which issues are real, chains them together and shows the business impact, such as reading another tenant's data. You need both: scanning continuously, testing periodically.

Black box, gray box or white box: which should I choose?

Gray box gives the best value for most companies. Testers get accounts for each role and basic architecture notes, so they spend the budget finding flaws instead of rediscovering your login page. Black box only makes sense when the question is specifically what an outsider with no knowledge can do.

What should a penetration test report include?

An executive summary in business terms, the scope and dates, and one entry per finding with severity, affected assets, evidence, reproduction steps and a specific fix. It should also list what was tested and found secure, so you can tell coverage from silence.

How long does a penetration test take?

A focused web application or API test usually runs one to two weeks of testing time, and a combined application, cloud and internal network test can take three to four. Add time before it for scoping and access setup, and after it for the report and retest.

Need a penetration test that leads to fixes?

We scope tests around the questions that matter to your business, test authorization across every role and tenant, and stay for the part most firms skip: fixing the findings and proving the fixes hold. If we built it, we're on the pager.

• Cybersecurity and compliance practice
• Talk to an engineer