Point-in-Time vs Continuous: Rethinking the Annual VAPT

The report is stale before the remediation meeting

A typical annual VAPT runs like this. Scoping in week one. Testing in weeks two and three. The report lands in week four. The walkthrough with engineering happens in week five, because two of the four people who need to be in the room are on release duty. By the time anyone opens a ticket, the application under test has had three or four deployments, two new API endpoints are live, a partner integration went up, and somebody spun up a staging environment in a cloud account that was not on the asset list.

Nine months later the auditor asks a simple question: show me that the high-severity findings from the last test were fixed. What exists is a PDF of findings, an email thread, a few Jira tickets in various states, and one engineer’s recollection that “that one was a false positive, I think”. There is no dated record of who closed what, on which build, verified by whom, with what evidence.

This is the common failure, and it is not a failure of testing. The testing was fine. The failure is that the organisation bought a document when what it needed was a control: a loop that detects exposure, assigns it, fixes it, proves the fix, and keeps a record an outsider will accept.

What a point-in-time test gives you, and what it does not

What it genuinely gives you

Point-in-time testing is not a waste of money. It does several things that nothing else does.

• Manual depth. A tester working by hand finds the things scanners structurally cannot: broken object-level authorisation between two tenants, a price field that can be set client-side, a password-reset flow where the token is predictable, a multi-step chain where three low findings combine into account takeover. This is the core value and it is still worth paying for.
• Proof of exploitability. A scanner says a parameter looks injectable. A tester shows you the extracted row. That difference is what moves a finding from the backlog into the sprint.
• A dated, independent baseline. Someone outside your team looked at a defined set of assets on defined dates and wrote down what they found. That has real value for a board, an insurer, a customer and a regulator.
• A forced inventory conversation. Scoping a test is the only time most organisations are made to write down what they actually run and who owns it. The asset list argument during week one is frequently the most useful output of the engagement.
• A procurement or regulatory box, legitimately ticked. Many contracts and sector guidelines require an independent test at a stated frequency. A point-in-time engagement satisfies that on its own terms.

What it cannot give you

A test report is a statement about a defined set of assets on a defined set of dates. It is not a statement about your security posture, and it should not be read as one.

• Any coverage of what shipped after the window closed. If you deploy weekly and test annually, roughly fifty of your fifty-two release weeks are untested by definition.
• Proof of remediation. Unless a retest is contracted and performed against production, the report proves only that problems existed. It says nothing about whether they still do.
• Anything about detection and response. Most engagements are run under rules that explicitly tell the blue team a test is happening. That is the right call for safety, but it means the engagement does not tell you whether a real intrusion would be noticed.
• Cloud configuration drift. A posture review on a Tuesday is accurate on that Tuesday. Security groups get opened for a demo on Thursday.
• Anything about assets you did not declare. The asset list was yours. Shadow infrastructure, a forgotten subdomain pointing at a decommissioned service, a vendor-hosted portal under your brand — none of it appears in a report that never saw it.

The gap between report delivery and closure

The interesting metric in most VAPT programmes is not how many findings were raised. It is the elapsed time between the report date and the date each finding was verified as closed, and how many findings re-appear in the next test. Both numbers are usually worse than the organisation believes, and the reasons are consistent.

• The finding was fixed; the cause was not. A missing authorisation check is patched on the one endpoint the tester found. The next three endpoints the team ships repeat the pattern, because the fix was a condition in a controller rather than middleware applied by default.
• The fix was applied to one instance of a shared component. An outdated library is upgraded in the service that was tested. Four other services use the same base image.
• Infrastructure fixes get overwritten. A hardening change is made by hand on a server; the next automated deployment restores the baseline template that does not contain it. The configuration finding returns looking like negligence when it is actually a pipeline problem.
• The retest happened against the wrong thing. Verification is performed in staging because production access was awkward to arrange. The fix is in staging. Production is three weeks behind.
• Ownership was assigned to “security”. Security does not own the code, the server template or the cloud account. A finding whose owner is a function rather than a named person does not get fixed; it gets discussed.
• The register lived inside the report. A PDF cannot be updated, assigned, re-prioritised or exported. Status therefore lives in email and in heads, which is exactly the state an auditor will not accept.
• Risk was accepted silently. Someone decided the medium findings were not worth the sprint capacity. That decision was correct and completely undocumented, so at audit it looks identical to having forgotten.

What “continuous” should mean in practice

Most vendors selling continuous security are selling more frequent scanning. Scanning more often, without a closure loop attached, produces a louder backlog and nothing else. A continuous programme worth paying for has four parts, and scanning is a supporting detail of the first one.

1. Retest on fix, not retest on calendar. When a team says a finding is fixed, verification happens against the environment where the fix is live, within days, by someone who did not write the fix. The register entry moves to closed only on that verification. This single change does more for a programme than any tooling decision.
2. A findings register that is the system of record. One row per finding per affected asset, with an owner who is a person, a severity, a remediation SLA driven by severity, a status, and an evidence reference. The report becomes an input to the register rather than the place status lives.
3. Release-triggered and change-triggered testing. Testing fires on events, not dates: a release that touches authentication, a new external endpoint, a new cloud account, a new third-party integration, a change to a privileged role, a new subdomain appearing in certificate transparency logs. Depth is matched to the change — a focused test of the changed flow, not a full re-run of the annual scope.
4. Closure evidence captured when closure happens. The request and response that now returns 403, the configuration export, the dependency version, the commit reference, the date, the name of the verifier. Captured at the moment, not reconstructed at audit. Reconstructed evidence is how three-week audit scrambles start.

Around those four, two things remain genuinely periodic. Attack-surface discovery should run on a short cycle, because its whole job is to find the assets your list is missing. And a deep manual test — the week where a human being tries to break your business logic with no particular hypothesis — still belongs on an annual or semi-annual cycle. Continuous does not replace it. Continuous keeps the gaps between those weeks from going dark.

Point-in-time and continuous, side by side

The honest comparison is not “one is better”. It is that they produce different things and miss different things.

Dimension	Point-in-time VAPT	Continuous programme
Coverage	Deep on the declared scope, for the duration of the test window. Zero coverage of anything added or changed afterwards.	Broad and current across a maintained asset inventory, with depth concentrated where change happened. Shallower on any single asset in any given month.
Evidence produced	A dated report, a findings list with reproduction steps, and a retest report if one was contracted. Status after delivery lives outside the deliverable.	A living register with per-finding history, verification records, closure evidence, documented risk acceptances, and exportable trend data on open counts and age.
Auditor acceptance	Accepted as proof that testing was performed. Rarely sufficient on its own for “show me remediation was effective”, which is the question that generates findings.	Answers both questions from one source: testing happened, and findings were closed with dated independent verification. Also answers the awkward follow-up about what is still open and why.
Cost shape	Lumpy. One procurement, one invoice, one budget line, usually in the quarter before the audit. Easy to approve, easy to defer.	Flat and predictable, plus internal cost that is often underestimated: somebody has to triage, assign and chase. Harder to approve once, easier to plan around.
What it misses	Everything that changed after the window. Undeclared assets. Configuration drift. Whether fixes held. Whether anyone would detect a real intrusion.	The long, unhurried manual hunt for business-logic flaws, if the programme is allowed to collapse into automated checks. Also useless if nobody can act within the SLA — it then just documents your backlog precisely.

A findings register an auditor will accept

This is the artefact that decides whether your programme is defensible. It does not need to be a product. A disciplined spreadsheet with change history beats an expensive platform nobody updates. What matters is the field set and the rules.

Field	What it holds	Why an auditor cares
Finding ID	Stable identifier that survives report cycles. One row per finding per affected asset instance.	Lets them trace the same issue across tests and see whether it recurred.
Source	Which engagement, scan, release test or vendor advisory raised it, with the date.	Establishes that findings come from somewhere traceable, not from memory.
Asset and environment	The specific host, repository, endpoint, cloud resource or account — and whether production.	“Fixed in staging” stops being mistakable for fixed.
Severity and rationale	Rating plus a sentence on business impact in your context, not just the generic score.	Shows prioritisation was a decision rather than an inherited number.
Owner	A named individual. Not a team, not a function.	Unowned findings are the single best predictor of a stalled programme.
Remediation SLA and due date	Derived from severity by a written policy, set on the day the finding is raised.	Gives them a standard to test you against — one you set yourself.
Status and history	Open, in progress, fixed-pending-verification, closed, risk accepted, false positive. Every transition dated, with who moved it.	An immutable trail is the difference between a record and an assertion.
Verification	Who verified, on what date, against which build or configuration, by what method.	This is the answer to “prove the fix worked”.
Closure evidence	Reference to the stored artefact: the blocked request, the config export, the version, the commit.	Evidence referenced but not stored is evidence you do not have.
Risk acceptance	Named accepting executive, written reason, compensating control, review date.	Turns a silent gap into a governed decision. Auditors accept governed decisions.

Four rules keep it usable. A severity may be changed, but never without a recorded reason and author. A finding may only reach closed through verification by someone other than the fixer. A false positive needs the same evidence discipline as a closure, because it is the easiest status to abuse. And a risk acceptance must carry an expiry date, so that a one-quarter deferral does not quietly become permanent.

Keep the register exportable as flat data. When a customer questionnaire or an auditor arrives, you want to produce something like this in minutes:

finding_id, source, source_date, asset, environment, severity,
owner, sla_days, due_date, status, status_date, verified_by,
verified_date, verified_build, evidence_ref, accepted_by, review_date

If that export is possible, most of what an auditor or a prospective customer wants to know is already answered. If it is not possible, no amount of report design will cover the gap.

Cadence set by change rate, not by the calendar

Annual testing is a calendar artefact. It exists because audits are annual, not because risk is. A better rule: test an asset when it changes, and set a floor for assets that do not change so that they are not forgotten.

The change signals worth wiring into a trigger are concrete. Release frequency to production. New or modified routes in your API specification. Infrastructure-as-code merge volume. Creation of cloud accounts, subscriptions or projects. New third-party integrations and new outbound data flows. Changes to privileged roles and service-account permissions. New hostnames appearing in certificate transparency logs. Authentication and session code touched by any commit.

Asset class	Change signal to watch	Event-driven testing	Calendar floor
Customer-facing web or mobile application under active development	Production releases; commits touching auth, session or payment flows	Focused test of the changed flow each release; authorisation matrix re-run when roles change	Full manual application test at least annually
Public and partner APIs	New or modified endpoints in the spec; new consumers onboarded	Authorisation, rate-limit and data-exposure tests on each new endpoint before it goes live	Full API test annually; partner scope review each contract cycle
Cloud accounts and infrastructure-as-code	IaC merges; new accounts; IAM policy changes	Posture and privilege-path checks on every merge to the production branch; alert on manual console changes	Deep IAM and configuration review semi-annually
External attack surface	New hostnames, certificates, open ports, exposed admin interfaces	Continuous discovery with triage of anything new within days	External network penetration test annually
Internal network and endpoints	New sites or segments; directory and group-policy changes; major patch cycles	Targeted testing after segmentation or directory changes	Internal test annually, or semi-annually in regulated environments
Stable third-party or vendor-hosted systems you do not change	Vendor advisories and version end-of-life; vendor security attestations	Review on advisory; re-scope when the vendor re-architects	Annual review; test where your contract permits it

Two practical notes. Event-driven testing only works if the events are visible to whoever runs the programme — that usually means read access to the release pipeline and the cloud change log, which is an access conversation, not a tooling one. And the calendar floor matters most for the assets nobody is changing, because those are the ones that silently reach end-of-life.

Answering a customer security questionnaire without a scramble

For a lot of Indian service businesses, the real forcing function is not the regulator. It is an enterprise customer’s security questionnaire arriving with a two-week deadline during a procurement cycle. The questions are predictable and repeat across customers.

They ask when you last had an independent penetration test and by whom. Whether high and critical findings from it are closed. How quickly you commit to remediate by severity. Whether you retest. Whether you scan dependencies and container images. How you handle vulnerability disclosure. Whether you have had a breach and what you did. What your open-finding position is right now.

Every one of those is answerable from a maintained register plus a current engagement report. If you have both, the questionnaire is a half-day of extraction. If you have neither, it is three weeks of reconstruction, and the answers you produce will be softer than the ones your competitor produces from evidence.

• A current dated report and retest report from an independent tester, with a scope statement that matches the service the customer is buying.
• A written remediation SLA by severity, and an export showing your actual performance against it — including the misses.
• An open-findings summary as of a stated date, with owners and due dates.
• Risk acceptances documented with approver, reason, compensating control and review date.
• Evidence of closure for at least the high and critical findings, retrievable without asking an engineer to remember.
• A one-page description of what triggers a test, so “annual” is not the whole answer.

One point on honesty here. If you have open high findings, say so, with the owner, the due date and the compensating control. Reviewers are not surprised by open findings; they are suspicious of a zero. A clean register that claims nothing is open invites exactly the follow-up questions you do not want.

When continuous is the wrong answer

Continuous is not universally correct, and we would rather say so than sell it. If your estate is small and slow-changing — a handful of SaaS applications you do not develop, one line-of-business system from a vendor who patches it, no in-house engineering team, no public API — then a continuous programme will mostly generate findings that nobody has the capacity to act on. For that estate, the proportionate answer is an annual external test, disciplined patching, hardened identity and multi-factor authentication everywhere, a short asset list someone actually maintains, and a register with ten rows in it instead of four hundred.

Before moving to continuous, three prerequisites need to be honestly true. Someone can act on a finding within the SLA you intend to write down — capacity, not intent. There is a tracker that development and infrastructure teams already live in, so findings land where work happens rather than in a security-only tool. And ownership is mappable: for every asset in scope, you can name the person who can change it. If any of those three is missing, fix it first. Continuous testing on top of a team that cannot remediate produces a precise, dated, well-evidenced record of your own inability to keep up, which is worse than a vague one.

The sequence that works in practice is unglamorous. Take your last report and build the register from it — owners, severities, SLAs, status, honest status. Retest the highs and criticals and record real closure evidence. Add attack-surface discovery so the asset list stops lying. Then, and only then, start wiring testing to release and change events. Most organisations get more risk reduction from the first two steps than from any amount of new tooling.

We do not publish invented case studies or recovery percentages. What we will do on a call is look at your last report and your current tracker and tell you which of these steps is actually your constraint. If what you need is a one-off test and nothing more, we will tell you that too.