Medical Device Cybersecurity Testing: Methods, Timing and Documentation

Introduction

Cybersecurity testing for medical devices has moved from recommended practice to submission requirement. The FDA expects security testing documentation in the premarket submission, and MDCG 2019-16 states plainly that testing is the primary means of security verification and validation.

What has not moved with it is how most manufacturers organise the work. Testing is still commonly treated as a single event before submission — a penetration test booked when the release candidate is ready — and that shape is precisely what regulators have begun to push back on. The two most consequential decisions in a cybersecurity testing programme are not which tools to use; they are what the scope covers and when the tests run.

This guide covers the regulatory basis on both sides of the Atlantic, the eight testing methods and what each one actually detects, where each belongs in the lifecycle, and how to document the results so that a reviewer who is not a security specialist can evaluate them.

Table of Contents

• Why testing became a submission requirement
• The regulatory framework
• The eight testing methods
• Penetration testing
• Fuzz testing
• SAST and DAST
• Vulnerability scanning
• Security feature testing and threat model validation
• Incident response testing
• When each test belongs in the lifecycle
• Building the programme
• Documentation for regulatory submission
• The mistakes that cost a review cycle
• Frequently asked questions
• Conclusions

Why testing became a submission requirement

The shift reflects a change in how cybersecurity failures are classified. A device that can be compromised is not an IT problem with clinical side effects; it is a device whose safety and performance cannot be assured. Ransomware disabling hospital systems, unauthorised access to implantable device programming, and manipulation of diagnostic output have each demonstrated that the consequence reaches the patient.

Regulators responded by making the demonstration evidential rather than declarative. It is no longer sufficient to describe the security architecture; the manufacturer has to show that the controls in it were tested and that they worked.

The regulatory framework

The four naming the same four methods is not coincidence. Security feature testing, fuzz testing, vulnerability scanning and penetration testing are the minimum set: a programme that omits one of them has to explain why, and a programme that includes only penetration testing has covered a quarter of what is expected.

The eight testing methods

Each method detects a different class of defect, and none substitutes for another. The table sets out what each one finds and, as importantly, what it cannot find.

Penetration testing

A simulated attack carried out by security specialists to identify and exploit weaknesses before an adversary does. It is the most widely required form of cybersecurity testing and the one most often scoped too narrowly.

For a medical device the scope has to reach beyond the software interfaces: hardware attack vectors, firmware, wireless protocols, debug ports and any cloud components. A test that covered the application and reported no findings has established that the application is sound, not that the device is.

The phases are conventional — scope definition, reconnaissance, vulnerability identification, exploitation, post-exploitation impact assessment, reporting — and the two that determine the value of the exercise are the first and the last. Scope decides what could be found; the report decides whether a reviewer can evaluate it.

Fuzz testing

Fuzzing supplies random, malformed or unexpected inputs and observes the behaviour. Unlike unit testing it is exploratory: the expected outcome is not known in advance, which is what makes it capable of surfacing defects no one thought to test for.

It is particularly effective on memory corruption, crash conditions and protocol handling errors — exactly the class exploited in real attacks and least likely to be caught by review. For any device exposing a network protocol or parsing external data formats, fuzzing is not optional in practice.

SAST and DAST

Static analysis examines source, bytecode or binary without executing it, and is integrated into the development environment so it runs continuously. It finds hardcoded credentials, weak cryptographic implementations, buffer overflows, injection patterns and unsafe API use. The FDA names credential testing specifically — hardcoded, default, easily guessed or easily compromised.

Dynamic analysis attacks the running application through its exposed interfaces, without source access, and finds what only exists at runtime: authentication bypass, session management flaws, injection that passes static review. For devices with a web interface, an API or a companion mobile app, DAST covers the most exposed surface.

The two are complements with almost no overlap. Running one and reporting it as code security testing leaves half the class uncovered.

Vulnerability scanning

Automated comparison of the device’s components, operating system and services against the CVE and National Vulnerability Database records. It is the operational half of SOUP management: the scan validates the anomaly evaluation that IEC 62304 requires for every third-party component, and it can only do so if the component inventory is complete and current.

That inventory is the Software Bill of Materials, and its accuracy sets the ceiling on what scanning can find. A component missing from the SBOM is a component whose vulnerabilities are never checked. The mechanics of maintaining the list are covered in our guide to SOUP management under IEC 62304.

Security feature testing and threat model validation

Security feature testing confirms that the controls specified in the design work as specified: authentication including any multi-factor mechanism, authorisation and role-based access, session management and timeout, encryption at rest and in transit, audit log completeness and integrity, and the secure update mechanism.

Threat model validation goes one level up and asks whether each control actually mitigates the threat it was assigned to. It is what creates the traceable chain from threat to control to test result that a submission is evaluated on, and it depends entirely on the quality of the threat model underneath it — which is built during design and covered in our guide to threat modeling and cybersecurity risk assessment.

Incident response testing

An exercise rather than a technical test: can the organisation detect, contain and recover from an incident affecting a device already in the field. It is a process-level activity and increasingly expected as evidence that the manufacturer has a credible plan for post-market events rather than a document describing one.

When each test belongs in the lifecycle

Timing is where most programmes go wrong, and the error is uniform: testing treated as a single event before submission. Regulators have pushed back on this specifically, and the FDA has raised concerns where penetration testing was performed long before the submission date, in some cases requiring a fresh test.

Smaller penetration tests during development are worth running even though they do not satisfy the submission requirement, because vulnerabilities are found in almost every one and finding them early is what makes them cheap to fix. External testers are preferred for the pre-release test, both for competence and to remove any argument about independence.

Building the programme

Scope from the threat model

The programme starts with the threat model, because the threat model defines the attack surface: interfaces, components and data flows an adversary could target. The test scope has to cover all of it — hardware interfaces, software APIs, wireless protocols, web and mobile interfaces, cloud components and the update mechanism.

A gap between the threat model scope and the test scope is one of the most common findings in a cybersecurity review, and it is trivially easy for a reviewer to spot: the two documents are read together, and anything in one and not the other is a question.

Depth from risk

Not every component warrants the same depth. Components handling patient data, controlling device functionality or exposed to the internet carry the heaviest coverage; components with limited connectivity and no safety-critical function carry less. All of them carry some. The allocation follows the same risk logic as the rest of the file, and it should trace to the ISO 14971 risk management output rather than being decided separately.

Tester independence and competence

The test report has to document who performed the test, their credentials, and the equipment and methods used. This is not administrative detail: without it, neither the manufacturer nor the regulator can judge whether the testing applied was appropriate, and inappropriately applied testing is difficult to detect from results alone.

Internal testing is acceptable in principle. For Class II and Class III devices, external independent testing is preferred and increasingly expected — and using the development team to penetration test its own code is unlikely to satisfy a reviewer regardless of the team’s skill.

Documentation for regulatory submission

A rigorous test with poor documentation is assessed the same as inadequate testing, because the reviewer can only evaluate what is written. The package has four parts.

Write for a reviewer who is not a security specialist. A finding that sounds benign in security terms — information disclosure through verbose error messages — has to be explained in terms of what an attacker could do with that information on this device, in this clinical environment. The generic CVSS score is the starting point of that explanation, not the end of it.

For an EU technical file the same material is organised inside the risk management file and the software documentation structure defined by IEC 62304 and IEC 81001-5-1, rather than as a standalone cybersecurity section.

The mistakes that cost a review cycle

Testing the software rather than the device. Debug ports, firmware update mechanisms, Bluetooth and Wi-Fi interfaces, USB connections and cloud backends are all attack surface. A scope limited to the application software is a scope that will be questioned.

Testing a build that is not the build. A penetration test on a pre-release version, followed by significant software changes before submission, produces evidence about software that was never shipped.

Tester credentials undocumented. The report does not say who tested, with what competence, using what methods — so the appropriateness of the testing cannot be assessed.

No traceability from findings to risk management. Test results evaluated on generic severity ratings alone, with no reference to the device’s threat model or clinical context. A low CVSS score can represent a critical risk on a specific device.

Only one method run. A penetration test presented as the cybersecurity testing programme. Four methods are named across the FDA guidance and MDCG 2019-16, and each finds a different class of defect.

No post-market schedule. Testing treated as a submission activity, with no defined trigger for retesting after significant software changes, after a significant vulnerability is disclosed in a component, or at periodic intervals.

Frequently asked questions

Is cybersecurity testing mandatory for medical devices?

Yes, in both major markets. The FDA expects security testing documentation in the premarket submission and Section 524B of the FD&C Act gives it authority to refuse submissions that lack adequate cybersecurity evidence. In the EU, Annex I of the MDR sets the requirements at GSPR 17.2, 17.4 and 23.4, and MDCG 2019-16 states that testing is the primary means of security verification and validation.

Which cybersecurity tests does the FDA require?

The guidance calls for verification and validation testing including known vulnerability assessment, malware testing, fuzz testing and structured penetration testing, and strongly encourages static and dynamic code analysis — including testing for hardcoded, default or easily guessed credentials. Evidence of threat model testing demonstrating that the risk controls work is also expected.

How often should penetration testing be repeated?

The final pre-submission test should be performed on the release candidate build and as close to the submission date as practical, because the FDA has questioned tests performed long before submission. After release, testing should be repeated whenever significant software changes are made, when a significant vulnerability is disclosed in a third-party component, and at defined periodic intervals — typically annually for higher-risk devices.

What is the difference between penetration testing and security feature testing?

Penetration testing looks for weaknesses nobody anticipated, by attempting to exploit the system as an adversary would. Security feature testing verifies that the controls the manufacturer specified — authentication, encryption, access control, logging, secure update — were implemented correctly and work as designed. The first tests for the unknown, the second confirms the known. Neither substitutes for the other.

Can internal staff perform the penetration testing?

In principle yes, but external independent testing is preferred and increasingly expected for Class II and Class III devices, both for competence and to remove any question about independence. Whoever performs it, the report must document the testers’ credentials and the equipment and methods used, since the appropriateness of the testing cannot otherwise be judged.

How does vulnerability scanning relate to SOUP management?

Scanning is the operational half of it. IEC 62304 requires the evaluation of published anomalies for every SOUP component, and the scan performs that evaluation against the CVE and NVD records. Its coverage is limited by the accuracy of the Software Bill of Materials: a component missing from the inventory is never checked.

What has to go in the cybersecurity testing documentation?

Four elements: the testing plan with scope, methods, tools and acceptance criteria; a test report for each method performed, including tester credentials; a vulnerability tracking matrix recording every finding, its remediation and the verification of that remediation; and a residual risk evaluation explaining why any unresolved finding is acceptable in the clinical context of the device.

Does a low CVSS score mean a finding can be closed?

No. CVSS scores are generic and take no account of the clinical context. A finding rated low in general terms may carry serious consequences on a specific device in a specific care setting, and the evaluation has to be made against the device’s threat model and risk assessment rather than against the score alone.

Conclusions

Cybersecurity testing is now a multi-method, lifecycle-spanning activity, and the two decisions that determine whether a programme survives review are made before any tool is run: what the scope covers, and when the tests happen.

Scope has to match the threat model, because the two documents are read together and any gap between them is a question. Timing has to put the definitive tests on the build that ships, because a test on an earlier build is evidence about software nobody will use. Everything else — which tools, which vendor, how deep — is a judgement that follows from those two.

The third point is about the writing. A finding is only as useful as the explanation of what it means for a patient, and the reviewer assessing it may not be a security specialist. Translating technical severity into clinical consequence is the part of the documentation that decides whether the testing counts.

If you are assembling the documentation, the SW Documentation Kit covers the IEC 62304 lifecycle with the cybersecurity set alongside it, and the Gold Kit adds the AI/ML module for devices containing a model.

Related articles

• Medical Device Cybersecurity Risk Assessment and Threat Modeling
• IEC 81001-5-1: Cybersecurity for Health Software
• IEC 62304: Safety Classes, Processes and Edition 2 Status
• SOUP Management Under IEC 62304
• Software Bill of Materials (SBOM) for Medical Devices
• Software as a Medical Device (SaMD): Classification and Regulatory Pathway