Regulatory Science Meets Cyber Science; Why It’s So Much More than a Pen Test

Anyone who has had professional interaction with the FDA has encountered the term regulatory science. It is used extensively within the agency to convey the scientific disciplines that FDA Centers employ in performing their regulatory functions. More than just examiners with a checklist of pass/fail criteria, the role of the FDA examiner requires a diverse technical, clinical, and analytical skillset that clearly qualifies as a field of science.

At Harbor Labs, we apply a very similar mindset to our professional titles, starting with the question, when does cybersecurity become cyberscience? When the work you perform involves information security, hardware security, computer science, clinical functionality, and an understanding of how all of this affects medical safety, you are certainly deserving of the title scientist. And this is precisely why the unique professional title Cyberscientist is given to most Harbor Labs technical staff positions. It is intended to recognize the diverse technical nature of our staff’s skillsets, as well as convey a subtle marketing identity to our community of clients.

We are frequently engaged by medical device manufacturers who are actively working on a regulatory submission, and have come to us because they “need a pen test performed.” While this is a perfectly reasonable request, at Harbor Labs, the term pen test is rarely used, and only then in a very specific context. The concept of a pen test, in which a variety of tools (Nessus, Metasploit, e.g.) are applied against a target system to identify known vulnerabilities, is only a subset of what is required to truly expose all potential flaws, weaknesses, and vulnerabilities in a medical system. We prefer instead to refer to the testing phase of our analysis as clinical cybersecurity testing. This more comprehensive term captures a broad variety of tests intended to stress the target system in ways that expose all categories of vulnerability. This can include fuzzing, reverse engineering, SAST, MITM, dynamic analysis, robustness (DoS and DDoS), software component analysis, and yes, various forms of COTS and custom pen testing.

But what makes this testing clinical? Cybersecurity analysis alone can be insufficient if the therapeutic, diagnostic, or other clinical functions of the target system are not exercised in parallel. Without this additional context, the severity and functional impact of a vulnerability could be unknowable, leaving an examiner unclear on its true significance. At Harbor Labs, clinical context is at the forefront of our analysis, and has included such testing methods as:

• The actuation of an infusion system by simulating a drug cassette’s behaviors
• Removing the arm of a surgical robot to force an error condition
• Processing actual samples of genetic material in a sequencer
• Attaching EKG leads to our cyberscientists to generate real-time test data

among other similar clinical exercises.

By integrating the medical characteristics and functions of the target device into the cybersecurity testing, the fidelity of the test results are far more meaningful and can lead to a clearer understanding of how security characteristics affect medical performance and patient safety.

When the results of such comprehensive testing are then combined with an understanding of the clinical functions of a target system, and the interactions that occur between the patient and clinician, only then is the testing sufficient for a CDRH examiner. Anyone who has been part of the more than 50% of all regulatory submissions that are rejected already understands this all too well.

Recognizing that medical cybersecurity is indeed a science and treating it as such will significantly reduce time to market for medical device manufacturers and lead to more positive regulatory outcomes for examiners and manufacturers alike.

Related Insights