Medical Device Production, QMS and NPS Software Risk Assessment, Validation, and Protocols

This is the first article of a three-part series on non-product software risk assessment, validation and testing. Download the entire series in one convenient PDF.

• Part 1: This article
• Part 2: Non-Product Software (NPS) Validation For Medical Device Manufacturers
• Part 3: Writing NPS Protocols And Testing For Medical Device Manufacturers

Here’s a relationship tip. If one day your spouse or partner asks you what you did today, never reply by saying that you were working on a master validation plan for non-product software (NPS). Tell them you were working on a program designed to improve the safety of medical devices. Trust me on this.

That tip aside, someday soon you WILL actually be working on this issue and you’re probably here now because – like thousands of other quality and regulatory professionals – you don’t have a clear sense of how to start this process, or even what is required. No problem – let’s get to it.

Develop an Inventory of Non-Product Software

First let’s define NPS, because it’s a broad category. In its simplest sense, NPS is any software that is used in the manufacturing, design, testing, maintenance, packaging, and distribution (and more) of a medical device. Even though NPS is not directly associated with your device, it can still impact the device’s safety.

Imagine, for example, that a high-speed camera inspection system controlled by a software application is not functioning correctly due to a software error. If validation is not performed, we would have no clue that the system is not working correctly and thus is not picking up a specific type of defect, thus allowing poor quality through with possibly no further inspections. Hence, you can think of validation as a means of double-checking that the software does what you want it to do.

You’ll notice that off-the-shelf (OTS) software is included in the list above and must be validated to some degree. We’ll explain that in a moment.

Start by Making a List of All NPS Software

When you start thinking about all the software involved in making or maintaining your medical devices, the prospect of evaluating it all can be daunting. Fear not. There’s a methodical way you can go about evaluating this myriad of NPS systems and meet regulatory requirements – you don’t need to be a software engineer to do it!

Start by creating a spreadsheet that lists as many NPS applications as you can think of. This is a good time to involve your IT department or even Purchasing, as they may already have a list. Determine what information is going to be needed to evaluate this software and the sources available. This can sometimes be challenging, because the vendor of a purchased application may not want to provide any details, or – in the case of OTS software – little information (or none) may be available. The list can get long, so it’s a good idea to group your software into categories. For instance, you may have one group for all software used in the QMS and another group for software used in equipment. You can further separate these into software that is standard off-the-shelf versus software that has been customized or developed internally.

Do a Risk Assessment of Non-Product Software, Including OTS and QMS Software

The risk assessment determines the categories of software, scope and amount of work to be done, what exactly needs to be completed, and what documentation you’ll produce to show your work. A risk assessment involves conducting a risk analysis and a risk evaluation. Assessment is a subset of the entire risk management process. Here are the steps:

1 – Identify the intended purpose of the NPS.

Here you’ll want to describe how the software is going to be used and provide information on the software development life cycle, functionality, maintenance/upgrades, and documentation. You will also define any risks or hazards related to the software itself or potentially to the finished device.

2 – Create a risk assessment document.

Among other things, this document will address whether the software is going to impact any aspect of quality or safety when it is used as intended. You also want to determine and describe whether or how the software access is controlled, how data integrity and records are managed, and cybersecurity risks.

3 – Categorize the risk level for each NPS.

Don’t overthink this. You can create a simple 1 to 5 rating (see the table below). Regardless of how you rate risk, it’s important that you define the factors that would place an NPS into each rating category. The probability of occurrence is also a key factor to document and should figure prominently in your overall assessment. Inputs to this categorization can come from GAMP 5 or ISO/TR 80002-2. Here’s one example of how you might assess software used for a complaint handling data system, with 1 = low risk and 5 = high risk.

Risk Level: 1 = LOW; 3 = MODERATE; 5 = HIGH

4 – Approve or accept the risk assessment determination.

Melting ice cream. Cats walking on your keyboard. Everything involves some sort of risk. It’s your job to identify the risks and figure out which ones are acceptable. It isn’t just your opinion here – this is determining the level of effort needed for NPS validation activities, which sets the stage for continued work. It’s important that you document how you came to your conclusion so that, for instance, if someone were to step into your job years later, that person would know exactly how you arrived at your conclusions.

You can evaluate risk rating with a probability of occurrence to have a better understanding of overall risk. Together these can be used to help determine the level of effort you need to put into validation, something we discuss in our next article. Be thorough, but don’t get carried away. Remember, this is a mini-analysis and you are looking at the impact risk has on the finished device, not patient safety. Make sure to look at the overall system and not just the software component. For more information on risk ranking, consider purchasing the GAMP 5 Guide and/or ISO/TR 80002-2:2017.

Assessing Off-the-Shelf (OTS) Software
OTS confuses many people, who wonder how you can validate software over which you have no control. For example, if you have built an elaborate Microsoft Excel spreadsheet to track complaint data, you do not need to validate Microsoft’s software code because you cannot change it. Instead, you will assess how you use the software, including, for instance, any macros you have created. The same is true for CAD design software. The game changes for software code that you are able to customize.

What’s Next?

The risk assessment process is just the beginning of your journey. Your next step will be to formulate a validation plan in which you describe your validation strategy, how validation will be performed, how often it will be required, who will be doing the work, and why validation is needed. Read the next article in this series to learn how to create that plan.

If validation is soon to become a bigger part of your job responsibilities, consider taking a deep dive into the topic with our medical device software validation training class or our non-product software validation training class. Both are designed to arm you with the knowledge you need to confidently assess and validate software. Our consulting team is also available to assist with medical device or NPS validation.