QA/RA Consulting, Auditing & Training
In this guide:
Stages in the clinical evaluation process
Defining the scope and drafting a plan
Making sure CER evaluators are qualified
Creating a literature review protocol
Identifying data needed to fulfill plan requirements
Crafting a literature search and review strategy
Choosing the appropriate data
Is your data valid and relevant?
Appraising the clinical data
Analyzing your datasets and drawing conclusions
Clinical data analysis
Alignment between clinical evaluation, IFU, and risk management
Are additional clinical investigations needed?
Writing your CER and when to update it
Compiling the clinical evaluation report
Creating your EU CER template
Create a CER checklist
How often your CER should be updated
Medical device regulatory professionals have been grappling with tighter requirements for clinical data to support clinical evidence since MEDDEV 2.7/1 Rev. 4 was released in mid-2016. Now that the European Medical Device Regulation (2017/745) is nearing implementation, clinical evaluation reports (CERs) have taken on new urgency.
Regardless of whether you are making updates to existing CERs or building one for a product launch, you want to make sure that you don’t experience any nasty surprises during your Notified Body audit and technical documentation review. The first step is to create a robust clinical evaluation plan.
As you probably know, medical device manufacturers can fulfill clinical evidence requirements using one or more of the following:
Before we go into detail on the process, here’s an overview of the various stages outlined in MEDDEV 2.7/1 Rev. 4 and where you will find guidance on each stage.
Published literature is a key component of the clinical data gathered by most companies. While it may be tempting to dig in and start doing literature searches straight away, it’s important that you understand what is needed in the first place. This “scoping” process is vital because you will need to explain and defend it to your Notified Body, and you will need to replicate it in the future.
Your clinical evaluation plan will define the extent of information gathered based on the General Safety and Performance Requirements in Annex I of the MDR. MEDDEV 2.7/1 Rev. 4 defines numerous aspects that should be considered in a thorough clinical evaluation plan. You can find much more detail on the elements to be considered during Stage 0 in Section 7 of the MEDDEV, which delineates between new medical devices and those that already have CE Marking.
MEDDEV 2.7/1 Rev. 4 states: Clinical evaluation is necessary and important because it ensures that the evaluation of safety and performance of the device is based on sufficient clinical evidence throughout the lifetime that the medical device is on the market.
Here are some of the aspects that need to be included in your CER scoping process. Some of these issues apply only to “new” devices getting CE Marking for the first time.
Of course, the amount of data deemed necessary to meet sufficient clinical evidence and the General Safety and Performance Requirements will also be determined by the nature of the device, its stage in the life cycle, and its safety record.
Article 1(a) of Annex XIV in the MDR provides additional detail about what the clinical evaluation plan should include. You’ll want to study this in addition to Section 7 of the MEDDEV before crafting your plan.
Before you dive into planning, you’ll also want to know that the MEDDEV sets some pretty specific guidelines for who can take on this important task. Evaluating clinical data is serious business, and regulators want to ensure that the people performing the evaluation are well qualified to do so. Section 6.4 of the MEDDEV outlines the basic requirements. Evaluators should have an appropriate college degree plus 5 years of professional experience or 10 years of experience if a degree is not a prerequisite. In addition, Section 6.4 goes on to say that evaluators should have knowledge of:
These qualifications will need to be documented and you can be certain that your Notified Body will review documentation related to reviewer qualifications.
Once you have created a robust clinical evaluation plan for your medical device(s), it’s time to get busy figuring out what data you need, drafting a protocol, and then taking stock of that data. This is considered Stages 1 and 2 of the CER process and the specifics can be found in MEDDEV 2.7/1 rev. 4, as shown below.
There are two broad categories of clinical data.
1 – Data not generated by your company (retrieved from online literature searches and other offline data)
2 – Data generated by your company, which can include:
For many companies, the data retrieved from literature searches will represent most, if not all, of the data they collect. That’s why it is so critically important that you develop a literature search strategy that is robust and can be replicated during subsequent updates to your CER. The output of your search and review should obviously include literature on your device and any identified equivalent device, plus a review of the current state of the art. Your literature search protocol should include the following elements (for more specifics on this, see Appendix A5 in MEDDEV 2.7/1 Rev. 4):
You should think of your plan as you would a standard operating procedure (SOP) or a detailed instruction. As an example, if you were to leave the company a year from now, your successor should be able to read your protocol and understand exactly what was done during the previous update.
Make sure you treat the literature searches related to device equivalence as a separate activity from the searches on current state of the art. Separately define and track search terms used, database sources, date parameters, etc. Also, be cognizant that your idea of “state of the art” may differ from reality – here’s why. Before finalizing your protocol, test it. Identify known papers relevant to your device or current state of the art and then test those search parameters to make sure those known papers appear in your search results.
Your literature searches should be extremely thorough and encompass a breadth of search criteria. They also need to be documented in detail so the results can be independently verified and replicated. Your selection of literature should be objective (the good and the bad) and justifiable. Nearly every published study has citations that may lead you to additional relevant data. Again, this is where planning and staying within your device search parameters become really important, because you can very quickly wander down a virtual rabbit hole and not be able to replicate how you got there.
While most publications are available in English (e.g., 93% on MEDLINE), make some attempts to do basic searches in French, German, or Japanese using online translation tools. Obviously, if you work for a larger company that has offices in Europe or Asia, getting help from native-speaking colleagues would be ideal. While the odds are good that you will find what you need in English, you might uncover valuable data that can be translated and further appraised if needed. Also, if you are updating an existing CER, we recommend that you start your search a few weeks before the end date of previous search to ensure you don’t miss anything that may have been added at the very end of your previous search period but had not yet been indexed online.
Keep in mind that some data may be publicly available but not easily accessible online. For instance, the label and IFU of a competitive device (if using one) may yield useful information. Also, presentations made at industry conferences may provide information that is not available (or has not yet been published) online.
During Stage 0, you came up with a clinical evaluation plan focused on determining which data you need and from what sources. In Stage 2, your focus will turn to choosing the right data among the data sets you have found. This requires yet more planning. Your data appraisal plan should address:
According to Section 9.2 of MEDDEV 2.7/1 Rev. 4, the criteria you select should “reflect the nature, history and intended clinical use of the device.” This is something you must document and justify based on current state of the art.
So you found plenty of data in Stage 1 that seem relevant to your device and its intended purpose. But how do you know if the quality is good? At first glance, you don’t. Sure, a paper published in The Lancet is instantly credible, but the quality of a publication cannot serve as the sole rationale for selection. Ultimately you need to dive headfirst into the pool and do investigative research of your own to appraise the quality of the data.
Doing this requires some skill, because it is on you to evaluate the methodology used to collect the data and therefore determine its scientific merit. As part of this appraisal process you will also need to weigh the contribution of each data set to the overall clinical evaluation. Section 9.3 of the MEDDEV contains seven pages of advice on how to evaluate the methodological quality and scientific validity of data sets.
Many companies use equivalency claims to avoid having to conduct redundant pre- or postmarket clinical studies that prove safety and performance. This part of the clinical evaluation process is not new. What is new is the level of scrutiny those comparative evaluations will endure. The clinical evaluation requirements in the existing Medical Devices Directive (93/42/EEC) and MEDDEV 2.7/1 Rev. 3 largely favored device equivalency but did not define it. Thus, some companies took a liberal view of “equivalent.” When MEDDEV 2.7/1 Rev. 4 was released in mid-2016, it gave manufacturers a lot less latitude for determining which devices could be considered equivalent. In fact, Appendix A1 in Rev. 4 is quite clear about the clinical, technical, and biological characteristics your device must have in common with an “equivalent” device.
The MDR furthers tightens the screws for Class III and implantable devices, requiring a more in-depth assessment and making it more challenging to leverage competitor data for new devices. That’s because Article 61, Section 5 of the MDR requires manufacturers of such devices to have access to the full technical documentation of the competitive device(s). It also instructs Notified Bodies to ask for proof that you have a contract in place granting you permanent access to that technical documentation. Good luck with that.
With an appraisal plan created and a grip on how to execute it, you can begin the hard work of appraising the data you have found. This appraisal must be done based on the complete text of publications you find, not just by reading the abstracts or summaries. For each document you appraise, you are required to document your appraisal of it to the point that it could reasonably be reviewed by others. The appraisal results should also support conclusions you are making about clinical safety and clinical performance of the finished device (e.g., citing non-device-related literature would be ranked low for appraisal).
Appendix A6 in MEDDEV 2.7/1 Rev. 4 can be helpful in performing your appraisal of data. It provides some examples of red flags that should make you pause, including clinical data that:
The issue of evaluating statistical methods and significance intimidates many RA/QA professionals. If you feel uncomfortable about your “stats skills” for conducting statistical analysis or reviewing statistical information, hire some outside expertise to evaluate these specific aspects. Better to invest in doing that now rather than have a Notified Body reviewer challenge your appraisal later.
Anyone who has ever researched and compiled an entire European clinical evaluation report knows that the devil is in the details. Finding appropriate clinical data is not necessarily the most challenging aspect of the process – it’s the appraisal and analysis of that data that causes regulatory heartburn.
Many professionals interchange appraisal and analysis, but those are actually two distinct steps in the process. When you appraise data, you are looking to make sure it has statistically significant data sets, uses proper statistical methods, has adequate controls, and properly collects mortality and/or serious adverse event data.
During the analysis stage (Stage 3) you will really dig in and conduct a comprehensive assessment to determine if the data you have found meets clinical safety requirements, clinical performance requirements, and General Safety and Performance Requirements (GSPR) of the EU Medical Device Regulation (MDR). You will evaluate the following:
During Stage 3 you are expected to analyze the data to ensure the clinical evaluation demonstrates that any risks are minimal and acceptable. You also need to consider all aspects of the device’s intended purpose. You’ll find more detail on this in Section 10.2 of MEDDEV 2.7/1 Rev. 4, and in Annex AVII. You will identify gaps related to:
Data from the literature you have appraised is often put into Excel tables to be analyzed. It’s a convenient way to compare different study details, patient populations, endpoints, adverse events, etc. This can end up being a sizable amount of data unto itself, but certain aspects can be parsed into “bite-sized” tables focused on particular issues. While spreadsheets are not especially wonderful for narrative explanations, they are useful as quick comparative tools and are extremely helpful in noting differences between studies when writing the summary.
Your analysis should also examine the alignment between the clinical evaluation, labeling/instructions for use (IFU), and the risk management file, as well as the current state of the art. Reviewers need to pay very close attention to make sure that, for example, the IFU and promotional materials are harmonized with regard to medical conditions and target populations. This analysis also needs to be consistent with the appraisal you conducted during Stage 2. Here’s what Appendix A7.1 of MEDDEV 2.7/1 Rev. 4 has to say about it:
“The information materials supplied by the manufacturer (including label, IFU, available promotional materials including accompanying documents possibly foreseen by the manufacturer), should be reviewed to ensure they are consistent with the relevant clinical data appraised in Stage 2 and that all the hazards, information on risk mitigation, and other clinically relevant information have been identified appropriately.”
It is expected that you will conduct the analysis with input from your risk management files and appropriate standards such as IEC 60601-1 (electrical safety) and EN 62366 (usability). The goal, of course, is to demonstrate that any risks associated with the intended purpose of your device are acceptable when weighed against the benefits it offers the patient or user.
How much is enough? There is no definitive rule that determines whether you have collected enough clinical data to meet the General Safety and Performance Requirements (GSPR) of the EU Medical Device Regulation (MDR). You need to conduct a detailed gap analysis and come to your own conclusion about whether supplemental clinical investigations will be required. At this time, you should also determine whether there are residual risks and uncertainties. This might include factors related to rare complications, long-term performance, or safety under widespread use. You’ll find more information on this in Appendix A2 of MEDDEV 2.7/1 Rev. 4.
You’ve done a lot of work to get this far. Writing a medical device clinical evaluation report (CER) is the culmination of a monumental effort to conduct literature searches, find/review literature, and/or conduct original clinical investigations. Data must be sourced, appraised, analyzed, and then summarized into your CER. This final process – writing the CER itself – is commonly referred to as Stage 4.
Because the contents of a clinical evaluation report vary according to the nature and history of the device being evaluated, neither MEDDEV 2.7/1 Rev. 4 nor the EU MDR provide a detailed CER template. However, Appendix A9 of the MEDDEV does provide nearly six pages of guidance on what the structure of your CER should look like and what content it should contain. Here’s the basic outline.
1 – Summary
2 – Scope of the clinical evaluation
3 – Clinical background, current knowledge, and state of the art
4 – Device under evaluation:
5 – Conclusions
6 – Date of next clinical evaluation
7 – Dates and signatures
8 – Qualification of the responsible evaluators
9 – References
You’ve done all the hard work, but are you sure you didn’t forget anything? The checklist found in Appendix A10 should help. Here’s an abbreviated version of it. (Note that this is only a partial list, to give you a flavor of what your CER checklist should include.)
If the CER covers several models, sizes, settings, or situations, are the conclusions correct for:
Section 6.2.3 of MEDDEV 2.7/1 Rev. 4 provides guidance to manufacturers on how often to update clinical evaluations. It says that the “manufacturer should define and justify the frequency” of CER updates. Typically, this is done in concert with your Notified Body audit and certificate renewal, but that predefined schedule can be tossed out the window if your postmarket surveillance activities uncover new risks.
Shown below is an example of how you could rationalize the frequency of updates to your CER. This sample shows a simple evaluation table for an ECG machine that is well established on the market and has decent clinical history. You can, of course, create your own rating system with more factors, and you could weight these factors as well. We’ve kept the table simple in order to illustrate the point that your NB wants to see that you have a systematic approach to determining your CER update schedule.
As you can see in this example, even though this device is an established ECG with good clinical history, due to its inherently complexity, where it is used on the body, and the target population on which it is used, updating the CER every few years is still recommended.
Regardless of your proposed rationalization for the update schedule, you must update the CER after getting new information from your postmarket surveillance activities that might change your current evaluation. For example, safety reports, newly published literature, or PMCF studies may uncover previously unknown safety concerns. Data from these sources must be to evaluated because it may change your risk/benefit profile. Remember that Annex XIV, Part B of the MDR mandates that your PMCF plan specify the “methods and procedures for proactively collecting and evaluating clinical data.”
Along with your risk management documentation and postmarket surveillance plan, the clinical evaluation report is a centerpiece of the technical documentation needed for CE Marking. While we think of the CER primarily as a regulatory exercise, it’s important to remember that the ultimate goal is the advancement of patient safety. Looking at it through that lens will help you focus on the right things and create a CER in full compliance with MEDDEV 2.7/1 Rev. 4 and the EU MDR (2017/745).
If you found this article to be informative and you want to take the next step in advancing your knowledge of all things CER, consider our EU CER training class. Our consultants are also available to help you with EU CER development and gap analysis.
US OfficeWashington DC
EU OfficeCork, Ireland
UNITED STATES
1055 Thomas Jefferson St. NW
Suite 304
Washington, DC 20007
Phone: 1.800.472.6477
EUROPE
4 Emmet House, Barrack Square
Ballincollig
Cork, Ireland
Phone: +353 21 212 8530