Posted By: By Donna Mitchell-Magaldi, Nerac Analyst

On March 20, 2010, the revised Guideline; MedDev 2.7.1 Rev 3 Clinical Evaluation: A Guide for Manufacturers and Notified Bodies became a requirement for obtaining and maintaining a CE Mark for all classes of devices marketed in the European Union. An important component of the MedDev 2.7.1 Rev 3 is the evaluation of the clinical literature (European Commission, 2009). The literature search is used to identify clinical data that is outside the manufacturer’s possession and can be used to assist in the evaluation of the device in question. The literature can address the specific product or an equivalent device (MedDev 2.7.1 Rev 3).

This can be a daunting task, as many devices may not have been evaluated clinically, or may have been around for so long that there is a plethora of information pertaining to the device.  The selection of appropriate articles is a key component for a successful evaluation. This is especially true if the clinical literature evaluation will serve as the bulk of your clinical evidence.

Although it’s tempting to include any and all articles pertaining to your device or a competitive equivalent, not all articles are created equal.  It’s important to focus on articles that are going to provide the best possible clinical evidence. Non-clinical studies such as animal studies, in vivo studies, in vitro studies (unless the device is an in vitro diagnostic device) and cadaver studies should not be included. Although the information in these studies is important, they are not included in the clinical literature review as the focus of proving safety and efficacy is on human subjects and must be consistent with the intended use of the device.

Bias and Confounding: A Question of Validity

Two elements that affect the validity of the data are bias and confounding. Both can significantly impact the quality of the study.  Three different types of bias we will discuss (there are many more) are selection bias, measurement bias, and analysis bias.  Selection bias occurs when subjects are allocated to a treatment group in such a way that produces a group that does not accurately represent the population, or in such a way that treatment groups are systematically different (collemergencymed). Measurement bias occurs when outcome is inaccurate due to instrument bias (may not have been properly calibrated), or biased expectations of study participants, researchers or care staff (collemergencymed).  Analysis bias occurs when relevant information is omitted or miscalculated. Examples of possible analysis bias are when patients are withdrawn and omitted from the analysis, or data is missing or inaccurate which leads to in inaccurate analysis and incorrect conclusions (May, 1981).

Confounding variables are variables in which the researcher cannot control and can affect the accuracy of study outcomes through misinterpretation of accurate measurements. A confounding variable is also known as a confounder. Confounders can be known or unknown. The adverse effects of a known confounder can be mitigated though designing a study that uses statistical methods to adjust for the confounding issue.  An unknown confounder, on the other hand is much more difficult to address by the shear nature of it being unknown. With an unknown confounder, there is always a risk that any association between a risk, intervention or outcome is being mediated through the unknown confounder (collemergencymed).

Importance of Clinical Study Design in Mitigating the Affects of Bias and Confounding Variables

Randomization is the best defense against confounding variables and bias (collemergencymed) (Farb, 2010). It accomplishes this by removing the potential bias in allocation of study subjects to different interventions, ensures study groups are compatible, can account for both known and unknown confounders, and ensure the validity of the statistics used to interpret the results.

Blinding or masking is also a tool used to reduce bias.  Studies that are single-blinded allow the investigator to know which patient received specific treatments, while the patient is unaware of which treatment they received (Kahan, 2009). In a double-blind study, both the Investigator and the subject are not provided with the treatment information. A modified double-blind study is in which the Investigator is not blinded, but the patient and observer who is responsible for assessing safety and efficacy are blinded to which patient received a particular treatment (Kahan, 2009).

Non-randomized trials, on the other hand, allow for opportunity of possible bias, provide data that may be inaccurate and may lead to misinterpretation of the data (Farb, 2010).

A cohort study is observational and not randomized; instead, treatment is decided by the patient’s provider. The study measures the same characteristics in patients. Patient groups may vary in their makeup by one characteristic for example, invasive procedure or noninvasive procedure. The study may use various multivariate analyses in order to correct for any possible confounding factors. The validity of the data is subject to selection bias (Sullivan & al, 2008).

Review articles provide much relevant background information on the device, but lack the clinical details required to critically evaluate the quality of the data presented. As a result, the data may not be as strong as the data presented in a randomized or blinded trial.

Published case series and case reports should be included, as the adverse events cited in these instances can help add valuable information that may not have been caught in the clinical studies. They can report rare events and even help identify early trends. However, it is not advised that the bulk of your evidence be comprised of case reports and case studies because these are single instances of adverse events and should not be used alone to evaluate overall performance and safety of the device in question.

This list of study designs is not exhaustive but covers most basic designs

Figure 1 provides a ranking of study designs types and their ability to provide qualified, accurate and effective evidence.

Figure 1: Ranking of Clinical Evidence Types

Weighing the Evidence

MedDev 2.7.1 Rev 3; Guidelines on Medical Devices Clinical Evaluation: A Guide for Manufacturers and Notified Bodies address the quality of articles in Appendix D.  The guidance provides a good example of the weighing Schedule D1. Schedule D1 takes into account the device, the intended use, the intended population and the quality of the data.  The Guidance further addressed evaluating the quality of the date with Schedule D2.  It’s important to note that the very first criterion on the D2 schedule addresses the issue of appropriate study design (European Commission, 2009).

 Upcoming Changes

The European Commission is currently in the process of ratifying the new Medical Device and In vitro Diagnostics Regulatory Framework (European Commission, 2013). These regulations will replace the directives and guidance that are currently in place. The draft regulations indicate that even more medical devices and some non-medical devices will require a clinical literature evaluation and that even more scrutiny will be placed on this process. Therefore, it’s imperative that the clinical literature evidence you provide in your evaluation consists of the most relevant, qualified and accurate data possible.

Bibliography

Collemergencymed. (n.d.). Bias and confounding. Retrieved March 5, 2013, from collemergencymed.ac.uk: http://www.collemergencymed.ac.uk/CEM/Research/technical_guide/biasconfound.htm

Farb, A. (2010, Sept 30). When is a randomized clinical trial appropriate vs a historical control vs. a performance goal. Retrieved March 5, 2013, from FDA.gov: http://www.fda.gov/downloads/MedicalDevices/NewsEvents/WorkshopsConferences/UCM240786.pdf

Kahan, J. S. (2009). Medical Device Develpment Regulation and Law; Chapter 6. PAREXEL.

May, G. a. (1981). The Randomized clinical trial: Bias in analysis. Circulation Vol 64 , 669-673.

Sullivan, B. M., & al, e. (2008). Overview of Study Designs in Clinical Research. Retrieved Marh 5, 2013, from nuh.edu: http://www.nuhs.edu/media/4133/5.1bAppraiseOverviewofStudyDesignsLecture.pdf