If sponsors want to apply the same scientific rigor to the evaluation of medical devices as applied to pharmaceutical or biotech agents, they should consider independent endpoint adjudication. Independent endpoint adjudication committees support the integrity of medical device trials. Independent and consistent adjudication of events using uniformly applied endpoint definitions and reporting processes mitigates variability and bias.
Endpoint adjudication committees (EACs)—also called clinical endpoint committees (CECs)—are increasingly commonplace in pharma trials but remain a bit of a novelty in device trials. EACs include independent experts who have the relevant therapeutic-area expertise and understand the clinical trial process in general and, in particular, the study protocol.
They review and classify efficacy and/or safety endpoints, providing standardized endpoint outcomes for statistical analysis. Done well, endpoint adjudication enhances the quality, validity and integrity of study results, primarily by reducing risk of bias and variation. Independent adjudication is essential in studies with complex or subjective clinical endpoints that require interpretation of data, especially when endpoints of interest lie outside an investigator’s therapeutic specialty.
Sponsors are increasingly adding EACs to their pharma and biopharma trials. But until recently, the same hasn’t been true for medical device trials.
And yet, these are trials that would benefit greatly from them. In general, device studies are often smaller; smaller studies generally have fewer endpoints to reach a significant finding. That means each error can have a profound effect on the study.
Standardization and Variation
By relying solely on investigators to interpret outcome events in a clinical trial, there is risk of introducing a significant degree of variability into the results. Each site may have its own definition of a clinical event which results in variability on conclusions drawn from the data.
Independent adjudication committees can improve the accuracy and precision of any subjective assessment. They also help mitigate variability in endpoint definitions associated with lengthier and more global trials.
Precision and Consistency
Let’s borrow an analogy from WCG Chief Scientific Officer Jonathan H. Seltzer, MD, MA, MBA: We’ve all been in situations—in the family, classroom or at work—where several people appear to be ill with the same thing. One person’s doctor says it’s a cold. Another person’s doctor calls it the flu. One calls it a viral syndrome while yet another attributes it to allergies. Yet everyone has the same symptoms.
This can certainly happen in medical device trials: Different investigators have different definitions for the same event.
An independent endpoint adjudication committee creates standard definitions for endpoints. So, returning to that cold/flu/allergy example: The EAC develops precise definitions for “cold” and “flu.” Then they look at the trial data to classify patients very specifically. This brings precision to the study endpoints, making regulators happy, improving quality of data and potentially reducing study length.
The risk for variability could be higher in device trials than in pharma or biopharma ones. There’s a real learning curve for practitioners who are using new devices. It’s not as straightforward as “take a pill or don’t take a pill.”
We also know that external factors may influence the interpretation or reporting of clinical endpoint events. For example, coding may be influenced by reimbursement incentives or local practice variation.1
Bias, however, poses the greatest challenge to standardized results.
Bias, Real and Perceived
One of the most significant differences between pharmaceutical and medical device trials is also the most compelling argument for an endpoint adjudication committee: the inability to blind the patient or the practitioner. Class III (high risk) and some Class II (intermediate risk) devices require clinical trials but, unlike drug trials, device studies are rarely blinded. A practitioner may believe in a device and, consciously or unconsciously, have a bias in its favor: They may believe it’s going to work—or not.
An investigator may be an enthusiast for the new therapy; and this may influence the interpretation of an event and how it relates to the device. Therefore, the interpretation of events at the site level might be prone to bias. For example, sites may underreport events associated with the device. The potential for bias—real or perceived—increases if the investigator has a financial or scientific relationship with the device manufacturer or a competitor.
A related concern is that an investigator might interpret events erroneously due to their direct involvement in the patient’s care, particularly relating to a complication or inadequate care. And then there’s the fear of liability or “responsibility,” which can introduce bias in assessment of endpoints. That risk is already a major issue for any surgeon and it’s likely to color their opinion of the medical device being investigated.
Aside from the bias itself is the appearance of bias, from the investigator to the sponsor. No one can ignore the inherent conflict of interest between taking difficult—and often costly—actions to do what is right for a trial and taking the actions that will propel the device or therapy forward to the next milestone. Sponsors need an arms-length relationship with the EAC members.
For medical device trials, an independent endpoint adjudication committee, looking at the blinded data, can bring rigor to the process and mitigate investigator bias.
Industry and Regulatory Consensus
We’re seeing growing consensus around the value of endpoint adjudication committees for device studies.
A March 2016 gathering on “The Role of Endpoint Adjudication in Medical Device Clinical Trials,” hosted by the U.S. FDA, the Cardiac Safety Research Consortium and MDEpiNet recommended that the majority of medical device studies be adjudicated. Dr. Seltzer chaired the meeting.
The resulting whitepaper, “Use of endpoint adjudication to improve the quality and validity of endpoint assessment for medical device development and post marketing evaluation: Rationale and best practices. A report from the cardiac safety research consortium,” provides a summary of the meeting and its recommendations.2
Moreover, although the U.S. FDA and the European Medicines Agency have yet to require endpoint adjudication committees for devices, both endorse them and include them in guidance. We believe this indicates endpoint adjudication will play an increasingly important role in approval. It will soon be expected, perhaps even required.
Streamlining the Process
Sponsors are often not equipped to create these committees as they may not have the expertise in adjudication or the resources to build the technology, create the charter, find the experts and identify the endpoints to monitor. Most medical device companies are smaller and lack the internal people, processes and technology to do this effectively.
And again, we have the question of objectivity. The sponsor and the CRO are in the midst of the clinical trial; they may not be seeing the signals the same way an outside expert would.
Independent adjudication of clinical outcomes limits bias, be it from the investigator, provider, sponsor or CRO. The use of a rigorously defined adjudication process helps ensure high-quality study results that regulators are more inclined to look upon favorably.
It all comes down to scientific rigor. And that doesn’t end with the trial.
Real-world data
While EACs typically focus on prospective review of clinical trial data, sponsors are gaining valuable, competitive advantage producing insights from the retrospective review of data from select trials as well.
FDA released guidance in 2017 on real-world evidence to support regulatory decision-making for medical devices.3 As regulators look for more real-world evidence, the use of post-market studies to provide data on safety and effectiveness for newly approved devices will become more common—even expected. And post-market data from registries and other sources likely need independent adjudication to ensure the accurate assessment of clinical outcomes.4
Thoughtful deployment of endpoint adjudication committees in the post-marketing phase has enabled companies to organize, define and gain insightful expertise across the lifecycle of a medical device, to commercialization and beyond.
References
1. Seltzer JH, et al. “Use of endpoint adjudication to improve the quality and validity of endpoint assessment for medical device development and post marketing evaluation: Rationale and best practices. A report from the cardiac safety research consortium.” Am Heart J. 2017 Aug;190:76-85.
2. Seltzer JH, et al., op. cit.
3. FDA Guidance “Use of Real-World Evidence to Support Regulatory Decision-Making for Medical Devices,” August 2017