1
|
Kaizar E, Lin CY, Faries D, Johnston J. Reweighting estimators to extend the external validity of clinical trials: methodological considerations. J Biopharm Stat 2023; 33:515-543. [PMID: 36688658 DOI: 10.1080/10543406.2022.2162067] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2022] [Accepted: 12/10/2022] [Indexed: 01/24/2023]
Abstract
Methods to extend the strong internal validity of randomized controlled trials to reliably estimate treatment effects in target populations are gaining attention. This paper enumerates steps recommended for undertaking such extended inference, discusses currently viable choices for each one, and provides recommendations. We demonstrate a complete extended inference from a clinical trial studying a pharmaceutical treatment for Alzheimer's disease (AD) to a realistic target population of European residents diagnosed with AD. This case study highlights approaches to overcoming practical difficulties and demonstrates limitations of reliably extending inference from a trial to a real-world population.
Collapse
Affiliation(s)
- Eloise Kaizar
- Department of Statistics, Ohio State University, Columbus, Ohio, USA
| | - Chen-Yen Lin
- FSP Biometrics, Syneos Health, Toronto, Ontario, Canada
| | - Douglas Faries
- Real World Analytics, Eli Lilly and Company, Indianapolis, Indiana, USA
| | - Joseph Johnston
- Value, Evidence, and Outcomes, Eli Lilly and Company, Indianapolis, Indiana, USA
| |
Collapse
|
2
|
Chen J, Xiang Y, Li L, Xu A, Hu W, Lin Z, Xu F, Lin D, Chen W, Lin H. Application of Surgical Decision Model for Patients With Childhood Cataract: A Study Based on Real World Data. Front Bioeng Biotechnol 2021; 9:657866. [PMID: 34513804 PMCID: PMC8427305 DOI: 10.3389/fbioe.2021.657866] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/24/2021] [Accepted: 05/04/2021] [Indexed: 11/13/2022] Open
Abstract
Reliable validated methods are necessary to verify the performance of diagnosis and therapy-assisted models in clinical practice. However, some validated results have research bias and may not reflect the results of real-world application. In addition, the conduct of clinical trials has executive risks for the indeterminate effectiveness of models and it is challenging to finish validated clinical trials of rare diseases. Real world data (RWD) can probably solve this problem. In our study, we collected RWD from 251 patients with a rare disease, childhood cataract (CC) and conducted a retrospective study to validate the CC surgical decision model. The consistency of the real surgical type and recommended surgical type was 94.16%. In the cataract extraction (CE) group, the model recommended the same surgical type for 84.48% of eyes, but the model advised conducting cataract extraction and primary intraocular lens implantation (CE + IOL) surgery in 15.52% of eyes, which was different from the real-world choices. In the CE + IOL group, the model recommended the same surgical type for 100% of eyes. The real-recommended matched rates were 94.22% in the eyes of bilateral patients and 90.38% in the eyes of unilateral patients. Our study is the first to apply RWD to complete a retrospective study evaluating a clinical model, and the results indicate the availability and feasibility of applying RWD in model validation and serve guidance for intelligent model evaluation for rare diseases.
Collapse
Affiliation(s)
- Jingjing Chen
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, China
| | - Yifan Xiang
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, China
| | - Longhui Li
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, China
| | - Andi Xu
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, China
| | - Weiling Hu
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, China
| | - Zhuoling Lin
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, China
| | - Fabao Xu
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, China
| | - Duoru Lin
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, China
| | - Weirong Chen
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, China
| | - Haotian Lin
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, China.,Center of Precision Medicine, Sun Yat-sen University, Guangzhou, China
| |
Collapse
|
3
|
Rogers JR, Lee J, Zhou Z, Cheung YK, Hripcsak G, Weng C. Contemporary use of real-world data for clinical trial conduct in the United States: a scoping review. J Am Med Inform Assoc 2021; 28:144-154. [PMID: 33164065 DOI: 10.1093/jamia/ocaa224] [Citation(s) in RCA: 24] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/05/2020] [Revised: 08/11/2020] [Accepted: 09/02/2020] [Indexed: 12/28/2022] Open
Abstract
OBJECTIVE Real-world data (RWD), defined as routinely collected healthcare data, can be a potential catalyst for addressing challenges faced in clinical trials. We performed a scoping review of database-specific RWD applications within clinical trial contexts, synthesizing prominent uses and themes. MATERIALS AND METHODS Querying 3 biomedical literature databases, research articles using electronic health records, administrative claims databases, or clinical registries either within a clinical trial or in tandem with methodology related to clinical trials were included. Articles were required to use at least 1 US RWD source. All abstract screening, full-text screening, and data extraction was performed by 1 reviewer. Two reviewers independently verified all decisions. RESULTS Of 2020 screened articles, 89 qualified: 59 articles used electronic health records, 29 used administrative claims, and 26 used registries. Our synthesis was driven by the general life cycle of a clinical trial, culminating into 3 major themes: trial process tasks (51 articles); dissemination strategies (6); and generalizability assessments (34). Despite a diverse set of diseases studied, <10% of trials using RWD for trial process tasks evaluated medications or procedures (5/51). All articles highlighted data-related challenges, such as missing values. DISCUSSION Database-specific RWD have been occasionally leveraged for various clinical trial tasks. We observed underuse of RWD within conducted medication or procedure trials, though it is subject to the confounder of implicit report of RWD use. CONCLUSION Enhanced incorporation of RWD should be further explored for medication or procedure trials, including better understanding of how to handle related data quality issues to facilitate RWD use.
Collapse
Affiliation(s)
- James R Rogers
- Department of Biomedical Informatics, Columbia University, New York, New York, USA
| | - Junghwan Lee
- Department of Biomedical Informatics, Columbia University, New York, New York, USA
| | - Ziheng Zhou
- Institute of Human Nutrition, Columbia University, New York, New York, USA
| | - Ying Kuen Cheung
- Department of Biostatistics, Columbia University, New York, New York, USA, and
| | - George Hripcsak
- Department of Biomedical Informatics, Columbia University, New York, New York, USA.,Medical Informatics Services, New York-Presbyterian Hospital, New York, New York, USA
| | - Chunhua Weng
- Department of Biomedical Informatics, Columbia University, New York, New York, USA
| |
Collapse
|
4
|
McComb M, Bies R, Ramanathan M. Machine learning in pharmacometrics: Opportunities and challenges. Br J Clin Pharmacol 2021; 88:1482-1499. [PMID: 33634893 DOI: 10.1111/bcp.14801] [Citation(s) in RCA: 41] [Impact Index Per Article: 13.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2020] [Revised: 02/08/2021] [Accepted: 02/12/2021] [Indexed: 12/13/2022] Open
Abstract
The explosive growth in medical devices, imaging and diagnostics, computing, and communication and information technologies in drug development and healthcare has created an ever-expanding data landscape that the pharmacometrics (PMX) research community must now traverse. The tools of machine learning (ML) have emerged as a powerful computational approach in other data-rich disciplines but its effective utilization in the pharmaceutical sciences and PMX modelling is in its infancy. ML-based methods can complement PMX modelling by enabling the information in diverse sources of big data, e.g. population-based public databases and disease-specific clinical registries, to be harnessed because they are capable of efficiently identifying salient variables associated with outcomes and delineating their interdependencies. ML algorithms are computationally efficient, have strong predictive capabilities and can enable learning in the big data setting. ML algorithms can be viewed as providing a computational bridge from big data to complement PMX modelling. This review provides an overview of the strengths and weaknesses of ML approaches vis-à-vis population methods, assesses current research into ML applications in the pharmaceutical sciences and provides perspective for potential opportunities and strategies for the successful integration and utilization of ML in PMX.
Collapse
Affiliation(s)
- Mason McComb
- Department of Pharmaceutical Sciences, University at Buffalo, University at Buffalo, State University of New York, Buffalo, NY, USA
| | - Robert Bies
- Department of Pharmaceutical Sciences, University at Buffalo, University at Buffalo, State University of New York, Buffalo, NY, USA.,Institute for Computational Data Science, University at Buffalo, NY, USA
| | - Murali Ramanathan
- Department of Pharmaceutical Sciences, University at Buffalo, University at Buffalo, State University of New York, Buffalo, NY, USA.,Department of Neurology, University at Buffalo, State University of New York, Buffalo, NY, USA
| |
Collapse
|
5
|
Goldstein BA. Five analytic challenges in working with electronic health records data to support clinical trials with some solutions. Clin Trials 2020; 17:370-376. [DOI: 10.1177/1740774520931211] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
Abstract
Electronic health records data are becoming a key data resource in clinical research. Owing to issues of data efficiency, electronic health records data are being used for clinical trials. This includes both large-scale pragmatic trails and smaller—more focused—point-of-care trials. While electronic health records data open up a number of scientific opportunities, they also present a number of analytic challenges. This article discusses five particular challenges related to organizing electronic health records data for analytic purposes. These are as follows: (1) data are not organized for research purposes, (2) data are both densely and irregularly observed, (3) we don’t have all data elements we may want or need, (4) data are both cross-sectional and longitudinal, and (5) data may be informatively observed. While laying out these challenges, the article notes how many of these challenges can be addressed by careful and thoughtful study design as well as by integration of clinicians and informaticians into the analytic team.
Collapse
|
6
|
Zaccardi F, Davies MJ, Khunti K. The present and future scope of real-world evidence research in diabetes: What questions can and cannot be answered and what might be possible in the future? Diabetes Obes Metab 2020; 22 Suppl 3:21-34. [PMID: 32250528 DOI: 10.1111/dom.13929] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/30/2019] [Revised: 11/18/2019] [Accepted: 11/18/2019] [Indexed: 12/16/2022]
Abstract
The last decade has witnessed an exponential growth in the opportunities to collect and link health-related data from multiple resources, including primary care, administrative, and device data. The availability of these "real-world," "big data" has fuelled also an intense methodological research into methods to handle them and extract actionable information. In medicine, the evidence generated from "real-world data" (RWD), which are not purposely collected to answer biomedical questions, is commonly termed "real-world evidence" (RWE). In this review, we focus on RWD and RWE in the area of diabetes research, highlighting their contributions in the last decade; and give some suggestions for future RWE diabetes research, by applying well-established and less-known tools to direct RWE diabetes research towards better personalized approaches to diabetes care. We underline the essential aspects to consider when using RWD and the key features limiting the translational potential of RWD in generating high-quality and applicable RWE. Only if viewed in the context of other study designs and statistical methods, with its pros and cons carefully considered, RWE will exploit its full potential as a complementary or even, in some cases, substitutive source of evidence compared to the expensive evidence obtained from randomized controlled trials.
Collapse
Affiliation(s)
- Francesco Zaccardi
- Diabetes Research Centre, Leicester Diabetes Centre, Leicester, UK
- Leicester Real World Evidence Unit, Leicester Diabetes Centre, Leicester, UK
| | - Melanie J Davies
- Diabetes Research Centre, Leicester Diabetes Centre, Leicester, UK
| | - Kamlesh Khunti
- Diabetes Research Centre, Leicester Diabetes Centre, Leicester, UK
- Leicester Real World Evidence Unit, Leicester Diabetes Centre, Leicester, UK
| |
Collapse
|