1
|
Fusaroli M, Raschi E, Poluzzi E, Hauben M. The evolving role of disproportionality analysis in pharmacovigilance. Expert Opin Drug Saf 2024; 23:981-994. [PMID: 38913869 DOI: 10.1080/14740338.2024.2368817] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2024] [Accepted: 06/12/2024] [Indexed: 06/26/2024]
Abstract
INTRODUCTION From 2009 to 2015, the IMI PROTECT conducted rigorous studies addressing questions about optimal implementation and significance of disproportionality analyses, leading to the development of Good Signal Detection Practices. The ensuing period witnessed the independent exploration of research paths proposed by IMI PROTECT, accumulating valuable experience and insights that have yet to be seamlessly integrated. AREAS COVERED This state-of-the-art review integrates IMI PROTECT recommendations with recent acquisitions and evolving challenges. It deals with defining the object of study, disproportionality methods, subgrouping, masking, drug-drug interaction, duplication, expectedness, the debated use of disproportionality results as risk measures, integration with other types of data. EXPERT OPINION Despite the ongoing skepticism regarding the usefulness of disproportionality analyses and individual case safety reports, their ability to timely detect safety signals regarding rare and unpredictable adverse reactions remains unparalleled. Moreover, recent exploration into their potential for characterizing safety signals revealed valuable insights concerning potential risk factors and the patient's perspective. To fully realize their potential beyond hypothesis generation and achieve a comprehensive evidence synthesis with other kinds of data and studies, each with their unique limitations and contributions, we need to investigate methods for more transparently communicating disproportionality results and mapping and addressing pharmacovigilance biases.
Collapse
Affiliation(s)
- Michele Fusaroli
- Pharmacology Unit, Department of Medical and Surgical Sciences, University of Bologna, Bologna, Italy
| | - Emanuel Raschi
- Pharmacology Unit, Department of Medical and Surgical Sciences, University of Bologna, Bologna, Italy
| | - Elisabetta Poluzzi
- Pharmacology Unit, Department of Medical and Surgical Sciences, University of Bologna, Bologna, Italy
| | - Manfred Hauben
- Department of Family and Community Medicine, New York Medical College, Valhalla, NY, USA
| |
Collapse
|
2
|
Matheny ME, Yang J, Smith JC, Walsh CG, Al-Garadi MA, Davis SE, Marsolo KA, Fabbri D, Reeves RR, Johnson KB, Dal Pan GJ, Ball R, Desai RJ. Enhancing Postmarketing Surveillance of Medical Products With Large Language Models. JAMA Netw Open 2024; 7:e2428276. [PMID: 39150707 DOI: 10.1001/jamanetworkopen.2024.28276] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 08/17/2024] Open
Abstract
Importance The Sentinel System is a key component of the US Food and Drug Administration (FDA) postmarketing safety surveillance commitment and uses clinical health care data to conduct analyses to inform drug labeling and safety communications, FDA advisory committee meetings, and other regulatory decisions. However, observational data are frequently deemed insufficient for reliable evaluation of safety concerns owing to limitations in underlying data or methodology. Advances in large language models (LLMs) provide new opportunities to address some of these limitations. However, careful consideration is necessary for how and where LLMs can be effectively deployed for these purposes. Observations LLMs may provide new avenues to support signal-identification activities to identify novel adverse event signals from narrative text of electronic health records. These algorithms may be used to support epidemiologic investigations examining the causal relationship between exposure to a medical product and an adverse event through development of probabilistic phenotyping of health outcomes of interest and extraction of information related to important confounding factors. LLMs may perform like traditional natural language processing tools by annotating text with controlled vocabularies with additional tailored training activities. LLMs offer opportunities for enhancing information extraction from adverse event reports, medical literature, and other biomedical knowledge sources. There are several challenges that must be considered when leveraging LLMs for postmarket surveillance. Prompt engineering is needed to ensure that LLM-extracted associations are accurate and specific. LLMs require extensive infrastructure to use, which many health care systems lack, and this can impact diversity, equity, and inclusion, and result in obscuring significant adverse event patterns in some populations. LLMs are known to generate nonfactual statements, which could lead to false positive signals and downstream evaluation activities by the FDA and other entities, incurring substantial cost. Conclusions and Relevance LLMs represent a novel paradigm that may facilitate generation of information to support medical product postmarket surveillance activities that have not been possible. However, additional work is required to ensure LLMs can be used in a fair and equitable manner, minimize false positive findings, and support the necessary rigor of signal detection needed for regulatory activities.
Collapse
Affiliation(s)
- Michael E Matheny
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee
- Department of Medicine, Vanderbilt University Medical Center, Nashville, Tennessee
- Department of Biostatistics, Vanderbilt University Medical Center, Nashville, Tennessee
- Geriatric Research Education and Clinical Care Service, Tennessee Valley Healthcare System VA, Nashville
| | - Jie Yang
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, Massachusetts
| | - Joshua C Smith
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee
| | - Colin G Walsh
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee
- Department of Medicine, Vanderbilt University Medical Center, Nashville, Tennessee
- Department of Psychiatry, Vanderbilt University Medical Center, Nashville, Tennessee
| | - Mohammed A Al-Garadi
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee
| | - Sharon E Davis
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee
| | - Keith A Marsolo
- Department of Population Health Sciences, Duke University School of Medicine, Durham, North Carolina
| | - Daniel Fabbri
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee
- Department of Computer Science, Vanderbilt University, Nashville, Tennessee
| | - Ruth R Reeves
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee
- Geriatric Research Education and Clinical Care Service, Tennessee Valley Healthcare System VA, Nashville
| | - Kevin B Johnson
- Department of Epidemiology and Informatics, University of Pennsylvania, Philadelphia
- Department of Pediatrics, University of Pennsylvania, Philadelphia
| | - Gerald J Dal Pan
- Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, Maryland
| | - Robert Ball
- Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, Maryland
| | - Rishi J Desai
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, Massachusetts
| |
Collapse
|
3
|
Fleurence RL, Kent S, Adamson B, Tcheng J, Balicer R, Ross JS, Haynes K, Muller P, Campbell J, Bouée-Benhamiche E, García Martí S, Ramsey S. Assessing Real-World Data From Electronic Health Records for Health Technology Assessment: The SUITABILITY Checklist: A Good Practices Report of an ISPOR Task Force. VALUE IN HEALTH : THE JOURNAL OF THE INTERNATIONAL SOCIETY FOR PHARMACOECONOMICS AND OUTCOMES RESEARCH 2024; 27:692-701. [PMID: 38871437 PMCID: PMC11182651 DOI: 10.1016/j.jval.2024.01.019] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/10/2024] [Accepted: 01/23/2024] [Indexed: 06/15/2024]
Abstract
This ISPOR Good Practices report provides a framework for assessing the suitability of electronic health records data for use in health technology assessments (HTAs). Although electronic health record (EHR) data can fill evidence gaps and improve decisions, several important limitations can affect its validity and relevance. The ISPOR framework includes 2 components: data delineation and data fitness for purpose. Data delineation provides a complete understanding of the data and an assessment of its trustworthiness by describing (1) data characteristics; (2) data provenance; and (3) data governance. Fitness for purpose comprises (1) data reliability items, ie, how accurate and complete the estimates are for answering the question at hand and (2) data relevance items, which assess how well the data are suited to answer the particular question from a decision-making perspective. The report includes a checklist specific to EHR data reporting: the ISPOR SUITABILITY Checklist. It also provides recommendations for HTA agencies and policy makers to improve the use of EHR-derived data over time. The report concludes with a discussion of limitations and future directions in the field, including the potential impact from the substantial and rapid advances in the diffusion and capabilities of large language models and generative artificial intelligence. The report's immediate audiences are HTA evidence developers and users. We anticipate that it will also be useful to other stakeholders, particularly regulators and manufacturers, in the future.
Collapse
Affiliation(s)
| | - Seamus Kent
- Erasmus School of Health & Policy Management, Erasmus University, Rotterdam, The Netherlands
| | | | | | | | - Joseph S Ross
- Yale School of Medicine, Yale University, New Haven, CT, USA
| | - Kevin Haynes
- Janssen Research and Development, Titusville, NJ, USA
| | - Patrick Muller
- Centre for Guidelines, National Institute for Health and Care Excellence, Manchester or London, England, UK
| | - Jon Campbell
- National Pharmaceutical Council, Washington, DC, USA
| | - Elsa Bouée-Benhamiche
- Public Health and Healthcare Division, Institut National du Cancer, Boulogne-Billancourt, France
| | - Sebastián García Martí
- Health Technology Assessment and Health Economics Department, Institute for Clinical Effectiveness and Health Policy, Buenos Aires, Argentina
| | - Scott Ramsey
- Hutchinson Institute for Cancer Outcomes Research, Fred Hutchinson Cancer Center, Seattle, WA, USA.
| |
Collapse
|
4
|
Weberpals J, Raman SR, Shaw PA, Lee H, Russo M, Hammill BG, Toh S, Connolly JG, Dandreo KJ, Tian F, Liu W, Li J, Hernández-Muñoz JJ, Glynn RJ, Desai RJ. A Principled Approach to Characterize and Analyze Partially Observed Confounder Data from Electronic Health Records. Clin Epidemiol 2024; 16:329-343. [PMID: 38798915 PMCID: PMC11127690 DOI: 10.2147/clep.s436131] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2023] [Accepted: 04/09/2024] [Indexed: 05/29/2024] Open
Abstract
Objective Partially observed confounder data pose challenges to the statistical analysis of electronic health records (EHR) and systematic assessments of potentially underlying missingness mechanisms are lacking. We aimed to provide a principled approach to empirically characterize missing data processes and investigate performance of analytic methods. Methods Three empirical sub-cohorts of diabetic SGLT2 or DPP4-inhibitor initiators with complete information on HbA1c, BMI and smoking as confounders of interest (COI) formed the basis of data simulation under a plasmode framework. A true null treatment effect, including the COI in the outcome generation model, and four missingness mechanisms for the COI were simulated: completely at random (MCAR), at random (MAR), and two not at random (MNAR) mechanisms, where missingness was dependent on an unmeasured confounder and on the value of the COI itself. We evaluated the ability of three groups of diagnostics to differentiate between mechanisms: 1)-differences in characteristics between patients with or without the observed COI (using averaged standardized mean differences [ASMD]), 2)-predictive ability of the missingness indicator based on observed covariates, and 3)-association of the missingness indicator with the outcome. We then compared analytic methods including "complete case", inverse probability weighting, single and multiple imputation in their ability to recover true treatment effects. Results The diagnostics successfully identified characteristic patterns of simulated missingness mechanisms. For MAR, but not MCAR, the patient characteristics showed substantial differences (median ASMD 0.20 vs 0.05) and consequently, discrimination of the prediction models for missingness was also higher (0.59 vs 0.50). For MNAR, but not MAR or MCAR, missingness was significantly associated with the outcome even in models adjusting for other observed covariates. Comparing analytic methods, multiple imputation using a random forest algorithm resulted in the lowest root-mean-squared-error. Conclusion Principled diagnostics provided reliable insights into missingness mechanisms. When assumptions allow, multiple imputation with nonparametric models could help reduce bias.
Collapse
Affiliation(s)
- Janick Weberpals
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA
| | - Sudha R Raman
- Department of Population Health Sciences, Duke University School of Medicine, Durham, NC, USA
| | - Pamela A Shaw
- Biostatistics Division, Kaiser Permanente Washington Health Research Institute, Seattle, WA, USA
| | - Hana Lee
- Office of Biostatistics, Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, MD, USA
| | - Massimiliano Russo
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA
| | - Bradley G Hammill
- Department of Population Health Sciences, Duke University School of Medicine, Durham, NC, USA
| | - Sengwee Toh
- Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, Boston, MA, USA
| | - John G Connolly
- Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, Boston, MA, USA
| | - Kimberly J Dandreo
- Department of Population Medicine, Harvard Pilgrim Health Care Institute, Boston, MA, USA
| | - Fang Tian
- Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, MD, USA
| | - Wei Liu
- Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, MD, USA
| | - Jie Li
- Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, MD, USA
| | - José J Hernández-Muñoz
- Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, MD, USA
| | - Robert J Glynn
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA
| | - Rishi J Desai
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA
| |
Collapse
|
5
|
Yamamoto K, Sakaguchi M, Onishi A, Yokoyama S, Matsui Y, Yamamoto W, Onizawa H, Fujii T, Murata K, Tanaka M, Hashimoto M, Matsuda S, Morinobu A. Energy landscape analysis and time-series clustering analysis of patient state multistability related to rheumatoid arthritis drug treatment: The KURAMA cohort study. PLoS One 2024; 19:e0302308. [PMID: 38709812 PMCID: PMC11073743 DOI: 10.1371/journal.pone.0302308] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/04/2023] [Accepted: 04/02/2024] [Indexed: 05/08/2024] Open
Abstract
Rheumatoid arthritis causes joint inflammation due to immune abnormalities, resulting in joint pain and swelling. In recent years, there have been considerable advancements in the treatment of this disease. However, only approximately 60% of patients achieve remission. Patients with multifactorial diseases shift between states from day to day. Patients may remain in a good or poor state with few or no transitions, or they may switch between states frequently. The visualization of time-dependent state transitions, based on the evaluation axis of stable/unstable states, may provide useful information for achieving rheumatoid arthritis treatment goals. Energy landscape analysis can be used to quantitatively determine the stability/instability of each state in terms of energy. Time-series clustering is another method used to classify transitions into different groups to identify potential patterns within a time-series dataset. The objective of this study was to utilize energy landscape analysis and time-series clustering to evaluate multidimensional time-series data in terms of multistability. We profiled each patient's state transitions during treatment using energy landscape analysis and time-series clustering. Energy landscape analysis divided state transitions into two patterns: "good stability leading to remission" and "poor stability leading to treatment dead-end." The number of patients whose disease status improved increased markedly until approximately 6 months after treatment initiation and then plateaued after 1 year. Time-series clustering grouped patients into three clusters: "toward good stability," "toward poor stability," and "unstable." Patients in the "unstable" cluster are considered to have clinical courses that are difficult to predict; therefore, these patients should be treated with more care. Early disease detection and treatment initiation are important. The evaluation of state multistability enables us to understand a patient's current state in the context of overall state transitions related to rheumatoid arthritis drug treatment and to predict future state transitions.
Collapse
Affiliation(s)
- Keiichi Yamamoto
- Division of Data Science, Center for Industrial Research and Innovation, Translational Research Institute for Medical Innovation, Osaka Dental University, Hirakata City, Osaka, Japan
| | - Masahiko Sakaguchi
- Department of Engineering Informatics, Faculty of Information and Communication Engineering, Osaka Electro-Communication University, Neyagawa City, Osaka, Japan
| | - Akira Onishi
- Department of Advanced Medicine for Rheumatic Diseases, Kyoto University Graduate School of Medicine, Sakyo, Kyoto, Japan
| | | | | | - Wataru Yamamoto
- Department of Advanced Medicine for Rheumatic Diseases, Kyoto University Graduate School of Medicine, Sakyo, Kyoto, Japan
- Department of Health Information Management, Kurashiki Sweet Hospital, Nakasho, Kurashiki, Kurashiki City, Okayama Prefecture, Japan
| | - Hideo Onizawa
- Department of Advanced Medicine for Rheumatic Diseases, Kyoto University Graduate School of Medicine, Sakyo, Kyoto, Japan
| | - Takayuki Fujii
- Department of Advanced Medicine for Rheumatic Diseases, Kyoto University Graduate School of Medicine, Sakyo, Kyoto, Japan
| | - Koichi Murata
- Department of Advanced Medicine for Rheumatic Diseases, Kyoto University Graduate School of Medicine, Sakyo, Kyoto, Japan
| | - Masao Tanaka
- Department of Advanced Medicine for Rheumatic Diseases, Kyoto University Graduate School of Medicine, Sakyo, Kyoto, Japan
| | - Motomu Hashimoto
- Department of Clinical Immunology, Osaka Metropolitan University Graduate School of Medicine, Osaka City, Japan
| | - Shuichi Matsuda
- Department of Advanced Medicine for Rheumatic Diseases, Kyoto University Graduate School of Medicine, Sakyo, Kyoto, Japan
| | - Akio Morinobu
- Department of Advanced Medicine for Rheumatic Diseases, Kyoto University Graduate School of Medicine, Sakyo, Kyoto, Japan
| |
Collapse
|
6
|
Weberpals J, Raman SR, Shaw PA, Lee H, Hammill BG, Toh S, Connolly JG, Dandreo KJ, Tian F, Liu W, Li J, Hernández-Muñoz JJ, Glynn RJ, Desai RJ. smdi: an R package to perform structural missing data investigations on partially observed confounders in real-world evidence studies. JAMIA Open 2024; 7:ooae008. [PMID: 38304248 PMCID: PMC10833461 DOI: 10.1093/jamiaopen/ooae008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/17/2023] [Revised: 01/09/2024] [Accepted: 01/16/2024] [Indexed: 02/03/2024] Open
Abstract
Objectives Partially observed confounder data pose a major challenge in statistical analyses aimed to inform causal inference using electronic health records (EHRs). While analytic approaches such as imputation are available, assumptions on underlying missingness patterns and mechanisms must be verified. We aimed to develop a toolkit to streamline missing data diagnostics to guide choice of analytic approaches based on meeting necessary assumptions. Materials and methods We developed the smdi (structural missing data investigations) R package based on results of a previous simulation study which considered structural assumptions of common missing data mechanisms in EHR. Results smdi enables users to run principled missing data investigations on partially observed confounders and implement functions to visualize, describe, and infer potential missingness patterns and mechanisms based on observed data. Conclusions The smdi R package is freely available on CRAN and can provide valuable insights into underlying missingness patterns and mechanisms and thereby help improve the robustness of real-world evidence studies.
Collapse
Affiliation(s)
- Janick Weberpals
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA 02120, United States
| | - Sudha R Raman
- Department of Population Health Sciences, Duke University School of Medicine, Durham, NC 27701, United States
| | - Pamela A Shaw
- Biostatistics Division, Kaiser Permanente Washington Health Research Institute, Seattle, WA 98101, United States
| | - Hana Lee
- Office of Biostatistics, Center for Drug Evaluation and Research, United States Food and Drug Administration, Silver Spring, MD 20993, United States
| | - Bradley G Hammill
- Department of Population Health Sciences, Duke University School of Medicine, Durham, NC 27701, United States
| | - Sengwee Toh
- Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, Boston, MA 02215, United States
| | - John G Connolly
- Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, Boston, MA 02215, United States
| | - Kimberly J Dandreo
- Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, Boston, MA 02215, United States
| | - Fang Tian
- Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, United States Food and Drug Administration, Silver Spring, MD 20993, United States
| | - Wei Liu
- Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, United States Food and Drug Administration, Silver Spring, MD 20993, United States
| | - Jie Li
- Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, United States Food and Drug Administration, Silver Spring, MD 20993, United States
| | - José J Hernández-Muñoz
- Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, United States Food and Drug Administration, Silver Spring, MD 20993, United States
| | - Robert J Glynn
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA 02120, United States
| | - Rishi J Desai
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA 02120, United States
| |
Collapse
|
7
|
Smith JC, Williamson BD, Cronkite DJ, Park D, Whitaker JM, McLemore MF, Osmanski JT, Winter R, Ramaprasan A, Kelley A, Shea M, Wittayanukorn S, Stojanovic D, Zhao Y, Toh S, Johnson KB, Aronoff DM, Carrell DS. Data-driven automated classification algorithms for acute health conditions: applying PheNorm to COVID-19 disease. J Am Med Inform Assoc 2024; 31:574-582. [PMID: 38109888 PMCID: PMC10873852 DOI: 10.1093/jamia/ocad241] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2023] [Revised: 10/19/2023] [Accepted: 11/27/2023] [Indexed: 12/20/2023] Open
Abstract
OBJECTIVES Automated phenotyping algorithms can reduce development time and operator dependence compared to manually developed algorithms. One such approach, PheNorm, has performed well for identifying chronic health conditions, but its performance for acute conditions is largely unknown. Herein, we implement and evaluate PheNorm applied to symptomatic COVID-19 disease to investigate its potential feasibility for rapid phenotyping of acute health conditions. MATERIALS AND METHODS PheNorm is a general-purpose automated approach to creating computable phenotype algorithms based on natural language processing, machine learning, and (low cost) silver-standard training labels. We applied PheNorm to cohorts of potential COVID-19 patients from 2 institutions and used gold-standard manual chart review data to investigate the impact on performance of alternative feature engineering options and implementing externally trained models without local retraining. RESULTS Models at each institution achieved AUC, sensitivity, and positive predictive value of 0.853, 0.879, 0.851 and 0.804, 0.976, and 0.885, respectively, at quantiles of model-predicted risk that maximize F1. We report performance metrics for all combinations of silver labels, feature engineering options, and models trained internally versus externally. DISCUSSION Phenotyping algorithms developed using PheNorm performed well at both institutions. Performance varied with different silver-standard labels and feature engineering options. Models developed locally at one site also worked well when implemented externally at the other site. CONCLUSION PheNorm models successfully identified an acute health condition, symptomatic COVID-19. The simplicity of the PheNorm approach allows it to be applied at multiple study sites with substantially reduced overhead compared to traditional approaches.
Collapse
Affiliation(s)
- Joshua C Smith
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN 37203, United States
| | - Brian D Williamson
- Kaiser Permanente Washington Health Research Institute, Seattle, WA 98101, United States
| | - David J Cronkite
- Kaiser Permanente Washington Health Research Institute, Seattle, WA 98101, United States
| | - Daniel Park
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN 37203, United States
| | - Jill M Whitaker
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN 37203, United States
| | - Michael F McLemore
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN 37203, United States
| | - Joshua T Osmanski
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN 37203, United States
| | - Robert Winter
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN 37203, United States
| | - Arvind Ramaprasan
- Kaiser Permanente Washington Health Research Institute, Seattle, WA 98101, United States
| | - Ann Kelley
- Kaiser Permanente Washington Health Research Institute, Seattle, WA 98101, United States
| | - Mary Shea
- Kaiser Permanente Washington Health Research Institute, Seattle, WA 98101, United States
| | - Saranrat Wittayanukorn
- Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, MD 20903, United States
| | - Danijela Stojanovic
- Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, MD 20903, United States
| | - Yueqin Zhao
- Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, MD 20903, United States
| | - Sengwee Toh
- Harvard Pilgrim Health Care Institute, Boston, MA 02215, United States
| | - Kevin B Johnson
- Department of Biostatistics, Epidemiology and Informatics, University of Pennsylvania, Philadelphia, PA 19104, United States
| | - David M Aronoff
- Department of Medicine, Indiana University School of Medicine, Indianapolis, IN 46202, United States
| | - David S Carrell
- Kaiser Permanente Washington Health Research Institute, Seattle, WA 98101, United States
| |
Collapse
|
8
|
Bey R, Cohen A, Trebossen V, Dura B, Geoffroy PA, Jean C, Landman B, Petit-Jean T, Chatellier G, Sallah K, Tannier X, Bourmaud A, Delorme R. Natural language processing of multi-hospital electronic health records for public health surveillance of suicidality. NPJ MENTAL HEALTH RESEARCH 2024; 3:6. [PMID: 38609541 PMCID: PMC10955903 DOI: 10.1038/s44184-023-00046-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/01/2023] [Accepted: 12/06/2023] [Indexed: 04/14/2024]
Abstract
There is an urgent need to monitor the mental health of large populations, especially during crises such as the COVID-19 pandemic, to timely identify the most at-risk subgroups and to design targeted prevention campaigns. We therefore developed and validated surveillance indicators related to suicidality: the monthly number of hospitalisations caused by suicide attempts and the prevalence among them of five known risks factors. They were automatically computed analysing the electronic health records of fifteen university hospitals of the Paris area, France, using natural language processing algorithms based on artificial intelligence. We evaluated the relevance of these indicators conducting a retrospective cohort study. Considering 2,911,920 records contained in a common data warehouse, we tested for changes after the pandemic outbreak in the slope of the monthly number of suicide attempts by conducting an interrupted time-series analysis. We segmented the assessment time in two sub-periods: before (August 1, 2017, to February 29, 2020) and during (March 1, 2020, to June 31, 2022) the COVID-19 pandemic. We detected 14,023 hospitalisations caused by suicide attempts. Their monthly number accelerated after the COVID-19 outbreak with an estimated trend variation reaching 3.7 (95%CI 2.1-5.3), mainly driven by an increase among girls aged 8-17 (trend variation 1.8, 95%CI 1.2-2.5). After the pandemic outbreak, acts of domestic, physical and sexual violence were more often reported (prevalence ratios: 1.3, 95%CI 1.16-1.48; 1.3, 95%CI 1.10-1.64 and 1.7, 95%CI 1.48-1.98), fewer patients died (p = 0.007) and stays were shorter (p < 0.001). Our study demonstrates that textual clinical data collected in multiple hospitals can be jointly analysed to compute timely indicators describing mental health conditions of populations. Our findings also highlight the need to better take into account the violence imposed on women, especially at early ages and in the aftermath of the COVID-19 pandemic.
Collapse
Affiliation(s)
- Romain Bey
- Innovation and Data unit, IT Department, Assistance Publique-Hôpitaux de Paris, Paris, France
| | - Ariel Cohen
- Innovation and Data unit, IT Department, Assistance Publique-Hôpitaux de Paris, Paris, France.
| | - Vincent Trebossen
- Child and Adolescent Psychiatry Department, Robert Debré University Hospital, Assistance Publique-Hôpitaux de Paris, Paris, France
| | - Basile Dura
- Innovation and Data unit, IT Department, Assistance Publique-Hôpitaux de Paris, Paris, France
| | - Pierre-Alexis Geoffroy
- Département de psychiatrie et d'addictologie, GHU Paris Nord, DMU neurosciences, Bichat - Claude Bernard Hospital, Assistance Publique-Hôpitaux de Paris, 75018, Paris, France
- GHU Paris - psychiatry & neurosciences, 1, rue Cabanis, 75014, Paris, France
- NeuroDiderot, Inserm, FHU I2-D2, université Paris Cité, 75019, Paris, France
- CNRS UPR 3212, Institute for cellular and integrative neurosciences, 67000, Strasbourg, France
| | - Charline Jean
- Innovation and Data unit, IT Department, Assistance Publique-Hôpitaux de Paris, Paris, France
- Université Paris-Est Créteil, INSERM, IMRB U955, Créteil, France
- Service Santé Publique & URC, Hôpital Henri Mondor, Assistance Publique-Hôpitaux de Paris, Créteil, France
| | - Benjamin Landman
- Child and Adolescent Psychiatry Department, Robert Debré University Hospital, Assistance Publique-Hôpitaux de Paris, Paris, France
| | - Thomas Petit-Jean
- Innovation and Data unit, IT Department, Assistance Publique-Hôpitaux de Paris, Paris, France
| | - Gilles Chatellier
- Innovation and Data unit, IT Department, Assistance Publique-Hôpitaux de Paris, Paris, France
- Université Paris Cité, Paris, France
| | - Kankoe Sallah
- URC PNVS, CIC-EC 1425, INSERM, Bichat - Claude Bernard Hospital, Assistance Publique-Hôpitaux de Paris, Paris, France
| | - Xavier Tannier
- Sorbonne Université, Inserm, Université Sorbonne Paris Nord, Laboratoire d'Informatique Médicale et d'Ingénierie des Connaissances pour la e-Santé (LIMICS), Paris, France
| | - Aurelie Bourmaud
- Université Paris Cité, Paris, France
- Clinical Epidemiology Unit, Robert Debré University Hospital, Assistance Publique-Hôpitaux de Paris, Paris, France
- CIC 1426, Inserm, Paris, France
| | - Richard Delorme
- Child and Adolescent Psychiatry Department, Robert Debré University Hospital, Assistance Publique-Hôpitaux de Paris, Paris, France
- Human Genetics and Cognitive Functions, Institut Pasteur, Paris, France
| |
Collapse
|
9
|
Desai RJ, Wang SV, Sreedhara SK, Zabotka L, Khosrow-Khavar F, Nelson JC, Shi X, Toh S, Wyss R, Patorno E, Dutcher S, Li J, Lee H, Ball R, Dal Pan G, Segal JB, Suissa S, Rothman KJ, Greenland S, Hernán MA, Heagerty PJ, Schneeweiss S. Process guide for inferential studies using healthcare data from routine clinical practice to evaluate causal effects of drugs (PRINCIPLED): considerations from the FDA Sentinel Innovation Center. BMJ 2024; 384:e076460. [PMID: 38346815 DOI: 10.1136/bmj-2023-076460] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/15/2024]
Affiliation(s)
- Rishi J Desai
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, MA 02120, USA
| | - Shirley V Wang
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, MA 02120, USA
| | - Sushama Kattinakere Sreedhara
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, MA 02120, USA
| | - Luke Zabotka
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, MA 02120, USA
| | - Farzin Khosrow-Khavar
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, MA 02120, USA
| | - Jennifer C Nelson
- Kaiser Permanente Washington Health Research Institute, Seattle, WA, USA
| | - Xu Shi
- Department of Biostatistics, University of Michigan, Ann Arbor, MI, USA
| | - Sengwee Toh
- Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, Boston, MA, USA
| | - Richard Wyss
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, MA 02120, USA
| | - Elisabetta Patorno
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, MA 02120, USA
| | - Sarah Dutcher
- US Food and Drug Administration, Silver Spring, MD, USA
| | - Jie Li
- US Food and Drug Administration, Silver Spring, MD, USA
| | - Hana Lee
- US Food and Drug Administration, Silver Spring, MD, USA
| | - Robert Ball
- US Food and Drug Administration, Silver Spring, MD, USA
| | | | - Jodi B Segal
- Department of Medicine, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Samy Suissa
- Departments of Epidemiology and Biostatistics, and Medicine, McGill University, Montreal, QC, Canada
| | | | - Sander Greenland
- Department of Epidemiology and Department of Statistics, University of California, Los Angeles, CA, USA
| | - Miguel A Hernán
- CAUSALab and Departments of Epidemiology and Biostatistics, Harvard T H Chan School of Public Health, Boston, MA, USA
| | | | - Sebastian Schneeweiss
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, MA 02120, USA
| |
Collapse
|
10
|
Ailawadhi S, Romanus D, Shah S, Fraeman K, Saragoussi D, Buus RM, Nguyen B, Cherepanov D, Lamerato L, Berger A. Development and validation of algorithms for identifying lines of therapy in multiple myeloma using real-world data. Future Oncol 2024. [PMID: 38231002 DOI: 10.2217/fon-2023-0696] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/18/2024] Open
Abstract
Aim: To validate algorithms based on electronic health data to identify composition of lines of therapy (LOT) in multiple myeloma (MM). Materials & methods: This study used available electronic health data for selected adults within Henry Ford Health (Michigan, USA) newly diagnosed with MM in 2006-2017. Algorithm performance in this population was verified via chart review. As with prior oncology studies, good performance was defined as positive predictive value (PPV) ≥75%. Results: Accuracy for identifying LOT1 (N = 133) was 85.0%. For the most frequent regimens, accuracy was 92.5-97.7%, PPV 80.6-93.8%, sensitivity 88.2-89.3% and specificity 94.3-99.1%. Algorithm performance decreased in subsequent LOTs, with decreasing sample sizes. Only 19.5% of patients received maintenance therapy during LOT1. Accuracy for identifying maintenance therapy was 85.7%; PPV for the most common maintenance therapy was 73.3%. Conclusion: Algorithms performed well in identifying LOT1 - especially more commonly used regimens - and slightly less well in identifying maintenance therapy therein.
Collapse
Affiliation(s)
- Sikander Ailawadhi
- Division of Hematology/Oncology, Department of Medicine, Mayo Clinic, Jacksonville, FL 32224, USA
| | - Dorothy Romanus
- Global Evidence & Outcomes, Takeda Development Center Americas, Inc. (TDCA), Lexington, MA 02421, USA
| | - Surbhi Shah
- Real-World Evidence, Evidera/PPD (part of Thermo fisher Scientific), Bethesda, MD 20814, USA
| | - Kathy Fraeman
- Real-World Evidence, Evidera/PPD (part of Thermo fisher Scientific), Bethesda, MD 20814, USA
| | - Delphine Saragoussi
- Real-World Evidence, Evidera/PPD (part of Thermo fisher Scientific), London, W6 8BJ, UK
| | - Rebecca Morris Buus
- Epidemiology and Scientific Affairs, Clinical Development Services Division, Evidera/PPD (part of Thermo Fisher Scientific), Bethesda, MD 20814, USA
| | - Binh Nguyen
- Medical Science and Strategy, Oncology, PPD (part of Thermo Fisher Scientific), Bethesda, MD 20814, USA
| | - Dasha Cherepanov
- Global Evidence & Outcomes, Takeda Development Center Americas, Inc. (TDCA), Lexington, MA 02421, USA
| | | | - Ariel Berger
- Real-World Evidence, Evidera/PPD (part of Thermo fisher Scientific), Bethesda, MD 20814, USA
| |
Collapse
|
11
|
Zang C, Zhang H, Xu J, Zhang H, Fouladvand S, Havaldar S, Cheng F, Chen K, Chen Y, Glicksberg BS, Chen J, Bian J, Wang F. High-throughput target trial emulation for Alzheimer's disease drug repurposing with real-world data. Nat Commun 2023; 14:8180. [PMID: 38081829 PMCID: PMC10713627 DOI: 10.1038/s41467-023-43929-1] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/13/2022] [Accepted: 11/24/2023] [Indexed: 12/18/2023] Open
Abstract
Target trial emulation is the process of mimicking target randomized trials using real-world data, where effective confounding control for unbiased treatment effect estimation remains a main challenge. Although various approaches have been proposed for this challenge, a systematic evaluation is still lacking. Here we emulated trials for thousands of medications from two large-scale real-world data warehouses, covering over 10 years of clinical records for over 170 million patients, aiming to identify new indications of approved drugs for Alzheimer's disease. We assessed different propensity score models under the inverse probability of treatment weighting framework and suggested a model selection strategy for improved baseline covariate balancing. We also found that the deep learning-based propensity score model did not necessarily outperform logistic regression-based methods in covariate balancing. Finally, we highlighted five top-ranked drugs (pantoprazole, gabapentin, atorvastatin, fluticasone, and omeprazole) originally intended for other indications with potential benefits for Alzheimer's patients.
Collapse
Affiliation(s)
- Chengxi Zang
- Department of Population Health Sciences, Weill Cornell Medicine, New York, NY, USA
- Institute of Artificial Intelligence for Digital Health, Weill Cornell Medicine, New York, NY, USA
| | - Hao Zhang
- Department of Population Health Sciences, Weill Cornell Medicine, New York, NY, USA
| | - Jie Xu
- Department of Health Outcomes & Biomedical Informatics, University of Florida, Gainesville, FL, USA
| | - Hansi Zhang
- Department of Health Outcomes & Biomedical Informatics, University of Florida, Gainesville, FL, USA
| | - Sajjad Fouladvand
- Institude for Biomedical Informatics (IBI) and Department of Computer Science, University of Kentucky, Lexington, KY, USA
| | - Shreyas Havaldar
- Hasso Plattner Institute for Digital Health at Mount Sinai, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | - Feixiong Cheng
- Genomic Medicine Institute, Lerner Research Institute, Cleveland Clinic, Cleveland, OH, USA
- Department of Molecular Medicine, Cleveland Clinic Lerner College of Medicine, Case Western Reserve University, Cleveland, OH, USA
- Case Comprehensive Cancer Center, Case Western Reserve University School of Medicine, Cleveland, OH, USA
| | - Kun Chen
- Department of Statistics, University of Connecticut, Storrs, CT, USA
| | - Yong Chen
- Department of Biostatistics, Epidemiology and Informatics (DBEI), the Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Benjamin S Glicksberg
- Hasso Plattner Institute for Digital Health at Mount Sinai, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | - Jin Chen
- Institude for Biomedical Informatics (IBI) and Department of Computer Science, University of Kentucky, Lexington, KY, USA
| | - Jiang Bian
- Department of Health Outcomes & Biomedical Informatics, University of Florida, Gainesville, FL, USA
| | - Fei Wang
- Department of Population Health Sciences, Weill Cornell Medicine, New York, NY, USA.
- Institute of Artificial Intelligence for Digital Health, Weill Cornell Medicine, New York, NY, USA.
| |
Collapse
|
12
|
Struja T, Matos J, Lam B, Cao Y, Liu X, Jia Y, Sauer CM, D'Couto H, Dankwa-Mullan I, Celi LA, Waschka AK. Evaluating equitable care in the ICU: Creating a causal inference framework to assess the impact of life-sustaining interventions across racial and ethnic groups. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2023:2023.10.12.23296933. [PMID: 37873267 PMCID: PMC10592988 DOI: 10.1101/2023.10.12.23296933] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/25/2023]
Abstract
Background Variability in the provision of intensive care unit (ICU)-interventions may lead to disparities between socially defined racial-ethnic groups. Research Question We used causal inference to examine the use of invasive mechanical ventilation (IMV), renal replacement therapy (RRT), and vasopressor agents (VP) to identify disparities in outcomes across race-ethnicity in patients with sepsis. Study Design and Methods Single-center, academic referral hospital in Boston, Massachusetts, USA. Retrospective analysis of treatment effect with a targeted trial design categorized by treatment assignment within the first 24 hours in the MIMIC-IV dataset (2008- 2019) using targeted maximum likelihood estimation. Of 76,943 ICU stays in MIMIC-IV, 32,971 adult stays fulfilling sepsis-3 criteria were included. The primary outcome was in-hospital mortality. Secondary outcomes were hospital-free days, and occurrence of nosocomial infection stratified by predicted mortality probability ranges and self-reported race-ethnicity. Average treatment effects by treatment type and race-ethnicity, Racial-ethnic group (REG) or White group (WG), were estimated. Results Of 19,419 admissions that met inclusion criteria, median age was 68 years, 57.4% were women, 82% were White, and mortality was 18.2%. There was no difference in mortality benefit associated with the administration of IMV, RRT, or VP between the REG and the WG. There was also no difference in hospital-free days or nosocomial infections. These findings are unchanged with different eligibility periods. Interpretation There were no differences in the treatment outcomes from three life-sustaining interventions in the ICU according to race-ethnicity. While there was no discernable harm from the treatments across mortality risk, there was also no measurable benefit. These findings highlight the need for research to understand better the risk-benefit of life-sustaining interventions in the ICU.
Collapse
|
13
|
Maro JC, Nguyen MD, Kolonoski J, Schoeplein R, Huang TY, Dutcher SK, Dal Pan GJ, Ball R. Six Years of the US Food and Drug Administration's Postmarket Active Risk Identification and Analysis System in the Sentinel Initiative: Implications for Real World Evidence Generation. Clin Pharmacol Ther 2023; 114:815-824. [PMID: 37391385 DOI: 10.1002/cpt.2979] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2023] [Accepted: 05/25/2023] [Indexed: 07/02/2023]
Abstract
Congress mandated the creation of a postmarket Active Risk Identification and Analysis (ARIA) system containing data on 100 million individuals for monitoring risks associated with drug and biologic products using data from disparate sources to complement the US Food and Drug Administration's (FDA's) existing postmarket capabilities. We report on the first 6 years of ARIA utilization in the Sentinel System (2016-2021). The FDA has used the ARIA system to evaluate 133 safety concerns; 54 of these evaluations have closed with regulatory determinations, whereas the rest remain in progress. If the ARIA system and the FDA's Adverse Event Reporting System are deemed insufficient to address a safety concern, then the FDA may issue a postmarket requirement to a product's manufacturer. One hundred ninety-seven ARIA insufficiency determinations have been made. The most common situation for which ARIA was found to be insufficient is the evaluation of adverse pregnancy and fetal outcomes following in utero drug exposure, followed by neoplasms and death. ARIA was most likely to be sufficient for thromboembolic events, which have high positive predictive value in claims data alone and do not require supplemental clinical data. The lessons learned from this experience illustrate the continued challenges using administrative claims data, especially to define novel clinical outcomes. This analysis can help to identify where more granular clinical data are needed to fill gaps to improve the use of real-world data for drug safety analyses and provide insights into what is needed to efficiently generate high-quality real-world evidence for efficacy.
Collapse
Affiliation(s)
- Judith C Maro
- Department of Population Medicine, Harvard Pilgrim Health Care Institute and Harvard Medical School, Boston, Massachusetts, USA
| | - Michael D Nguyen
- Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, Maryland, USA
| | - Joy Kolonoski
- Department of Population Medicine, Harvard Pilgrim Health Care Institute and Harvard Medical School, Boston, Massachusetts, USA
| | - Ryan Schoeplein
- Department of Population Medicine, Harvard Pilgrim Health Care Institute and Harvard Medical School, Boston, Massachusetts, USA
| | - Ting-Ying Huang
- Department of Population Medicine, Harvard Pilgrim Health Care Institute and Harvard Medical School, Boston, Massachusetts, USA
| | - Sarah K Dutcher
- Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, Maryland, USA
| | - Gerald J Dal Pan
- Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, Maryland, USA
| | - Robert Ball
- Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, Maryland, USA
| |
Collapse
|
14
|
Niazi SK. The Coming of Age of AI/ML in Drug Discovery, Development, Clinical Testing, and Manufacturing: The FDA Perspectives. Drug Des Devel Ther 2023; 17:2691-2725. [PMID: 37701048 PMCID: PMC10493153 DOI: 10.2147/dddt.s424991] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2023] [Accepted: 08/24/2023] [Indexed: 09/14/2023] Open
Abstract
Artificial intelligence (AI) and machine learning (ML) represent significant advancements in computing, building on technologies that humanity has developed over millions of years-from the abacus to quantum computers. These tools have reached a pivotal moment in their development. In 2021 alone, the U.S. Food and Drug Administration (FDA) received over 100 product registration submissions that heavily relied on AI/ML for applications such as monitoring and improving human performance in compiling dossiers. To ensure the safe and effective use of AI/ML in drug discovery and manufacturing, the FDA and numerous other U.S. federal agencies have issued continuously updated, stringent guidelines. Intriguingly, these guidelines are often generated or updated with the aid of AI/ML tools themselves. The overarching goal is to expedite drug discovery, enhance the safety profiles of existing drugs, introduce novel treatment modalities, and improve manufacturing compliance and robustness. Recent FDA publications offer an encouraging outlook on the potential of these tools, emphasizing the need for their careful deployment. This has expanded market opportunities for retraining personnel handling these technologies and enabled innovative applications in emerging therapies such as gene editing, CRISPR-Cas9, CAR-T cells, mRNA-based treatments, and personalized medicine. In summary, the maturation of AI/ML technologies is a testament to human ingenuity. Far from being autonomous entities, these are tools created by and for humans designed to solve complex problems now and in the future. This paper aims to present the status of these technologies, along with examples of their present and future applications.
Collapse
|
15
|
Antoon JW, Feinstein JA, Goldman JL, Kyler KE, Shah SS. Advancing pediatric medication safety using real-world data: Current problems and potential solutions. J Hosp Med 2023; 18:865-869. [PMID: 36855275 PMCID: PMC10460821 DOI: 10.1002/jhm.13068] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/31/2022] [Revised: 01/17/2023] [Accepted: 02/03/2023] [Indexed: 03/02/2023]
Affiliation(s)
- James W. Antoon
- Monroe Carell Jr. Children’s Hospital at Vanderbilt, Nashville, Tennessee, USA
- Department of Pediatrics, Division of Hospital Medicine, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - James A. Feinstein
- Adult and Child Consortium for Health Outcomes Research & Delivery Science, Children’s Hospital Colorado, University of Colorado, Aurora, Colorado, USA
| | - Jennifer L. Goldman
- Divisions of Infectious Diseases and Clinical Pharmacology, Children’s Mercy Hospitals and Clinics, Kansas City, Missouri, USA
| | - Kathryn E. Kyler
- Division of Hospital Medicine, Children’s Mercy Hospitals and Clinics, Kansas City, Missouri, USA
| | - Samir S. Shah
- Divisions of Hospital Medicine and Infectious Diseases, Cincinnati Children’s Hospital Medical Center & Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, Ohio, USA
| | | |
Collapse
|
16
|
Davis SE, Zabotka L, Desai RJ, Wang SV, Maro JC, Coughlin K, Hernández-Muñoz JJ, Stojanovic D, Shah NH, Smith JC. Use of Electronic Health Record Data for Drug Safety Signal Identification: A Scoping Review. Drug Saf 2023; 46:725-742. [PMID: 37340238 DOI: 10.1007/s40264-023-01325-0] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 05/31/2023] [Indexed: 06/22/2023]
Abstract
INTRODUCTION Pharmacovigilance programs protect patient health and safety by identifying adverse event signals through postmarketing surveillance of claims data and spontaneous reports. Electronic health records (EHRs) provide new opportunities to address limitations of traditional approaches and promote discovery-oriented pharmacovigilance. METHODS To evaluate the current state of EHR-based medication safety signal identification, we conducted a scoping literature review of studies aimed at identifying safety signals from routinely collected patient-level EHR data. We extracted information on study design, EHR data elements utilized, analytic methods employed, drugs and outcomes evaluated, and key statistical and data analysis choices. RESULTS We identified 81 eligible studies. Disproportionality methods were the predominant analytic approach, followed by data mining and regression. Variability in study design makes direct comparisons difficult. Studies varied widely in terms of data, confounding adjustment, and statistical considerations. CONCLUSION Despite broad interest in utilizing EHRs for safety signal identification, current efforts fail to leverage the full breadth and depth of available data or to rigorously control for confounding. The development of best practices and application of common data models would promote the expansion of EHR-based pharmacovigilance.
Collapse
Affiliation(s)
- Sharon E Davis
- Department of Biomedical Informatics, Vanderbilt University Medical Center, 2525 West End Ave, Suite 1475, Nashville, TN, 37203, USA
- Vanderbilt University School of Medicine, Nashville, TN, USA
| | | | - Rishi J Desai
- Brigham and Women's Hospital, Boston, MA, USA
- Harvard Medical School, Boston, MA, USA
| | - Shirley V Wang
- Brigham and Women's Hospital, Boston, MA, USA
- Harvard Medical School, Boston, MA, USA
| | - Judith C Maro
- Harvard Medical School, Boston, MA, USA
- Harvard Pilgrim Health Care Institute, Boston, MA, USA
| | | | | | | | - Nigam H Shah
- School of Medicine, Stanford University, Stanford, CA, USA
- Stanford Health Care, Palo Alto, CA, USA
| | - Joshua C Smith
- Department of Biomedical Informatics, Vanderbilt University Medical Center, 2525 West End Ave, Suite 1475, Nashville, TN, 37203, USA.
- Vanderbilt University School of Medicine, Nashville, TN, USA.
| |
Collapse
|
17
|
Dipietro L, Gonzalez-Mego P, Ramos-Estebanez C, Zukowski LH, Mikkilineni R, Rushmore RJ, Wagner T. The evolution of Big Data in neuroscience and neurology. JOURNAL OF BIG DATA 2023; 10:116. [PMID: 37441339 PMCID: PMC10333390 DOI: 10.1186/s40537-023-00751-2] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 01/28/2023] [Accepted: 05/08/2023] [Indexed: 07/15/2023]
Abstract
Neurological diseases are on the rise worldwide, leading to increased healthcare costs and diminished quality of life in patients. In recent years, Big Data has started to transform the fields of Neuroscience and Neurology. Scientists and clinicians are collaborating in global alliances, combining diverse datasets on a massive scale, and solving complex computational problems that demand the utilization of increasingly powerful computational resources. This Big Data revolution is opening new avenues for developing innovative treatments for neurological diseases. Our paper surveys Big Data's impact on neurological patient care, as exemplified through work done in a comprehensive selection of areas, including Connectomics, Alzheimer's Disease, Stroke, Depression, Parkinson's Disease, Pain, and Addiction (e.g., Opioid Use Disorder). We present an overview of research and the methodologies utilizing Big Data in each area, as well as their current limitations and technical challenges. Despite the potential benefits, the full potential of Big Data in these fields currently remains unrealized. We close with recommendations for future research aimed at optimizing the use of Big Data in Neuroscience and Neurology for improved patient outcomes. Supplementary Information The online version contains supplementary material available at 10.1186/s40537-023-00751-2.
Collapse
Affiliation(s)
| | - Paola Gonzalez-Mego
- Spaulding Rehabilitation/Neuromodulation Lab, Harvard Medical School, Cambridge, MA USA
| | | | | | | | | | - Timothy Wagner
- Highland Instruments, Cambridge, MA USA
- Harvard-MIT Division of Health Sciences and Technology, Cambridge, MA USA
| |
Collapse
|
18
|
Beetler DJ, Di Florio DN, Law EW, Groen CM, Windebank AJ, Peterson QP, Fairweather D. The evolving regulatory landscape in regenerative medicine. Mol Aspects Med 2023; 91:101138. [PMID: 36050142 PMCID: PMC10162454 DOI: 10.1016/j.mam.2022.101138] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2022] [Accepted: 08/12/2022] [Indexed: 01/17/2023]
Abstract
Regenerative medicine as a field has emerged as a new component of modern medicine and medical research that encompasses a wide range of products including cellular and acellular therapies. As this new field emerged, regulatory agencies like the Food and Drug Administration (FDA) rapidly adapted existing regulatory frameworks to address the transplantation, gene therapy, cell-based therapeutics, and acellular biologics that fall under the broader regenerative medicine umbrella. Where it has not been possible to modify existing regulation and processes, entirely new frameworks have been generated with the intention of providing flexible, forward-facing systems to regulate this rapidly growing field. This review discusses the current state of FDA regulatory affairs in the context of stem cells and extracellular vesicles by highlighting gaps in the current regulatory system and then discussing where regulatory science in regenerative medicine may be headed based on these gaps and the FDA's historical ability to deal with emerging fields. Lastly, we utilize case studies in stem cell and acellular based treatments to demonstrate how regulatory science has evolved in regenerative medicine and highlight the ongoing clinical efforts and challenges of these therapies.
Collapse
Affiliation(s)
- Danielle J Beetler
- Center for Clinical and Translational Science, Mayo Clinic, Rochester, MN, 55902, USA; Department of Cardiovascular Medicine, Mayo Clinic, Jacksonville, FL, 32224, USA
| | - Damian N Di Florio
- Center for Clinical and Translational Science, Mayo Clinic, Rochester, MN, 55902, USA; Department of Cardiovascular Medicine, Mayo Clinic, Jacksonville, FL, 32224, USA
| | - Ethan W Law
- Center for Clinical and Translational Science, Mayo Clinic, Rochester, MN, 55902, USA; Center for Regenerative Medicine, Mayo Clinic, Rochester, MN, 55902, USA
| | - Chris M Groen
- Department of Neurology, Mayo Clinic, Rochester, MN, 55902, USA
| | - Anthony J Windebank
- Center for Clinical and Translational Science, Mayo Clinic, Rochester, MN, 55902, USA; Department of Neurology, Mayo Clinic, Rochester, MN, 55902, USA
| | - Quinn P Peterson
- Center for Clinical and Translational Science, Mayo Clinic, Rochester, MN, 55902, USA; Center for Regenerative Medicine, Mayo Clinic, Rochester, MN, 55902, USA
| | - DeLisa Fairweather
- Center for Clinical and Translational Science, Mayo Clinic, Rochester, MN, 55902, USA; Department of Cardiovascular Medicine, Mayo Clinic, Jacksonville, FL, 32224, USA; Department of Environmental Health Sciences and Engineering, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, 21205, USA.
| |
Collapse
|
19
|
Mayo-Wilson E, Qureshi R, Li T. Conducting separate reviews of benefits and harms could improve systematic reviews and meta-analyses. Syst Rev 2023; 12:67. [PMID: 37061724 PMCID: PMC10105415 DOI: 10.1186/s13643-023-02234-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/02/2022] [Accepted: 04/10/2023] [Indexed: 04/17/2023] Open
Abstract
Guidance for systematic reviews of interventions recommends both benefits and harms be included. Systematic reviews may reach conclusions about harms (or lack of harms) that are not true when reviews include only some relevant studies, rely on incomplete data from eligible studies, use inappropriate methods for synthesizing data, and report results selectively. Separate reviews about harms could address some of these problems, and we argue that conducting separate reviews of harms is a feasible alternative to current standards and practices. Systematic reviews of potential benefits could be organized around the use of interventions for specific health problems. Systematic reviews of potential harms could be broader, including more diverse study designs and including all people at risk of harms (who might use the same intervention to treat different health problems). Multiple reviews about benefits could refer to a single review of harms. This approach could improve the reliability, completeness, and efficiency of systematic reviews.
Collapse
Affiliation(s)
- Evan Mayo-Wilson
- Department of Epidemiology, UNC Gillings School of Global Public Health, Chapel Hill, NC, 27599, USA.
| | - Riaz Qureshi
- Department of Ophthalmology, School of Medicine, University of Colorado Anschutz Medical Campus, Aurora, CO, USA
| | - Tianjing Li
- Department of Ophthalmology, School of Medicine, University of Colorado Anschutz Medical Campus, Aurora, CO, USA
| |
Collapse
|
20
|
Li Y, Tian Y, Pei S, Xie B, Xu X, Wang B. Worldwide Trends in Registering Real-World Studies at ClinicalTrials.gov: A Cross-Sectional Analysis. Int J Gen Med 2023; 16:1123-1136. [PMID: 37007912 PMCID: PMC10065426 DOI: 10.2147/ijgm.s402478] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2022] [Accepted: 03/22/2023] [Indexed: 03/29/2023] Open
Abstract
Objective The purpose of this study was to characterize real-world studies (RWSs) registered at ClinicalTrials.gov to help investigators better conduct relevant research in clinical practice. Methods A retrospective analysis of 944 studies was performed on February 28, 2023. Results A total of 944 studies were included. The included studies involved a total of 48 countries. China was the leading country in terms of the total number of registered studies (37.9%, 358), followed by the United States (19.7%, 186). Regarding intervention type, 42.4% (400) of the studies involved drugs, and only 9.1% (86) of the studies involved devices. Only 8.5% (80) of the studies mentioned both the detailed study design type and data source in the "Brief Summary". A total of 49.4% (466) of studies had a sample size of 500 participants and above. Overall, 63% (595) of the studies were single-center studies. A total of 213 conditions were covered in the included studies. One-third of the studies (32.7%, 309) involved neoplasms (or tumors). China and the United States were very different regarding the study of different conditions. Conclusion Although the pandemic has provided new opportunities for RWSs, the rigor of scientific research still needs to be emphasized. Special attention needs to be given to the correct and comprehensive description of the study design in the Brief Summary of registered studies, thereby promoting communication and understanding. In addition, deficiencies in ClinicalTrials.gov registration data remain prominent.
Collapse
Affiliation(s)
- Yuanxiao Li
- Department of Pediatric Gastroenterology, Lanzhou University Second Hospital, Lanzhou, People’s Republic of China
| | - Ying Tian
- Department of Clinical Medicine, Lanzhou University Second Hospital, Lanzhou, People’s Republic of China
| | - Shufen Pei
- Department of Clinical Medicine, North Sichuan Medical College, Nanchong, People’s Republic of China
| | - Baoyuan Xie
- Department of Clinical Medicine, Lanzhou University Second Hospital, Lanzhou, People’s Republic of China
| | - Xiaonan Xu
- Department of Pediatric Gastroenterology, Lanzhou University Second Hospital, Lanzhou, People’s Republic of China
| | - Bin Wang
- Department of Infectious Diseases, Second Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, People’s Republic of China
- Correspondence: Bin Wang, 88 Jie Fang Lu, Shangcheng District, Hangzhou, Zhejiang, 310009, People’s Republic of China, Email
| |
Collapse
|
21
|
Lovis C, Siebel J, Fuhrmann S, Fischer A, Sedlmayr M, Weidner J, Bathelt F. Assessment and Improvement of Drug Data Structuredness From Electronic Health Records: Algorithm Development and Validation. JMIR Med Inform 2023; 11:e40312. [PMID: 36696159 PMCID: PMC9909518 DOI: 10.2196/40312] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2022] [Revised: 09/27/2022] [Accepted: 11/18/2022] [Indexed: 11/19/2022] Open
Abstract
BACKGROUND Digitization offers a multitude of opportunities to gain insights into current diagnostics and therapies from retrospective data. In this context, real-world data and their accessibility are of increasing importance to support unbiased and reliable research on big data. However, routinely collected data are not readily usable for research owing to the unstructured nature of health care systems and a lack of interoperability between these systems. This challenge is evident in drug data. OBJECTIVE This study aimed to present an approach that identifies and increases the structuredness of drug data while ensuring standardization according to Anatomical Therapeutic Chemical (ATC) classification. METHODS Our approach was based on available drug prescriptions and a drug catalog and consisted of 4 steps. First, we performed an initial analysis of the structuredness of local drug data to define a point of comparison for the effectiveness of the overall approach. Second, we applied 3 algorithms to unstructured data that translated text into ATC codes based on string comparisons in terms of ingredients and product names and performed similarity comparisons based on Levenshtein distance. Third, we validated the results of the 3 algorithms with expert knowledge based on the 1000 most frequently used prescription texts. Fourth, we performed a final validation to determine the increased degree of structuredness. RESULTS Initially, 47.73% (n=843,980) of 1,768,153 drug prescriptions were classified as structured. With the application of the 3 algorithms, we were able to increase the degree of structuredness to 85.18% (n=1,506,059) based on the 1000 most frequent medication prescriptions. In this regard, the combination of algorithms 1, 2, and 3 resulted in a correctness level of 100% (with 57,264 ATC codes identified), algorithms 1 and 3 resulted in 99.6% (with 152,404 codes identified), and algorithms 1 and 2 resulted in 95.9% (with 39,472 codes identified). CONCLUSIONS As shown in the first analysis steps of our approach, the availability of a product catalog to select during the documentation process is not sufficient to generate structured data. Our 4-step approach reduces the problems and reliably increases the structuredness automatically. Similarity matching shows promising results, particularly for entries with no connection to a product catalog. However, further enhancement of the correctness of such a similarity matching algorithm needs to be investigated in future work.
Collapse
Affiliation(s)
| | - Joscha Siebel
- Institute for Medical Informatics and Biometry, Carl Gustav Carus Faculty of Medicine, Technische Universität Dresden, Dresden, Germany
| | - Saskia Fuhrmann
- Center for Evidence-Based Healthcare, Carl Gustav Carus Faculty of Medicine, Technische Universität Dresden, Dresden, Germany.,Hospital Pharmacy, University Hospital Carl Gustav Carus, Dresden, Germany
| | - Andreas Fischer
- Hospital Pharmacy, University Hospital Carl Gustav Carus, Dresden, Germany
| | - Martin Sedlmayr
- Center for Evidence-Based Healthcare, Carl Gustav Carus Faculty of Medicine, Technische Universität Dresden, Dresden, Germany
| | - Jens Weidner
- Institute for Medical Informatics and Biometry, Carl Gustav Carus Faculty of Medicine, Technische Universität Dresden, Dresden, Germany
| | - Franziska Bathelt
- Institute for Medical Informatics and Biometry, Carl Gustav Carus Faculty of Medicine, Technische Universität Dresden, Dresden, Germany
| |
Collapse
|
22
|
Hennessy S, Berlin JA. Real-World Trends in the Evaluation of Medical Products. Am J Epidemiol 2023; 192:1-5. [PMID: 36217921 DOI: 10.1093/aje/kwac172] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2022] [Revised: 08/15/2022] [Accepted: 08/26/2022] [Indexed: 01/11/2023] Open
Abstract
There is a compelling need to evaluate the real-world health effects of medical products outside of tightly controlled preapproval clinical trials. This is done through pharmacoepidemiology, which is the study of the health effects of medical products (including drugs, biologicals, and medical devices and diagnostics) in populations, often using nonrandomized designs. Recent developments in pharmacoepidemiology span changes in the focus of research questions, research designs, data used, and statistical analysis methods. Developments in these areas are thought to improve the value of the evidence produced by such studies, and are prompting greater use of real-world evidence to inform clinical, regulatory, and reimbursement decisions.
Collapse
|
23
|
Brown JS, Mendelsohn AB, Nam YH, Maro JC, Cocoros NM, Rodriguez-Watson C, Lockhart CM, Platt R, Ball R, Dal Pan GJ, Toh S. The US Food and Drug Administration Sentinel System: a national resource for a learning health system. J Am Med Inform Assoc 2022; 29:2191-2200. [PMID: 36094070 PMCID: PMC9667154 DOI: 10.1093/jamia/ocac153] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2022] [Revised: 06/18/2022] [Accepted: 08/18/2022] [Indexed: 07/23/2023] Open
Abstract
The US Food and Drug Administration (FDA) created the Sentinel System in response to a requirement in the FDA Amendments Act of 2007 that the agency establish a system for monitoring risks associated with drug and biologic products using data from disparate sources. The Sentinel System has completed hundreds of analyses, including many that have directly informed regulatory decisions. The Sentinel System also was designed to support a national infrastructure for a learning health system. Sentinel governance and guiding principles were designed to facilitate Sentinel's role as a national resource. The Sentinel System infrastructure now supports multiple non-FDA projects for stakeholders ranging from regulated industry to other federal agencies, international regulators, and academics. The Sentinel System is a working example of a learning health system that is expanding with the potential to create a global learning health system that can support medical product safety assessments and other research.
Collapse
Affiliation(s)
- Jeffrey S Brown
- Department of Population Medicine, Harvard Pilgrim Health Care Institute and Harvard Medical School, Boston, Massachusetts, USA
| | - Aaron B Mendelsohn
- Department of Population Medicine, Harvard Pilgrim Health Care Institute and Harvard Medical School, Boston, Massachusetts, USA
| | - Young Hee Nam
- Department of Population Medicine, Harvard Pilgrim Health Care Institute and Harvard Medical School, Boston, Massachusetts, USA
| | - Judith C Maro
- Department of Population Medicine, Harvard Pilgrim Health Care Institute and Harvard Medical School, Boston, Massachusetts, USA
| | - Noelle M Cocoros
- Department of Population Medicine, Harvard Pilgrim Health Care Institute and Harvard Medical School, Boston, Massachusetts, USA
| | - Carla Rodriguez-Watson
- Reagan-Udall Foundation for the Food and Drug Administration, Washington, District of Columbia, USA
| | - Catherine M Lockhart
- Biologics and Biosimilars Collective Intelligence Consortium, Alexandria, Virginia, USA
| | - Richard Platt
- Department of Population Medicine, Harvard Pilgrim Health Care Institute and Harvard Medical School, Boston, Massachusetts, USA
| | - Robert Ball
- Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, Food and Drug Administration, Silver Spring, Maryland, USA
| | - Gerald J Dal Pan
- Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, Food and Drug Administration, Silver Spring, Maryland, USA
| | - Sengwee Toh
- Corresponding Author: Sengwee Toh, ScD, Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, 401 Park Drive, Suite 401 East, Boston, MA 02215, USA;
| |
Collapse
|
24
|
Gonzalez-Hernandez G, Krallinger M, Muñoz M, Rodriguez-Esteban R, Uzuner Ö, Hirschman L. Challenges and opportunities for mining adverse drug reactions: perspectives from pharma, regulatory agencies, healthcare providers and consumers. Database (Oxford) 2022; 2022:6682867. [PMID: 36050787 PMCID: PMC9436770 DOI: 10.1093/database/baac071] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/13/2022] [Revised: 07/08/2022] [Accepted: 08/25/2022] [Indexed: 11/17/2022]
Abstract
Monitoring drug safety is a central concern throughout the drug life cycle. Information about toxicity and adverse events is generated at every stage of this life cycle, and stakeholders have a strong interest in applying text mining and artificial intelligence (AI) methods to manage the ever-increasing volume of this information. Recognizing the importance of these applications and the role of challenge evaluations to drive progress in text mining, the organizers of BioCreative VII (Critical Assessment of Information Extraction in Biology) convened a panel of experts to explore ‘Challenges in Mining Drug Adverse Reactions’. This article is an outgrowth of the panel; each panelist has highlighted specific text mining application(s), based on their research and their experiences in organizing text mining challenge evaluations. While these highlighted applications only sample the complexity of this problem space, they reveal both opportunities and challenges for text mining to aid in the complex process of drug discovery, testing, marketing and post-market surveillance. Stakeholders are eager to embrace natural language processing and AI tools to help in this process, provided that these tools can be demonstrated to add value to stakeholder workflows. This creates an opportunity for the BioCreative community to work in partnership with regulatory agencies, pharma and the text mining community to identify next steps for future challenge evaluations.
Collapse
Affiliation(s)
- Graciela Gonzalez-Hernandez
- Department of Computational Biomedicine, Cedars-Sinai Medical Center , 700 N. San Vicente Blvd., West Hollywood, CA 90069, USA
| | - Martin Krallinger
- Life Sciences—Text Mining, Barcelona Supercomputing Center , Plaça Eusebi Güell, 1-3, Barcelona 08034, Spain
| | - Monica Muñoz
- Division of Pharmacovigilance, Office of Surveillance and Epidemiology, Center of Drug Evaluation and Research, FDA , 10903 New Hampshire Ave, Silver Spring, MD 20993, USA
| | - Raul Rodriguez-Esteban
- Roche Innovation Center Basel, Roche Pharmaceuticals , Grenzacherstrasse 124, Basel 4070, Switzerland
| | - Özlem Uzuner
- Information Sciences and Technology, George Mason University , 4400 University Dr, Fairfax, VA 22030, USA
| | - Lynette Hirschman
- MITRE Labs, The MITRE Corporation , 202 Burlington Rd., Bedford, MA 01730, USA
| |
Collapse
|
25
|
Zhang D, Song J, Dharmarajan S, Jung TH, Lee H, Ma Y, Zhang R, Levenson M. The Use of Machine Learning in Regulatory Drug Safety Evaluation. Stat Biopharm Res 2022. [DOI: 10.1080/19466315.2022.2108135] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/16/2022]
Affiliation(s)
- Di Zhang
- Division of Biometrics VII, Office of Biostatistics, Center for Drug Evaluation and Research, U.S. Food and Drug Administration
| | - Jaejoon Song
- Division of Biometrics VII, Office of Biostatistics, Center for Drug Evaluation and Research, U.S. Food and Drug Administration
| | - Sai Dharmarajan
- Division of Biometrics VII, Office of Biostatistics, Center for Drug Evaluation and Research, U.S. Food and Drug Administration
| | - Tae Hyun Jung
- Division of Biometrics VII, Office of Biostatistics, Center for Drug Evaluation and Research, U.S. Food and Drug Administration
| | - Hana Lee
- Division of Biometrics VII, Office of Biostatistics, Center for Drug Evaluation and Research, U.S. Food and Drug Administration
| | - Yong Ma
- Division of Biometrics VII, Office of Biostatistics, Center for Drug Evaluation and Research, U.S. Food and Drug Administration
| | - Rongmei Zhang
- Division of Biometrics VII, Office of Biostatistics, Center for Drug Evaluation and Research, U.S. Food and Drug Administration
| | - Mark Levenson
- Division of Biometrics VII, Office of Biostatistics, Center for Drug Evaluation and Research, U.S. Food and Drug Administration
| |
Collapse
|
26
|
Polak TB, Cucchi DGJ, van Rosmalen J, Uyl-de Groot CA, Darrow JJ. Generating Evidence from Expanded Access Use of Rare Disease Medicines: Challenges and Recommendations. Front Pharmacol 2022; 13:913567. [PMID: 35677436 PMCID: PMC9168458 DOI: 10.3389/fphar.2022.913567] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2022] [Accepted: 05/05/2022] [Indexed: 12/05/2022] Open
Abstract
Patients with rare diseases often have limited or no options for approved treatments or participation in clinical trials. In such cases, expanded access (or “compassionate use”) provides a potential means of accessing unapproved investigational medicines. It is also possible to capture and analyze clinical data from such use, but doing so is controversial. In this perspective, we offer examples of evidence derived from expanded access programs for rare diseases to illustrate its potential value to the decision-making of regulators and payers in the European Union and the United States. We discuss ethical and regulatory aspects to the use of expanded access data, with a focus on rare disease medicines. The heterogeneous approach to expanded access among countries within the European Union leaves uncertainties to what extent data can be collected and analyzed. We recommend the issuance of new guidance on data collection during expanded access, harmonization of European pathways, and an update of existing European compassionate use guidance. We hereby aim to clarify the supportive role of expanded access in evidence generation. Harmonization across Europe of expanded access regulations could reduce manufacturer burdens, improve patient access, and yield better data. These changes would better balance the need to generate quality evidence with the desire for pre-approval access to investigational medicine.
Collapse
Affiliation(s)
- Tobias B Polak
- Erasmus School of Health Policy and Management, Erasmus University Rotterdam, Rotterdam, Netherlands.,Department of Biostatistics, Erasmus University Rotterdam, Rotterdam, Netherlands.,Department of Epidemiology, Erasmus University Rotterdam, Rotterdam, Netherlands.,Real-World Data Department, myTomorrows, Amsterdam, Netherlands
| | - David G J Cucchi
- Department of Internal Medicine, Franciscus Gasthuis & Vlietland, Rotterdam, Netherlands.,Department of Hematology, Cancer Center Amsterdam, Vrije Universiteit Amsterdam, Amsterdam University Medical Center, Amsterdam, Netherlands
| | - Joost van Rosmalen
- Department of Biostatistics, Erasmus University Rotterdam, Rotterdam, Netherlands.,Department of Epidemiology, Erasmus University Rotterdam, Rotterdam, Netherlands
| | - Carin A Uyl-de Groot
- Erasmus School of Health Policy and Management, Erasmus University Rotterdam, Rotterdam, Netherlands
| | - Jonathan J Darrow
- Department of Law and Taxation, Bentley University, Waltham, MA, United States.,Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, MA, United States
| |
Collapse
|
27
|
Chan JTH, Liew DFL, Stojanova J, McMaster C. Better Pharmacovigilance Through Artificial Intelligence: What Is Needed To Make This A Reality? HEALTH POLICY AND TECHNOLOGY 2022. [DOI: 10.1016/j.hlpt.2022.100638] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
|