1
|
Agius R, Riis-Jensen AC, Wimmer B, da Cunha-Bang C, Murray DD, Poulsen CB, Bertelsen MB, Schwartz B, Lundgren JD, Langberg H, Niemann CU. Deployment and validation of the CLL treatment infection model adjoined to an EHR system. NPJ Digit Med 2024; 7:147. [PMID: 38839920 PMCID: PMC11153589 DOI: 10.1038/s41746-024-01132-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2023] [Accepted: 05/08/2024] [Indexed: 06/07/2024] Open
Abstract
Research algorithms are seldom externally validated or integrated into clinical practice, leaving unknown challenges in deployment. In such efforts, one needs to address challenges related to data harmonization, the performance of an algorithm in unforeseen missingness, automation and monitoring of predictions, and legal frameworks. We here describe the deployment of a high-dimensional data-driven decision support model into an EHR and derive practical guidelines informed by this deployment that includes the necessary processes, stakeholders and design requirements for a successful deployment. For this, we describe our deployment of the chronic lymphocytic leukemia (CLL) treatment infection model (CLL-TIM) as a stand-alone platform adjoined to an EPIC-based Danish Electronic Health Record (EHR), with the presentation of personalized predictions in a clinical context. CLL-TIM is an 84-variable data-driven prognostic model utilizing 7-year medical patient records and predicts the 2-year risk composite outcome of infection and/or treatment post-CLL diagnosis. As an independent validation cohort for this deployment, we used a retrospective population-based cohort of patients diagnosed with CLL from 2018 onwards (n = 1480). Unexpectedly high levels of missingness for key CLL-TIM variables were exhibited upon deployment. High dimensionality, with the handling of missingness, and predictive confidence were critical design elements that enabled trustworthy predictions and thus serves as a priority for prognostic models seeking deployment in new EHRs. Our setup for deployment, including automation and monitoring into EHR that meets Medical Device Regulations, may be used as step-by-step guidelines for others aiming at designing and deploying research algorithms into clinical practice.
Collapse
Affiliation(s)
- Rudi Agius
- Department of Hematology, Rigshospitalet, Copenhagen University Hospital, Copenhagen, Denmark
| | | | - Bettina Wimmer
- SP Sundhedsdata, The Data Unit, Capital Region of Denmark, Copenhagen, Denmark
| | - Caspar da Cunha-Bang
- Department of Hematology, Rigshospitalet, Copenhagen University Hospital, Copenhagen, Denmark
| | - Daniel Dawson Murray
- Center of Excellence for Health, Immunity, and Infections (CHIP), Rigshospitalet, Copenhagen University Hospital, Copenhagen, Denmark
| | | | | | - Berit Schwartz
- Rigshospitalets Innoovationscenter, Copenhagen University Hospital Rigshopsitalet, Copenhagen, Denmark
| | - Jens Dilling Lundgren
- Center of Excellence for Health, Immunity, and Infections (CHIP), Rigshospitalet, Copenhagen University Hospital, Copenhagen, Denmark
| | - Henning Langberg
- Rigshospitalets Innoovationscenter, Copenhagen University Hospital Rigshopsitalet, Copenhagen, Denmark
| | - Carsten Utoft Niemann
- Department of Hematology, Rigshospitalet, Copenhagen University Hospital, Copenhagen, Denmark.
- Department of Clinical Medicine, University of Copenhagen, Copenhagen, Denmark.
| |
Collapse
|
2
|
Shao Y, Kay NE, Gale RP, Liang Y. Challenges in analyzing clinical trials testing Bruton tyrosine-kinase-inhibitora in chronic lymphocytic leukaemia. Leukemia 2024:10.1038/s41375-024-02294-8. [PMID: 38824147 DOI: 10.1038/s41375-024-02294-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/25/2023] [Revised: 05/16/2024] [Accepted: 05/20/2024] [Indexed: 06/03/2024]
Affiliation(s)
- Yingqi Shao
- State Key Laboratory of Experimental Hematology, National Clinical Research Center for Blood Diseases, Haihe Laboratory of Cell Ecosystem, Institute of Hematology & Blood Diseases Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Tianjin, China.
| | - Neil E Kay
- Division of Hematology, and Department of Immunology, Mayo Clinic, Rochester, MN, USA
| | - Robert Peter Gale
- Department of Hematologic Oncology, State Key Laboratory of Oncology in South China, Collaborative Innovation, Center for Cancer Medicine, Sun Yat-sen University Cancer Center, Guangzhou, China
- Centre for Haematology, Department of Immunology and Inflammation, Imperial College of Science, Technology & Medicine, London, London, UK
| | - Yang Liang
- State Key Laboratory of Experimental Hematology, National Clinical Research Center for Blood Diseases, Haihe Laboratory of Cell Ecosystem, Institute of Hematology & Blood Diseases Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Tianjin, China.
- Department of Hematologic Oncology, State Key Laboratory of Oncology in South China, Collaborative Innovation, Center for Cancer Medicine, Sun Yat-sen University Cancer Center, Guangzhou, China.
| |
Collapse
|
3
|
Morabito F, Adornetto C, Monti P, Amaro A, Reggiani F, Colombo M, Rodriguez-Aldana Y, Tripepi G, D’Arrigo G, Vener C, Torricelli F, Rossi T, Neri A, Ferrarini M, Cutrona G, Gentile M, Greco G. Genes selection using deep learning and explainable artificial intelligence for chronic lymphocytic leukemia predicting the need and time to therapy. Front Oncol 2023; 13:1198992. [PMID: 37719021 PMCID: PMC10501728 DOI: 10.3389/fonc.2023.1198992] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/02/2023] [Accepted: 07/31/2023] [Indexed: 09/19/2023] Open
Abstract
Analyzing gene expression profiles (GEP) through artificial intelligence provides meaningful insight into cancer disease. This study introduces DeepSHAP Autoencoder Filter for Genes Selection (DSAF-GS), a novel deep learning and explainable artificial intelligence-based approach for feature selection in genomics-scale data. DSAF-GS exploits the autoencoder's reconstruction capabilities without changing the original feature space, enhancing the interpretation of the results. Explainable artificial intelligence is then used to select the informative genes for chronic lymphocytic leukemia prognosis of 217 cases from a GEP database comprising roughly 20,000 genes. The model for prognosis prediction achieved an accuracy of 86.4%, a sensitivity of 85.0%, and a specificity of 87.5%. According to the proposed approach, predictions were strongly influenced by CEACAM19 and PIGP, moderately influenced by MKL1 and GNE, and poorly influenced by other genes. The 10 most influential genes were selected for further analysis. Among them, FADD, FIBP, FIBP, GNE, IGF1R, MKL1, PIGP, and SLC39A6 were identified in the Reactome pathway database as involved in signal transduction, transcription, protein metabolism, immune system, cell cycle, and apoptosis. Moreover, according to the network model of the 3D protein-protein interaction (PPI) explored using the NetworkAnalyst tool, FADD, FIBP, IGF1R, QTRT1, GNE, SLC39A6, and MKL1 appear coupled into a complex network. Finally, all 10 selected genes showed a predictive power on time to first treatment (TTFT) in univariate analyses on a basic prognostic model including IGHV mutational status, del(11q) and del(17p), NOTCH1 mutations, β2-microglobulin, Rai stage, and B-lymphocytosis known to predict TTFT in CLL. However, only IGF1R [hazard ratio (HR) 1.41, 95% CI 1.08-1.84, P=0.013), COL28A1 (HR 0.32, 95% CI 0.10-0.97, P=0.045), and QTRT1 (HR 7.73, 95% CI 2.48-24.04, P<0.001) genes were significantly associated with TTFT in multivariable analyses when combined with the prognostic factors of the basic model, ultimately increasing the Harrell's c-index and the explained variation to 78.6% (versus 76.5% of the basic prognostic model) and 52.6% (versus 42.2% of the basic prognostic model), respectively. Also, the goodness of model fit was enhanced (χ2 = 20.1, P=0.002), indicating its improved performance above the basic prognostic model. In conclusion, DSAF-GS identified a group of significant genes for CLL prognosis, suggesting future directions for bio-molecular research.
Collapse
Affiliation(s)
| | - Carlo Adornetto
- Department of Mathematics and Computer Science, University of Calabria, Cosenza, Italy
| | - Paola Monti
- Mutagenesis and Cancer Prevention Unit, Istituto di Ricovero e Cura a Carattere Scientifico (IRCCS) Ospedale Policlinico San Martino, Genoa, Italy
| | - Adriana Amaro
- Tumor Epigenetics Unit, Istituto di Ricovero e Cura a Carattere Scientifico (IRCCS) Ospedale Policlinico San Martino, Genoa, Italy
| | - Francesco Reggiani
- Tumor Epigenetics Unit, Istituto di Ricovero e Cura a Carattere Scientifico (IRCCS) Ospedale Policlinico San Martino, Genoa, Italy
| | - Monica Colombo
- Molecular Pathology Unit, Istituto di Ricovero e Cura a Carattere Scientifico (IRCCS) Ospedale Policlinico San Martino, Genoa, Italy
| | | | - Giovanni Tripepi
- Consiglio Nazionale delle Ricerche, Istituto di Fisiologia Clinica del Consiglio Nazionale delle Ricerche (CNR), Reggio Calabria, Italy
| | - Graziella D’Arrigo
- Consiglio Nazionale delle Ricerche, Istituto di Fisiologia Clinica del Consiglio Nazionale delle Ricerche (CNR), Reggio Calabria, Italy
| | - Claudia Vener
- Department of Oncology and Hemato-Oncology, University of Milan, Milan, Italy
| | - Federica Torricelli
- Laboratory of Translational Research, Azienda Unità Sanitaria Locale - Istituto di Ricovero e Cura a Crabtree Scientifico (USL-IRCCS) of Reggio Emilia, Reggio Emilia, Italy
| | - Teresa Rossi
- Laboratory of Translational Research, Azienda Unità Sanitaria Locale - Istituto di Ricovero e Cura a Crabtree Scientifico (USL-IRCCS) of Reggio Emilia, Reggio Emilia, Italy
| | - Antonino Neri
- Scientific Directorate, Azienda Unità Sanitaria Locale - Istituto di Ricovero e Cura a Carattere Scientifico (USL-IRCCS) of Reggio Emilia, Reggio Emilia, Italy
| | - Manlio Ferrarini
- Unità Operariva (UO) Molecular Pathology, Ospedale Policlinico San Martino Istituto di Ricovero e Cura a Carattere Scientifico (IRCCS), Genoa, Italy
| | - Giovanna Cutrona
- Molecular Pathology Unit, Istituto di Ricovero e Cura a Carattere Scientifico (IRCCS) Ospedale Policlinico San Martino, Genoa, Italy
| | - Massimo Gentile
- Hematology Unit, Department of Onco-Hematology, Azienda Ospedaliera (A.O.) of Cosenza, Cosenza, Italy
- Department of Pharmacy and Health and Nutritional Sciences, University of Calabria, Cosenza, Italy
| | - Gianluigi Greco
- Department of Mathematics and Computer Science, University of Calabria, Cosenza, Italy
| |
Collapse
|
4
|
Skånland SS, Brown JR. PI3K inhibitors in chronic lymphocytic leukemia: where do we go from here? Haematologica 2023; 108:9-21. [PMID: 35899388 PMCID: PMC9827175 DOI: 10.3324/haematol.2022.281266] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2022] [Accepted: 07/21/2022] [Indexed: 02/04/2023] Open
Abstract
Phosphatidylinositol 3-kinase (PI3K) inhibitors are effective in chronic lymphocytic leukemia (CLL). However, the severe toxicity profile associated with the first-generation inhibitors idelalisib and duvelisib, combined with the availability of other more tolerable agents, have limited their use. CLL is still considered incurable, and relapse after treatment, development of resistance, and treatment intolerance are common. It is therefore of interest to optimize the administration of currently approved PI3K inhibitors and to develop next-generation agents to improve tolerability, so that this class of agents will be considered an effective and safe treatment option when needed. These efforts are reflected in the large number of emerging clinical trials with PI3K inhibitors in CLL. Current strategies to overcome treatment limitations include intermittent dosing, which is established for copanlisib and zandelisib and under investigation for duvelisib and parsaclisib. A second strategy is to combine the PI3K inhibitor with another novel agent, either as a continuous regimen or a fixedduration regimen, to deepen responses. In addition to these approaches, it is of interest to identify higher-resolution actionable biomarkers that can predict treatment responses and toxicity, and inform personalized treatment decisions. Here, we discuss the current status of PI3K inhibitors in CLL, factors limiting the use of currently approved PI3K inhibitors in CLL, current strategies to overcome these limitations, and where to go next.
Collapse
Affiliation(s)
- Sigrid S Skånland
- Department of Cancer Immunology, Institute for Cancer Research, Oslo University Hospital, Oslo, Norway; K. G. Jebsen Centre for B Cell Malignancies, Institute of Clinical Medicine, University of Oslo, Oslo.
| | - Jennifer R Brown
- Department of Medical Oncology, Dana-Farber Cancer Institute, Boston, MA, USA; Harvard Medical School, Boston, MA
| |
Collapse
|
5
|
Flobak Å, Skånland SS, Hovig E, Taskén K, Russnes HG. Functional precision cancer medicine: drug sensitivity screening enabled by cell culture models. Trends Pharmacol Sci 2022; 43:973-985. [PMID: 36163057 DOI: 10.1016/j.tips.2022.08.009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/13/2022] [Revised: 08/20/2022] [Accepted: 08/22/2022] [Indexed: 10/31/2022]
Abstract
Functional precision medicine is a new, emerging area that can guide cancer treatment by capturing information from direct perturbations of tumor-derived, living cells, such as by drug sensitivity screening. Precision cancer medicine as currently implemented in clinical practice has been driven by genomics, and current molecular tumor boards rely extensively on genomic characterization to advise on therapeutic interventions. However, genomic biomarkers can only guide treatment decisions for a fraction of the patients. In this review we provide an overview of the current state of functional precision medicine, highlight advances for drug-sensitivity screening enabled by cell culture models, and discuss how artificial intelligence (AI) can be coupled to functional precision medicine to guide patient stratification.
Collapse
Affiliation(s)
- Åsmund Flobak
- The Cancer Clinic, St. Olav University Hospital, Trondheim, Norway; Department of Clinical and Molecular Medicine, Norwegian University of Science and Technology, Trondheim, Norway
| | - Sigrid S Skånland
- Department of Cancer Immunology, Institute for Cancer Research, Oslo University Hospital, Oslo, Norway; K.G. Jebsen Centre for B Cell Malignancies, Institute of Clinical Medicine, University of Oslo, Oslo, Norway
| | - Eivind Hovig
- Department of Tumor Biology, Institute for Cancer Research, Oslo University Hospital, Oslo, Norway; Department of Informatics, Centre for Bioinformatics, University of Oslo, Oslo, Norway
| | - Kjetil Taskén
- Department of Cancer Immunology, Institute for Cancer Research, Oslo University Hospital, Oslo, Norway; K.G. Jebsen Centre for B Cell Malignancies, Institute of Clinical Medicine, University of Oslo, Oslo, Norway.
| | - Hege G Russnes
- Department of Pathology, Oslo University Hospital, Oslo, Norway; Department of Cancer Genetics, Institute for Cancer Research, Oslo University Hospital, Oslo, Norway; Institute of Clinical Medicine, University of Oslo, Oslo, Norway
| |
Collapse
|
6
|
Andersen MA, Grand MK, Brieghel C, Siersma V, Andersen CL, Niemann CU. Pre-diagnostic trajectories of lymphocytosis predict time to treatment and death in patients with chronic lymphocytic leukemia. COMMUNICATIONS MEDICINE 2022; 2:50. [PMID: 35603299 PMCID: PMC9098503 DOI: 10.1038/s43856-022-00117-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/01/2021] [Accepted: 04/25/2022] [Indexed: 12/02/2022] Open
Abstract
Background The dynamics of pre-diagnostic lymphocytosis in patients with ensuing chronic lymphocytic leukemia (CLL) need to be explored as a better understanding of disease progression may improve treatment options and even lead to disease avoidance approaches. Our aim was to investigate the development of lymphocytosis prior to diagnosis in a population-based cohort of patients with CLL and to assess the prognostic information in these pre-diagnostic measurements. Methods All patients diagnosed with CLL in the Greater Copenhagen area between 2008 and 2016 were included in the study. Pre-diagnostic blood test results were obtained from the Copenhagen Primary Care Laboratory Database encompassing all blood tests requested by Copenhagen general practitioners. Using pre-diagnostic measurements, we developed a model to assess the prognosis following diagnosis. Our model accounts for known prognostic factors and corresponds to lymphocyte dynamics after diagnosis. Results We explore trajectories of lymphocytosis, associated with known recurrent mutations. We show that the pre-diagnostic trajectories are an independent predictor of time to treatment. The implementation of pre-diagnostic lymphocytosis slope groups improved the model predictions (compared to CLL-IPI alone) for treatment throughout the period. The model can manage the heterogeneous data that are to be expected from the real-world setting and adds further prognostic information. Conclusions Our findings further knowledge of the development of CLL and may eventually make prophylactic measures possible. While clinicians largely agree that patients with chronic lymphocytic leukemia (CLL) have increased levels of white blood cells in the years preceding their diagnosis, there is less certainty as to how and when this increase occurs. A better understanding of how white blood cell levels change during this period might help us to predict who will become ill and require treatment. In this work, we explore patterns of white blood cell growth and develop a tool to predict the time to treatment for CLL based on these growth rates. Using our tool in the clinic might help clinicians to decide who needs treatment for CLL and when, potentially leading to better outcomes for patients. Andersen et al. evaluate lymphocyte dynamics in chronic lymphocytic leukemia (CLL) patients prior to diagnosis. The authors develop a model to predict risk of requiring CLL treatment or death based on pre-diagnostic lymphocyte growth rates.
Collapse
|
7
|
Prediction of clinical outcome in CLL based on recurrent gene mutations, CLL-IPI variables, and (para)clinical data. Blood Adv 2022; 6:3716-3728. [PMID: 35468622 PMCID: PMC9631547 DOI: 10.1182/bloodadvances.2021006351] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/25/2021] [Accepted: 04/06/2022] [Indexed: 12/03/2022] Open
Abstract
Baseline data (but not genetics) added to CLL-IPI variables improve predictive performance of ML models in CLL. Risk factors predictive of death within a 5-year outlook are mostly similar to risk factors predictive of infection within a 2-year outlook.
A highly variable clinical course, immune dysfunction, and a complex genetic blueprint pose challenges for treatment decisions and the management of risk of infection in patients with chronic lymphocytic leukemia (CLL). In recent years, the use of machine learning (ML) technologies has made it possible to attempt to untangle such heterogeneous disease entities. In this study, using 3 classes of variables (international prognostic index for CLL [CLL-IPI] variables, baseline [para]clinical data, and data on recurrent gene mutations), we built ML predictive models to identify the individual risk of 4 clinical outcomes: death, treatment, infection, and the combined outcome of treatment or infection. Using the predictive models, we assessed to what extent the different classes of variables are predictive of the 4 different outcomes, within both a short-term 2-year outlook and a long-term 5-year outlook after CLL diagnosis. By adding the baseline (para)clinical data to CLL-IPI variables, predictive performance was improved, whereas no further improvement was observed when including the data on recurrent genetic mutations. We discovered 2 main clusters of variables predictive of treatment and infection. Further emphasizing the high mortality resulting from infection in CLL, we found a close similarity between variables predictive of infection in the short-term outlook and those predictive of death in the long-term outlook. We conclude that at the time of CLL diagnosis, routine (para)clinical data are more predictive of patient outcome than recurrent mutations. Future studies on modeling genetics and clinical outcome should always consider the inclusion of several (para)clinical data to improve performance.
Collapse
|