1
|
Eslami M, Tabarestani S, Adjouadi M. A unique color-coded visualization system with multimodal information fusion and deep learning in a longitudinal study of Alzheimer's disease. Artif Intell Med 2023; 140:102543. [PMID: 37210151 PMCID: PMC10204620 DOI: 10.1016/j.artmed.2023.102543] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2022] [Revised: 03/28/2023] [Accepted: 04/02/2023] [Indexed: 05/22/2023]
Abstract
PURPOSE Automated diagnosis and prognosis of Alzheimer's Disease remain a challenging problem that machine learning (ML) techniques have attempted to resolve in the last decade. This study introduces a first-of-its-kind color-coded visualization mechanism driven by an integrated ML model to predict disease trajectory in a 2-year longitudinal study. The main aim of this study is to help capture visually in 2D and 3D renderings the diagnosis and prognosis of AD, therefore augmenting our understanding of the processes of multiclass classification and regression analysis. METHOD The proposed method, Machine Learning for Visualizing AD (ML4VisAD), is designed to predict disease progression through a visual output. This newly developed model takes baseline measurements as input to generate a color-coded visual image that reflects disease progression at different time points. The architecture of the network relies on convolutional neural networks. With 1123 subjects selected from the ADNI QT-PAD dataset, we use a 10-fold cross-validation process to evaluate the method. Multimodal inputs* include neuroimaging data (MRI, PET), neuropsychological test scores (excluding MMSE, CDR-SB, and ADAS to avoid bias), cerebrospinal fluid (CSF) biomarkers with measures of amyloid beta (ABETA), phosphorylated tau protein (PTAU), total tau protein (TAU), and risk factors that include age, gender, years of education, and ApoE4 gene. FINDINGS/RESULTS Based on subjective scores reached by three raters, the results showed an accuracy of 0.82 ± 0.03 for a 3-way classification and 0.68 ± 0.05 for a 5-way classification. The visual renderings were generated in 0.08 msec for a 23 × 23 output image and in 0.17 ms for a 45 × 45 output image. Through visualization, this study (1) demonstrates that the ML visual output augments the prospects for a more accurate diagnosis and (2) highlights why multiclass classification and regression analysis are incredibly challenging. An online survey was conducted to gauge this visualization platform's merits and obtain valuable feedback from users. All implementation codes are shared online on GitHub. CONCLUSION This approach makes it possible to visualize the many nuances that lead to a specific classification or prediction in the disease trajectory, all in context to multimodal measurements taken at baseline. This ML model can serve as a multiclass classification and prediction model while reinforcing the diagnosis and prognosis capabilities by including a visualization platform.
Collapse
Affiliation(s)
- Mohammad Eslami
- Harvard Ophthalmology AI lab, Schepens Eye Research Institute of Massachusetts Eye and Ear, Harvard Medical School, Boston, MA, USA; Center for Advanced Technology and Education, Florida International University, Miami, FL, United States.
| | - Solale Tabarestani
- Center for Advanced Technology and Education, Florida International University, Miami, FL, United States.
| | - Malek Adjouadi
- Center for Advanced Technology and Education, Florida International University, Miami, FL, United States.
| |
Collapse
|
3
|
Hosten N, Bülow R, Völzke H, Domin M, Schmidt CO, Teumer A, Ittermann T, Nauck M, Felix S, Dörr M, Markus MRP, Völker U, Daboul A, Schwahn C, Holtfreter B, Mundt T, Krey KF, Kindler S, Mksoud M, Samietz S, Biffar R, Hoffmann W, Kocher T, Chenot JF, Stahl A, Tost F, Friedrich N, Zylla S, Hannemann A, Lotze M, Kühn JP, Hegenscheid K, Rosenberg C, Wassilew G, Frenzel S, Wittfeld K, Grabe HJ, Kromrey ML. SHIP-MR and Radiology: 12 Years of Whole-Body Magnetic Resonance Imaging in a Single Center. Healthcare (Basel) 2021; 10:33. [PMID: 35052197 PMCID: PMC8775435 DOI: 10.3390/healthcare10010033] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2021] [Revised: 12/17/2021] [Accepted: 12/20/2021] [Indexed: 12/16/2022] Open
Abstract
The Study of Health in Pomerania (SHIP), a population-based study from a rural state in northeastern Germany with a relatively poor life expectancy, supplemented its comprehensive examination program in 2008 with whole-body MR imaging at 1.5 T (SHIP-MR). We reviewed more than 100 publications that used the SHIP-MR data and analyzed which sequences already produced fruitful scientific outputs and which manuscripts have been referenced frequently. Upon reviewing the publications about imaging sequences, those that used T1-weighted structured imaging of the brain and a gradient-echo sequence for R2* mapping obtained the highest scientific output; regarding specific body parts examined, most scientific publications focused on MR sequences involving the brain and the (upper) abdomen. We conclude that population-based MR imaging in cohort studies should define more precise goals when allocating imaging time. In addition, quality control measures might include recording the number and impact of published work, preferably on a bi-annual basis and starting 2 years after initiation of the study. Structured teaching courses may enhance the desired output in areas that appear underrepresented.
Collapse
Affiliation(s)
- Norbert Hosten
- Institute of Diagnostic Radiology and Neuroradiology, University Medicine Greifswald, 17475 Greifswald, Germany; (N.H.); (R.B.); (M.D.); (K.H.); (C.R.)
| | - Robin Bülow
- Institute of Diagnostic Radiology and Neuroradiology, University Medicine Greifswald, 17475 Greifswald, Germany; (N.H.); (R.B.); (M.D.); (K.H.); (C.R.)
| | - Henry Völzke
- Institute for Community Medicine, University Medicine Greifswald, 17475 Greifswald, Germany; (H.V.); (C.O.S.); (A.T.); (T.I.); (W.H.); (J.-F.C.)
- German Centre for Cardiovascular Research (DZHK), Partner Site Greifswald, 10785 Berlin, Germany; (M.N.); (S.F.); (M.D.); (M.R.P.M.); (U.V.); (N.F.); (S.Z.); (A.H.)
| | - Martin Domin
- Institute of Diagnostic Radiology and Neuroradiology, University Medicine Greifswald, 17475 Greifswald, Germany; (N.H.); (R.B.); (M.D.); (K.H.); (C.R.)
| | - Carsten Oliver Schmidt
- Institute for Community Medicine, University Medicine Greifswald, 17475 Greifswald, Germany; (H.V.); (C.O.S.); (A.T.); (T.I.); (W.H.); (J.-F.C.)
| | - Alexander Teumer
- Institute for Community Medicine, University Medicine Greifswald, 17475 Greifswald, Germany; (H.V.); (C.O.S.); (A.T.); (T.I.); (W.H.); (J.-F.C.)
- German Centre for Cardiovascular Research (DZHK), Partner Site Greifswald, 10785 Berlin, Germany; (M.N.); (S.F.); (M.D.); (M.R.P.M.); (U.V.); (N.F.); (S.Z.); (A.H.)
| | - Till Ittermann
- Institute for Community Medicine, University Medicine Greifswald, 17475 Greifswald, Germany; (H.V.); (C.O.S.); (A.T.); (T.I.); (W.H.); (J.-F.C.)
| | - Matthias Nauck
- German Centre for Cardiovascular Research (DZHK), Partner Site Greifswald, 10785 Berlin, Germany; (M.N.); (S.F.); (M.D.); (M.R.P.M.); (U.V.); (N.F.); (S.Z.); (A.H.)
- Institute of Clinical Chemistry and Laboratory Medicine, University Medicine Greifswald, 17475 Greifswald, Germany
| | - Stephan Felix
- German Centre for Cardiovascular Research (DZHK), Partner Site Greifswald, 10785 Berlin, Germany; (M.N.); (S.F.); (M.D.); (M.R.P.M.); (U.V.); (N.F.); (S.Z.); (A.H.)
- Department of Internal Medicine B, University Medicine Greifswald, 17475 Greifswald, Germany
| | - Marcus Dörr
- German Centre for Cardiovascular Research (DZHK), Partner Site Greifswald, 10785 Berlin, Germany; (M.N.); (S.F.); (M.D.); (M.R.P.M.); (U.V.); (N.F.); (S.Z.); (A.H.)
- Department of Internal Medicine B, University Medicine Greifswald, 17475 Greifswald, Germany
| | - Marcello Ricardo Paulista Markus
- German Centre for Cardiovascular Research (DZHK), Partner Site Greifswald, 10785 Berlin, Germany; (M.N.); (S.F.); (M.D.); (M.R.P.M.); (U.V.); (N.F.); (S.Z.); (A.H.)
- Department of Internal Medicine B, University Medicine Greifswald, 17475 Greifswald, Germany
| | - Uwe Völker
- German Centre for Cardiovascular Research (DZHK), Partner Site Greifswald, 10785 Berlin, Germany; (M.N.); (S.F.); (M.D.); (M.R.P.M.); (U.V.); (N.F.); (S.Z.); (A.H.)
- Interfaculty Institute of Genetics and Functional Genomics, University Medicine Greifswald, 17475 Greifswald, Germany
| | - Amro Daboul
- Department of Prosthetic Dentistry, Gerodontology and Biomaterials, University Medicine Greifswald, 17475 Greifswald, Germany; (A.D.); (C.S.); (T.M.); (S.S.); (R.B.)
| | - Christian Schwahn
- Department of Prosthetic Dentistry, Gerodontology and Biomaterials, University Medicine Greifswald, 17475 Greifswald, Germany; (A.D.); (C.S.); (T.M.); (S.S.); (R.B.)
| | - Birte Holtfreter
- Department of Restorative Dentistry, Periodontology, Endodontology, and Preventive and Pediatric Dentistry, University Medicine Greifswald, 17475 Greifswald, Germany; (B.H.); (T.K.)
| | - Torsten Mundt
- Department of Prosthetic Dentistry, Gerodontology and Biomaterials, University Medicine Greifswald, 17475 Greifswald, Germany; (A.D.); (C.S.); (T.M.); (S.S.); (R.B.)
| | - Karl-Friedrich Krey
- Department of Orthodontics, University Medicine Greifswald, 17475 Greifswald, Germany;
| | - Stefan Kindler
- Department of Oral and Maxillofacial Surgery/Plastic Surgery, University Medicine Greifswald, 17475 Greifswald, Germany; (S.K.); (M.M.)
| | - Maria Mksoud
- Department of Oral and Maxillofacial Surgery/Plastic Surgery, University Medicine Greifswald, 17475 Greifswald, Germany; (S.K.); (M.M.)
| | - Stefanie Samietz
- Department of Prosthetic Dentistry, Gerodontology and Biomaterials, University Medicine Greifswald, 17475 Greifswald, Germany; (A.D.); (C.S.); (T.M.); (S.S.); (R.B.)
| | - Reiner Biffar
- Department of Prosthetic Dentistry, Gerodontology and Biomaterials, University Medicine Greifswald, 17475 Greifswald, Germany; (A.D.); (C.S.); (T.M.); (S.S.); (R.B.)
| | - Wolfgang Hoffmann
- Institute for Community Medicine, University Medicine Greifswald, 17475 Greifswald, Germany; (H.V.); (C.O.S.); (A.T.); (T.I.); (W.H.); (J.-F.C.)
- German Centre for Cardiovascular Research (DZHK), Partner Site Greifswald, 10785 Berlin, Germany; (M.N.); (S.F.); (M.D.); (M.R.P.M.); (U.V.); (N.F.); (S.Z.); (A.H.)
- German Centre for Neurodegenerative Diseases (DZNE), Partner Site Rostock/Greifswald, 17489 Greifswald, Germany
| | - Thomas Kocher
- Department of Restorative Dentistry, Periodontology, Endodontology, and Preventive and Pediatric Dentistry, University Medicine Greifswald, 17475 Greifswald, Germany; (B.H.); (T.K.)
| | - Jean-Francois Chenot
- Institute for Community Medicine, University Medicine Greifswald, 17475 Greifswald, Germany; (H.V.); (C.O.S.); (A.T.); (T.I.); (W.H.); (J.-F.C.)
| | - Andreas Stahl
- Clinic of Ophthalmology, University Medicine Greifswald, 17475 Greifswald, Germany; (A.S.); (F.T.)
| | - Frank Tost
- Clinic of Ophthalmology, University Medicine Greifswald, 17475 Greifswald, Germany; (A.S.); (F.T.)
| | - Nele Friedrich
- German Centre for Cardiovascular Research (DZHK), Partner Site Greifswald, 10785 Berlin, Germany; (M.N.); (S.F.); (M.D.); (M.R.P.M.); (U.V.); (N.F.); (S.Z.); (A.H.)
- Institute of Clinical Chemistry and Laboratory Medicine, University Medicine Greifswald, 17475 Greifswald, Germany
| | - Stephanie Zylla
- German Centre for Cardiovascular Research (DZHK), Partner Site Greifswald, 10785 Berlin, Germany; (M.N.); (S.F.); (M.D.); (M.R.P.M.); (U.V.); (N.F.); (S.Z.); (A.H.)
- Institute of Clinical Chemistry and Laboratory Medicine, University Medicine Greifswald, 17475 Greifswald, Germany
| | - Anke Hannemann
- German Centre for Cardiovascular Research (DZHK), Partner Site Greifswald, 10785 Berlin, Germany; (M.N.); (S.F.); (M.D.); (M.R.P.M.); (U.V.); (N.F.); (S.Z.); (A.H.)
- Institute of Clinical Chemistry and Laboratory Medicine, University Medicine Greifswald, 17475 Greifswald, Germany
| | - Martin Lotze
- Functional Imaging Unit, Institute of Diagnostic Radiology and Neuroradiology, University Medicine Greifswald, 17475 Greifswald, Germany;
| | - Jens-Peter Kühn
- Institute and Policlinic of Diagnostic and Interventional Radiology, Medical University, Carl-Gustav Carus, 01307 Dresden, Germany;
| | - Katrin Hegenscheid
- Institute of Diagnostic Radiology and Neuroradiology, University Medicine Greifswald, 17475 Greifswald, Germany; (N.H.); (R.B.); (M.D.); (K.H.); (C.R.)
| | - Christian Rosenberg
- Institute of Diagnostic Radiology and Neuroradiology, University Medicine Greifswald, 17475 Greifswald, Germany; (N.H.); (R.B.); (M.D.); (K.H.); (C.R.)
| | - Georgi Wassilew
- Clinic of Orthopedics, University Medicine Greifswald, 17475 Greifswald, Germany;
| | - Stefan Frenzel
- Department of Psychiatry and Psychotherapy, University Medicine Greifswald, 17475 Greifswald, Germany; (S.F.); (K.W.); (H.J.G.)
| | - Katharina Wittfeld
- Department of Psychiatry and Psychotherapy, University Medicine Greifswald, 17475 Greifswald, Germany; (S.F.); (K.W.); (H.J.G.)
- German Center of Neurodegenerative Diseases (DZNE), Rostock/Greifswald, Site Greifswald, 17489 Greifswald, Germany
| | - Hans J. Grabe
- Department of Psychiatry and Psychotherapy, University Medicine Greifswald, 17475 Greifswald, Germany; (S.F.); (K.W.); (H.J.G.)
- German Center of Neurodegenerative Diseases (DZNE), Rostock/Greifswald, Site Greifswald, 17489 Greifswald, Germany
| | - Marie-Luise Kromrey
- Institute of Diagnostic Radiology and Neuroradiology, University Medicine Greifswald, 17475 Greifswald, Germany; (N.H.); (R.B.); (M.D.); (K.H.); (C.R.)
- Correspondence:
| |
Collapse
|
6
|
Phenotyping chronic tinnitus patients using self-report questionnaire data: cluster analysis and visual comparison. Sci Rep 2020; 10:16411. [PMID: 33009468 PMCID: PMC7532444 DOI: 10.1038/s41598-020-73402-8] [Citation(s) in RCA: 17] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2020] [Accepted: 09/16/2020] [Indexed: 12/02/2022] Open
Abstract
Chronic tinnitus is a complex, multi-factorial symptom that requires careful assessment and management. Evidence-based therapeutic approaches involve audiological and psychological treatment components. However, not everyone benefits from treatment. The identification and characterisation of patient subgroups (or “phenotypes”) may provide clinically relevant information. Due to the large number of assessment tools, data-driven methods appear to be promising. The acceptance of these empirical results can be further strengthened by a comprehensive visualisation. In this study, we used cluster analysis to identify distinct tinnitus phenotypes based on self-report questionnaire data and implemented a visualisation tool to explore phenotype idiosyncrasies. 1228 patients with chronic tinnitus from the Charité Tinnitus Center in Berlin were included. At baseline, each participant completed 14 questionnaires measuring tinnitus distress, -loudness, frequency and location, depressivity, perceived stress, quality of life, physical and mental health, pain perception, somatic symptom expression and coping attitudes. Four distinct patient phenotypes emerged from clustering: avoidant group (56.8%), psychosomatic group (14.1%), somatic group (15.2%), and distress group (13.9%). Radial bar- and line charts allowed for visual inspection and juxtaposition of major phenotype characteristics. The phenotypes differed in terms of clinical information including psychological symptoms, quality of life, coping attitudes, stress, tinnitus-related distress and pain, as well as socio-demographics. Our findings suggest that identifiable patient subgroups and their visualisation may allow for stratified treatment strategies and research designs.
Collapse
|