2
|
Chou A, Torres-Espin A, Kyritsis N, Huie JR, Khatry S, Funk J, Hay J, Lofgreen A, Shah R, McCann C, Pascual LU, Amorim E, Weinstein PR, Manley GT, Dhall SS, Pan JZ, Bresnahan JC, Beattie MS, Whetstone WD, Ferguson AR. Expert-augmented automated machine learning optimizes hemodynamic predictors of spinal cord injury outcome. PLoS One 2022; 17:e0265254. [PMID: 35390006 PMCID: PMC8989303 DOI: 10.1371/journal.pone.0265254] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2021] [Accepted: 02/25/2022] [Indexed: 11/18/2022] Open
Abstract
Artificial intelligence and machine learning (AI/ML) is becoming increasingly more accessible to biomedical researchers with significant potential to transform biomedicine through optimization of highly-accurate predictive models and enabling better understanding of disease biology. Automated machine learning (AutoML) in particular is positioned to democratize artificial intelligence (AI) by reducing the amount of human input and ML expertise needed. However, successful translation of AI/ML in biomedicine requires moving beyond optimizing only for prediction accuracy and towards establishing reproducible clinical and biological inferences. This is especially challenging for clinical studies on rare disorders where the smaller patient cohorts and corresponding sample size is an obstacle for reproducible modeling results. Here, we present a model-agnostic framework to reinforce AutoML using strategies and tools of explainable and reproducible AI, including novel metrics to assess model reproducibility. The framework enables clinicians to interpret AutoML-generated models for clinical and biological verifiability and consequently integrate domain expertise during model development. We applied the framework towards spinal cord injury prognostication to optimize the intraoperative hemodynamic range during injury-related surgery and additionally identified a strong detrimental relationship between intraoperative hypertension and patient outcome. Furthermore, our analysis captured how evolving clinical practices such as faster time-to-surgery and blood pressure management affect clinical model development. Altogether, we illustrate how expert-augmented AutoML improves inferential reproducibility for biomedical discovery and can ultimately build trust in AI processes towards effective clinical integration.
Collapse
Affiliation(s)
- Austin Chou
- Weill Institute for Neurosciences, Brain and Spinal Injury Center, University of California, San Francisco (UCSF), San Francisco, California, United States of America
- Department of Neurological Surgery, University of California, San Francisco (UCSF), San Francisco, California, United States of America
- Zuckerberg San Francisco General Hospital and Trauma Center, San Francisco, California, United States of America
| | - Abel Torres-Espin
- Weill Institute for Neurosciences, Brain and Spinal Injury Center, University of California, San Francisco (UCSF), San Francisco, California, United States of America
- Department of Neurological Surgery, University of California, San Francisco (UCSF), San Francisco, California, United States of America
- Zuckerberg San Francisco General Hospital and Trauma Center, San Francisco, California, United States of America
| | - Nikos Kyritsis
- Weill Institute for Neurosciences, Brain and Spinal Injury Center, University of California, San Francisco (UCSF), San Francisco, California, United States of America
- Department of Neurological Surgery, University of California, San Francisco (UCSF), San Francisco, California, United States of America
- Zuckerberg San Francisco General Hospital and Trauma Center, San Francisco, California, United States of America
| | - J. Russell Huie
- Weill Institute for Neurosciences, Brain and Spinal Injury Center, University of California, San Francisco (UCSF), San Francisco, California, United States of America
- Department of Neurological Surgery, University of California, San Francisco (UCSF), San Francisco, California, United States of America
- Zuckerberg San Francisco General Hospital and Trauma Center, San Francisco, California, United States of America
| | - Sarah Khatry
- DataRobot, Inc., Boston, Massachusetts, United States of America
| | - Jeremy Funk
- DataRobot, Inc., Boston, Massachusetts, United States of America
| | - Jennifer Hay
- DataRobot, Inc., Boston, Massachusetts, United States of America
| | - Andrew Lofgreen
- DataRobot, Inc., Boston, Massachusetts, United States of America
| | - Rajiv Shah
- DataRobot, Inc., Boston, Massachusetts, United States of America
| | - Chandler McCann
- DataRobot, Inc., Boston, Massachusetts, United States of America
| | - Lisa U. Pascual
- Orthopedic Trauma Institute, Department of Orthopedic Surgery, University of California, San Francisco (UCSF), San Francisco, California, United States of America
| | - Edilberto Amorim
- Zuckerberg San Francisco General Hospital and Trauma Center, San Francisco, California, United States of America
- Department of Neurology, University of California, San Francisco (UCSF), San Francisco, California, United States of America
| | - Philip R. Weinstein
- Department of Neurological Surgery, University of California, San Francisco (UCSF), San Francisco, California, United States of America
- Department of Neurology, University of California, San Francisco (UCSF), San Francisco, California, United States of America
- Weill Institute for Neurosciences, Institute for Neurodegenerative Diseases, Spine Center, University of California, San Francisco (UCSF), San Francisco, California, United States of America
| | - Geoffrey T. Manley
- Weill Institute for Neurosciences, Brain and Spinal Injury Center, University of California, San Francisco (UCSF), San Francisco, California, United States of America
- Department of Neurological Surgery, University of California, San Francisco (UCSF), San Francisco, California, United States of America
- Zuckerberg San Francisco General Hospital and Trauma Center, San Francisco, California, United States of America
| | - Sanjay S. Dhall
- Weill Institute for Neurosciences, Brain and Spinal Injury Center, University of California, San Francisco (UCSF), San Francisco, California, United States of America
- Department of Neurological Surgery, University of California, San Francisco (UCSF), San Francisco, California, United States of America
- Zuckerberg San Francisco General Hospital and Trauma Center, San Francisco, California, United States of America
| | - Jonathan Z. Pan
- Weill Institute for Neurosciences, Brain and Spinal Injury Center, University of California, San Francisco (UCSF), San Francisco, California, United States of America
- Department of Anesthesia and Perioperative Care, University of California, San Francisco (UCSF), San Francisco, California, United States of America
| | - Jacqueline C. Bresnahan
- Weill Institute for Neurosciences, Brain and Spinal Injury Center, University of California, San Francisco (UCSF), San Francisco, California, United States of America
- Department of Neurological Surgery, University of California, San Francisco (UCSF), San Francisco, California, United States of America
- Zuckerberg San Francisco General Hospital and Trauma Center, San Francisco, California, United States of America
| | - Michael S. Beattie
- Weill Institute for Neurosciences, Brain and Spinal Injury Center, University of California, San Francisco (UCSF), San Francisco, California, United States of America
- Department of Neurological Surgery, University of California, San Francisco (UCSF), San Francisco, California, United States of America
- Zuckerberg San Francisco General Hospital and Trauma Center, San Francisco, California, United States of America
| | - William D. Whetstone
- Department of Emergency Medicine, University of California, San Francisco (UCSF), San Francisco, California, United States of America
| | - Adam R. Ferguson
- Weill Institute for Neurosciences, Brain and Spinal Injury Center, University of California, San Francisco (UCSF), San Francisco, California, United States of America
- Department of Neurological Surgery, University of California, San Francisco (UCSF), San Francisco, California, United States of America
- Zuckerberg San Francisco General Hospital and Trauma Center, San Francisco, California, United States of America
| | | |
Collapse
|