1
|
Kuersten A. Prudently Evaluating Medical Adaptive Machine Learning Systems. THE AMERICAN JOURNAL OF BIOETHICS : AJOB 2024; 24:76-79. [PMID: 39283387 DOI: 10.1080/15265161.2024.2388759] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/20/2024]
Affiliation(s)
- Andreas Kuersten
- American Law Division, Congressional Research Service, Library of Congress
| |
Collapse
|
2
|
Lukkien DRM, Stolwijk NE, Ipakchian Askari S, Hofstede BM, Nap HH, Boon WPC, Peine A, Moors EHM, Minkman MMN. AI-Assisted Decision-Making in Long-Term Care: Qualitative Study on Prerequisites for Responsible Innovation. JMIR Nurs 2024; 7:e55962. [PMID: 39052315 PMCID: PMC11310645 DOI: 10.2196/55962] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/01/2024] [Revised: 04/16/2024] [Accepted: 05/24/2024] [Indexed: 07/27/2024] Open
Abstract
BACKGROUND Although the use of artificial intelligence (AI)-based technologies, such as AI-based decision support systems (AI-DSSs), can help sustain and improve the quality and efficiency of care, their deployment creates ethical and social challenges. In recent years, a growing prevalence of high-level guidelines and frameworks for responsible AI innovation has been observed. However, few studies have specified the responsible embedding of AI-based technologies, such as AI-DSSs, in specific contexts, such as the nursing process in long-term care (LTC) for older adults. OBJECTIVE Prerequisites for responsible AI-assisted decision-making in nursing practice were explored from the perspectives of nurses and other professional stakeholders in LTC. METHODS Semistructured interviews were conducted with 24 care professionals in Dutch LTC, including nurses, care coordinators, data specialists, and care centralists. A total of 2 imaginary scenarios about AI-DSSs were developed beforehand and used to enable participants articulate their expectations regarding the opportunities and risks of AI-assisted decision-making. In addition, 6 high-level principles for responsible AI were used as probing themes to evoke further consideration of the risks associated with using AI-DSSs in LTC. Furthermore, the participants were asked to brainstorm possible strategies and actions in the design, implementation, and use of AI-DSSs to address or mitigate these risks. A thematic analysis was performed to identify the opportunities and risks of AI-assisted decision-making in nursing practice and the associated prerequisites for responsible innovation in this area. RESULTS The stance of care professionals on the use of AI-DSSs is not a matter of purely positive or negative expectations but rather a nuanced interplay of positive and negative elements that lead to a weighed perception of the prerequisites for responsible AI-assisted decision-making. Both opportunities and risks were identified in relation to the early identification of care needs, guidance in devising care strategies, shared decision-making, and the workload of and work experience of caregivers. To optimally balance the opportunities and risks of AI-assisted decision-making, seven categories of prerequisites for responsible AI-assisted decision-making in nursing practice were identified: (1) regular deliberation on data collection; (2) a balanced proactive nature of AI-DSSs; (3) incremental advancements aligned with trust and experience; (4) customization for all user groups, including clients and caregivers; (5) measures to counteract bias and narrow perspectives; (6) human-centric learning loops; and (7) the routinization of using AI-DSSs. CONCLUSIONS The opportunities of AI-assisted decision-making in nursing practice could turn into drawbacks depending on the specific shaping of the design and deployment of AI-DSSs. Therefore, we recommend considering the responsible use of AI-DSSs as a balancing act. Moreover, considering the interrelatedness of the identified prerequisites, we call for various actors, including developers and users of AI-DSSs, to cohesively address the different factors important to the responsible embedding of AI-DSSs in practice.
Collapse
Affiliation(s)
- Dirk R M Lukkien
- Vilans Centre of Expertise of Long Term Care, Utrecht, Netherlands
- Copernicus Institute of Sustainable Development, Utrecht University, Utrecht, Netherlands
| | | | - Sima Ipakchian Askari
- Vilans Centre of Expertise of Long Term Care, Utrecht, Netherlands
- Human Technology Interaction, Eindhoven University of Technology, Eindhoven, Netherlands
| | - Bob M Hofstede
- Vilans Centre of Expertise of Long Term Care, Utrecht, Netherlands
- Human Technology Interaction, Eindhoven University of Technology, Eindhoven, Netherlands
| | - Henk Herman Nap
- Vilans Centre of Expertise of Long Term Care, Utrecht, Netherlands
- Human Technology Interaction, Eindhoven University of Technology, Eindhoven, Netherlands
| | - Wouter P C Boon
- Copernicus Institute of Sustainable Development, Utrecht University, Utrecht, Netherlands
| | - Alexander Peine
- Faculty of Humanities, Open University of The Netherlands, Heerlen, Netherlands
| | - Ellen H M Moors
- Copernicus Institute of Sustainable Development, Utrecht University, Utrecht, Netherlands
| | - Mirella M N Minkman
- Vilans Centre of Expertise of Long Term Care, Utrecht, Netherlands
- TIAS School for Business and Society, Tilburg University, Tilburg, Netherlands
| |
Collapse
|
3
|
Sáez C, Ferri P, García-Gómez JM. Resilient Artificial Intelligence in Health: Synthesis and Research Agenda Toward Next-Generation Trustworthy Clinical Decision Support. J Med Internet Res 2024; 26:e50295. [PMID: 38941134 PMCID: PMC11245653 DOI: 10.2196/50295] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/26/2023] [Revised: 04/16/2024] [Accepted: 05/18/2024] [Indexed: 06/29/2024] Open
Abstract
Artificial intelligence (AI)-based clinical decision support systems are gaining momentum by relying on a greater volume and variety of secondary use data. However, the uncertainty, variability, and biases in real-world data environments still pose significant challenges to the development of health AI, its routine clinical use, and its regulatory frameworks. Health AI should be resilient against real-world environments throughout its lifecycle, including the training and prediction phases and maintenance during production, and health AI regulations should evolve accordingly. Data quality issues, variability over time or across sites, information uncertainty, human-computer interaction, and fundamental rights assurance are among the most relevant challenges. If health AI is not designed resiliently with regard to these real-world data effects, potentially biased data-driven medical decisions can risk the safety and fundamental rights of millions of people. In this viewpoint, we review the challenges, requirements, and methods for resilient AI in health and provide a research framework to improve the trustworthiness of next-generation AI-based clinical decision support.
Collapse
Affiliation(s)
- Carlos Sáez
- Biomedical Data Science Lab, Instituto Universitario de Tecnologías de la Información y Comunicaciones, Universitat Politècnica de València, Valencia, Spain
| | - Pablo Ferri
- Biomedical Data Science Lab, Instituto Universitario de Tecnologías de la Información y Comunicaciones, Universitat Politècnica de València, Valencia, Spain
| | - Juan M García-Gómez
- Biomedical Data Science Lab, Instituto Universitario de Tecnologías de la Información y Comunicaciones, Universitat Politècnica de València, Valencia, Spain
| |
Collapse
|
4
|
Faust L, Wilson P, Asai S, Fu S, Liu H, Ruan X, Storlie C. Considerations for Quality Control Monitoring of Machine Learning Models in Clinical Practice. JMIR Med Inform 2024; 12:e50437. [PMID: 38941140 PMCID: PMC11245651 DOI: 10.2196/50437] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/07/2023] [Revised: 08/22/2023] [Accepted: 05/04/2024] [Indexed: 06/29/2024] Open
Abstract
Integrating machine learning (ML) models into clinical practice presents a challenge of maintaining their efficacy over time. While existing literature offers valuable strategies for detecting declining model performance, there is a need to document the broader challenges and solutions associated with the real-world development and integration of model monitoring solutions. This work details the development and use of a platform for monitoring the performance of a production-level ML model operating in Mayo Clinic. In this paper, we aimed to provide a series of considerations and guidelines necessary for integrating such a platform into a team's technical infrastructure and workflow. We have documented our experiences with this integration process, discussed the broader challenges encountered with real-world implementation and maintenance, and included the source code for the platform. Our monitoring platform was built as an R shiny application, developed and implemented over the course of 6 months. The platform has been used and maintained for 2 years and is still in use as of July 2023. The considerations necessary for the implementation of the monitoring platform center around 4 pillars: feasibility (what resources can be used for platform development?); design (through what statistics or models will the model be monitored, and how will these results be efficiently displayed to the end user?); implementation (how will this platform be built, and where will it exist within the IT ecosystem?); and policy (based on monitoring feedback, when and what actions will be taken to fix problems, and how will these problems be translated to clinical staff?). While much of the literature surrounding ML performance monitoring emphasizes methodological approaches for capturing changes in performance, there remains a battery of other challenges and considerations that must be addressed for successful real-world implementation.
Collapse
Affiliation(s)
- Louis Faust
- Robert D and Patricia E Kern Center for the Science of Health Care Delivery, Mayo Clinic, Rochester, MN, United States
| | - Patrick Wilson
- Robert D and Patricia E Kern Center for the Science of Health Care Delivery, Mayo Clinic, Rochester, MN, United States
| | - Shusaku Asai
- Robert D and Patricia E Kern Center for the Science of Health Care Delivery, Mayo Clinic, Rochester, MN, United States
| | - Sunyang Fu
- Department of Artificial Intelligence and Informatics, Mayo Clinic, Rochester, MN, United States
| | - Hongfang Liu
- Department of Artificial Intelligence and Informatics, Mayo Clinic, Rochester, MN, United States
| | - Xiaoyang Ruan
- Department of Artificial Intelligence and Informatics, Mayo Clinic, Rochester, MN, United States
| | - Curt Storlie
- Robert D and Patricia E Kern Center for the Science of Health Care Delivery, Mayo Clinic, Rochester, MN, United States
| |
Collapse
|
5
|
Eguale T, Bastardot F, Song W, Motta-Calderon D, Elsobky Y, Rui A, Marceau M, Davis C, Ganesan S, Alsubai A, Matthews M, Volk LA, Bates DW, Rozenblum R. A Machine Learning Application to Classify Patients at Differing Levels of Risk of Opioid Use Disorder: Clinician-Based Validation Study. JMIR Med Inform 2024; 12:e53625. [PMID: 38842167 PMCID: PMC11185289 DOI: 10.2196/53625] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/20/2023] [Revised: 03/15/2024] [Accepted: 04/20/2024] [Indexed: 06/07/2024] Open
Abstract
Background Despite restrictive opioid management guidelines, opioid use disorder (OUD) remains a major public health concern. Machine learning (ML) offers a promising avenue for identifying and alerting clinicians about OUD, thus supporting better clinical decision-making regarding treatment. Objective This study aimed to assess the clinical validity of an ML application designed to identify and alert clinicians of different levels of OUD risk by comparing it to a structured review of medical records by clinicians. Methods The ML application generated OUD risk alerts on outpatient data for 649,504 patients from 2 medical centers between 2010 and 2013. A random sample of 60 patients was selected from 3 OUD risk level categories (n=180). An OUD risk classification scheme and standardized data extraction tool were developed to evaluate the validity of the alerts. Clinicians independently conducted a systematic and structured review of medical records and reached a consensus on a patient's OUD risk level, which was then compared to the ML application's risk assignments. Results A total of 78,587 patients without cancer with at least 1 opioid prescription were identified as follows: not high risk (n=50,405, 64.1%), high risk (n=16,636, 21.2%), and suspected OUD or OUD (n=11,546, 14.7%). The sample of 180 patients was representative of the total population in terms of age, sex, and race. The interrater reliability between the ML application and clinicians had a weighted kappa coefficient of 0.62 (95% CI 0.53-0.71), indicating good agreement. Combining the high risk and suspected OUD or OUD categories and using the review of medical records as a gold standard, the ML application had a corrected sensitivity of 56.6% (95% CI 48.7%-64.5%) and a corrected specificity of 94.2% (95% CI 90.3%-98.1%). The positive and negative predictive values were 93.3% (95% CI 88.2%-96.3%) and 60.0% (95% CI 50.4%-68.9%), respectively. Key themes for disagreements between the ML application and clinician reviews were identified. Conclusions A systematic comparison was conducted between an ML application and clinicians for identifying OUD risk. The ML application generated clinically valid and useful alerts about patients' different OUD risk levels. ML applications hold promise for identifying patients at differing levels of OUD risk and will likely complement traditional rule-based approaches to generating alerts about opioid safety issues.
Collapse
Affiliation(s)
- Tewodros Eguale
- School of Pharmacy, Massachusetts College of Pharmacy and Health Sciences, Boston, MA, United States
- Division of General Internal Medicine, Brigham and Women's Hospital, Boston, MA, United States
| | - François Bastardot
- Innovation and Clinical Research Directorate, Lausanne University Hospital (CHUV), Lausanne, Switzerland
- Medical Directorate, Lausanne University Hospital (CHUV), Lausanne, Switzerland
| | - Wenyu Song
- Division of General Internal Medicine, Brigham and Women's Hospital, Boston, MA, United States
- Harvard Medical School, Boston, MA, United States
| | | | - Yasmin Elsobky
- Division of General Internal Medicine, Brigham and Women's Hospital, Boston, MA, United States
- Alexandria University, Alexandria, Egypt
| | - Angela Rui
- Division of General Internal Medicine, Brigham and Women's Hospital, Boston, MA, United States
| | - Marlika Marceau
- Clinical Quality and IS Analysis, Mass General Brigham, Somerville, MA, United States
| | - Clark Davis
- Division of General Internal Medicine, Brigham and Women's Hospital, Boston, MA, United States
| | - Sandya Ganesan
- Division of General Internal Medicine, Brigham and Women's Hospital, Boston, MA, United States
| | - Ava Alsubai
- Division of General Internal Medicine, Brigham and Women's Hospital, Boston, MA, United States
| | - Michele Matthews
- School of Pharmacy, Massachusetts College of Pharmacy and Health Sciences, Boston, MA, United States
- Department of Pharmacy, Brigham and Women's Hospital, Boston, MA, United States
| | - Lynn A Volk
- Clinical Quality and IS Analysis, Mass General Brigham, Somerville, MA, United States
| | - David W Bates
- Division of General Internal Medicine, Brigham and Women's Hospital, Boston, MA, United States
- Harvard Medical School, Boston, MA, United States
- Clinical Quality and IS Analysis, Mass General Brigham, Somerville, MA, United States
- Harvard TH Chan School of Public Health, Boston, MA, United States
| | - Ronen Rozenblum
- Division of General Internal Medicine, Brigham and Women's Hospital, Boston, MA, United States
- Harvard Medical School, Boston, MA, United States
| |
Collapse
|
6
|
Ge J, Buenaventura A, Berrean B, Purvis J, Fontil V, Lai JC, Pletcher MJ. Applying human-centered design to the construction of a cirrhosis management clinical decision support system. Hepatol Commun 2024; 8:e0394. [PMID: 38407255 PMCID: PMC10898661 DOI: 10.1097/hc9.0000000000000394] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/17/2023] [Accepted: 12/13/2023] [Indexed: 02/27/2024] Open
Abstract
BACKGROUND Electronic health record (EHR)-based clinical decision support is a scalable way to help standardize clinical care. Clinical decision support systems have not been extensively investigated in cirrhosis management. Human-centered design (HCD) is an approach that engages with potential users in intervention development. In this study, we applied HCD to design the features and interface for a clinical decision support system for cirrhosis management, called CirrhosisRx. METHODS We conducted technical feasibility assessments to construct a visual blueprint that outlines the basic features of the interface. We then convened collaborative-design workshops with generalist and specialist clinicians. We elicited current workflows for cirrhosis management, assessed gaps in existing EHR systems, evaluated potential features, and refined the design prototype for CirrhosisRx. At the conclusion of each workshop, we analyzed recordings and transcripts. RESULTS Workshop feedback showed that the aggregation of relevant clinical data into 6 cirrhosis decompensation domains (defined as common inpatient clinical scenarios) was the most important feature. Automatic inference of clinical events from EHR data, such as gastrointestinal bleeding from hemoglobin changes, was not accepted due to accuracy concerns. Visualizations for risk stratification scores were deemed not necessary. Lastly, the HCD co-design workshops allowed us to identify the target user population (generalists). CONCLUSIONS This is one of the first applications of HCD to design the features and interface for an electronic intervention for cirrhosis management. The HCD process altered features, modified the design interface, and likely improved CirrhosisRx's overall usability. The finalized design for CirrhosisRx proceeded to development and production and will be tested for effectiveness in a pragmatic randomized controlled trial. This work provides a model for the creation of other EHR-based interventions in hepatology care.
Collapse
Affiliation(s)
- Jin Ge
- Department of Medicine, Division of Gastroenterology and Hepatology, University of California—San Francisco, San Francisco, California, USA
| | - Ana Buenaventura
- School of Medicine Technology Services, University of California—San Francisco, San Francisco, California, USA
| | - Beth Berrean
- School of Medicine Technology Services, University of California—San Francisco, San Francisco, California, USA
| | - Jory Purvis
- School of Medicine Technology Services, University of California—San Francisco, San Francisco, California, USA
| | - Valy Fontil
- Family Health Centers, NYU-Langone Medical Center, Brooklyn, New York, USA
| | - Jennifer C. Lai
- Department of Medicine, Division of Gastroenterology and Hepatology, University of California—San Francisco, San Francisco, California, USA
| | - Mark J. Pletcher
- Department of Epidemiology and Biostatistics, University of California—San Francisco, San Francisco, California, USA
| |
Collapse
|
7
|
Walton NA, Nagarajan R, Wang C, Sincan M, Freimuth RR, Everman DB, Walton DC, McGrath SP, Lemas DJ, Benos PV, Alekseyenko AV, Song Q, Gamsiz Uzun E, Taylor CO, Uzun A, Person TN, Rappoport N, Zhao Z, Williams MS. Enabling the clinical application of artificial intelligence in genomics: a perspective of the AMIA Genomics and Translational Bioinformatics Workgroup. J Am Med Inform Assoc 2024; 31:536-541. [PMID: 38037121 PMCID: PMC10797281 DOI: 10.1093/jamia/ocad211] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/11/2023] [Revised: 10/09/2023] [Accepted: 10/26/2023] [Indexed: 12/02/2023] Open
Abstract
OBJECTIVE Given the importance AI in genomics and its potential impact on human health, the American Medical Informatics Association-Genomics and Translational Biomedical Informatics (GenTBI) Workgroup developed this assessment of factors that can further enable the clinical application of AI in this space. PROCESS A list of relevant factors was developed through GenTBI workgroup discussions in multiple in-person and online meetings, along with review of pertinent publications. This list was then summarized and reviewed to achieve consensus among the group members. CONCLUSIONS Substantial informatics research and development are needed to fully realize the clinical potential of such technologies. The development of larger datasets is crucial to emulating the success AI is achieving in other domains. It is important that AI methods do not exacerbate existing socio-economic, racial, and ethnic disparities. Genomic data standards are critical to effectively scale such technologies across institutions. With so much uncertainty, complexity and novelty in genomics and medicine, and with an evolving regulatory environment, the current focus should be on using these technologies in an interface with clinicians that emphasizes the value each brings to clinical decision-making.
Collapse
Affiliation(s)
- Nephi A Walton
- Division of Medical Genetics, University of Utah School of Medicine, Salt Lake City, UT 84112 ,United States
| | - Radha Nagarajan
- Enterprise Information Services, Cedars-Sinai Medical Center, Los Angeles, CA 90025, United States
- Information Services Department, Children’s Hospital of Orange County, Orange, CA 92868, United States
| | - Chen Wang
- Division of Computational Biology, Department of Quantitative Health Sciences, Center for Individualized Medicine, Mayo Clinic, Rochester, MN 55905, United States
| | - Murat Sincan
- Flatiron Health, New York, NY 10013, United States
- Department of Internal Medicine, Sanford School of Medicine, University of South Dakota, Sioux Falls, SD 57107, United States
| | - Robert R Freimuth
- Department of Artificial Intelligence and Informatics, Center for Individualized Medicine, Mayo Clinic, Rochester, MN 55905, United States
| | - David B Everman
- EverMed Genetics and Genomics Consulting LLC, Greenville, SC 29607, United States
| | | | - Scott P McGrath
- CITRIS Health, CITRIS and Banatao Institute, University of California Berkeley, Berkeley, CA 94720, United States
| | - Dominick J Lemas
- Department of Health Outcomes and Biomedical Informatics, University of Florida, Gainesville, FL 32610, United States
| | - Panayiotis V Benos
- Department of Epidemiology, University of Florida, Gainesville, FL 32610, United States
| | - Alexander V Alekseyenko
- Department of Public Health Sciences, Biomedical Informatics Center, Medical University of South Carolina, Charleston, SC 29403, United States
| | - Qianqian Song
- Department of Health Outcomes and Biomedical Informatics, University of Florida, Gainesville, FL 32610, United States
| | - Ece Gamsiz Uzun
- Department of Pathology and Laboratory Medicine, Rhode Island Hospital and Lifespan Medical Center, Providence, RI 02915, United States
- Department of Pathology and Laboratory Medicine, Warren Alpert Medical School of Brown University, Providence, RI 02915, United States
| | - Casey Overby Taylor
- Departments of Medicine and Biomedical Engineering, Johns Hopkins School of Medicine, Baltimore, MD 21205, United States
| | - Alper Uzun
- Department of Pathology and Laboratory Medicine, Warren Alpert Medical School of Brown University, Providence, RI 02915, United States
- Legorreta Cancer Center, Brown University, Providence, RI 02915, United States
| | - Thomas Nate Person
- Department of Bioinformatics and Genomics, Huck Institutes of the Life Sciences, Penn State University, Bloomsburg, PA 16802, United States
| | - Nadav Rappoport
- Department of Software and Information Systems Engineering, Ben-Gurion University of the Negev, Beer Sheva 8410501, Israel
| | - Zhongming Zhao
- Center for Precision Health, McWilliams School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, TX 77030, United States
| | - Marc S Williams
- Department of Genomic Health, Geisinger, Danville, PA 17822, United States
| |
Collapse
|
8
|
Liu S, Wei S, Lehmann HP. Applicability Area: A novel utility-based approach for evaluating predictive models, beyond discrimination. AMIA ... ANNUAL SYMPOSIUM PROCEEDINGS. AMIA SYMPOSIUM 2024; 2023:494-503. [PMID: 38222359 PMCID: PMC10785877] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Subscribe] [Scholar Register] [Indexed: 01/16/2024]
Abstract
Translating prediction models into practice and supporting clinicians' decision-making demand demonstration of clinical value. Existing approaches to evaluating machine learning models emphasize discriminatory power, which is only a part of the medical decision problem. We propose the Applicability Area (ApAr), a decision-analytic utility-based approach to evaluating predictive models that communicate the range of prior probability and test cutoffs for which the model has positive utility; larger ApArs suggest a broader potential use of the model. We assess ApAr with simulated datasets and with three published medical datasets. ApAr adds value beyond the typical area under the receiver operating characteristic curve (AUROC) metric analysis. As an example, in the diabetes dataset, the top model by ApAr was ranked as the 23rd best model by AUROC. Decision makers looking to adopt and implement models can leverage ApArs to assess if the local range of priors and utilities is within the respective ApArs.
Collapse
Affiliation(s)
- Star Liu
- Johns Hopkins University School of Medicine, Baltimore, MD, United States
| | - Shixiong Wei
- Johns Hopkins University School of Medicine, Baltimore, MD, United States
| | - Harold P Lehmann
- Johns Hopkins University School of Medicine, Baltimore, MD, United States
| |
Collapse
|
9
|
Cho I, Cho J, Hong JH, Choe WS, Shin H. Utilizing standardized nursing terminologies in implementing an AI-powered fall-prevention tool to improve patient outcomes: a multihospital study. J Am Med Inform Assoc 2023; 30:1826-1836. [PMID: 37507147 PMCID: PMC10586045 DOI: 10.1093/jamia/ocad145] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2023] [Revised: 07/14/2023] [Accepted: 07/17/2023] [Indexed: 07/30/2023] Open
Abstract
OBJECTIVES Standardized nursing terminologies (SNTs) are necessary to ensure consistent knowledge expression and compare the effectiveness of nursing practice across settings. This study investigated whether SNTs can support semantic interoperability and outcoming tracking over time by implementing an AI-powered CDS tool for fall prevention across multiple EMR systems. MATERIALS AND METHODS The study involved 3 tertiary academic hospitals and 1 public hospital with different EMR systems and nursing terms, and employed an AI-powered CDS tool that determines the fall risk within the next hour (prediction model) and recommends tailored care plans (CDS functions; represented by SNTs). The prediction model was mapped to local data elements and optimized using local data sets. The local nursing statements in CDS functions were mapped using an ICNP-based inpatient fall-prevention catalog. Four implementation models were compared, and patient outcomes and nursing activities were observed longitudinally at one site. RESULTS The postimplementation approach was practical for disseminating the AI-powered CDS tool for nursing. The 4 hospitals successfully implemented prediction models with little performance variation; the AUROCs were 0.8051-0.9581. The nursing process data contributed markedly to fall-risk predictions. The local nursing statements on preventing falls covered 48.0%-86.7% of statements. There was no significant longitudinal decrease in the fall rate (P = .160, 95% CI = -1.21 to 0.21 per 1000 hospital days), but rates of interventions provided by nurses were notably increased. CONCLUSION SNTs contributed to achieving semantic interoperability among multiple EMR systems to disseminate AI-powered CDS tools and automatically track nursing and patient outcomes.
Collapse
Affiliation(s)
- Insook Cho
- Nursing Department, Inha University, Incheon, Republic of Korea
- Division of General Internal Medicine, The Center for Patient Safety Research and Practice, Brigham and Women’s Hospital, Boston, Massachusetts, USA
| | - Jiseon Cho
- Department of Nursing, National Health Insurance Service Ilsan Hospital, Gyeonggi-do, Republic of Korea
| | - Jeong Hee Hong
- Department of Nursing, Samsung Medical Center, Seoul, Republic of Korea
| | - Wha Suk Choe
- Department of Nursing, Inha University Hospital, Incheon, Republic of Korea
| | - HyeKyeong Shin
- Graduate School, Nursing Department, Inha University, Incheon, Republic of Korea
| |
Collapse
|
10
|
Lee T, Lukac PJ, Vangala S, Kowsari K, Vu V, Fogelman S, Pfeffer MA, Bell DS. Evaluating the predictive ability of natural language processing in identifying tertiary/quaternary cases in prioritization workflows for interhospital transfer. JAMIA Open 2023; 6:ooad069. [PMID: 37600073 PMCID: PMC10435371 DOI: 10.1093/jamiaopen/ooad069] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2023] [Revised: 07/26/2023] [Accepted: 08/06/2023] [Indexed: 08/22/2023] Open
Abstract
Objectives Tertiary and quaternary (TQ) care refers to complex cases requiring highly specialized health services. Our study aimed to compare the ability of a natural language processing (NLP) model to an existing human workflow in predictively identifying TQ cases for transfer requests to an academic health center. Materials and methods Data on interhospital transfers were queried from the electronic health record for the 6-month period from July 1, 2020 to December 31, 2020. The NLP model was allowed to generate predictions on the same cases as the human predictive workflow during the study period. These predictions were then retrospectively compared to the true TQ outcomes. Results There were 1895 transfer cases labeled by both the human predictive workflow and the NLP model, all of which had retrospective confirmation of the true TQ label. The NLP model receiver operating characteristic curve had an area under the curve of 0.91. Using a model probability threshold of ≥0.3 to be considered TQ positive, accuracy was 81.5% for the NLP model versus 80.3% for the human predictions (P = .198) while sensitivity was 83.6% versus 67.7% (P<.001). Discussion The NLP model was as accurate as the human workflow but significantly more sensitive. This translated to 15.9% more TQ cases identified by the NLP model. Conclusion Integrating an NLP model into existing workflows as automated decision support could translate to more TQ cases identified at the onset of the transfer process.
Collapse
Affiliation(s)
- Timothy Lee
- Altamed Health Services, Commerce, CA, United States
| | - Paul J Lukac
- Department of Pediatrics, University of California, Los Angeles, Los Angeles, CA, United States
- Office of Health Informatics and Analytics, University of California, Los Angeles, Los Angeles, CA, United States
| | - Sitaram Vangala
- Department of Medicine Statistics Core, University of California, Los Angeles, Los Angeles, CA, United States
| | - Kamran Kowsari
- Office of Health Informatics and Analytics, University of California, Los Angeles, Los Angeles, CA, United States
| | - Vu Vu
- Office of Health Informatics and Analytics, University of California, Los Angeles, Los Angeles, CA, United States
| | | | - Michael A Pfeffer
- Department of Medicine, Stanford University, Palo Alto, CA, United States
| | - Douglas S Bell
- Department of Medicine, University of California, Los Angeles, Los Angeles, CA, United States
| |
Collapse
|
11
|
Douthit BJ, McCoy AB, Nelson SD. The Impact of Clinical Decision Support on Health Disparities and the Digital Divide. Yearb Med Inform 2023; 32:169-178. [PMID: 37414030 PMCID: PMC10751127 DOI: 10.1055/s-0043-1768722] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/08/2023] Open
Abstract
OBJECTIVES This literature review summarizes relevant studies from the last three years (2020-2022) related to clinical decision support (CDS) and CDS impact on health disparities and the digital divide. This survey identifies current trends and synthesizes evidence-based recommendations and considerations for future development and implementation of CDS tools. METHODS We conducted a search in PubMed for literature published between 2020 and 2022. Our search strategy was constructed as a combination of the MEDLINE®/PubMed® Health Disparities and Minority Health Search Strategy and relevant CDS MeSH terms and phrases. We then extracted relevant data from the studies, including priority population when applicable, domain of influence on the disparity being addressed, and the type of CDS being used. We also made note of when a study discussed the digital divide in some capacity and organized the comments into general themes through group discussion. RESULTS Our search yielded 520 studies, with 45 included at the conclusion of screening. The most frequent CDS type in this review was point-of-care alerts/reminders (33.3%). Health Care System was the most frequent domain of influence (71.1%), and Blacks/African Americans were the most frequently included priority population (42.2%). Throughout the literature, we found four general themes related to the technology divide: inaccessibility of technology, access to care, trust of technology, and technology literacy.This survey revealed the diversity of CDS being used to address health disparities and several barriers which may make CDS less effective or potentially harmful to certain populations. Regular examinations of literature that feature CDS and address health disparities can help to reveal new strategies and patterns for improving healthcare.
Collapse
Affiliation(s)
- Brian J. Douthit
- Post-Doctoral Research Fellow: United States Department of Veterans Affairs, Vanderbilt University, Nashville, TN, USA
| | - Allison B. McCoy
- Assistant Professor: Department of Biomedical Informatics, Vanderbilt University, Nashville, TN, USA
- Director: Clinical Informatics Core, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Scott D. Nelson
- Associate Professor: Department of Biomedical Informatics, Vanderbilt University, Nashville, TN, USA
- Program Director: MS in Applied Clinical Informatics Program (MS-ACI), Vanderbilt University, Nashville, TN, USA
- Clinical Director: HealthIT, Vanderbilt University Medical Center, Nashville, TN, USA
| |
Collapse
|
12
|
Loftus TJ, Altieri MS, Balch JA, Abbott KL, Choi J, Marwaha JS, Hashimoto DA, Brat GA, Raftopoulos Y, Evans HL, Jackson GP, Walsh DS, Tignanelli CJ. Artificial Intelligence-enabled Decision Support in Surgery: State-of-the-art and Future Directions. Ann Surg 2023; 278:51-58. [PMID: 36942574 DOI: 10.1097/sla.0000000000005853] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/23/2023]
Abstract
OBJECTIVE To summarize state-of-the-art artificial intelligence-enabled decision support in surgery and to quantify deficiencies in scientific rigor and reporting. BACKGROUND To positively affect surgical care, decision-support models must exceed current reporting guideline requirements by performing external and real-time validation, enrolling adequate sample sizes, reporting model precision, assessing performance across vulnerable populations, and achieving clinical implementation; the degree to which published models meet these criteria is unknown. METHODS Embase, PubMed, and MEDLINE databases were searched from their inception to September 21, 2022 for articles describing artificial intelligence-enabled decision support in surgery that uses preoperative or intraoperative data elements to predict complications within 90 days of surgery. Scientific rigor and reporting criteria were assessed and reported according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews guidelines. RESULTS Sample size ranged from 163-2,882,526, with 8/36 articles (22.2%) featuring sample sizes of less than 2000; 7 of these 8 articles (87.5%) had below-average (<0.83) area under the receiver operating characteristic or accuracy. Overall, 29 articles (80.6%) performed internal validation only, 5 (13.8%) performed external validation, and 2 (5.6%) performed real-time validation. Twenty-three articles (63.9%) reported precision. No articles reported performance across sociodemographic categories. Thirteen articles (36.1%) presented a framework that could be used for clinical implementation; none assessed clinical implementation efficacy. CONCLUSIONS Artificial intelligence-enabled decision support in surgery is limited by reliance on internal validation, small sample sizes that risk overfitting and sacrifice predictive performance, and failure to report confidence intervals, precision, equity analyses, and clinical implementation. Researchers should strive to improve scientific quality.
Collapse
Affiliation(s)
- Tyler J Loftus
- Department of Surgery, University of Florida Health, Gainesville, FL
- American College of Surgeons Health Information Technology Committee and Artificial Intelligence Subcommittee, Chicago, IL
| | - Maria S Altieri
- American College of Surgeons Health Information Technology Committee and Artificial Intelligence Subcommittee, Chicago, IL
- Department of Surgery, University of Pennsylvania, Philadelphia, PA
| | - Jeremy A Balch
- Department of Surgery, University of Florida Health, Gainesville, FL
- American College of Surgeons Health Information Technology Committee and Artificial Intelligence Subcommittee, Chicago, IL
| | - Kenneth L Abbott
- Department of Surgery, University of Florida Health, Gainesville, FL
- American College of Surgeons Health Information Technology Committee and Artificial Intelligence Subcommittee, Chicago, IL
| | - Jeff Choi
- American College of Surgeons Health Information Technology Committee and Artificial Intelligence Subcommittee, Chicago, IL
- Department of Surgery, Stanford University, Stanford, CA
| | - Jayson S Marwaha
- American College of Surgeons Health Information Technology Committee and Artificial Intelligence Subcommittee, Chicago, IL
- Department of Surgery, Beth Israel Deaconess Medical Center
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA
| | - Daniel A Hashimoto
- American College of Surgeons Health Information Technology Committee and Artificial Intelligence Subcommittee, Chicago, IL
- Department of Surgery, University of Pennsylvania Perelman School of Medicine
- General Robotics, Automation, Sensing, and Perception Laboratory, University of Pennsylvania School of Engineering and Applied Science, Philadelphia, PA
| | - Gabriel A Brat
- American College of Surgeons Health Information Technology Committee and Artificial Intelligence Subcommittee, Chicago, IL
- Department of Surgery, Beth Israel Deaconess Medical Center
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA
| | - Yannis Raftopoulos
- American College of Surgeons Health Information Technology Committee and Artificial Intelligence Subcommittee, Chicago, IL
- Weight Management Program, Holyoke Medical Center, Holyoke, MA
| | - Heather L Evans
- American College of Surgeons Health Information Technology Committee and Artificial Intelligence Subcommittee, Chicago, IL
- Department of Surgery, Medical University of South Carolina, Charleston, SC
| | - Gretchen P Jackson
- American College of Surgeons Health Information Technology Committee and Artificial Intelligence Subcommittee, Chicago, IL
- Digital, Intuitive Surgical, Sunnyvale, CA; Departments of Pediatric Surgery, Pediatrics, and Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN
| | - Danielle S Walsh
- American College of Surgeons Health Information Technology Committee and Artificial Intelligence Subcommittee, Chicago, IL
- Department of Surgery, University of Kentucky, Lexington, KY
| | - Christopher J Tignanelli
- American College of Surgeons Health Information Technology Committee and Artificial Intelligence Subcommittee, Chicago, IL
- Department of Surgery
- Institute for Health Informatics
- Program for Clinical Artificial Intelligence, Center for Learning Health Systems Science, University of Minnesota, Minneapolis, MN
| |
Collapse
|
13
|
Novak LL, Russell RG, Garvey K, Patel M, Thomas Craig KJ, Snowdon J, Miller B. Clinical use of artificial intelligence requires AI-capable organizations. JAMIA Open 2023; 6:ooad028. [PMID: 37152469 PMCID: PMC10155810 DOI: 10.1093/jamiaopen/ooad028] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2022] [Revised: 03/18/2023] [Accepted: 04/11/2023] [Indexed: 05/09/2023] Open
Abstract
Artificial intelligence-based algorithms are being widely implemented in health care, even as evidence is emerging of bias in their design, problems with implementation, and potential harm to patients. To achieve the promise of using of AI-based tools to improve health, healthcare organizations will need to be AI-capable, with internal and external systems functioning in tandem to ensure the safe, ethical, and effective use of AI-based tools. Ideas are starting to emerge about the organizational routines, competencies, resources, and infrastructures that will be required for safe and effective deployment of AI in health care, but there has been little empirical research. Infrastructures that provide legal and regulatory guidance for managers, clinician competencies for the safe and effective use of AI-based tools, and learner-centric resources such as clear AI documentation and local health ecosystem impact reviews can help drive continuous improvement.
Collapse
Affiliation(s)
- Laurie Lovett Novak
- Corresponding Author: Laurie Lovett Novak, PhD, MHSA, Department of Biomedical Informatics, Vanderbilt University Medical Center, 2525 West End Ave, Suite 1475, Nashville, TN 37203, USA;
| | - Regina G Russell
- Department of Medical Education and Administration and Office of Undergraduate Medical Education, Vanderbilt University School of Medicine, Nashville, Tennessee, USA
| | - Kim Garvey
- Department of Anesthesiology and the Center for Advanced Mobile Healthcare Learning, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - Mehool Patel
- Department of Internal Medicine, Northeastern Ohio Medical University (NEOMED), Rootstown, Ohio, USA
- Department of Internal Medicine, Western Reserve Hospital, Cuyahoga Falls, Ohio, USA
| | - Kelly Jean Thomas Craig
- Clinical Evidence Development, Aetna®, Medical Affairs CVS Health®, Wellesley, Massachusetts, USA
| | - Jane Snowdon
- Corporate Technical Strategy, IBM® Corporation, Yorktown Heights, New York, USA
| | - Bonnie Miller
- Department of Medical Education and Administration and Center for Advanced Mobile Healthcare Learning, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| |
Collapse
|
14
|
Zygmont ME, Ikuta I, Nguyen XV, Frigini LAR, Segovis C, Naeger DM. Clinical Decision Support: Impact on Appropriate Imaging Utilization. Acad Radiol 2023; 30:1433-1440. [PMID: 36336523 DOI: 10.1016/j.acra.2022.10.006] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/02/2022] [Revised: 10/01/2022] [Accepted: 10/03/2022] [Indexed: 11/06/2022]
Affiliation(s)
- Matthew E Zygmont
- Department of Radiology and Imaging Sciences, Emory University School of Medicine, Atlanta, Georgia.
| | - Ichiro Ikuta
- Department of Radiology & Biomedical Imaging, Neuroradiology, Yale University School of Medicine, New Haven, Connecticut
| | - Xuan V Nguyen
- Department of Radiology, The Ohio State University Wexner Medical Center, Columbus, Ohio
| | | | - Colin Segovis
- Department of Radiology and Imaging Sciences, Emory University School of Medicine, Atlanta, Georgia
| | - David M Naeger
- Denver Health and Hospital Authority, Department of Radiology, Denver CO, and the University of Colorado School of Medicine, Aurora, Colorado
| |
Collapse
|
15
|
Russell RG, Lovett Novak L, Patel M, Garvey KV, Craig KJT, Jackson GP, Moore D, Miller BM. Competencies for the Use of Artificial Intelligence-Based Tools by Health Care Professionals. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2023; 98:348-356. [PMID: 36731054 DOI: 10.1097/acm.0000000000004963] [Citation(s) in RCA: 34] [Impact Index Per Article: 34.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/18/2023]
Abstract
PURPOSE The expanded use of clinical tools that incorporate artificial intelligence (AI) methods has generated calls for specific competencies for effective and ethical use. This qualitative study used expert interviews to define AI-related clinical competencies for health care professionals. METHOD In 2021, a multidisciplinary team interviewed 15 experts in the use of AI-based tools in health care settings about the clinical competencies health care professionals need to work effectively with such tools. Transcripts of the semistructured interviews were coded and thematically analyzed. Draft competency statements were developed and provided to the experts for feedback. The competencies were finalized using a consensus process across the research team. RESULTS Six competency domain statements and 25 subcompetencies were formulated from the thematic analysis. The competency domain statements are: (1) basic knowledge of AI: explain what AI is and describe its health care applications; (2) social and ethical implications of AI: explain how social, economic, and political systems influence AI-based tools and how these relationships impact justice, equity, and ethics; (3) AI-enhanced clinical encounters: carry out AI-enhanced clinical encounters that integrate diverse sources of information in creating patient-centered care plans; (4) evidence-based evaluation of AI-based tools: evaluate the quality, accuracy, safety, contextual appropriateness, and biases of AI-based tools and their underlying data sets in providing care to patients and populations; (5) workflow analysis for AI-based tools: analyze and adapt to changes in teams, roles, responsibilities, and workflows resulting from implementation of AI-based tools; and (6) practice-based learning and improvement regarding AI-based tools: participate in continuing professional development and practice-based improvement activities related to use of AI tools in health care. CONCLUSIONS The 6 clinical competencies identified can be used to guide future teaching and learning programs to maximize the potential benefits of AI-based tools and diminish potential harms.
Collapse
Affiliation(s)
- Regina G Russell
- R.G. Russell is director of learning system outcomes, Office of Undergraduate Medical Education, and assistant professor of medical education and administration, Vanderbilt University School of Medicine, Nashville Tennessee; ORCID: https://orcid.org/0000-0002-5540-7073
| | - Laurie Lovett Novak
- L.L. Novak is director, Center of Excellence in Applied Artificial Intelligence, Vanderbilt University Medical Center, and associate professor of biomedical informatics, Vanderbilt University School of Medicine, Nashville, Tennessee; ORCID: https://orcid.org/0000-0002-0415-4301
| | - Mehool Patel
- M. Patel is associate chief health officer and chief medical officer of provider analytics, IBM Watson Health, Cambridge, Massachusetts, and clinical professor, Northeast Ohio Medical University, Rootstown, Ohio
| | - Kim V Garvey
- K.V. Garvey is research instructor in anesthesiology, Vanderbilt University School of Medicine, and director of operations, Center for Advanced Mobile Healthcare Learning, Vanderbilt University Medical Center, Nashville, Tennessee; ORCID: https://orcid.org/0000-0002-2427-0182
| | - Kelly Jean Thomas Craig
- K.J.T. Craig is lead director, Clinical Evidence Development, Aetna Medical Affairs, CVS Health. At the time this work was completed, the author was deputy chief science officer of evidence-based practice, Center for AI, Research, and Evaluation, IBM Watson Health, Cambridge, Massachusetts; ORCID: https://orcid.org/0000-0002-9954-2795
| | - Gretchen P Jackson
- G.P. Jackson is vice president and scientific medical officer, Intuitive Surgical, Sunnyvale, California, and associate professor of surgery, pediatrics, and biomedical informatics, Vanderbilt University School of Medicine, Nashville, Tennessee. At the beginning of this work, the author was vice president and chief science officer, IBM Watson Health, Cambridge, Massachusetts; ORCID: https://orcid.org/0000-0002-3242-8058
| | - Don Moore
- D. Moore is emeritus professor of medical education and administration, Vanderbilt University School of Medicine, Nashville, Tennessee
| | - Bonnie M Miller
- B.M. Miller is professor of medical education and administration, Vanderbilt University School of Medicine, and director, Center for Advanced Mobile Healthcare Learning, Vanderbilt University Medical Center, Nashville, Tennessee; ORCID: https://orcid.org/0000-0002-7333-3389
| |
Collapse
|
16
|
Ramgopal S, Sanchez-Pinto LN, Horvat CM, Carroll MS, Luo Y, Florin TA. Artificial intelligence-based clinical decision support in pediatrics. Pediatr Res 2023; 93:334-341. [PMID: 35906317 PMCID: PMC9668209 DOI: 10.1038/s41390-022-02226-1] [Citation(s) in RCA: 33] [Impact Index Per Article: 33.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/21/2022] [Revised: 06/29/2022] [Accepted: 07/18/2022] [Indexed: 11/24/2022]
Abstract
Machine learning models may be integrated into clinical decision support (CDS) systems to identify children at risk of specific diagnoses or clinical deterioration to provide evidence-based recommendations. This use of artificial intelligence models in clinical decision support (AI-CDS) may have several advantages over traditional "rule-based" CDS models in pediatric care through increased model accuracy, with fewer false alerts and missed patients. AI-CDS tools must be appropriately developed, provide insight into the rationale behind decisions, be seamlessly integrated into care pathways, be intuitive to use, answer clinically relevant questions, respect the content expertise of the healthcare provider, and be scientifically sound. While numerous machine learning models have been reported in pediatric care, their integration into AI-CDS remains incompletely realized to date. Important challenges in the application of AI models in pediatric care include the relatively lower rates of clinically significant outcomes compared to adults, and the lack of sufficiently large datasets available necessary for the development of machine learning models. In this review article, we summarize key concepts related to AI-CDS, its current application to pediatric care, and its potential benefits and risks. IMPACT: The performance of clinical decision support may be enhanced by the utilization of machine learning-based algorithms to improve the predictive performance of underlying models. Artificial intelligence-based clinical decision support (AI-CDS) uses models that are experientially improved through training and are particularly well suited toward high-dimensional data. The application of AI-CDS toward pediatric care remains limited currently but represents an important area of future research.
Collapse
Affiliation(s)
- Sriram Ramgopal
- Division of Emergency Medicine, Ann & Robert H. Lurie Children's Hospital of Chicago, Department of Pediatrics, Northwestern University Feinberg School of Medicine, Chicago, IL, USA.
| | - L. Nelson Sanchez-Pinto
- grid.16753.360000 0001 2299 3507Division of Critical Care Medicine, Ann & Robert H. Lurie Children’s Hospital of Chicago, Department of Pediatrics, Northwestern University Feinberg School of Medicine, Chicago, IL USA ,grid.16753.360000 0001 2299 3507Department of Preventive Medicine (Health and Biomedical Informatics), Feinberg School of Medicine, Northwestern University, Chicago, IL USA
| | - Christopher M. Horvat
- grid.21925.3d0000 0004 1936 9000Department of Critical Care Medicine, UPMC Children’s Hospital of Pittsburgh, University of Pittsburgh School of Medicine, Pittsburgh, PA USA
| | - Michael S. Carroll
- grid.16753.360000 0001 2299 3507Data Analytics and Reporting, Ann & Robert H. Lurie Children’s Hospital of Chicago, Department of Pediatrics, Northwestern University Feinberg School of Medicine, Chicago, IL USA
| | - Yuan Luo
- grid.16753.360000 0001 2299 3507Department of Preventive Medicine (Health and Biomedical Informatics), Feinberg School of Medicine, Northwestern University, Chicago, IL USA
| | - Todd A. Florin
- grid.16753.360000 0001 2299 3507Division of Emergency Medicine, Ann & Robert H. Lurie Children’s Hospital of Chicago, Department of Pediatrics, Northwestern University Feinberg School of Medicine, Chicago, IL USA
| |
Collapse
|
17
|
Petersen C, Berner ES, Cardillo A, Fultz Hollis K, Goodman KW, Koppel R, Korngiebel DM, Lehmann CU, Solomonides AE, Subbian V. AMIA's code of professional and ethical conduct 2022. J Am Med Inform Assoc 2022; 30:3-7. [PMID: 36228119 PMCID: PMC9748526 DOI: 10.1093/jamia/ocac192] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2022] [Accepted: 10/06/2022] [Indexed: 01/24/2023] Open
Affiliation(s)
- Carolyn Petersen
- Department of Artificial Intelligence and Informatics, Mayo Clinic, Rochester, Minnesota, USA
| | - Eta S Berner
- Department of Health Services Administration, University of Alabama at Birmingham, Birmingham, Alabama, USA
| | - Anthony Cardillo
- Department of Emergency Medicine, NYU Langone Health, New York, New York, USA
| | - Kate Fultz Hollis
- Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland, Oregon, USA
| | - Kenneth W Goodman
- Institute for Bioethics and Health Policy, University of Miami Miller School of Medicine, Miami, Florida, USA
| | - Ross Koppel
- Department of Sociology and Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Department of Biomedical Informatics, Jacob’s School of Medicine, University of Buffalo (SUNY), Buffalo, New York, USA
| | - Diane M Korngiebel
- Google, LLC, Mountain View, California, USA
- Department of Biomedical Informatics & Medical Education, University of Washington School of Medicine, Seattle, Washington, USA
| | - Christoph U Lehmann
- Clinical Informatics Center, Departments of Pediatrics, Population & Data Science, and Bioinformatics, University of Texas Southwestern Medical Center, Dallas, Texas, USA
| | - Anthony E Solomonides
- Outcomes Research Network, Research Institute, NorthShore University HealthSystem, Evanston, Illinois, USA
| | - Vignesh Subbian
- Department of Biomedical Engineering, The University of Arizona, Tucson, Arizona, USA
- Department of Systems & Industrial Engineering, The University of Arizona, Tucson, Arizona, USA
| |
Collapse
|
18
|
Salwei ME, Carayon P. A Sociotechnical Systems Framework for the Application of Artificial Intelligence in Health Care Delivery. JOURNAL OF COGNITIVE ENGINEERING AND DECISION MAKING 2022; 16:194-206. [PMID: 36704421 PMCID: PMC9873227 DOI: 10.1177/15553434221097357] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/14/2023]
Abstract
In the coming years, artificial intelligence (AI) will pervade almost every aspect of the health care delivery system. AI has the potential to improve patient safety (e.g. diagnostic accuracy) as well as reduce the burden on clinicians (e.g. documentation-related workload); however, these benefits are yet to be realized. AI is only one element of a larger sociotechnical system that needs to be considered for effective AI application. In this paper, we describe the current challenges of integrating AI into clinical care and propose a sociotechnical systems (STS) approach for AI design and implementation. We demonstrate the importance of an STS approach through a case study on the design and implementation of a clinical decision support (CDS). In order for AI to reach its potential, the entire work system as well as clinical workflow must be systematically considered throughout the design of AI technology.
Collapse
Affiliation(s)
- Megan E. Salwei
- Center for Research and Innovation in Systems Safety, Department of Anesthesiology, Vanderbilt University Medical Center, Nashville, TN
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN
| | - Pascale Carayon
- Department of Industrial and Systems Engineering, University of Wisconsin-Madison, Madison, WI
- Wisconsin Institute for Healthcare Systems Engineering, University of Wisconsin-Madison, Madison, WI
| |
Collapse
|
19
|
Davis SE, Walsh CG, Matheny ME. Open questions and research gaps for monitoring and updating AI-enabled tools in clinical settings. Front Digit Health 2022; 4:958284. [PMID: 36120717 PMCID: PMC9478183 DOI: 10.3389/fdgth.2022.958284] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2022] [Accepted: 08/11/2022] [Indexed: 11/15/2022] Open
Abstract
As the implementation of artificial intelligence (AI)-enabled tools is realized across diverse clinical environments, there is a growing understanding of the need for ongoing monitoring and updating of prediction models. Dataset shift-temporal changes in clinical practice, patient populations, and information systems-is now well-documented as a source of deteriorating model accuracy and a challenge to the sustainability of AI-enabled tools in clinical care. While best practices are well-established for training and validating new models, there has been limited work developing best practices for prospective validation and model maintenance. In this paper, we highlight the need for updating clinical prediction models and discuss open questions regarding this critical aspect of the AI modeling lifecycle in three focus areas: model maintenance policies, performance monitoring perspectives, and model updating strategies. With the increasing adoption of AI-enabled tools, the need for such best practices must be addressed and incorporated into new and existing implementations. This commentary aims to encourage conversation and motivate additional research across clinical and data science stakeholders.
Collapse
Affiliation(s)
- Sharon E. Davis
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, United States
| | - Colin G. Walsh
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, United States
- Department of Medicine, Vanderbilt University Medical Center, Nashville, TN, United States
- Department of Psychiatry, Vanderbilt University Medical Center, Nashville, TN, United States
| | - Michael E. Matheny
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, United States
- Department of Medicine, Vanderbilt University Medical Center, Nashville, TN, United States
- Department of Biostatistics, Vanderbilt University Medical Center, Nashville, TN, United States
- Tennessee Valley Healthcare System VA Medical Center, Veterans Health Administration, Nashville, TN, United States
| |
Collapse
|
20
|
Ackermann K, Baker J, Festa M, McMullan B, Westbrook J, Li L. Computerized Clinical Decision Support Systems for the Early Detection of Sepsis Among Pediatric, Neonatal, and Maternal Inpatients: Scoping Review. JMIR Med Inform 2022; 10:e35061. [PMID: 35522467 PMCID: PMC9123549 DOI: 10.2196/35061] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/02/2021] [Revised: 02/27/2022] [Accepted: 03/19/2022] [Indexed: 11/13/2022] Open
Abstract
BACKGROUND Sepsis is a severe condition associated with extensive morbidity and mortality worldwide. Pediatric, neonatal, and maternal patients represent a considerable proportion of the sepsis burden. Identifying sepsis cases as early as possible is a key pillar of sepsis management and has prompted the development of sepsis identification rules and algorithms that are embedded in computerized clinical decision support (CCDS) systems. OBJECTIVE This scoping review aimed to systematically describe studies reporting on the use and evaluation of CCDS systems for the early detection of pediatric, neonatal, and maternal inpatients at risk of sepsis. METHODS MEDLINE, Embase, CINAHL, Cochrane, Latin American and Caribbean Health Sciences Literature (LILACS), Scopus, Web of Science, OpenGrey, ClinicalTrials.gov, and ProQuest Dissertations and Theses Global (PQDT) were searched by using a search strategy that incorporated terms for sepsis, clinical decision support, and early detection. Title, abstract, and full-text screening was performed by 2 independent reviewers, who consulted a third reviewer as needed. One reviewer performed data charting with a sample of data. This was checked by a second reviewer and via discussions with the review team, as necessary. RESULTS A total of 33 studies were included in this review-13 (39%) pediatric studies, 18 (55%) neonatal studies, and 2 (6%) maternal studies. All studies were published after 2011, and 27 (82%) were published from 2017 onward. The most common outcome investigated in pediatric studies was the accuracy of sepsis identification (9/13, 69%). Pediatric CCDS systems used different combinations of 18 diverse clinical criteria to detect sepsis across the 13 identified studies. In neonatal studies, 78% (14/18) of the studies investigated the Kaiser Permanente early-onset sepsis risk calculator. All studies investigated sepsis treatment and management outcomes, with 83% (15/18) reporting on antibiotics-related outcomes. Usability and cost-related outcomes were each reported in only 2 (6%) of the 31 pediatric or neonatal studies. Both studies on maternal populations were short abstracts. CONCLUSIONS This review found limited research investigating CCDS systems to support the early detection of sepsis among pediatric, neonatal, and maternal patients, despite the high burden of sepsis in these vulnerable populations. We have highlighted the need for a consensus definition for pediatric and neonatal sepsis and the study of usability and cost-related outcomes as critical areas for future research. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID) RR2-10.2196/24899.
Collapse
Affiliation(s)
- Khalia Ackermann
- Centre for Health Systems and Safety Research, Australian Institute of Health Innovation, Macquarie University, Australia
| | - Jannah Baker
- Centre for Health Systems and Safety Research, Australian Institute of Health Innovation, Macquarie University, Australia
| | - Marino Festa
- Kids Critical Care Research, Department of Paediatric Intensive Care, Children's Hospital at Westmead, Sydney, Australia
| | - Brendan McMullan
- Department of Immunology and Infectious Diseases, Sydney Children's Hospital, Randwick, Sydney, Australia
- Faculty of Medicine & Health, University of New South Wales, Sydney, Australia
| | - Johanna Westbrook
- Centre for Health Systems and Safety Research, Australian Institute of Health Innovation, Macquarie University, Australia
| | - Ling Li
- Centre for Health Systems and Safety Research, Australian Institute of Health Innovation, Macquarie University, Australia
| |
Collapse
|
21
|
Solomonides AE, Koski E, Atabaki SM, Weinberg S, McGreevey JD, Kannry JL, Petersen C, Lehmann CU. Defining AMIA's artificial intelligence principles. J Am Med Inform Assoc 2022; 29:585-591. [PMID: 35190824 PMCID: PMC8922174 DOI: 10.1093/jamia/ocac006] [Citation(s) in RCA: 41] [Impact Index Per Article: 20.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2022] [Accepted: 01/14/2022] [Indexed: 08/08/2023] Open
Abstract
Recent advances in the science and technology of artificial intelligence (AI) and growing numbers of deployed AI systems in healthcare and other services have called attention to the need for ethical principles and governance. We define and provide a rationale for principles that should guide the commission, creation, implementation, maintenance, and retirement of AI systems as a foundation for governance throughout the lifecycle. Some principles are derived from the familiar requirements of practice and research in medicine and healthcare: beneficence, nonmaleficence, autonomy, and justice come first. A set of principles follow from the creation and engineering of AI systems: explainability of the technology in plain terms; interpretability, that is, plausible reasoning for decisions; fairness and absence of bias; dependability, including "safe failure"; provision of an audit trail for decisions; and active management of the knowledge base to remain up to date and sensitive to any changes in the environment. In organizational terms, the principles require benevolence-aiming to do good through the use of AI; transparency, ensuring that all assumptions and potential conflicts of interest are declared; and accountability, including active oversight of AI systems and management of any risks that may arise. Particular attention is drawn to the case of vulnerable populations, where extreme care must be exercised. Finally, the principles emphasize the need for user education at all levels of engagement with AI and for continuing research into AI and its biomedical and healthcare applications.
Collapse
Affiliation(s)
| | - Eileen Koski
- Center for Computational Health, IBM T. J. Watson Research Center, Yorktown Heights, New York, USA
| | - Shireen M Atabaki
- Pediatrics; Emergency Medicine, The George Washington University School of Medicine Children s National Hospital, Washington, District of Columbia, USA
| | - Scott Weinberg
- Public Policy, American Medical Informatics Association, Rockville, Maryland, USA
| | - John D McGreevey
- Center for Applied Health Informatics and Office of the Chief Medical Information Officer, University of Pennsylvania Health System, Philadelphia, Pennsylvania, USA
| | - Joseph L Kannry
- Department of Medicine, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Carolyn Petersen
- Health Education & Content Services, Mayo Clinic, Rochester, Minnesota, USA
| | - Christoph U Lehmann
- Clinical Informatics Center, University of Texas Southwestern Medical Center, Dallas, Texas, USA
| |
Collapse
|
22
|
Ackermann K, Baker J, Green M, Fullick M, Varinli H, Westbrook J, Li L. Computerized Clinical Decision Support Systems for the Early Detection of Sepsis Among Adult Inpatients: Scoping Review. J Med Internet Res 2022; 24:e31083. [PMID: 35195528 PMCID: PMC8908200 DOI: 10.2196/31083] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2021] [Revised: 09/23/2021] [Accepted: 10/29/2021] [Indexed: 12/21/2022] Open
Abstract
Background Sepsis is a significant cause of morbidity and mortality worldwide. Early detection of sepsis followed promptly by treatment initiation improves patient outcomes and saves lives. Hospitals are increasingly using computerized clinical decision support (CCDS) systems for the rapid identification of adult patients with sepsis. Objective This scoping review aims to systematically describe studies reporting on the use and evaluation of CCDS systems for the early detection of adult inpatients with sepsis. Methods The protocol for this scoping review was previously published. A total of 10 electronic databases (MEDLINE, Embase, CINAHL, the Cochrane database, LILACS [Latin American and Caribbean Health Sciences Literature], Scopus, Web of Science, OpenGrey, ClinicalTrials.gov, and PQDT [ProQuest Dissertations and Theses]) were comprehensively searched using terms for sepsis, CCDS, and detection to identify relevant studies. Title, abstract, and full-text screening were performed by 2 independent reviewers using predefined eligibility criteria. Data charting was performed by 1 reviewer with a second reviewer checking a random sample of studies. Any disagreements were discussed with input from a third reviewer. In this review, we present the results for adult inpatients, including studies that do not specify patient age. Results A search of the electronic databases retrieved 12,139 studies following duplicate removal. We identified 124 studies for inclusion after title, abstract, full-text screening, and hand searching were complete. Nearly all studies (121/124, 97.6%) were published after 2009. Half of the studies were journal articles (65/124, 52.4%), and the remainder were conference abstracts (54/124, 43.5%) and theses (5/124, 4%). Most studies used a single cohort (54/124, 43.5%) or before-after (42/124, 33.9%) approach. Across all 124 included studies, patient outcomes were the most frequently reported outcomes (107/124, 86.3%), followed by sepsis treatment and management (75/124, 60.5%), CCDS usability (14/124, 11.3%), and cost outcomes (9/124, 7.3%). For sepsis identification, the systemic inflammatory response syndrome criteria were the most commonly used, alone (50/124, 40.3%), combined with organ dysfunction (28/124, 22.6%), or combined with other criteria (23/124, 18.5%). Over half of the CCDS systems (68/124, 54.8%) were implemented alongside other sepsis-related interventions. Conclusions The current body of literature investigating the implementation of CCDS systems for the early detection of adult inpatients with sepsis is extremely diverse. There is substantial variability in study design, CCDS criteria and characteristics, and outcomes measured across the identified literature. Future research on CCDS system usability, cost, and impact on sepsis morbidity is needed. International Registered Report Identifier (IRRID) RR2-10.2196/24899
Collapse
Affiliation(s)
- Khalia Ackermann
- Centre for Health Systems and Safety Research, Australian Institute of Health Innovation, Macquarie University, Australia
| | - Jannah Baker
- Centre for Health Systems and Safety Research, Australian Institute of Health Innovation, Macquarie University, Australia
| | | | - Mary Fullick
- Clinical Excellence Commission, Sydney, Australia
| | | | - Johanna Westbrook
- Centre for Health Systems and Safety Research, Australian Institute of Health Innovation, Macquarie University, Australia
| | - Ling Li
- Centre for Health Systems and Safety Research, Australian Institute of Health Innovation, Macquarie University, Australia
| |
Collapse
|
23
|
Röösli E, Bozkurt S, Hernandez-Boussard T. Peeking into a black box, the fairness and generalizability of a MIMIC-III benchmarking model. Sci Data 2022; 9:24. [PMID: 35075160 PMCID: PMC8786878 DOI: 10.1038/s41597-021-01110-7] [Citation(s) in RCA: 22] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2021] [Accepted: 12/10/2021] [Indexed: 11/13/2022] Open
Abstract
As artificial intelligence (AI) makes continuous progress to improve quality of care for some patients by leveraging ever increasing amounts of digital health data, others are left behind. Empirical evaluation studies are required to keep biased AI models from reinforcing systemic health disparities faced by minority populations through dangerous feedback loops. The aim of this study is to raise broad awareness of the pervasive challenges around bias and fairness in risk prediction models. We performed a case study on a MIMIC-trained benchmarking model using a broadly applicable fairness and generalizability assessment framework. While open-science benchmarks are crucial to overcome many study limitations today, this case study revealed a strong class imbalance problem as well as fairness concerns for Black and publicly insured ICU patients. Therefore, we advocate for the widespread use of comprehensive fairness and performance assessment frameworks to effectively monitor and validate benchmark pipelines built on open data resources.
Collapse
Affiliation(s)
- Eliane Röösli
- School of Life Sciences, Swiss Federal Institute of Technology (EPFL), Lausanne, Switzerland
- Department of Medicine (Biomedical Informatics), Stanford University, Stanford, CA, USA
| | - Selen Bozkurt
- Department of Medicine (Biomedical Informatics), Stanford University, Stanford, CA, USA
| | - Tina Hernandez-Boussard
- Department of Medicine (Biomedical Informatics), Stanford University, Stanford, CA, USA.
- Department of Biomedical Data Sciences, Stanford University, Stanford, CA, USA.
| |
Collapse
|
24
|
Rodriguez DV, Lawrence K, Luu S, Yu JL, Feldthouse DM, Gonzalez J, Mann D. Development of a computer-aided text message platform for user engagement with a digital Diabetes Prevention Program: a case study. J Am Med Inform Assoc 2021; 29:155-162. [PMID: 34664647 PMCID: PMC8714274 DOI: 10.1093/jamia/ocab206] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2021] [Revised: 09/03/2021] [Accepted: 09/09/2021] [Indexed: 11/12/2022] Open
Abstract
Digital Diabetes Prevention Programs (dDPP) are novel mHealth applications that leverage digital features such as tracking and messaging to support behavior change for diabetes prevention. Despite their clinical effectiveness, long-term engagement to these programs remains a challenge, creating barriers to adherence and meaningful health outcomes. We partnered with a dDPP vendor to develop a personalized automatic message system (PAMS) to promote user engagement to the dDPP platform by sending messages on behalf of their primary care provider. PAMS innovates by integrating into clinical workflows. User-centered design (UCD) methodologies in the form of iterative cycles of focus groups, user interviews, design workshops, and other core UCD activities were utilized to defined PAMS requirements. PAMS uses computational tools to deliver theory-based, automated, tailored messages, and content to support patient use of dDPP. In this article, we discuss the design and development of our system, including key requirements and features, the technical architecture and build, and preliminary user testing.
Collapse
Affiliation(s)
- Danissa V Rodriguez
- Department of Population Health, NYU Grossman School of Medicine, New York, New York, USA
| | - Katharine Lawrence
- Department of Population Health, NYU Grossman School of Medicine, New York, New York, USA
| | - Son Luu
- Department of Population Health, NYU Grossman School of Medicine, New York, New York, USA
| | - Jonathan L Yu
- Department of Population Health, NYU Grossman School of Medicine, New York, New York, USA
| | - Dawn M Feldthouse
- Medical Center Information Technology, NYU Langone Health, New York, New York, USA
| | - Javier Gonzalez
- Medical Center Information Technology, NYU Langone Health, New York, New York, USA
| | - Devin Mann
- Department of Population Health, NYU Grossman School of Medicine, New York, New York, USA
- Medical Center Information Technology, NYU Langone Health, New York, New York, USA
| |
Collapse
|
25
|
Chen LC, Sheu JT, Chuang YJ, Tsao Y. Predicting the Travel Distance of Patients to Access Healthcare Using Deep Neural Networks. IEEE JOURNAL OF TRANSLATIONAL ENGINEERING IN HEALTH AND MEDICINE 2021; 10:4900411. [PMID: 35141054 PMCID: PMC8809644 DOI: 10.1109/jtehm.2021.3134106] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/14/2021] [Revised: 11/17/2021] [Accepted: 11/22/2021] [Indexed: 12/23/2022]
Abstract
OBJECTIVE Improving geographical access remains a key issue in determining the sufficiency of regional medical resources during health policy design. However, patient choices can be the result of the complex interactivity of various factors. The aim of this study is to propose a deep neural network approach to model the complex decision of patient choice in travel distance to access care, which is an important indicator for policymaking in allocating resources. METHOD We used the 4-year nationwide insurance data of Taiwan and accumulated the possible features discussed in earlier literature. This study proposes the use of a convolutional neural network (CNN)-based framework to make predictions. The model performance was tested against other machine learning methods. The proposed framework was further interpreted using Integrated Gradients (IG) to analyze the feature weights. RESULTS We successfully demonstrated the effectiveness of using a CNN-based framework to predict the travel distance of patients, achieving an accuracy of 0.968, AUC of 0.969, sensitivity of 0.960, and specificity of 0.989. The CNN-based framework outperformed all other methods. In this research, the IG weights are potentially explainable; however, the relationship does not correspond to known indicators in public health. CONCLUSIONS Our results demonstrate the feasibility of the deep learning-based travel distance prediction model. It has the potential to guide policymaking in resource allocation. Clinical and Translational Impact Statement- Deep learning technology is feasible in investigating the distance that patients would travel while accessing care. It is a tool that integrates complex interactive variables with highly imbalanced data distributions.
Collapse
Affiliation(s)
- Li-Chin Chen
- Research Center for Information Technology InnovationAcademia Sinica, NankangTaipei115Taiwan
| | - Ji-Tian Sheu
- Department of Health Care ManagementChang Gung University, GuishanTaoyuan333Taiwan
| | - Yuh-Jue Chuang
- Department of Health Care ManagementChang Gung University, GuishanTaoyuan333Taiwan
| | - Yu Tsao
- Research Center for Information Technology InnovationAcademia Sinica, NankangTaipei115Taiwan
| |
Collapse
|
26
|
Garvey KV, Craig KJT, Russell RG, Novak L, Moore D, Preininger AM, Jackson GP, Miller BM. The Potential and the Imperative: the Gap in AI-Related Clinical Competencies and the Need to Close It. MEDICAL SCIENCE EDUCATOR 2021; 31:2055-2060. [PMID: 34956712 PMCID: PMC8651813 DOI: 10.1007/s40670-021-01377-w] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 08/14/2021] [Indexed: 05/27/2023]
Affiliation(s)
- Kim V. Garvey
- Center for Advanced Mobile Healthcare Learning, Vanderbilt University Medical Center, Nashville, TN USA
| | | | - Regina G. Russell
- Office of Undergraduate Medical Education, Vanderbilt University School of Medicine, Nashville, TN USA
| | - Laurie Novak
- Center of Excellence in Applied Artificial Intelligence, Department of Bioinformatics, Vanderbilt University Medical Center, Nashville, TN USA
| | - Don Moore
- Vanderbilt University School of Medicine, Nashville, TN USA
| | | | - Gretchen P. Jackson
- AI Research and Evaluation, IBM Watson Health, Cambridge, MA USA
- IBM Watson Health, Cambridge, MA USA
| | - Bonnie M. Miller
- Office of Health Sciences Education, Vanderbilt University Medical Center, 2525 West End Avenue, Office 1586, TN Nashville, USA
| |
Collapse
|
27
|
Ji M, Chen X, Genchev GZ, Wei M, Yu G. Status of AI-Enabled Clinical Decision Support Systems Implementations in China. Methods Inf Med 2021; 60:123-132. [PMID: 34695871 DOI: 10.1055/s-0041-1736461] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
Abstract
BACKGROUND AI-enabled Clinical Decision Support Systems (AI + CDSSs) were heralded to contribute greatly to the advancement of health care services. There is an increased availability of monetary funds and technical expertise invested in projects and proposals targeting the building and implementation of such systems. Therefore, understanding the actual system implementation status in clinical practice is imperative. OBJECTIVES The aim of the study is to understand (1) the current situation of AI + CDSSs clinical implementations in Chinese hospitals and (2) concerns regarding AI + CDSSs current and future implementations. METHODS We investigated 160 tertiary hospitals from six provinces and province-level cities. Descriptive analysis, two-sided Fisher exact test, and Mann-Whitney U-test were utilized for analysis. RESULTS Thirty-eight of the surveyed hospitals (23.75%) had implemented AI + CDSSs. There were statistical differences on grade, scales, and medical volume between the two groups of hospitals (implemented vs. not-implemented AI + CDSSs, p <0.05). On the 5-point Likert scale, 81.58% (31/38) of respondents rated their overall satisfaction with the systems as "just neutral" to "satisfied." The three most common concerns were system functions improvement and integration into the clinical process, data quality and availability, and methodological bias. CONCLUSION While AI + CDSSs were not yet widespread in Chinese clinical settings, professionals recognize the potential benefits and challenges regarding in-hospital AI + CDSSs.
Collapse
Affiliation(s)
- Mengting Ji
- Department of Oncology, Ren Ji Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - Xiaoyun Chen
- China Digital Medicine Press, Beijing, People's Republic of China
| | - Georgi Z Genchev
- Department of Information Technology, Shanghai Children's Hospital, Shanghai, People's Republic of China.,Bulgarian Institute for Genomics and Precision Medicine, Sofia, Bulgaria
| | - Mingyue Wei
- Department of Information Technology, Shanghai Children's Hospital, Shanghai, People's Republic of China
| | - Guangjun Yu
- Department of Information Technology, Shanghai Children's Hospital, Shanghai, People's Republic of China
| |
Collapse
|
28
|
Gilbert S, Fenech M, Hirsch M, Upadhyay S, Biasiucci A, Starlinger J. Algorithm Change Protocols in the Regulation of Adaptive Machine Learning-Based Medical Devices. J Med Internet Res 2021; 23:e30545. [PMID: 34697010 PMCID: PMC8579211 DOI: 10.2196/30545] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2021] [Revised: 08/23/2021] [Accepted: 09/22/2021] [Indexed: 01/29/2023] Open
Abstract
One of the greatest strengths of artificial intelligence (AI) and machine learning (ML) approaches in health care is that their performance can be continually improved based on updates from automated learning from data. However, health care ML models are currently essentially regulated under provisions that were developed for an earlier age of slowly updated medical devices—requiring major documentation reshape and revalidation with every major update of the model generated by the ML algorithm. This creates minor problems for models that will be retrained and updated only occasionally, but major problems for models that will learn from data in real time or near real time. Regulators have announced action plans for fundamental changes in regulatory approaches. In this Viewpoint, we examine the current regulatory frameworks and developments in this domain. The status quo and recent developments are reviewed, and we argue that these innovative approaches to health care need matching innovative approaches to regulation and that these approaches will bring benefits for patients. International perspectives from the World Health Organization, and the Food and Drug Administration’s proposed approach, based around oversight of tool developers’ quality management systems and defined algorithm change protocols, offer a much-needed paradigm shift, and strive for a balanced approach to enabling rapid improvements in health care through AI innovation while simultaneously ensuring patient safety. The draft European Union (EU) regulatory framework indicates similar approaches, but no detail has yet been provided on how algorithm change protocols will be implemented in the EU. We argue that detail must be provided, and we describe how this could be done in a manner that would allow the full benefits of AI/ML-based innovation for EU patients and health care systems to be realized.
Collapse
Affiliation(s)
- Stephen Gilbert
- Ada Health GmbH, Berlin, Germany.,Else Kröner-Fresenius Center for Digital Health, Faculty of Medicine Carl Gustav Carus, Technische Universität Dresden, Dresden, Germany
| | - Matthew Fenech
- Ada Health GmbH, Berlin, Germany.,Una Health GmbH, Berlin, Germany
| | - Martin Hirsch
- Ada Health GmbH, Berlin, Germany.,Institute for AI in Medicine, University Hospital of Giessen and Marburg, Marburg, Germany
| | | | - Andrea Biasiucci
- Laboratory for Investigative Neurophysiology, University Hospital Center and University of Lausanne, Lausanne, Switzerland.,confinis ag, Sursee, Switzerland
| | | |
Collapse
|
29
|
Kollef MH, Shorr AF, Bassetti M, Timsit JF, Micek ST, Michelson AP, Garnacho-Montero J. Timing of antibiotic therapy in the ICU. Crit Care 2021; 25:360. [PMID: 34654462 PMCID: PMC8518273 DOI: 10.1186/s13054-021-03787-z] [Citation(s) in RCA: 57] [Impact Index Per Article: 19.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/04/2021] [Accepted: 10/06/2021] [Indexed: 12/15/2022] Open
Abstract
Severe or life threatening infections are common among patients in the intensive care unit (ICU). Most infections in the ICU are bacterial or fungal in origin and require antimicrobial therapy for clinical resolution. Antibiotics are the cornerstone of therapy for infected critically ill patients. However, antibiotics are often not optimally administered resulting in less favorable patient outcomes including greater mortality. The timing of antibiotics in patients with life threatening infections including sepsis and septic shock is now recognized as one of the most important determinants of survival for this population. Individuals who have a delay in the administration of antibiotic therapy for serious infections can have a doubling or more in their mortality. Additionally, the timing of an appropriate antibiotic regimen, one that is active against the offending pathogens based on in vitro susceptibility, also influences survival. Thus not only is early empiric antibiotic administration important but the selection of those agents is crucial as well. The duration of antibiotic infusions, especially for β-lactams, can also influence antibiotic efficacy by increasing antimicrobial drug exposure for the offending pathogen. However, due to mounting antibiotic resistance, aggressive antimicrobial de-escalation based on microbiology results is necessary to counterbalance the pressures of early broad-spectrum antibiotic therapy. In this review, we examine time related variables impacting antibiotic optimization as it relates to the treatment of life threatening infections in the ICU. In addition to highlighting the importance of antibiotic timing in the ICU we hope to provide an approach to antimicrobials that also minimizes the unnecessary use of these agents. Such approaches will increasingly be linked to advances in molecular microbiology testing and artificial intelligence/machine learning. Such advances should help identify patients needing empiric antibiotic therapy at an earlier time point as well as the specific antibiotics required in order to avoid unnecessary administration of broad-spectrum antibiotics.
Collapse
Affiliation(s)
- Marin H Kollef
- Division of Pulmonary and Critical Care Medicine, Washington University School of Medicine, 660 South Euclid Avenue, MSC 8052-43-14, St. Louis, MO, 63110, USA.
| | - Andrew F Shorr
- Pulmonary and Critical Care Medicine, Medstar Washington Hospital, Washington, DC, USA
| | - Matteo Bassetti
- Infectious Diseases Unit, Department of Health Sciences, San Martino Policlinico Hospital - IRCCS, University of Genoa, Genoa, Italy
| | - Jean-Francois Timsit
- AP-HP, Bichat Claude Bernard Hospital, Medical and Infectious Diseases ICU (MI2), IAME, INSERM, Université de Paris, Paris, France
| | - Scott T Micek
- Department of Pharmacy Practice, University of Health Sciences and Pharmacy, St. Louis, MO, USA
| | - Andrew P Michelson
- Division of Pulmonary and Critical Care Medicine, Washington University School of Medicine, 660 South Euclid Avenue, MSC 8052-43-14, St. Louis, MO, 63110, USA
| | | |
Collapse
|
30
|
Akbar S, Lyell D, Magrabi F. Automation in nursing decision support systems: A systematic review of effects on decision making, care delivery, and patient outcomes. J Am Med Inform Assoc 2021; 28:2502-2513. [PMID: 34498063 PMCID: PMC8510331 DOI: 10.1093/jamia/ocab123] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/18/2021] [Revised: 05/24/2021] [Accepted: 06/03/2021] [Indexed: 01/07/2023] Open
Abstract
OBJECTIVE The study sought to summarize research literature on nursing decision support systems (DSSs ); understand which steps of the nursing care process (NCP) are supported by DSSs, and analyze effects of automated information processing on decision making, care delivery, and patient outcomes. MATERIALS AND METHODS We conducted a systematic review in accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement. PubMed, CINAHL, Cochrane, Embase, Scopus, and Web of Science were searched from January 2014 to April 2020 for studies focusing on DSSs used exclusively by nurses and their effects. Information about the stages of automation (information acquisition, information analysis, decision and action selection, and action implementation), NCP, and effects was assessed. RESULTS Of 1019 articles retrieved, 28 met the inclusion criteria, each studying a unique DSS. Most DSSs were concerned with two NCP steps: assessment (82%) and intervention (86%). In terms of automation, all included DSSs automated information analysis and decision selection. Five DSSs automated information acquisition and only one automated action implementation. Effects on decision making, care delivery, and patient outcome were mixed. DSSs improved compliance with recommendations and reduced decision time, but impacts were not always sustainable. There were improvements in evidence-based practice, but impact on patient outcomes was mixed. CONCLUSIONS Current nursing DSSs do not adequately support the NCP and have limited automation. There remain many opportunities to enhance automation, especially at the stage of information acquisition. Further research is needed to understand how automation within the NCP can improve nurses' decision making, care delivery, and patient outcomes.
Collapse
Affiliation(s)
- Saba Akbar
- Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, Sydney, Australia
| | - David Lyell
- Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, Sydney, Australia
| | - Farah Magrabi
- Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, Sydney, Australia
| |
Collapse
|
31
|
Kashyap S, Morse KE, Patel B, Shah NH. A survey of extant organizational and computational setups for deploying predictive models in health systems. J Am Med Inform Assoc 2021; 28:2445-2450. [PMID: 34423364 PMCID: PMC8510384 DOI: 10.1093/jamia/ocab154] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2021] [Revised: 06/07/2021] [Accepted: 07/11/2021] [Indexed: 11/13/2022] Open
Abstract
OBJECTIVE Artificial intelligence (AI) and machine learning (ML) enabled healthcare is now feasible for many health systems, yet little is known about effective strategies of system architecture and governance mechanisms for implementation. Our objective was to identify the different computational and organizational setups that early-adopter health systems have utilized to integrate AI/ML clinical decision support (AI-CDS) and scrutinize their trade-offs. MATERIALS AND METHODS We conducted structured interviews with health systems with AI deployment experience about their organizational and computational setups for deploying AI-CDS at point of care. RESULTS We contacted 34 health systems and interviewed 20 healthcare sites (58% response rate). Twelve (60%) sites used the native electronic health record vendor configuration for model development and deployment, making it the most common shared infrastructure. Nine (45%) sites used alternative computational configurations which varied significantly. Organizational configurations for managing AI-CDS were distinguished by how they identified model needs, built and implemented models, and were separable into 3 major types: Decentralized translation (n = 10, 50%), IT Department led (n = 2, 10%), and AI in Healthcare (AIHC) Team (n = 8, 40%). DISCUSSION No singular computational configuration enables all current use cases for AI-CDS. Health systems need to consider their desired applications for AI-CDS and whether investment in extending the off-the-shelf infrastructure is needed. Each organizational setup confers trade-offs for health systems planning strategies to implement AI-CDS. CONCLUSION Health systems will be able to use this framework to understand strengths and weaknesses of alternative organizational and computational setups when designing their strategy for artificial intelligence.
Collapse
Affiliation(s)
- Sehj Kashyap
- Stanford Center for Biomedical Informatics Research, Stanford University, Stanford, California, USA
| | - Keith E Morse
- Division of Pediatric Hospital Medicine, Department of Pediatrics, Stanford University School of Medicine, Stanford, California, USA
| | - Birju Patel
- Stanford Center for Biomedical Informatics Research, Stanford University, Stanford, California, USA
| | - Nigam H Shah
- Stanford Center for Biomedical Informatics Research, Stanford University, Stanford, California, USA
| |
Collapse
|
32
|
Shashikumar SP, Wardi G, Malhotra A, Nemati S. Artificial intelligence sepsis prediction algorithm learns to say "I don't know". NPJ Digit Med 2021; 4:134. [PMID: 34504260 PMCID: PMC8429719 DOI: 10.1038/s41746-021-00504-6] [Citation(s) in RCA: 28] [Impact Index Per Article: 9.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2021] [Accepted: 08/09/2021] [Indexed: 01/07/2023] Open
Abstract
Sepsis is a leading cause of morbidity and mortality worldwide. Early identification of sepsis is important as it allows timely administration of potentially life-saving resuscitation and antimicrobial therapy. We present COMPOSER (COnformal Multidimensional Prediction Of SEpsis Risk), a deep learning model for the early prediction of sepsis, specifically designed to reduce false alarms by detecting unfamiliar patients/situations arising from erroneous data, missingness, distributional shift and data drifts. COMPOSER flags these unfamiliar cases as indeterminate rather than making spurious predictions. Six patient cohorts (515,720 patients) curated from two healthcare systems in the United States across intensive care units (ICU) and emergency departments (ED) were used to train and externally and temporally validate this model. In a sequential prediction setting, COMPOSER achieved a consistently high area under the curve (AUC) (ICU: 0.925-0.953; ED: 0.938-0.945). Out of over 6 million prediction windows roughly 20% and 8% were identified as indeterminate amongst non-septic and septic patients, respectively. COMPOSER provided early warning within a clinically actionable timeframe (ICU: 12.2 [3.2 22.8] and ED: 2.1 [0.8 4.5] hours prior to first antibiotics order) across all six cohorts, thus allowing for identification and prioritization of patients at high risk for sepsis.
Collapse
Affiliation(s)
| | - Gabriel Wardi
- Department of Emergency Medicine, University of California San Diego, San Diego, USA
- Division of Pulmonary, Critical Care and Sleep Medicine, University of California San Diego, San Diego, USA
| | - Atul Malhotra
- Division of Pulmonary, Critical Care and Sleep Medicine, University of California San Diego, San Diego, USA
| | - Shamim Nemati
- Division of Biomedical Informatics, University of California San Diego, San Diego, USA.
| |
Collapse
|
33
|
Hernandez-Boussard T, Lundgren MP, Shah N. Conflicting information from the Food and Drug Administration: Missed opportunity to lead standards for safe and effective medical artificial intelligence solutions. J Am Med Inform Assoc 2021; 28:1353-1355. [PMID: 33674865 PMCID: PMC8661389 DOI: 10.1093/jamia/ocab035] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/01/2021] [Accepted: 02/10/2021] [Indexed: 11/14/2022] Open
Abstract
The Food & Drug Administration (FDA) is considering the permanent exemption of premarket notification requirements for several Class I and II medical device products, including several artificial Intelligence (AI)-driven devices. The exemption is based on the need to rapidly more quickly disseminate devices to the public, estimated cost-savings, a lack of documented adverse events reported to the FDA's database. However, this ignores emerging issues related to AI-based devices, including utility, reproducibility and bias that may not only affect an individual but entire populations. We urge the FDA to reinforce the messaging on safety and effectiveness regulations of AI-based Software as a Medical Device products to better promote fair AI-driven clinical decision tools and for preventing harm to the patients we serve.
Collapse
Affiliation(s)
| | | | - Nigam Shah
- Department of Medicine, Stanford University, Stanford, California, USA
| |
Collapse
|
34
|
Lyell D, Coiera E, Chen J, Shah P, Magrabi F. How machine learning is embedded to support clinician decision making: an analysis of FDA-approved medical devices. BMJ Health Care Inform 2021; 28:bmjhci-2020-100301. [PMID: 33853863 PMCID: PMC8054073 DOI: 10.1136/bmjhci-2020-100301] [Citation(s) in RCA: 28] [Impact Index Per Article: 9.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2020] [Revised: 02/10/2021] [Accepted: 03/02/2021] [Indexed: 12/20/2022] Open
Abstract
Objective To examine how and to what extent medical devices using machine learning (ML) support clinician decision making. Methods We searched for medical devices that were (1) approved by the US Food and Drug Administration (FDA) up till February 2020; (2) intended for use by clinicians; (3) in clinical tasks or decisions and (4) used ML. Descriptive information about the clinical task, device task, device input and output, and ML method were extracted. The stage of human information processing automated by ML-based devices and level of autonomy were assessed. Results Of 137 candidates, 59 FDA approvals for 49 unique devices were included. Most approvals (n=51) were since 2018. Devices commonly assisted with diagnostic (n=35) and triage (n=10) tasks. Twenty-three devices were assistive, providing decision support but left clinicians to make important decisions including diagnosis. Twelve automated the provision of information (autonomous information), such as quantification of heart ejection fraction, while 14 automatically provided task decisions like triaging the reading of scans according to suspected findings of stroke (autonomous decisions). Stages of human information processing most automated by devices were information analysis, (n=14) providing information as an input into clinician decision making, and decision selection (n=29), where devices provide a decision. Conclusion Leveraging the benefits of ML algorithms to support clinicians while mitigating risks, requires a solid relationship between clinician and ML-based devices. Such relationships must be carefully designed, considering how algorithms are embedded in devices, the tasks supported, information provided and clinicians’ interactions with them.
Collapse
Affiliation(s)
- David Lyell
- Australian Institute of Health Innovation, Faculty of Medicine, Health and Human Sciences, Macquarie University, Sydney, NSW, Australia
| | - Enrico Coiera
- Australian Institute of Health Innovation, Faculty of Medicine, Health and Human Sciences, Macquarie University, Sydney, NSW, Australia
| | - Jessica Chen
- Australian Institute of Health Innovation, Faculty of Medicine, Health and Human Sciences, Macquarie University, Sydney, NSW, Australia
| | - Parina Shah
- Australian Institute of Health Innovation, Faculty of Medicine, Health and Human Sciences, Macquarie University, Sydney, NSW, Australia
| | - Farah Magrabi
- Australian Institute of Health Innovation, Faculty of Medicine, Health and Human Sciences, Macquarie University, Sydney, NSW, Australia
| |
Collapse
|
35
|
Embi PJ. Algorithmovigilance-Advancing Methods to Analyze and Monitor Artificial Intelligence-Driven Health Care for Effectiveness and Equity. JAMA Netw Open 2021; 4:e214622. [PMID: 33856479 DOI: 10.1001/jamanetworkopen.2021.4622] [Citation(s) in RCA: 25] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/14/2022] Open
Affiliation(s)
- Peter J Embi
- Regenstrief Institute Inc, Indiana University School of Medicine, Indianapolis
| |
Collapse
|
36
|
Smith J. Setting the agenda: an informatics-led policy framework for adaptive CDS. J Am Med Inform Assoc 2020; 27:1831-1833. [PMID: 33301025 PMCID: PMC7727380 DOI: 10.1093/jamia/ocaa239] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/10/2020] [Indexed: 03/31/2024] Open
Affiliation(s)
- Jeffery Smith
- American Medical Informatics Association, Bethesda, Maryland, USA
| |
Collapse
|