1
|
Wang Y, Liu S, Spiteri AG, Huynh ALH, Chu C, Masters CL, Goudey B, Pan Y, Jin L. Understanding machine learning applications in dementia research and clinical practice: a review for biomedical scientists and clinicians. Alzheimers Res Ther 2024; 16:175. [PMID: 39085973 PMCID: PMC11293066 DOI: 10.1186/s13195-024-01540-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/22/2024] [Accepted: 07/21/2024] [Indexed: 08/02/2024]
Abstract
Several (inter)national longitudinal dementia observational datasets encompassing demographic information, neuroimaging, biomarkers, neuropsychological evaluations, and muti-omics data, have ushered in a new era of potential for integrating machine learning (ML) into dementia research and clinical practice. ML, with its proficiency in handling multi-modal and high-dimensional data, has emerged as an innovative technique to facilitate early diagnosis, differential diagnosis, and to predict onset and progression of mild cognitive impairment and dementia. In this review, we evaluate current and potential applications of ML, including its history in dementia research, how it compares to traditional statistics, the types of datasets it uses and the general workflow. Moreover, we identify the technical barriers and challenges of ML implementations in clinical practice. Overall, this review provides a comprehensive understanding of ML with non-technical explanations for broader accessibility to biomedical scientists and clinicians.
Collapse
Affiliation(s)
- Yihan Wang
- The Florey Institute of Neuroscience and Mental Health, 30 Royal Parade, Parkville, VIC, 3052, Australia
- Florey Department of Neuroscience and Mental Health, The University of Melbourne, 30 Royal Parade, Parkville, VIC, 3052, Australia
| | - Shu Liu
- The Florey Institute of Neuroscience and Mental Health, 30 Royal Parade, Parkville, VIC, 3052, Australia
- Florey Department of Neuroscience and Mental Health, The University of Melbourne, 30 Royal Parade, Parkville, VIC, 3052, Australia
- The ARC Training Centre in Cognitive Computing for Medical Technologies, The University of Melbourne, Carlton, VIC, 3010, Australia
| | - Alanna G Spiteri
- The Florey Institute of Neuroscience and Mental Health, 30 Royal Parade, Parkville, VIC, 3052, Australia
| | - Andrew Liem Hieu Huynh
- Department of Aged Care, Austin Health, Heidelberg, VIC, 3084, Australia
- Department of Medicine, Austin Health, University of Melbourne, Heidelberg, VIC, 3084, Australia
| | - Chenyin Chu
- The Florey Institute of Neuroscience and Mental Health, 30 Royal Parade, Parkville, VIC, 3052, Australia
- Florey Department of Neuroscience and Mental Health, The University of Melbourne, 30 Royal Parade, Parkville, VIC, 3052, Australia
| | - Colin L Masters
- The Florey Institute of Neuroscience and Mental Health, 30 Royal Parade, Parkville, VIC, 3052, Australia
| | - Benjamin Goudey
- Florey Department of Neuroscience and Mental Health, The University of Melbourne, 30 Royal Parade, Parkville, VIC, 3052, Australia
- The ARC Training Centre in Cognitive Computing for Medical Technologies, The University of Melbourne, Carlton, VIC, 3010, Australia
| | - Yijun Pan
- The Florey Institute of Neuroscience and Mental Health, 30 Royal Parade, Parkville, VIC, 3052, Australia.
- Florey Department of Neuroscience and Mental Health, The University of Melbourne, 30 Royal Parade, Parkville, VIC, 3052, Australia.
| | - Liang Jin
- The Florey Institute of Neuroscience and Mental Health, 30 Royal Parade, Parkville, VIC, 3052, Australia
- Florey Department of Neuroscience and Mental Health, The University of Melbourne, 30 Royal Parade, Parkville, VIC, 3052, Australia
| |
Collapse
|
2
|
Groom LL, Feldthouse D, Robertiello G, Fletcher J, Squires A. A Pilot Study Toward Development of the Digital Literacy, Usability, and Acceptability of Technology Instrument for Healthcare. Comput Inform Nurs 2024:00024665-990000000-00203. [PMID: 38913989 DOI: 10.1097/cin.0000000000001156] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/26/2024]
Abstract
Electronic health record proficiency is critical for health professionals to deliver and document patient care. There is scarce research on this topic within undergraduate nursing student populations. The purpose of this study is to describe the psychometric evaluation of the Digital Literacy, Usability, and Acceptability of Technology Instrument for Healthcare. A cross-sectional pilot study for psychometric evaluation of the instrument was conducted using data collected through an emailed survey. Exploratory factor analysis, inter-item and adjusted item-total correlations, and Cronbach's α calculated subscale reliability. A total of 297 nursing students completed the survey. A seven-factor structure best fit the data: technology use-engagement, technology use-confidence, technology use-history, electronic health record-ease of use, electronic health record-comparability, and electronic health record-burden. Cronbach's α indicated good to very good internal consistency (α = .68 to .89). The instrument effectively measured digital literacy, acceptance, and usability of an electronic health record and may be implemented with good to very good reliability across varied healthcare simulation and training experiences.
Collapse
Affiliation(s)
- Lisa L Groom
- Author Affiliations: Quinnipiac University Rory Meyers College of Nursing (Drs Groom, Robertiello, Fletcher, and Squires); and Medical Center Information Technology, NYU Langone Health, New York (Dr Groom and Ms Feldthouse)
| | | | | | | | | |
Collapse
|
3
|
Klawunn R, Albrecht UV, Katzmarzyk D, Dierks ML. Beyond technology acceptance-a focused ethnography on the implementation, acceptance and use of new nursing technology in a German hospital. Front Digit Health 2024; 6:1330988. [PMID: 38736709 PMCID: PMC11085260 DOI: 10.3389/fdgth.2024.1330988] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2023] [Accepted: 04/11/2024] [Indexed: 05/14/2024] Open
Abstract
Introduction Hospitalised patients could benefit from the emergence of novel technologies for nursing care. There are numerous technical products available, but these rarely find their way into practice. Further knowledge is required about the circumstances under which technology in nursing is accepted and used. In the research project "Centre for Implementing Nursing Care Innovations", technical innovations are implemented on a trauma surgery inpatient ward in Germany. After implementation, it was investigated: Which implemented technologies are accepted/rejected, and which factors influence the acceptance/rejection of technology for nurses? Material and methods A focused ethnography was used, containing two approaches: First, participant observation was conducted to examine nurses' and patients' interaction with technologies. Observations were fixed in a field research diary and analysed using evaluative qualitative content analysis. Second, a questionnaire was used by nurses to provide information about the use frequency and technology suitability. The results of the study were consolidated and analysed using the UTAUT model. Results Seven studied technologies can be summarised in four result categories: (1) A Mobilising mattress, a Special projector and a Sound pillow are accepted and used by nurses and patients, because they offer a way to provide high quality care with little additional effort. (2) A Fall prevention system is consistently used in patient care as a work obligation, but since nurses consider the system error-prone, acceptance is low. (3) An Interactive therapy ball is accepted but nurses cannot use it due to the high workload. (4) An App for nurse-patient communication and a work-equipment tracking system are not used or accepted because nurses do not see a practical benefit in the systems. Discussion Acceptance or rejection of a product does not necessarily equate to use or non-use of the technology. Before implementation, technology acceptance among users occurs as prejudice-when users are given time to experiment with technology, intention-to-use can stabilize into sustained use. Accepted and used technologies can serve to mask problems (such as staff shortages) and encourage problematic developments, such as the reduction of contact time at the bedside. Therefore, technology acceptance should be qualified in asking to what accepted technology contributes.
Collapse
Affiliation(s)
- Ronny Klawunn
- Department for Patient Orientation and Health Education, Institute for Epidemiology, Social Medicine and Health System Research, Hannover Medical School (MHH), Hanover, Germany
| | - Urs-Vito Albrecht
- Department of Digital Medicine, Medical Faculty OWL, Bielefeld University, Bielefeld, Germany
| | - Deliah Katzmarzyk
- Department for Patient Orientation and Health Education, Institute for Epidemiology, Social Medicine and Health System Research, Hannover Medical School (MHH), Hanover, Germany
| | - Marie-Luise Dierks
- Department for Patient Orientation and Health Education, Institute for Epidemiology, Social Medicine and Health System Research, Hannover Medical School (MHH), Hanover, Germany
| |
Collapse
|
4
|
Evans RP, Bryant LD, Russell G, Absolom K. Trust and acceptability of data-driven clinical recommendations in everyday practice: A scoping review. Int J Med Inform 2024; 183:105342. [PMID: 38266426 DOI: 10.1016/j.ijmedinf.2024.105342] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2023] [Revised: 12/08/2023] [Accepted: 01/14/2024] [Indexed: 01/26/2024]
Abstract
BACKGROUND Increasing attention is being given to the analysis of large health datasets to derive new clinical decision support systems (CDSS). However, few data-driven CDSS are being adopted into clinical practice. Trust in these tools is believed to be fundamental for acceptance and uptake but to date little attention has been given to defining or evaluating trust in clinical settings. OBJECTIVES A scoping review was conducted to explore how and where acceptability and trustworthiness of data-driven CDSS have been assessed from the health professional's perspective. METHODS Medline, Embase, PsycInfo, Web of Science, Scopus, ACM Digital, IEEE Xplore and Google Scholar were searched in March 2022 using terms expanded from: "data-driven" AND "clinical decision support" AND "acceptability". Included studies focused on healthcare practitioner-facing data-driven CDSS, relating directly to clinical care. They included trust or a proxy as an outcome, or in the discussion. The preferred reporting items for systematic reviews and meta-analyses extension for scoping reviews (PRISMA-ScR) is followed in the reporting of this review. RESULTS 3291 papers were screened, with 85 primary research studies eligible for inclusion. Studies covered a diverse range of clinical specialisms and intended contexts, but hypothetical systems (24) outnumbered those in clinical use (18). Twenty-five studies measured trust, via a wide variety of quantitative, qualitative and mixed methods. A further 24 discussed themes of trust without it being explicitly evaluated, and from these, themes of transparency, explainability, and supporting evidence were identified as factors influencing healthcare practitioner trust in data-driven CDSS. CONCLUSION There is a growing body of research on data-driven CDSS, but few studies have explored stakeholder perceptions in depth, with limited focused research on trustworthiness. Further research on healthcare practitioner acceptance, including requirements for transparency and explainability, should inform clinical implementation.
Collapse
Affiliation(s)
- Ruth P Evans
- University of Leeds, Woodhouse Lane, Leeds LS2 9JT, UK.
| | | | - Gregor Russell
- Bradford District Care Trust, Bradford, New Mill, Victoria Rd, BD18 3LD, UK.
| | - Kate Absolom
- University of Leeds, Woodhouse Lane, Leeds LS2 9JT, UK.
| |
Collapse
|
5
|
Giddings R, Joseph A, Callender T, Janes SM, van der Schaar M, Sheringham J, Navani N. Factors influencing clinician and patient interaction with machine learning-based risk prediction models: a systematic review. Lancet Digit Health 2024; 6:e131-e144. [PMID: 38278615 DOI: 10.1016/s2589-7500(23)00241-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2023] [Revised: 10/20/2023] [Accepted: 11/14/2023] [Indexed: 01/28/2024]
Abstract
Machine learning (ML)-based risk prediction models hold the potential to support the health-care setting in several ways; however, use of such models is scarce. We aimed to review health-care professional (HCP) and patient perceptions of ML risk prediction models in published literature, to inform future risk prediction model development. Following database and citation searches, we identified 41 articles suitable for inclusion. Article quality varied with qualitative studies performing strongest. Overall, perceptions of ML risk prediction models were positive. HCPs and patients considered that models have the potential to add benefit in the health-care setting. However, reservations remain; for example, concerns regarding data quality for model development and fears of unintended consequences following ML model use. We identified that public views regarding these models might be more negative than HCPs and that concerns (eg, extra demands on workload) were not always borne out in practice. Conclusions are tempered by the low number of patient and public studies, the absence of participant ethnic diversity, and variation in article quality. We identified gaps in knowledge (particularly views from under-represented groups) and optimum methods for model explanation and alerts, which require future research.
Collapse
Affiliation(s)
- Rebecca Giddings
- Lungs for Living Research Centre, UCL Respiratory, University College London, London, UK.
| | - Anabel Joseph
- Lungs for Living Research Centre, UCL Respiratory, University College London, London, UK
| | - Thomas Callender
- Lungs for Living Research Centre, UCL Respiratory, University College London, London, UK
| | - Sam M Janes
- Lungs for Living Research Centre, UCL Respiratory, University College London, London, UK
| | - Mihaela van der Schaar
- Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Cambridge, UK; The Alan Turing Institute, London, UK
| | - Jessica Sheringham
- Department of Applied Health Research, University College London, London, UK
| | - Neal Navani
- Lungs for Living Research Centre, UCL Respiratory, University College London, London, UK
| |
Collapse
|
6
|
Epizitone A, Moyane SP, Agbehadji IE. A Data-Driven Paradigm for a Resilient and Sustainable Integrated Health Information Systems for Health Care Applications. J Multidiscip Healthc 2023; 16:4015-4025. [PMID: 38107085 PMCID: PMC10725635 DOI: 10.2147/jmdh.s433299] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2023] [Accepted: 11/02/2023] [Indexed: 12/19/2023] Open
Abstract
Introduction Many transformations and uncertainties, such as the fourth industrial revolution and pandemics, have propelled healthcare acceptance and deployment of health information systems (HIS). External and internal determinants aligning with the global course influence their deployments. At the epic is digitalization, which generates endless data that has permeated healthcare. The continuous proliferation of complex and dynamic healthcare data is the digitalization frontier in healthcare that necessitates attention. Objective This study explores the existing body of information on HIS for healthcare through the data lens to present a data-driven paradigm for healthcare augmentation paramount to attaining a sustainable and resilient HIS. Method Preferred Reporting Items for Systematic Reviews and Meta-Analyses: PRISMA-compliant in-depth literature review was conducted systematically to synthesize and analyze the literature content to ascertain the value disposition of HIS data in healthcare delivery. Results This study details the aspects of a data-driven paradigm for robust and sustainable HIS for health care applications. Data source, data action and decisions, data sciences techniques, serialization of data sciences techniques in the HIS, and data insight implementation and application are data-driven features expounded. These are essential data-driven paradigm building blocks that need iteration to succeed. Discussions Existing literature considers insurgent data in healthcare challenging, disruptive, and potentially revolutionary. This view echoes the current healthcare quandary of good and bad data availability. Thus, data-driven insights are essential for building a resilient and sustainable HIS. People, technology, and tasks dominated prior HIS frameworks, with few data-centric facets. Improving healthcare and the HIS requires identifying and integrating crucial data elements. Conclusion The paper presented a data-driven paradigm for a resilient and sustainable HIS. The findings show that data-driven track and components are essential to improve healthcare using data analytics insights. It provides an integrated footing for data analytics to support and effectively assist health care delivery.
Collapse
Affiliation(s)
- Ayogeboh Epizitone
- ICT and Society Research Group, Department of Information and Corporate Management, Durban University of Technology, Durban, South Africa
| | - Smangele Pretty Moyane
- Department of Information and Corporate Management, Durban University of Technology, Durban, South Africa
| | - Israel Edem Agbehadji
- Centre for Transformative Agricultural and Food Systems, School of Agricultural, Earth and Environmental Sciences, University of KwaZulu-Natal, Pietermaritzburg, South Africa
| |
Collapse
|
7
|
Susanto AP, Lyell D, Widyantoro B, Berkovsky S, Magrabi F. Effects of machine learning-based clinical decision support systems on decision-making, care delivery, and patient outcomes: a scoping review. J Am Med Inform Assoc 2023; 30:2050-2063. [PMID: 37647865 PMCID: PMC10654852 DOI: 10.1093/jamia/ocad180] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2023] [Revised: 08/01/2023] [Accepted: 08/23/2023] [Indexed: 09/01/2023] Open
Abstract
OBJECTIVE This study aims to summarize the research literature evaluating machine learning (ML)-based clinical decision support (CDS) systems in healthcare settings. MATERIALS AND METHODS We conducted a review in accordance with the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta Analyses extension for Scoping Review). Four databases, including PubMed, Medline, Embase, and Scopus were searched for studies published from January 2016 to April 2021 evaluating the use of ML-based CDS in clinical settings. We extracted the study design, care setting, clinical task, CDS task, and ML method. The level of CDS autonomy was examined using a previously published 3-level classification based on the division of clinical tasks between the clinician and CDS; effects on decision-making, care delivery, and patient outcomes were summarized. RESULTS Thirty-two studies evaluating the use of ML-based CDS in clinical settings were identified. All were undertaken in developed countries and largely in secondary and tertiary care settings. The most common clinical tasks supported by ML-based CDS were image recognition and interpretation (n = 12) and risk assessment (n = 9). The majority of studies examined assistive CDS (n = 23) which required clinicians to confirm or approve CDS recommendations for risk assessment in sepsis and for interpreting cancerous lesions in colonoscopy. Effects on decision-making, care delivery, and patient outcomes were mixed. CONCLUSION ML-based CDS are being evaluated in many clinical areas. There remain many opportunities to apply and evaluate effects of ML-based CDS on decision-making, care delivery, and patient outcomes, particularly in resource-constrained settings.
Collapse
Affiliation(s)
- Anindya Pradipta Susanto
- Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, Sydney, NSW 2109, Australia
- Faculty of Medicine, Universitas Indonesia, Jakarta, DKI Jakarta 10430, Indonesia
| | - David Lyell
- Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, Sydney, NSW 2109, Australia
| | - Bambang Widyantoro
- Faculty of Medicine, Universitas Indonesia, Jakarta, DKI Jakarta 10430, Indonesia
- National Cardiovascular Center Harapan Kita Hospital, Jakarta, DKI Jakarta 11420, Indonesia
| | - Shlomo Berkovsky
- Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, Sydney, NSW 2109, Australia
| | - Farah Magrabi
- Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, Sydney, NSW 2109, Australia
| |
Collapse
|
8
|
Lambert SI, Madi M, Sopka S, Lenes A, Stange H, Buszello CP, Stephan A. An integrative review on the acceptance of artificial intelligence among healthcare professionals in hospitals. NPJ Digit Med 2023; 6:111. [PMID: 37301946 DOI: 10.1038/s41746-023-00852-5] [Citation(s) in RCA: 29] [Impact Index Per Article: 29.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2022] [Accepted: 05/25/2023] [Indexed: 06/12/2023] Open
Abstract
Artificial intelligence (AI) in the domain of healthcare is increasing in prominence. Acceptance is an indispensable prerequisite for the widespread implementation of AI. The aim of this integrative review is to explore barriers and facilitators influencing healthcare professionals' acceptance of AI in the hospital setting. Forty-two articles met the inclusion criteria for this review. Pertinent elements to the study such as the type of AI, factors influencing acceptance, and the participants' profession were extracted from the included studies, and the studies were appraised for their quality. The data extraction and results were presented according to the Unified Theory of Acceptance and Use of Technology (UTAUT) model. The included studies revealed a variety of facilitating and hindering factors for AI acceptance in the hospital setting. Clinical decision support systems (CDSS) were the AI form included in most studies (n = 21). Heterogeneous results with regard to the perceptions of the effects of AI on error occurrence, alert sensitivity and timely resources were reported. In contrast, fear of a loss of (professional) autonomy and difficulties in integrating AI into clinical workflows were unanimously reported to be hindering factors. On the other hand, training for the use of AI facilitated acceptance. Heterogeneous results may be explained by differences in the application and functioning of the different AI systems as well as inter-professional and interdisciplinary disparities. To conclude, in order to facilitate acceptance of AI among healthcare professionals it is advisable to integrate end-users in the early stages of AI development as well as to offer needs-adjusted training for the use of AI in healthcare and providing adequate infrastructure.
Collapse
Affiliation(s)
- Sophie Isabelle Lambert
- AIXTRA-Competence Center for Training and Patient Safety, Medical Faculty, RWTH Aachen University, Pauwelsstraße 30, 52074, Aachen, Germany.
- Department of Anesthesiology, Uniklinik RWTH Aachen, Pauwelsstraße 30, 52074, Aachen, Germany.
| | - Murielle Madi
- Department of Nursing Science, Uniklinik RWTH Aachen, Pauwelsstraße 30, 52074, Aachen, Germany.
| | - Saša Sopka
- AIXTRA-Competence Center for Training and Patient Safety, Medical Faculty, RWTH Aachen University, Pauwelsstraße 30, 52074, Aachen, Germany
- Department of Anesthesiology, Uniklinik RWTH Aachen, Pauwelsstraße 30, 52074, Aachen, Germany
| | - Andrea Lenes
- AIXTRA-Competence Center for Training and Patient Safety, Medical Faculty, RWTH Aachen University, Pauwelsstraße 30, 52074, Aachen, Germany
| | - Hendrik Stange
- Fraunhofer Society for the Advancement of Applied Research. Fraunhofer-Institute for Intelligent Analysis and Information Systems IAIS, Schloss Birlinghoven 1, 53757, Sankt Augustin, Bonn, Germany
| | - Claus-Peter Buszello
- Fraunhofer Society for the Advancement of Applied Research. Fraunhofer-Institute for Intelligent Analysis and Information Systems IAIS, Schloss Birlinghoven 1, 53757, Sankt Augustin, Bonn, Germany
| | - Astrid Stephan
- Department of Nursing Science, Uniklinik RWTH Aachen, Pauwelsstraße 30, 52074, Aachen, Germany
- Fliedner University of Applied Sciences, Geschwister-Aufricht-Straße, 940489, Düsseldorf, Germany
| |
Collapse
|
9
|
Wieben AM, Walden RL, Alreshidi BG, Brown SF, Cato K, Coviak CP, Cruz C, D'Agostino F, Douthit BJ, Forbes TH, Gao G, Johnson SG, Lee MA, Mullen-Fortino M, Park JI, Park S, Pruinelli L, Reger A, Role J, Sileo M, Schultz MA, Vyas P, Jeffery AD. Data Science Implementation Trends in Nursing Practice: A Review of the 2021 Literature. Appl Clin Inform 2023; 14:585-593. [PMID: 37150179 PMCID: PMC10411069 DOI: 10.1055/a-2088-2893] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/29/2022] [Accepted: 05/03/2023] [Indexed: 05/09/2023] Open
Abstract
OBJECTIVES The goal of this work was to provide a review of the implementation of data science-driven applications focused on structural or outcome-related nurse-sensitive indicators in the literature in 2021. By conducting this review, we aim to inform readers of trends in the nursing indicators being addressed, the patient populations and settings of focus, and lessons and challenges identified during the implementation of these tools. METHODS We conducted a rigorous descriptive review of the literature to identify relevant research published in 2021. We extracted data on model development, implementation-related strategies and measures, lessons learned, and challenges and stakeholder involvement. We also assessed whether reports of data science application implementations currently follow the guidelines of the Developmental and Exploratory Clinical Investigations of DEcision support systems driven by AI (DECIDE-AI) framework. RESULTS Of 4,943 articles found in PubMed (NLM) and CINAHL (EBSCOhost), 11 were included in the final review and data extraction. Systems leveraging data science were developed for adult patient populations and were primarily deployed in hospital settings. The clinical domains targeted included mortality/deterioration, utilization/resource allocation, and hospital-acquired infections/COVID-19. The composition of development teams and types of stakeholders involved varied. Research teams more frequently reported on implementation methods than implementation results. Most studies provided lessons learned that could help inform future implementations of data science systems in health care. CONCLUSION In 2021, very few studies report on the implementation of data science-driven applications focused on structural- or outcome-related nurse-sensitive indicators. This gap in the sharing of implementation strategies needs to be addressed in order for these systems to be successfully adopted in health care settings.
Collapse
Affiliation(s)
- Ann M. Wieben
- University of Wisconsin-Madison School of Nursing, Madison, Wisconsin, United States
| | - Rachel Lane Walden
- Annette and Irwin Eskind Family Biomedical Library, Vanderbilt University, Nashville, Tennessee, United States
| | - Bader G. Alreshidi
- Medical-Surgical Nursing Department, College of Nursing, University of Hail, Hail, Saudi Arabia
| | | | - Kenrick Cato
- Department of Emergency Medicine, Columbia University School of Nursing, New York, New York, United States
| | - Cynthia Peltier Coviak
- Kirkhof College of Nursing, Grand Valley State University, Allendale, Michigan, United States
| | - Christopher Cruz
- Global Health Technology and Informatics, Chevron, San Ramon, California, United States
| | - Fabio D'Agostino
- Department of Medicine and Surgery, Saint Camillus International University of Health Sciences, Rome, Italy
| | - Brian J. Douthit
- Department of Biomedical Informatics, United States Department of Veterans Affairs, Vanderbilt University, Nashville, Tennessee, United States
| | - Thompson H. Forbes
- Department of Advanced Nursing Practice and Education, East Carolina University College of Nursing, Greenville, North Carolina, United States
| | - Grace Gao
- Atlanta VA Quality Scholars Program, Joseph Maxwell Cleland, Atlanta VA Medical Center, North Druid Hills, Georgia, United States
| | - Steve G. Johnson
- Institute for Health Informatics, University of Minnesota, Minneapolis, Minnesota, United States
| | | | | | - Jung In Park
- Sue and Bill Gross School of Nursing, University of California, Irvine, United States
| | - Suhyun Park
- College of Nursing and College of Medicine, University of Florida, Gainesville, Florida, United States
| | - Lisiane Pruinelli
- College of Nursing and College of Medicine, University of Florida, Gainesville, Florida, United States
| | | | - Jethrone Role
- Loma Linda University Health, Loma Linda, California, United States
| | - Marisa Sileo
- Boston Children's Hospital, Boston, Massachusetts, United States
| | | | - Pankaj Vyas
- University of Arizona College of Nursing, Tucson, Arizona, United States
| | - Alvin D. Jeffery
- U.S. Department of Veterans Affairs, Vanderbilt University School of Nursing, Tennessee Valley Healthcare System, Nashville, Tennessee, United States
| |
Collapse
|
10
|
Tricco AC, Hezam A, Parker A, Nincic V, Harris C, Fennelly O, Thomas SM, Ghassemi M, McGowan J, Paprica PA, Straus SE. Implemented machine learning tools to inform decision-making for patient care in hospital settings: a scoping review. BMJ Open 2023; 13:e065845. [PMID: 36750280 PMCID: PMC9906263 DOI: 10.1136/bmjopen-2022-065845] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/09/2023] Open
Abstract
OBJECTIVES To identify ML tools in hospital settings and how they were implemented to inform decision-making for patient care through a scoping review. We investigated the following research questions: What ML interventions have been used to inform decision-making for patient care in hospital settings? What strategies have been used to implement these ML interventions? DESIGN A scoping review was undertaken. MEDLINE, Embase, Cochrane Central Register of Controlled Trials (CENTRAL) and the Cochrane Database of Systematic Reviews (CDSR) were searched from 2009 until June 2021. Two reviewers screened titles and abstracts, full-text articles, and charted data independently. Conflicts were resolved by another reviewer. Data were summarised descriptively using simple content analysis. SETTING Hospital setting. PARTICIPANT Any type of clinician caring for any type of patient. INTERVENTION Machine learning tools used by clinicians to inform decision-making for patient care, such as AI-based computerised decision support systems or "'model-based'" decision support systems. PRIMARY AND SECONDARY OUTCOME MEASURES Patient and study characteristics, as well as intervention characteristics including the type of machine learning tool, implementation strategies, target population. Equity issues were examined with PROGRESS-PLUS criteria. RESULTS After screening 17 386 citations and 3474 full-text articles, 20 unique studies and 1 companion report were included. The included articles totalled 82 656 patients and 915 clinicians. Seven studies reported gender and four studies reported PROGRESS-PLUS criteria (race, health insurance, rural/urban). Common implementation strategies for the tools were clinician reminders that integrated ML predictions (44.4%), facilitated relay of clinical information (17.8%) and staff education (15.6%). Common barriers to successful implementation of ML tools were time (11.1%) and reliability (11.1%), and common facilitators were time/efficiency (13.6%) and perceived usefulness (13.6%). CONCLUSIONS We found limited evidence related to the implementation of ML tools to assist clinicians with patient healthcare decisions in hospital settings. Future research should examine other approaches to integrating ML into hospital clinician decisions related to patient care, and report on PROGRESS-PLUS items. FUNDING Canadian Institutes of Health Research (CIHR) Foundation grant awarded to SES and the CIHR Strategy for Patient Oriented-Research Initiative (GSR-154442). SCOPING REVIEW REGISTRATION: https://osf.io/e2mna.
Collapse
Affiliation(s)
- Andrea C Tricco
- Knowledge Translation Program, St Michael's Hospital Li Ka Shing Knowledge Institute, Toronto, Ontario, Canada
- Epidemiology Division and Institute of Health Policy, Management and Evaluation, University of Toronto Dalla Lana School of Public Health, Toronto, Ontario, Canada
| | - Areej Hezam
- Knowledge Translation Program, St Michael's Hospital Li Ka Shing Knowledge Institute, Toronto, Ontario, Canada
| | - Amanda Parker
- Knowledge Translation Program, St Michael's Hospital Li Ka Shing Knowledge Institute, Toronto, Ontario, Canada
| | - Vera Nincic
- Knowledge Translation Program, St Michael's Hospital Li Ka Shing Knowledge Institute, Toronto, Ontario, Canada
| | - Charmalee Harris
- Knowledge Translation Program, St Michael's Hospital Li Ka Shing Knowledge Institute, Toronto, Ontario, Canada
| | - Orna Fennelly
- Irish Centre for High End Computing (ICHEC), National University of Ireland Galway, Galway, Ireland
| | - Sonia M Thomas
- Knowledge Translation Program, St Michael's Hospital Li Ka Shing Knowledge Institute, Toronto, Ontario, Canada
| | - Marco Ghassemi
- Knowledge Translation Program, St Michael's Hospital Li Ka Shing Knowledge Institute, Toronto, Ontario, Canada
| | - Jessie McGowan
- School of Epidemiology and Public Health, University of Ottawa, Ottawa, Ontario, Canada
| | - P Alison Paprica
- Institute for Health Policy, Management and Evaluation, University of Toronto Dalla Lana School of Public Health, Toronto, Ontario, Canada
| | - Sharon E Straus
- Knowledge Translation Program, St Michael's Hospital Li Ka Shing Knowledge Institute, Toronto, Ontario, Canada
- Department of Medicine, University of Toronto, Toronto, Ontario, Canada
| |
Collapse
|
11
|
Evaluation of a Machine Learning-Based Dysphagia Prediction Tool in Clinical Routine: A Prospective Observational Cohort Study. Dysphagia 2023:10.1007/s00455-022-10548-9. [PMID: 36625964 PMCID: PMC9831015 DOI: 10.1007/s00455-022-10548-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/11/2022] [Accepted: 12/28/2022] [Indexed: 01/11/2023]
Abstract
Based on a large number of pre-existing documented electronic health records (EHR), we developed a machine learning (ML) algorithm for detection of dysphagia and aspiration pneumonia. The aim of our study was to prospectively apply this algorithm in two large patient cohorts. The tool was integrated in the hospital information system of a secondary care hospital in Austria. Based on existing data such as diagnoses, laboratory, and medication, dysphagia risk was predicted automatically, and patients were stratified into three risk groups. Patients' risk groups and risk factors were visualized in a web application. Prospective predictions of 1270 admissions to geriatric or internal medicine departments were compared with the occurrence of dysphagia or aspiration pneumonia of routinely documented events. The discriminative performance for internal medicine patients (n = 885) was excellent with an AUROC of 0.841, a sensitivity of 74.2%, and a specificity of 84.1%. For the smaller geriatric cohort (n = 221), the AUROC was 0.758, sensitivity 44.4%, and specificity 93.0%. For both cohorts, calibration plots showed a slight overestimation of the risk. This is the first study to evaluate the performance of a ML-based prediction tool for dysphagia in a prospective clinical setting. Future studies should validate the predictions on data of systematic dysphagia screening by specialists and evaluate user satisfaction and acceptance. The ML-based dysphagia prediction tool achieved an excellent performance in the internal medicine cohort. More data are needed to determine the performance in geriatric patients.
Collapse
|
12
|
Tulk Jesso S, Kelliher A, Sanghavi H, Martin T, Henrickson Parker S. Inclusion of Clinicians in the Development and Evaluation of Clinical Artificial Intelligence Tools: A Systematic Literature Review. Front Psychol 2022; 13:830345. [PMID: 35465567 PMCID: PMC9022040 DOI: 10.3389/fpsyg.2022.830345] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2021] [Accepted: 02/09/2022] [Indexed: 12/11/2022] Open
Abstract
The application of machine learning (ML) and artificial intelligence (AI) in healthcare domains has received much attention in recent years, yet significant questions remain about how these new tools integrate into frontline user workflow, and how their design will impact implementation. Lack of acceptance among clinicians is a major barrier to the translation of healthcare innovations into clinical practice. In this systematic review, we examine when and how clinicians are consulted about their needs and desires for clinical AI tools. Forty-five articles met criteria for inclusion, of which 24 were considered design studies. The design studies used a variety of methods to solicit and gather user feedback, with interviews, surveys, and user evaluations. Our findings show that tool designers consult clinicians at various but inconsistent points during the design process, and most typically at later stages in the design cycle (82%, 19/24 design studies). We also observed a smaller amount of studies adopting a human-centered approach and where clinician input was solicited throughout the design process (22%, 5/24). A third (15/45) of all studies reported on clinician trust in clinical AI algorithms and tools. The surveyed articles did not universally report validation against the “gold standard” of clinical expertise or provide detailed descriptions of the algorithms or computational methods used in their work. To realize the full potential of AI tools within healthcare settings, our review suggests there are opportunities to more thoroughly integrate frontline users’ needs and feedback in the design process.
Collapse
Affiliation(s)
- Stephanie Tulk Jesso
- Fralin Biomedical Research Institute, Virginia Tech, Roanoke, VA, United States.,Institute for Creativity, Arts, and Technology, Blacksburg, VA, United States
| | - Aisling Kelliher
- Department of Computer Science, College of Engineering, Virginia Tech, Blacksburg, VA, United States
| | | | - Thomas Martin
- Institute for Creativity, Arts, and Technology, Blacksburg, VA, United States.,Department of Electrical and Computer Engineering, College of Engineering, Virginia Tech, Blacksburg, VA, United States
| | - Sarah Henrickson Parker
- Fralin Biomedical Research Institute, Virginia Tech, Roanoke, VA, United States.,Department of Health Systems and Implementation Science, Virginia Tech Carilion School of Medicine, Roanoke, VA, United States
| |
Collapse
|
13
|
Wang Q, Hou L, Hong JC, Yang X, Zhang M. Impact of Face-Recognition-Based Access Control System on College Students' Sense of School Identity and Belonging During COVID-19 Pandemic. Front Psychol 2022; 13:808189. [PMID: 35250739 PMCID: PMC8889067 DOI: 10.3389/fpsyg.2022.808189] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2021] [Accepted: 01/28/2022] [Indexed: 11/23/2022] Open
Abstract
In the context of coronavirus pandemic (COVID-19), the face-recognition-based access control system (FACS) has been intensively adopted to protect students’ and teachers’ health and safety in school. However, the impact of FACS, as a new technology, on students’ attitude toward accepting FACS has remained unknown from the psychological halo effect. Drawn on “halo effect” theory where psychological effects affect the sense of social identity and belonging, the present study explored college students’ sense of school identity and belonging in using FACS during COVID-19 based on the technology acceptance model (TAM). Data collected from 391 college students was analyzed using SEM to verify the relationship among perceived usefulness (PU), perceived ease of use (PEU), intention to use (IU), school identity, and school belonging. The results show that PU and PEU can positively predict IU, and consequentially can positively predict school identity and school belonging. Our study expands the application of halo effect theory to study FACS acceptance based on TAM, and provides strong evidence to support the effect of school FACS during the pandemic. The findings of this study also suggest that FACS acceptance can enhance students’ sense of school identity and belonging.
Collapse
Affiliation(s)
- Qiang Wang
- College of Elementary Education and College of Education, Capital Normal University, Beijing, China
| | - Lan Hou
- College of Elementary Education and College of Education, Capital Normal University, Beijing, China
| | - Jon-Chao Hong
- Institute for Research Excellence in Learning Sciences, National Taiwan Normal University, Taipei, Taiwan
| | - Xiantong Yang
- Faculty of Psychology, Beijing Normal University, Beijing, China
| | - Mengmeng Zhang
- School of Education, Minzu University of China, Beijing, China
| |
Collapse
|
14
|
Von Gerich H, Moen H, Block LJ, Chu CH, DeForest H, Hobensack M, Michalowski M, Mitchell J, Nibber R, Olalia MA, Pruinelli L, Ronquillo CE, Topaz M, Peltonen LM. Artificial Intelligence -based technologies in nursing: A scoping literature review of the evidence. Int J Nurs Stud 2021; 127:104153. [DOI: 10.1016/j.ijnurstu.2021.104153] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/11/2021] [Revised: 11/23/2021] [Accepted: 12/01/2021] [Indexed: 12/20/2022]
|