1
|
Nabecker S, Nation K, Gilfoyle E, Abelairas-Gomez C, Koota E, Lin Y, Greif R. Cognitive aids used in simulated resuscitation: A systematic review. Resusc Plus 2024; 19:100675. [PMID: 38873274 PMCID: PMC11170275 DOI: 10.1016/j.resplu.2024.100675] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2024] [Revised: 05/15/2024] [Accepted: 05/16/2024] [Indexed: 06/15/2024] Open
Abstract
Objectives To compare the effectiveness of cognitive aid use during resuscitation with no use of cognitive aids on cardiopulmonary resuscitation quality and performance. Methods This systematic review followed the PICOST format. All randomised controlled trials and non-randomised studies evaluating cognitive aid use during (simulated) resuscitation were included in any setting. Unpublished studies were excluded. We did not include studies that reported cognitive aid use during training for resuscitation alone. Medline, Embase and Cochrane databases were searched from inception until July 2019 (updated August 2022, November 2023, and 23 April 2024). We did not search trial registries. Title and abstract screening, full-text screening, data extraction, risk of bias assessment (using RoB2 and ROBINS-I), and certainty of evidence (using GRADE) were performed by two researchers. PRISMA reporting standards were followed, and registration (PROSPERO CRD42020159162, version 19 July 2022) was performed. No funding has been obtained. Results The literature search identified 5029 citations. After removing 512 duplicates, reviewing the titles and abstracts of the remaining articles yielded 103 articles for full-text review. Hand-searching identified 3 more studies for full-text review. Of these, 29 studies were included in the final analysis. No clinical studies involving patients were identified. The review was limited to indirect evidence from simulation studies only. The results are presented in five different populations: healthcare professionals managing simulated resuscitations in neonates, children, adult advanced life support, and other emergencies; as well as lay providers managing resuscitations. Main outcomes were adherence to protocol or process, adherence to protocol or process assessed by performance score, CPR performance and retention, and feasibility of chatbot guidance. The risk of bias assessment ranged from low to high. Studies in neonatal, paediatric and adult life support delivered by healthcare professionals showed benefits of using cognitive aids, however, some studies evaluating resuscitations by lay providers reported undesirable effects. The performance of a meta-analysis was not possible due to significant methodological heterogeneity. The certainty of evidence was rated as moderate to very low due to serious indirectness, (very) serious risk of bias, serious inconsistency and (very) serious imprecision. Conclusion Because of the very low certainty evidence from simulation studies, we suggest that cognitive aids should be used by healthcare professionals during resuscitation. In contrast, we do not suggest use of cognitive aids for lay providers, based on low certainty evidence.
Collapse
Affiliation(s)
- Sabine Nabecker
- Department of Anesthesiology and Pain Management, Sinai Health System, University of Toronto, Toronto, Canada
| | - Kevin Nation
- New Zealand Resuscitation Council, Wellington, New Zealand
| | - Elaine Gilfoyle
- Department of Critical Care Medicine, The Hospital for Sick Children, Toronto, Canada
| | - Cristian Abelairas-Gomez
- Faculty of Education Sciences and CLINURSID Research Group, Universidade de Santiago de Compostela, Santiago de Compostela, Spain
- Simulation and Intensive Care Unit of Santiago (SICRUS) Research Group, Health Research Institute of Santiago, University Hospital of Santiago de Compostela-CHUS, Santiago de Compostela, Spain
| | - Elina Koota
- HUS Joint Resources, Helsinki University Hospital, University of Helsinki, Helsinki, Finland
| | - Yiqun Lin
- KidSIM Simulation Education and Research Program, Alberta Children’s Hospital, University of Calgary, Calgary, Alberta, Canada
| | - Robert Greif
- University of Bern, Bern, Switzerland
- Department of Surgical Science, University of Torino, Torino, Italy
| |
Collapse
|
2
|
Tawfik M, Schiff E, Mosavian R, Campisi C, Shen A, Lin J, Windsor AM, Weingarten‐Arams J, Soshnick SH, Nishisaki A, Je S, Maa T, Harwayne‐Gidansky I, Fortunov RM, Yang CJ. Validation of a Novel Mobile Application for Assessing Pediatric Tracheostomy Emergency Simulations. OTO Open 2024; 8:e145. [PMID: 38974176 PMCID: PMC11222740 DOI: 10.1002/oto2.145] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2024] [Accepted: 04/27/2024] [Indexed: 07/09/2024] Open
Abstract
Objective Pediatric tracheostomy is associated with high morbidity and mortality, yet clinician knowledge and quality of tracheostomy care may vary widely. In situ simulation is effective at detecting and mitigating related latent safety threats, but evaluation via retrospective video review has disadvantages (eg, delayed analysis, and potential data loss). We evaluated whether a novel mobile application is accurate and reliable for assessment of in situ tracheostomy emergency simulations. Methods A novel mobile application was developed for assessment of tracheostomy emergency in situ simulation team performance. After 1.25 hours of training, 6 raters scored 10 tracheostomy emergency simulation videos for the occurrence and timing of 12 critical steps. To assess accuracy, rater scores were compared to a reference standard to determine agreement for occurrence or absence of critical steps and a timestamp within ±5 seconds. Interrater reliability was determined through Cohen's and Fleiss' kappa and intraclass correlation coefficient. Results Raters had 86.0% agreement with the reference standard when considering step occurrence and timing, and 92.8% agreement when considering only occurrence. The average timestamp difference from the reference standard was 1.3 ± 18.5 seconds. Overall interrater reliability was almost perfect for both step occurrence (Fleiss' kappa of 0.81) and timing of step (intraclass correlation coefficient of 0.99). Discussion Using our novel mobile application, raters with minimal training accurately and reliably assessed videos of tracheostomy emergency simulations and identified areas for future refinement. Implications for Practice With refinements, this innovative mobile application is an effective tool for real-time data capture of time-critical steps in in situ tracheostomy emergency simulations.
Collapse
Affiliation(s)
| | | | | | | | - Amanda Shen
- Albert Einstein College of MedicineBronxNew YorkUSA
| | - Juan Lin
- Albert Einstein College of MedicineBronxNew YorkUSA
| | - Alanna M. Windsor
- Albert Einstein College of MedicineBronxNew YorkUSA
- Department of Otorhinolaryngology–Head and Neck SurgeryMontefiore Medical CenterBronxNew YorkUSA
| | - Jacqueline Weingarten‐Arams
- Albert Einstein College of MedicineBronxNew YorkUSA
- Department of Pediatrics, Division of Pediatric Critical Care MedicineChildren's Hospital at MontefioreBronxNew YorkUSA
| | - Sara H. Soshnick
- Albert Einstein College of MedicineBronxNew YorkUSA
- Department of Pediatrics, Division of Pediatric Critical Care MedicineChildren's Hospital at MontefioreBronxNew YorkUSA
| | - Akira Nishisaki
- University of Pennsylvania Perelman School of MedicinePhiladelphiaPennsylvaniaUSA
- Department of Anesthesiology and Critical Care Medicine, Division of Pediatric Critical Care MedicineChildren's Hospital of PhiladelphiaPhiladelphiaPennsylvaniaUSA
| | - Sangmo Je
- Department of Anesthesiology and Critical Care Medicine, Division of Pediatric Critical Care MedicineChildren's Hospital of PhiladelphiaPhiladelphiaPennsylvaniaUSA
| | - Tensing Maa
- Ohio State University College of MedicineColumbusOhioUSA
- Department of Pediatrics, Division of Pediatric Critical Care MedicineNationwide Children's HospitalColumbusOhioUSA
| | - Ilana Harwayne‐Gidansky
- Department of Pediatrics, Division of Pediatric Critical Care Medicine, The Bernard & Millie Duker Children's HospitalAlbany Medical CenterAlbanyNew YorkUSA
- Albany Medical CollegeAlbanyNew YorkUSA
| | - Regine M. Fortunov
- Department of Pediatrics, Division of NeonatologyBaylor College of MedicineHoustonTexasUSA
- Texas Children's HospitalHoustonTexasUSA
| | - Christina J. Yang
- Albert Einstein College of MedicineBronxNew YorkUSA
- Department of Otorhinolaryngology–Head and Neck SurgeryMontefiore Medical CenterBronxNew YorkUSA
| | | |
Collapse
|
3
|
Pham C, Govender R, Tehami S, Chavez S, Adepoju OE, Liaw W. ChatGPT's Performance in Cardiac Arrest and Bradycardia Simulations Using the American Heart Association's Advanced Cardiovascular Life Support Guidelines: Exploratory Study. J Med Internet Res 2024; 26:e55037. [PMID: 38648098 DOI: 10.2196/55037] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2023] [Revised: 02/22/2024] [Accepted: 03/10/2024] [Indexed: 04/25/2024] Open
Abstract
BACKGROUND ChatGPT is the most advanced large language model to date, with prior iterations having passed medical licensing examinations, providing clinical decision support, and improved diagnostics. Although limited, past studies of ChatGPT's performance found that artificial intelligence could pass the American Heart Association's advanced cardiovascular life support (ACLS) examinations with modifications. ChatGPT's accuracy has not been studied in more complex clinical scenarios. As heart disease and cardiac arrest remain leading causes of morbidity and mortality in the United States, finding technologies that help increase adherence to ACLS algorithms, which improves survival outcomes, is critical. OBJECTIVE This study aims to examine the accuracy of ChatGPT in following ACLS guidelines for bradycardia and cardiac arrest. METHODS We evaluated the accuracy of ChatGPT's responses to 2 simulations based on the 2020 American Heart Association ACLS guidelines with 3 primary outcomes of interest: the mean individual step accuracy, the accuracy score per simulation attempt, and the accuracy score for each algorithm. For each simulation step, ChatGPT was scored for correctness (1 point) or incorrectness (0 points). Each simulation was conducted 20 times. RESULTS ChatGPT's median accuracy for each step was 85% (IQR 40%-100%) for cardiac arrest and 30% (IQR 13%-81%) for bradycardia. ChatGPT's median accuracy over 20 simulation attempts for cardiac arrest was 69% (IQR 67%-74%) and for bradycardia was 42% (IQR 33%-50%). We found that ChatGPT's outputs varied despite consistent input, the same actions were persistently missed, repetitive overemphasis hindered guidance, and erroneous medication information was presented. CONCLUSIONS This study highlights the need for consistent and reliable guidance to prevent potential medical errors and optimize the application of ChatGPT to enhance its reliability and effectiveness in clinical practice.
Collapse
Affiliation(s)
- Cecilia Pham
- Tilman J Fertitta Family College of Medicine, University of Houston, Houston, TX, United States
| | - Romi Govender
- Tilman J Fertitta Family College of Medicine, University of Houston, Houston, TX, United States
| | - Salik Tehami
- Tilman J Fertitta Family College of Medicine, University of Houston, Houston, TX, United States
| | - Summer Chavez
- Tilman J Fertitta Family College of Medicine, University of Houston, Houston, TX, United States
- Humana Integrated Health Sciences Institute, University of Houston, Houston, TX, United States
- Department of Health Systems and Population Health Sciences, Tilman J Fertitta Family College of Medicine, Houston, TX, United States
| | - Omolola E Adepoju
- Tilman J Fertitta Family College of Medicine, University of Houston, Houston, TX, United States
- Humana Integrated Health Sciences Institute, University of Houston, Houston, TX, United States
- Department of Health Systems and Population Health Sciences, Tilman J Fertitta Family College of Medicine, Houston, TX, United States
| | - Winston Liaw
- Tilman J Fertitta Family College of Medicine, University of Houston, Houston, TX, United States
- Humana Integrated Health Sciences Institute, University of Houston, Houston, TX, United States
- Department of Health Systems and Population Health Sciences, Tilman J Fertitta Family College of Medicine, Houston, TX, United States
| |
Collapse
|
4
|
Corazza F, Fiorese E, Arpone M, Tardini G, Frigo AC, Cheng A, Da Dalt L, Bressan S. The impact of cognitive aids on resuscitation performance in in-hospital cardiac arrest scenarios: a systematic review and meta-analysis. Intern Emerg Med 2022; 17:2143-2158. [PMID: 36031672 PMCID: PMC9420676 DOI: 10.1007/s11739-022-03041-6] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/10/2021] [Accepted: 06/20/2022] [Indexed: 11/24/2022]
Abstract
Different cognitive aids have been recently developed to support the management of cardiac arrest, however, their effectiveness remains barely investigated. We aimed to assess whether clinicians using any cognitive aids compared to no or alternative cognitive aids for in-hospital cardiac arrest (IHCA) scenarios achieve improved resuscitation performance. PubMed, EMBASE, the Cochrane Library, CINAHL and ClinicalTrials.gov were systematically searched to identify studies comparing the management of adult/paediatric IHCA simulated scenarios by health professionals using different or no cognitive aids. Our primary outcomes were adherence to guideline recommendations (overall team performance) and time to critical resuscitation actions. Random-effects model meta-analyses were performed. Of the 4.830 screened studies, 16 (14 adult, 2 paediatric) met inclusion criteria. Meta-analyses of eight eligible adult studies indicated that the use of electronic/paper-based cognitive aids, in comparison with no aid, was significantly associated with better overall resuscitation performance [standard mean difference (SMD) 1.16; 95% confidence interval (CI) 0.64; 1.69; I2 = 79%]. Meta-analyses of the two paediatric studies, showed non-significant improvement of critical actions for resuscitation (adherence to guideline recommended sequence of actions, time to defibrillation, rate of errors in defibrillation, time to start chest compressions), except for significant shorter time to amiodarone administration (SMD - 0.78; 95% CI - 1.39; - 0.18; I2 = 0). To conclude, the use of cognitive aids appears to have benefits in improving the management of simulated adult IHCA scenarios, with potential positive impact on clinical practice. Further paediatric studies are necessary to better assess the impact of cognitive aids on the management of IHCA scenarios.
Collapse
Affiliation(s)
- Francesco Corazza
- Division of Pediatric Emergency Medicine, University Hospital of Padova, Padova, Italy
| | - Elena Fiorese
- Department of Women's and Children's Health, University of Padova, Padova, Italy
| | - Marta Arpone
- Department of Women's and Children's Health, University of Padova, Padova, Italy
| | - Giacomo Tardini
- Department of Women's and Children's Health, University of Padova, Padova, Italy
| | - Anna Chiara Frigo
- Biostatistics, Epidemiology and Public Health Unit, Department of Cardiac, Thoracic, Vascular Sciences and Public Health, University of Padova, Padova, Italy
| | - Adam Cheng
- Departments of Paediatrics and Emergency Medicine, Alberta Children's Hospital, University of Calgary, Calgary, Canada
| | - Liviana Da Dalt
- Division of Pediatric Emergency Medicine, University Hospital of Padova, Padova, Italy
- Department of Women's and Children's Health, University of Padova, Padova, Italy
| | - Silvia Bressan
- Division of Pediatric Emergency Medicine, University Hospital of Padova, Padova, Italy.
- Department of Women's and Children's Health, University of Padova, Padova, Italy.
| |
Collapse
|
5
|
Matthews S, Atkins B, Walton N, Mitchell JA, Jennings G, Buttery AK. Development and Use of a Cardiac Clinical Guideline Mobile App in Australia: Acceptability and Multi-Methods Study. JMIR Form Res 2022; 6:e35599. [PMID: 35930350 PMCID: PMC9391980 DOI: 10.2196/35599] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/10/2021] [Revised: 03/03/2022] [Accepted: 06/20/2022] [Indexed: 11/28/2022] Open
Abstract
Background Implementation of clinical guidelines into routine practice remains highly variable. Strategies to increase guideline uptake include developing digital tools and mobile apps for use in clinical practice. The National Heart Foundation of Australia in collaboration with the Cardiac Society of Australia and New Zealand published 3 key cardiac clinical guidelines, including the Australian clinical guidelines for the (1) prevention and detection of atrial fibrillation, (2) detection and management of heart failure, and (3) management of acute coronary syndromes. To improve access and uptake for health care providers, we developed the Smart Heart Guideline App. Objective This study aims to evaluate the acceptability, implementation, and usability of an Australian-specific cardiac guidelines mobile app. Methods We used an iterative multiple methods development and implementation approach. First, we conducted a cross-sectional web-based survey with end users (n=504 health professionals) in 2017 to determine the acceptability of an Australian-specific cardiac clinical guidelines mobile app. Second, the Smart Heart Guidelines app was created using a design, user testing, and revision process. The app includes interactive algorithms and flowcharts to inform diagnosis and management at the point of care. The freely available app was launched in October 2019 on iOS and Android operating systems and promoted and implemented using multiple methods. Third, data from 2 annual national cross-sectional general practitioner (GP) surveys in 2019 and 2020 were evaluated to understand the awareness and use of the clinical guidelines and the app. Fourth, data from the app stores were analyzed between October 1, 2019, and June 30, 2021, to evaluate usage. Results Most health professionals surveyed (447/504, 89%) reported accessing resources electronically, and most (318/504, 63%) reported that they would use an Australian-specific cardiac guidelines app. GPs surveyed in 2019 were aware of the heart failure (159/312, 51%) and atrial fibrillation (140/312, 45%) guidelines, and in 2020, a total of 34 of 189 (18%) reported that they were aware of the app. The app was downloaded 11,313 times (7483, 66% from the Apple App Store; 3830, 34% from Google Play) during the first 20-month period. Most downloads (6300/7483, 84%) were a result of searching for the app in the stores. Monthly download rates varied. App Store data showed that people used the app twice (on average 2.06 times) during the 20 months. Many (3256/3830, 85%) Android users deleted the app. Conclusions Health professionals supported the development of the Smart Heart Guidelines app. Although initial downloads were promising, the frequency of using the app was low and deletion rates were high. Further evaluation of users’ experience of the most and least useful components of the app is needed.
Collapse
Affiliation(s)
- Stacey Matthews
- National Heart Foundation of Australia, Melbourne, Australia
- Royal Melbourne Hospital, Cardiothoracic Surgery Unit, Parkville, Australia
| | - Brooke Atkins
- National Heart Foundation of Australia, Melbourne, Australia
| | - Natalie Walton
- National Heart Foundation of Australia, Melbourne, Australia
| | | | - Garry Jennings
- National Heart Foundation of Australia, Melbourne, Australia
- Baker Heart and Diabetes Institute, Melbourne, Australia
| | | |
Collapse
|