Improving Emergency Department Patient-Physician Conversation Through an Artificial Intelligence Symptom-Taking Tool: Mixed Methods Pilot Observational Study

Quality of interaction between clinicians and artificial intelligence systems. A systematic review

Introduction

Artificial intelligence (AI) has the potential to improve healthcare quality when thoughtfully integrated into clinical practice. Current evaluations of AI solutions tend to focus solely on model performance. There is a critical knowledge gap in the assessment of AI-clinician interactions. We systematically reviewed existing literature to identify interaction traits that can be used to assess the quality of AI-clinician interactions.

Methods

We performed a systematic review of published studies to June 2022 that reported elements of interactions that impacted the relationship between clinicians and AI-enabled clinical decision support systems. Due to study heterogeneity, we conducted a narrative synthesis of the different interaction traits identified from this review. Two study authors categorised the AI-clinician interaction traits based on their shared constructs independently. After the independent categorisation, both authors engaged in a discussion to finalise the categories.

Results

From 34 included studies, we identified 210 interaction traits. The most common interaction traits included usefulness, ease of use, trust, satisfaction, willingness to use and usability. After removing duplicate or redundant traits, 90 unique interaction traits were identified. Unique interaction traits were then classified into seven categories: usability and user experience, system performance, clinician trust and acceptance, impact on patient care, communication, ethical and professional concerns, and clinician engagement and workflow.

Discussion

We identified seven categories of interaction traits between clinicians and AI systems. The proposed categories may serve as a foundation for a framework assessing the quality of AI-clinician interactions.

Healthcare leaders' experiences of implementing artificial intelligence for medical history-taking and triage in Swedish primary care: an interview study

BACKGROUND

Artificial intelligence (AI) holds significant promise for enhancing the efficiency and safety of medical history-taking and triage within primary care. However, there remains a dearth of knowledge concerning the practical implementation of AI systems for these purposes, particularly in the context of healthcare leadership. This study explores the experiences of healthcare leaders regarding the barriers to implementing an AI application for automating medical history-taking and triage in Swedish primary care, as well as the actions they took to overcome these barriers. Furthermore, the study seeks to provide insights that can inform the development of AI implementation strategies for healthcare.

METHODS

We adopted an inductive qualitative approach, conducting semi-structured interviews with 13 healthcare leaders representing seven primary care units across three regions in Sweden. The collected data were subsequently analysed utilizing thematic analysis. Our study adhered to the Consolidated Criteria for Reporting Qualitative Research to ensure transparent and comprehensive reporting.

RESULTS

The study identified implementation barriers encountered by healthcare leaders across three domains: (1) healthcare professionals, (2) organization, and (3) technology. The first domain involved professional scepticism and resistance, the second involved adapting traditional units for digital care, and the third inadequacies in AI application functionality and system integration. To navigate around these barriers, the leaders took steps to (1) address inexperience and fear and reduce professional scepticism, (2) align implementation with digital maturity and guide patients towards digital care, and (3) refine and improve the AI application and adapt to the current state of AI application development.

CONCLUSION

The study provides valuable empirical insights into the implementation of AI for automating medical history-taking and triage in primary care as experienced by healthcare leaders. It identifies the barriers to this implementation and how healthcare leaders aligned their actions to overcome them. While progress was evident in overcoming professional-related and organizational-related barriers, unresolved technical complexities highlight the importance of AI implementation strategies that consider how leaders handle AI implementation in situ based on practical wisdom and tacit understanding. This underscores the necessity of a holistic approach for the successful implementation of AI in healthcare.

Strategies to Measure and Improve Emergency Department Performance: A Review

Emergency Departments (EDs) globally face escalating challenges such as overcrowding, resource limitations, and increased patient demand. This study aims to identify and analyze strategies to enhance the structural performance of EDs, with a focus on reducing overcrowding, optimizing resource allocation, and improving patient outcomes. Through a comprehensive review of the literature and observational studies, the research highlights the effectiveness of various approaches, including triage optimization, dynamic staffing, technological integration, and strategic resource management. Key findings indicate that tailored strategies, such as implementing advanced triage protocols and leveraging telemedicine, can significantly reduce wait times and enhance patient throughput. Furthermore, evidence suggests that dynamic staffing models and the integration of cutting-edge diagnostic tools contribute to operational efficiency and improved quality of care. These strategies, when combined, offer a multifaceted solution to the complex challenges faced by EDs, promising better patient care and satisfaction. The study underscores the need for a comprehensive approach, incorporating both organizational and technological innovations, to address the evolving needs of emergency healthcare.

Holistic Human-Serving Digitization of Health Care Needs Integrated Automated System-Level Assessment Tools

Digital health tools, platforms, and artificial intelligence- or machine learning-based clinical decision support systems are increasingly part of health delivery approaches, with an ever-greater degree of system interaction. Critical to the successful deployment of these tools is their functional integration into existing clinical routines and workflows. This depends on system interoperability and on intuitive and safe user interface design. The importance of minimizing emergent workflow stress through human factors research and purposeful design for integration cannot be overstated. Usability of tools in practice is as important as algorithm quality. Regulatory and health technology assessment frameworks recognize the importance of these factors to a certain extent, but their focus remains mainly on the individual product rather than on emergent system and workflow effects. The measurement of performance and user experience has so far been performed in ad hoc, nonstandardized ways by individual actors using their own evaluation approaches. We propose that a standard framework for system-level and holistic evaluation could be built into interacting digital systems to enable systematic and standardized system-wide, multiproduct, postmarket surveillance and technology assessment. Such a system could be made available to developers through regulatory or assessment bodies as an application programming interface and could be a requirement for digital tool certification, just as interoperability is. This would enable health systems and tool developers to collect system-level data directly from real device use cases, enabling the controlled and safe delivery of systematic quality assessment or improvement studies suitable for the complexity and interconnectedness of clinical workflows using developing digital health technologies.

Proof of Concept: Using ChatGPT to Teach Emergency Physicians How to Break Bad News

Background Breaking bad news is an essential skill for practicing physicians, particularly in the field of emergency medicine (EM). Patient-physician communication teaching has previously relied on standardized patient scenarios and objective structured clinical examination formats. The novel use of artificial intelligence (AI) chatbot technology, such as Chat Generative Pre-trained Transformer (ChatGPT), may provide an alternative role in graduate medical education in this area. As a proof of concept, the author demonstrates how providing detailed prompts to the AI chatbot can facilitate the design of a realistic clinical scenario, enable active roleplay, and deliver effective feedback to physician trainees. Methods ChatGPT-3.5 language model was utilized to assist in the roleplay of breaking bad news. A detailed input prompt was designed to outline rules of play and grading assessment via a standardized scale. User inputs (physician role), chatbot outputs (patient role) and ChatGPT-generated feedback were recorded. Results ChatGPT set up a realistic training scenario on breaking bad news based on the initial prompt. Active roleplay as a patient in an emergency department setting was accomplished, and clear feedback was provided to the user through the application of the Setting up, Perception, Invitation, Knowledge, Emotions with Empathy, and Strategy or Summary (SPIKES) framework for breaking bad news. Conclusion The novel use of AI chatbot technology to assist educators is abundant with potential. ChatGPT was able to design an appropriate scenario, provide a means for simulated patient-physician roleplay, and deliver real-time feedback to the physician user. Future studies are required to expand use to a targeted group of EM physician trainees and provide best practice guidelines for AI use in graduate medical education.

Electronic Discharge Communication Tools Used in Pediatric Emergency Departments: Systematic Review

BACKGROUND

Electronic discharge communication tools (EDCTs) are increasingly common in pediatric emergency departments (EDs). These tools have been shown to improve patient-centered communication, support postdischarge care at home, and reduce unnecessary return visits to the ED.

OBJECTIVE

This study aimed to map and assess the evidence base for EDCTs used in pediatric EDs according to their functionalities, intended purpose, implementation context features, and outcomes.

METHODS

A systematic review was conducted following PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) procedures for identification, screening, and eligibility. A total of 7 databases (EBSCO, MEDLINE, CINAHL, PsycINFO, EMBASE Scopus, and Web of Science) were searched for studies published between 1989 and 2021. Studies evaluating discharge communication-related outcomes using electronic tools (eg, text messages, videos, and kiosks) in pediatric EDs were included. In all, 2 researchers independently assessed the eligibility. Extracted data related to study identification, methodology, settings and demographics, intervention features, outcome implementation features, and practice, policy, and research implications. The Mixed Method Appraisal Tool was used to assess methodological quality. The synthesis of results involved structured tabulation, vote counting, recoding into common metrics, inductive thematic analysis, descriptive statistics, and heat mapping.

RESULTS

In total, 231 full-text articles and abstracts were screened for review inclusion with 49 reports (representing 55 unique tools) included. In all, 70% (26/37) of the studies met at least three of five Mixed Method Appraisal Tool criteria. The most common EDCTs were videos, text messages, kiosks, and phone calls. The time required to use the tools ranged from 120 seconds to 80 minutes. The EDCTs were evaluated for numerous presenting conditions (eg, asthma, fracture, head injury, fever, and otitis media) that required a range of at-home care needs after the ED visit. The most frequently measured outcomes were knowledge acquisition, caregiver and patient beliefs and attitudes, and health service use. Unvalidated self-report measures were typically used for measurement. Health care provider satisfaction or system-level impacts were infrequently measured in studies. The directionality of primary outcomes pointed to positive effects for the primary measure (44/55, 80%) or no significant difference (10/55, 18%). Only one study reported negative findings, with an increase in return visits to the ED after receiving the intervention compared with the control group.

CONCLUSIONS

This review is the first to map the broad literature of EDCTs used in pediatric EDs. The findings suggest a promising evidence base, demonstrating that EDCTs have been successfully integrated across clinical contexts and deployed via diverse technological modalities. Although caregiver and patient satisfaction with EDCTs is high, future research should use robust trials using consistent measures of communication quality, clinician experience, cost-effectiveness, and health service use to accumulate evidence regarding these outcomes.

TRIAL REGISTRATION

PROSPERO CRD42020157500; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=157500.

Investigating the Potential for Clinical Decision Support in Sub-Saharan Africa With AFYA (Artificial Intelligence-Based Assessment of Health Symptoms in Tanzania): Protocol for a Prospective, Observational Pilot Study

BACKGROUND

Low- and middle-income countries face difficulties in providing adequate health care. One of the reasons is a shortage of qualified health workers. Diagnostic decision support systems are designed to aid clinicians in their work and have the potential to mitigate pressure on health care systems.

OBJECTIVE

The Artificial Intelligence-Based Assessment of Health Symptoms in Tanzania (AFYA) study will evaluate the potential of an English-language artificial intelligence-based prototype diagnostic decision support system for mid-level health care practitioners in a low- or middle-income setting.

METHODS

This is an observational, prospective clinical study conducted in a busy Tanzanian district hospital. In addition to usual care visits, study participants will consult a mid-level health care practitioner, who will use a prototype diagnostic decision support system, and a study physician. The accuracy and comprehensiveness of the differential diagnosis provided by the diagnostic decision support system will be evaluated against a gold-standard differential diagnosis provided by an expert panel.

RESULTS

Patient recruitment started in October 2021. Participants were recruited directly in the waiting room of the outpatient clinic at the hospital. Data collection will conclude in May 2022. Data analysis is planned to be finished by the end of June 2022. The results will be published in a peer-reviewed journal.

CONCLUSIONS

Most diagnostic decision support systems have been developed and evaluated in high-income countries, but there is great potential for these systems to improve the delivery of health care in low- and middle-income countries. The findings of this real-patient study will provide insights based on the performance and usability of a prototype diagnostic decision support system in low- or middle-income countries.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04958577; http://clinicaltrials.gov/ct2/show/NCT04958577.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/34298.

Study protocol for a pilot prospective, observational study investigating the condition suggestion and urgency advice accuracy of a symptom assessment app in sub-Saharan Africa: the AFYA-'Health' Study

INTRODUCTION

Due to a global shortage of healthcare workers, there is a lack of basic healthcare for 4 billion people worldwide, particularly affecting low-income and middle-income countries. The utilisation of AI-based healthcare tools such as symptom assessment applications (SAAs) has the potential to reduce the burden on healthcare systems. The purpose of the AFYA Study (AI-based Assessment oF health sYmptoms in TAnzania) is to evaluate the accuracy of the condition suggestions and urgency advice provided by a user on a Swahili language Ada SAA.

METHODS AND ANALYSIS

This study is designed as an observational prospective clinical study. The setting is a waiting room of a Tanzanian district hospital. It will include patients entering the outpatient clinic with various conditions and age groups, including children and adolescents. Patients will be asked to use the SAA before proceeding to usual care. After usual care, they will have a consultation with a study-provided physician. Patients and healthcare practitioners will be blinded to the SAA's results. An expert panel will compare the Ada SAA's condition suggestions and urgency advice to usual care and study provided differential diagnoses and triage. The primary outcome measures are the accuracy and comprehensiveness of the Ada SAA evaluated against the gold standard differential diagnoses.

ETHICS AND DISSEMINATION

Ethical approval was received by the ethics committee (EC) of Muhimbili University of Health and Allied Sciences with an approval number MUHAS-REC-09-2019-044 and the National Institute for Medical Research, NIMR/HQ/R.8c/Vol. I/922. All amendments to the protocol are reported and adapted on the basis of the requirements of the EC. The results from this study will be submitted to peer-reviewed journals, local and international stakeholders, and will be communicated in editorials/articles by Ada Health.

TRIAL REGISTRATION NUMBER

NCT04958577.