Virtual Standardized Patients vs Academic Training for Learning Motivational Interviewing Skills in the US Department of Veterans Affairs and the US Military: A Randomized Trial

Roles, Users, Benefits, and Limitations of Chatbots in Health Care: Rapid Review

BACKGROUND

Chatbots, or conversational agents, have emerged as significant tools in health care, driven by advancements in artificial intelligence and digital technology. These programs are designed to simulate human conversations, addressing various health care needs. However, no comprehensive synthesis of health care chatbots' roles, users, benefits, and limitations is available to inform future research and application in the field.

OBJECTIVE

This review aims to describe health care chatbots' characteristics, focusing on their diverse roles in the health care pathway, user groups, benefits, and limitations.

METHODS

A rapid review of published literature from 2017 to 2023 was performed with a search strategy developed in collaboration with a health sciences librarian and implemented in the MEDLINE and Embase databases. Primary research studies reporting on chatbot roles or benefits in health care were included. Two reviewers dual-screened the search results. Extracted data on chatbot roles, users, benefits, and limitations were subjected to content analysis.

RESULTS

The review categorized chatbot roles into 2 themes: delivery of remote health services, including patient support, care management, education, skills building, and health behavior promotion, and provision of administrative assistance to health care providers. User groups spanned across patients with chronic conditions as well as patients with cancer; individuals focused on lifestyle improvements; and various demographic groups such as women, families, and older adults. Professionals and students in health care also emerged as significant users, alongside groups seeking mental health support, behavioral change, and educational enhancement. The benefits of health care chatbots were also classified into 2 themes: improvement of health care quality and efficiency and cost-effectiveness in health care delivery. The identified limitations encompassed ethical challenges, medicolegal and safety concerns, technical difficulties, user experience issues, and societal and economic impacts.

CONCLUSIONS

Health care chatbots offer a wide spectrum of applications, potentially impacting various aspects of health care. While they are promising tools for improving health care efficiency and quality, their integration into the health care system must be approached with consideration of their limitations to ensure optimal, safe, and equitable use.

Designing and Evaluating an Emotionally Responsive Virtual Patient Simulation

INTRODUCTION

Virtual patient (VP) simulations have been widely used for healthcare training, education, and assessment. However, few VP systems have integrated emotion sensing and analyzed how a user's emotions may influence the overall training experience. This article presents a VP that can recognize and respond to 5 human emotions (anger, disgust, fear, joy, and sadness), as well as 2 facial expressions (smiling and eye contact).

METHODS

The VP was developed by combining the capabilities of a facial recognition system, a tone analyzer, a cloud-based artificial intelligence chatbot, and interactive 3-dimensional avatars created in a high-fidelity game engine (Unity). The system was tested with healthcare professionals at Changzhou Traditional Chinese Medicine Hospital.

RESULTS

A total of 65 participants (38 females and 27 males) aged between 23 and 57 years (mean = 38.35, SD = 11.48) completed the survey, and 19 participants were interviewed. Most participants perceived that the VP was useful in improving their communication skills, particularly their nonverbal communication skills. They also reported that adding users' affective states as an additional interaction increased engagement of the VP and helped them build connections with the VP.

CONCLUSIONS

The emotionally responsive VP seemed to be functionally complete and usable. However, some technical limitations need to be addressed before the system's official implementation in real-world clinical practice. Future development will include improving the accuracy of the speech recognition system, using more sophisticated emotion sensing software, and developing a natural user interface.

Effects of Training in Cognitive Behavioural Therapy and Motivational Interviewing on Mental Health Practitioner Behaviour: A Systematic Review and Meta-Analysis

Effective training of mental health professionals is crucial for bridging the gap between research and practice when delivering cognitive behavioural therapy (CBT) and motivational interviewing (MI) within community settings. However, previous research has provided inconclusive evidence regarding the impact of training efforts. The current study aimed to systematically search, review and synthesize the literature on CBT and MI training to assess its effect on practitioner behavioural outcomes. Following prospective registration, a literature search was conducted for studies where mental health practitioners were exposed to training in face-to-face CBT or MI, reporting on at least one quantitative practitioner behavioural outcome. A total of 116 studies were eligible for the systematic review, and 20 studies were included in four meta-analyses. The systematic review highlights the need to establish psychometrically valid outcome measures for practitioner behaviour. Results of the meta-analyses suggest that training has a greater effect on practitioner behaviour change compared to receiving no training or reading a treatment manual. Training combined with consultation/supervision was found to be more effective than training alone, and no differences were found between face-to-face and online training. Results should be interpreted with caution due to methodological limitations in the primary studies, large heterogeneity, and small samples in the meta-analyses. Future directions are discussed.

Motivational Interviewing Education in North American Family Medicine Clerkships: A CERA Study

BACKGROUND AND OBJECTIVES

Many health conditions are preventable or modifiable through behavioral changes. Motivational interviewing (MI) is an evidence-based communication technique that explores a patient's reasons for behavioral changes. This study assesses the current landscape of MI training in North American Family Medicine (FM) clerkships.

METHODS

We analyzed data gathered as part of the 2022 Council of Academic Family Medicine's Educational Research Alliance (CERA) survey of FM clerkship directors (CDs). The survey was distributed via email invitation to 159 US and Canadian FM CDs in June 2022.

RESULTS

Of the 94 responses received, 61% indicated that MI training is provided in their FM clerkship. Medical school type, class size, and location were associated with MI training priority, offerings, and duration in the clerkship, respectively. CD experience correlated with MI training duration; student MI skill training level was associated with MI training duration and priority; the rigor of student MI skills evaluation was correlated with MI teaching methods and training duration; self-reported student MI competency was associated with the length of time students spent with FM community preceptors as well as MI training priority and teaching methods; and several items emerged as predictors of student, CD, and FM faculty MI training expansion.

CONCLUSIONS

Opportunities exist to enhance the volume, content, and rigor of MI training in North American FM clerkships as well as to improve self-reported student MI competency within those clerkships.

Virtual Standardized Patients Versus Traditional Academic Training for Improving Clinical Competence Among Traditional Chinese Medicine Students: Prospective Randomized Controlled Trial

BACKGROUND

The practical training course of internal medicine of traditional Chinese medicine (PTC-IMTCM) is primarily based on traditional case teaching, which can be stressful for teachers. The use of virtual standardized patient (VSP) applications could be an alternative; however, there is limited evidence regarding their feasibility and effectiveness.

OBJECTIVE

This study aimed to build a VSP-TCM application according to the characteristics of PTC-IMTCM and the needs of students and to compare its efficacy with that of traditional teaching in improving TCM clinical competence among students.

METHODS

A prequestionnaire investigation was conducted before the course, and a VSP-TCM system was developed based on the results of the questionnaire. A randomized controlled trial was then conducted between February 26, 2020, and August 20, 2021. A total of 84 medical students were included and were divided into 2 groups: an observation group, trained with VSP-TCM (n=42, 50%), and a control group, trained with traditional academic training (n=42, 50%). Formative and summative assessments were conducted to evaluate teaching effectiveness. After completing the course, the students were administered a questionnaire to self-assess their satisfaction with the course. A questionnaire was also administered to 15 teachers to uncover their perspectives on VSP-TCM.

RESULTS

All participants completed the study. In the formative assessment, the VSP-TCM group performed better in medical interviewing ability (mean 7.19, SD 0.63, vs mean 6.83, SD 0.81; P=.04), clinical judgment (mean 6.48, SD 0.98, vs mean 5.86, SD 1.04; P=.006), and comprehensive ability (mean 6.71, SD 0.59, vs mean 6.40, SD 0.58; P=.02) than the control group. Similarly, in the summative evaluation, the VSP-TCM group performed better in the online systematic knowledge test (OSKT; mean 86.62, SD 2.71, vs mean 85.38, SD 2.62; P=.046), application of TCM technology (mean 87.86, SD 3.04, vs mean 86.19, SD 3.08; P=.02), TCM syndrome differentiation and therapeutic regimen (mean 90.93, SD 2.42, vs mean 89.60, SD 2.86; P=.03), and communication skills (mean 90.67, SD 4.52, vs mean 88.24, SD 4.56; P=.02) than the control group. There was no significant difference in medical writing between both groups (mean 75.07, SD 3.61, vs mean 75.71, SD 2.86; P=.37). The postcourse feedback questionnaire indicated that VSP-TCM can better enhance students' TCM thinking ability (n=39, 93%, vs n=37, 88%; P=.002), medical history collection (n=38, 90%, vs n=30, 72; P=.001), syndrome differentiation and treatment and critical thinking (n=38, 90%, vs n=37, 88%; P=.046), comprehensive clinical application ability (n=40, 95%, vs n=36, 86%; P=.009), interpersonal communication skills (n=36, 86%, vs n=28, 67%; P=.01), and autonomous learning ability (n=37, 88%, vs n=28, 67%; P=.01) than traditional academic training. Similarly, the teachers held a positive perspective on VSP-TCM.

CONCLUSIONS

VSP-TCM enhances students' TCM clinical competence and dialectical thinking and improves their ability to work autonomously. Moreover, the VSP-TCM system is feasible, practical, and cost-effective and thus merits further promotion in TCM education.

Virtual Worlds Technology to Enhance Training for Primary Care Providers in Assessment and Management of Posttraumatic Stress Disorder Using Motivational Interviewing: Pilot Randomized Controlled Trial

BACKGROUND

Many individuals with posttraumatic stress disorder (PTSD) first present to primary care rather than specialty mental health care. Primary care providers often lack the training required to assess and treat patients with PTSD. Virtual trainings have emerged as a convenient and effective way of training primary care providers in PTSD assessment and communication methods (ie, motivational interviewing [MI]).

OBJECTIVE

The aim of this study was to conduct a pilot randomized controlled trial of a synchronous Virtual Worlds (VW; a virtual world where learners were immersed as avatars) training versus an asynchronous web-based training on PTSD and MI, comparing the feasibility, acceptability, usability, and preliminary efficacy of 2 different training platforms among primary care providers.

METHODS

Participating primary care providers were randomized to a VW and a web-based PTSD training. Outcomes were collected at baseline, posttraining, and 90-days follow-up. Standardized patient interviews measured participants' communication skills in assessing and managing patients with PTSD symptoms.

RESULTS

Compared to the web-based training, the VW training platform achieved larger learning gains in MI (ie, partnership and empathy) and in discussing pharmacotherapy and psychotherapy for PTSD. Both VW and web-based trainings led to increases in PTSD knowledge and primary care providers' self-confidence.

CONCLUSIONS

The asynchronous web-based PTSD training improved PTSD-related knowledge and self-confidence but was not as effective as the VW immersive experience in teaching MI or clinical management. Because VW training is synchronous and new for many learners, it required more time, facilitation, and technical support. As computer technology improves, VW educational interventions may become more feasible, particularly in teaching clinical skills.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03898271; https://tinyurl.com/mu479es5.

Exploring the Study of Simulation as a Continuing Professional Development Strategy for Physicians

INTRODUCTION

Practicing physicians have the responsibility to engage in lifelong learning. Although simulation is an effective experiential educational strategy, physicians seldom select it for continuing professional development (CPD) for reasons that are poorly understood. The objective of this study was to explore existing evidence on simulation-based CPD and the factors influencing physicians' engagement in simulation-based CPD.

METHODS

A scoping review of the literature on simulation-based CPD included MEDLINE, Embase, and CINAHL databases. Studies involving the use of simulation for practicing physicians' CPD were included. Information related to motivations for participating in simulation-based CPD, study objectives, research question(s), rationale(s), reasons for using simulation, and simulation features was abstracted.

RESULTS

The search yielded 8609 articles, with 6906 articles undergoing title and abstract screening after duplicate removal. Six hundred sixty-one articles underwent full-text screening. Two hundred twenty-five studies (1993-2021) were reviewed for data abstraction. Only four studies explored physicians' motivations directly, while 31 studies described incentives or strategies used to enroll physicians in studies on simulation-based CPD. Most studies focused on leveraging or demonstrating the utility of simulation for CPD. Limited evidence suggests that psychological safety, direct relevance to clinical practice, and familiarity with simulation may promote future engagement.

DISCUSSION

Although simulation is an effective experiential educational method, factors explaining its uptake by physicians as a CPD strategy are unclear. Additional evidence of simulation effectiveness may fail to convince physicians to participate in simulation-based CPD unless personal, social, educational, or contextual factors that shape physicians' motivations and choices to engage in simulation-based CPD are explored.

Technology and Mental Health: State of the Art for Assessment and Treatment

Technology is ubiquitous in society and is now being extensively used in mental health applications. Both assessment and treatment strategies are being developed and deployed at a rapid pace. The authors review the current domains of technology utilization, describe standards for quality evaluation, and forecast future developments. This review examines technology-based assessments of cognition, emotion, functional capacity and everyday functioning, virtual reality approaches to assessment and treatment, ecological momentary assessment, passive measurement strategies including geolocation, movement, and physiological parameters, and technology-based cognitive and functional skills training. There are many technology-based approaches that are evidence based and are supported through the results of systematic reviews and meta-analyses. Other strategies are less well supported by high-quality evidence at present, but there are evaluation standards that are well articulated at this time. There are some clear challenges in selection of applications for specific conditions, but in several areas, including cognitive training, randomized clinical trials are available to support these interventions. Some of these technology-based interventions have been approved by the U.S. Food and Drug administration, which has clear standards for which types of applications, and which claims about them, need to be reviewed by the agency and which are exempt.

Technology-Based Methods for Training Counseling Skills in Behavioral Health: a Scoping Review

In the present review, we consider technology-based methods for training and monitoring counseling skills in behavioral health (i.e., addictions, mental health, and behavioral medicine). We provide an overview of topical foci and design features, as well as review the available research. The Arksey and O’Malley framework for scoping review was used and there were two project phases. First, we reviewed and charted design features and training topics. Second, we reviewed and charted published research evaluating training outcomes. The search process yielded six commercial companies or academic research centers targeting online training of behavioral health counseling skills. These programs could be categorized by an avatar (i.e., computer-generated) or video (i.e., human actor) client interface, as well as by a completely interactive experience (i.e., virtual reality) or an experience with a pre-programmed, branch-logic interaction (i.e., computer simulation). One final company provided monitoring services only, without an explicit training component. The literature in this area is in its nascent stages, with primarily pilot scope and comparatively less progress if contrasted with fields such as general medicine. Online training and monitoring of behavioral health counseling skills is a promising emerging field with positive qualities such as scalability, resource efficiency, and standardization. Future research should emphasize (1) between-group randomized clinical trials, (2) comparisons to standard training practices, and (3) alignment with professional competency standards.

Research Hotspots and Trend Exploration on the Clinical Translational Outcome of Simulation-Based Medical Education: A 10-Year Scientific Bibliometric Analysis From 2011 to 2021

BACKGROUND

In recent decades, an increasing number of studies have focused on the clinical translational effect of simulation-based medical education (SBME). However, few scientific bibliometric studies have analyzed the research hotspots and publication trends. This study aimed to investigate research hotspots and future direction in the clinical translational outcome of SBME via bibliometrics.

METHOD

Relevant publications on the clinical translational outcomes of SBME from 2011 to 2021 were identified and retrieved from the Web of Science Core Collection (WOSCC). Software including VOSviewer (1.6.17) and CiteSpace (5.8R3) and a platform (bibliometric.com) were employed to conduct bibliographic and visualized analysis on the literature.

RESULTS

A total of 1,178 publications were enrolled. An increasing number of publications were observed in the past decades from 48 in 2011 to 175 in 2021. The United States accounted for the largest number of publications (488, 41.4%) and citations (10,432); the University of Toronto and Northwestern University were the leading institutions. Academic Medicine was the most productive journal concerning this field. McGaghie W C and Konge L were the most influential authors in this area. The hot topic of the translational outcome of SBME was divided into 3 stages, laboratory phase, individual skill improvement, and patient outcome involving both technical skills and non-technical skills. Translational research of comprehensive impact and collateral outcomes could be obtained in the future.

CONCLUSION

From the overall trend of 10 years of research, we can see that the research is roughly divided into three phases, from laboratory stage, individual skill improvement to the patient outcomes, and comprehensive impacts such as skill retention and collateral effect as cost-effectiveness is a major trend of future research. More objective evaluation measurement should be designed to assess the diverse impact and further meta-analysis and randomized controlled trials are needed to provide more clinical evidence of SBME as translational science.

Virtual, Augmented, and Alternate Reality in Medical Education: Socially Distanced but Fully Immersed

BACKGROUND

Advancements in technology continue to transform the landscape of medical education. The need for technology-enhanced distance learning has been further accelerated by the coronavirus disease (COVID-19) pandemic. The relatively recent emergence of virtual reality (VR), augmented reality (AR), and alternate reality has expanded the possible applications of simulation-based education (SBE) outside of the traditional simulation laboratory, making SBE accessible asynchronously and in geographically diverse locations.

OBJECTIVE

In this review, we will explore the evidence base for use of emerging technologies in SBE as well as the strengths and limitations of each modality in a variety of settings.

METHODS

PubMed was searched for peer-reviewed articles published between 1995 and 2021 that focused on VR in medical education. The search terms included medical education, VR, simulation, AR, and alternate reality. We also searched reference lists from selected articles to identify additional relevant studies.

RESULTS

VR simulations have been used successfully in resuscitation, communication, and bronchoscopy training. In contrast, AR has demonstrated utility in teaching anatomical correlates with the use of diagnostic imaging, such as point-of-care ultrasound. Alternate reality has been used as a tool for developing clinical reasoning skills, longitudinal patient panel management, and crisis resource management via multiplayer platforms.

CONCLUSION

Although each of these modalities has a variety of educational applications in health profession education, there are benefits and limitations to each that are important to recognize prior to the design and implementation of educational content, including differences in equipment requirements, cost, and scalability.

Adapting to the challenges of the global pandemic on genetic counselor education: Evaluating students' satisfaction with virtual clinical experiences

Travel restrictions, physical distancing, and limits to clinical placements due to the global pandemic raised enormous challenges for genetic counseling education in 2020. In response, we created authentic virtual clinical experiences in our Master of Genetic Counseling program, mimicking clinical practice: virtual simulation with standardized clients, and virtual clinical placements, including intake calls, triage, consultations, teamwork and time management, and genetic counseling with standardized clients. The virtual clinical experiences involved online pre-brief, simulation, and debrief. We aimed to evaluate students' satisfaction with this learning method. Between April and November 2020, we distributed an anonymous online survey to all participating students using a modified version of a validated satisfaction with simulation scale. We analyzed the combined responses from first- and second-year virtual clinical experiences using descriptive statistics and content analysis. The total number of possible responses was 120. The mean response rate was 68.36% (n = 82.03), with a mean of 16.41 participants responding to each survey from each year group. Of the first-year participants, 53% (n = 10) had not observed a genetic counseling consultation before attending the virtual clinical placement. Overall, 92.5% of responses indicated that students were satisfied with the virtual clinical experiences (SD = 0.05). 100% (n = 82) of responses indicated that working with standardized clients was beneficial to learning, encouraged reflection on clinical ability and was a valuable learning experience overall. However, 37.78% (n = 17) of those who participated in the virtual simulation found that the use of Zoom detracted from their clinical learning. The virtual clinical experiences increased first-year students' confidence about clinical placement and prepared second-year students for telehealth. In conclusion, the adaptation to virtual clinical experiences enhanced learning for most students, prepared them for practice, met the requirements of the accreditation body and enabled all of our final year students to graduate on time.

Virtual Standardized Patients for Mental Health Education

PURPOSE OF REVIEW

The training of psychiatrists and other mental health professionals requires education on a range of interpersonal, communication, and psychotherapy techniques. Classroom and workshop training must be augmented by experiential learning with feedback for skill implementation with fidelity. Virtual standardized patients (VSPs) are computerized conversational agents that can support experiential learning through standardized, consequence-free training environments at reduced costs.

RECENT FINDINGS

Research on mental health VSPs is rife with feasibility and acceptability pilot studies across various training populations and settings. Users have generally reported positive reactions to training with VSPs, though frustrations with some VSP speech recognition or VSP response relevance has been reported. Several studies have demonstrated a promising transfer of clinical skills from VSP training to human standardized patients and randomized trials supporting improved skill relative to reading or academic study are encouraging. As technology improves and natural language processing and accurate computer response generation for broad ranging conversational topics emerges, the field would benefit from research on the characteristics of effective VSPs for a range of purposes and trainee populations. Well-designed randomized evaluations of VSPs relative to best practices in education are needed, particularly regarding the impact of VSPs on clinical practice among actual patients.