Simulation Training in Psychiatry for Medical Education: A Review

Educational Utility of Clinical Vignettes Generated in Japanese by ChatGPT-4: Mixed Methods Study

BACKGROUND

Evaluating the accuracy and educational utility of artificial intelligence-generated medical cases, especially those produced by large language models such as ChatGPT-4 (developed by OpenAI), is crucial yet underexplored.

OBJECTIVE

This study aimed to assess the educational utility of ChatGPT-4-generated clinical vignettes and their applicability in educational settings.

METHODS

Using a convergent mixed methods design, a web-based survey was conducted from January 8 to 28, 2024, to evaluate 18 medical cases generated by ChatGPT-4 in Japanese. In the survey, 6 main question items were used to evaluate the quality of the generated clinical vignettes and their educational utility, which are information quality, information accuracy, educational usefulness, clinical match, terminology accuracy (TA), and diagnosis difficulty. Feedback was solicited from physicians specializing in general internal medicine or general medicine and experienced in medical education. Chi-square and Mann-Whitney U tests were performed to identify differences among cases, and linear regression was used to examine trends associated with physicians' experience. Thematic analysis of qualitative feedback was performed to identify areas for improvement and confirm the educational utility of the cases.

RESULTS

Of the 73 invited participants, 71 (97%) responded. The respondents, primarily male (64/71, 90%), spanned a broad range of practice years (from 1976 to 2017) and represented diverse hospital sizes throughout Japan. The majority deemed the information quality (mean 0.77, 95% CI 0.75-0.79) and information accuracy (mean 0.68, 95% CI 0.65-0.71) to be satisfactory, with these responses being based on binary data. The average scores assigned were 3.55 (95% CI 3.49-3.60) for educational usefulness, 3.70 (95% CI 3.65-3.75) for clinical match, 3.49 (95% CI 3.44-3.55) for TA, and 2.34 (95% CI 2.28-2.40) for diagnosis difficulty, based on a 5-point Likert scale. Statistical analysis showed significant variability in content quality and relevance across the cases (P<.001 after Bonferroni correction). Participants suggested improvements in generating physical findings, using natural language, and enhancing medical TA. The thematic analysis highlighted the need for clearer documentation, clinical information consistency, content relevance, and patient-centered case presentations.

CONCLUSIONS

ChatGPT-4-generated medical cases written in Japanese possess considerable potential as resources in medical education, with recognized adequacy in quality and accuracy. Nevertheless, there is a notable need for enhancements in the precision and realism of case details. This study emphasizes ChatGPT-4's value as an adjunctive educational tool in the medical field, requiring expert oversight for optimal application.

Short-term effects of a simulation-based training program on suicide risk assessment and intervention for first-year psychiatry residents

OBJECTIVES

Adequate training of mental health professionals in suicide risk assessment and intervention is crucial. Simulation-based education is a relevant method to acquire competences for challenging health interventions. Here, we aimed to evaluate the effects of a simulation-based training program on first-year psychiatry residents' skills, knowledge, attitudes and satisfaction.

METHODS

We conducted pre- and immediately post-training assessments in 153 psychiatry residents during their first or second postgraduate semester in Paris, France, in 2020 and 2021. The simulation-based training occurred a few weeks after a two-hour theoretical lecture on suicidal behaviors. It consisted of a full-day training of small groups (n=5 to 9) with six scenarios played by professional actors and facilitated by two trained psychiatrists, and a 45-60minute debriefing session after each simulation. Educational objectives focused on basic aspects of patient-psychiatrist relation, the investigation of suicidal ideation, and basic interventions in various contexts (outpatient consultation, emergency room, telephone) and patient profiles. Outcome measures included the Suicide Intervention Response Inventory (SIRI-2), a short questionnaire on basic knowledge, a self-confidence four-item scale, and a satisfaction questionnaire.

RESULTS

There were significant improvements in skills, basic knowledge, and self-confidence from pre- to post-training. Trainees also expressed a high level of satisfaction, an appreciation for this innovative pedagogy and a wish for more similar training.

CONCLUSIONS

This practical training program based on simulation improved self-confidence, knowledge and skills in suicide risk assessment and intervention in the short-term among first year psychiatry residents. Results from this study are therefore promising. Longitudinal studies are, however, needed to evaluate the persistence of changes over time, positive changes in care in real settings and health benefits for patients. In a time of easily accessible and rapidly growing factual medical knowledge, of necessary need for skilled professionals and of growing demand in mental health care, the development of simulation training in psychiatry should be a pedagogical and a public health priority.

Embracing different languages and local differences: Co-constructive patient simulation strengthens host countries' clinical training in psychiatry

BACKGROUND

Global education in psychiatry is heavily influenced by knowledge from Western, high-income countries, which obscures local voices and expertise.

AIM

To adapt a human simulation model to psychiatric education in a context that is specific to local languages and cultures.

METHODS

We conducted an observational study consisting of six human simulation sessions with standardized patients from two host countries, speaking their native languages, and following an adaptation of the co-constructive patient simulation (CCPS) model. As local faculty became increasingly familiar with the CCPS approach, they took on the role of facilitators-in their country's native language.

RESULTS

Fifty-three learners participated: 19 child and adolescent psychiatry trainees and 3 faculty members in Türkiye (as a group that met online during 3 consecutive months); and 24 trainees and 7 faculty in Israel (divided into 3 groups, in parallel in-person sessions during a single training day). Each of the six cases reflected local realities and clinical challenges, and was associated with specific learning goals identified by each case-writing trainee.

CONCLUSION

Human simulation has not been fully incorporated into psychiatric education: The creation of immersive clinical experiences and the strengthening of reflective practice are two areas ripe for development. Our adaptations of CCPS can also strengthen local and regional networks and psychiatric communities of practice. Finally, the model can help question and press against hegemonies in psychiatric training that overshadow local expertise.

E-learning as a tool of suicide prevention training: A meta-analysis and systematic review

Suicide is a global health challenge. One prevention strategy is teaching individuals how to detect and respond to suicidality. These training have increasingly been delivered online. We searched WoS, Scopus, and PubMed from inception until the 20 September 2023 to evaluate e-learning efficacy as standardized mean changes and standardized mean differences. We synthesized main results using multilevel meta-analyses and subgroups using random-effects meta-analyses. Robins-I, RoB-II and trim-and-fill were used to assess the risk of bias. Of the 6516 initially screened articles, 26 were included. Overall, e-learning increased suicide prevention skills. Subgroups reported differing results: e-learning affected knowledge and self-efficacy more than behavior and attitudes. Efficacy, short duration, and low-cost suggest that e-learning may be feasible in teaching basic suicide prevention skills to lay people. However, current evidence suggests that health care professionals should not rely on e-learning as a training modality, except when no other form of training is available. Preregisteration: CRD42020218978.

Simulation training in suicide risk assessment and intervention: a systematic review and meta-analysis

PURPOSE

Suicide is a major cause of preventable death worldwide. Adequate training in risk assessment and intervention is key to suicide prevention. The use of simulation (role plays, simulated patients, virtual reality…) for practical training is a promising tool in mental health. The purpose of this study was to assess the effectiveness of simulation training in suicide risk assessment and intervention for healthcare professionals and gatekeepers.

METHODS

We conducted a systematic review in Medline and PsycINFO up to 31 July 2021 of randomized controlled trials (RCTs), non-randomized controlled trials, and pre/post-test studies. RCTs were furthermore included in a meta-analysis. We assessed the methodological quality of all studies with the Medical Education Research Study Quality Instrument, and the Cochrane Risk of Bias tool 2.0 for RCTs. Primary outcomes were changes in Kirkpatrick criteria: attitudes, skills, knowledge, behaviors, and patient outcomes.

RESULTS

We included 96 articles representing 43,656 participants. Most pre/post-test (n = 65) and non-randomized controlled (n = 14) studies showed significant improvement in attitudes, skills, knowledge, and behaviors. The meta-analysis of 11 RCTs showed positive changes in attitudes immediately after training and at 2-4 months post-training; in self-perceived skills at 6 months post-training; but not in factual knowledge. Studies assessing benefits for patients are still limited.

CONCLUSIONS

The heterogeneity of methodological designs, interventions, and trained populations combined with a limited number of RCTs and studies on patients' outcomes limit the strength of the evidence. However, preliminary findings suggest that simulation is promising for practical training in suicidal crisis intervention and should be further studied.

Promoting Social Participation and Recovery with Virtual Reality Based Interventions among people with Mental Health and Substance Use disorders: A Qualitative study (Preprint)

BACKGROUND

People with mental health disorders (MHDs) and substance use disorders (SUDs) are a highly vulnerable group, particularly affected by social exclusion, marginalization, and disconnectedness. Virtual reality technology holds a potential for simulating social environments and interactions to mitigate the social barriers and marginalization faced by people recovering from MHDs and SUDs. However, it is still unclear how we can harness the greater ecological validity of virtual reality-based interventions targeting social and functional impairments in individuals with MHDs and SUDs.

OBJECTIVE

The aim of this paper was to explore how service providers in community-based MHD and SUD health care services perceive the barriers to social participation among adults recovering from MHDs and SUDs to provide a broader understanding of how learning experiences can be modeled to promote social participation in virtual reality environments.

METHODS

Two semistructured, open-ended, and dual-moderator focus group interviews were conducted with participants representing different community-based MHD and SUD health care services. Service providers were recruited from their MHD and SUD services in our collaborating municipality in Eastern Norway. We recruited the first participant group at a municipal MHD and SUD assisted living facility for service users with ongoing excessive substance use and severe social dysfunctionality. We recruited the second participant group at a community-based follow-up care service aimed at clients with a broad range of MHDs and SUDs and various levels of social functioning. The qualitative data extracted in the interviews were analyzed, using reflexive thematic analysis.

RESULTS

The analysis of the service providers' perceptions of the barriers to social participation among clients with MHDs and SUDs revealed the following five main themes: challenging or lacking social connections, impaired cognitive functions, negative self-perception, impaired personal functioning, and insufficient social security. The barriers identified are interrelated in a cluster of cognitive, socioemotional, and functional impairments, leading to a severe and diverse complex of barriers to social participation.

CONCLUSIONS

Social participation relies on people's capability to use their present social opportunities. Promoting basic human functioning is key to promoting social participation among people with MHDs and SUDs. The findings in this study indicate a need to address cognitive functioning, socioemotional learning, instrumental skills, and complex social functions to meet the complexity and diversity of the identified barriers to social functioning in our target group. Virtual reality-based interventions for promoting social participation should be sequenced into distinct scenarios dedicated to specific learning goals to build complex learning in a step-by-step process based on successively more complex levels of human and social functioning.

Learning About and Destigmatizing Substance Use Disorders: a Video-Based Educational Module Using Simulated Patients

OBJECTIVE

The authors developed and tested video clips depicting three paradigmatic substance use disorder (SUD) clinical scenarios, each portrayed by a different simulated patient interacting with the same clinician.

METHODS

The authors embedded 21 short video clips (with a cumulative duration of 27 min) into a 2-h session on SUDs. The didactic was delivered online through synchronous videoconferencing with Zoom. The primary outcome compared learners' scores on the Attitudes and Confidence in the Treatment of Patients with Substance Use Disorders (ACT-SUDS) before and after participating in the didactic.

RESULTS

Fifty-eight second-year medical students participated and completed the survey prior to the didactic; 42 (72%) of them completed the survey immediately after. Compared to baseline, ACT-SUDS increased after the didactic: from 3.7 ± 0.5 to 4.0 ± 0.4 (mean difference = 0.4 [95% confidence interval = 0.2-0.5], paired-t = 5.75, p < 0.001), as did each of its four subscales: confidence, enjoyment, SUD as a medical disorder, and attitudes toward Alcoholics Anonymous (AA; t ≥ 3.0, p ≤ 0.005).

CONCLUSIONS

The video-based educational module proved easy to implement in the virtual classroom and led to measurable changes in perceptions and attitudes toward SUDs. The module is available to view or freely download and is amenable for adaptation by end-use instructors.

Experiences of French medical students during their clerkship in adolescent psychiatry: a qualitative study

The epidemiology of adolescent psychiatric disorders and the relational complexity of their management make exposure to adolescent psychiatry essential during medical school. However, some clinical particularities can complicate the students' learning experiences. Our work aimed to explore the experience of being a medical student during clerkships in adolescent psychiatry. Following the Interpretative Phenomenological Analysis qualitative approach, 20 semi-structured face-to-face interviews were conducted with medical students at the end of their clerkship in adolescent psychiatry. Three super-ordered themes emerged to describe their experience: in-depth self-exploration calling on emotions, thoughts and experiences; changes in the view of adolescent mental health; better understanding of the role and meaning of adolescent psychiatric care and how to approach it. Identification between students and patients could result from time-related factors (the end of adolescent brain remodeling, long, demanding studies, and financial and material dependence). In addition, the predominant use of non-analytical clinical reasoning processes-less valued in the rest of the graduate curriculum-poses a challenge for students. Indeed, for a student to find his or her place in adolescent psychiatry requires the student to reinvent him or herself, because the codes are different (no gown, less well-defined tasks, etc.). Finally, the excess prevalence of mental disorders among medical students requires increased vigilance on the part of tutors. For all these reasons, close, attentive tutoring seems essential to support students, while these clerkships afford a real opportunity for students to broaden their interpersonal skills.

Making It Real: From Telling to Showing, Sharing, and Doing in Psychiatric Education

OBJECTIVE

Innovations in contemporary medical education could inform remedies to address enduring challenges such as the marginalization and stigmatization of psychiatry, of mental illnesses, and of those affected by them.

METHODS

In blending the works of Bleakley, Bligh, and Brown (2011) and of Kumagai and Naidu (2015), we developed an overarching heuristic with practical relevance and concrete applications to psychiatric education.

RESULTS

We identify three areas to enhance psychiatric education embedded into this blended framework: 1) Showing, or the more accurate depiction or imaging of mental illnesses and of psychiatric practice, as exemplified by the incorporation into didactic content of asynchronous video-based clinical materials produced with specific educational objectives in mind; 2) Sharing, or addressing the image problem of mental illnesses, of those living with or affected by them, and of psychiatry as a profession, as exemplified by psychiatrists embracing their role as experts by professional and personal experience when sharing their own journeys with mental illness, treatment, and recovery; and 3) Doing, or reimagining reflective psychiatric practice, as exemplified by the novel methodology of co-constructive patient simulation (CCPS), through which learners can engage in reflective practice and supervision in a participatory and democratic setting that does not privilege participants' hierarchical standing.

CONCLUSION

The blended model and the sample applications we describe offer a range of teaching, learning, and professional development opportunities, should psychiatric educators choose to pursue them and reap their promise.