Can we rely on simulated patients' satisfaction with their consultation for assessing medical students' communication skills? A cross-sectional study

Are evaluations in simulated medical encounters reliable among rater types? A comparison between standardized patient and outside observer ratings of OSCEs

Objective

By analyzing Objective Structured Clinical Examination (OSCE) evaluations of first-year interns' communication with standardized patients (SP), our study aimed to examine the differences between ratings of SPs and a set of outside observers with training in healthcare communication.

Methods

Immediately following completion of OSCEs, SPs evaluated interns' communication skills using 30 items. Later, two observers independently coded video recordings using the same items. We conducted two-tailed t-tests to examine differences between SP and observers' ratings.

Results

Rater scores differed significantly on 21 items (p < .05), with 20 of the 21 differences due to higher SP in-person evaluation scores. Items most divergent between SPs and observers included items related to empathic communication and nonverbal communication.

Conclusion

Differences between SP and observer ratings should be further investigated to determine if additional rater training is needed or if a revised evaluation measure is needed. Educators may benefit from adjusting evaluation criteria to decrease the number of items raters must complete and may do so by encompassing more global questions regarding various criteria. Furthermore, evaluation measures may be strengthened by undergoing reliability and validity testing.

Innovation

This study highlights the strengths and limitations to rater types (observers or SPs), as well as evaluation methods (recorded or in-person).

Effect of Experiential Communication Skills Education on Graduate Medical Education Trainees' Communication Behaviors: A Systematic Review

PURPOSE

A better understanding of how communication skills education impacts trainees' communication skills is important for continual improvement in graduate medical education (GME). Guided by the Kirkpatrick Model, this review focused on studies that measured communication skills in either simulated or clinical settings. The aim of this systematic review was to examine the effect of experiential communication skills education on GME trainees' communication behaviors.

METHOD

Five databases were searched for studies published between 2001 and 2021 using terms representing the concepts of medical trainees, communication, training, and skills and/or behaviors. Included studies had an intervention design, focused only on GME trainees as learners, used experiential methods, and had an outcome measure of communication skills behavior that was assessed by a simulated or standardized patient (SP), patient, family member, or outside observer. Studies were examined for differences in outcomes based on study design; simulated versus clinical evaluation setting; outside observer versus SP, patient, or family member evaluator; and length of training.

RESULTS

Seventy-seven studies were ultimately included. Overall, 54 (70%) studies reported some positive findings (i.e., change in behavior). There were 44 (57%) single-group pre-post studies, 13 (17%) nonrandomized control studies, and 20 (26%) randomized control studies. Positive findings were frequent in single-group designs (80%) and were likely in nonrandomized (62%) and randomized (55%) control trials. Positive findings were likely in studies evaluating communication behavior in simulated (67%) and clinical (78%) settings as well as in studies with outside observer (63%) and SP, patient, and family member (64%) evaluators.

CONCLUSIONS

This review demonstrates strong support that experiential communication skills education can impact GME trainees' communication behaviors. Marked heterogeneity in communication trainings and evaluation measures, even among subgroups, did not allow for meta-analysis or comparative efficacy evaluation of different studies. Future studies would benefit from homogeneity in curricular and evaluation measures.

Interprofessional communication in medical simulation: findings from a scoping review and implications for academic medicine

BACKGROUND

Interprofessional communication is fundamental to the delivery of healthcare and can be taught in medical school and other health professional schools through interprofessional education (IPE) activities. Simulation centers have become a predominant location for simulation IPE activities with infrastructure able to support high fidelity activities in a controlled environment. In this secondary analysis of a scoping review conducted on simulation-based IPE, we describe the characteristics of previously reported simulation IPE activities involving undergraduate medical students in a simulation center focused on interprofessional communication.

METHODS

Electronic searches of PubMed, CINAHL, and ERIC databases in accordance with PRISMA-ScR guidelines were conducted to isolate relevant articles from 2016-2020. In total, 165 peer-reviewed articles met inclusion criteria and data extraction linked to four research questions was applied by one individual and the accuracy was confirmed by a second individual. A secondary analysis was performed to describe what existing approaches for simulation IPE in simulation center settings have been used to explicitly achieve interprofessional communication competencies in undergraduate medical education. A sub-dataset was developed from the original scoping review and identified 21 studies describing simulation IPE activities that took place in dedicated simulation centers, targeted the IPEC interprofessional communication domain, and involved undergraduate medical students.

RESULTS

Though diverse, the majority of simulation IPE activities described high-fidelity approaches involving standardized patients and utilized assessment tools with established validity evidence in IPE activities to measure learning outcomes. A minority of simulation IPE activities were described as hybrid and utilized more than one resource or equipment for the activity and only two were longitudinal in nature. Learning outcomes were focused predominantly on modification of attitudes/perceptions and few targeted higher levels of assessment.

CONCLUSIONS

Educators charged with developing simulation IPE activities for medical students focused on interprofessional communication should incorporate assessment tools that have validity evidence from similar activities, target higher level learning outcomes, and leverage hybrid models to develop longitudinal simulation IPE activities. Though an ideal environment to achieve higher level learning outcomes, simulation centers are not required for meaningful simulation IPE activities.

Comparing perceptions of student abilities in diabetes self-management education and support among students, faculty and patients

BACKGROUND AND PURPOSE

Diabetes self-management education and support (DSMES) equips patients with diabetes with the knowledge needed for appropriate management. The purpose of this study was to compare perceptions of student confidence/aptitude held by students, pharmacy faculty preceptors, and patients regarding student teaching of a DSMES class.

EDUCATIONAL ACTIVITY AND SETTING

The study was a prospective assessment of fourth-year ambulatory care advanced pharmacy practice experience students. Students taught a single DSMES class and evaluated their confidence using a 14-item survey before and after the class. Patients who participated in the class and a pharmacy faculty observer completed the same instrument. Responses were compared using Friedman and Wilcoxon signed-rank tests, as appropriate.

FINDINGS

Twenty-six students completed the survey. Overall, students' self-perceived confidence scores significantly increased for all questions after teaching the DSMES class (P < .001 for all questions). Confidence scores among students and faculty preceptors were similar with no significant differences in perceived confidence. There were some significant differences found among student and patient scores, with patients assessing student's perceived self-confidence higher than the students on three items (P < .05).

SUMMARY

Student pharmacists' perceptions of their own confidence and abilities improved from before to after teaching a DSMES class. Student and faculty preceptor confidence scores were similar. There were few differences between student and patient confidence scores, with patients rating students highly on their perceived confidence in teaching a DSMES class. Patient feedback is important to consider when evaluating student confidence and abilities.

Comparison of self and simulated patient assessments of first-year medical students' Interpersonal and Communication Skills (ICS) during Objective Structured Clinical Examinations (OSCE)

BACKGROUND

Interpersonal and communication skills (ICS) are important core competencies in medical education and certification. In this study, we identified self- and simulated patient (SP)-reported ratings of US first-year medical students' ICS and the influence of age and gender on performance appraisal during the Objective-Structured Clinical Examination (OSCE).

METHODS

OSCE participants, including 172 first-year medical students and 15 SPs were asked to evaluate the students' ICS using the American Board of Internal Medicine-Patient-Satisfaction Questionnaire (ABIM-PSQ), electronically and via paper, respectively. Self- and SP-reported ratings of students' ICS were presented as the median on a 5-point Likert-scale and as three categories defined as "good," "very good," and "inadequate."

RESULTS

SPs assessed all 172 students in the OSCE, while 43.6% of students assessed their own performance. The majority of students and SPs evaluated the students' ICS as very good. 23.3% of SPs and 5.3% of students rated the medical students' ability to encourage patient question-asking and answer questions as inadequate (P <  0.002). Neither age nor gender influenced the medical students' self-assessment of ICS. Female SPs assigned lower scores to students in regard to respecting patients and encouraging patient question-asking and answering. Older SPs was more likely to assign lower scores on all survey questions.

CONCLUSIONS

In the present study, self- and SP-reported ratings of first-year medical students' ICS were mainly "very good" with no influence of students' age or gender. Older age and female gender among the SPs were associated with a reduction in SP-reported ratings of students' ICS.

Social ties between team members affect patient satisfaction: a data-driven approach to handling complex network analyses

Research from outside the medical field suggests that social ties between team-members influence knowledge sharing, improve coordination, and facilitate task completion. However, the relative importance of social ties among team-members for patient satisfaction remains unknown. In this study, we explored the association between social ties within emergency teams performing simulated caesarean sections (CS) and patient-actor satisfaction. Two hundred seventy-two participants were allocated to 33 teams performing two emergency CSs in a simulated setting. We collected data on social ties between team-members, measured as affective, personal and professional ties. Ties were rated on 5-point Likert scales. In addition, participants' clinical experience, demographic data and their knowledge about team members' roles were surveyed. Perceived patient satisfaction was measured on a 5-point Likert scale. Data was analysed with a linear regression model using elastic net regularization. In total, 109 predictor variables were analysed including 84 related to social ties and 25 related to clinical experience, demographics and knowledge test scores. Of the 84 variables reflecting social ties, 34 (41%) had significant association with patient satisfaction, p < 0.01. By contrast, a significant association with patient satisfaction was found for only one (4%) of the 25 variables reflecting clinical experience, demographics and knowledge of team roles. Affective ties and personal ties were found to be far more important predictors in the statistical model than professional ties and predictors relating to clinical experience. Social ties between emergency team members may be important predictors of patient satisfaction. The results from this study help to enhance our conceptual understanding of social ties and their implications for team-dynamics. Our study challenges existing views of team-performance by placing emphasis on achieving collective competence through affective and personal social ties, rather than focusing on traditional measures of expertise.

Impact of simulated patients on students' self-assessment of competency in practice of geriatric dentistry

PURPOSE

The evaluation of the effectiveness of simulation training in dentistry has previously been focused primarily on psychomotor hand skills. This study explored the impact of simulated patient (SP) encounters in simulation on students' self-assessment of competency in their clinical and communication abilities with geriatric patients.

METHODS

Students from 2 cohorts were recruited for this study. Cohort 1 (n = 30) participated in the standard curriculum with no simulation training and served as the control group. Cohort 2 (n = 34) participated in a SP experience, simulating the initial stages of a care visit for 2 nursing home patients. Students' perceptions of competency to perform these clinical and communication tasks were assessed. A group debriefing session was held 5 weeks post-simulation where Cohort 2 completed a student feedback form.

RESULTS

A statistically significant change (P < 0.00001) was noted for both cohorts in their self-reported competence to perform clinical tasks following exposure to an independent clinical experience. In addition to this gain, individuals in Cohort 2 demonstrated improvements following simulation and expressed different responses of impact to questions related to treatment, pharmacology, and managing a complex medical history.

CONCLUSIONS

This study suggests that simulation of patient interactions using SPs can strengthen students' self-assessment of competency in their abilities, leading to more genuine interactions with actual patients. These findings will help inform the design of future SP encounters as a component of an evolving humanistic curriculum.

Have medical students' attitudes towards clinical communication skills changed over a 12- year period? A comparative long-term study

BACKGROUND

Attitudes towards learning clinical communication skills at the end of medical school are likely to reflect the students' training and motivation for the continued development of their skills as doctors. Students from two Norwegian medical schools, one with a traditional, and the other with an integrated curriculum, were approached in 2003 and 2015; with regard to changes in students' attitudes towards acquiring communication skills in two diverse learning environments. This comparison might reveal the effects of the training programs from a long-term perspective, as neither of the medical schools made any major curriculum changes within the study period.

METHODS

The samples comprised final-year medical students. Two separate cross-sectional surveys performed 12 years apart (2003 and 2015) used items from the Communication Skills Attitude Scale in addition to age and gender. The traditional curriculum included only theoretical teaching and no contact with patients was made during the first 2 to 2.5 years of medical school. However, the integrated curriculum combined training in theoretical and clinical communication skills with early patient contact from the beginning.

RESULTS

Attitudes improved from the first to the second survey at both schools, however, students from the integrated school reported more positive attitudes than those from the traditional school. Female students from the integrated school contributed the most to the difference in attitudes in both surveys.

CONCLUSIONS

Students in both traditional and integrated curricula improved their attitudes from the first to the second assessment. However, compared with the traditional curriculum, the integrated one fostered even higher levels of positive attitudes towards acquiring communication skills, and a pronounced influence was observed on female students. These findings suggest that an educational program with greater emphasis on improving attitudes among male students may be required.

“It Hurts as If…”: Pain-Associated Language, Visual Characterization, and Storytelling in Hmong Adults

Objective

Pain is challenging to diagnose and manage in primary care, especially when patients have limited English proficiency (LEP). Little is known about whether LEP patients can provide pain information that is consistent with the process and the content that providers expect in a clinical interaction. We explore how LEP Hmong patients communicate their pain to providers in primary care settings.

Methods

A qualitative study with 67 Hmong participants (63% female and x̄ age = 53.7 years) were recruited from a Midwestern state. Semistructured interviews on pain communication were conducted, audio-recorded, transcribed, and analyzed using directed content analysis.

Results

The Hmong participants described pain using stories that generally had the same dimensions of information that providers require for pain assessment. These included references to time, causality, associated symptoms or related experiences, intensity, and consequences of pain. However, the participants expressed some pain dimensions in language that was not shared by providers: visual metaphors that were generally in reference to pain quality and fewer words for pain location, intensity, and some qualities. Participants used two strategies to decide whether they should tell their pain story: assessing the provider and determining whether their story was appreciated. The perception that providers underappreciated their stories resulted in dissatisfaction and undertreatment of pain. Ultimately, this resulted in having less frequent contact with providers or changing providers.

Conclusions

Findings demonstrate a discordance in the expected process and content of the clinical interaction between LEP Hmong patients and providers, suggesting the need for culturally appropriate pain assessments in this population.

Using conversation analysis to inform role play and simulated interaction in communications skills training for healthcare professionals: identifying avenues for further development through a scoping review

BACKGROUND

This paper responds to previously published debate in this journal around the use of sociolinguistic methods in communication skills training (CST), which has raised the significant question of how far consultations with simulated patients reflect real clinical encounters. This debate concluded with a suggestion that sociolinguistic methods offer an alternative analytic lens for evaluating CST. We demonstrate here that the utility of sociolinguistic methods in CST is not limited to critique, but also presents an important tool for development and delivery.

METHODS

Following a scoping review of the use of role play and simulated interaction in CST for healthcare professionals, we consider the use of the specific sociolinguistic approach of conversation analysis (CA), which has been applied to the study of health communication in a wide range of settings, as well as to the development of training.

DISCUSSION

Role play and simulated interaction have been criticised by both clinicians and sociolinguists for a lack of authenticity as compared to real life interactions. However they contain a number of aspects which healthcare professionals report finding particularly useful: the need to think on one's feet in real time, as in actual interaction with patients; the ability to receive feedback on the simulation; and the ability to watch and reflect on how others approach the same simulation task in real time. Since sociolinguistic approaches can help to identify inauthenticity in role play and simulation, they can also be used to improve authenticity. Analysis of real-life interactions using sociolinguistic methods, and CA in particular, can identify actual interactional practices that are used by particular patient groups. These practices can then be used to inform the training of actors simulating patients. In addition, the emphasis of CA on talk as joint activity means that proper account can be taken of the way in which simulated interaction is co-constructed between simulator and trainee. We suggest that as well as identifying potential weaknesses in current role play and simulation practice, conversation analysis offers the potential to enhance and develop the authenticity of these training methods.

Emotion recognition and extraversion of medical students interact to predict their empathic communication perceived by simulated patients

BACKGROUND

This study assessed the impact of medical students' emotion recognition ability and extraversion on their empathic communication, as perceived by simulated patients in a training context.

METHODS

This study used a crossed-effect data structure and examined 245 students in their fourth year of medical school. The students' personality traits were assessed based on a self-assessment questionnaire of the short form of the Big Five Inventory; their emotion recognition ability was measured using a performance test (Diagnostic Analysis of Nonverbal Accuracy-2, Adult Facial Expressions). Simulated patients evaluated the medical students' empathic communication.

RESULTS

Students with a combination of high emotion recognition ability and extraversion received more positive ratings from simulated patients than their fellow students with a combination of emotion recognition ability and low extraversion. The main effects of emotion recognition or extraversion were not sufficient to yield similar effects. There were no other effects related to the remaining Big Five variables.

CONCLUSIONS

The results support the hypothesis that to build rapport with patients, medical staff need to combine emotional capabilities with a dispositional interest in interpersonal encounters.

Tracking Veterinary Students' Acquisition of Communication Skills and Clinical Communication Confidence by Comparing Student Performance in the First and Twenty-Seventh Standardized Client Encounters

The American Veterinary Medical Association's Council on Education mandates that veterinary students graduate with competence in clinical communication. Communication competence facilitates the successful transition of students into clinical practice by improving veterinary client satisfaction and reducing the risk of litigation. Curricular experimentation with communications training has led to innovative approaches to this content area, including the adoption of standardized patients from human health care education. Although the use of standardized clients (SCs) is time and resource intensive, their use by veterinary educators has allowed students to work through authentic cases in a safe learning environment in which learner goals drive training. Despite the increasing popularity of SC-based communications curricula, outcomes assessment studies that track the resultant student growth are lacking. Ninety students in the inaugural class of 2018 at Midwestern University College of Veterinary Medicine participated in 27 SC encounters over eight consecutive quarters. Students' use of Calgary-Cambridge Guide communication skills was compared in the first and twenty-seventh encounters. On the basis of SC evaluations of student performance, students increased their use of all communication skills in the final encounter compared with baseline. Students were also more likely to take a comprehensive patient history, build rapport, and retain the client. Mean scores for pre- and post-encounter communication confidence, as self-reported by students, also increased. These findings support that a SC-rich curriculum facilitates student acquisition of communication skills and promotes confidence when students approach entry-level clinical tasks such as history taking.

Using root metaphors to analyze communication between nurses and patients: a qualitative study

BACKGROUND

Metaphors in communication can serve to convey individuals' backgrounds, contexts, experiences, and worldviews. Metaphors used in a health care setting can help achieve consensual communication in professional-patient relationships. Patients use metaphors to describe symptoms, or how disease affects them. Health professionals draw on shared understanding of such metaphors to better comprehend and meet patient needs, and to communicate information that patients can more easily integrate into their lives. This study incorporated a theoretical framework based on four worldviews, each with an underlying foundational metaphor (root metaphor). The use of these root metaphors (formism, mechanism, contextualism, and organicism) can have an explanatory function and serve to impart new meanings, as each type of metaphor can lead to a particular interpretation. The study aimed to extract and discuss the root metaphors, with a view to analyzing the communication between health professionals and patients.

METHODS

In a case study in Spain over a six-month period, we analyzed the content of recorded, transcribed interviews conducted by one nurse with 32 patients who had chronic illnesses. We inductively extracted five categories that emerged from the interviews: blood sugar, cholesterol, exercise, blood pressure, and diet. We then examined these categories from the standpoint of each of the four root metaphors using two approaches: A series (deductive) and an emergent (inductive) approach.

RESULTS

The results show that the nurse tended to primarily use two worldviews: mechanism and formism. In contrast, patients tended to favor mechanism when discussing cholesterol, blood pressure, and blood sugar levels, whereas contextualism was predominant when the category was diet or exercise.

CONCLUSIONS

This study adds to the existing literature on health professionals and patients' communication. It shows how the use of Pepper's root metaphors help to analyze the communication between the nurse and patients. Furthermore, it shows they are both using different root metaphors when they are talking about illness and treatments especially regarding blood sugar, cholesterol, exercise, blood pressure, and diet. Further qualitative and quantitative studies are needed to solidly these findings.

Incorporating patient partner scores into high stakes assessment: an observational study into opinions and attitudes

BACKGROUND

Volunteer patients (also known as patient partners (PPs)) play a vital role in undergraduate healthcare curricula. They frequently take part in objective structured clinical examinations (OSCE) and rate aspects of students' performance. However, the inclusion and weighting of PP marks varies, while attitudes and opinions regarding how (and if) they should contribute towards the pass/fail outcome are uncertain.

METHODS

A prospective observational study was conducted to explore beliefs of PPs regarding inclusion of their scores in a high stakes undergraduate OSCE in a single UK medical school. All PPs delivering components of the local MBChB curriculum were asked to participate in the questionnaire study. Quantitative and qualitative data were analysed using descriptive statistics and framework analysis respectively.

RESULTS

Fifty out of 160 (31% response rate) PPs completed the questionnaire; 70% had participated in a final year OSCE. Thirty (60%) felt their marks should be incorporated into a student's overall score, while 28% were uncertain. The main reasons for inclusion were recognition of the patient perspective (31%) and their ability to assess attitudes and professionalism (27%), while reasons against inclusion included lack of PP qualification/training (18%) and concerns relating to consistency (14%). The majority of PPs were uncertain what proportion of the total mark they should contribute, although many felt that 5-10% of the total score was reasonable. Most respondents (70%) felt that globally low PP scores should not result in an automatic fail and many (62%) acknowledged that prior to mark inclusion, further training was required.

CONCLUSION

These data show that most respondents considered it reasonable to "formalise their expertise" by contributing marks in the overall assessment of students in a high stakes OSCE, although what proportion they believe this should represent was variable. Some expressed concerns that using marks towards progress decisions may alter PP response patterns. It would therefore seem reasonable to compare outcomes (i.e. pass/fail status) using historical data both incorporating and not incorporating PP marks to evaluate the effects of doing so. Further attention to existing PP training programmes is also required in order to provide clear instruction on how to globally rate students to ensure validity and consistency.

Communication skills of medical students during the OSCE: Gender-specific differences in a longitudinal trend study

BACKGROUND

Communication skills are essential in a patient-centred health service and therefore in medical teaching. Although significant differences in communication behaviour of male and female students are known, gender differences in the performance of students are still under-reported. The aim of this study was to analyse gender differences in communication skills of medical students in the context of an OSCE exam (OSCE = Objective Structured Clinical Examination).

METHODS

In a longitudinal trend study based on seven semester-cohorts, it was analysed if there are gender differences in medical students' communication skills. The students (self-perception) and standardized patients (SP) (external perception) were asked to rate the communication skills using uniform questionnaires. Statistical analysis was performed by using frequency analyses and t-tests in SPSS 21.

RESULTS

Across all ratings in the self- and the external perception, there was a significant gender difference in favour of female students performing better in the dimensions of empathy, structure, verbal expression and non-verbal expression. The results of male students deteriorated across all dimensions in the external perception between 2011 and 2014.

DISCUSSION & CONCLUSION

It is important to consider if gender-specific teaching should be developed, considering the reported differences between female and male students.