Comparison of self and simulated patient assessments of first-year medical students' Interpersonal and Communication Skills (ICS) during Objective Structured Clinical Examinations (OSCE)

Assessment of communication skills in health professions education; Ottawa 2024 consensus statement

Despite the increasing inclusion of communication skills in accreditation standards and an increase in time dedicated to teaching these skills, communication is often regarded as a separate skill and is therefore, not consistently represented in overall systems of assessment in Health Professions Education (HPE). The ascendence of competency-based medical education, programmatic assessment, artificial intelligence, and widespread use of telehealth, alongside changing patient expectations warrant an update in thinking about the assessment of communication skills in health professions education. This consensus statement draws on existing literature, expert pinion, and emerging challenges to situate the assessment of communication skills in the contemporary health professions education context. The statement builds on previous work to offer an update on the topic and include new developments related to assessment, particularly: the challenges and opportunities associated with systems of assessment; patient and peer perspectives in assessment; assessment of interprofessional communication, cross-cultural communication, digital communication; and assessment using digital technologies. Consensus was reached through extensive discussion among the authors and other experts in HPE, exploration of the literature, and discussion during an Ottawa 2024 conference workshop. The statement puts forward a summary of available evidence with suggestions for what educators and curriculum developers should consider in their planning and design of the assessment of communication.

Investigating the Use of Simulated Patient Ratings in Objective Structured Clinical Examinations to Enhance Undergraduate Medical Assessments in Riyadh, Saudi Arabia

BACKGROUND

Objective structured clinical examinations (OSCE) are the gold standard of clinical assessment, and are used to conduct undergraduate family medicine clinical assessment at King Faisal Specialist Hospital and Research Centre (KFSHRC). Some studies have suggested that simulated patient (SP) ratings could provide a better measure of empathy and communication skills than physician scores. The objective of this study is to further explore the effectiveness of simulated patient (SP) ratings in undergraduate OSCE assessments.

METHODS

The research employed a mixed-method approach. Three OSCE assessments for final-year students were selected. Both physicians and SPs evaluated each student, providing global ratings across four domains. The quantitative aspect involved comparing physician and SP scores and assessing correlation. The qualitative aspect involved interviewing SPs to establish what student behaviours led to higher or lower scores.

RESULTS

Moderate correlation was found between physician ratings and SP ratings (r=0.53, p<0.01). Internal consistency of the SP ratings was lower than physician scores. SPs considered themselves to be patient advocates and were keen to give formative feedback. The ability of the trainee to truly listen was a major concern. Scoring for SPs was relatively holistic in nature.

CONCLUSIONS

The results demonstrate that SP scores have slightly weaker reliability but are still relevant and offer a completely different perspective, enriching the assessment data. Assessment should take patient or SP perspectives into account, and not rely solely on the expert physician. Changing the assessment methods will lead to necessary changes in student approach to the OSCE and improve authenticity and validity.

Development of a validated assessment tool for medical students using simulated patients: an 8-year panel survey

BACKGROUND

The use of simulated patients (SPs) to assess medical students' clinical performance is gaining prominence, underscored by patient safety perspective. However, few reports have investigated the validity of such assessment. Here, we examined the validity and reliability of an assessment tool that serves as a standardized tool for SPs to assess medical students' medical interview.

METHODS

This longitudinal survey was conducted at Keio University School of Medicine in Japan from 2014 to 2021. To establish content validity, the simulated patient assessment tool (SPAT) was developed by several medical education specialists from 2008 to 2013. A cohort of 36 SPs assessed the performance of 831 medical students in clinical practice medical interview sessions from April 2014 to December 2021. The assessment's internal structure was analyzed using descriptive statistics (maximum, minimum, median, mean, and standard deviation) for the SPAT's 13 item total scores. Structural validity was examined with exploratory factor analysis, and internal consistency with Cronbach's alpha coefficients. The mean SPAT total scores across different SPs and scenarios were compared using one way analysis of variance (ANOVA). Convergent validity was determined by correlating SPAT with the post-clinical clerkship obstructive structured clinical examination (post-CC OSCE) total scores using Pearson's correlation coefficient.

RESULTS

Of the 831 assessment sheets, 36 with missing values were excluded, leaving 795 for analysis. Thirty-five SPs, excluding one SP who quit in 2014, completed 795 assessments, for a response rate of 95.6%. Exploratory factor analysis revealed two factors, communication and physician performance. The overall Cronbach's alpha coefficient was 0.929. Significant differences in SPAT total scores were observed across SPs and scenarios via one-way ANOVA. A moderate correlation (r =.212, p <.05) was found between SPAT and post-CC OSCE total scores, indicating convergent validity.

CONCLUSIONS

Evidence for the validity of SPAT was examined. These findings may be useful in the standardization of SP assessment of the scenario-based clinical performance of medical students.

A comparative analysis of student, educator, and simulated parent ratings of video-recorded medical student consultations in pediatrics

BACKGROUND

Simulation-based education (SBE) affords learners opportunities to develop communication skills, including those related to pediatrics. Feedback is an integral part of SBE, and while much research into feedback from multiple sources exists, the findings are mixed. The aim of this comparative study was to replicate some of this work in a novel area, pediatric medical education, to better understand how multisource feedback (self, educator, and simulated parent) may inform learning and curriculum design.

METHODS

During their pediatric rotation, medical students participated in a consultation with a simulated parent, engaged in video-assisted self-reflection, and received feedback from both an educator and the simulated parent through an e-learning platform. The Pediatric Consultation Skills Assessment Tool (PCAT) was used for self-assessment and educator feedback, and the Consultation and Relational Empathy (CARE) measure was used for simulated parent feedback.

RESULTS

Our results showed that high-performing students underrated their performance, and low-performing students overrated their performance. Feedback from multiple sources helps to identify both areas of weakness in student performance and areas of weakness in student self-appraisal. Overall, general areas of weakness identified for the learners related to making contingency plans and providing easy-to-understand explanations for simulated parents. Some simulated parent feedback did not align with educator and student ratings, highlighting the value of including the simulated parent perspective. Our findings question whether a third party can reliably judge the simulated parent's level of understanding.

CONCLUSION

Multisource feedback allows students to develop layered insights into their performance and supports self-appraisal. Aggregating feedback through an e-learning platform allows educators to gain greater insights into the strengths and weakness of students and design a more tailored teaching plan to support student needs.

Factors influencing older adults' satisfaction with caregivers' communication

Effective communication skills are crucial for caregivers to provide quality care and meet the unique needs of patients of all ages. However, older patients have specific communication requirements, and their satisfaction depends on several factors.

Objective

This study aimed to evaluate the level of satisfaction among older adults in Marrakech, Morocco, regarding the communication skills of their caregivers, and to identify the factors influencing this satisfaction.

Methods

This is a cross-sectional study conducted between March and July 2022 among 204 people aged 60 years and older who presented to the Mouhamed VI University Hospital of Marrakech, Morocco, for various care services. The older adults' satisfaction with caregivers' communication was assessed by the American Board of Internal Medicine (ABIM) patient satisfaction questionnaire. Sociodemographic and clinical characteristics of the participants were collected through interview and consultation of medical records. Multiple linear regression was used to determine potential factors influencing the total satisfaction score.

Results

The total satisfaction score of older adults with caregiver' communication was 2.55±0.95 and the mean scores of the lowest subscales were answering questions, greeting and listening. Analysis revealed that having visual disorders (B=-0.276±0.12; p=0.029) and receiving affective touch from caregivers (B=0.745±0.12; p=0.001) were the main factors associated with older adults' satisfaction with caregiver' communication.

Conclusion

Older people are not sufficiently satisfied with caregivers' communication skills, especially those with vision problems and those who have not received affective touch from caregivers. Caregivers need to be aware of the specific needs of older patients and use appropriate communication techniques.

Are evaluations in simulated medical encounters reliable among rater types? A comparison between standardized patient and outside observer ratings of OSCEs

Objective

By analyzing Objective Structured Clinical Examination (OSCE) evaluations of first-year interns' communication with standardized patients (SP), our study aimed to examine the differences between ratings of SPs and a set of outside observers with training in healthcare communication.

Methods

Immediately following completion of OSCEs, SPs evaluated interns' communication skills using 30 items. Later, two observers independently coded video recordings using the same items. We conducted two-tailed t-tests to examine differences between SP and observers' ratings.

Results

Rater scores differed significantly on 21 items (p < .05), with 20 of the 21 differences due to higher SP in-person evaluation scores. Items most divergent between SPs and observers included items related to empathic communication and nonverbal communication.

Conclusion

Differences between SP and observer ratings should be further investigated to determine if additional rater training is needed or if a revised evaluation measure is needed. Educators may benefit from adjusting evaluation criteria to decrease the number of items raters must complete and may do so by encompassing more global questions regarding various criteria. Furthermore, evaluation measures may be strengthened by undergoing reliability and validity testing.

Innovation

This study highlights the strengths and limitations to rater types (observers or SPs), as well as evaluation methods (recorded or in-person).

Why and How Civic Health Should Be Incorporated Into Medical Education

Civic health refers to the ability of a community to organize and collectively address problems that affect the well-being of its members through democratic participation. Civic health should be an integral part of the medical school curriculum because improving a community's civic health shifts the distribution of power toward patients, better enabling them to address social determinants of health that are affecting their well-being. This article details how to effectively integrate civic health curriculum into already-existing medical education frameworks, outlines how these interventions will improve both patient care and the student experience, and addresses barriers that might restrict the implementation. Civic health can be integrated into the didactic curriculum in the form of lunchtime guest lectures, panels with community organizations, and small-group discussions; it can be integrated into experiential curriculum by distributing QR codes to aid in voter registration, organizing voter registration drives, and participating in nonpartisan canvassing. This civic health content can be integrated into existing social justice curricula without massive investment or structural change. Medical students are capable and effective messengers of civic health and can affect change at all levels of training. Notably, because civic health is directly actionable, it can be a source of motivation rather than burnout for medical students. As students develop into medical professionals, the training in civic health improves their understanding of social determinants of health and enables them to play an important role in promoting civic engagement and empowering patients with the democratic tools necessary to enact social change.

Student characteristics associated with interpersonal skills in medical consultations

BACKGROUND

The quality of medical care depends on effective physician-patient communication. Interpersonal skills can be improved through teaching, but the determinants are poorly understood. We therefore assessed the factors associated with the interpersonal skills of medical students during simulated medical consultations.

METHODS

We conducted a cross-sectional study of fourth-year medical students participating in simulated consultations with standardized patients. Each video-recorded medical consultation was independently assessed by two raters, using a cross-cultural adaptation of the Four Habits Coding Scheme (4-HCS) into French. We then collected information on demographics and education-related characteristics. The relationship between the overall 4-HCS score and student characteristics was modeled using univariable and multivariable linear regression.

RESULTS

Our analytical sample included 165 medical students for analysis. The factors significantly associated with 4-HCS score were gender (β = - 4.8, p = 0.011) and completion of an international clinical placement (β = 6.2, p = 0.002) or a research laboratory clerkship (β = 6.5, p = 0.005). Education-related characteristics, multiple-choice examinations in the first to third preclinical years, and number of medicine or surgery clerkships were not significantly associated with 4-HCS score.

CONCLUSIONS

Undergraduate students with higher level of interpersonal skills during video-recorded medical consultations with standardized patients are more likely to be female, to have completed international clinical placement as part of the ERASMUS exchange program or research laboratory clerkship.

The influence of OSPE and PBL on competency-based pharmacy student self-assessment

BACKGROUND

In addition to vocational education and experience, the pharmaceutical profession's proper pursuit requires acquiring and continuously improving professional competencies. In recent years, the need has been increasingly highlighted for developing a medical education system based on helping students develop their competencies. It is necessary to adapt the tools and methods for assessing competencies during formal education. It will enable students to know the directions of further personal or professional development.

OBJECTIVE

The study aimed to compare pharmacy students' self-assessment outcomes before and after the Objective Structured Practical Examination (OSPE), which finished the Pharmaceutical Care course (PCc). The study's purpose was also to compare the outcomes of the self-assessment of competencies between the students of two academic years for whom classes on the PCc were provided by different methods.

METHODS

The study was conducted over two academic years (2018/2019 and 2019/2020) among 5th-year students enrolled at the Faculty of Pharmacy of the Jagiellonian University Medical College (JUMC) at the end of the Pharmaceutical Care course. Different teaching methods were used in the delivery of the course in these academic years. The students self-assessed their competencies using a questionnaire consisting of a list of personal and patient care competencies. The students completed the questionnaire before and after the OSPE, which followed the completion of the PCc.

RESULTS

Students' professional competencies as self-assessed after the exam were higher than those assessed before the exam. Differences were observed in both personal and patient care competencies. Students taking the course in the 2019/2020 academic year set their pre-OSPE competencies higher than students taking the PCc in 2018/2019.

CONCLUSION

The self-assessment scores increased for most competencies included in the study following the OSPE. This may suggest that taking part in the exam, involvement in patient's case simulations, and self-assessment of performance at individual stages of the exam contributed to increased subjective assessment of professional competencies.

Efficacy of a Standardised Patient Simulation Programme for Chronicity and End-of-Life Care Training in Undergraduate Nursing Students

BACKGROUND

Standardised patient simulations seem to be useful for improving the communication skills of health sciences students. However, it is important to define the effectiveness of these types of interventions in complex scenarios linked to disease chronicity and end-of-life contexts.

METHODS

A quasi-experimental study with pre- and post-intervention measures was carried out in a single group. A total of 161 nursing students completed different assessment instruments to measure their attitudes towards communication (Attitude Toward Communication Scale), self-efficacy (Self-Efficacy of Communication Skills, SE-12), and communication skills (Health Professionals Communication Skills Scale, EHC-PS) before and after simulation training with standardised patients. The objective of the program was to train students in non-technical skills for complex situations involving chronicity and end-of-life care. It comprised eight sessions lasting 2.5 h each.

RESULTS

The results showed notable baseline gender differences in attitudes towards communication and in the informative communication dimension, with women obtaining higher scores. The participants' self-efficacy and communication skills significantly improved after completing the intervention, with no significant differences being found for the attitudes towards communication variable.

CONCLUSION

The standardised patient simulation programme for complex scenarios related to chronicity and end-of-life contexts improved communication self-efficacy and communication skills in these nursing students. In future work it will be important to analyse the influence of gender and attitudes towards communication as variables in the learning of communication skills in nursing students.