Development of an instrument for medical students' self-assessment of facets of competence for patient-centred care

OBJECTIVE

To develop a facets of competence self-assessment instrument (FOCSI) with operationalised items for ten competence facets required for patient-centred care at the beginning of residency.

METHODS

We conducted focus groups and cognitive interviews with final-year medical students to develop items that match students' clinical experience. We tested 50 items in two samples and analysed model fit and internal consistency of all possible combinations to identify the optimal ten-item-solution. Item analysis was performed as well as correlation with six personality traits.

RESULTS

An optimal ten-item solution for the self-assessment instrument emerged for sample 1 (n = 101, 27.2 ± 3.5 years, 75.2 % female). We validated the model fit with sample 2 (n = 135, 27.7 ± 3.9 years, 66.7 % female): χ2(35) = 49.3, p = .055, CFI = .94, RMSEA = .055, SRMR = .058, Cronbach's alpha = .78. The personality factors 'Conscientiousness' and 'Extraversion' correlate positively with most FOCSI items.

CONCLUSION

The operationalised FOCSI items support undergraduate medical students close to graduation in realistic self-assessment of facets of competences for patient-centred care in their transition to residency.

PRACTICE IMPLICATIONS

Realistic self-assessment of facets of competence will provide medical students with the opportunity to monitor their competence development as part of self-directed learning for gaining adaptive expertise in professional, patient-centred care.

"Fit for the finals" - project report on a telemedical training with simulated patients, peers, and assessors for the licensing exam

Background

Undergraduate medical students take the licensing exam (M3) as a two-day oral-practical examination. The main requirements are to demonstrate history taking skills and coherent case presentations. The aim of this project was to establish a training in which students can test their communication skills during history taking and their clinical reasoning skills in focused case presentations.

Methods

In the newly developed training, final-year students took four telemedical histories in the role of physicians from simulated patients (SP). They received further findings for two SPs and presented these in a handover, in which they also received a handover of two SPs which they had not seen themselves. Each student presented one of the two received SPs in a case discussion with a senior physician. Feedback was given to the participants on their communication and interpersonal skills by the SPs with the ComCare questionnaire and on the case presentation by the senior physician. Sixty-two students from the universities of Hamburg and Freiburg in their final year participated in September 2022 and evaluated the training.

Results

Participants felt that the training was very appropriate for exam preparation. The SPs' feedback on communication and the senior physician's feedback on clinical reasoning skills received the highest ratings in importance to the students. Participants highly valued the practice opportunity for structured history taking and case presentation and would like to have more such opportunities in the curriculum.

Conclusion

Essential elements of the medical licensing exam can be represented, including feedback, in this telemedical training and it can be offered independent of location.

A simulation-based OSCE with case presentation and remote rating - development of a prototype

Simulation-based examination formats improve the possibility to assess medical students' competences during their performance. Additionally, videotaping of simulations allows for remote rating, providing advantages for raters, students, and exam organizers. We describe a simulation-based OSCE prototype with remote rating of students' competences, developed to replace a conventional OSCE at Hamburg Medical Faculty. The assessment consists of two phases: a consultation phase with four simulated patient encounters and a case presentation phase where four students present two cases each. All encounters from the consultation and the presentation phase are to be videotaped and remotely rated by clinical raters. Advanced medical students (year 4) are to be assessed regarding their clinical knowledge as well as physician-patient-communication, clinical reasoning competence, and patient management competence. We provide detailed schedules for the simulation-based OSCE procedure and a roster for organization. When piloting the assessment, we encountered two major obstacles with respect to legal obligations regarding examination time and videotaping which allowed us to provide tips on how to successfully implement this assessment prototype. Remote rating will, when successfully implemented, help students to concentrate on their consultation or presentation tasks, reduce raters' time constraints and also allow for randomized rating. Using established instruments for competence-rating rather than OSCE checklists provides an additional feature for this simulation-based OSCE prototype. Legal issues can be avoided by using the prototype for formative assessment but should be addressed in advance when it is planned to be used as summative assessment.

Assessment of final-year medical students' entrustable professional activities after education on an interprofessional training ward: A case-control study

INTRODUCTION

Interprofessional training wards (ITWs) are implemented to provide medical students with a holistic and authentic health care experience to improve their clinical competencies. Controlled outcome studies assessing students' competencies after ITW-training are uncommon. In this case-control study, we assessed final-year medical students who received ITW-training regarding entrustable professional activities (EPAs) and communicative as well as social competencies.

METHODS

In March 2021, 32 final-year students, 16 with (ITW group) and 16 without (control group) a previous four-week placement on an ITW participated in a training simulating the first day of residency. The simulated patients assessed students' communication and interpersonal skills for history taking with the ComCare index after every consultation. Twelve prospective EPAs were assessed by three senior physicians after watching videos of the students' case presentations.

RESULTS

While baseline characteristics and ComCare index ratings were not significantly different between the two groups, the overall mean entrustment level for the 12 EPAs was significantly higher (p < 0.001) in the ITW group compared to the control group (median = 3.15 versus 2.22). The interrater reliability for all EPAs was high and entrustment in students from the ITW group was significantly higher in 10 out of 12 EPAs.

DISCUSSION

ITW training seems to prepare medical students well to practice competencies which are relevant for prospective entrustment decisions and can be deduced by senior physicians from case presentations. Further studies with larger student cohorts are needed to corroborate this finding and observable EPAs could also be defined to assess students' competencies after ITW training.

Validation of the ComCare index for rater-based assessment of medical communication and interpersonal skills

OBJECTIVES

To develop and validate a short instrument to assess undergraduate medical students' communication and interpersonal skills in videographed history taking situations with simulated patients.

METHODS

Sixty-seven undergraduate medical students participating in an assessment including videographed physician-patient encounters for history taking with five simulated patients were included in this study. The last video of each participant's consultation hour was rated by two independent assessors with the eight-item ComCare index for assessment of communication and interpersonal skills newly designed for the external rater perspective (ComCareR). We compared the sum scores of the ComCareR with ratings of the same videos with the Kalamazoo Communication Skills Assessment Form from an observational perspective (KCSAFd-video) and the Global Rating scale (GR), which also measure communication and interpersonal skills.

RESULTS

The ComCareR showed an excellent interrater reliability (ICC = .85). We found a small but significant correlation with the KCSAFd-video Interpersonal Competence (ρ = .34, 95% CI [.10,.54]) and a high positive correlation with the GR (ρ = .59, 95% CI [.40,.73]).

CONCLUSIONS

The ComCareR is a valid and brief index for holistic assessment of communication and interpersonal skills in physician-patient encounters.

PRACTICE IMPLICATIONS

The ComCareR can be used for quick rater-based assessment of physicians' communication and interpersonal skills.

Evaluating the Global Rating scale's psychometric properties to assess communication skills of undergraduate medical students in video-recorded simulated patient encounters

OBJECTIVES

To evaluate the psychometric properties of the Global Rating scale (GR) as an observer-based tool to assess communication skills of undergraduate medical students in video-recorded patient encounters.

METHODS

Seventy advanced undergraduate medical students participated in a simulation-based assessment including patient consultations. Simulated patients rated these encounters with the Consultation and Relational Empathy (CARE) scale. Two independent, blinded raters assessed the videos of the encounters with the GR and another blinded rater with the Clinical Reasoning Indicators Scale (CRI-HT-S). To assess the GR's psychometric properties, we analysed reliability by means of a G-study, interrater reliability by ICC, convergent validity (correlation of GR and CARE), and divergent validity (correlation of GR and CRI-HT-S).

RESULTS

We analysed 325 videos of 65 students (56.9% female, mean age 26.1 ± 2.2 years). The G-coefficient was.90. Interrater reliability of the GR was ICC = .95, 95% CI [.91,.97]. CARE and GR correlated moderately (ρ = .47, 95% CI [.25,.65]). GR and CRI-HT-S did not correlate (ρ = .09, 95% CI [-.16,.34]).

CONCLUSIONS

With excellent reliability and adequate validity, the quality of the GR as assessment instrument for communication skills could be demonstrated.

PRACTICE IMPLICATIONS

The GR is a suitable instrument for video-based rating of communication skills.

Final-year medical students' self-assessment of facets of competence for beginning residents

BACKGROUND

Final-year undergraduate medical students often do not feel well prepared for their start of residency training. Self-assessment of competences is important so that medical trainees can take responsibility for their learning. In this study, we investigated how final-year medical students self-assessed their competences as they neared their transition to postgraduate training. The aim was to identify areas for improvement in undergraduate training.

METHODS

In the academic year 2019/2020, a national online survey was sent to final-year undergraduate medical students via their respective medical schools. The survey included ten facets of competence (FOC) most relevant for beginning residents. The participants were asked to self-assess their competence for each FOC on a 5-point Likert scale (1: strongly disagree to 5: strongly agree). We established an order of self-assessed FOC performance by means and calculated paired t-tests. Gender differences were assessed with independent t-tests.

RESULTS

A total of 1083 students from 35 medical schools completed the questionnaire. Mean age was 27.2 ± 3.1 years and 65.8% were female. Students rated their performance highest in the FOCs 'Teamwork and collegiality' and 'Empathy and openness' (97.1 and 95.0% 'strongly agree' or 'agree', respectively) and lowest in 'Verbal communication with colleagues and supervisors' and 'Scientifically and empirically grounded method of working' (22.8 and 40.2% 'strongly disagree', 'disagree', or 'neither agree nor disagree', respectively). Women rated their performance of 'Teamwork and collegiality', 'Empathy and openness', and 'Knowing and maintaining own personal bounds and possibilities' significantly higher than men did (Cohen's d > .2), while men showed higher self-assessed performance in 'Scientifically and empirically grounded method of working' than women (Cohen's d = .38). The FOCs 'Responsibility', 'Knowing and maintaining own personal bounds and possibilities', 'Structure, work planning, and priorities', 'Coping with mistakes', and 'Scientifically and empirically grounded method of working' revealed lower self-assessed performance than the order of FOC relevance established by physicians for beginning residents.

CONCLUSIONS

The differences between the level of students' self-assessed FOC performance and physicians' ranking of FOC relevance revealed areas for improvement in undergraduate medical education related to health system sciences. Final-year students might benefit from additional or better training in management skills, professionalism, and evidence-based medicine. Surveys of self-assessed competences may be useful to monitor competence development during undergraduate training.

General population's knowledge about the anatomical locations of organs and medical terms today and 50 years ago: a replication study

Background: Physicians are frequently not aware that patients may not be familiar with the meaning of medical terms or have limited knowledge about the location of organs. These aspects of functional health competence require particular attention when designing communication curricula for undergraduate medical students. The aim of our study was to evaluate the knowledge of laypersons about the anatomical locations of organs and the definitions of commonly used medical terms as relevant aspects of health literacy. Furthermore, we wished to compare it with the knowledge of a historical patient cohort who performed this study 50 years ago. Methods: In this replication study, multiple-choice questionnaires with simple anatomy and common medical terms which were published in 1970 were distributed among a convenience sample of lay volunteers (n=537) from the streets of Hamburg, Germany. Sociodemographic data including sex, age, highest educational school achievement, occupation in a field associated with medicine, and German as first language were also collected. The percentage of laypersons' correct answers was compared to the percentage of correct answers of a historical patient cohort (n=234) published in 1970 to identify the development of health literacy as basis for curricular planning. Results: Laypersons showed significantly more correct answers in four of eight simple anatomical locations of organs (p<0.001). For seven commonly used medical terms laypersons only gave significantly more correct answers for the definitions of "jaundice" (p<0.001) and "diarrhoea" (p=0.001) compared to the historical cohort from 1970. Participants with a senior high school degree performed significantly better with respect to total scores of correct organ locations (p<0.001, d=0.35) and correct definitions of medical terms (p=0.001, d=0.30) than participants who completed junior high school. Conclusion: The definitions of common medical terms and the correct anatomical locations of organs by laypersons have increased during the past 50 years but could still need improvement by school education and media information of better quality. Medical educators should know about the low health literacy of laypersons with respect to these aspects to raise medical students' awareness for this problem and to provide communication training for medical students to use comprehensible language during history taking and shared decision making.

Are different medical school admission tests associated with the outcomes of a simulation-based OSCE?

BACKGROUND

Medical school admission procedures have the common goal to select applicants with the greatest potential of becoming successful physicians. Hamburg Medical Faculty selects medical students by grade point average (GPA) and employs a two-step selection process of a natural sciences test (HAM-Nat), in some cases followed by multiple mini-interviews (HAM-Int). Multiple mini-interviews can predict non-cognitive outcomes, while GPA has predictive validity for cognitive outcomes. The aim of our study was to explore communication skills and clinical knowledge of advanced medical students according to their respective admission procedure.

METHODS

In July 2019, 146 students grouped according to their admission procedure into GPA-only (19.2 %), HAM-Nat (33.6 %), HAM-Int (30.8 %), and Waiting List (16.4 %) participated in four OSCE stations which equally assessed students' communication skills (OSCE part 1) and clinical knowledge (OSCE part 2) in simulated patient encounters, rated by physicians with checklists. Additionally, psychosocial assessors ranked communication skills with a global rating scale (GR). The students also participated in a multiple choice (MC) exam testing clinical knowledge. Kruskal-Wallis analyses of variance of test performance and Spearman correlation of instruments were calculated.

RESULTS

Students from the Waiting List group performed significantly worse on the MC exam compared to GPA-only and HAM-Int (adjusted p = .029 and 0.018, respectively). No significant differences were found between the admission groups with respect to communication skills. Global Rating and OSCE part 1 (communication) correlated significantly (ρ = 0.228, p = .006) as did OSCE part 2 (clinical knowledge) and MC exam (ρ = 0.242, p = .003), indicating criterion validity. Constructs did not overlap, indicating divergent validity.

CONCLUSIONS

Advanced medical students selected for undergraduate studies by multiple mini-interviews assessing psychosocial skills showed similar communication skills compared to students admitted to medical school by other entryways. It is unclear whether these similarities are due to an effective undergraduate longitudinal communication curriculum. Assessing baseline communication skills of all medical students at entry-level may aid with this question.

Spinocerebellar ataxia type 14: refining clinicogenetic diagnosis in a rare adult-onset disorder

Objectives

Genetic variant classification is a challenge in rare adult‐onset disorders as in SCA‐PRKCG (prior spinocerebellar ataxia type 14) with mostly private conventional mutations and nonspecific phenotype. We here propose a refined approach for clinicogenetic diagnosis by including protein modeling and provide for confirmed SCA‐PRKCG a comprehensive phenotype description from a German multi‐center cohort, including standardized 3D MR imaging.

Methods

This cross‐sectional study prospectively obtained neurological, neuropsychological, and brain imaging data in 33 PRKCG variant carriers. Protein modeling was added as a classification criterion in variants of uncertain significance (VUS).

Results

Our sample included 25 cases confirmed as SCA‐PRKCG (14 variants, thereof seven novel variants) and eight carriers of variants assigned as VUS (four variants) or benign/likely benign (two variants). Phenotype in SCA‐PRKCG included slowly progressive ataxia (onset at 4–50 years), preceded in some by early‐onset nonprogressive symptoms. Ataxia was often combined with action myoclonus, dystonia, or mild cognitive‐affective disturbance. Inspection of brain MRI revealed nonprogressive cerebellar atrophy. As a novel finding, a previously not described T2 hyperintense dentate nucleus was seen in all SCA‐PRKCG cases but in none of the controls.

Interpretation

In this largest cohort to date, SCA‐PRKCG was characterized as a slowly progressive cerebellar syndrome with some clinical and imaging features suggestive of a developmental disorder. The observed non‐ataxia movement disorders and cognitive‐affective disturbance may well be attributed to cerebellar pathology. Protein modeling emerged as a valuable diagnostic tool for variant classification and the newly described T2 hyperintense dentate sign could serve as a supportive diagnostic marker of SCA‐PRKCG.

Need for cognitive closure, tolerance for ambiguity, and perfectionism in medical school applicants

BACKGROUND

Physicians have to deal with uncertainty on a daily basis, which requires high tolerance for ambiguity. When medical decisions have to be made in ambiguous situations, low levels of need for cognitive closure and high levels of adaptive perfectionism are beneficial. It might be useful to measure such personality traits during medical school selection processes. In our study, we explored the expression of need for cognitive closure, tolerance for ambiguity, and perfectionism in medical school applicants who participated in a multiple mini-interview selection process with respect to the final decision of admission or rejection.

METHODS

After participating in the multiple mini-interview procedure (HAM-Int) at Hamburg Medical School in August 2019, 189 medical school applicants filled out a questionnaire including the Multidimensional Perfectionism Scale by Hewitt and Flett (MPS-H), the Multidimensional Perfectionism Scale by Frost (MPS-F), the Tolerance for Ambiguity Scale (TAS), the 16-Need for Cognitive Closure Scale (16-NCCS), and sociodemographic data. After the final admission decision, the scores of need for cognitive closure, tolerance for ambiguity, and perfectionism of admitted and rejected applicants were compared. We also assessed the predictive power of need for cognitive closure and age for the admission decision in a binary logistic regression model.

RESULTS

Compared to the admitted applicants, the rejected applicants showed a significantly higher need for cognitive closure (p = .009). A high need for cognitive closure correlated significantly positively with maladaptive perfectionism (p < .001) and significantly negatively with tolerance for ambiguity (p < .001). Low need for cognitive closure and older age were associated with a positive admission decision.

CONCLUSIONS

Regarding the personality traits need for cognitive closure, tolerance for ambiguity, and perfectionism we identified interesting differences and correlations of relevance for physicians' daily work in medical school applicants who were admitted or rejected after participating in a multiple mini-interview selection procedure. Further studies are needed to investigate these characteristics and their development longitudinally in medical students and to correlate them with students' medical performance.

The relationship between perfectionism and symptoms of depression in medical school applicants

BACKGROUND

The association between perfectionism and depression in the medical profession can ultimately influence physicians' performance negatively. In medical students, especially maladaptive perfectionism is connected with distress and lower academic performance. The expression of perfectionism and symptoms of depression at the time of medical school application is not known. Therefore, we explored perfectionism and symptoms of depression in participants of multiple mini-interviews for medical school admission and investigated possible differences between applicants who were eventually admitted or rejected.

METHODS

After the multiple mini-interviews admission procedure at Hamburg Medical School in August 2018, 146 applicants filled out a questionnaire including sociodemographic data and the following validated instruments: Multidimensional Perfectionism Scale by Hewitt and Flett (MPS-H), Multidimensional Perfectionism Scale by Frost (MPS-F), Patient Health Questionnaire (PHQ-9), Generalized Anxiety Disorder Scale (GAD-7), and a 10-item version of the Big Five Inventory (BFI-10). The two groups of admitted and rejected applicants were compared and the correlation between symptoms of depression and perfectionism further explored.

RESULTS

The admitted applicants were significantly more extrovert and had lower depression scores compared to the rejected applicants. In both groups, the composite scales of Adaptive Perfectionism (r = .21, p = .011) and Maladaptive Perfectionism (r = .43, p < .001) as well as their components correlated significantly with the PHQ-9 results. Maladaptive Perfectionism accounted for about 18% of variance in the PHQ-9 score.

CONCLUSIONS

Rejected medical school applicants who participated in a multiple mini-interviews admission procedure showed higher levels of depression symptoms than admitted applicants. The degree of depressive symptoms can be partly explained by Maladaptive Perfectionism scores. Since coping in medical school and in postgraduate medical education require robust mental health, perfectionism questionnaires could be an additional tool in medical school selection processes.