Impact of student ethnicity and primary childhood language on communication skill assessment in a clinical performance examination

Inconsistencies in rater-based assessments mainly affect borderline candidates: but using simple heuristics might improve pass-fail decisions

INTRODUCTION

Research in various areas indicates that expert judgment can be highly inconsistent. However, expert judgment is indispensable in many contexts. In medical education, experts often function as examiners in rater-based assessments. Here, disagreement between examiners can have far-reaching consequences. The literature suggests that inconsistencies in ratings depend on the level of performance a to-be-evaluated candidate shows. This possibility has not been addressed deliberately and with appropriate statistical methods. By adopting the theoretical lens of ecological rationality, we evaluate if easily implementable strategies can enhance decision making in real-world assessment contexts.

METHODS

We address two objectives. First, we investigate the dependence of rater-consistency on performance levels. We recorded videos of mock-exams and had examiners (N=10) evaluate four students' performances and compare inconsistencies in performance ratings between examiner-pairs using a bootstrapping procedure. Our second objective is to provide an approach that aids decision making by implementing simple heuristics.

RESULTS

We found that discrepancies were largely a function of the level of performance the candidates showed. Lower performances were rated more inconsistently than excellent performances. Furthermore, our analyses indicated that the use of simple heuristics might improve decisions in examiner pairs.

DISCUSSION

Inconsistencies in performance judgments continue to be a matter of concern, and we provide empirical evidence for them to be related to candidate performance. We discuss implications for research and the advantages of adopting the perspective of ecological rationality. We point to directions both for further research and for development of assessment practices.

Predictors of Pharmacy Student Performance on Written and Clinical Examinations in a Flipped Classroom Curriculum

Objective. To examine the effects of student demographics, prior academic performance, course engagement, and time management on pharmacy students' performance on course examinations and objective structured clinical examinations (OSCEs).Methods. Study participants were one cohort of pharmacy students enrolled in a five-year combined Bachelor and Master of Pharmacy degree program at one institution. Variables included student demographics, baseline factors (language assessment and situational judgement test scores), prior academic performance (high school admission rank), course engagement, and student time management of pre-class online activities. Data were collected from course, learning management system, and institutional databases. Data were analyzed for univariate, bivariate, and multivariate associations (four linear regression models) between explanatory factors and outcome variables.Results. Three years of data on 159 pharmacy students were obtained and entered in the dataset. Significant positive predictors of OSCE communication performance included domestic (ie, Australian) student designation, higher baseline written English proficiency, and pre-class online activity completion. Positive predictors of OSCE problem-solving included workshop attendance and low empathy as measured by a baseline situational judgment test (SJT). Positive predictors of performance on year 2 end-of-course examinations included the Australian Tertiary Academic Rank, completing pre-class online activities prior to lectures, and high integrity as measured by an SJT.Conclusion. Several explanatory factors predicted pharmacy students' examination and OSCE performance in the regression models. Future research should continue to study additional contexts, explanatory factors, and outcome variables.

Oral examinations in undergraduate medical education - What is the 'value added' to evaluation?

BACKGROUND

Given the long tradition of oral examinations in surgical training, surgical clerkships continue to use oral examinations to evaluate medical students even though the value of oral examination at the post-graduate level has been questioned. The key issue in the context of undergraduate surgical training then is to understand value of the oral examination in assessment. The goal of this study is to clarify what oral examinations do, or appear to, test and how this complements other methods of assessment.

METHODS

The study is a retrospective, qualitative study of comments provided by examiners on the oral examination score sheets evaluating performance of students completing their core surgery clerkship at an academic medical center. Through immersion in and initial familiarization with the data we develop a scheme of codes for labeling the data for subsequent synthesis. Using these inductive codes, all comments were reviewed and analyzed to determine what qualities examiners detect, or naturally comment on, when administering and scoring the oral examinations.

RESULTS

Thirteen substantive codes (Communication, Critical Thinking, Decisiveness, Demeanor, Differential Diagnosis, Focus, Knowledge, Management, Organization, Pace, Prompting, Thoroughness, and Work Up) and three valence codes (Negative, Neutral, and Positive) were developed and used to code the data. The most universal code was 'Knowledge', used by 43 (100%) of examiners; the most frequently used code was 'Work Up', applied to the comments 437 (21.1%) times. Overall, positive valence was attached to 1146 (55.2%) of codes and negative valence to 879 (42.3%) codes. The most discriminating codes in grading were 'Demeanor', 'Focus', and 'Organization'.

CONCLUSIONS

Oral examinations provide rich opportunity for testing qualities readily tested on other examinations but also many intangible qualities that are otherwise less well or not well tested. As such, the 'value-added' by oral examinations likely justifies their continued use in the evaluation of surgical trainees. The identification of testable qualities should aid in the development of a standardized scoring rubric, the use of which may aid in minimizing subjectivity and bias in what otherwise is a rich assessment tool.

The Predictive Value of Full-length Practice Exams for the New MCAT Exam for Premedical Students

OBJECTIVE

To investigate whether full-length practice exams could predict performance in the new MCAT exam in a pilot group.

METHODS

A dataset of the first group of students who enrolled in the premedical post-baccalaureate program, established at Khalifa University of Science and Technology in the 2018-2019 academic year, were used. Nineteen students from the group were subsequently admitted to the newly launched Doctor of Medicine Program at KU in the 2019 fall semester.

RESULTS

Performance in the full-length practice exams may significantly predict the MCAT score (β = 0.74, t = 6.50, P = 0.000), independent of English proficiency.

CONCLUSION

These results are the first to provide direct empirical evidence supporting that doing full-length practice exams before the MCAT test day is a good strategy for preparation. Given the size of the cohort and the amount of time the program has been in existence, further studies are thus required to support this initial result.

Am I Being Understood? Veterinary Students' Perceptions of the Relationship between Their Language Background, Communication Ability, and Clinical Learning

During clinical workplace learning, effective communication between veterinary students and clinical staff is of paramount importance to facilitating learning, assessment, and patient care. Although studies in health sciences education have indicated that students may experience communication difficulties as a result of linguistic, cultural, and other factors and that these difficulties can affect clinical learning and academic outcomes, this has not yet been explored in veterinary clinical educational contexts. In this study, the authors sought to identify whether final-year veterinary students perceived that their communication ability influenced their clinical learning and, if so, whether language background was of significance. Seventy-one students from a final-year cohort at an Australian veterinary school completed a student perception survey at the end of their clinical training. Exploratory factor analysis was used to investigate the extent to which learners perceived that their communication ability influenced their clinical learning. Two factors explained 72.3% of total variance. Factor 1 related to communication ability as a source of concern; Factor 2 related to comprehending and contributing to clinical conversations. Communication ability as a source of concern differed significantly ( p < .001) between students who did and did not have an English-speaking background, but there was no significant difference between these two student groups for Factor 2. Although language background was associated with self-perceived communication ability, evidence also emerged that students may experience communication challenges during clinical learning, irrespective of their language background.

How Small Differences in Assessed Clinical Performance Amplify to Large Differences in Grades and Awards: A Cascade With Serious Consequences for Students Underrepresented in Medicine

While students entering medical schools are becoming more diverse, trainees in residency programs in competitive specialties and academic medicine faculty have not increased in diversity. As part of an educational continuous quality improvement process at the University of California, San Francisco, School of Medicine, the authors examined data for the classes of 2013-2016 to determine whether differences existed between underrepresented in medicine (UIM) and not-UIM students' clinical performance (clerkship director ratings and number of clerkship honors grades awarded) and honor society membership-all of which influence residency selection and academic career choices.This analysis demonstrated differences that consistently favored not-UIM students. Whereas the size and magnitude of differences in clerkship director ratings were small, UIM students received approximately half as many honors grades as not-UIM students and were three times less likely to be selected for honor society membership.The authors use these findings to illustrate the amplification cascade, a phenomenon in which small differences in assessed performance lead to larger differences in grades and selection for awards. The amplification cascade raises concerns about opportunities for UIM students to compete successfully for competitive residency programs and potentially enter academic careers. Using a fishbone diagram, a continuous quality improvement root cause analysis tool, the authors contextualize their institutional results. They describe potential causes of group differences, drawing from the education disparities literature, and propose interventions and future research. They also share countermeasures adopted at their institution and encourage other medical schools to consider similar exploration of their institutional data.

Ethnic and social disparities in different types of examinations in undergraduate pre-clinical training

Medical schools are increasingly faced with a more diverse student population. Generally, ethnic minority students are reported to underperform compared with those from the ethnic majority. However, there are inconsistencies in findings in different types of examinations. Additionally, little is known about the performance of first-generation university students and about performance differences across ethnic minority groups. This study aimed to investigate underperformance across ethnic minority groups and by first-generation university students in different types of written tests and clinical skills examinations during pre-clinical training. A longitudinal prospective cohort study of progress on a 3-year Dutch Bachelor of Medicine course was conducted. Participants included 2432 students who entered the course over a consecutive 6-year period (2008-2013). Compared with Dutch students, the three non-Western ethnic minority groups (Turkish/Moroccan/African, Surinamese/Antillean and Asian) underperformed in the clinical problem solving tests, the language test and the OSCEs. Findings on the theoretical end-of-block tests and writing skills tests, and results for Western minority students were less consistent. Age, gender, pre-university grade point average and additional socio-demographic variables (including first-generation university student, first language, and medical doctor parent) could explain the ethnicity-related differences in theoretical examinations, but not in language, clinical and writing skills examinations. First-generation university students only underperformed in the language test. Apparently, underperformance differs both across ethnic subgroups and between different types of written and clinical examinations. Medical schools should ensure their assessment strategies create a level playing field for all students and explore reasons for underperformance in the clinical and writing skills examinations.

A systematic review of assessment and intervention strategies for effective clinical communication in culturally and linguistically diverse students

OBJECTIVE

Culturally and linguistically diverse (CALD) students often experience difficulties with the clinical communication skills that are essential for successful interactions in the workplace. However, there is little evidence on the effectiveness of assessment and intervention strategies for this population. The two aims of this study were: to evaluate the effectiveness of assessment tools in identifying and describing the clinical communication difficulties of CALD health care students; and to determine whether communication programmes improved their clinical communication skills.

METHODS

Systematic review based on the Cochrane protocol. Articles were identified through a search of established databases using MeSH and key search terms. Studies published in English from 1990 to March 2015 were included if they described assessment strategies or a training programme for communication skills of CALD students. Studies were excluded if they did not describe implementation of a specific assessment or intervention programme. Data were extracted independently by the first author and verified by the second author. Quality was measured by the Best Evidence Medical Education guide and the Educational Interventions Critical Appraisal Tool. The Kirkpatrick hierarchy was used to measure impact. Meta-analysis was not conducted because of the heterogeneity of programme design and outcome measures.

RESULTS

One hundred and twenty-nine articles met the criteria for full text review. Eighty-six articles were excluded. Thirteen articles addressing assessment and 30 articles reporting on communication training programmes were included in this review. Assessment tools used rubrics and rating scales effectively. Intervention studies focused on speech and language skills (n = 20), interpersonal skills (n = 7) and faculty-level support (n = 5). Although 17 studies reported positive findings on student satisfaction, only eight reported improved skills post-training.

CONCLUSIONS

The development of effective assessment and intervention programmes should have an integrated design and include specific outcome measures to increase educational impact.

Pre-admission criteria and pre-clinical achievement: Can they predict medical students performance in the clinical phase?

RATIONAL

Various factors affect medical students' performance during clinical phase. Identifying these factors would help in mentoring weak students and help in selection process for residency programmes.

OBJECTIVE

Our study objective is to evaluate the impact of pre-admission criteria, and pre-clinical grade point average (GPA) on undergraduate medical students' performance during clinical phase.

METHOD

This study has a cross-sectional design that includes fifth- and sixth-year female medical students (71). Data of clinical and pre-clinical GPA in medical school and pre-admission to medical school tests scores were collected.

RESULTS

A significant correlation between clinical GPA with the pre-clinical GPA was observed (p < 0.05). Such significant correlation was not seen with other variables under study. A regression analysis was performed, and the only significant predictor of students clinical performance was the pre-clinical GPA (p < 0.001). However, no significant difference between students' clinical and pre-clinical GPA for both cohorts was observed (p > 0.05).

CONCLUSION

Pre-clinical GPA is strongly correlated with and can predict medical students' performance during clinical years. Our study highlighted the importance of evaluating the academic performances of students in pre-clinical years before they move into clinical years in order to identify weak students to mentor them and monitor their progress.

Racial and Gender Influences on Pass Rates for the UK and Ireland Specialty Board Examinations

INTRODUCTION

We explored effects of gender, ethnic origin, first language, and training status on scores in the Intercollegiate Specialty Board examinations in the UK and Ireland across the computer-marked written section and in the face-to-face oral and clinical section.

METHODS

Demographic characteristics and examination results from 9987 attempts across 177 sittings from 2009 to 2013 were analyzed in an analysis of variance by training status, gender, ethnic origin, first language, and section (computer-marked multiple-choice examination vs face-to-face oral and clinical examination).

RESULTS

We found increasing alignment between examiner and candidate characteristics during this period, with a 50% increase in examiners of Asian ethnic origin and a 60% increase in examiners whose first language is not English. The strongest factor in the analysis of variance was training status (F[2, 9818] = 27.67, p < 0.001), with candidates in training significantly outperforming others. Within "core candidates" (first attempt, in training), we found significant main effects for ethnic origin (F[5, 4809] = 2.36, p = 0.04), and first language (F[2, 4809] = 5.29, p = 0.003), but no interaction effects between these factors and section (both F < 1, p > 0.05).

CONCLUSIONS

Training status was the most important factor in candidates' results. Although the analysis showed significant effects of ethnic origin and first language within "core candidates," these differences were statistically indistinguishable between the 2 sections of the examination, suggesting that the differential attainment by these factors cannot be attributed to examiner bias in a face-to-face examination.

The effect of English proficiency and ethnicity on academic performance and progress

Non-local ethnicity or nationality and lower English proficiency have been linked with poor performance in health professional education. This study sought to compare the relative contributions of ethnicity and English proficiency, and to do so in a context where students had not been selected via interviews or some other proxy for language proficiency. Ethnicity, citizenship, and demographic data, as well as academic performance and progress were obtained for three successive cohorts (entering in 2007, 2008 and 2009) into a three-year pharmacy programme, following a common first-year. Complete data was available for all 297 students entering via this path. By the end of the programme, controlling for grades at entry, students from some non-local ethnic backgrounds (p < .001) underperformed local students, as did males (p < .001) and those who had been identified as having low English proficiency in year 1 (p < .01). Males (p < .01) and one non-local ethnic group (p < .01) were also more likely to have to repeat a year of study. Ethnicity was a stronger predictor than English proficiency or acculturation, but English proficiency was still independently and additively predictive. This suggests that targeted support strategies for students with lower English language proficiency are still important, but that cultural differences should not be underestimated.

Predictive validity of a new integrated selection process for medical school admission

BACKGROUND

This paper is an evaluation of an integrated selection process utilising previous academic achievement [Universities Admission Index (UAI)], a skills test [Undergraduate Medicine and Health Sciences Admission Test (UMAT)], and a structured interview, introduced (in its entirety) in 2004 as part of curriculum reform of the undergraduate Medicine Program at the University of New South Wales (UNSW), Australia. Demographic measures of gender, country of birth, educational background and rurality are considered.

METHOD

Admission scores and program outcomes of 318 students enrolled in 2004 and 2005 were studied. Regression analyses were undertaken to determine whether selection scores predicted overall, knowledge-based and clinical-based learning outcomes after controlling for demographics.

RESULTS

UAI attained the highest values in predicting overall and knowledge-based outcomes. The communication dimension of the interview achieved similar predictive values as UAI for clinical-based outcomes, although predictive values were relatively low. The UMAT did not predict any performance outcome. Female gender, European/European-derived country of birth and non-rurality were significant predictors independent of UAI scores.

CONCLUSION

Results indicate promising validity for an integrated selection process introduced for the Medicine Program at UNSW, with UAI and interview predictive of learning outcomes. Although not predictive, UMAT may have other useful roles in an integrated selection process. Further longitudinal research is proposed to monitor and improve the validity of the integrated student selection process.

Poorer verbal working memory for a second language selectively impacts academic achievement in university medical students

Working memory (WM) is often poorer for a second language (L2). In low noise conditions, people listening to a language other than their first language (L1) may have similar auditory perception skills for that L2 as native listeners, but do worse in high noise conditions, and this has been attributed to the poorer WM for L2. Given that WM is critical for academic success in children and young adults, these speech in noise effects have implications for academic performance where the language of instruction is L2 for a student. We used a well-established Speech-in-Noise task as a verbal WM (vWM) test, and developed a model correlating vWM and measures of English proficiency and/or usage to scholastic outcomes in a multi-faceted assessment medical education program. Significant differences in Speech-Noise Ratio (SNR₅₀) values were observed between medical undergraduates who had learned English before or after five years of age, with the latter group doing worse in the ability to extract whole connected speech in the presence of background multi-talker babble (Student-t tests, p < 0.001). Significant negative correlations were observed between the SNR₅₀ and seven of the nine variables of English usage, learning styles, stress, and musical abilities in a questionnaire administered to the students previously. The remaining two variables, Perceived Stress Scale (PSS) and the Age of Acquisition of English (AoAoE) were significantly positively correlated with the SNR₅₀, showing that those with a poorer capacity to discriminate simple English sentences from noise had learnt English later in life and had higher levels of stress – all characteristics of the international students. Local students exhibited significantly lower SNR₅₀ scores and were significantly younger when they first learnt English. No significant correlation was detected between the SNR₅₀ and the students’ Visual/Verbal Learning Style (r = −0.023). Standard multiple regression was carried out to assess the relationship between language proficiency and verbal working memory (SNR₅₀) using 5 variables of L2 proficiency, with the results showing that the variance in SNR₅₀ was significantly predicted by this model (r² = 0.335). Hierarchical multiple regression was then used to test the ability of three independent variable measures (SNR₅₀, age of acquisition of English and English proficiency) to predict academic performance as the dependent variable in a factor analysis model which predicted significant performance differences in an assessment requiring communications skills (p = 0.008), but not on a companion assessment requiring knowledge of procedural skills, or other assessments requiring factual knowledge. Thus, impaired vWM for an L2 appears to affect specific communications-based assessments in university medical students.

Does applicant personality influence multiple mini-interview performance and medical school acceptance offers?

PURPOSE

To examine relationships among applicant personality, Multiple Mini-Interview (MMI) performance, and medical school acceptance offers.

METHOD

The authors conducted an observational study of applicants who participated in the MMI at the University of California, Davis, School of Medicine during the 2010-2011 admissions cycle and responded to the Big Five Inventory measuring their personality factors (agreeableness, conscientiousness, extraversion, neuroticism, openness). Individuals' MMI performance at 10 stations was summarized as a total score. Regression analyses examined associations of personality factors with MMI score, and associations of personality factors and MMI score with acceptance offers. Covariates included sociodemographic and academic performance measures.

RESULTS

Among the 444 respondents, those with extraversion scores in the top (versus bottom) quartile had significantly higher MMI scores (adjusted parameter estimate = 5.93 higher, 95% CI: 4.27-7.59; P < .01). In a model excluding MMI score, top (versus bottom) quartile agreeableness (AOR = 3.22; 95% CI 1.57-6.58; P < .01) and extraversion (AOR = 3.61; 95% CI 1.91-6.82; P < .01) were associated with acceptance offers. After adding MMI score to the model, high agreeableness (AOR = 4.77; 95% CI 1.95-11.65; P < .01) and MMI score (AOR 1.33; 95% CI 1.26-1.42; P < .01) were associated with acceptance offers.

CONCLUSIONS

Extraversion was associated with MMI performance, whereas both extraversion and agreeableness were associated with acceptance offers. Adoption of the MMI may affect diversity in medical student personalities, with potential implications for students' professional growth, specialty distribution, and patient care.

Ethnic disparities in undergraduate pre-clinical and clinical performance

CONTEXT

Research from numerous medical schools has shown that students from ethnic minorities underperform compared with those from the ethnic majority. However, little is known about why this underperformance occurs and whether there are performance differences among ethnic minority groups.

OBJECTIVES

This study aimed to investigate underperformance across ethnic minority groups in undergraduate pre-clinical and clinical training.

METHODS

A longitudinal prospective cohort study of progress on a 6-year undergraduate medical course was conducted in a Dutch medical school. Participants included 1661 Dutch and 696 non-Dutch students who entered the course over a consecutive 6-year period (2002-2007). Main outcome measures were performance in Year 1 and in the pre-clinical and clinical courses. Odds ratios (ORs) with 95% confidence intervals (CIs) were estimated by logistic regression analysis for ethnic subgroups (Surinamese/Antillean, Turkish/Moroccan/African, Asian, Western) compared with Dutch students, adjusted for age, gender, pre-university grade point average (pu-GPA), additional socio-demographic variables (first-generation immigrant, urban background, first-generation university student, first language, medical doctor as parent) and previous performance at medical school.

RESULTS

Compared with Dutch students, Surinamese and Antillean students specifically underperformed in the Year 1 course (pass rate: 37% versus 64%; adjusted OR 0.40, 95% CI 0.27-0.60) and the pre-clinical course (pass rate: 19% versus 41%; adjusted OR 0.57, 95% CI 0.35-0.93). On the clinical course all non-Dutch subgroups were less likely than Dutch students to receive a grade of ≥ 8.0 (at least three of five grades: 54-77% versus 88%; adjusted ORs: 0.17-0.45).

CONCLUSIONS

Strong ethnic disparities exist in medical school performance even after adjusting for age, gender, pu-GPA and socio-demographic variables. More subjective grading cannot be ruled out as a cause of lower grades in clinical training, but other possible explanations should be studied further to mitigate the disparities.

Association of the pre-internship objective structured clinical examination in final year medical students with comprehensive written examinations

AIM

The purpose of this study is to evaluate the association of the pre-internship Objective Structured Clinical Examination (OSCE) in final year medical students with comprehensive written examinations. SUBJECTS AND MATERIAL: All medical students of October 2004 admission who took part in the October 2010 National Comprehensive Pre-internship Examination (NCPE) and pre-internship OSCE were included in the study (n = 130). OSCE and NCPE scores and medical grade point average (GPA) were collected.

RESULTS

GPA was highly correlated with NCPE (r = 0.76 and P<0.001) and moderately with OSCE (r = 0.68 and P < 0.001). Similarly a moderate correlation was observed between NCPE and OSCE scores(r = 0.6 and P < 0.001).Linear stepwise regression shows r(2) of a model applying GPA as predictor of OSCE score is 0.46 (β = 0.68 and P < 0.001), while addition of gender to the model increases r(2) to 0.59 (β = 0.61 and 0.36, for GPA and male gender, respectively and P < 0.001). Logistic forward regression models shows male gender and GPA are the only dependent predictors of high score in OSCE. OR of GPA and male gender for high OSCE score are 4.89 (95% CI = 2.37-10.06) and 6.95 (95% CI = 2.00-24.21), respectively (P < 0.001).

DISCUSSION

Our findings indicate OSCE and examination which mainly evaluate knowledge, judged by GPA and NCPE are moderately to highly correlated. Our results illustrate the interwoven nature of knowledge and clinical skills. In other words, certain level of knowledge is crucial for appropriate clinical performance. Our findings suggest neither OSCE nor written forms of assessments can replace each other. They are complimentary and should also be combined by other evaluations to cover all attributes of clinical competence efficiently.

Virtual humans versus standardized patients: which lead residents to more correct diagnoses?

PURPOSE

Medical educators frequently use standardized patient (SP) encounters to bridge the gap between didactic education and practical application. Typically, SPs are healthy adults with no consistent physical findings; however, highly immersive virtual humans (VHs) may enable the consistent presentation of abnormal physical findings to multiple learners across multiple repetitions. Thus, the authors conducted this study to compare how frequently junior anesthesiology residents suspected obstructive sleep apnea (OSA) in preoperative assessments of SPs versus a VH.

METHOD

The authors presented a patient whose case included the historical features of OSA (snoring, daytime fatigue, observed apnea, hypertension, and obesity). Three SPs (in 2008) and one VH (in 2009) were necessary to run the residents through the assessment. The VH appeared morbidly obese and had a neck circumference of 40 cm [corrected]. An airway exam of the VH displayed an image of redundant soft tissue, prominent tongue, and tonsillar hypertrophy. The VH responded to natural speech by recognizing "triggers" in a human's voice. The 849 triggers and 259 VH responses were designed with a technique that collects information from user interactions.

RESULTS

Five of 21 residents (23.8%) suspected OSA after interviewing the SPs, whereas 11 of 13 residents (84.6%) suspected OSA after interviewing the VH (odds ratio of 17.6; 95% CI of 2.9-107).

CONCLUSIONS

Residents suspected OSA much more frequently after interviewing the VH than after interviewing the SPs. The VH provides a unique opportunity to display numerous abnormal physical findings as part of SP encounters.

The influence of language family on academic performance in Year 1 and 2 MBBS students

OBJECTIVES

Generally, in most countries around the world, local medical students outperform, in an academic sense, international students. In an endeavour to understand if this effect is caused by language proficiency skills, we investigated academic differences between local and international MBBS students categorised by native language families.

METHODS

Data were available and obtained for medical students in their first and second years of study in 2002, 2003, 2005 and 2006. Information on social demographics, personal history and language(s) spoken at home was collected, as well as academic assessment results for each student. Statistical analysis was carried out with a dataset pertaining to a total of 872 students.

RESULTS

Local students performed better than international students in first- (p < 0.001) as well as second-year (p < 0.001) assessments. In addition, there was a main interaction effect between language family and origin in the first year (p < 0.05). For international students only, there was a main effect for language in the second year (p < 0.05), with students from Sino-Tibetan language family backgrounds obtaining higher mean scores than students from English or Indo-European language family backgrounds.

CONCLUSIONS

Our results confirmed that, overall, local students perform better academically than international students. However, given that language family differences exist, this may reflect acculturation rather than simply English language skills.

Impact of student ethnicity and patient-centredness on communication skills performance

OBJECTIVES

The development of patient-centred attitudes by health care providers is critical to improving health care quality. A prior study showed that medical students with more patient-centred attitudes scored higher in communication skills as judged by standardised patients (SPs) than students with less patient-centred attitudes. We designed this multicentre study to examine the relationships among students' demographic characteristics, patient-centredness and communication scores on an SP examination.

METHODS

Early Year 4 medical students at three US schools completed a 12-item survey during an SP examination. Survey items addressed demographics (gender, ethnicity, primary childhood language) and patient-centredness. Factor analysis on the patient-centredness items defined specific patient-centred attitudes. We used multiple regression analysis incorporating demographic characteristics, school and patient-centredness items and examined the effect of these variables on the outcome variable of communication score.

RESULTS

A total of 351 students took the SP examination and 329 (94%) completed the patient-centredness questionnaire. Responses indicated generally high patient-centredness. Student ethnicity and medical school were significantly associated with communication scores; gender and primary childhood language were not. Two attitudinal factors were identified: patient perspective and impersonal attitude. Multiple regression analysis revealed that school and scores on the impersonal factor were associated with communication scores. The effect size was modest.

CONCLUSIONS

In a medical student SP examination, modest differences in communication scores based on ethnicity were observed and can be partially explained by student attitudes regarding patient-centredness. Curricular interventions to enhance clinical experiences, teaching and feedback are needed to address key elements of a patient-centred approach to care.

Reflective practice enriches clerkship students' cross-cultural experiences

AIM

To describe a curriculum incorporating written reflection followed by reflective discussion with the goal of enhancing students' recognition and handling of cross-cultural and health disparity issues in different healthcare delivery settings. PROGRAM AND SETTING: This required curriculum was implemented within a 4-week family medicine clerkship (n = 188 students, 6 to 12 per rotation) in 23 successive rotations over 2 years. Electronic submission of a written assignment in response to structured questions was followed by in-class discussion in week 4.

PROGRAM EVALUATION

Outcomes were students' session evaluations, thematic analysis of student responses, and analysis of faculty facilitators' reflections about discussion sessions. Students' cultural knowledge about their patients' health beliefs around diabetes was assessed using multiple choice questions at the beginning and end of the clerkship.

RESULTS

One hundred percent of students submitted narratives. Student evaluations demonstrated high acceptance, appreciation of sessions and faculty. Analyses of written assignments and in-class discussions identified recurring themes. Students achieved greater synthesis and more nuanced understanding of cross-cultural encounters after discussion. Self-rating of confidence in addressing cultural issues after the curriculum was high at 3.17 +/- SD 0.57 (1-4). Cultural knowledge scores improved significantly. Core components for success were clerkship director support, required participation, experienced faculty facilitators without evaluative roles, a structured assignment and formal forum for trigger question discussion.

DISCUSSION

Written reflection followed by facilitated peer discussion adds value to simple 'exposure' to cross-cultural clinical experiences for medical students.

Using standardized patients to assess communication skills in medical and nursing students

BACKGROUND

A number of recent developments in medical and nursing education have highlighted the importance of communication and consultation skills (CCS). Although such skills are taught in all medical and nursing undergraduate curriculums, there is no comprehensive screening or assessment programme of CCS using professionally trained Standardized Patients Educators (SPE's) in Ireland. This study was designed to test the content, process and acceptability of a screening programme in CCS with Irish medical and nursing students using trained SPE's and a previously validated global rating scale for CCS.

METHODS

Eight tutors from the Schools of Nursing and Medicine at University College Cork were trained in the use of a validated communication skills and attitudes holistic assessment tool. A total of forty six medical students (Year 2 of 5) and sixty four nursing students (Year 2/3 of 4) were selected to under go individual CCS assessment by the tutors via an SPE led scenario. Immediate formative feedback was provided by the SPE's for the students. Students who did not pass the assessment were referred for remediation CCS learning.

RESULTS

Almost three quarters of medical students (33/46; 72%) and 81% of nursing students (56/64) passed the CCS assessment in both communication and attitudes categories. All nursing students had English as their first language. Nine of thirteen medical students referred for enhanced learning in CCS did not have English as their first language.

CONCLUSIONS

A significant proportion of both medical and nursing students required referral for enhanced training in CCS. Medical students requiring enhanced training were more likely not to have English as a first language.

Assessing medical students' skills in working with interpreters during patient encounters: a validation study of the Interpreter Scale

PURPOSE

Interpreted patient encounters require distinct communication skills. The absence of available reliable, valid, and practical measures hinders the assessment of these skills; therefore, the authors aimed to construct and validate the Interpreter Scale (IS).

METHOD

The authors constructed the IS based on expert consensus and prior studies. They administered the IS to two classes (n = 182) in an interpreted standardized patient (SP) case setting. Standardized interpreters in the examination room assessed, using the IS, students' communication skills. Concurrently, SPs, using the validated Patient-Physician Interaction scale (PPI) and the Interpreter Impact Rating Scale (IIRS), also assessed students' skills. Trained observers watched DVDs and used the Faculty Observer Rating Scale (FORS) to assess student performance. A prior study documented the qualities of the IIRS and FORS. The authors determined the internal consistency reliability and examined construct validity of IS scores through factor analysis and concordance with other measures' scores.

RESULTS

IS reliability analysis yielded Cronbach alpha = 0.77. Factor analysis demonstrated two IS dimensions. Nine items, "managing the encounter," and four items, "setting the stage," explained 76% and 15% of score variance, respectively. IS and FORS scores significantly correlated (r = 0.385; P < .0001). IS factor 1 scores significantly correlated (all P < .0001) with FORS (r = 0.402), IIRS (r = 0.277), and PPI (r = 0.332) scores.

CONCLUSIONS

The IS has reasonable internal consistency reliability and construct validity to warrant use for formatively measuring student communication skills in interpreted SP encounters, and it needs testing in actual patient encounters.

Validating measures of third year medical students' use of interpreters by standardized patients and faculty observers

BACKGROUND

Increasing prevalence of limited English proficiency patient encounters demands effective use of interpreters. Validated measures for this skill are needed.

OBJECTIVE

We describe the process of creating and validating two new measures for rating student skills for interpreter use.

SETTING

Encounters using standardized patients (SPs) and interpreters within a clinical practice examination (CPX) at one medical school.

MEASUREMENTS

Students were assessed by SPs using the interpreter impact rating scale (IIRS) and the physician patient interaction (PPI) scale. A subset of 23 encounters was assessed by 4 faculty raters using the faculty observer rating scale (FORS). Internal consistency reliability was assessed by Cronbach's coefficient alpha (alpha). Interrater reliability of the FORS was examined by the intraclass correlation coefficient (ICC). The FORS and IIRS were compared and each was correlated with the PPI.

RESULTS

Cronbach's alpha was 0.90 for the 7-item IIRS and 0.88 for the 11-item FORS. ICC among 4 faculty observers had a mean of 0.61 and median of 0.65 (0.20, 0.86). Skill measured by the IIRS did not significantly correlate with FORS but correlated with the PPI.

CONCLUSIONS

We developed two measures with good internal reliability for use by SPs and faculty observers. More research is needed to clarify the reasons for the lack of concordance between these measures and which may be more valid for use as a summative assessment measure.