Validation of a competence-based assessment of medical students' performance in the physician's role

"Fit for the finals" - project report on a telemedical training with simulated patients, peers, and assessors for the licensing exam

Background

Undergraduate medical students take the licensing exam (M3) as a two-day oral-practical examination. The main requirements are to demonstrate history taking skills and coherent case presentations. The aim of this project was to establish a training in which students can test their communication skills during history taking and their clinical reasoning skills in focused case presentations.

Methods

In the newly developed training, final-year students took four telemedical histories in the role of physicians from simulated patients (SP). They received further findings for two SPs and presented these in a handover, in which they also received a handover of two SPs which they had not seen themselves. Each student presented one of the two received SPs in a case discussion with a senior physician. Feedback was given to the participants on their communication and interpersonal skills by the SPs with the ComCare questionnaire and on the case presentation by the senior physician. Sixty-two students from the universities of Hamburg and Freiburg in their final year participated in September 2022 and evaluated the training.

Results

Participants felt that the training was very appropriate for exam preparation. The SPs' feedback on communication and the senior physician's feedback on clinical reasoning skills received the highest ratings in importance to the students. Participants highly valued the practice opportunity for structured history taking and case presentation and would like to have more such opportunities in the curriculum.

Conclusion

Essential elements of the medical licensing exam can be represented, including feedback, in this telemedical training and it can be offered independent of location.

A simulation-based OSCE with case presentation and remote rating - development of a prototype

Simulation-based examination formats improve the possibility to assess medical students' competences during their performance. Additionally, videotaping of simulations allows for remote rating, providing advantages for raters, students, and exam organizers. We describe a simulation-based OSCE prototype with remote rating of students' competences, developed to replace a conventional OSCE at Hamburg Medical Faculty. The assessment consists of two phases: a consultation phase with four simulated patient encounters and a case presentation phase where four students present two cases each. All encounters from the consultation and the presentation phase are to be videotaped and remotely rated by clinical raters. Advanced medical students (year 4) are to be assessed regarding their clinical knowledge as well as physician-patient-communication, clinical reasoning competence, and patient management competence. We provide detailed schedules for the simulation-based OSCE procedure and a roster for organization. When piloting the assessment, we encountered two major obstacles with respect to legal obligations regarding examination time and videotaping which allowed us to provide tips on how to successfully implement this assessment prototype. Remote rating will, when successfully implemented, help students to concentrate on their consultation or presentation tasks, reduce raters' time constraints and also allow for randomized rating. Using established instruments for competence-rating rather than OSCE checklists provides an additional feature for this simulation-based OSCE prototype. Legal issues can be avoided by using the prototype for formative assessment but should be addressed in advance when it is planned to be used as summative assessment.

Implicit expression of uncertainty in medical students during different sequences of clinical reasoning in simulated patient handovers

BACKGROUND

Dealing with medical uncertainty is an essential competence of physicians. During handovers, communication of uncertainty is important for patient safety, but is often not explicitly expressed and can hamper medical decisions. This study examines medical students' implicit expression of uncertainty in different sequences of clinical reasoning during simulated patient handovers.

METHODS

In 2018, eighty-seven final-year medical students participated in handovers of three simulated patient cases, which were videotaped and transcribed verbatim. Sequences of clinical reasoning and language references to implicit uncertainty that attenuate and strengthen information based on a framework were identified, categorized, and analyzed with chi-square goodness-of-fit tests.

RESULTS

A total of 6358 sequences of clinical reasoning were associated with the four main categories "statement", "assessment", "consideration", and "implication", with statements occurring significantly (p<0.001) most frequently. Attenuated sequences of clinical reasoning occurred significantly (p<0.003) more frequently than strengthened sequences. Implications were significantly more often attenuated than strengthened (p<0.003). Statements regarding results occurred significantly more often plain or strengthened than statements regarding actions (p<0.0025).

CONCLUSION

Implicit expressions of uncertainty in simulated medical students' handovers occur in different degrees during clinical reasoning. These findings could contribute to courses on clinical case presentations by including linguistic terms and implicit expressions of uncertainty and making them explicit.

Assessment of final-year medical students' entrustable professional activities after education on an interprofessional training ward: A case-control study

INTRODUCTION

Interprofessional training wards (ITWs) are implemented to provide medical students with a holistic and authentic health care experience to improve their clinical competencies. Controlled outcome studies assessing students' competencies after ITW-training are uncommon. In this case-control study, we assessed final-year medical students who received ITW-training regarding entrustable professional activities (EPAs) and communicative as well as social competencies.

METHODS

In March 2021, 32 final-year students, 16 with (ITW group) and 16 without (control group) a previous four-week placement on an ITW participated in a training simulating the first day of residency. The simulated patients assessed students' communication and interpersonal skills for history taking with the ComCare index after every consultation. Twelve prospective EPAs were assessed by three senior physicians after watching videos of the students' case presentations.

RESULTS

While baseline characteristics and ComCare index ratings were not significantly different between the two groups, the overall mean entrustment level for the 12 EPAs was significantly higher (p < 0.001) in the ITW group compared to the control group (median = 3.15 versus 2.22). The interrater reliability for all EPAs was high and entrustment in students from the ITW group was significantly higher in 10 out of 12 EPAs.

DISCUSSION

ITW training seems to prepare medical students well to practice competencies which are relevant for prospective entrustment decisions and can be deduced by senior physicians from case presentations. Further studies with larger student cohorts are needed to corroborate this finding and observable EPAs could also be defined to assess students' competencies after ITW training.

Multisource feedback: an overview of its use and application as a formative assessment

Multisource feedback (MSF), often termed 360-degree feedback, is a formative performance assessment in which data about an individual's observable workplace behaviors are collected through questionnaires from those interacting with the individual; data are aggregated for anonymity and confidentiality; the aggregated data, along with self-assessment if available, are provided to the individual; and the recipient meets with a trusted individual to review the data and develop an action plan. It is used along the continuum of medical education. This article provides an overview of MSF's utility, its evidence base and cautions.

A video-based situational judgement test of medical students' communication competence in patient encounters: Development and first evaluation

OBJECTIVE

We developed and evaluated the Video-Based Assessment of Medical Communication Competence (VA-MeCo), a construct-driven situational judgement test measuring medical students' communication competence in patient encounters.

METHODS

In the construction phase, we conducted two expert studies (n_panel1 = 6, n_panel2 = 13) to ensure curricular and content validity and sufficient expert agreement on the answer key. In the evaluation phase, we conducted a cognitive pre-test (n = 12) and a pilot study (n = 117) with medical students to evaluate test usability and acceptance, item statistics and test reliability depending on the applied scoring method (raw consensus vs. pairwise comparison scoring).

RESULTS

The results of the expert interviews indicated good curricular and content validity. Expert agreement on the answer key was high (ICCs> .86). The pilot study showed favourable usability and acceptance by students. Irrespective of the scoring method, reliability for the complete test (Cronbach's α >.93) and its subscales (α >.83) was high.

CONCLUSION

There is promising evidence that medical communication competence can be validly and reliably measured using a construct-driven and video-based situational judgement test.

PRACTICE IMPLICATIONS

Video-based SJTs allow efficient online assessment of medical communication competence and are well accepted by students and educators.

Validation of the ComCare index for rater-based assessment of medical communication and interpersonal skills

OBJECTIVES

To develop and validate a short instrument to assess undergraduate medical students' communication and interpersonal skills in videographed history taking situations with simulated patients.

METHODS

Sixty-seven undergraduate medical students participating in an assessment including videographed physician-patient encounters for history taking with five simulated patients were included in this study. The last video of each participant's consultation hour was rated by two independent assessors with the eight-item ComCare index for assessment of communication and interpersonal skills newly designed for the external rater perspective (ComCareR). We compared the sum scores of the ComCareR with ratings of the same videos with the Kalamazoo Communication Skills Assessment Form from an observational perspective (KCSAFd-video) and the Global Rating scale (GR), which also measure communication and interpersonal skills.

RESULTS

The ComCareR showed an excellent interrater reliability (ICC = .85). We found a small but significant correlation with the KCSAFd-video Interpersonal Competence (ρ = .34, 95% CI [.10,.54]) and a high positive correlation with the GR (ρ = .59, 95% CI [.40,.73]).

CONCLUSIONS

The ComCareR is a valid and brief index for holistic assessment of communication and interpersonal skills in physician-patient encounters.

PRACTICE IMPLICATIONS

The ComCareR can be used for quick rater-based assessment of physicians' communication and interpersonal skills.

Evaluating the Global Rating scale's psychometric properties to assess communication skills of undergraduate medical students in video-recorded simulated patient encounters

OBJECTIVES

To evaluate the psychometric properties of the Global Rating scale (GR) as an observer-based tool to assess communication skills of undergraduate medical students in video-recorded patient encounters.

METHODS

Seventy advanced undergraduate medical students participated in a simulation-based assessment including patient consultations. Simulated patients rated these encounters with the Consultation and Relational Empathy (CARE) scale. Two independent, blinded raters assessed the videos of the encounters with the GR and another blinded rater with the Clinical Reasoning Indicators Scale (CRI-HT-S). To assess the GR's psychometric properties, we analysed reliability by means of a G-study, interrater reliability by ICC, convergent validity (correlation of GR and CARE), and divergent validity (correlation of GR and CRI-HT-S).

RESULTS

We analysed 325 videos of 65 students (56.9% female, mean age 26.1 ± 2.2 years). The G-coefficient was.90. Interrater reliability of the GR was ICC = .95, 95% CI [.91,.97]. CARE and GR correlated moderately (ρ = .47, 95% CI [.25,.65]). GR and CRI-HT-S did not correlate (ρ = .09, 95% CI [-.16,.34]).

CONCLUSIONS

With excellent reliability and adequate validity, the quality of the GR as assessment instrument for communication skills could be demonstrated.

PRACTICE IMPLICATIONS

The GR is a suitable instrument for video-based rating of communication skills.

Frequency of medical students' language expressing implicit uncertainty in simulated handovers

OBJECTIVES

The aim of this study was to investigate the number and type of implicit expressions of uncertainty by medical students during simulated patient handovers.

METHODS

Eighty-seven volunteer medical students, a convenience sample collected on a first-come, first-served basis, participated in simulated handovers. They each worked with three simulated patients who presented with different chief complaints and personal conditions. The handovers were video recorded and transcribed. A framework of implicit expressions of uncertainty was used to identify and count modifiers that attenuate or strengthen medical information using MAXQDA lexical search. We analysed the findings with respect to the patients' contexts.

RESULTS

Implicit uncertainty expressions which attenuate or strengthen information occurred in almost equal frequency, 1879 (55%) versus 1505 (45%). Attenuators were found most frequently in the category 'Questionable', 1041 (55.4%), strengtheners in the category 'Focused', 1031 (68.5%). Most attenuators and strengtheners were found in the handover of two patients with challenging personal conditions ('angry man', 434 (23.1%) versus 323 (21.5%); 'unfocused woman', 354 (19.4%) versus 322 (21.4%)) and one patient with abnormal laboratory findings ('elevated creatinine', 379 (20.2%) versus 285 (18.9%)).

CONCLUSIONS

Medical students use a variety of implicit expressions of uncertainty in simulated handovers. These findings provide an opportunity for medical educators to design communication courses that raise students' awareness for content-dependent implicit expressions of uncertainty and provide strategies to communicate uncertainty explicitly.

Final-year medical students' self-assessment of facets of competence for beginning residents

BACKGROUND

Final-year undergraduate medical students often do not feel well prepared for their start of residency training. Self-assessment of competences is important so that medical trainees can take responsibility for their learning. In this study, we investigated how final-year medical students self-assessed their competences as they neared their transition to postgraduate training. The aim was to identify areas for improvement in undergraduate training.

METHODS

In the academic year 2019/2020, a national online survey was sent to final-year undergraduate medical students via their respective medical schools. The survey included ten facets of competence (FOC) most relevant for beginning residents. The participants were asked to self-assess their competence for each FOC on a 5-point Likert scale (1: strongly disagree to 5: strongly agree). We established an order of self-assessed FOC performance by means and calculated paired t-tests. Gender differences were assessed with independent t-tests.

RESULTS

A total of 1083 students from 35 medical schools completed the questionnaire. Mean age was 27.2 ± 3.1 years and 65.8% were female. Students rated their performance highest in the FOCs 'Teamwork and collegiality' and 'Empathy and openness' (97.1 and 95.0% 'strongly agree' or 'agree', respectively) and lowest in 'Verbal communication with colleagues and supervisors' and 'Scientifically and empirically grounded method of working' (22.8 and 40.2% 'strongly disagree', 'disagree', or 'neither agree nor disagree', respectively). Women rated their performance of 'Teamwork and collegiality', 'Empathy and openness', and 'Knowing and maintaining own personal bounds and possibilities' significantly higher than men did (Cohen's d > .2), while men showed higher self-assessed performance in 'Scientifically and empirically grounded method of working' than women (Cohen's d = .38). The FOCs 'Responsibility', 'Knowing and maintaining own personal bounds and possibilities', 'Structure, work planning, and priorities', 'Coping with mistakes', and 'Scientifically and empirically grounded method of working' revealed lower self-assessed performance than the order of FOC relevance established by physicians for beginning residents.

CONCLUSIONS

The differences between the level of students' self-assessed FOC performance and physicians' ranking of FOC relevance revealed areas for improvement in undergraduate medical education related to health system sciences. Final-year students might benefit from additional or better training in management skills, professionalism, and evidence-based medicine. Surveys of self-assessed competences may be useful to monitor competence development during undergraduate training.

Changing trends in assessment: Effectiveness of Direct observation of procedural skills (DOPS) as an assessment tool in anesthesiology postgraduate students

Background and Aims:

National Medical commission of India (NMC) has introduced Competency based Medical Education (CBME) following the international trend. Competency based assessment differs from traditional assessment and we need to adopt to newer work place-based assessments (WPBA). Direct Observation of Procedural Skills (DOPS) is one such assessment tool, which assesses procedural skills of a student. Hence this study was conducted to evaluate the effectiveness of DOPS among Anesthesiology postgraduate students.

Material and Methods:

A prospective interventional study was conducted, where in 55 postgraduate students underwent DOPS assessment while performing three routine core skills of Anesthesiology. Observations were documented (Pre-DOPS-Score) and immediate verbal feedback was given. Students were educated through demonstration of all three procedures on actual patients by three faculty members using the DOPS checklist. The students were finally assessed again with proforma and DOPS scores were documented (Post-DOPS Score). Mean Pre-DOPS and Post-DOPS assessment Scores were compared. Student and faculty Perceptions were taken regarding the DOPS assessment.

Results:

There was significant improvement in the procedural skill performance after DOPS with a statistically significant difference. Average time taken for DOPS test was 11 min. Average time taken for giving feedback was 13 min. Students and faculty members gave positive feedback about DOPS.

Conclusion:

DOPS is very feasible and effective assessment tool, improves procedural skills of Anaesthesiology postgraduate students, helps to develop good clinical skills which finally brings good clinical care.

It does not have to be either or! Assessing competence in medicine should be a continuum between an analytic and a holistic approach

Assessing competence is a tremendous challenge in medical education. There are two contrasting approaches in competence assessment: an analytic approach that aims to precisely measure observable constituents and facets of competence and a holistic approach that focuses on a comprehensive assessment of competences in complex real situations reflecting actual performance. We would like to contribute to the existing discourse about medical competence and its assessment by proposing an approach that can provide orientation for the development of competence-based assessment concepts in undergraduate and postgraduate medical education. The approach follows Kane's framework of an "argument-based approach" to validity and is based on insights into task complexity, testing and learning theories as well as the importance of the learning environment. It describes a continuum from analytic to holistic approaches to assess the constituents and facets of competence to performance. We conclude that the complexity of a task should determine the selection of the assessment and suggest to use this approach to reorganize and adapt competence assessment.

Variable trajectory: a systematic review, analytic synthesis and construct domain consolidation of international measures of competence in doctors and medical students

BACKGROUND

Competence is assessed throughout a doctor's career. Failure to identify and manage impaired competence can have critical consequences. Consistent conceptualisation and accurate measurement of this construct is imperative. Therefore, the objective of this review was to identify and evaluate measures used to assess competence in doctors and medical students.

METHODS

A systematic search of the published literature was undertaken between December 2019 and February 2020 for articles reporting on the measurement of competence in doctors and/or medical students. Searches were conducted in the PsychSOURCE, US National Library of Medicine National Institutes of Health, MEDLINE (PubMed), The Cochrane Central Register of Controlled Trials and Web of Science electronic databases. Citation screening and forward citation tracking of included studies were carried out to identify any further relevant papers for inclusion. One thousand one hundred and thirty-six potentially relevant articles were screened. An analytic synthesis approach was implemented to the identification, organisation and interpretation of homogenous study and measure characteristics.

RESULTS

Twelve competence domains were identified from the 153 identified measures. Knowledge and procedural competence domains were the dominant focus of publications reporting current medical practice, but less so in research-based studies which more frequently assessed interpersonal, psychological, physiological and ethical competencies. In the 105 included articles, the reporting of measurement instrument quality was varied, with comprehensive reporting only present in 53.6% of measures; validation for some of the measures was particularly limited.

DISCUSSION

While this review included a considerable number of publications reporting the measurement of competence in doctors and medical students, the heterogeneity of the measures and variation of findings limit the ability to evaluate their validity and generalisability. However, this review presents a resource for researchers and medical educators which may inform operational practice and future research.

PROSPERO REGISTRATION NUMBER

CRD42020162156.

Final-year medical students' competence profiles according to the modified requirement tracking questionnaire

BACKGROUND

Undergraduate medical education is supposed to equip medical students with basic competences to select any specialty of their choice for postgraduate training. Medical specialties are characterized by a great diversity of their daily work routines and require different sets of competence facets. This study examines the self-assessed competence profiles of final-year undergraduate medical students and their specialty choice for postgraduate training. Students' profiles, who wish to choose anaesthesiology, internal medicine, or paediatrics, are compared with the physicians' competence profiles from these three disciplines.

METHODS

In this study, 148 volunteer final-year undergraduate medical students completed the modified requirement-tracking (R-Track) questionnaire for self-assessment of their competence profiles. The R-Track questionnaire contains 63 competence facets assigned to six areas of competence: "Mental abilities", "Sensory abilities", "Psychomotor & multitasking abilities", "Social interactive competences", "Motivation", and "Personality traits". The expression of the different competence facets had to be assessed on a 5-point Likert scale (1: "very low" to 5: "very high"). Additionally, socio-demographic data and the participants' first choice of a medical speciality for postgraduate education were collected. We used analysis of variance (ANOVA) for mean score comparison of subgroups and least significant difference (LSD) tests for post hoc analysis.

RESULTS

The competence area with the highest rating was "Motivation" (3.70 ± 0.47) while "Psychomotor & multitasking abilities" received the lowest rating (3.34 ± 0.68). Individual facets of competence ranked from "In need of harmony" (4.36 ± 0.72), followed by "Tactfulness" (4.26 ± 0.64), and "Cooperation/Agreeableness" (4.24 ± 0.53) to "Risk orientation" (2.90 ± 0.92), "Mathematical reasoning" (2.87 ± 1.25), and "Sanctioning" (2.26 ± 0.93). The students' competence profiles showed 100 % congruence with physicians' competence profiles of the postgraduate specialty of their choice for internal medicine, 33.3 % for paediatrics, and 0 % for anaesthesiology.

CONCLUSIONS

Undergraduate medical students could define their competence profiles with the modified R-Track questionnaire and compare them with the profile of their desired specialty for postgraduate training to discover possible learning gaps or to detect good specialty matches. A combination of students' competence self-assessment with an external assessment of students' facets of competence could provide curricular planners with useful information how to design learning opportunities for specific facets of competence.

Perspective matters: assessment of medical students' communication and interpersonal skills by simulated patients from the internal and external patient perspective

Background: Communication and interpersonals skills are important qualities of professionalism in medicine. In medical curricula, they are usually acquired in communication trainings and assessed in OSCEs. Studies show correlations as well as differences between communication ratings of examiners and simulated patients. In our study, simulated patients assessed medical students' communication and interpersonal skills after a consultation hour from the internal and the external patient perspective. Methods: In December 2019, 52 final-year medical students participated in a consultation hour as part of a simulated first day of residency assessment. They were assessed twice with a questionnaire for communication and interpersonal skills (ComCare) by the simulated patients: directly after each consultation from the internal perspective of the patient's view (internal perspective) and four month later from the 208 consultation videos from an external perspective of the patient's view (external perspective). All eight ComCare items were assessed on a five-point Likert scale (1=full disagreement to 5=full agreement). Differences between the item means of internal and external perspective were examined by paired t-tests. Results: Overall, significantly higher ratings were found for all ComCare items from the external perspective except for the item "Interest". Ratings for the items "Language" and "Listening" were significantly higher from the external perspective for all simulated patients. Significantly higher ratings for all items from the external perspective were observed for two simulated patients. Conclusion: Simulated patients' ratings after a conversation seem to represent a more authentic view on students' communication and interpersonal skills because of the emotionally experienced situation. The evaluation of those skills from a simulated patient perspective could be a valuable complement to communication ratings by examiners.

Development and pilot test of ComCare - a questionnaire for quick assessment of communicative and social competences in medical students after interviews with simulated patients

Background: Physicians' communicative and social competences are highly relevant for doctor-patient relationships. Simulation-based learning is frequently used to provide students with learning experiences resembling realistic medical situations. This study aims to assess communication and interpersonal skills in medical students after simulated consultations with a newly designed short questionnaire. Methods: In 2019, 103 final year students participated in a simulated consultation hour seeing four simulated patients. Communicative and social competences were assessed by a questionnaire including items for communication (Com) and interpersonal (Care) skills. The questionnaire was used by the simulated patients (ComCareP) after each consultation and as self-assessment by the students (ComCareD) after the fourth consultation. An explorative factor analysis was performed and the results of ComCareP and ComCareD were compared with respect to students' sex and advancement in their final year. Results: All ComCareP items loaded on one factor, which explained 50.7% of the variance. The participants self-assessed their communication and interpersonal skills significantly better than the simulated patients. No significant differences were found for students' sexes or advancement in their final year except for the item "responding to patients' needs satisfactorily" which was significantly lower in students at the end of their final year. Patients' general "satisfaction with the consultation" was higher while physicians' general "satisfaction with the consultation" was lower than their total ComCare mean score. The general satisfaction with the consultation showed a significant positive correlation with both ComCares' total mean scores. Conclusion: The ComCare measures communication and interpersonal skills as one factor. It can be used directly after consultations and shows significant positive correlation with the general satisfaction with a consultation. Since simulated patients' satisfaction with the consultation was higher than their ComCare score, other factors than communication and interpersonal skills could play a role for patient satisfaction with a conversation and need to be further investigated.

Evaluation of a telemedicine-based training for final-year medical students including simulated patient consultations, documentation, and case presentation

Background: Focused history taking, knowledge-based clinical reasoning, and adequate case presentation during hand-offs represent important facets of competence of practicing physicians. Based on a validated 360-degree assessment simulating a first day of residency we developed a training for final-year medical students including patient consultation, patient management, and patient hand-off. Due to the COVID-19 pandemic the training was changed to a telemedicine format and evaluated. Methods: In 2019, 103 final-year students participated in a newly designed competence-based training including a consultation hour with simulated patients, a patient management phase with an electronic patient chart, and a case presentation in hand-off format. Due to social distancing regulations, the training was not allowed to take place in this way. Therefore, we changed the training to a telemedicine format. In May 2020, 32 students participated in the telemedicine training. A 5-point Likert scale (1: does not apply to 5: fully applies) was used for the evaluation items. The two formats were compared with t-tests. Results: The students were similarly satisfied with the content of the training independently of its format. Both groups found the patient cases interesting (presence: 4.68 ± 0.49, telemedicine: 4.66 ± 0.48). With respect to the telemedicine format, participants were glad that an option had been found that could be offered throughout the final year (4.94 ± 0.24) despite the COVID-19 pandemic and they regarded it as a very useful training for their final examination (4.94 ± 0.24). Conclusion: The telemedicine format of the competence-based training worked as well as the presence format. In its telemedicine format, the training can be offered to students independently of their location.

Assessing clinical reasoning in undergraduate medical students during history taking with an empirically derived scale for clinical reasoning indicators

BACKGROUND

The clinical reasoning process, which requires biomedical knowledge, knowledge about problem-solving strategies, and knowledge about reasons for diagnostic procedures, is a key element of physicians' daily practice but difficult to assess. The aim of this study was to empirically develop a Clinical Reasoning Indicators-History Taking-Scale (CRI-HT-S) and to assess the clinical reasoning ability of advanced medical students during a simulation involving history taking.

METHODS

The Clinical Reasoning Indictors-History Taking-Scale (CRI-HT-S) including a 5-point Likert scale for assessment was designed from clinical reasoning indicators identified in a qualitative study in 2017. To assess indicators of clinical reasoning ability, 65 advanced medical students (semester 10, n = 25 versus final year, n = 40) from three medical schools participated in a 360-degree competence assessment in the role of beginning residents during a simulated first workday in hospital. This assessment included a consultation hour with five simulated patients which was videotaped. Videos of 325 patient consultations were assessed using the CRI-HT-S. A factor analysis was conducted and the students' results were compared according to their advancement in undergraduate medical training.

RESULTS

The clinical reasoning indicators of the CRI-HT-S loaded on three factors relevant for clinical reasoning: 1) focusing questions, 2) creating context, and 3) securing information. Students reached significantly different scores (p < .001) for the three factors (factor 1: 4.07 ± .47, factor 2: 3.72 ± .43, factor 3: 2.79 ± .83). Students in semester 10 reached significantly lower scores for factor 3 than students in their final year (p < .05).

CONCLUSIONS

The newly developed CRI-HT-S worked well for quantitative assessment of clinical reasoning indicators during history taking. Its three-factored structure helped to explore different aspects of clinical reasoning. Whether the CRI-HT-S has the potential to be used as a scale in objective structured clinical examinations (OCSEs) or in workplace-based assessments of clinical reasoning has to be investigated in further studies with larger student cohorts.

Development of competency model for family physicians against the background of 'internet plus healthcare' in China: a mixed methods study

BACKGROUND

Identification of the service competences of family physicians is central to ensuring high-quality primary care and improving patient outcomes. However, little is known about how to assess the family physicians' service competences in primary care settings. It is necessary to develop and validate a general model of core competences of the family physician under the stage of construction of family doctor system and implementation of 'Internet Plus Healthcare' service model in China.

METHODS

The literature review, behavioural event interviews, expert consultation and questionnaire survey were performed. The scale's 35 questions were measured by response rate, highest score, lowest score, and average score for each. Delphi method was used to assess content validity, Cronbach's α to estimate reliability, and factor analysis to test structural validity. Respondents were randomly divided into two groups; data for one group were used for exploratory factor analysis (EFA) to explore possible model structure. Confirmatory factor analysis (CFA) was then performed.

RESULTS

Effective response rate was 93.56%. Cronbach's α coefficient of the scale was 0.977. Factor analysis showed KMO of 0.988. Bartlett's test showed χ2 of 22 917.515 (df = 630), p < .001. Overall authority grade of expert consultation was 0.80, and Kendall's coefficient of concordance W was 0.194. By EFA, the five-factor model was retained after thorough consideration, and four items with factor loading less than 0.4 were proposed to obtain a five-dimension, 32-item scale. CFA was performed on the new structure, showing high goodness-of-fit test (NFI = 0.98, TLI = 0.91, SRMSR = 0.05, RMSEA = 0.04). Overall Cronbach's α coefficients of the scale and each sub-item were greater than 0.9.

CONCLUSIONS

The scale has good reliability, validity, and credibility and can therefore serve as an effective tool for assessment of Chinese family physicians' service competences.

Implicit expression of uncertainty - suggestion of an empirically derived framework

BACKGROUND

Uncertainty occurs in physicians' daily work in almost every clinical context and is also present in the clinical reasoning process. The way physicians communicate uncertainty in their thinking process during handoffs is crucial for patient safety because uncertainty has diverse effects on individuals involved in patient care. Dealing with uncertainty and expressing uncertainty are important processes in the development of professional identity of undergraduate medical students. Many studies focused on how to deal with uncertainty and whether uncertainty is explicitly expressed. Hardly any research has been done regarding implicit expression of uncertainty. Therefore, we studied the ways in which medical students in the role of beginning residents implicitly express uncertainty during simulated handoffs.

METHODS

Sixty-seven advanced undergraduate medical students participated in a simulated first day of residency including a consultation hour, a patient management phase with interprofessional interaction, and a patient handoff. We transcribed the videographed handoffs verbatim and extracted language with respect to expression of uncertainty using a grounded theory approach. Text sequences expressing patient related information were analyzed and coded with respect to language aspects which implicitly modified plain information with respect to increasing or decreasing uncertainty. Concepts and categories were developed and discussed until saturation of all aspects was reached.

RESULTS

We discovered a framework of implicit expressions of uncertainty regarding diagnostic and treatment-related decisions within four categories: "Statement", "Assessment", "Consideration", and "Implication". Each category was related to either the subcategory "Actions" or "Results" within the diagnostic or therapeutic decisions. Within each category and subcategory, we found a subset of expressions, which implicitly attenuated or strengthened plain information thereby increasing uncertainty or certainty, respectively. Language that implicitly attenuated plain information belonged to the categories questionable, incomplete, alterable, and unreliable while we could ascribe implicit strengtheners to the categories assertive, adequate, focused, and reliable.

CONCLUSIONS

Our suggested framework of implicit expression of uncertainty may help to raise the awareness for expression of uncertainty in the clinical reasoning process and provide support for making uncertainty explicit in the teaching process. This may lead to more transparent communication processes among health care professionals and eventually to improved patient safety.