Midclerkship feedback in the surgical clerkship: the "Professionalism, Reporting, Interpreting, Managing, Educating, and Procedural Skills" application utilizing learner self-assessment

Designing feedback processes in the workplace-based learning of undergraduate health professions education: a scoping review

BACKGROUND

Feedback processes are crucial for learning, guiding improvement, and enhancing performance. In workplace-based learning settings, diverse teaching and assessment activities are advocated to be designed and implemented, generating feedback that students use, with proper guidance, to close the gap between current and desired performance levels. Since productive feedback processes rely on observed information regarding a student's performance, it is imperative to establish structured feedback activities within undergraduate workplace-based learning settings. However, these settings are characterized by their unpredictable nature, which can either promote learning or present challenges in offering structured learning opportunities for students. This scoping review maps literature on how feedback processes are organised in undergraduate clinical workplace-based learning settings, providing insight into the design and use of feedback.

METHODS

A scoping review was conducted. Studies were identified from seven databases and ten relevant journals in medical education. The screening process was performed independently in duplicate with the support of the StArt program. Data were organized in a data chart and analyzed using thematic analysis. The feedback loop with a sociocultural perspective was used as a theoretical framework.

RESULTS

The search yielded 4,877 papers, and 61 were included in the review. Two themes were identified in the qualitative analysis: (1) The organization of the feedback processes in workplace-based learning settings, and (2) Sociocultural factors influencing the organization of feedback processes. The literature describes multiple teaching and assessment activities that generate feedback information. Most papers described experiences and perceptions of diverse teaching and assessment feedback activities. Few studies described how feedback processes improve performance. Sociocultural factors such as establishing a feedback culture, enabling stable and trustworthy relationships, and enhancing student feedback agency are crucial for productive feedback processes.

CONCLUSIONS

This review identified concrete ideas regarding how feedback could be organized within the clinical workplace to promote feedback processes. The feedback encounter should be organized to allow follow-up of the feedback, i.e., working on required learning and performance goals at the next occasion. The educational programs should design feedback processes by appropriately planning subsequent tasks and activities. More insight is needed in designing a full-loop feedback process, in which specific attention is needed in effective feedforward practices.

Evaluating Teaching Effectiveness of Medical Humanities in an Integrated Clerkship Program by a Novel Prospective Propensity Score Matching Framework

Background: This study aims to rigorously compare the effectiveness of the educational programs of a new integrated clinical clerkship in medicine (3 months) and surgery (3 months) at a cancer center with the conventional subspecialty-based rotations at a tertiary teaching hospital, by this prospective, pre-post comparative method. Methods: Between 2013–2016, we compared 69 students who had selected the integrated clerkship that emphasized clinical competency and medical humanities training with 138 matched peers who had completed conventional clerkships during the same period. Outcome measures for medical humanities included empathy, patient-centeredness, and other values and skills related to holistic health care professionalism by introducing prospective propensity score matching (PSM). Results: At baseline, no significant between-group differences existed. At the completion of the core clerkships, students receiving the integrative clerkship had significantly higher scores on the Patient–Practitioner Orientation Scale (PPOS) and the Professionalism Climate in Clinical Teaching Environment (PCI), and similar Jefferson Scale of Physician Empathy Student Version (JSPE) scores, as compared with the comparison group. We also found that the students in this program did not perform worse than those in the traditional internship group in the comprehensive and formative OSCE medical clinical skills test. Conclusions: Our study develops an empirical basis for rigorous evaluation to design medical education to improve the medical humanities values and skills of interns. Features of the new integrated clerkship program that we developed include substantial participation by the students in patient-centered in-hospital culture, as well as reflection, discussion, and feedback on actual clinical cases.

Temporal changes in medical student perceptions of their clinical skills and needs using a repeated self-assessment instrument

BACKGROUND

Medical student needs in clinical skill training may change over time, but data on this topic are limited. This study uses repeated self-assessments on clinical rotations during medical school to evaluate students' perceptions of their clinical skill growth.

METHODS

A self-assessment rating was completed by students during each clinical rotation as they progressed through their core clinical rotation year. The instrument consisted of questions on 5 clinical skill categories where students rated their performance as "below", "at" or "above" expected, and open-ended questions on strengths and challenges. We evaluated changes in self-ratings between the first (n=136) and third (n=118) quarters by matched-pair analysis of the shift in responses between time points using a Sign Test. We also identified the main themes from the students' responses to open-ended questions.

RESULTS

We found 22.4 % and 13.3 % of students increased their self-assessment ratings on "Oral Presentation Skills" and on "Differential Diagnosis", respectively. In contrast, perceived ability to communicate with patients saw the largest negative shifts. "Patient Interaction" was the most commonly identified area of strength and "Knowledge and Organization" was most frequently cited as a barrier.

CONCLUSIONS

Students demonstrated a positive shift in perceived competence in some core clinical skills that are not strongly emphasized in the preclinical curriculum, likely reflecting increased exposure over time. However, their perceived competence in communication skills declined. This may reflect initial over-estimation or true decline due to competing needs/interests. These patterns of change can inform the design of longitudinal curricula that anticipate and address students' needs during clinical rotations, such as placing increased emphasis on presentation skills and differential diagnosis earlier in the curriculum, and adding more emphasis to communication skills in later phases.

Assessment of Professionalism During the Emergency Medicine Clerkship Using the National Clinical Assessment Tool for Medical Students in Emergency Medicine

OBJECTIVES

In 2016, a national consensus conference created the National Clinical Assessment Tool for Medical Students in Emergency Medicine (NCAT-EM), a standardized end-of-shift assessment tool. We report the first large-scale analysis of professionalism concerns collected from May 2017 through December 2018 by a multisite consortium using the NCAT-EM. Our primary objective was to characterize the nature and frequency of professionalism concerns. Our secondary objective was to identify characteristics associated with giving or receiving a professionalism flag.

METHODS

The consortium database includes assessments for all students on EM clerkships at participating sites. This report presents descriptive statistics about the frequency of different flags, the distribution of flags among different student categories, assessor and student characteristics, and distribution of global assessment scores on assessments citing concerns. We used Fisher's exact test to look for associations between the frequency of professionalism flags and the sex of the students and assessors and across student categories. We used logistic regression to look for relationships between professionalism concerns and global assessment scores as well as intent to apply in EM.

RESULTS

We screened 6,768 assessments of 784 students by 719 assessors from 13 sites. After excluding assessments without flags and assessments with apparent data entry errors, we analyzed 57 (0.8%) assessments containing 79 flags. The most frequent flags were punctuality (25/79, 31.6%) and initiative (20/79, 25.3%). Few students received flags (42/784, 5.4%). Few assessors flagged concerns (41/719, 5.7%). We detected no correlation between the frequency of flags and whether a student was applying in EM or between the sex of students and assessors. Global scores of lower one-third appeared more often in assessments with a flag (30/57, 52.6% vs. 233/6,711, 3.5%).

CONCLUSIONS

Only 5.4% of students received flags. Punctuality and initiative accounted for a majority of citations. Professionalism flags correlated strongly with lower global assessment scores.

Informed self-assessment versus preceptor evaluation: a comparative study of pediatric procedural skills acquisition of fifth year medical students

BACKGROUND

Simulation training is widely used in medical education as students rarely perform clinical procedures, and confidence can influence practitioners' ability to perform procedures. Thus, this study assessed students' perceptions and experiences of a pediatric skills program and compared their informed self-assessment with their preceptor-evaluated performance competency for several pediatric clinical procedures.

METHODS

A total of 65 final-year medical students attended a weeklong pediatric skills training course by the University of Tripoli that used a manikin and various clinical scenarios to simulate real-life cases. Participants completed questionnaires self-assessing their performance skills, while examiners evaluated each students' competency on five procedural skills (lumbar puncture, nasogastric tube insertion, umbilical vein catheterization, intraosseous access, and suprapubic aspiration) using an objective structured clinical examination (OSCE) model. Differences between agreement levels in question responses were evaluated through a nonparametric chi-square test for a goodness of test fit, and the relationship between confidence levels and the OSCE scores for each procedure was assessed using Spearman's rank-order correlation.

RESULTS

All participants completed the informed self-assessment questionnaire and OSCE stations. The frequency differences in agreement levels in students' questionnaire responses were statistically significant. No significant differences were found between students' self-assessment and preceptors' evaluation scores. For each procedure's passing score rate, umbilical vein catheterization had the highest passing rate (78.5%) and nasogastric tube placement the lowest (56.9%). The mean performance scores were above passing for all procedures. The Wilcoxon signed-rank test revealed no significant differences between participants' self-assessment and their preceptor-evaluated competency; students correctly perceived and assessed their ability to perform each procedure.

CONCLUSIONS

High competence in several life-saving procedures was demonstrated among final-year medical students. The need for consistent and timely feedback, methods to increase medical students' confidence, and further development and improvement of competency-based assessments are also highlighted.

Effects of Feedback on Midwifery Students' Self-Assessed Performance and Their Self-Assessment Ability: A Quasi-Experimental Study

This study aimed to evaluate the effects of verbal and written feedback in clinical midwifery placement on students' self-assessed performance and their self-assessment ability. This three-group quasi-experimental study was conducted on 120 students. Participants in the control group received clinical education through the routine method, while in the feedback groups received either verbal or written feedback methods on the basis of the sandwich feedback model. In the last day of clinical education, a checklist was simultaneously filled out by participants and a second instructor. There was significant direct correlation between the scores of performance assessment by both the second instructor and students in the control group (r = .38, p = .01), the verbal feedback group (r = .63, p < .001), and the written feedback group (r = .74, p < .001). The rates of student-instructor agreement in the control, verbal feedback, and written feedback groups were 32.5%, 70%, and 77.5%, respectively. Feedback is effective in significantly improving students' self-assessment ability.

Point-of-Encounter Assessment: Using Health Belief Model Constructs to Change Grading Behaviors

OBJECTIVE

Fourth-year medical students need feedback to improve. Even during 1-month rotations, there needs to be a formal mid-clerkship feedback session. Better feedback involves multiple surgical evaluators at multiple levels. Constructs from the Health Belief Model of behavior change were used to assess faculty and resident grading behaviors to create a more usable evaluation system. A point-of-encounter (POE) system was created. The objective of this study was to review the efficacy of a POE clinical evaluation card (CEC) system which was initiated to increase evaluator's participation in grading and formative feedback prior to mid-clerkship evaluation.

DESIGN

The study was a 1-year retrospective cohort study reviewing the CECs for level of evaluator, content, and student compliance. A Likert-type scale survey regarding the usage of the clinical cards was also completed by evaluators.

SETTING

Texas Tech University Health Sciences Center at El Paso, during 2 fourth-year medical student rotations, Subinternship (Sub-I), and Surgical Intensive Care Unit (SICU).

PARTICIPANTS

34 fourth-year medical students and 20 evaluators.

RESULTS

Students turned in a mean of 10 cards, 75% in SICU and 65% in Sub-I turned in all 10 cards. There were significantly greater advanced residents evaluating during Sub-I vs SICU: mean evals by PGY3 were 1.9 vs 0.75 (p = .01) and mean evals by PGY5 were 1.4 vs 0.1 (p < .0001). There were significantly more faculty completing evaluations during SICU vs Sub-I: 2.5 faculty evals/student vs 1.4 faculty evals/student (p = .023). Evaluator ratings were high on a 5-point Likert-type scale, with most responses near the "strongly agree" rating of 4.7 to 4.8.

CONCLUSIONS

Use of POE CECs met goals of having at least 7 CECs turned in by mid-clerkship and 10 at end-clerkships. Formative evaluations by mid-clerkship went from 0 to 7 evaluations. Evaluator surveys highlighted clarity and efficiency as reasons for using CECs.