Teaching feedback to first-year medical students: long-term skill retention and accuracy of student self-assessment

Effects of basic endoscopic handling and care training on gastrointestinal endoscopy logistics

Background and study aims  The current practice of endoscopists is undergoing a dramatic revolution due to emerging endoscopy practices. Increasing use of gastrointestinal endoscopy has led to hospital budgets setting aside funds specifically related to damage to endoscopic instruments. Therefore, training in understanding endoscopic equipment, handling techniques, and equipment care can be helpful in addressing this issue. The aim of this study was to investigate the effects of educational courses and training about basic endoscopic handling and care in gastrointestinal endoscopic care and services. Methods  A number of new endoscopists, nurses, and nurse assistants were enrolled in a course for training in basic endoscopic handling and care. Data on the type of damage, cause, cost, and timing of endoscopic repair were prospectively collected. Data from the post-training period then were compared with retrospective data from the pre-training period. Results  This study demonstrated that after training, there was less damage to endoscopes, lower costs associated with it, and repair times were shorter for endoscopes than before the training course. Post-training results indicated savings of a total of $ 40,617.21 or £ 29,539.78 and 102.6 days per damaged endoscope. Conclusions  Basic endoscopic handling and care training plays an important role for both endoscopists and nurses, as well as in endoscopy facilities, specifically in avoiding the nuisance of unwanted and broken endoscopes. This could be beneficial for both hospital finances and endoscopic services.

Improving effectiveness of dental students' feedback and course evaluation

Dental students providing feedback about a course they take, in a timely manner, benefits not only teachers, but also indirectly the students themselves, especially if given with confidence in a constructive manner. Therefore, the aim of this study was to train students on how to give feedback, to ask them to provide feedback before and after the instructions were given, and analyze the change in their responses. Participants were students who attended the second-year preclinical course in prosthodontics. They were asked to provide feedback anonymously with online surveys after completing modules of the course during the academic year. There was no intervention prior to the first feedback; however, before providing the second feedback, students were asked to read a 1-page handout related to feedback modalities. Following this, an interactive workshop in feedback was provided prior to the third survey. The received responses were ranked as either: neutral, positive, negative, or constructive and were analyzed using a mixed repeated measures test with Bonferroni correction at a 0.05 significance level. The results showed a higher number of constructive and positive responses than both neutral and negative feedback (P ≤ 0.05) within the same surveys, but no interaction effect was found between the surveys (P = 0.076). Our data showed an increase in constructive feedback provided by students after the 2 different training methods, but the modality of delivery did not seem to significantly influence the results. In summary, training students on how to provide constructive feedback may be beneficial for teachers to improve their courses.

Improving Student Feedback Quality: A Simple Model Using Peer Review and Feedback Rubrics

Constructive feedback is an important aspect of medical education to help students improve performance in cognitive and clinical skills assessments. However, for students to appropriately act on feedback, they must recognize quality feedback and have the opportunity to practice giving, receiving, and acting on feedback. We incorporated feedback literacy into a case-based concept mapping small group-learning course. Student groups engaged in peer review of group-constructed concept maps and provided written peer feedback. Faculty also provided written feedback on group concept maps and used a simple rubric to assess the quality of peer feedback. Groups were provided feedback on a weekly basis providing an opportunity for timely improvement. Precourse and postcourse evaluations along with peer-review feedback assessment scores were used to show improvement in both group and individual student feedback quality. Feedback quality was compared to a control student cohort that engaged in the identical course without implementing peer review or feedback assessment. Student feedback quality was significantly improved with feedback training compared to the control cohort. Furthermore, our analysis shows that this skill transferred to the quality of student feedback on course evaluations. Feedback training using a simple rubric along with opportunities to act on feedback greatly enhanced student feedback quality.

Evaluation of medical student retention of clinical skills following simulation training

BACKGROUND

Adequate clinical skills training is a challenge for present day medical education. Simulation Based Education (SBE) is playing an increasingly important role in healthcare education worldwide to teach invasive procedures. The impact of this teaching on students along with retention of what is taught is not fully understood. The purpose of this study was to evaluate the retention levels of practical skills taught and assessed by SBE and to explore the degree of re-training required to restore decayed performance. In exploring this aim, the study further investigates how skilled performance decays over time and which dimensions of clinical skills were more likely to decay.

METHODS

Study participants were 51 final year medical students. They were provided with online pre-course videos and procedural guides asynchronously with repeatedly access. 7 of the skills taught over 2 years using task trainers were selected. Following demonstration from faculty, students practiced in small groups with faculty facilitated supervision and peer support prior to formal testing. Score sheets with itemised procedure checklists detailing the minimum passing standard (MPS) for each skill were designed. To test retention of skills, 18 months later, there was an unannounced test to demonstrate proficiency in the skills. Students were asked to complete a questionnaire indicating how many times and where they had practiced or performed the skills.

RESULTS

55% of the students were deficient in 3 or more skills and 4% were not competent in 5 or more skills. A significant number of students had never practiced some skills following the initial teaching session. A relationship was noted with the number of times students self-declared that they had practiced and their performance. Decay is evident in both psychomotor and cognitive domains of the skills.

CONCLUSION

A curriculum with deliberate practice significantly increases the competence of students in defined clinical skills. Deliberate practice of clinical skills, under supervision of an engaged instructor, is a key component of the mastery model. Experiences and assessments in the clinical setting need to be augmented with focus on direct observation and focused feedback to reinforce the skills acquired in the simulated setting.

Impact of training and student self-identification on frequency, constructiveness, and professionalism of pharmacy student evaluations of teaching

INTRODUCTION

The purpose of this study was to simultaneously assess impact of enhanced training and anonymity on frequency, constructiveness, and professionalism of open-ended comments provided on course and instructor evaluations.

METHODS

In a 2 × 2 factorial study, didactic students at one doctor of pharmacy program were randomized to receive enhanced training in providing constructive feedback on student evaluations of teaching or an unrelated control activity at the beginning of the academic year, and to provide or not provide an electronic signature at the time of evaluation completion for two consecutive semesters. After ensuring intercoder reliability and agreement among all investigators, one external investigator coded all open-ended comments based on the following factors: comment provided/comment not provided, constructive/less than constructive, professional/unprofessional, training/no training, and signed/unsigned.

RESULTS

Of 836 opportunities to respond to open-ended items, 646 (77.3%) written comments were provided, which was similar by training (76.6% vs. 78.0%, p = 0.631) and signature (76.1% vs. 78.5%, p = 0.401) status. Of the 646 comments, 85.1% (n = 550) were constructive and 98.0% (n = 633) were professional. Students in the untrained/signed group were associated with decreased constructiveness (78.3%, OR 0.43, 95% CI 0.25-0.76, p = 0.003); overall, training was associated with increased constructiveness (88.3%, OR 1.70, 95% CI 1.09-2.65, p = 0.019). No factors were negatively or positively associated with professionalism when accounting for covariates.

DISCUSSION

Most students provided comments on course and instructor evaluations; most were constructive and professional.

CONCLUSION

Study group assignment did not substantially impact comment frequency or professionalism. Assignment to enhanced training, regardless of signature status, significantly increased constructiveness.

Using Small Case-Based Learning Groups as a Setting for Teaching Medical Students How to Provide and Receive Peer Feedback

As future physicians, nearly all medical students will be required to provide face-to-face feedback. Moreover, receiving high quality feedback from multiple perspectives is particularly valuable during the pre-clerkship training period. To address these needs, we developed a straightforward, easy to implement exercise that affords students the opportunity to practice giving and receiving feedback with peers. We describe how this exercise has been tailored to fit within the case-based learning small groups of our first-year curriculum and how to enhance the activity by weaving the basic principles of quality feedback into preparation sessions. This exercise has been valued greatly by students.

Self-evaluation and peer-feedback of medical students' communication skills using a web-based video annotation system. Exploring content and specificity

OBJECTIVE

Self-evaluation and peer-feedback are important strategies within the reflective practice paradigm for the development and maintenance of professional competencies like medical communication. Characteristics of the self-evaluation and peer-feedback annotations of medical students' video recorded communication skills were analyzed.

METHOD

Twenty-five year 4 medical students recorded history-taking consultations with a simulated patient, uploaded the video to a web-based platform, marked and annotated positive and negative events. Peers reviewed the video and self-evaluations and provided feedback. Analyzed were the number of marked positive and negative annotations and the amount of text entered. Topics and specificity of the annotations were coded and analyzed qualitatively.

RESULTS

Students annotated on average more negative than positive events. Additional peer-feedback was more often positive. Topics most often related to structuring the consultation. Students were most critical about their biomedical topics. Negative annotations were more specific than positive annotations. Self-evaluations were more specific than peer-feedback and both show a significant correlation. Four response patterns were detected that negatively bias specificity assessment ratings.

CONCLUSION

Teaching students to be more specific in their self-evaluations may be effective for receiving more specific peer-feedback.

PRACTICE IMPLICATIONS

Videofragmentrating is a convenient tool to implement reflective practice activities like self-evaluation and peer-feedback to the classroom in the teaching of clinical skills.

Effects of longitudinal small-group learning on delivery and receipt of communication skills feedback

OBJECTIVES

Although feedback is a critical component of learning, recent data suggest that learners may discount feedback they receive. The emotional threat inherent in feedback can contribute to its ineffectiveness, particularly for sensitive topics like communication skills. Longitudinal relationships among peers may increase their sense of safety and soften the perceived threat of feedback to allow students to give, receive and potentially more effectively incorporate feedback. We studied the effects of prior shared learning experiences among medical students in the delivery and receipt of feedback on clinical (communication) skills.

METHODS

During a formative clinical skills examination, we divided Year 3 students at a US medical school into two subgroups comprising, respectively, small-group classmates from a 2-year longitudinal pre-clerkship clinical skills course (with prior peer-learning relationships), and peers with no prior shared small-group coursework. Students in both subgroups observed peers in a simulated clinical case and then provided feedback, which was videotaped, transcribed and coded. Feedback recipients also completed a survey on their perceptions of the feedback.

RESULTS

Students valued the feedback they received and intended to enact it, regardless of whether they had prior peer-learning relationships. Coding of feedback revealed high specificity. Feedback providers who had prior peer-learning relationships with recipients provided more specific corrective feedback on communication skills than those with no such relationships (p = 0.014); there was no significant difference between subgroups in the provision of reinforcing feedback on communication skills.

CONCLUSIONS

Year 3 medical student peers can deliver specific feedback on clinical skills; prior peer-learning relationships in pre-clerkship clinical skills courses enrich the provision of specific corrective feedback about communication skills. Feedback between peers with pre-existing peer-learning relationships represents an additional and potentially underutilised method of helping students improve clinical skills in sensitive realms such as interpersonal communication.

Delivery and use of individualised feedback in large class medical teaching

BACKGROUND

Formative feedback that encourages self-directed learning in large class medical teaching is difficult to deliver. This study describes a new method, blueprinted feedback, and explores learner's responses to assess its appropriate use within medical science teaching.

METHODS

Mapping summative assessment items to their relevant learning objectives creates a blueprint which can be used on completion of the assessment to automatically create a list of objectives ranked by the attainment of the individual student. Two surveys targeted medical students in years 1, 2 and 3. The behaviour-based survey was released online several times, with 215 and 22 responses from year 2, and 187, 180 and 21 responses from year 3. The attitude-based survey was interviewer-administered and released once, with 22 responses from year 2 and 3, and 20 responses from year 1.

RESULTS

88-96% of learners viewed the blueprinted feedback report, whilst 39% used the learning objectives to guide further learning. Females were significantly more likely to revisit learning objectives than males (p = 0.012). The most common reason for not continuing learning was a 'hurdle mentality' of focusing learning elsewhere once a module had been assessed.

CONCLUSIONS

Blueprinted feedback contains the key characteristics required for effective feedback so that with further education and support concerning its use, it could become a highly useful tool for the individual and teacher.

Workplace-based assessment and students' approaches to learning: a qualitative inquiry

BACKGROUND

We have performed this research to assess the effect of work-place based assessment (WBA) practice on medical students' learning approaches.

METHODS

The research was conducted at the King Saud bin Abdulaziz University for Health Sciences, College of Medicine from 1 March to 31 July 2012. We conducted a qualitative, phenomenological research utilizing semi-structured individual interviews with medical students exposed to WBA. The audio-taped interviews were transcribed verbatim, analyzed, and themes were identified. We preformed investigators' triangulation, member checking with clinical supervisors and we triangulated the data with a similar research performed prior to the implementation of WBA.

RESULTS

WBA results in variable learning approaches. Based on several affecting factors; clinical supervisors, faculty-given feedback, and assessment function, students may swing between surface, deep and effort and achievement learning approaches. Students' and supervisors' orientations on the process of WBA, utilization of peer feedback and formative rather than summative assessment facilitate successful implementation of WBA and lead to students' deeper approaches to learning. Interestingly, students and their supervisors have contradicting perceptions to WBA.

CONCLUSION

A change in culture to unify students' and supervisors' perceptions of WBA, more accommodation of formative assessment, and feedback may result in students' deeper approach to learning.

Peer assisted learning in surgical skills laboratory training: a pilot study

BACKGROUND

Peer assisted learning (PAL) has been rarely investigated in surgical skills training.

AIMS

Describe feedback residents give peers on surgical skills with and without guidelines, determine the association between feedback and actual performance, evaluate resident satisfaction with PAL.

METHOD

Participants evaluated peers using a validated knot-tying checklist and provided feedback on suturing without a guideline. Feedback comments were coded by type and an expert scored performance of each participant. Residents completed a satisfaction questionnaire.

RESULTS

Comments were generally specific. Feedback was twice as likely with the use of a guideline. Specific feedback correlated significantly with expert knot-tying score but not suturing score. Most participants felt peer feedback was helpful and were motivated to practice surgical skills after PAL sessions.

CONCLUSIONS

Surgical residents can provide high quality specific feedback to peers on surgical skills using performance guidelines. Further exploration of effective PAL methodology in surgical skills laboratory training is needed.

Self-assessment and students' study strategies in a community of clinical practice: a qualitative study

BACKGROUND

Self-assessment is recognized as a necessary skill for lifelong learning. It is widely reported to offer numerous advantages to the learner. The research evaluated the impact of students' and supervisors' self-assessment and feedback training on students' perceptions and practices of self-assessment. Moreover, it evaluated the effect of self-assessment process on students' study strategies within a community of clinical practice.

METHODS

We conducted a qualitative phenomenological study from May 2008 to December 2009. We held 37 semi-structured individual interviews with three different cohorts of undergraduate medical students until we reached data saturation. The cohorts were exposed to different contexts while experiencing their clinical years' assessment program. In the interviews, students' perceptions and interpretations of 'self-assessment practice' and 'supervisor-provided feedback' within different contexts and the resulting study strategies were explored.

RESULTS

The analysis of interview data with the three cohorts of students yielded three major themes: strategic practice of self-assessment, self-assessment and study strategies, and feedback and study strategies. It appears that self-assessment is not appropriate within a summative context, and its implementation requires cultural preparation. Despite education and orientation on the two major components of the self-assessment process, feedback was more effective in enhancing deeper study strategies.

CONCLUSION

This research suggests that the theoretical advantages linked to the self-assessment process are a result of its feedback component rather than the practice of self-assessment isolated from feedback. Further research exploring the effects of different contextual and personal factors on students' self-assessment is needed.

Skills development using role-play in a first-year pharmacy practice course

OBJECTIVES

To evaluate the usefulness of a role-play model in developing students' patient-care skills in a first-year undergraduate pharmacy practice course.

DESIGN

A role-play model was developed and implemented in workshops across 2 semesters of a year-long course. Students performed different roles, including that of a pharmacist and a patient, and documented case notes in a single interaction.

ASSESSMENT

Student perceptions of the usefulness of the approach in acquiring skills were measured by surveying students during both semesters. All student assessments (N=130 in semester1; N=129 in semester 2) also were analyzed for skills in verbal communication, information gathering, counselling and making recommendations, and accurately documenting information. A majority of students found the approach useful in developing skills. An analysis of student assessments revealed that role-playing was not as effective in building skills related to accurate documentation as it was in other areas of patient care.

CONCLUSIONS

Role play is useful for developing patient-care skills in communication and information gathering but not for documentation of case notes.

Measuring learning gain during a one-day introductory bronchoscopy course

BACKGROUND

Rigorous assessment of medical knowledge and technical skill inspires learning, reinforces confidence, and reassures the public. Identifying curricular effectiveness using objective measures of learning is therefore crucial for competency-oriented program development in a learner-centric educational environment. The aim of this study was to determine whether various measures of learning, including class-average normalized gain, can be used to assess the effectiveness of a one-day introductory bronchoscopy course curriculum.

METHODS

We conducted a quasi-experimental one-group pre-test/post-test study at the University of California, Irvine. The group comprised 24 first-year pulmonary and critical care trainees from eight training institutions in southern California. Class-average normalized gain, single-student normalized gain, absolute gain, and relative gain were used as objective measures of cognitive knowledge and bronchoscopy technical skill learning. A class-average normalized gain of 30% was used to determine curricular effectiveness. Perceived educational value using Likert-scale surveys and post-course questionnaires was determined during and 3 months after course participation.

RESULTS

Mean test scores of cognitive knowledge improved significantly from 48 to 66% (p = 0.043). Absolute gain for the class was 18%, relative gain was 37%, class average normalized gain was 34%, and the average of the single-student normalized gains g(ave) was 29%. Mean test scores of technical skill improved significantly from 43 to 77% (p = 0.017). Absolute gain was 34%, relative gain was 78%, class average normalized gain was 60%, and the average of the single-student normalized gains g(ave) was 59%. Statistically significant improvements in absolute gain were noted in all five elements of technical skill (p < 0.05). Likert-scale surveys, questionnaires, and surveys demonstrated strong perceived educational value.

CONCLUSION

The effectiveness of a one-day introductory bronchoscopy curriculum was demonstrated using a pre-test/post-test model with calculation of normalized gain and related metrics.