Explicit feedback to enhance the effect of an interim assessment: a cross-over study on learning effect and gender difference

Summative and Formative Style Anatomy Practical Examinations: Do They Have Impact on Students' Performance and Drive for Learning?

Anatomical knowledge is commonly assessed by practical examinations that are often administered in summative format. The format of anatomy practical examination was changed at the Lee Kong Chian School of Medicine in Singapore from summative (graded; must pass) to formative (ungraded; no pass/fail) in academic year (AY) 2017-2018. Both assessment formats were undertaken online, but the formative mode used a team-based learning activity comprising individual and team assessments. This gave an unique opportunity to investigate: (1) the impact of two different online assessment formats on student performance in practical examination; (2) the impact of new formative practical examination on students' performance in summative examinations; and (3) students' opinions of these two practical examination formats. The class of 2021 perceptions was obtained as they experienced both formats. A retrospective cohort study was also conducted to analyze the Year 2 students' performance in anatomy practical and year-end summative examinations of cohorts AY 2015-2016, AY 2016-2017 (summative format), and AY 2017-2018 (formative format). There were no significant differences in students' performance between two practical examination formats. The cohort who experienced the formative format, performed significantly better in summative examinations (mean ± SD: 82.32 ± 10.22%) compared with the cohort who experienced the summative format (73.77 ± 11.09%) (P < 0.001). Students highlighted positive features of the formative practical examination, including team reinforcement of learning, instant feedback, and enhanced learning. These findings indicate that students continue to study for anatomy practical examination without the need for external drivers. The team-based learning style practical examination enhances students' performance in summative examinations.

The impact of feedback during formative testing on study behaviour and performance of (bio)medical students: a randomised controlled study

BACKGROUND

A potential concern of formative testing using web-based applications ("apps") is provision of limited feedback. Adopting a randomised controlled trial in 463 first year (bio) medical students, we explored if providing immediate, detailed feedback during "app"-based formative testing can further improve study behaviour and study performance of (bio)medical students.

METHODS

Students had access to a formative testing "app", which involved 7 formative test modules throughout the 4-week course. In a randomised order, subjects received the "app" with (n = 231, intervention) or without (n = 232, control) detailed feedback during the formative test modules.

RESULTS

No differences in app-use was found between groups (P = 0.15), whereas the intervention group more frequently reviewed information compared to controls (P = 0.007). Exam scores differed between non-/moderate-/intensive- users of the "app" (P < 0.001). No differences in exam scores were found between intervention (6.6 ± 1.1) versus control (6.6 ± 1.1, P = 0.18). Time spent studying was significantly higher compared to previous courses in moderate- and intensive-users (P = 0.006 and < 0.001, respectively), but not in non-users (P = 0.55). Time spent studying did not differ between groups (P > 0.05).

CONCLUSIONS

Providing detailed feedback did not further enhance the effect of a web-based application of formative testing on study behaviour or study performance in (bio)medical students, possibly because of a ceiling-effect.

Does quantity ensure quality? Standardized OSCE-stations for outcome-oriented evaluation of practical skills at different medical faculties

BACKGROUND

Practical skills are often assessed using Objective Structured Clinical Skill Exams (OSCE). Nevertheless, in Germany, interchange and agreement between different medical faculties or a general agreement on the minimum standard for passing is lacking.

METHODS

We developed standardized OSCE-stations for assessing structured clinical examination of knee and shoulder joint with identical checklists and evaluation standards. These were implemented into the OSCE-course at five different medical faculties. Learning objectives for passing the stations were agreed beforehand. At each faculty, one reference examiner scored independently of the local examiner. Outcome of the students at the standardized station was compared between faculties and correlated to their total outcome at the OSCE, to their results at the Part One of the National Medical Licensing Examination as a reference test during medical studies and to their previous amount of lessons in examining joints.

RESULTS

Comparing the results of the reference examiner, outcome at the station differed significantly between some of the participating medical faculties. Depending on the faculty, mean total results at the knee-examination-station differed from 64.4% to 77.9% and at the shoulder-examination-station from 62.6% to 79.2%. Differences were seen in knowledge-based items and also in competencies like communication and professional manner. There was a weak correlation between outcome at the joint-examination-OSCE-station and Part One of the National Medical Licensing Examination, and a modest correlation between outcome at the joint-examination-station and total OSCE-result. Correlation to the previous amount of lessons in examining joint was also weak.

CONCLUSION

Although addressing approved learning objectives, different outcomes were achieved when testing a clinical skill at different medical faculties with a standardized OSCE-station. Results can be used as a tool for evaluating lessons, training and curricula at the different sites. Nevertheless, this study shows the importance of information exchange and agreement upon certain benchmarks and evaluation standards when assessing practical skills.

Impact of structured verbal feedback module in medical education: A questionnaire- and test score-based analysis

Introduction:

Feedback is a divalent bond between the supplier (teacher) and the recipient (student). The strength of the bond depends on the instructional design of the feedback. Feedback is central to medical education in promoting self-directed learning in students. In the present study, a structured verbal feedback module was prepared, implemented, and evaluated.

Methods:

The study was done on 280 students from four consecutive batches (2011 to 2014) of the 1^st year MBBS students exposed to different types and modes of feedback. Analysis was done using student feedback questionnaire for the perception of students to verbal feedback. Quantitative analysis using post hoc test and ANOVA for the impact of type of feedback (verbal or written) and effect of modes (individual or group) of verbal feedback on test score performance were done.

Result:

In this study, ≥95% of the students preferred verbal feedback of both positive and negative attributes in student questionnaires. It was observed that verbal feedback sessions made a difference of up to 2–2.4 grade points in the mean score of batch when compared to the written feedback. The initial mean test score (T1) of 2011 + 2012 and 2013 + 2014 was not statistically significant (P = 0.113). But, in all subsequent tests (T2, T3, and T4), there was a statistically significant difference in the mean test scores (P = 0.000).

Conclusion:

(1) Students prefer verbal one-to-one feedback over written feedback. (2) Verbal feedback changes learning process and causes sustained improvement in learning strategies.

Students' perceptions on feedback module in pharmacology

CONTEXT

Feedback is an integral part of formative assessment though underutilized in medical education. The objective of this study was to review our feedback module through students' perceptions.

METHODOLOGY

We have developed a feedback module which is practiced by us for last 10 years for term ending examination that gives collective feedback to the whole class, followed by individual student-teacher interactions. Students were also exposed to 6-7 multiple choice questions (MCQs) based assessment during the course of pharmacology. Immediately after each MCQ test the answer keys is displayed along with an explanation. Two classes of students were requested to give their perceptions about the feedback by responding on Likert scale for the statements in the questionnaire. All the 206 students who volunteered for the study were enrolled in the study. Mann-Whitney test was used to calculate the difference in perceptions.

RESULTS

Of 278 students of two classes, 206 responded (74%). Students' agreement varied from 93% to 98% for 5 items in the questionnaire for the feedback after term ending examinations. Perception of students attending one or more than one feedback session did not differ significantly. For MCQs, tests agreement was 91% to 98% for the 4 items. There was no significant difference between two classes in their perceptions regarding feedback practices (P < 0.05).

CONCLUSION

Students gave a favorable opinion for our feedback module. In the medical colleges with a large number of students, this module is feasible for feedback in formative assessment in the form of written tests.

Immediate detailed feedback to test-enhanced learning: an effective online educational tool

BACKGROUND

Test-enhanced learning has gained popularity because it is an effective way to increase retention of knowledge; provided the student receives the correct answer soon after the test is taken.

AIM

To determine whether detailed feedback provided to test-enhanced learning questions is an effective online educational tool for improving performance on complex biomedical information exams.

METHODS

A series of online multiple choice tests were developed to test knowledge of biomedical information that students were expected to know after each patient-case. Following submission of the student answers, one cohort (n = 52) received answers only while the following year, a second cohort (n = 51) received the answers with detailed feedback explaining why each answer was correct or incorrect.

RESULTS

Students in both groups progressed through the series of online tests with little assessor intervention. Students receiving the answers along with the explanations within their feedback performed significantly better in the final biomedical information exam than those students receiving correct answers only.

CONCLUSIONS

This pilot study found that the detailed feedback to test-enhanced learning questions is an important online learning tool. The increase in student performance in the complex biomedical information exam in this study suggests that detailed feedback should be investigated not only for increasing knowledge, but also be investigated for its effect on retention and application of knowledge.

On doctor-patient relationship and feedback interventions

None