Case-based discussion: assessment tool or teaching aid?

Factors that may improve paediatric workplace-based assessments: an exploratory study

OBJECTIVES

To establish if paediatric trainees are satisfied with the current workplace-based assessment (WBA) process. To identify factors that contribute both positively and negatively to the educational experience during WBAs. To find out if trainees and their supervisors experience any challenges conducting WBAs. To establish potential ways to improve future assessments.

DESIGN

Qualitative semistructured interviews.

SETTING

Participants included fifteen trainees (ST1-8) in general paediatric and subspecialty posts and four consultants or equivalent across five hospital sites in the Thames Valley Deanery. All participants had regular exposure to WBAs.

INTERVENTIONS

Interviews were undertaken between June 2020 and January 2021 via video link. Data collection and analysis were conducted iteratively using constant comparison until theoretical sufficiency was achieved.

MAIN OUTCOME MEASURE

Using Constructivist Grounded Theory, a theoretical framework, grounded in the data, was developed that depicted the core elements that should be present to optimise WBAs.

RESULTS

A number of key components were reported to affect the educational value of WBAs. A positive departmental culture towards education and training is essential. Chosen cases should be challenging, and direct observation or in-depth discussion, depending on the assessment type, is fundamental. Timely constructive feedback and immediate completion of the assessment form are also imperative.

CONCLUSION

Some trainees experienced WBAs where these key components aligned, but many did not, and this negatively affected their learning. Three main challenges or future targets for further improvements include increasing time, improving training and optimising technology.

A Study to Evaluate Students’ Performance, Engagement, and Progression in Higher Education Based on Feedforward Teaching Approach

Learners who enter higher education (HE) at the foundational level are susceptible to many challenges that impact their performance, engagement, and progression. Not all students who enter HE at the foundational level will progress and attain their course qualifications. In addition, many university lecturers struggle to give effective support to their students. This study focuses on feedforward teaching approaches that define ways to enhance learning by using advanced organisational strategies to offer relevant supporting concepts and meaningful verbal material. To date, there are insufficient literature reviews on feedforward approaches to facilitate students’ subsequent learning. Providing better academic support for students and a strong foundation for independent learning is the focus of this paper. Therefore, the main contributions of this paper are identifying the key feedforward features and suggesting effective feedforward approaches. This study was undertaken to rigorously implement feedforward approaches that would support groups of students in modules at the foundational entry-level. At the end of module delivery, different students’ data sets were analysed related to the progression rates, standard deviation, and mean. In addition, the student satisfaction questionnaire (module evaluation survey) and feedback survey were also considered for engagement and retention purposes. The outcomes from this exercise suggest that feedforward approaches allow students to increase their overall effort when attempting summative assessments and, thus, improve their performance, engagement, and retention.

Can case-based discussions in a group setting be used to assess residents' clinical skills?

OBJECTIVES

The purpose of this study was to explore residents' and assessors' perception of a new group assessment concept.

METHODS

This qualitative study consists of observations of four group assessment sessions, followed by semi-structured interviews with six residents and four assessors (specialists in internal medicine), who all volunteered to be interviewed. All residents at a medical department (eleven to fifteen each time) and four assessors participated in four group assessments, where the residents' clinical skills were assessed through case-based discussions. An external consultant (an anthropologist) performed the observations and the interviews. Notes from the observations and the interviews were analyzed using an inductive approach.

RESULTS

Eight of the ten interviewed participants preferred group assessment to individual assessment. Results from the interviews suggested that the group assessments were more consistent and that the level of discussion was perceived to be higher in the group discussions compared to the one-to-one discussions. All residents indicated that they had acquired new knowledge during their assessment and reported having learned from listening to the assessment of their peers. Assessors similarly reported gaining new knowledge.

CONCLUSIONS

The residents and assessors expressed very favourable attitudes toward the new group assessment concept. The assessment process was perceived to be higher in quality and more consistent, contributing to learning for all participating doctors in the department. Group assessment is feasible and acceptable, and provides a promising tool for assessment of clinical skills in the future.

Comparison of Active Learning Techniques: Audience Response Questions Versus Small Group Discussion on Immediate- and Long-term Knowledge Gain

OBJECTIVES

Active learning techniques help with motivation, involvement, and retention during didactics. There are few studies comparing different active learning methods, and these have yielded mixed results. The objective of this study was to compare the effect of two active learning methods-small-group discussion and audience response system (ARS)-on immediate- and long-term knowledge gain.

METHODS

This was a prospective experimental study of emergency medicine (EM) subinterns and residents. Participants were randomized into two groups, and baseline knowledge was assessed with a multiple-choice pretest. Didactic sessions on salicylate toxicity and ocular trauma were given to both groups utilizing either small-group discussion or ARS. A crossover design was utilized to ensure that both groups received instruction by each method. A multiple-choice posttest was administered following the didactics and again 2 months later. Pre- and posttests were identical. All test items were written by an academic faculty member with advanced training in medical education and item writing and were based on the goals and objectives of the session. Test items were piloted with a reference group of learners. Didactic instructors were blinded to test items. Data were analyzed using a linear mixed-effects model.

RESULTS

Thirty-eight subinterns and residents participated in the study. Both instructional methods showed immediate- and long-term knowledge gain. The linear mixed-effects model did not demonstrate any significant difference between instructional methods on immediate knowledge gain (mean difference = 0.18, p = 0.62, 95% confidence interval [CI] = -0.52 to 0.88) or long-term knowledge gain (mean difference = -0.42, p = 0.36, 95% CI = -1.32 to 0.47).

CONCLUSION

In this small study, there was no significant difference between instructional methods on immediate- and long-term knowledge gain in EM subinterns and residents.

Workplace-based assessment: how to use case-based discussion as a formative assessment

Workplace-based assessments are increasingly used as a way of gaining insight into clinician performance in real-life situations. Although some can be used to inform a summative (pass/fail) assessment, many have a much greater role in the formative assessment of trainees, and can be used as tools for teaching and training and in identifying the development needs of trainees. There is considerable variation between different European countries in the use of formative, workplace-based assessment, such as a structured case-based discussion (CbD), during training. This article gives an overview of how to use CbD as a formative assessment for higher specialist trainees, and gives access to a downloadable record form which can be used by trainers.

Case-based discussion is a structured method of formative assessment which has been found to be valuable by both trainees and supervisors. This article describes the process, and offers a proforma for supervisors who have no access to this facility.http://bit.ly/2HYkOVJ

Use of work-based assessments in postgraduate medical education

Work-based assessments are ubiquitous in postgraduate medical training in the UK. This article discusses the variety of these assessments and explores barriers to their use and solutions for improving the educational value of these tools for adult learners. The focus should be on feedback and learning rather than assessment, and this may promote discussion of more challenging scenarios where the opportunity for learning is greater. Mobile devices may help reduce the administrative, geographical and time constraints of completing work-based assessments.

The effect of case-based discussion of topics with experts on learners' opinions: implications for spinal education and training

PURPOSE

This investigation aimed to examine the extent to which case-based discussion with experts could influence the audience's opinions on the treatment of patients during a continuing medical education event for spine surgeons.

METHODS

We conducted a prospective controlled crossover study of 90 surgeons. During a continuing medical education activity using case-based discussion, quiz questions were used which asked participants (attendants and faculty group) their opinions on the best choices about diagnosis and treatment in a number of cases. No answer was considered correct, but we evaluated the number of participants choosing each specific answer among a number of valid options. Quiz questions were collected with an automated response system at the entry and at the end of each case discussion. Change in participant's opinions was estimated from the change in the preferred answers between the entry and exit quizzes. Chi-square analysis was performed to determine significance.

RESULTS

Sixty-two attendants out of eighty three (75%) and six faculties out of twelve (50%) responded to the survey. After the case discussion, 68.2% (p < 0.04, Chi-square test) of the attendants changed their opinion on the appropriate treatment. The faculty answers, however, showed no significant change in opinions regarding the identification of the appropriate treatment.

CONCLUSIONS

On the basis of our results, case-based discussion driven by experts, as a form of teaching, has a measurable effect in terms of changes in the learners' opinions.

Chart-stimulated Recall as a Learning Tool for Improving Radiology Residents' Reports

RATIONALE AND OBJECTIVES

Workplace-based assessments gauge the highest tier of clinical competence. Chart-stimulated recall (CSR) is a workplace-based assessment method that complements chart audit with an interview based on the residents' notes. It allows evaluation of the residents' knowledge and heuristics while providing opportunities for feedback and self-reflection. We evaluated the utility of CSR for improving the radiology residents' reporting skills.

MATERIALS AND METHODS

Residents in each year of training were randomly assigned to an intervention group (n = 12) or a control group (n = 13). Five pre-intervention and five post-intervention reports of each resident were independently evaluated by three blinded reviewers using a modified Bristol Radiology Report Assessment Tool. The study intervention comprised a CSR interview tailored to each individual resident's learning needs based on the pre-intervention assessment. The CSR process focused on the clinical relevance of the radiology reports. Student's t test (P < .05) was used to compare pre- and post-intervention scores of each group.

RESULTS

A total of 125 pre-intervention and 125 post-intervention reports were evaluated (total 750 assessments). The Cronbach's alpha for the study tool was 0.865. A significant improvement was seen in the cumulative 19-item score (66% versus 73%, P < .001) and the global rating score (59% versus 72%, P < .001) of the intervention group after the CSR. The reports of the control group did not demonstrate any significant improvement.

CONCLUSION

CSR is a feasible workplace-based assessment method for improving reporting skills of the radiology residents.

The effect of case-based discussion of topics with experts on learners' opinions: implications for spinal education and training

PURPOSE

This investigation aimed to examine the extent to which case-based discussion with experts could influence the audience's opinions on the treatment of patients during a continuing medical education event for spine surgeons.

METHODS

We conducted a prospective controlled crossover study of 90 surgeons. During a continuing medical education activity using case-based discussion, quiz questions were used which asked participants (attendants and faculty group) their opinions on the best choices about diagnosis and treatment in a number of cases. No answer was considered correct, but we evaluated the number of participants choosing each specific answer among a number of valid options. Quiz questions were collected with an automated response system at the entry and at the end of each case discussion. Change in participant's opinions was estimated from the change in the preferred answers between the entry and exit quizzes. Chi-square analysis was performed to determine significance.

RESULTS

Sixty-two attendants out of eighty three (75%) and six faculties out of twelve (50%) responded to the survey. After the case discussion, 68.2% (p < 0.04, Chi-square test) of the attendants changed their opinion on the appropriate treatment. The faculty answers, however, showed no significant change in opinions regarding the identification of the appropriate treatment.

CONCLUSIONS

On the basis of our results, case-based discussion driven by experts, as a form of teaching, has a measurable effect in terms of changes in the learners' opinions.

Workplace-based assessment: a review of user perceptions and strategies to address the identified shortcomings

Workplace based assessments (WBAs) are now commonplace in postgraduate medical training. User acceptability and engagement is essential to the success of any medical education innovation. To this end, possessing an insight into trainee and trainer perceptions towards WBAs will help identify the major problems, permitting strategies to be introduced to improve WBA implementation. A review of literature was performed to identify studies examining trainee and trainer perceptions towards WBAs. Studies were excluded if non-English or sampling a non-medical/dental population. The identified literature was synthesised for the purpose of this critical narrative review. It is clear that there is widespread negativity towards WBAs in the workplace. This has negatively impacted on the effectiveness of WBA tools as learning aids. This negativity exists in trainees but also to an extent in their trainers. Insight gained from the literature reveals three dominant problems with WBA implementation: poor understanding as to the purpose of WBAs; insufficient time available for undertaking these assessments; and inadequate training of trainers. Approaches to addressing these three problems with WBA implementation are discussed. It is likely that a variety of solutions will be required. The prevalence of negativity towards WBAs is substantial in both trainees and trainers, eroding the effectiveness of learning that is consequent upon them. The educational community must now listen to the concerns being raised by the users and consider the range of strategies being proposed to improve the experiences of trainees, and their trainers.

Toward competency-based curriculum: Application of workplace-based assessment tools in the National Saudi Arabian Anesthesia Training Program

Background:

The anesthesia training program of the Saudi Commission for health specialties has introduced a developed competency-based anesthesia residency program starting from 2015 with the utilization of the workplace-based assessment (WBA) tools, namely mini-clinical exercises (mini-CEX), direct observation of procedural skills (DOPS), and case-based discussion (CBD).

Objectives:

This work aimed to describe the process of development of anesthesia-specific list of mini-CEX, DOPS, and CBD tools within the Saudi Arabian Anesthesia Training Programs.

Materials and Methods:

To introduce the main concepts of formative WBA tools and to develop anesthesia-specific applications for each of the selected WBA tools, four 1-day workshops were held at the level of major training committees at eastern (Dammam), western (Jeddah), and central (Riyadh) regions in the Kingdom were conducted. Sixty-seven faculties participated in these workshops.

Results:

After conduction of the four workshops, the anesthesia-specific applications setting of mini-CEX, DOPS, and CBD tools among the 5-year levels were fully described. The level of the appropriate consultation skills was divided according to the case complexity adopted from the American Society of Anesthesiologists physical classification for adult and obstetric and pediatric patient as well as the type of the targeted anesthetic procedure.

Conclusion:

WBA anesthesia-specific lists of mini-CEX, DOPS, and CBD forms were easily incorporated first into guidelines to help the first stage of implementation of formative assessment in the Saudi Arabian Anesthesia Residency Program, and this can be helpful to replicate such program within other various training programs in Saudi Arabia and abroad.

Using patients' charts to assess medical trainees in the workplace: a systematic review

OBJECTIVES

The objective of this review is to summarize and critically appraise existing evidence on the use of chart stimulated recall (CSR) and case-based discussion (CBD) as an assessment tool for medical trainees.

METHODS

Medline, Embase, CINAHL, PsycINFO, Educational Resources Information Centre (ERIC), Web of Science, and the Cochrane Central Register of Controlled Trials were searched for original articles on the use of CSR or CBD as an assessment method for trainees in all medical specialties.

RESULTS

Four qualitative and three observational non-comparative studies were eligible for this review. The number of patient-chart encounters needed to achieve sufficient reliability varied across studies. None of the included studies evaluated the content validity of the tool. Both trainees and assessors expressed high level of satisfaction with the tool; however, inadequate training, different interpretation of the scoring scales and skills needed to give feedback were addressed as limitations for conducting the assessment.

CONCLUSION

There is still no compelling evidence for the use of patient's chart to evaluate medical trainees in the workplace. A body of evidence that is valid, reliable, and documents the educational effect in support of the use of patients' charts to assess medical trainees is needed.

Otolaryngology WBAs in the Wales Deanery: the first six years

‘The provision of excellent care for the surgical patient, delivered safely, is at the heart of the curriculum.’ 1 Workplace-based assessments (WBAs) are an integral part of the assessment component of the UK’s intercollegiate surgical curriculum. The curriculum is web-based and accessed through www.iscp.ac.uk .