Implementation of workplace-based assessment in general practice

Assessing Residents in the Department of Surgery at a Tertiary Care Centre Using Mini-Clinical Evaluation Exercise (Mini-CEX)

OBJECTIVE

This study aimed to introduce, sensitize, and train our postgraduate students and faculty of the department of general surgery with the use of mini-Clinical Evaluation Exercise (mini‑CEX) and to assess the perception of students and faculty towards it.

MATERIAL AND METHODS

A cross‑sectional observational study was conducted over a period of four months. Ten surgery residents in the department were asked to volunteer to participate and five professors conducted the session. Five sessions of mini‑CEX (nine points) were conducted for each resident in different settings of the out‑patient department (OPD) and in‑patient department (IPD). A total of five skills were tested. Feedback from faculty and residents regarding the perception of mini‑CEX was also taken.

RESULTS

A statistically significant difference in mean scores of all domains was observed comparing the first and last assessment (p<0.05). Hundred percent of the residents scored superior category (7-9) in the final assessment in all domains, whereas the maximum was in a satisfactory scoring grade in 1st assessment. The time taken for the assessment significantly reduced from 1st assessment to the last assessment in OPD and IPD settings (p=0.001). The mini-CEX assessment tool got 100% feedback from faculty in terms of skill improvement, method, attitude of residents, and ability to identify gaps in knowledge. However, one assessor thought that "time given for assessment" was inadequate and more effort was required than the usual traditional assessment methods. The most identified problem faced by residents was that the "time given during assessment" was less (50%); however, overall residents also found it valid, effective, and helpful in identifying knowledge gaps and improving clinical and communication skills.

CONCLUSION

Mini‑CEX improves the learning environment in residency and also leads to improvement in medical interviewing skills, physical examination skills, humanistic qualities/professionalism, and counseling skills. So, it can be used for residency training in clinical departments.

Family Medicine Resident Experience Toward Workplace-Based Assessment Form in Improving Clinical Teaching: An Exploratory Qualitative Study

Background

Workplace-Based Assessment (WPBA) has been widely utilized for assessing performance in training sites for both formative and summative purposes. Currently, with the recently updated duration of the family medicine (FM) training program in Saudi Arabia from four years to three years, the possible impact of such a change on assessment would need to be investigated. This objective was to explore the experiences of FM residents regarding the usage of WPBA as an assessment tool for improving clinical teaching at King Abdulaziz Hospital (KAH), Al Ahsa, Saudi Arabia.

Methods

The study involves an exploratory qualitative phenomenological approach targeting family medicine resident in KAH was used. Purposive sampling techniques were used. In this descriptive study, data was collected through the utilization of 1:1 semi-structured interviews guided by directive prompts. All recorded interviews were transcribed verbatim. An inductive analytical approach was applied for thematic analysis of transcripts.

Results

Fifteen participants were individually interviewed until data saturation was reached. The themes that emerged were organized into the categories of underlying principles of WPBA, the impact of the learning environment, associated opportunities and challenges, and making WPBA more effective. Participants expressed that the orientation provided by the program was insufficient, although the core principles were clear to them. They valued the senior peers' support and encouragement for the creation of a positive learning environment. However, time limit, workload, and a lack of optimum ideal implementation reduced the educational value and effectiveness of WPBA among senior residents.

Conclusion

The study examined residents' experiences with WPBA and concluded that low levels of satisfaction were attributed to implementation-related problems. Improvements should be made primarily in two areas: better use of available resources and more systematic prior planning. Revision and assignment of the selection process were suggested, in addition to the implementation of the new curriculum. The research will assist stakeholders in selecting and carrying out evaluation techniques that will enhance residents' abilities.

Effect of Workplace-Based Assessment Utilization as a Formative Assessment for Learning Among Family Medicine Postgraduates at the Faculty of Medicine, Menoufia University: A Prospective Study

Background Workplace-based assessment (WBA) is a group of assessment approaches that assesses the trainees' performance through their observation and monitoring in real clinical settings and then provides them with constructive and relevant feedback. Many WBA tools are available, including the mini-clinical evaluation exercise (mini-CEX), direct observation of procedural skills (DOPS), case-based discussions, and multisource feedback (peers, seniors, and patients). A WBA can help medical students improve their clinical competencies and ensure that qualified physicians graduate. Methods This prospective study was done in the family medicine department at the Menoufia Faculty of Medicine in Egypt and passed through two phases. Phase I was introducing an orientation lecture for family medicine staff and a convenient sample of 21 family medicine postgraduates about WBA. Phase II was conducting a monthly mini-CEX and DOPS for the postgraduates. Finally, students' satisfaction with the WBA was assessed, and all collected data were analyzed via Statistical Package for Social Science (SPSS) version 23 (IBM Corp., Armonk, NY). Results A total of 105 feedback sheets were obtained. These feedback sheets were subdivided into 63 mini-CEX feedback sheets (21 sheets from each mini-CEX session for three sessions) and 42 DOPS feedback sheets (21 sheets from each DOPS session for two sessions), all of which were collected and analyzed. A significant improvement was detected in the mini-CEX and DOPS feedback scores of the postgraduates throughout the consecutive sessions (9.5 ± 2.7, 24.9 ± 2.5, 27.29 ± 1.5) (P < 0.001) for Mini-CEX and (6.1 ± 1.8 versus 9.0 ± 1.2) (P < 0.001) for DOPS. About 93% of the postgraduates recommended the application of WBA for their peers, and 86% of them requested to perform it again for other different clinical cases and procedures. Conclusion Workplace-based assessment in the form of Mini-CEX and DOPS revealed its ability to improve clinical knowledge and skills among family medicine postgraduates who became motivated to undergo it again in search of improving their clinical performance and reducing their stresses related to final summative and objective structured clinical examinations (OSCEs).

Implementation and Use of Workplace-Based Assessment in Clinical Learning Environments: A Scoping Review

PURPOSE

Workplace-based assessment (WBA) serves a critical role in supporting competency-based medical education (CBME) by providing assessment data to inform competency decisions and support learning. Many WBA systems have been developed, but little is known about how to effectively implement WBA. Filling this gap is important for creating suitable and beneficial assessment processes that support large-scale use of CBME. As a step toward filling this gap, the authors describe what is known about WBA implementation and use to identify knowledge gaps and future directions.

METHOD

The authors used Arksey and O'Malley's 6-stage scoping review framework to conduct the review, including: (1) identifying the research question; (2) identifying relevant studies; (3) study selection; (4) charting the data; (5) collating, summarizing, and reporting the results; and (6) consulting with relevant stakeholders.

RESULTS

In 2019-2020, the authors searched and screened 726 papers for eligibility using defined inclusion and exclusion criteria. One hundred sixty-three met inclusion criteria. The authors identified 5 themes in their analysis: (1) Many WBA tools and programs have been implemented, and barriers are common across fields and specialties; (2) Theoretical perspectives emphasize the need for data-driven implementation strategies; (3) User perceptions of WBA vary and are often dependent on implementation factors; (4) Technology solutions could provide useful tools to support WBA; and (5) Many areas of future research and innovation remain.

CONCLUSIONS

Knowledge of WBA as an implemented practice to support CBME remains constrained. To remove these constraints, future research should aim to generate generalizable knowledge on WBA implementation and use, address implementation factors, and investigate remaining knowledge gaps.

Examining the educational impact of the mini-CEX: a randomised controlled study

BACKGROUND

The purpose of this study is to evaluate the mini-Clinical Evaluation Exercise (mini-CEX) as a formative assessment tool among undergraduate medical students, in terms of student perceptions, effects on direct observation and feedback, and educational impact.

METHODS

Cluster randomised study of 38 fifth-year medical students during a 16-week clinical placement. Hospitals were randomised to provide a minimum of 8 mini-CEXs per student (intervention arm) or continue with ad-hoc feedback (control arm). After finishing their clinical placement, students completed an Objective Structured Clinical Examination (OSCE), a written test and a survey.

RESULTS

All participants in the intervention group completed the pre-planned number of assessments, and 60% found them to be useful during their clinical placement. Overall, there were no statistically significant differences between groups in reported quantity or quality of direct observation and feedback. Observed mean scores were marginally higher on the OSCE and written test in the intervention group, but not statistically significant.

CONCLUSIONS

There is considerable potential in assessing medical students during clinical placements and routine practice, but the educational impact of formative assessments remains mostly unknown. This study contributes with a robust study design, and may serve as a basis for future research.

How to enhance and assess reflection in specialist training: a mixed method validation study of a new tool for global assessment of reflection ability

BACKGROUND

In Danish GP training we had the ambition to enhance and assess global reflection ability, but since we found no appropriate validated method in the literature, we decided to develop a new assessment tool. This tool is based on individual trainee developed mind maps and structured trainer-trainee discussions related to specific complex competencies. We named the tool Global Assessment of Reflection ability (GAR) and conducted a mixed method validation study. Our goal was to investigate whether it is possible to enhance and assess reflection ability using the tool.

METHODS

In order to investigate acceptability, feasibility, face validity, and construct validity of the tool we conducted a mixed method validation study that combined 1) qualitative data obtained from 750 GP trainers participating in train-the-trainer courses, 2) a questionnaire survey sent to 349 GP trainers and 214 GP trainees and 3) a thorough analysis of eight trainer-trainee discussions.

RESULTS

Our study showed an immediate high acceptance of the GAR tool. Both trainers and trainees found the tool feasible, useful, and relevant with acceptable face validity. Rating of eight audio recordings showed that the tool can demonstrate reflection during assessment of complex competencies.

CONCLUSIONS

We have developed an assessment tool (GAR) to enhance and assess reflection. GAR was found to be acceptable, feasible, relevant and with good face- and construct validity. GAR seems to be able to enhance the trainees' ability to reflect and provide a good basis for assessment in relation to complex competencies.

Workplace based assessment: A review of available tools and their relevance

Workplace-based assessment (WPBA) appears to be a promising tool for more comprehensive assessment of the learners. Relevant literature in this direction was collated and analyzed for its relevance, salience, and merit. Many WPBA tools are being used by various educational institutions which spans across multiple domains and over the entire duration of the workplace learning. It caters to holistic assessment with structured measures, real-time feedback, and continuous professional development. It is being used more for formative assessment and has limited utility in summative assessment as on date. WPBA tools have promising prospect in bringing novelty, objectivity, and holistic approach in assessment.