Evaluating the impact of an integrated multidisciplinary head & neck competency-based anatomy & radiology teaching approach in radiation oncology: a prospective cohort study

Evaluating competency-based medical education: a systematized review of current practices

BACKGROUND

Few published articles provide a comprehensive overview of the available evidence on the topic of evaluating competency-based medical education (CBME) curricula. The purpose of this review is therefore to synthesize the available evidence on the evaluation practices for competency-based curricula employed in schools and programs for undergraduate and postgraduate health professionals.

METHOD

This systematized review was conducted following the systematic reviews approach with minor modifications to synthesize the findings of published studies that examined the evaluation of CBME undergraduate and postgraduate programs for health professionals.

RESULTS

Thirty-eight articles met the inclusion criteria and reported evaluation practices in CBME curricula from various countries and regions worldwide, such as Canada, China, Turkey, and West Africa. 57% of the evaluated programs were at the postgraduate level, and 71% were in the field of medicine. The results revealed variation in reporting evaluation practices, with numerous studies failing to clarify evaluations' objectives, approaches, tools, and standards as well as how evaluations were reported and communicated. It was noted that questionnaires were the primary tool employed for evaluating programs, often combined with interviews or focus groups. Furthermore, the utilized evaluation standards considered the well-known competencies framework, specialized association guidelines, and accreditation criteria.

CONCLUSION

This review calls attention to the importance of ensuring that reports of evaluation experiences include certain essential elements of evaluation to better inform theory and practice.

Successful implementation of online educational lectures of the German Society for Radiation Oncology (DEGRO)

PURPOSE

Modern digital teaching formats have become increasingly important in recent years, in part due to the COVID-19 pandemic. In January 2021, an online-based webinar series was established by the German Society for Radiation Oncology (DEGRO) and the young DEGRO (yDEGRO) working group. In the monthly 120-minute courses, selected lecturers teach curricular content as preparation for the board certification exam for radiation oncology.

METHODS

The evaluation of the 24 courses between 01.2021 and 12.2022 was performed using a standardized questionnaire with 21 items (recording epidemiological characteristics of the participants, didactic quality, content quality). A Likert scale (1-4) was used in combination with binary and open-ended questions.

RESULTS

A combined total of 4200 individuals (1952 in 2021 and 2248 in 2022) registered for the courses, and out of those, 934 participants (455 in 2021 and 479 in 2022) later provided evaluations for the respective courses (36% residents, 35% specialists, 21% medical technicians for radiology [MTR], 8% medical physics experts [MPE]). After 2 years, 74% of the DEGRO Academy curriculum topics were covered by the monthly webinars. The overall rating by participants was positive (mean 2021: 1.33 and 2022: 1.25) and exceeded the curriculum offered at each site for 70% of participants. Case-based learning was identified as a particularly well-rated method.

CONCLUSION

The DEGRO webinar expands the digital teaching opportunities in radiation oncology. The consistently high number of participants confirms the need for high-quality teaching and underlines the advantages of e‑learning methods. Optimization opportunities were identified through reevaluation of feedback from course participants. In its design as a teaching format for a multiprofessional audience, the webinar series could be used as a practice model of online teaching for other disciplines.

Comparison of case-based learning using Watson for oncology and traditional method in teaching undergraduate medical students

BACKGROUND

Watson for Oncology (WFO) is a decision-making system generated by artificial intelligence (AI) and has been widely used in treatment recommendations of cancer patients. However, the application of WFO in clinical teaching among medical students has not been reported.

OBJECTIVE

To establish a novel teaching and learning method with WFO in undergraduate medical students and evaluate its efficiency and students' satisfaction compared with traditional case-based learning model.

METHODS

72 undergraduates majoring in clinical medicine in Wuhan University were enrolled and were randomly divided into the WFO-based group and the control group. 36 students in the WFO-based group learned clinical oncology cases via WFO platform while 36 students in the control group using traditional teaching methods. After the course, final examination and questionnaire survey of teaching assessment were conducted on the two groups of students.

RESULTS

According to the questionnaire survey of teaching assessment, WFO-based group showed significant higher score in the aspect of cultivating ability of independent learning (17.67 ± 1.39 vs. 15.17 ± 2.02, P = 0.018), increasing knowledge mastery (17.75 ± 1.10 vs. 16.25 ± 1.18, P = 0.001), enhancing learning interest (18.41 ± 1.42 vs. 17.00 ± 1.37, P = 0.002), increasing course participation (18.33 ± 1.67 vs. 15.75 ± 1.67, P = 0.001) and the overall course satisfaction (89.25 ± 5.92 vs. 80.75 ± 3.42, P = 0.001) than those of the control group students.

CONCLUSION

Our practice has established a novel clinical case-based teaching pattern with WFO, providing undergraduate students with convenient and scientific training and guidance. It empowers students with improved learning experiences and equips them with essential tools for clinical practices.

Medical students' perception and academic performance after team-based and seminar-based learning in human anatomy

Objectives

Student-centered educational strategies like problem-based learning (PBL), case-based learning (CBL), team-based learning (TBL), and seminars enhance group and self-learning. This study was carried out to evaluate students' achievements in anatomy topics delivered through TBL sessions and seminars and to survey student preferences regarding these two modalities in anatomy learning.

Methods

TBL was conducted through individual readiness assurance tests (IRATs), group readiness assurance tests (GRATs), mini-lectures, and application exercises. Seminars included pretests, peer lecturing, and posttests. The performance of 117 students in three TBL sessions and three seminars was compared after standardizing the questions. The students were second-year (42), third-year (40), and fourth-year (35) students at the College of Medicine, University of Bisha, KSA, during the 2019/2020 academic year.

Results

A gradual increase in the means of TBL grades was noticed among second-, third-, and four-year students (means ± SD: 68.6 ± 9.56, 82.8 ± 12.25, and 92.7 ± 4.70, respectively), but their seminar grades were nearly stationary (means ± SD: 80.0 ± 9.66, 85.11 ± 10.16, and 85.9 ± 8.80, respectively). Cohen's d-test to check the strength of the relationship between the two activities showed 1.03, 0.16, and 0.74 in the same order. We statistically analyzed perception and preference questionnaire results received from 39, 35, and 28 second-, third-, and four-year students, respectively. The majority of the students selected TBL as their preferred learning modality. However, their acceptance of the seminars was very poor

Conclusions

It can be concluded that TBL is more beneficial to the students, even in practical sciences like anatomy, most likely because group peer teaching enhances the sense of collegial competition, as opposed to the self-learning nature of seminars, which might suppress the sense of competition.

Evaluation of the impact of teaching on delineation variation during a virtual stereotactic ablative radiotherapy contouring workshop

Introduction:

Variation in delineation of target volumes/organs at risk (OARs) is well recognised in radiotherapy and may be reduced by several methods including teaching. We evaluated the impact of teaching on contouring variation for thoracic/pelvic stereotactic ablative radiotherapy (SABR) during a virtual contouring workshop.

Materials and methods:

Target volume/OAR contours produced by workshop participants for three cases were evaluated against reference contours using DICE similarity coefficient (DSC) and line domain error (LDE) metrics. Pre- and post-workshop DSC results were compared using Wilcoxon signed ranks test to determine the impact of teaching during the workshop.

Results:

Of 50 workshop participants, paired pre- and post-workshop contours were available for 21 (42%), 20 (40%) and 22 (44%) participants for primary lung cancer, pelvic bone metastasis and pelvic node metastasis cases, respectively. Statistically significant improvements post-workshop in median DSC and LDE results were observed for 6 (50%) and 7 (58%) of 12 structures, respectively, although the magnitude of DSC/LDE improvement was modest in most cases. An increase in median DSC post-workshop ≥0·05 was only observed for GTVbone, IGTVlung and SacralPlex, and reduction in median LDE > 1 mm was only observed for GTVbone, CTVbone and SacralPlex. Post-workshop, median DSC values were >0·7 for 75% of structures. For 92% of the structures, post-workshop contours were considered to be acceptable or within acceptable variation following review by the workshop faculty.

Conclusions:

This study has demonstrated that virtual SABR contouring training is feasible and was associated with some improvements in contouring variation for multiple target volumes/OARs.

Is Remote Learning as Effective as In-Person Learning for Contouring Education? A Prospective Comparison of Face-to-Face vs. Online Delivery of the Anatomy and Radiology Contouring Bootcamp

PURPOSE

The Anatomy and Radiology Contouring (ARC) Bootcamp was a face-to-face (F2F) intervention providing integrated education for radiation oncology (RO) residents and medical physicists. To increase access, we launched an online offering in 2019. We evaluated the impact of the online course on participants' knowledge acquisition, contouring skills, and self-confidence by comparing it to the F2F course.  METHODS AND MATERIALS: Using modules, the online course offers similar content to the F2F comparator. Participants from the 2019 F2F and the 2019-2020 online course completed pre- and post-evaluations, assessing anatomy/radiology knowledge, contouring skills, self-confidence, and course satisfaction.

RESULTS

There were 180 (F2F: n=40; online: n=140) enrolled and 57 (F2F: n=30; online: n=27) participants completed both evaluations. The online course had a wider geographic participation (19 countries) than F2F (4 countries). F2F had primarily RO resident participation (80%), compared to online (41%). Both cohorts demonstrated similar improvements in self-confidence pertaining to their anatomy/radiology knowledge, contouring skills, and in interpreting radiology images (all p < 0.001). Both the online (mean ± SD improvement: 6.6 ± 6.7 on a 40-point scale; p < 0.001) and F2F (3.7 ± 5.7; p=0.002) groups showed anatomy/radiology knowledge improvement. Only the F2F group demonstrated improvement with the contouring assessment (F2F: 0.10 ± 0.17 on a 1-point Dice scale; p=0.004; online: 0.07 ± 0.16; p=0.076). Both cohorts perceived the course as a positive experience (F2F: 4.8 ± 0.4 on a 5-point scale; online: 4.5 ± 0.6), stated it would improve their professional practice (F2F: 4.6 ± 0.5; online: 4.2 ± 0.8), and said they would recommend it to others (F2F: 4.8 ± 0.4; online: 4.4 ± 0.6).

CONCLUSIONS

The online ARC Bootcamp demonstrated improved self-confidence, knowledge scores, and high satisfaction levels among participants. The offering had lower completion rates but was more accessible to geographic regions, provided a flexible learning experience, and allowed for ongoing education during the COVID-19 pandemic.

Implementation and evaluation of an online anatomy, radiology and contouring bootcamp for radiation therapists

BACKGROUND

As new treatments and technologies have been introduced in radiation oncology, the clinical roles of radiation therapists (RTs) have expanded. However, there are few formal learning opportunities for RTs. An online, anatomy, radiology and contouring bootcamp (ARC Bootcamp) originally designed for medical residents was identified as a prospective educational tool for RTs. The purpose of this study was to evaluate an RT edition of the ARC Bootcamp on knowledge, contouring, and confidence, as well as to identify areas for future modification.

METHODS

Fifty licensed RTs were enrolled in an eight-week, multidisciplinary, online RT ARC Bootcamp. Contouring practice was available throughout the course using an online contouring platform. Outcomes were evaluated using a pre-course and post-course multiple-choice quiz (MCQ), contouring evaluation and qualitative self-efficacy and satisfaction survey.

RESULTS

Of the fifty enrolled RTs, 30 completed the course, and 26 completed at least one of the post-tests. Nineteen contouring dice similarity coefficient (DSC) scores were available for paired pre- and post-course analysis. RTs demonstrated a statistically significant increase in mean DSC scoring pooled across all contouring structures (mean ± SD improvement: 0.09 ± 0.18 on a scale from 0 to 1, p=0.020). For individual contouring structures, 3/15 reached significance in contouring improvement. MCQ scores were available for 26 participants and increased after RT ARC Bootcamp participation with a mean ± SD pre-test score of 18.6 ± 4.2 (46.5%); on a 40-point scale vs. post-test score of 24.5 ± 4.3 (61.4%) (p < 0.001). RT confidence in contouring, anatomy knowledge and radiographic identification improved after course completion (p < 0.001). Feedback from RTs recommended more contouring instruction, less in-depth anatomy review and more time to complete the course.

CONCLUSIONS

The RT ARC Bootcamp was an effective tool for improving anatomy and radiographic knowledge among RTs. The course demonstrated improvements in contouring and overall confidence. However, only approximately half of the enrolled RTs completed the course, limiting statistical power. Future modifications will aim to increase relevance to RTs and improve completion rates.

Development, Implementation, and Initial Participant Feedback of an Online Anatomy and Radiology Contouring Bootcamp in Radiation Oncology

BACKGROUND

The Anatomy and Radiology Contouring (ARC) Bootcamp was a face-to-face (F2F) course designed to ensure radiation oncology residents were equipped with the knowledge and skillset to use radiation therapy techniques properly. The ARC Bootcamp was proven to be a useful educational intervention for improving learners' knowledge of anatomy and radiology and contouring ability. An online version of the course was created to increase accessibility to the ARC Bootcamp and provide a flexible, self-paced learning environment. This study aimed to describe the instructional design model used to create the online offering and report participants' motivation to enroll in the course and the online ARC Bootcamp's strengths and improvement areas.

METHODS

The creation of the online course followed the analysis, design, development, implementation, and evaluation (ADDIE) framework. The course was structured in a linear progression of locked modules consisting of radiology and contouring lectures, anatomy labs, and integrated evaluations.

RESULTS

The online course launched on the platform Teachable in November 2019, and by January 2021, 140 participants had enrolled in the course, with 27 participants completing all course components. The course had broad geographic participation with learners from 19 different countries. Of the participants enrolled, 34% were female, and most were radiation oncology residents (56%), followed by other programs (24%), such as medical physics residents or medical students. The primary motivator for participants to enroll was to improve their subject knowledge/skill (44%). The most common strength identified by participants was the course's quality (41%), and the most common improvement area was to incorporate more course content (41%).

CONCLUSIONS

The creation of the online ARC Bootcamp using the ADDIE framework was feasible. The course is accessible to diverse geographic regions and programs and provides a flexible learning environment; however, the course completion rate was low. Participants' feedback regarding their experiences will inform future offerings of the online course.

Learning radiotherapy: the state of the art

BACKGROUND

The last two decades have seen revolutionary developments in both radiotherapy technology and postgraduate medical training. Trainees are expected to attain competencies using a mix of experiential learning, formal postgraduate teaching, self-directed learning and peer education. Radiation (Clinical) Oncology is a recognised 'craft specialty' where the apprenticeship model of training is applicable. This scoping review examines the evidence in relation to how medical trainees learn radiotherapy.

METHODS

A systematic search of MEDINE and EMBASE was undertaken to identify studies of trainee and/or trainer experience of radiotherapy learning published 1999-2018. Results pertaining to Medical Oncology, workforce trends, undergraduate radiotherapy exposure, academic training, global health, non-medical staff, health service infrastructure and recruitment to training programmes were not included.

RESULTS

A total of 146 publications were included in the synthesis. Five themes were apparent through careful iterative analysis representing broadly inter-related issues. Most articles studied radiotherapy training from the perspective of the trainee doctor. Most literature reports results of observational, local or national surveys with a tightly defined scope. Considerable variation exists within hospitals, within countries, over time and between different curricular areas.

CONCLUSIONS

Medical education has not kept pace with changes in the field of radiotherapy and large differences are demonstrated in experience between trainees in different hospitals, countries and training stages. Interpersonal relationships, departmental organisation, and national curricula impact on training quality. Qualitative and quantitative research examining modern radiotherapy learning has been uncommon and uncoordinated, until recently. To date no single study has been designed to comprehensively assess a department's training scheme.

Systematic review of educational interventions to improve contouring in radiotherapy

BACKGROUND AND PURPOSE

Contouring is a critical step in the radiotherapy process, but there is limited research on how to teach it and no consensus about the best method. We summarize the current evidence regarding improvement of contouring skills.

METHODS AND MATERIALS

Comprehensive literature search of the Pubmed-MEDLINE database, EMBASE database and Cochrane Library to identify relevant studies (independently examined by two investigators) that included baseline contouring followed by a re-contouring assessment after an educational intervention.

RESULTS

598 papers were identified. 16 studies met the inclusion criteria representing 370 participants (average number of participants per study of 23; range (4-141). Regarding the teaching methodology, 5/16 used onsite courses, 8/16 online courses, and 2/16 used blended learning. Study quality was heterogenous. There were only 3 randomized studies and only 3 analyzed the dosimetric impact of improving contouring homogeneity. Dice similarity coefficient was the most common evaluation metric (7/16), and in all these studies at least some contours improved significantly post-intervention. The time frame for evaluating the learning effect of the teaching intervention was almost exclusively short-time, with only one study evaluating the long-term utility of the educational program beyond 6 months.

CONCLUSION

The literature on educational interventions designed to improve contouring performance is limited and heterogenous. Onsite, online and blended learning courses have all been shown to be helpful, however, sample sizes are small and impact assessment is almost exclusively short-term and typically does not take into account the effect on treatment planning. The most effective teaching methodology/format is unknown and impact on daily clinical practice is uncertain.

Harmonization of practices between radiotherapy centres in the Nord and Pas-de-Calais regions (France): A three-year evaluation

PURPOSE

The delineation of volumes of interest can be a source of significant interobserver variability. The purpose of this study was to improve the homogeneity of delineation between oncologist-radiotherapists in the territorial departments of Nord and Pas-de-Calais (France) through discussions of clinical cases and the adoption of common published reference documents.

MATERIALS AND METHODS

All eleven radiotherapy centres in the Nord and Pas-de-Calais departments of France participated. The localizations assessed to date included prostate, head and neck, breast and brain cancers. For each localization, the junior or senior physician(s) in charge of pathology delineated the volumes of interest according to their usual practices. Validated indices, including the Dice similarity coefficient, were used to quantify the delineation differences. The anonymized results were presented at two to three annual meetings. A second delineation of the clinical cases was then carried out to quantify homogenization. An evaluation of dosimetry practices was also conducted for prostate cancer. Wilcoxon assay matched data were used.

RESULTS

Our work showed either satisfactory delineation concordance after the initial assessment or improved delineation concordance. For prostate cancer, the Dice similarity coefficient values were greater than 0.6 initially in two of the three clinical cases. For head and neck cancers, a statistically significant improvement was observed for only one of the clinical target volumes. More than half of the Dice similarity coefficient values were greater than 0.6 in the first comparison. The study of clinical cases of breast cancer allowed a homogenization of the delineation of five of the six lymph node clinical target volumes. The dosimetry study of prostate cancer allowed for a homogenization of practices.

CONCLUSION

This work makes it possible to harmonize the delineation practices around validated standards. An extension to the entire Hauts-de-France region is planned.

Simulation as More Than a Treatment-Planning Tool: A Systematic Review of the Literature on Radiation Oncology Simulation-Based Medical Education

PURPOSE

Simulation-based medical education (SBME) is gaining prominence as a tool to meet Accreditation Council for Graduate Medical Education-mandated competency-based assessment educational goals. SBME is used in radiation oncology, although the type and extent are not clear. This study reports a systematic literature review designed to clarify the type and extent of radiation oncology SBME.

METHODS AND MATERIALS

The systematic review focused on radiation oncology SBME literature. The methods followed Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. The inclusion criteria were identified according to the PICOS (population, intervention, comparison, outcome, and setting) framework. The population included undergraduate, graduate, and continuing medical education learners. Studies were limited to English-language studies published on or after January 1, 1990, in peer-reviewed journals. PubMed, MedEdPORTAL, and in-press articles were searched. The PubMed search was conducted using predefined search terms. References and similar articles were examined. Medical Subject Headings terms in selected articles were reviewed to ensure relevant terms were included.

RESULTS

Fifty-four SBME publications met the inclusion criteria. Only 9 of 54 studies (17%) self-identified as SBME. SBME types included screen-based simulators (56%), simulated environments (13%), virtual reality and haptic systems (13%), simulated patients (11%), part-task trainers (6%), and computer-based systems with mannequins (2%). A variety of radiation oncology skill sets were addressed, including contouring (54%), treatment planning (20%), clinical decision making (17%), anatomy and/or radiology (13%), radiation biology and/or physics (13%), communication skills and/or patient education (13%), brachytherapy (13%), and immobilization (11%). A target learning population was defined in 47 studies, including residents (53%), attending physicians (36%), medical students (21%), medical physicists (11%), radiation therapists (9%), nurses (6%), administrative staff (4%), and dosimetrists (4%). Learner feedback was reported in 32 studies.

CONCLUSIONS

Overall, this systematic literature review provides context and guidance for future radiation oncology SBME development. Appropriately framing SBME reports in the radiation oncology literature will facilitate development, implementation, and evaluation of SBME interventions. SBME resources should be centralized to facilitate dissemination and share resources.

A Foundation for Enterprise Imaging: HIMSS-SIIM Collaborative White Paper

Care providers today routinely obtain valuable clinical multimedia with mobile devices, scope cameras, ultrasound, and many other modalities at the point of care. Image capture and storage workflows may be heterogeneous across an enterprise, and as a result, they often are not well incorporated in the electronic health record. Enterprise Imaging refers to a set of strategies, initiatives, and workflows implemented across a healthcare enterprise to consistently and optimally capture, index, manage, store, distribute, view, exchange, and analyze all clinical imaging and multimedia content to enhance the electronic health record. This paper is intended to introduce Enterprise Imaging as an important initiative to clinical and informatics leadership, and outline its key elements of governance, strategy, infrastructure, common multimedia content, acquisition workflows, enterprise image viewers, and image exchange services.

Evaluating the impact of a Canadian national anatomy and radiology contouring boot camp for radiation oncology residents

BACKGROUND

Radiation therapy treatment planning has advanced over the past 2 decades, with increased emphasis on 3-dimensional imaging for target and organ-at-risk (OAR) delineation. Recent studies suggest a need for improved resident instruction in this area. We developed and evaluated an intensive national educational course ("boot camp") designed to provide dedicated instruction in site-specific anatomy, radiology, and contouring using a multidisciplinary (MDT) approach.

METHODS

The anatomy and radiology contouring (ARC) boot camp was modeled after prior single-institution pilot studies and a needs-assessment survey. The boot camp incorporated joint lectures from radiation oncologists, anatomists, radiologists, and surgeons, with hands-on contouring instruction and small group interactive seminars using cadaveric prosections and correlative axial radiographs. Outcomes were evaluated using pretesting and posttesting, including anatomy/radiology multiple-choice questions (MCQ), timed contouring sessions (evaluated relative to a gold standard using Dice similarity metrics), and qualitative questions on satisfaction and perceived effectiveness. Analyses of pretest versus posttest scores were performed using nonparametric paired testing.

RESULTS

Twenty-nine radiation oncology residents from 10 Canadian universities participated. As part of their current training, 29%, 75%, and 21% receive anatomy, radiology, and contouring instruction, respectively. On posttest scores, the MCQ knowledge scores improved significantly (pretest mean 60% vs posttest mean 80%, P<.001). Across all contoured structures, there was a 0.20 median improvement in students' average Dice score (P<.001). For individual structures, significant Dice improvements occurred in 10 structures. Residents self-reported an improved ability to contour OARs and interpret radiographs in all anatomic sites, 92% of students found the MDT format effective for their learning, and 93% found the boot camp more effective than educational sessions at their own institutions. All of the residents (100%) would recommend this course to others.

CONCLUSIONS

The ARC boot camp is an effective intervention for improving radiation oncology residents' knowledge and understanding of anatomy and radiology in addition to enhancing their confidence and accuracy in contouring.