Enhancing the role of case-oriented peer review to improve quality and safety in radiation oncology: Executive summary

The Future of Safety and Quality in Radiation Oncology

The increasing complexity of radiation therapy treatment presents new potentials for error and suboptimal care. High-performing programs thus not only require adherence to, but also ongoing improvement of, key safety and quality practices. In this article, we review these practices including standardization, risk analysis, peer review, and maintenance of strong safety culture, while also describing recent innovations and promising future directions. We specifically highlight the growing role of artificial intelligence in radiation oncology, both as a tool to deliver safe, high-quality care and as a potential new source of safety challenges.

"If You're Talking, I Think You're Muted": Follow-up Analysis of Weekly Peer Review Discussion and Plan Changes After Transitioning From Virtual to In-Person Format

AIMS

During the COVID-19 public health emergency, we previously identified decreased rates of radiotherapy (RT) peer review (PR) discussion and plan changes in virtual versus in-person PR conferences. To expand on these findings, we continued to prospectively collect data on all PR conferences from 2021 to 2023 and performed a follow-up analysis before and after the transition back to in-person PR.

MATERIALS AND METHODS

A prospectively maintained database of weekly PR cases was queried for consecutive cases reviewed before and after the transition from virtual to in-person conferences. Rates of PR discussion and change recommendations were summarized and compared between the virtual and in-person groups. A survey was developed and administered to assess participants' perceived levels of engagement, opinions on optimal PR format, and preferences for future meetings before and 3 months after the transition back to in-person PR.

RESULTS

In total, 2,103 RT plans were reviewed: 1,590 virtually and 513 after the transition back to in-person. There was no difference in faculty attendance between groups. The proportion of cases with PR discussion increased from virtual (9.8%) to in-person (25.5%) format (p < 0.001). In the virtual group, 8.1% of cases had 1 topic and 1.7% had 2+ topics discussed. This increased to 15.8% and 9.7% during in-person PR, respectively (p < 0.001). The rate of change recommendation also increased from 1.5% (virtual) to 3.3% (in-person, p = 0.016). Among cases with at least 1 topic discussed, there was no difference in changes. Survey-reported distraction significantly decreased from virtual to in-person PR (p < 0.001).

CONCLUSION

Upon returning to in-person PR conferences, peer discussion and plan change recommendations significantly increased and returned to pre-pandemic levels, and participants' perceived levels of distraction were reduced. In an increasingly virtual world, additional efforts to develop best practices that maximize PR discussion and minimize distraction outside virtual conferences are warranted.

Does a peer review group consensus process for MR-Linac patients affect clinical care? Evaluation of impact and feasibility

Background and purpose

Peer review is an important component of quality assurance in radiotherapy. To our knowledge, there are no studies reporting on the feasibility and outcomes of the peer review process for magnetic resonance (MR) guided radiotherapy (MRgRT) on the MR linear accelerator (MR-Linac) despite the planning complexity involved and its evolving clinical indications. This study aimed to quantify the rate of change in treatment plans post-peer review and the time and resources required.

Materials and methods

Fifty-five cases presented at weekly MR-Linac peer review meetings across two centres from 8 June to 21 September 2023 were prospectively collected. Cases were analysed to determine the rate and extent of plan changes based on the Peer Review Audit Tool for radiation oncology (PRAT) developed by the Royal Australian and New Zealand College of Radiologists (RANZCR).

Results

Peer review resulted in changes to 36.4 % of treatment plans (n = 20), with 3.6 % (n = 2) having major changes requiring deferment of treatment. The most frequent changes were to organs at risk (OAR) volumes involving both delineation and increased OAR sparing (16.4 %, n = 9), total dose and fractionation (10.9 %, n = 6) and target volume dose coverage (5.5 %, n = 3). Patients with SBRT plans (39.1 % cf 22.2 %), oligometastatic/oligoprogressive sites (38.1 % cf 30.7 %) and reirradiation cases (41.2 % cf 34.2 %) had higher rates of change. Cases took a mean of 7 min (range 2-15 minutes) to discuss.

Conclusion

The high rates of plan changes support the value of peer review in MRgRT. We recommend, where possible that all MRgRT cases, particularly those involving SBRT plans, oligometastatic/oligoprogressive sites, and/or reirradiation, be subject to peer review.

Peer Review in Radiation Oncology: Where Does the Middle East, North Africa, and Türkiye Region Stand?

PURPOSE

This study aims to assess the status of radiation oncology peer review procedures across the Middle East, North Africa, and Türkiye (MENAT) region.

METHODS

A cross-sectional electronic survey was conducted among radiotherapy centers in the MENAT region in March 2024. It assessed peer review practices, departmental demographics, perceived importance of peer review, and potential barriers.

RESULTS

Data from 177 radiation oncology centers revealed varying peer review implementation across the MENAT region. Egypt had the highest participation (16.4%) among all responders. Most centers (31%) treated 500-1,000 cases annually. The majority (77.4%) implemented peer review, with varying levels between countries and across different centers. Advanced radiotherapy techniques significantly correlated with implementation of peer review (P < .05). Peer review meetings were mostly scheduled on a weekly basis (46%) and organized by radiation oncologists (84.7%). Target volume contouring (89%) and radiotherapy prescription (82%) were frequently peer-reviewed. Respondents with peer review at their institutions significantly valued peer review for education, adherence to guidelines, improving planning protocols, and reducing variation in practice institutions without peer review (P < .05). The most frequently reported barriers to peer review were having a high number of patients (56%) and shortage of time (54%).

CONCLUSION

Peer review is essential for improving the quality of practice in radiation oncology. Despite some discrepancies, numerous obstacles, and challenges in implementation, it is instrumental in the improvement of patient care in most centers throughout the region. Raising awareness among radiation oncologists about the importance of peer review is paramount to lead to better outcomes.

Pilot study of radiation oncology peer review in low middle income country (LMIC) through cloud-based platform

PURPOSE

Peer review is an essential step in clinical quality assurance for radiation therapy. There are very little data on peer reviews from low-middle-income countries (LMIC). With increasing access to advanced technologies in LMIC also, peer review is becoming more important to ensure quality and standard of care. We evaluated cloud-based e-Peer review in our network of cancer centers in India with an aim to study its feasibility and impact on care.

MATERIALS AND METHODS

Four out of 13 cancer centers across India were selected for this pilot study. All team members were trained on the e-Peer review platform before the initiation of the study. A lead dosimetrist from a centralized planning site was selected to share new cases every week. Cases treated with only definitive intent were selected. The link to the cases was sent through an email to reviewing physicians. The following aspects were reviewed for each case. 1) Work up and staging. 2) Treatment intent and prescription. 3) Target contours. 4) Normal organ at risk contours. 5) Dose-volume-histogram (DVH) with clinical goals attached. Cases were marked as "Not Appropriate," "Appropriate," "Appropriate with minor finding," and "Represent with major revisions" as per volume and plan review.

RESULTS

Over a period of 3 months, 100 cases underwent peer review before the start of treatment. Median turnover time was 48 (interquartile range: 24-96) hours. The median time for review was 8 min with time to review cases requiring major and minor changes being 12 and 6 min, respectively (P < 0.001). Of all the cases reviewed, no changes, minor changes, and major changes were suggested for 36%, 48%, and 16% of cases, respectively. The most frequent reason for major changes was contouring corrections (15%). Also, 31.3% of major changes underwent recontouring and replanning before initiation of treatment.

CONCLUSION

Peer review was feasible in our setting through this cloud-based peer review system, with median turnover time and time taken for review being 48 h and 8 min, respectively. Like published data from the Western world, peer review led to changes that could impact patient care delivery and outcome. We plan to implement this across the remaining centers in our network.

Technical Note: Improving the workflow in a carbon ion therapy center with custom software for enhanced patient care

Carbon-ion radiation therapy (CIRT) is an up-and-coming modality for cancer treatment. Implementation of CIRT requires collaboration among specialists like radiation oncologists, medical physicists, and other healthcare professionals. Effective communication among team members is necessary for the success of CIRT. However, the current workflows involving data management, treatment planning, scheduling, and quality assurance (QA) can be susceptible to errors, leading to delays and decreased efficiency. With the aim of addressing these challenges, a team of medical physicists developed an in-house workflow management software using FileMaker Pro. This tool has streamlined the workflow and improved the efficiency and quality of patient care.

Implementation and Efficacy of a Large-Scale Radiation Oncology Case-Based Peer-Review Quality Program across a Multinational Cancer Network

PURPOSE

With expansion of academic cancer center networks across geographically-dispersed sites, ensuring high-quality delivery of care across all network affiliates is essential. We report on the characteristics and efficacy of a radiation oncology peer-review quality assurance (QA) system implemented across a large-scale multinational cancer network.

METHODS AND MATERIALS

Since 2014, weekly case-based peer-review QA meetings have been standard for network radiation oncologists with radiation oncology faculty at a major academic center. This radiotherapy (RT) QA program involves pre-treatment peer-review of cases by disease site, with disease-site subspecialized main campus faculty members. This virtual QA platform involves direct review of the proposed RT plan as well as supporting data, including relevant pathology and imaging studies for each patient. Network RT plans were scored as being concordant or nonconcordant based on national guidelines, institutional recommendations, and/or expert judgment when considering individual patient-specific factors for a given case. Data from January 1, 2014, through December 31, 2019, were aggregated for analysis.

RESULTS

Between 2014 and 2019, across 8 network centers, a total of 16,601 RT plans underwent peer-review. The network-based peer-review case volume increased over the study period, from 958 cases in 2014 to 4,487 in 2019. A combined global nonconcordance rate of 4.5% was noted, with the highest nonconcordance rates among head-and-neck cases (11.0%). For centers that joined the network during the study period, we observed a significant decrease in the nonconcordance rate over time (3.1% average annual decrease in nonconcordance, P = 0.01); among centers that joined the network prior to the study period, nonconcordance rates remained stable over time.

CONCLUSIONS

Through a standardized QA platform, network-based multinational peer-review of RT plans can be achieved. Improved concordance rates among newly added network affiliates over time are noted, suggesting a positive impact of network membership on the quality of delivered cancer care.

Systematic Implementation of Effective Quality Assurance Processes for the Assessment of Radiation Target Volumes in Head and Neck Cancer

PURPOSE

Significant heterogeneity exists in clinical quality assurance (QA) practices within radiation oncology departments, with most chart rounds lacking prospective peer-reviewed contour evaluation. This has the potential to significantly affect patient outcomes, particularly for head and neck cancers (HNC) given the large variance in target volume delineation. With this understanding, we incorporated a prospective systematic peer contour-review process into our workflow for all patients with HNC. This study aims to assess the effectiveness of implementing prospective peer review into practice for our National Cancer Institute Designated Cancer Center and to report factors associated with contour modifications.

METHODS AND MATERIALS

Starting in November 2020, our department adopted a systematic QA process with real-time metrics, in which contours for all patients with HNC treated with radiation therapy were prospectively peer reviewed and graded. Contours were graded with green (unnecessary), yellow (minor), or red (major) colors based on the degree of peer-recommended modifications. Contours from November 2020 through September 2021 were included for analysis.

RESULTS

Three hundred sixty contours were included. Contour grades were made up of 89.7% green, 8.9% yellow, and 1.4% red grades. Physicians with >12 months of clinical experience were less likely to have contour changes requested than those with <12 months (8.3% vs 40.9%; P < .001). Contour grades were significantly associated with physician case load, with physicians presenting more than the median number of 50 cases having significantly less modifications requested than those presenting <50 (6.7% vs 13.3%; P = .013). Physicians working with a resident or fellow were less likely to have contour changes requested than those without a trainee (5.2% vs 12.6%; P = .039). Frequency of major modification requests significantly decreased over time after adoption of prospective peer contour review, with no red grades occurring >6 months after adoption.

CONCLUSIONS

This study highlights the importance of prospective peer contour-review implementation into systematic clinical QA processes for HNC. Physician experience proved to be the highest predictor of approved contours. A growth curve was demonstrated, with major modifications declining after prospective contour review implementation. Even within a high-volume academic practice with subspecialist attendings, >10% of patients had contour changes made as a direct result of prospective peer review.

ACR-ARS Practice Parameter for Radiation Oncology

BACKGROUND

This practice parameter was revised collaboratively by the American College of Radiology (ACR), and the American Radium Society. This practice parameter provides updated reference literature regarding radiation oncology practice and its key personnel.

METHODS

This practice parameter was developed according to the process described under the heading The Process for Developing ACR Practice Parameters and Technical Standards on the ACR website ( https://www.acr.org/Clinical-Resources/Practice-Parameters-and-Technical-Standards ) by the Committee on Practice Parameters-Radiation Oncology of the ACR Commission on Radiation Oncology in collaboration with the American Radium Society.

RESULTS

This practice parameter provides a comprehensive update to the reference literature regarding radiation oncology practice in general. The overall roles of the radiation oncologist, the Qualified Medical Physicist, and other specialized personnel involved in the delivery of external-beam radiation therapy are discussed. The use of radiation therapy requires detailed attention to equipment, patient and personnel safety, equipment maintenance and quality assurance, and continuing staff education. Because the practice of radiation oncology occurs in a variety of clinical environments, the judgment of a qualified radiation oncologist should be used to apply these practice parameters to individual practices. Radiation oncologists should follow the guiding principle of limiting radiation exposure to patients and personnel while accomplishing therapeutic goals.

CONCLUSION

This practice parameter can be used as an effective tool to guide radiation oncology practice by successfully incorporating the close interaction and coordination among radiation oncologists, medical physicists, dosimetrists, nurses, and radiation therapists.

The impact of transportation mode, socioeconomic deprivation and rurality on travel times to radiotherapy and surgical services for patients with prostate cancer: A national population-based evaluation

BACKGROUND

The distances that patients have to travel can influence their access to cancer treatment. We investigated the determinants of travel time, separately for journeys by car and public transport, to centres providing radical surgery or radiotherapy for prostate cancer.

METHODS

Using national cancer registry records linked to administrative hospital data, we identified patients who had radical surgery or radiotherapy for prostate cancer between January 2017 and December 2018 in the English National Health Service. Estimated travel times from the patients' residential area to the nearest specialist surgical or radiotherapy centre were estimated for journeys by car and by public transport.

RESULTS

We included 13,186 men who had surgery and 26,581 who had radiotherapy. Estimated travel times by public transport (74.4 mins for surgery and 69.4 mins for radiotherapy) were more than twice as long as by car (33.4 mins and 29.1mins, respectively). Patients living in more socially deprived neighbourhoods in rural areas had the longest travel times to the nearest cancer treatment centres by car (62.0 mins for surgery and 52.1 mins for radiotherapy). Conversely patients living in more affluent neighbourhoods in urban conurbations had the shortest (18.7 mins for surgery and 17.9 mins for radiotherapy).

CONCLUSION

Travel times to cancer centres vary widely according to mode of transport, socioeconomic deprivation, and rurality. Policies changing the geographical configuration of cancer services should consider the impact on the expected travel times both by car and by public transport to avoid enhancing existing inequalities in access to treatment and patient outcomes.

Implementation of a Novel Chart Rounds Application to Facilitate Peer Review in a Virtual Academic Environment

Purpose

Peer review in the form of chart rounds is a critical component of quality assurance and safety in radiation therapy treatments. Radiation therapy departments have undergone significant changes that impose challenges to meaningful review, including institutional growth and increasing use of virtual environment. We discuss the implementation of a novel chart rounds (NCR) format and application adapted to modern peer review needs at a single high-volume multisite National Cancer Institute designated cancer center.

Methods and Materials

A working group was created to improve upon the prior institutional chart rounds format (standard chart rounds or SCR). Using a novel in-house application and format redesign, an NCR was created and implemented to accomplish stated goals. Data regarding the SCR and NCR system were then extracted for review.

Results

SCR consisted of 2- 90-minute weekly sessions held to review plans across all disease sites, review of 49 plans per hour on average. NCR uses 1-hour long sessions divided by disease site, enabling additional time to be spent per patient (11 plans per hour on average) and more robust discussion. The NCR application is able to automate a list of plans requiring peer review from the institutional treatment planning system. The novel application incorporates features that enable efficient and accurate review of plans in the virtual setting across multiple sites. A systematic scoring system is integrated into the application to record feedback. Over 5 months of use of the NCR, 1160 plans have been reviewed with 143 scored as requiring minor changes, 32 requiring major changes and 307 with comments. Major changes triggered treatment replan. Feedback from scoring is incorporated into physician workflow to ensure changes are addressed.

Conclusion

The presented NCR format and application enables standardized and highly reliable peer review of radiation therapy plans that is robust across a variety of complex planning scenarios and could be implemented globally.

Enhancing quality assurance in radiotherapy for gynaecological cancers: implementation of an on-demand peer review process

OBJECTIVES

Ensuring high-quality radiotherapy requires peer-reviewing target volumes. The Royal College of Radiologists recommends peer review specifically for individual target volumes in cases of gynaecological cancers. This study presents the outcomes of implementing an on-demand peer review system for gynaecological cancers within our institute.

METHODS

The peer review process was planned for gynaecological cancer cases intended for curative radiotherapy. After junior clinical oncologists (COs) completed the segmentation, two senior COs specializing in gynaecological cancers conducted the peer review. All peer review outcomes were recorded prospectively. The audit process compliance, the proportion of patients requiring major and minor modifications in target volumes, the direction of changes, and the factors influencing these changes were reported.

RESULTS

A total of 230 patients were eligible, and out of these, 204 (88.3%) patients underwent at least one peer review. Among the patients, 108 required major modifications in their target volumes. P-charts revealed a stabilization in the need for major modifications at the end of three months, indicating that 38.2% and 28% of patients still required major modifications for the nodal and primary CTV, respectively. Multivariable analysis demonstrated that major modifications were associated with the use of extended field radiotherapy and radical radiation in non-cervical primary cases.

CONCLUSIONS

An on-demand peer review system was feasible and resulted in clinically meaningful, major modifications in the target volumes for 53% of patients.

ADVANCES IN KNOWLEDGE

Gynaecological cancers require ongoing peer review to ensure quality of care in radiotherapy. A flexible on-demand system not only ensures that patient treatment start is not delayed but also has an important educational role for junior trainees.

Radiation Oncology Peer Review in a Community Setting: The Value of Prospective Review

Peer review is an important component of any radiation oncology continuous quality improvement program. While limited guidelines exist, there is no consensus about how peer review should be performed, and large variations exist among different institutions. The purpose of this report is to describe our experience with peer review at a busy Radiation Oncology clinic and to evaluate the difference between prospective and retrospective peer review. We also performed a failure modes and effects analysis (FMEA) of the peer review process. Starting in 2015, every peer review session was tracked, including recommended changes to treatment plans. We reviewed the frequency, types and severity of these changes. A team of physicians and physicists conducted an FMEA of the peer review process. Between April 2015 and June 2020, a total of 3,691 patients were peer-reviewed. Out of those, 1,903 were prospective reviews (51.6%). Plans reviewed before treatment were almost 4.5 times more likely to be changed by peer review than those reviewed after the start of treatment (0.9% vs 0.2%). Plan changes after the start of treatment had a higher severity than changes prior to the start of treatment. FMEA identified several critical components of peer review. While there is no national standard for peer review, it is evident that prospective peer review is preferable. There may be a subconscious reluctance to change plans already underway, which could be a barrier to improving plans with the peer review process. Rather than reviewing in a group setting, it would be ideal to individually assign review tasks that are embedded in the clinical flow, assuring prospective review for all patients prior to final physician approval. Individual review rather than group review may be more candid, due to interpersonal concerns about publicly disagreeing with colleagues.

Analysis of Retrospective Versus Prospective Peer Review in a Multisite Academic Radiation Department

Purpose

Our multisite academic radiation department reviewed our experience with transitioning from weekly primarily retrospective to daily primarily prospective peer review to improve plan quality and decrease the rate of plan revisions after treatment start.

Methods and Materials

This study was an institutional review board-approved prospective comparison of radiation treatment plan review outcomes of plans reviewed weekly (majority within 1 week after treatment start) versus plans reviewed daily (majority before treatment start, except brachytherapy, frame-based radiosurgery, and some emergent plans). Deviations were based on peer comments and considered major if plan revisions were recommended before the next fraction and minor if modifications were suggested but not required. Categorical variables were compared using χ2 distribution tests of independence; means were compared using independent t tests.

Results

In all, 798 patients with 1124 plans were reviewed: 611 plans weekly and 513 plans daily. Overall, 76 deviations (6.8%) were noted. Rates of any deviation were increased in the daily era (8.6% vs 5.2%; P = .026), with higher rates of major deviations in the daily era (4.1% vs 1.6%; P = .012). Median working days between initial simulation and treatment was the same across eras (8 days). Deviations led to a plan revision at a higher rate in the daily era (84.1% vs 31.3%; P < .001).

Conclusions

Daily prospective peer review is feasible in a multisite academic setting. Daily peer review with emphasis on prospective plan evaluation increased constructive plan feedback, plan revisions, and plan revisions being implemented before treatment start.

Analysis of Radiation Therapy Quality Assurance in NRG Oncology RTOG 0848

PURPOSE

NRG/Radiation Therapy Oncology Group 0848 is a 2-step randomized trial to evaluate the benefit of the addition of concurrent fluoropyrimidine and radiation therapy (RT) after adjuvant chemotherapy (second step) for patients with resected pancreatic head adenocarcinoma. Real-time quality assurance (QA) was performed on each patient who underwent RT. This analysis aims to evaluate adherence to protocol-specified contouring and treatment planning and to report the types and frequencies of deviations requiring revisions.

METHODS AND MATERIALS

In addition to a web-based contouring atlas, the protocol outlined step-by-step instructions for generating the clinical treatment volume through the creation of specific regions of interest. The planning target volume was a uniform 0.5 cm clinical treatment volume expansion. One of 2 radiation oncology study chairs independently reviewed each plan. Plans with unacceptable deviations were returned for revision and resubmitted until approved. Treatment started after final approval of the RT plan.

RESULTS

From 2014 to 2018, 354 patients were enrolled in the second randomization. Of these, 160 patients received RT and were included in the QA analysis. Resubmissions were more common for patients planned with 3-dimensional conformal RT (43%) than with intensity modulated RT (31%). In total, at least 1 resubmission of the treatment plan was required for 33% of patients. Among patients requiring resubmission, most only needed 1 resubmission (87%). The most common reasons for resubmission were unacceptable deviations with respect to the preoperative gross target volume (60.7%) and the pancreaticojejunostomy (47.5%).

CONCLUSION

One-third of patients required resubmission to meet protocol compliance criteria, demonstrating the continued need for expending resources on real-time, pretreatment QA in trials evaluating the use of RT, particularly for pancreas cancer. Rigorous QA is critically important for clinical trials involving RT to ensure that the true effect of RT is assessed. Moreover, RT QA serves as an educational process through providing feedback from specialists to practicing radiation oncologists on best practices.

Systematic Review and Meta-analysis of the Association Between Radiation Therapy Treatment Volume and Patient Outcomes

PURPOSE

Evidence of a volume-outcome association in cancer surgery has shaped the centralization of cancer services; however, it is unknown whether a similar association exists for radiation therapy. The objective of this study was to determine the association between radiation therapy treatment volume and patient outcomes.

METHODS AND MATERIALS

This systematic review and meta-analysis included studies that compared outcomes of patients who underwent definitive radiation therapy at high-volume radiation therapy facilities (HVRFs) versus low-volume facilities (LVRFs). The systematic review used Ovid MEDLINE and Embase. For the meta-analysis, a random effects model was used. Absolute effects and hazard ratios (HRs) were used to compare patient outcomes.

RESULTS

The search identified 20 studies assessing the association between radiation therapy volume and patient outcomes. Seven of the studies looked at head and neck cancers (HNCs). The remaining studies covered cervical (4), prostate (4), bladder (3), lung (2), anal (2), esophageal (1), brain (2), liver (1), and pancreatic cancer (1). The meta-analysis demonstrated that HVRFs were associated with a lower chance of death compared with LVRFs (pooled HR, 0.90; 95% CI, 0.87- 0.94). HNCs had the strongest evidence of a volume-outcome association for both nasopharyngeal cancer (pooled HR, 0.74; 95% CI, 0.62-0.89) and nonnasopharyngeal HNC subsites (pooled HR, 0.80; 95% CI, 0.75-0.84), followed by prostate cancer (pooled HR, 0.92; 95% CI, 0.86-0.98). The remaining cancer types showed weak evidence of an association. The results also demonstrate that some centers defined as HVRFs are undertaking very few procedures per annum (<5 radiation therapy cases per year).

CONCLUSIONS

An association between radiation therapy treatment volume and patient outcomes exists for most cancer types. Centralization of radiation therapy services should be considered for cancer types with the strongest volume-outcome association, but the effect on equitable access to services needs to be explicitly considered.

Contour subregion error detection methodology using deep learning auto-segmentation

BACKGROUND

Inaccurate manual organ delineation is one of the high-risk failure modes in radiation treatment. Numerous automated contour quality assurance (QA) systems have been developed to assess contour acceptability; however, manual inspection of flagged cases is a time-consuming and challenging process, and can lead to users overlooking the exact error location.

PURPOSE

Our aim is to develop and validate a contour QA system that can effectively detect and visualize subregional contour errors, both qualitatively and quantitatively.

METHODS/MATERIALS

A novel contour subregion error detection (CSED) system was developed using subregional surface distance discrepancies between manual and deep learning auto-segmentation (DLAS) contours. A validation study was conducted using a head and neck public dataset containing 339 cases and evaluated according to knowledge-based pass criteria derived from a clinical training dataset of 60 cases. A blind qualitative evaluation was conducted, comparing the results from the CSED system with manual labels. Subsequently, the CSED-flagged cases were re-examined by a radiation oncologist.

RESULTS

The CSED system could visualize the diverse types of subregional contour errors qualitatively and quantitatively. In the validation dataset, the CSED system resulted in true positive rates (TPR) of 0.814, 0.800, and 0.771; false positive rates (FPR) of 0.310, 0.267, and 0.298; and accuracies of 0.735, 0.759, and 0.730, for brainstem and left and right parotid contours, respectively. The CSED-assisted manual review caught 13 brainstem, 19 left parotid, and 21 right parotid contour errors missed by conventional human review. The TPR/FPR/accuracy of the CSED-assisted manual review improved to 0.836/0.253/0.784, 0.831/0.171/0.830, and 0.808/0.193/0.807 for each structure, respectively. Further, the time savings achieved through CSED-assisted review improved by 75%, with the time for review taking 24.81 ± 12.84, 26.75 ± 10.41, and 28.71 ± 13.72 s for each structure, respectively.

CONCLUSIONS

The CSED system enables qualitative and quantitative detection, localization, and visualization of manual segmentation subregional errors utilizing DLAS contours as references. The use of this system has been shown to help reduce the risk of high-risk failure modes resulting from inaccurate organ segmentation.

Bridging the Communication Gaps: A Prospective Single-Arm Pilot Study Testing the Feasibility of Interdisciplinary Radiotherapy Planning in Locally Advanced Lung Cancer

RATIONALE AND OBJECTIVES

The treatment of locally advanced lung cancer (LALC) with radiotherapy (RT) can be challenging. Multidisciplinary collaboration between radiologists and radiation oncologists (ROs) may optimize RT planning, reduce uncertainty in follow-up imaging interpretation, and improve outcomes.

MATERIALS AND METHODS

In this prospective clinical treatment trial (clinicaltrials.gov NCT04844736), 37 patients receiving definitive RT for LALC, six attending ROs, and three thoracic radiologists were consented and enrolled across four treatment centers. Prior to RT plan finalization, representative computed tomography (CT) slices with overlaid outlines of preliminary irradiation targets were shared with the team of radiologists. The primary endpoint was to assess feasibility of receiving feedback no later than 4 business days of RT simulation on at least 50% of plans.

RESULTS

Thirty-seven patients with lung cancer were enrolled, and 35 of 37 RT plans were reviewed. Of the 35 patients reviewed, mean age was 69 years. For 27 of 37 plans (73%), feedback was received within 4 or fewer days (interquartile range 3-4 days). Thirteen of 35 cases (37%) received feedback that the delineated target potentially did not include all sites suspicious for tumor involvement. In total, changes to the RT plan were recommended for over- or undercoverage in 16 of 35 cases (46%) and implemented in all cases. Radiology review resulted in no treatment delays and substantial changes to irradiated volumes: gross tumor volume, -1.9 to +96.1%; planning target volume, -37.5 to +116.5%.

CONCLUSION

Interdisciplinary collaborative RT planning using a simplified workflow was feasible, produced no treatment delays, and prompted substantial changes in RT targets.

Risk management patterns in radiation oncology-results of a national survey within the framework of the Patient Safety in German Radiation Oncology (PaSaGeRO) project

PURPOSE

Risk management (RM) is a key component of patient safety in radiation oncology (RO). We investigated current approaches on RM in German RO within the framework of the Patient Safety in German Radiation Oncology (PaSaGeRO) project. Aim was not only to evaluate a status quo of RM purposes but furthermore to discover challenges for sustainable RM that should be addressed in future research and recommendations.

METHODS

An online survey was conducted from June to August 2021, consisting of 18 items on prospective and reactive RM, protagonists of RM, and self-assessment concerning RM. The survey was designed using LimeSurvey and invitations were sent by e‑mail. Answers were requested once per institution.

RESULTS

In all, 48 completed questionnaires from university hospitals, general and non-academic hospitals, and private practices were received and considered for evaluation. Prospective and reactive RM was commonly conducted within interprofessional teams; 88% of all institutions performed prospective risk analyses. Most institutions (71%) reported incidents or near-events using multiple reporting systems. Results were presented to the team in 71% for prospective analyses and 85% for analyses of incidents. Risk conferences take place in 46% of institutions. 42% nominated a manager/committee for RM. Knowledge concerning RM was mostly rated "satisfying" (44%). However, 65% of all institutions require more information about RM by professional societies.

CONCLUSION

Our results revealed heterogeneous patterns of RM in RO departments, although most departments adhered to common recommendations. Identified mismatches between recommendations and implementation of RM provide baseline data for future research and support definition of teaching content.

A Decade of Prospective Peer Review: Impact on Safety Culture and Lessons Learned in a Multicenter Radiation Medicine Department

PURPOSE

Quality assurance (QA) is critical to the success of radiation therapy (RT) for patients with cancer and affects clinical outcomes. We report longitudinal findings of a prospective peer review evaluation system implemented at a major academic health system as part of RT QA during a 10-year period.

METHODS AND MATERIALS

All cases treated within our department undergo prospective multidisciplinary peer review and are assigned a grade (A, B, or C). "A" cases require no changes, "B" cases require minor modification, and "C" cases require major modification before treatment planning. The z-ratio test for the significance of the difference between the 5-year baseline (2012-2016) and follow-up (2017-2021) period was used to compare grades between the 2 periods. A 2-tailed P value <.05 was considered significant.

RESULTS

Of the 20,069 cases, 15,659 (78%) were curative and were analyzed. The fraction of A cases decreased from 74.8% (baseline) to 64.5% (follow-up), whereas B cases increased from 19.4% to 35.4% and C cases decreased from 5.8% to 0.1%. Of the 9 treatment locations, the main hospital site had a higher percentage of A grades relative to community locations in the baseline (78.6% vs 67.8%; P < .002) and follow-up (66.9% vs 62.3%; P < .002) periods. There was a decrease in the percentage of A cases from the baseline to the follow-up period regardless of plan type (complex vs intermediate vs simple). There was a decrease in the percentage of A cases among specialists from baseline to follow-up (78.2% to 67.7%; P < .002) and among generalists from baseline to follow-up (69.7% to 61.7%; P < .002).

CONCLUSIONS

Our 10-year experience in contour peer review identified increased opportunities in improving treatment plan quality over time. The drop in A scores and rise in B scores suggests increased scrutiny and findings-based improvements over time, whereas the drop in C scores indicates amelioration of "major failures" addressed in the startup years. Peer review rounds upstream of treatment planning provide valuable RT QA and should be considered by other departments to enhance the quality and consistency of RT.

The Effect of Surgeon Referral and a Radiation Oncologist Productivity-Based Metric on Radiation Therapy Receipt Among Elderly Women With Early Stage Breast Cancer: Analysis From a Tertiary Cancer Network

Purpose

: Guidelines for early-stage breast cancer allow for radiation therapy (RT) omission after breast conserving surgery among older women, though high utilization of RT persists. This study explored surgeon referral and the effect of a productivity-based bonus metric for radiation oncologists in an academic institution with centralized quality assurance review.

Methods and materials

: We evaluated patients ≥70 years of age treated with breast conserving surgery for estrogen receptor (ER)+ pT1N0 breast cancer at a single tertiary cancer network between 2015 and 2018. The primary outcomes were radiation oncology referral and RT receipt. Covariables included patient and physician characteristics and treatment decisions before versus after productivity metric implementation. Univariable generalized linear effects models explored associations between these outcomes and covariables.

Results

: Of 703 patients included, 483 (69%) were referred to radiation oncology and 273 (39%) received RT (among those referred, 57% received RT). No difference in RT receipt pre- versus post-productivity metric implementation was observed (P = .57). RT receipt was associated with younger patient age (70-74 years; odds ratio [OR], 2.66; 95% confidence interval [CI], 1.54-4.57) and higher grade (grade 3; OR, 7.75; 95% CI, 3.33-18.07). Initial referral was associated with younger age (70-74; OR, 5.64; 95% CI, 3.37-0.45) and higher performance status (Karnofsky performance status ≥90; OR, 5.34; 95% CI, 2.63-10.83).

Conclusions

: Nonreferral to radiation oncology accounted for half of RT omission but was based on age and Karnofsky performance status, in accordance with guidelines. Lack of radiation oncologist practice change in response to misaligned financial incentives is reassuring, potentially reflecting incentive design and/or centralized quality assurance review. Multi-institutional studies are needed to confirm these findings.

Quality and Safety Considerations in Intensity Modulated Radiation Therapy: An ASTRO Safety White Paper Update

PURPOSE

This updated report on intensity modulated radiation therapy (IMRT) is part of a series of consensus-based white papers previously published by the American Society for Radiation Oncology (ASTRO) addressing patient safety. Since the first white papers were published, IMRT went from widespread use to now being the main delivery technique for many treatment sites. IMRT enables higher radiation doses to be delivered to more precise targets while minimizing the dose to uninvolved normal tissue. Due to the associated complexity, IMRT requires additional planning and safety checks before treatment begins and, therefore, quality and safety considerations for this technique remain important areas of focus.

METHODS AND MATERIALS

ASTRO convened an interdisciplinary task force to assess the original IMRT white paper and update content where appropriate. Recommendations were created using a consensus-building methodology, and task force members indicated their level of agreement based on a 5-point Likert scale, from "strongly agree" to "strongly disagree." A prespecified threshold of ≥75% of raters who select "strongly agree" or "agree" indicated consensus.

CONCLUSIONS

This IMRT white paper primarily focuses on quality and safety processes in planning and delivery. Building on the prior version, this consensus paper incorporates revised and new guidance documents and technology updates. IMRT requires an interdisciplinary team-based approach, staffed by appropriately trained individuals as well as significant personnel resources, specialized technology, and implementation time. A comprehensive quality assurance program must be developed, using established guidance, to ensure IMRT is performed in a safe and effective manner. Patient safety in the delivery of IMRT is everyone's responsibility, and professional organizations, regulators, vendors, and end-users must work together to ensure the highest levels of safety.

Head and neck radiotherapy quality assurance conference for dedicated review of delineated targets and organs at risk: results of a prospective study

Purpose

Head and neck (HN) radiotherapy (RT) is complex, involving multiple target and organ at risk (OAR) structures delineated by the radiation oncologist. Site-agnostic peer review after RT plan completion is often inadequate for thorough review of these structures. In-depth review of RT contours is critical to maintain high-quality RT and optimal patient outcomes.

Materials and Methods

In August 2020, the HN RT Quality Assurance Conference, a weekly teleconference that included at least one radiation oncology HN specialist, was activated at our institution. Targets and OARs were reviewed in detail prior to RT plan creation. A parallel implementation study recorded patient factors and outcomes of these reviews. A major change was any modification to the high-dose planning target volume (PTV) or the prescription dose/fractionation; a minor change was modification to the intermediate-dose PTV, low-dose PTV, or any OAR. We analysed the results of consecutive RT contour review in the first 20 months since its initiation.

Results

A total of 208 patients treated by 8 providers were reviewed: 86·5% from the primary tertiary care hospital and 13·5% from regional practices. A major change was recommended in 14·4% and implemented in 25 of 30 cases (83·3%). A minor change was recommended in 17·3% and implemented in 32 of 36 cases (88·9%). A survey of participants found that all (n = 11) strongly agreed or agreed that the conference was useful.

Conclusion

Dedicated review of RT targets/OARs with a HN subspecialist is associated with substantial rates of suggested and implemented modifications to the contours.

A digital physician peer to automatically detect erroneous prescriptions in radiotherapy

Appropriate dosing of radiation is crucial to patient safety in radiotherapy. Current quality assurance depends heavily on a physician peer-review process, which includes a review of the treatment plan's dose and fractionation. Potentially, physicians may not identify errors during this manual peer review due to time constraints and caseload. A novel prescription anomaly detection algorithm is designed that utilizes historical data from the past to predict anomalous cases. Such a tool can serve as an electronic peer who will assist the peer-review process providing extra safety to the patients. In our primary model, we create two dissimilarity metrics, R and F. R defining how far a new patient's prescription is from historical prescriptions. F represents how far away a patient's feature set is from that of the group with an identical or similar prescription. We flag prescription if either metric is greater than specific optimized cut-off values. We use thoracic cancer patients (n = 2504) as an example and extracted seven features. Our testing set f1 score is between 73%-94% for different treatment technique groups. We also independently validate our results by conducting a mock peer review with three thoracic specialists. Our model has a lower type II error rate compared to the manual peer-review by physicians.

Improving Quality Metrics in Radiation Oncology: Implementation of Pretreatment Peer Review for Stereotactic Body Radiation Therapy in Patients with Thoracic Cancer

Purpose

Traditional peer reviews occur weekly, and can take place up to 1 week after the start of treatment. The American Society for Radiation Oncology peer-review white paper identified stereotactic body radiation therapy (SBRT) as a high priority for contour/plan review before the start of treatment, considering both the rapid-dose falloff and short treatment course. Yet, peer-review goals for SBRT must also balance physician time demands and the desire to avoid routine treatment delays that would occur in the setting of a 100% pretreatment (pre-Tx) review compliance requirement or prolonging the standard treatment planning timeline. Herein, we report on our pilot experience of a pre-Tx peer review of thoracic SBRT cases.

Methods and Materials

From March 2020 to August 2021, patients undergoing thoracic SBRT were identified for pre-Tx review, and placed on a quality checklist. We implemented twice-weekly meetings for detailed pre-Tx review of organ-at-risk/target contours and dose constraints in the treatment planning system for SBRT cases. Our quality metric goal was to peer review ≥90% of SBRT cases before exceeding 25% of the dose delivered. We used a statistical process control chart with sigma limits (ie, standard deviations [SDs]) to access compliance rates with pre-Tx review implementation.

Results

We identified 252 patients treated with SBRT to 294 lung nodules. When comparing pre-Tx review completion from initial rollout to full implementation, our rates improved from 19% to 79% (ie, from 1 sigma limit [SDs]) below to >2 sigma limits (SDs) above. Additionally, early completion of any form of contour/plan review (defined as any pre-Tx or standard review completed before exceeding 25% of the dose delivered) increased from 67% to 85% (March 2020-November 2020) to 76% to 94% (December 2020-August 2021).

Conclusions

We successfully implemented a sustainable workflow for detailed pre-Tx contour/plan review for thoracic SBRT cases in the context of twice-weekly disease site-specific peer-review meetings. We reached our quality improvement objective to peer review ≥90% of SBRT cases before exceeding 25% of the dose delivered. This process was feasible to conduct in an integrated network of sites across our system.

Attitudes and access to resources and strategies to improve quality of radiotherapy among US radiation oncologists: A mixed methods study

INTRODUCTION

We aimed to assess contouring-related practices among US radiation oncologists and explore how access to and use of resources and quality improvement strategies vary based on individual- and organization-level factors.

METHODS

We conducted a mixed methods study with a sequential explanatory design. Surveys were emailed to a random 10% sample of practicing US radiation oncologists. Participating physicians were invited to a semi-structured interview. Kruskal-Wallis and Wilcoxon rank-sum tests and a multivariable regression model were used to evaluate associations. Interview data were coded using thematic content analysis.

RESULTS

Survey overall response rate was 24%, and subsequent completion rate was 97%. Contouring-related questions arise in ≥50% of clinical cases among 73% of respondents. Resources accessed first include published atlases (75%) followed by consulting another radiation oncologist (60%). Generalists access consensus guidelines more often than disease-site specialists (P = 0.04), while eContour.org is more often used by generalists (OR 4.3, 95% CI 1.2-14.8) and younger physicians (OR 1.33 for each 5-year increase, 95% CI 1.08-1.67). Common physician-reported barriers to optimizing contour quality are time constraints (58%) and lack of access to disease-site specialists (21%). Forty percent (40%, n = 14) of physicians without access to disease-site specialists indicated it could facilitate the adoption of new treatments. Almost all (97%) respondents have formal peer review, but only 43% have contour-specific review, which is more common in academic centres (P = 0.02).

CONCLUSION

Potential opportunities to improve radiation contour quality include improved access to disease-site specialists and contour-specific peer review. Physician time must be considered when designing new strategies.

Veterans Affairs Radiation Oncology Quality Surveillance Program and American Society for Radiation Oncology Quality Measures Initiative

INTRODUCTION

Ensuring high quality, evidence-based radiation therapy for patients is of the upmost importance. As a part of the largest integrated health system in America, the Department of Veterans Affairs National Radiation Oncology Program (VA-NROP) established a quality surveillance initiative to address the challenge and necessity of providing the highest quality of care for veterans treated for cancer.

METHODS

As part of this initiative, the VA-NROP contracted with the American Society for Radiation Oncology (ASTRO) to commission five Blue-Ribbon Panels for lung, prostate, rectal, breast, and head & neck cancers experts. This group worked collaboratively with the VA-NROP to develop consensus quality measures. In addition to the site-specific measures, an additional Blue-Ribbon Panel comprised of the chairs and other members of the disease sites was formed to create 18 harmonized quality measures for all five sites (13 quality, 4 surveillance, and 1 aspirational).

CONCLUSION

The VA-NROP and ASTRO collaboration have created quality measures spanning five disease sites to help improve patient outcomes. These will be used for the ongoing quality surveillance of veterans receiving radiation therapy through the VA and its community partners.

ETHICS BOARD APPROVAL

N/A - No human subjects were required.

Peer review quality assurance in stereotactic body radiotherapy planning: the impact of case volume

Purpose:

Peer review is an essential quality assurance component of radiation therapy planning. A growing body of literature has demonstrated substantial rates of suggested plan changes resulting from peer review. There remains a paucity of data on the impact of peer review rounds for stereotactic body radiation therapy (SBRT). We therefore aim to evaluate the outcomes of peer review in this specific patient cohort.

Methods and materials:

We conducted a retrospective review of all SBRT cases that underwent peer review from July 2015 to June 2018 at a single institution. Weekly peer review rounds are grouped according to cancer subsite and attended by radiation oncologists, medical physicists and medical radiation technologists. We prospectively compiled ‘learning moments’, defined as cases with suggested changes or where an educational discussion occurred beyond routine management, and critical errors, defined as errors which could alter clinical outcomes, recorded prospectively during peer review. Plan changes implemented after peer review were documented.

Results:

Nine hundred thirty-four SBRT cases were included. The most common treatment sites were lung (518, 55%), liver (196, 21%) and spine (119, 13%). Learning moments were identified in 161 cases (17%) and translated into plan changes in 28 cases (3%). Two critical errors (0.2%) were identified: an inadequate planning target volume margin and an incorrect image set used for contouring. There was a statistically significantly higher rate of learning moments for lower-volume SBRT sites (defined as ≤30 cases/year) versus higher-volume SBRT sites (29% vs 16%, respectively; p = 0.001).

Conclusions:

Peer review for SBRT cases revealed a low rate of critical errors, but did result in implemented plan changes in 3% of cases, and either educational discussion or suggestions of plan changes in 17% of cases. All SBRT sites appear to benefit from peer review, though lower-volume sites may require particular attention.

Radiation oncology peer review in Australia and New Zealand

Peer review is a part of high quality care within radiation oncology, designed to achieve the best outcomes for patients. We discuss the importance of and evidence for peer review in clinical practice. The Royal Australia and New Zealand College of Radiologists (RANZCR) has evolved a Peer Review Assessment Tool (PRAT) since 1999. We report the results of a RANZCR faculty survey conducted in radiation oncology facilities across Australia and New Zealand to guide the 2019 PRAT revision process, and discuss the development and implementation of the 2019 PRAT. Peer-review processes are now mandated as a component of Australian and International Quality Standards. Several practical recommendations might address challenges for effective implementation of peer review process in routine clinical practice. This includes prioritising tumour sites and treatment techniques for peer review within the time and resources constraints of each institution, improving resource allocation, ensuring optimal timing and duration for peer review meetings, and adopting multi-centre virtual peer review meeting where necessary.

Knowledge-based quality control of organ delineations in radiation therapy

PURPOSE

To reduce the likelihood of errors in organ delineations used for radiotherapy treatment planning, a knowledge-based quality control (KBQC) system, which discriminates between valid and anomalous delineations is developed.

METHOD AND MATERIALS

The KBQC is comprised of a group-wise inference system and anomaly detection modules trained using historical priors from 296 locally advanced lung and prostate cancer patient computational tomographies (CTs). The inference system discriminates different organs based on shape, relational, and intensity features. For a given delineated image set, the inference system solves a combinatorial optimization problem that results in an organ group whose relational features follow those of the training set considering the posterior probabilities obtained from support vector machine (SVM), discriminant subspace ensemble (DSE), and artificial neural network (ANN) classifiers. These classifiers are trained on nonrelational features with a 10-fold cross-validation scheme. The anomaly detection module is a bank of ANN autoencoders, each corresponding with an organ, trained on nonrelational features. A heuristic rule detects anomalous organs that exceed predefined organ-specific tolerances for the feature reconstruction error and the classifier's posterior probabilities. Independent data sets with anomalous delineations were used to test the overall performance of the KBQC system. The anomalous delineations were manually manipulated, computer-generated, or propagated based on a transformation obtained by imperfect registrations. Both peer-review-based scoring system and shape similarity coefficient (DSC) were used to label regions of interest (ROIs) as normal or anomalous in two independent test cohorts.

RESULTS

The accuracy of the classifiers was ≥ $\ge$ 99.8%, and the minimum per-class F1-scores were 0.99, 0.99, and 0.98 for SVM, DSE, and ANN, respectively. The group-wise inference system reduced the miss-classification likelihood for the test data set with anomalous delineations compared to each individual classifier and a fused classifier that used the average posterior probability of all classifiers. For 15 independent locally advanced lung patients, the system detected > $>$ 79% of the anomalous ROIs. For 1320 auto-segmented abdominopelvic organs, the anomaly detection system identified anomalous delineations, which also had low Dice similarity coefficient values with respect to manually delineated organs in the training data set.

CONCLUSION

The KBQC system detected anomalous delineations with superior accuracy compared to classification methods that judge only based on posterior probabilities.

Peer Review of Head and Neck Cancer Planning Target Volumes in Radiation Oncology

Purpose

Radiation treatment plans undergo peer review during chart rounds, but changes to treatment volumes would require replanning. Our group implemented weekly head and neck cancer “volume rounds” to peer review all target volumes for head and neck cancer before radiation therapy (RT) planning and chart rounds.

Methods and Materials

We analyzed modifications made to planning target volumes (PTVs) at volume rounds for consecutive nonproton head and neck cancer cases from May 2020 to May 2021. Nine head and neck radiation oncologists participated in weekly volume rounds during this time. Recommendations were categorized as no changes, minor changes, major changes, additional workup (eg, biopsy or imaging), and consultation or tumor board discussion needed before the start of RT. Minor changes to PTVs generally did not require a second review before treatment planning while major changes did.

Results

PTVs for 511 cases involving 432 patients underwent peer review and 298 (58.3%) of these cases did not require any modifications before treatment planning. Minor and major changes were recommended in 75 (14.7%) and 86 (16.8%) cases, respectively. Forty-five (8.8%) cases were recommended to have additional workup and 23 (4.5%) required additional consultation with nonradiation surgeons or medical oncologists. Of the 45 cases that were recommended for additional workup, 40 underwent biopsy or imaging. Positive findings on imaging or biopsy occurred in 13 patients, leading to a significant change in management, including 4 patients who underwent additional surgery after positive findings before the start of RT.

Conclusions

Prospective peer review during head and neck cancer volume rounds led to frequent minor and major alterations to PTVs. Significant changes in the overall treatment plan, such as additional surgery before start of RT, occurred in a minority of patients.

Evaluation of a prospective radiation oncology departmental team review process using standardized simulation directives

INTRODUCTION

The primary objective of this study is to evaluate the utility and value of an institutional, multi-disciplinary radiation oncology team review process prior to radiotherapy (RT) simulation.

METHODS

Over a period of 3 months and through an iterative team-based process, a standardized simulation requisition directive (SSRD) was developed, piloted, modified, and subsequently implemented for all patients treated with external beam RT at a single tertiary care institution from January to December 2020. The SSRDs were reviewed at a daily multi-disciplinary radiation oncology team review conference; modifications consequential to the review were prospectively recorded in a quality database.

RESULTS

1,500 consecutive SSRDs were prospectively reviewed for this study. 397 modifications on 290 (19.3%) SSRDs were recorded and parsed into 5 main categories and 18 subcategories. The most common modifications resulted from changes in immobilization device (n=88, 22.2%), RT care path (n=56, 14.1%), and arm positioning (n=43, 10.8%). On univariate analysis, modifications were associated with RT intent, scan parameters, tumor site, and consultation type. An increased rate modifications was observed for patients had telemedicine consults (n=101, 22.7%) compared to in-person consultations (n=189, 17.9%) (p=0.032). Using logistic regression analysis, there was also a statistically significant relationship between postoperative RT delivery and modification rates (OR: 2.913, 95% CI: 1.014-8.372) (p=0.0126). Overall, only 14 patients (0.9%) needed re-simulation during the entire study period.

CONCLUSIONS

Prospective multi-disciplinary radiation oncology team review prior to simulation identifies actionable change in approximately 19% of procedures, and results in an extremely low rate (<1%) of re-simulation. As departmental processes transition to virtual platforms, thorough attention is needed to identify patients at higher risk of simulation modifications.

The impact of radiation therapist-led structured peer review meetings on compliance to Radiation Oncology Practice Standards

INTRODUCTION

Regular tumour-specific peer review meetings (TPRMs) were established by our group during 2016. A dedicated Quality Assurance Radiation Therapist (QART) was employed in 2018 to co-ordinate the meetings and for each patient, complete the Peer Review Audit Tool (PRAT) of the Royal Australian and New Zealand College of Radiologists (RANZCR). The aim of the current quality assurance study was to investigate the impact of the TPRMs and appointment of the QART on compliance to relevant RANZCR Radiation Oncology Practice Standards (ROPS).

METHODS

Tumour-specific peer review meetings for eight tumour sites were assessed across our group's three hospitals from January 2017 to December 2019. Data from meetings were collected using the PRAT or from paper-based minutes and assessed against four ROPS (ROPS 3, 4, 8 and 9). Compliance with each of the four standards was measured by presence of the required documentation and presentation at TPRM, as recorded by the PRAT.

RESULTS

There was an increase in the overall number of peer review cases audited from 173 in the 2017 calendar year to 469 in 2018 and 619 in 2019, representing 7%, 18% and 22% of all treatment courses started during these years, respectively. Staging was the most incompletely documented item across all years for audited patients. The request for radiation treatment plan modifications increased year-on-year: modifications were requested for 5% of plans in 2017 (8/172), 18% in 2018 (81/452) and 19% (119/619) in 2019.

CONCLUSION

This study has shown that an increase in the number of cases for peer-review audit corresponded to the QART-facilitated TPRMs. Application of the PRAT has identified radiation treatment plan modifications that would otherwise go undetected and without opportunity to improve the quality of patients' treatment or avoid harm.

Interprofessional Image Verification Workshop for Physician and Physics Residents: A Multi-Institutional Experience

PURPOSE

Verification of patient position through pretreatment setup imaging is crucial in modern radiation therapy. As treatment complexity increases and technology evolves, physicist-physician collaboration becomes imperative for safe and successful radiation delivery. Despite the importance of both, residency programs lack formal interprofessional education (IPE) activities or structured training for image verification. Here we show the impact of an interprofessional image verification workshop for residents in a multi-institutional setting.

METHODS

The workshop included a lecture by the attending physicist and physician, and hands-on image registration practice by learners (medical physics residents, MP; and radiation oncology residents, RO). All participants filled out pre- and postactivity surveys and rated their comfort from 1 to 10 in (A) selecting what type of imaging to order for a given case and (B) independently assessing the setup quality based on imaging. A paired 1-tailed t test (α = 0.05) was used to evaluate significance; Spearman rank correlation coefficient was used to assess correlation of ratings and RO postgraduate year (PGY). Surveys had free-response questions about IPE and image verification activities in residency.

RESULTS

A total of 71 residents from 7 institutions participated between 2018 and 2020. Pre- and postsurveys were completed by 50 residents (38RO, 12MP) and showed an increase in (A) from 5.5 ± 2.2 to 7.1 ± 1.6 (P < .001) and in (B) from 5.1 ± 2.3 to 6.8 ± 1.5 (P < .001), with significant increases per subgroup (A_{Δ, RO} = 1.8 ± 1.7, P < .001; B_{Δ, RO} = 1.9 ± 1.8, P <. 001; A_{Δ, MP} = 1.1 ± 1.4, P = .012; B_{Δ, MP} = 1.2 ± 1.6, P = .016). RO confidence scores moderately correlated with PGY. Survey responses indicated that image verification training is mostly unstructured, with extent of exposure varying by program and attending; most with little-to-no training. Time constraints were identified as the main barrier. IPE was noted as a useful way to incorporate different perspectives into the process.

CONCLUSIONS

Formal image verification training increases resident comfort with setup imaging review and provides opportunities for interprofessional collaboration in radiation oncology residency programs.

The predictive value of segmentation metrics on dosimetry in organs at risk of the brain

BACKGROUND

Fully automatic medical image segmentation has been a long pursuit in radiotherapy (RT). Recent developments involving deep learning show promising results yielding consistent and time efficient contours. In order to train and validate these systems, several geometric based metrics, such as Dice Similarity Coefficient (DSC), Hausdorff, and other related metrics are currently the standard in automated medical image segmentation challenges. However, the relevance of these metrics in RT is questionable. The quality of automated segmentation results needs to reflect clinical relevant treatment outcomes, such as dosimetry and related tumor control and toxicity. In this study, we present results investigating the correlation between popular geometric segmentation metrics and dose parameters for Organs-At-Risk (OAR) in brain tumor patients, and investigate properties that might be predictive for dose changes in brain radiotherapy.

METHODS

A retrospective database of glioblastoma multiforme patients was stratified for planning difficulty, from which 12 cases were selected and reference sets of OARs and radiation targets were defined. In order to assess the relation between segmentation quality -as measured by standard segmentation assessment metrics- and quality of RT plans, clinically realistic, yet alternative contours for each OAR of the selected cases were obtained through three methods: (i) Manual contours by two additional human raters. (ii) Realistic manual manipulations of reference contours. (iii) Through deep learning based segmentation results. On the reference structure set a reference plan was generated that was re-optimized for each corresponding alternative contour set. The correlation between segmentation metrics, and dosimetric changes was obtained and analyzed for each OAR, by means of the mean dose and maximum dose to 1% of the volume (Dmax 1%). Furthermore, we conducted specific experiments to investigate the dosimetric effect of alternative OAR contours with respect to the proximity to the target, size, particular shape and relative location to the target.

RESULTS

We found a low correlation between the DSC, reflecting the alternative OAR contours, and dosimetric changes. The Pearson correlation coefficient between the mean OAR dose effect and the Dice was -0.11. For Dmax 1%, we found a correlation of -0.13. Similar low correlations were found for 22 other segmentation metrics. The organ based analysis showed that there is a better correlation for the larger OARs (i.e. brainstem and eyes) as for the smaller OARs (i.e. optic nerves and chiasm). Furthermore, we found that proximity to the target does not make contour variations more susceptible to the dose effect. However, the direction of the contour variation with respect to the relative location of the target seems to have a strong correlation with the dose effect.

CONCLUSIONS

This study shows a low correlation between segmentation metrics and dosimetric changes for OARs in brain tumor patients. Results suggest that the current metrics for image segmentation in RT, as well as deep learning systems employing such metrics, need to be revisited towards clinically oriented metrics that better reflect how segmentation quality affects dose distribution and related tumor control and toxicity.

Metrics to evaluate the performance of auto-segmentation for radiation treatment planning: A critical review

Advances in artificial intelligence-based methods have led to the development and publication of numerous systems for auto-segmentation in radiotherapy. These systems have the potential to decrease contour variability, which has been associated with poor clinical outcomes and increased efficiency in the treatment planning workflow. However, there are no uniform standards for evaluating auto-segmentation platforms to assess their efficacy at meeting these goals. Here, we review the most frequently used evaluation techniques which include geometric overlap, dosimetric parameters, time spent contouring, and clinical rating scales. These data suggest that many of the most commonly used geometric indices, such as the Dice Similarity Coefficient, are not well correlated with clinically meaningful endpoints. As such, a multi-domain evaluation, including composite geometric and/or dosimetric metrics with physician-reported assessment, is necessary to gauge the clinical readiness of auto-segmentation for radiation treatment planning.

Physician trajectories of abandoning long-course breast radiotherapy and their cost impact

OBJECTIVE

To examine variation in trajectories of abandoning conventionally fractionated whole-breast irradiation (CF-WBI) for adjuvant breast radiotherapy among physician peer groups and the associated cost implications.

DATA SOURCES

Medicare claims data were obtained from the Chronic Conditions Data Warehouse for fee-for-service beneficiaries with breast cancer in 2011-2014.

STUDY DESIGN

We used social network methods to identify peer groups of physicians that shared patients. For each physician peer group in each time period (T1 = 2011-2012 and T2 = 2013-2014), we calculated a risk-adjusted rate of CF-WBI use among eligible women, after adjusting for patient clinical characteristics. We applied a latent class growth analysis to these risk-adjusted rates to identify distinct trajectories of CF-WBI use among physician peer groups. We further estimated potential savings to the Medicare program by accelerating abandonment of CF-WBI in T2 using a simulation model.

DATA COLLECTION/EXTRACTION METHODS

Use of conventionally fractionated whole-breast irradiation was determined from Medicare claims among women ≥ 66 years of age who underwent adjuvant radiotherapy after breast conserving surgery.

PRINCIPAL FINDINGS

Among 215 physician peer groups caring for 16 988 patients, there were four distinct trajectories of abandoning CF-WBI: (a) persistent high use (mean risk-adjusted utilization rate: T1 = 94.3%, T2 = 90.6%); (b) decreased high use (T1 = 81.3%, T2 = 65.3%); (c) decreased medium use (T1 = 60.1%, T2 = 44.0%); and (d) decreased low use (T1 = 31.6%, T2 = 23.6%). Peer groups with a smaller proportion of patients treated at free-standing radiation facilities and a larger proportion of physicians that were surgeons tended to follow trajectories with lower use of CF-WBI. If all physician peer groups had practice patterns in T2 similar to those in the "decreased low use" trajectory, the Medicare program could save $83.3 million (95% confidence interval: $58.5 million-$112.2 million).

CONCLUSIONS

Physician peer groups had distinct trajectories of abandoning CF-WBI. Physician composition and setting of radiotherapy were associated with the different trajectories. Distinct practice patterns across the trajectories had important cost implications.

Challenges in the target volume definition of lung cancer radiotherapy

Radiotherapy, with or without systemic treatment has an important role in the management of lung cancer. In order to deliver the treatment accurately, the clinician must precisely outline the gross tumour volume (GTV), mostly on computed tomography (CT) images. However, due to the limited contrast between tumour and non-malignant changes in the lung tissue, it can be difficult to distinguish the tumour boundaries on CT images leading to large interobserver variation and differences in interpretation. Therefore the definition of the GTV has often been described as the weakest link in radiotherapy with its inaccuracy potentially leading to missing the tumour or unnecessarily irradiating normal tissue. In this article, we review the various techniques that can be used to reduce delineation uncertainties in lung cancer.

A Comprehensive Analysis of a Prospective Multidisciplinary Peer Review Process Before Radiation Therapy Simulation

PURPOSE

Although peer review in radiation oncology (RO) has been recommended to improve quality of care, an analysis of modifications resulting from an RO multidisciplinary presimulation standardized review process has yet to be empirically demonstrated.

METHODS AND MATERIALS

A standardized simulation directive was used for patients undergoing simulation for external beam radiation therapy at a single tertiary care institution. The simulation directives were presented, and all aspects were reviewed by representatives from key RO disciplines. Modifications to the original directives were prospectively captured in a quality improvement registry. Association between key variables and the incidence of modifications were performed using Fisher exact test and t test.

RESULTS

A registry of 500 consecutive simulations for patients undergoing radiation therapy was reviewed. A median of 105 simulations occurred per month. All simulation directives were entered by a physician a median of 3 days before simulation (range, 1-76 days). The treatment intent was curative for 269 patients (53.8%), palliative for 203 patients (40.6%), and benign for 3 patients (0.6%). Twenty-five (5%) patients did not have a treatment intent selected. Based on RO multidisciplinary review, 105 directives (21%) were modified from the original intent, with 29 (5.8%) requiring more than 1 modification. A total of 149 modifications were made and categorized as changes to patient positioning and immobilization (n = 100, 20%), treatment site and care path (n = 34, 6.8%), simulation coordination activities (n = 6, 1.2%), and treatment technique and planning instructions (n = 9, 1.8%). A higher proportion of modifications occurred at the time of multidisciplinary review in patients receiving more complex treatments (intensity modulated radiation therapy/stereotactic radiosurgery/stereotactic body radiation therapy [IMRT/SRS/SBRT] vs 3-dimensional radiation therapy [3DCRT] radiation therapy, 25% vs 16%, P < .025).

CONCLUSIONS

Given the complexity of radiation therapy simulation, standardization of directives with prospective RO multidisciplinary presimulation peer review is critical to optimizing department processes and reducing errors. Approximately 1 in 5 patients benefits from this peer review process, especially patients treated with IMRT/SRS/SBRT.

Development, implementation, and associated challenges of a new HDR brachytherapy program

Developing any new radiation oncology program requires planning and analysis of the current state of the facility and its capacity to take on another program. Staff must consider a large number of factors to establish a feasible, safe, and sustainable program. We present a simple and generic outline that lays out the process for developing and implementing a new HDR brachytherapy program in any setting, but with particular emphasis on challenges associated with starting the program in a limited resource setting. The sections include feasibility of a program, starting cases, machine and equipment selection, and quality and safety.

Impact of Treating Physician on Radiation Therapy Related Severe Toxicities in Men with Prostate Cancer

PURPOSE

The impact of treating physician on radiation therapy (RT) related toxicity is unclear. We carried out a secondary analysis of a randomized controlled study to determine whether the risk of RT-related late toxicities in patients with prostate cancer varies depending on the treating radiation oncologist.

METHODS AND MATERIALS

This is a secondary analysis of a phase 3 randomized controlled study in which patients with prostate cancer with Gleason score ≤7, clinical stage T1b-T3a, and prostate-specific antigen <30 ng/mL were randomized to receive androgen suppression for 6 months, starting either 4 months before or concurrently with definitive prostate radiation therapy. Incidence of late RT-related toxicity was estimated using Kaplan-Meier methods. We applied multivariable semiparametric shared frailty models with gamma distribution to determine the between-physician variation in the hazard of late RT-related grade ≥3 gastrointestinal, genitourinary, or overall toxicity. Patient level covariables included age, risk group, year of enrollment, and treatment regimen. Frailty variance, a measure of unexplained heterogeneity, was estimated with 95% confidence intervals (CIs). Statistical significance was suggested when the lower limit of the 95% CI for the frailty variance was >0. The Commenges-Andersen test was used for P value estimation.

RESULTS

Overall, 426 patients were treated by 9 radiation oncologists. On log-rank test, there was a significant difference in the cumulative incidence of overall grade ≥3 toxicities (P = .001) and grade ≥3 gastrointestinal toxicity (P = .01) among the physician-based clusters. The frailty variance for overall late grade ≥3 toxicity was 0.31 (95% CI, 0.02-1.39; P = .01). The frailty variance for the grade ≥3 gastrointestinal and genitourinary toxicity was 0.84 (95% CI, 0.00-4.20; P = .11) and 0.11 (95% CI, 0.00-1.13; P = .31), respectively.

CONCLUSIONS

In our study, the hazard of overall RT-related late grade ≥3 toxicity varied significantly depending on treating radiation oncologist. Further studies are required to explore the underlying processes that lead to such variations in clinical trials involving radiation therapy in prostate cancer.

Impact of Simulation-Based Training on Radiation Therapists' Workload, Situation Awareness, and Performance

Purpose

This study aimed to assess the impact of simulation-based training intervention on radiation therapy therapist (RTT) mental workload, situation awareness, and performance during routine quality assurance (QA) and treatment delivery tasks.

Methods and Materials

As part of a prospective institutional review board-approved study, 32 RTTs completed routine QA and treatment delivery tasks on clinical scenarios in a simulation laboratory. Participants, randomized to receive (n = 16) versus not receive (n = 16) simulation-based training had pre- and postintervention assessments of mental workload, situation awareness, and performance. We used linear regression models to compare the postassessment scores between the study groups while controlling for baseline scores. Mental workload was quantified subjectively using the NASA Task Load Index. Situation awareness was quantified subjectively using the situation awareness rating technique and objectively using the situation awareness global assessment technique. Performance was quantified based on procedural compliance (adherence to preset/standard QA timeout tasks) and error detection (detection and correction of embedded treatment planning errors).

Results

Simulation-based training intervention was associated with significant improvements in overall performance (P < .01), but had no significant impact on mental workload or subjective/objective quantifications of situation awareness.

Conclusions

Simulation-based training might be an effective tool to improve RTT performance of QA-related tasks.

A Blinded, Prospective Study of Error Detection During Physician Chart Rounds in Radiation Oncology

PURPOSE

Peer review during physician chart rounds is a major quality assurance and patient safety step in radiation oncology. However, the effectiveness of chart rounds in detecting problematic treatment plans is unknown. We performed a prospective blinded study of error detection at chart rounds to clarify the effectiveness of this quality assurance step.

METHODS AND MATERIALS

Radiation Oncology Incident Learning System publications were queried for problematic plans approved for treatment that would be detectable at chart rounds. A resident physician, physicist, and dosimetrist collaboratively generated 20 treatment plans with simulated errors identical in nature to those reported to the Radiation Oncology Incident Learning System. These were inserted randomly into weekly chart rounds over 9 weeks, with a median of 2 problematic plans presented per chart rounds (range, 1-4). Data were collected on detection, attendance, length, and number of cases presented at chart rounds. Data were analyzed using descriptive statistics and univariable logistic regression with odds ratios.

RESULTS

The median length of chart rounds over the study period was 60 minutes (range, 42-79); median number of cases presented per chart rounds was 45 (range, 38-50). The overall detection rate was 55% (11 of 20). Detection rates were higher for cases presented earlier in chart rounds: 75% versus 25% of problematic plans were detected within 30 minutes of start of chart rounds versus after 30 minutes (odds ratio, 0.11; 95% confidence interval, 0.01-0.88; P = .037). Detection rates showed a trend toward increase during the study period but this was not significant: 33% in weeks 1 to 5 and 73% during weeks 6 to 9 (5.3; 95% confidence interval, 0.78-36; P = .08).

CONCLUSIONS

The detection of clinically significant problematic plans during chart rounds could be significantly improved. Problematic plans are more frequently detected earlier in chart rounds and inserting such plans into chart rounds may enhance detection; however, larger studies are needed to confirm these findings. A multi-institutional study is planned.