Utilizing knowledge from prior plans in the evaluation of quality assurance

Neural Network Model Based on Branch Architecture for the Quality Assurance of Volumetric Modulated Arc Therapy

Radiation therapy relies on quality assurance (QA) to verify dose delivery accuracy. However, current QA methods suffer from operation lag as well as inaccurate performance. Hence, to address these shortcomings, this paper proposes a QA neural network model based on branch architecture, which is based on the analysis of the category features of the QA complexity metrics. The designed branch network focuses on category features, which effectively improves the feature extraction capability for complexity metrics. The branch features extracted by the model are fused to predict the GPR for more accurate QA. The performance of the proposed method was validated on the collected dataset. The experiments show that the prediction performance of the model outperforms other QA methods; the average prediction errors for the test set are 2.12% (2%/2 mm), 1.69% (3%/2 mm), and 1.30% (3%/3 mm). Moreover, the results indicate that two-thirds of the validation samples' model predictions perform better than the clinical evaluation results, suggesting that the proposed model can assist physicists in the clinic.

Implementation of a knowledge-based decision support system for treatment plan auditing through automation

BACKGROUND

Independent auditing is a necessary component of a comprehensive quality assurance (QA) program and can also be utilized for continuous quality improvement (QI) in various radiotherapy processes. Two senior physicists at our institution have been performing a time intensive manual audit of cross-campus treatment plans annually, with the aim of further standardizing our planning procedures, updating policies and guidelines, and providing training opportunities of all staff members.

PURPOSE

A knowledge-based automated anomaly-detection algorithm to provide decision support and strengthen our manual retrospective plan auditing process was developed. This standardized and improved the efficiency of the assessment of our external beam radiotherapy (EBRT) treatment planning across all eight campuses of our institution.

METHODS

A total of 843 external beam radiotherapy plans for 721 lung patients from January 2020 to March 2021 were automatically acquired from our clinical treatment planning and management systems. From each plan, 44 parameters were automatically extracted and pre-processed. A knowledge-based anomaly detection algorithm, namely, "isolation forest" (iForest), was then applied to the plan dataset. An anomaly score was determined for each plan using recursive partitioning mechanism. Top 20 plans ranked with the highest anomaly scores for each treatment technique (2D/3D/IMRT/VMAT/SBRT) including auto-populated parameters were used to guide the manual auditing process and validated by two plan auditors.

RESULTS

The two auditors verified that 75.6% plans with the highest iForest anomaly scores have similar concerning qualities that may lead to actionable recommendations for our planning procedures and staff training materials. The time to audit a chart was approximately 20.8 min on average when done manually and 14.0 min when done with the iForest guidance. Approximately 6.8 min were saved per chart with the iForest method. For our typical internal audit review of 250 charts annually, the total time savings are approximately 30 hr per year.

CONCLUSION

iForest effectively detects anomalous plans and strengthens our cross-campus manual plan auditing procedure by adding decision support and further improve standardization. Due to the use of automation, this method was efficient and will be used to establish a standard plan auditing procedure, which could occur more frequently.

A clinically relevant online patient QA solution with daily CT scans and EPID-based in vivo dosimetry: a feasibility study on rectal cancer

Objective. Adaptive radiation therapy (ART) could protect organs at risk (OARs) while maintain high dose coverage to targets. However, there is still a lack of efficient online patient quality assurance (QA) methods, which is an obstacle to large-scale adoption of ART. We aim to develop a clinically relevant online patient QA solution for ART using daily CT scans and EPID-based in vivo dosimetry. Approach. Ten patients with rectal cancer at our center were included. Patients’ daily CT scans and portal images were collected to generate reconstructed 3D dose distributions. Contours of targets and OARs were recontoured on these daily CT scans by a clinician or an auto-segmentation algorithm, then dose-volume indices were calculated, and the percent deviation of these indices to their original plans were determined. This deviation was regarded as the metric for clinically relevant patient QA. The tolerance level was obtained using a 95% confidence interval of the QA metric distribution. These deviations could be further divided into anatomically relevant or delivery relevant indicators for error source analysis. Finally, our QA solution was validated on an additional six clinical patients. Main results. In rectal cancer, the 95% confidence intervals of the QA metric for PTV ΔD 95 (%) were [−3.11%, 2.35%], and for PTV ΔD 2 (%) were [−0.78%, 3.23%]. In validation, 68% for PTV ΔD 95 (%), and 79% for PTV ΔD 2 (%) of the 28 fractions are within tolerances of the QA metrics. one patient’s dosimetric impact of anatomical variations during treatment were observed through the source of error analysis. Significance. The online patient QA solution using daily CT scans and EPID-based in vivo dosimetry is clinically feasible. Source of error analysis has the potential for distinguishing sources of error and guiding ART for future treatments.

Automatic Segmentation of Clinical Target Volumes for Post-Modified Radical Mastectomy Radiotherapy Using Convolutional Neural Networks

Background

This study aims to construct and validate a model based on convolutional neural networks (CNNs), which can fulfil the automatic segmentation of clinical target volumes (CTVs) of breast cancer for radiotherapy.

Methods

In this work, computed tomography (CT) scans of 110 patients who underwent modified radical mastectomies were collected. The CTV contours were confirmed by two experienced oncologists. A novel CNN was constructed to automatically delineate the CTV. Quantitative evaluation metrics were calculated, and a clinical evaluation was conducted to evaluate the performance of our model.

Results

The mean Dice similarity coefficient (DSC) of the proposed model was 0.90, and the 95th percentile Hausdorff distance (95HD) was 5.65 mm. The evaluation results of the two clinicians showed that 99.3% of the chest wall CTV slices could be accepted by clinician A, and this number was 98.9% for clinician B. In addition, 9/10 of patients had all slices accepted by clinician A, while 7/10 could be accepted by clinician B. The score differences between the AI (artificial intelligence) group and the GT (ground truth) group showed no statistically significant difference for either clinician. However, the score differences in the AI group were significantly different between the two clinicians. The Kappa consistency index was 0.259. It took 3.45 s to delineate the chest wall CTV using the model.

Conclusion

Our model could automatically generate the CTVs for breast cancer. AI-generated structures of the proposed model showed a trend that was comparable, or was even better, than those of human-generated structures. Additional multicentre evaluations should be performed for adequate validation before the model can be completely applied in clinical practice.

Artificial Intelligence and the Medical Physicist: Welcome to the Machine

Artificial intelligence (AI) is a branch of computer science dedicated to giving machines or computers the ability to perform human-like cognitive functions, such as learning, problem-solving, and decision making. Since it is showing superior performance than well-trained human beings in many areas, such as image classification, object detection, speech recognition, and decision-making, AI is expected to change profoundly every area of science, including healthcare and the clinical application of physics to healthcare, referred to as medical physics. As a result, the Italian Association of Medical Physics (AIFM) has created the “AI for Medical Physics” (AI4MP) group with the aims of coordinating the efforts, facilitating the communication, and sharing of the knowledge on AI of the medical physicists (MPs) in Italy. The purpose of this review is to summarize the main applications of AI in medical physics, describe the skills of the MPs in research and clinical applications of AI, and define the major challenges of AI in healthcare.

A knowledge-based quantitative approach to characterize treatment plan quality: Application to prostate VMAT planning

PURPOSE

To characterize treatment plan (TP) quality, a quantitative quality control (QC) tool is proposed. The tool is validated using volumetric modulated arc therapy (VMAT) plans for treatment of prostate cancer by estimating the achievable organ at risk (OAR) sparing, based on the knowledge learned from prior plans.

METHODS

Prostate TP quality was investigated by evaluating the achieved OAR sparing in the rectum and bladder, based on their proximity to target surface. The knowledge base used in this work comprises 450 plans, consisting of 181 homogenous prostate plans and 269 simultaneous integrated boost (SIB) prostate plans. A knowledge-based algorithm was used to relate the absorbed doses of the OARs (rectum and bladder) and their proximity to the planning target volume (PTV). A metric (M_q,r value) was calculated to characterize the OAR sparing based on the weighted differences of the mean doses at binned distances to the PTV surface. The 90% probability ellipse of the normally distributed OARs M_q,r values was considered to define a threshold above which the treatment plan was re-optimized.

RESULTS

Following re-optimization, 8/11 of the homogenous plans and 6/13 of the SIB plans outside the 90% probability ellipse could be re-optimized to gain better OAR sparing while achieving the same or better target coverage. However, 3/4 of the homogenous TPs and 1/9 of the SIB TPs between 80% and 90% were improved. M_q,r values of bladder and rectum after re-optimizing the plans in both groups of homogenous and SIB showed lower values compared to the corresponding values before re-optimization, which implies that better OARs sparing was achieved.

CONCLUSIONS

This work demonstrates an effective anatomy-specific QC tool for identifying suboptimal plans and determining the achievable OAR sparing for each individual patient anatomy.

Implementation of an optimization method for parotid gland sparing during inverse planning for head and neck cancer radiotherapy

PURPOSE

To guide parotid gland (PG) sparing at the dose planning step, a specific model based on overlap between PTV and organ at risk (Moore et al.) was developed and evaluated for VMAT in head-and-neck (H&N) cancer radiotherapy.

MATERIALS AND METHODS

One hundred and sixty patients treated for locally advanced H&N cancer were included. A model optimization was first performed (20 patients) before a model evaluation (110 patients). Thirty cases were planned with and without the model to quantify the PG dose sparing. The inter-operator variability was evaluated on one case, planned by 12 operators with and without the model. The endpoints were PG mean dose (D_mean), PTV homogeneity and number of monitor units (MU).

RESULTS

The PG D_mean predicted by the model was reached in 89% of cases. Using the model significantly reduced the PG D_mean: -6.1±4.3Gy. Plans with the model showed lower PTV dose homogeneity and more MUs (+10.5% on average). For the inter-operator variability, PG dose volume histograms without the optimized model were significantly different compared to those with the model; the D_mean standard deviation for the ipsilateral PG decreased from 2.2Gy to 1.2Gy. For the contralateral PG, this value decreased from 2.9Gy to 0.8Gy.

CONCLUSION

During the H&N inverse planning, the optimized model guides to the lowest PG achievable mean dose, allowing a significant PG mean dose reduction of -6.1Gy. Integrating this method at the treatment-planning step significantly reduced the inter-patient and inter-operator variabilities.

Optimizing efficiency and safety in external beam radiotherapy using automated plan check (APC) tool and six sigma methodology

PURPOSE

To develop and implement an automated plan check (APC) tool using a Six Sigma methodology with the aim of improving safety and efficiency in external beam radiotherapy.

METHODS

The Six Sigma define-measure-analyze-improve-control (DMAIC) framework was used by measuring defects stemming from treatment planning that were reported to the departmental incidence learning system (ILS). The common error pathways observed in the reported data were combined with our departmental physics plan check list, and AAPM TG-275 identified items. Prioritized by risk priority number (RPN) and severity values, the check items were added to the APC tool developed using Varian Eclipse Scripting Application Programming Interface (ESAPI). At 9 months post-APC implementation, the tool encompassed 89 check items, and its effectiveness was evaluated by comparing RPN values and rates of reported errors. To test the efficiency gains, physics plan check time and reported error rate were prospectively compared for 20 treatment plans.

RESULTS

The APC tool was successfully implemented for external beam plan checking. FMEA RPN ranking re-evaluation at 9 months post-APC demonstrated a statistically significant average decrease in RPN values from 129.2 to 83.7 (P < .05). After the introduction of APC, the average frequency of reported treatment-planning errors was reduced from 16.1% to 4.1%. For high-severity errors, the reduction was 82.7% for prescription/plan mismatches and 84.4% for incorrect shift note. The process shifted from 4σ to 5σ quality for isocenter-shift errors. The efficiency study showed a statistically significant decrease in plan check time (10.1 ± 7.3 min, P = .005) and decrease in errors propagating to physics plan check (80%).

CONCLUSIONS

Incorporation of APC tool has significantly reduced the error rate. The DMAIC framework can provide an iterative and robust workflow to improve the efficiency and quality of treatment planning procedure enabling a safer radiotherapy process.

Applications and limitations of machine learning in radiation oncology

Machine learning approaches to problem-solving are growing rapidly within healthcare, and radiation oncology is no exception. With the burgeoning interest in machine learning comes the significant risk of misaligned expectations as to what it can and cannot accomplish. This paper evaluates the role of machine learning and the problems it solves within the context of current clinical challenges in radiation oncology. The role of learning algorithms within the workflow for external beam radiation therapy are surveyed, considering simulation imaging, multimodal fusion, image segmentation, treatment planning, quality assurance, and treatment delivery and adaptation. For each aspect, the clinical challenges faced, the learning algorithms proposed, and the successes and limitations of various approaches are analyzed. It is observed that machine learning has largely thrived on reproducibly mimicking conventional human-driven solutions with more efficiency and consistency. On the other hand, since algorithms are generally trained using expert opinion as ground truth, machine learning is of limited utility where problems or ground truths are not well-defined, or if suitable measures of correctness are not available. As a result, machines may excel at replicating, automating and standardizing human behaviour on manual chores, meanwhile the conceptual clinical challenges relating to definition, evaluation, and judgement remain in the realm of human intelligence and insight.

Knowledge-based planning for intensity-modulated radiation therapy: A review of data-driven approaches

Purpose

Intensity‐Modulated Radiation Therapy (IMRT), including its variations (including IMRT, Volumetric Arc Therapy (VMAT), and Tomotherapy), is a widely used and critically important technology for cancer treatment. It is a knowledge‐intensive technology due not only to its own technical complexity, but also to the inherently conflicting nature of maximizing tumor control while minimizing normal organ damage. As IMRT experience and especially the carefully designed clinical plan data are accumulated during the past two decades, a new set of methods commonly termed knowledge‐based planning (KBP) have been developed that aim to improve the quality and efficiency of IMRT planning by learning from the database of past clinical plans. Some of this development has led to commercial products recently that allowed the investigation of KBP in numerous clinical applications. In this literature review, we will attempt to present a summary of published methods of knowledge‐based approaches in IMRT and recent clinical validation results.

Methods

In March 2018, a literature search was conducted in the NIH Medline database using the PubMed interface to identify publications that describe methods and validations related to KBP in IMRT including variations such as VMAT and Tomotherapy. The search criteria were designed to have a broad scope to capture relevant results with high sensitivity. The authors filtered down the search results according to a predefined selection criteria by reviewing the titles and abstracts first and then by reviewing the full text. A few papers were added to the list based on the references of the reviewed papers. The final set of papers was reviewed and summarized here.

Results

The initial search yielded a total of 740 articles. A careful review of the titles, abstracts, and eventually the full text and then adding relevant articles from reviewing the references resulted in a final list of 73 articles published between 2011 and early 2018. These articles described methods for developing knowledge models for predicting such parameters as dosimetric and dose‐volume points, voxel‐level doses, and objective function weights that improve or automate IMRT planning for various cancer sites, addressing different clinical and quality assurance needs, and using a variety of machine learning approaches. A number of articles reported carefully designed clinical studies that assessed the performance of KBP models in realistic clinical applications. Overwhelming majority of the studies demonstrated the benefits of KBP in achieving comparable and often improved quality of IMRT planning while reducing planning time and plan quality variation.

Conclusions

The number of KBP‐related studies has been steadily increasing since 2011 indicating a growing interest in applying this approach to clinical applications. Validation studies have generally shown KBP to produce plans with quality comparable to expert planners while reducing the time and efforts to generate plans. However, current studies are mostly retrospective and leverage relatively small datasets. Larger datasets collected through multi‐institutional collaboration will enable the development of more advanced models to further improve the performance of KBP in complex clinical cases. Prospective studies will be an important next step toward widespread adoption of this exciting technology.

Assessing the Role of Artificial Intelligence (AI) in Clinical Oncology: Utility of Machine Learning in Radiotherapy Target Volume Delineation

The fields of radiotherapy and clinical oncology have been rapidly changed by the advances of technology. Improvement in computer processing power and imaging quality heralded precision radiotherapy allowing radiotherapy to be delivered efficiently, safely and effectively for patient benefit. Artificial intelligence (AI) is an emerging field of computer science which uses computer models and algorithms to replicate human-like intelligence and perform specific tasks which offers a huge potential to healthcare. We reviewed and presented the history, evolution and advancement in the fields of radiotherapy, clinical oncology and machine learning. Radiotherapy target delineation is a complex task of outlining tumour and organ at risks volumes to allow accurate delivery of radiotherapy. We discussed the radiotherapy planning, treatment delivery and reviewed how technology can help with this challenging process. We explored the evidence and clinical application of machine learning to radiotherapy. We concluded on the challenges, possible future directions and potential collaborations to achieve better outcome for cancer patients.

Automation in intensity modulated radiotherapy treatment planning-a review of recent innovations

Radiotherapy treatment planning of complex radiotherapy techniques, such as intensity modulated radiotherapy and volumetric modulated arc therapy, is a resource-intensive process requiring a high level of treatment planner intervention to ensure high plan quality. This can lead to variability in the quality of treatment plans and the efficiency in which plans are produced, depending on the skills and experience of the operator and available planning time. Within the last few years, there has been significant progress in the research and development of intensity modulated radiotherapy treatment planning approaches with automation support, with most commercial manufacturers now offering some form of solution. There is a rapidly growing number of research articles published in the scientific literature on the topic. This paper critically reviews the body of publications up to April 2018. The review describes the different types of automation algorithms, including the advantages and current limitations. Also included is a discussion on the potential issues with routine clinical implementation of such software, and highlights areas for future research.

Deep nets vs expert designed features in medical physics: An IMRT QA case study

PURPOSE

The purpose of this study was to compare the performance of Deep Neural Networks against a technique designed by domain experts in the prediction of gamma passing rates for Intensity Modulated Radiation Therapy Quality Assurance (IMRT QA).

METHOD

A total of 498 IMRT plans across all treatment sites were planned in Eclipse version 11 and delivered using a dynamic sliding window technique on Clinac iX or TrueBeam Linacs. Measurements were performed using a commercial 2D diode array, and passing rates for 3%/3 mm local dose/distance-to-agreement (DTA) were recorded. Separately, fluence maps calculated for each plan were used as inputs to a convolution neural network (CNN). The CNNs were trained to predict IMRT QA gamma passing rates using TensorFlow and Keras. A set of model architectures, inspired by the convolutional blocks of the VGG-16 ImageNet model, were constructed and implemented. Synthetic data, created by rotating and translating the fluence maps during training, was created to boost the performance of the CNNs. Dropout, batch normalization, and data augmentation were utilized to help train the model. The performance of the CNNs was compared to a generalized Poisson regression model, previously developed for this application, which used 78 expert designed features.

RESULTS

Deep Neural Networks without domain knowledge achieved comparable performance to a baseline system designed by domain experts in the prediction of 3%/3 mm Local gamma passing rates. An ensemble of neural nets resulted in a mean absolute error (MAE) of 0.70 ± 0.05 and the domain expert model resulted in a 0.74 ± 0.06.

CONCLUSIONS

Convolutional neural networks (CNNs) with transfer learning can predict IMRT QA passing rates by automatically designing features from the fluence maps without human expert supervision. Predictions from CNNs are comparable to a system carefully designed by physicist experts.

Internet-based computer technology on radiotherapy

Recent rapid development of Internet-based computer technologies has made possible many novel applications in radiation dose delivery. However, translational speed of applying these new technologies in radiotherapy could hardly catch up due to the complex commissioning process and quality assurance protocol. Implementing novel Internet-based technology in radiotherapy requires corresponding design of algorithm and infrastructure of the application, set up of related clinical policies, purchase and development of software and hardware, computer programming and debugging, and national to international collaboration. Although such implementation processes are time consuming, some recent computer advancements in the radiation dose delivery are still noticeable. In this review, we will present the background and concept of some recent Internet-based computer technologies such as cloud computing, big data processing and machine learning, followed by their potential applications in radiotherapy, such as treatment planning and dose delivery. We will also discuss the current progress of these applications and their impacts on radiotherapy. We will explore and evaluate the expected benefits and challenges in implementation as well.

IMRT QA using machine learning: A multi-institutional validation

Purpose

To validate a machine learning approach to Virtual intensity‐modulated radiation therapy (IMRT) quality assurance (QA) for accurately predicting gamma passing rates using different measurement approaches at different institutions.

Methods

A Virtual IMRT QA framework was previously developed using a machine learning algorithm based on 498 IMRT plans, in which QA measurements were performed using diode‐array detectors and a 3%local/3 mm with 10% threshold at Institution 1. An independent set of 139 IMRT measurements from a different institution, Institution 2, with QA data based on portal dosimetry using the same gamma index, was used to test the mathematical framework. Only pixels with ≥10% of the maximum calibrated units (CU) or dose were included in the comparison. Plans were characterized by 90 different complexity metrics. A weighted poison regression with Lasso regularization was trained to predict passing rates using the complexity metrics as input.

Results

The methodology predicted passing rates within 3% accuracy for all composite plans measured using diode‐array detectors at Institution 1, and within 3.5% for 120 of 139 plans using portal dosimetry measurements performed on a per‐beam basis at Institution 2. The remaining measurements (19) had large areas of low CU, where portal dosimetry has a larger disagreement with the calculated dose and as such, the failure was expected. These beams need further modeling in the treatment planning system to correct the under‐response in low‐dose regions. Important features selected by Lasso to predict gamma passing rates were as follows: complete irradiated area outline (CIAO), jaw position, fraction of MLC leafs with gaps smaller than 20 or 5 mm, fraction of area receiving less than 50% of the total CU, fraction of the area receiving dose from penumbra, weighted average irregularity factor, and duty cycle.

Conclusions

We have demonstrated that Virtual IMRT QA can predict passing rates using different measurement techniques and across multiple institutions. Prediction of QA passing rates can have profound implications on the current IMRT process.