1
|
Cheung MY, Netherton TJ, Court LE, Veeraraghavan A, Balakrishnan G. Metric-guided Image Reconstruction Bounds via Conformal Prediction. ARXIV 2024:arXiv:2404.15274v1. [PMID: 38711427 PMCID: PMC11071610] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Subscribe] [Scholar Register] [Indexed: 05/08/2024]
Abstract
Recent advancements in machine learning have led to novel imaging systems and algorithms that address ill-posed problems. Assessing their trustworthiness and understanding how to deploy them safely at test time remains an important and open problem. We propose a method that leverages conformal prediction to retrieve upper/lower bounds and statistical inliers/outliers of reconstructions based on the prediction intervals of downstream metrics. We apply our method to sparse-view CT for downstream radiotherapy planning and show 1) that metric-guided bounds have valid coverage for downstream metrics while conventional pixel-wise bounds do not and 2) anatomical differences of upper/lower bounds between metric-guided and pixel-wise methods. Our work paves the way for more meaningful reconstruction bounds. Code available at https://github.com/matthewyccheung/conformal-metric.
Collapse
Affiliation(s)
- Matt Y Cheung
- Department of Electrical and Computer Engineering, Rice University
| | - Tucker J Netherton
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center
| | - Laurence E Court
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center
| | | | | |
Collapse
|
2
|
Hernandez S, Burger H, Nguyen C, Paulino AC, Lucas JT, Faught AM, Duryea J, Netherton T, Rhee DJ, Cardenas C, Howell R, Fuentes D, Pollard-Larkin J, Court L, Parkes J. Validation of an automated contouring and treatment planning tool for pediatric craniospinal radiation therapy. Front Oncol 2023; 13:1221792. [PMID: 37810961 PMCID: PMC10556471 DOI: 10.3389/fonc.2023.1221792] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2023] [Accepted: 09/06/2023] [Indexed: 10/10/2023] Open
Abstract
Purpose Treatment planning for craniospinal irradiation (CSI) is complex and time-consuming, especially for resource-constrained centers. To alleviate demanding workflows, we successfully automated the pediatric CSI planning pipeline in previous work. In this work, we validated our CSI autosegmentation and autoplanning tool on a large dataset from St. Jude Children's Research Hospital. Methods Sixty-three CSI patient CT scans were involved in the study. Pre-planning scripts were used to automatically verify anatomical compatibility with the autoplanning tool. The autoplanning pipeline generated 15 contours and a composite CSI treatment plan for each of the compatible test patients (n=51). Plan quality was evaluated quantitatively with target coverage and dose to normal tissue metrics and qualitatively with physician review, using a 5-point Likert scale. Three pediatric radiation oncologists from 3 institutions reviewed and scored 15 contours and a corresponding composite CSI plan for the final 51 test patients. One patient was scored by 3 physicians, resulting in 53 plans scored total. Results The algorithm automatically detected 12 incompatible patients due to insufficient junction spacing or head tilt and removed them from the study. Of the 795 autosegmented contours reviewed, 97% were scored as clinically acceptable, with 92% requiring no edits. Of the 53 plans scored, all 51 brain dose distributions were scored as clinically acceptable. For the spine dose distributions, 92%, 100%, and 68% of single, extended, and multiple-field cases, respectively, were scored as clinically acceptable. In all cases (major or minor edits), the physicians noted that they would rather edit the autoplan than create a new plan. Conclusions We successfully validated an autoplanning pipeline on 51 patients from another institution, indicating that our algorithm is robust in its adjustment to differing patient populations. We automatically generated 15 contours and a comprehensive CSI treatment plan for each patient without physician intervention, indicating the potential for increased treatment planning efficiency and global access to high-quality radiation therapy.
Collapse
Affiliation(s)
- Soleil Hernandez
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX, United States
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, United States
| | - Hester Burger
- Department Medical Physics, Groote Schuur Hospital and University of Cape Town, Cape Town, South Africa
| | - Callistus Nguyen
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, United States
| | - Arnold C. Paulino
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, United States
| | - John T. Lucas
- Department of Radiation Oncology, St. Jude Children’s Research Hospital, Memphis, TN, United States
| | - Austin M. Faught
- Department of Radiation Oncology, St. Jude Children’s Research Hospital, Memphis, TN, United States
| | - Jack Duryea
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, United States
| | - Tucker Netherton
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, United States
| | - Dong Joo Rhee
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, United States
| | - Carlos Cardenas
- Department of Radiation Oncology, University of Alabama at Birmingham, Birmingham, AL, United States
| | - Rebecca Howell
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX, United States
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, United States
| | - David Fuentes
- Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, United States
| | - Julianne Pollard-Larkin
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX, United States
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, United States
| | - Laurence Court
- The University of Texas MD Anderson Cancer Center UTHealth Houston Graduate School of Biomedical Sciences, Houston, TX, United States
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, United States
| | - Jeannette Parkes
- Department of Radiation Oncology, Groote Schuur Hospital and University of Cape Town, Cape Town, South Africa
| |
Collapse
|
3
|
Oh K, Gronberg MP, Netherton TJ, Sengupta B, Cardenas CE, Court LE, Ford EC. A deep-learning-based dose verification tool utilizing fluence maps for a cobalt-60 compensator-based intensity-modulated radiation therapy system. Phys Imaging Radiat Oncol 2023; 26:100440. [PMID: 37342210 PMCID: PMC10277917 DOI: 10.1016/j.phro.2023.100440] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/13/2022] [Revised: 04/15/2023] [Accepted: 04/17/2023] [Indexed: 06/22/2023] Open
Abstract
Background and purpose A novel cobalt-60 compensator-based intensity-modulated radiation therapy (IMRT) system was developed for a resource-limited environment but lacked an efficient dose verification algorithm. The aim of this study was to develop a deep-learning-based dose verification algorithm for accurate and rapid dose predictions. Materials and methods A deep-learning network was employed to predict the doses from static fields related to beam commissioning. Inputs were a cube-shaped phantom, a beam binary mask, and an intersecting volume of the phantom and beam binary mask, while output was a 3-dimensional (3D) dose. The same network was extended to predict patient-specific doses for head and neck cancers using two different approaches. A field-based method predicted doses for each field and combined all calculated doses into a plan, while the plan-based method combined all nine fluences into a plan to predict doses. Inputs included patient computed tomography (CT) scans, binary beam masks, and fluence maps truncated to the patient's CT in 3D. Results For static fields, predictions agreed well with ground truths with average deviations of less than 0.5% for percent depth doses and profiles. Even though the field-based method showed excellent prediction performance for each field, the plan-based method showed better agreement between clinical and predicted dose distributions. The distributed dose deviations for all planned target volumes and organs at risk were within 1.3 Gy. The calculation speed for each case was within two seconds. Conclusions A deep-learning-based dose verification tool can accurately and rapidly predict doses for a novel cobalt-60 compensator-based IMRT system.
Collapse
Affiliation(s)
- Kyuhak Oh
- Department of Radiation Oncology, University of Washington Medical Center, Seattle, WA 98195, USA
- Department of Radiation Physics, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Mary P. Gronberg
- Department of Radiation Physics, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Tucker J. Netherton
- Department of Radiation Physics, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Bishwambhar Sengupta
- Department of Radiation Oncology, University of Washington Medical Center, Seattle, WA 98195, USA
| | - Carlos E. Cardenas
- Department of Radiation Oncology, University of Alabama, Birmingham, AL 35233, USA
| | - Laurence E. Court
- Department of Radiation Physics, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Eric C. Ford
- Department of Radiation Oncology, University of Washington Medical Center, Seattle, WA 98195, USA
| |
Collapse
|
4
|
Xiao Y, Cardenas C, Rhee DJ, Netherton T, Zhang L, Nguyen C, Douglas R, Mumme R, Skett S, Patel T, Trauernicht C, Chung C, Simonds H, Aggarwal A, Court L. Customizable landmark-based field aperture design for automated whole-brain radiotherapy treatment planning. J Appl Clin Med Phys 2023; 24:e13839. [PMID: 36412092 PMCID: PMC10018662 DOI: 10.1002/acm2.13839] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/09/2022] [Revised: 10/18/2022] [Accepted: 10/21/2022] [Indexed: 11/23/2022] Open
Abstract
PURPOSE To develop and evaluate an automated whole-brain radiotherapy (WBRT) treatment planning pipeline with a deep learning-based auto-contouring and customizable landmark-based field aperture design. METHODS The pipeline consisted of the following steps: (1) Auto-contour normal structures on computed tomography scans and digitally reconstructed radiographs using deep learning techniques, (2) locate the landmark structures using the beam's-eye-view, (3) generate field apertures based on eight different landmark rules addressing different clinical purposes and physician preferences. Two parallel approaches for generating field apertures were developed for quality control. The performance of the generated field shapes and dose distributions were compared with the original clinical plans. The clinical acceptability of the plans was assessed by five radiation oncologists from four hospitals. RESULTS The performance of the generated field apertures was evaluated by the Hausdorff distance (HD) and mean surface distance (MSD) from 182 patients' field apertures used in the clinic. The average HD and MSD for the generated field apertures were 16 ± 7 and 7 ± 3 mm for the first approach, respectively, and 17 ± 7 and 7 ± 3 mm, respectively, for the second approach. The differences regarding HD and MSD between the first and the second approaches were 1 ± 2 and 1 ± 3 mm, respectively. A clinical review of the field aperture design, conducted using 30 patients, achieved a 100% acceptance rate for both the first and second approaches, and the plan review achieved a 100% acceptance rate for the first approach and a 93% acceptance rate for the second approach. The average acceptance rate for meeting lens dosimetric recommendations was 80% (left lens) and 77% (right lens) for the first approach, and 70% (both left and right lenses) for the second approach, compared with 50% (left lens) and 53% (right lens) for the clinical plans. CONCLUSION This study provided an automated pipeline with two field aperture generation approaches to automatically generate WBRT treatment plans. Both quantitative and qualitative evaluations demonstrated that our novel pipeline was comparable with the original clinical plans.
Collapse
Affiliation(s)
- Yao Xiao
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Carlos Cardenas
- Department of Radiation Oncology, The University of Alabama-Birmingham, Birmingham, Alabama, USA
| | - Dong Joo Rhee
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Tucker Netherton
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Lifei Zhang
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Callistus Nguyen
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Raphael Douglas
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Raymond Mumme
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Stephen Skett
- Department of Medical Physics, Guy's & St Thomas NHS Foundation Trust, London, UK
| | - Tina Patel
- Department of Medical Physics, Guy's & St Thomas NHS Foundation Trust, London, UK
| | - Chris Trauernicht
- Division of Medical Physics, Stellenbosch University and Tygerberg Academic Hospital, Stellenbosch, South Africa
| | - Caroline Chung
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Hannah Simonds
- Division of Radiation Oncology, Stellenbosch University and Tygerberg Academic Hospital, Stellenbosch, South Africa
| | - Ajay Aggarwal
- Department of Medical Physics, Guy's & St Thomas NHS Foundation Trust, London, UK
| | - Laurence Court
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| |
Collapse
|
5
|
Hernandez S, Nguyen C, Parkes J, Burger H, Rhee DJ, Netherton T, Mumme R, Vega JGDL, Duryea J, Leone A, Paulino AC, Cardenas C, Howell R, Fuentes D, Pollard-Larkin J, Court L. Automating the treatment planning process for 3D-conformal pediatric craniospinal irradiation therapy. Pediatr Blood Cancer 2023; 70:e30164. [PMID: 36591994 DOI: 10.1002/pbc.30164] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/15/2022] [Revised: 11/23/2022] [Accepted: 11/28/2022] [Indexed: 01/03/2023]
Abstract
PURPOSE Pediatric patients with medulloblastoma in low- and middle-income countries (LMICs) are most treated with 3D-conformal photon craniospinal irradiation (CSI), a time-consuming, complex treatment to plan, especially in resource-constrained settings. Therefore, we developed and tested a 3D-conformal CSI autoplanning tool for varying patient lengths. METHODS AND MATERIALS Autocontours were generated with a deep learning model trained:tested (80:20 ratio) on 143 pediatric medulloblastoma CT scans (patient ages: 2-19 years, median = 7 years). Using the verified autocontours, the autoplanning tool generated two lateral brain fields matched to a single spine field, an extended single spine field, or two matched spine fields. Additional spine subfields were added to optimize the corresponding dose distribution. Feathering was implemented (yielding nine to 12 fields) to give a composite plan. Each planning approach was tested on six patients (ages 3-10 years). A pediatric radiation oncologist assessed clinical acceptability of each autoplan. RESULTS The autocontoured structures' average Dice similarity coefficient ranged from .65 to .98. The average V95 for the brain/spinal canal for single, extended, and multi-field spine configurations was 99.9% ± 0.06%/99.9% ± 0.10%, 99.9% ± 0.07%/99.4% ± 0.30%, and 99.9% ± 0.06%/99.4% ± 0.40%, respectively. The average maximum dose across all field configurations to the brainstem, eyes (L/R), lenses (L/R), and spinal cord were 23.7 ± 0.08, 24.1 ± 0.28, 13.3 ± 5.27, and 25.5 ± 0.34 Gy, respectively (prescription = 23.4 Gy/13 fractions). Of the 18 plans tested, all were scored as clinically acceptable as-is or clinically acceptable with minor, time-efficient edits preferred or required. No plans were scored as clinically unacceptable. CONCLUSION The autoplanning tool successfully generated pediatric CSI plans for varying patient lengths in 3.50 ± 0.4 minutes on average, indicating potential for an efficient planning aid in a resource-constrained settings.
Collapse
Affiliation(s)
- Soleil Hernandez
- The University of Texas MD Anderson Cancer Center UTHealth Graduate School of Biomedical Sciences, Houston, Texas, USA.,Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Callistus Nguyen
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Jeannette Parkes
- Department of Radiation Oncology, Groote Schuur Hospital and University of Cape Town, Cape Town, South Africa
| | - Hester Burger
- Department Medical Physics, Groote Schuur Hospital and University of Cape Town, Cape Town, South Africa
| | - Dong Joo Rhee
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Tucker Netherton
- The University of Texas MD Anderson Cancer Center UTHealth Graduate School of Biomedical Sciences, Houston, Texas, USA.,Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Raymond Mumme
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Jean Gumma-De La Vega
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Jack Duryea
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Alexandrea Leone
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Arnold C Paulino
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Carlos Cardenas
- Department of Radiation Oncology, University of Alabama at Birmingham, Birmingham, Alabama, USA
| | - Rebecca Howell
- The University of Texas MD Anderson Cancer Center UTHealth Graduate School of Biomedical Sciences, Houston, Texas, USA.,Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - David Fuentes
- The University of Texas MD Anderson Cancer Center UTHealth Graduate School of Biomedical Sciences, Houston, Texas, USA.,Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Julianne Pollard-Larkin
- The University of Texas MD Anderson Cancer Center UTHealth Graduate School of Biomedical Sciences, Houston, Texas, USA.,Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Laurence Court
- The University of Texas MD Anderson Cancer Center UTHealth Graduate School of Biomedical Sciences, Houston, Texas, USA.,Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| |
Collapse
|
6
|
Mehrens H, Douglas R, Gronberg M, Nealon K, Zhang J, Court L. Statistical process control to monitor use of a web-based autoplanning tool. J Appl Clin Med Phys 2022; 23:e13803. [PMID: 36300872 PMCID: PMC9797174 DOI: 10.1002/acm2.13803] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2022] [Revised: 08/29/2022] [Accepted: 09/02/2022] [Indexed: 01/01/2023] Open
Abstract
PURPOSE To investigate the use of statistical process control (SPC) for quality assurance of an integrated web-based autoplanning tool, Radiation Planning Assistant (RPA). METHODS Automatically generated plans were downloaded and imported into two treatment planning systems (TPSs), RayStation and Eclipse, in which they were recalculated using fixed monitor units. The recalculated plans were then uploaded back to the RPA, and the mean dose differences for each contour between the original RPA and the TPSs plans were calculated. SPC was used to characterize the RPA plans in terms of two comparisons: RayStation TPS versus RPA and Eclipse TPS versus RPA for three anatomical sites, and variations in the machine parameters dosimetric leaf gap (DLG) and multileaf collimator transmission factor (MLC-TF) for two algorithms (Analytical Anisotropic Algorithm [AAA]) and Acuros in the Eclipse TPS. Overall, SPC was used to monitor the process of the RPA, while clinics would still perform their routine patient-specific QA. RESULTS For RayStation, the average mean percent dose differences across all contours were 0.65% ± 1.05%, -2.09% ± 0.56%, and 0.28% ± 0.98% and average control limit ranges were 1.89% ± 1.32%, 2.16% ± 1.31%, and 2.65% ± 1.89% for the head and neck, cervix, and chest wall, respectively. In contrast, Eclipse's average mean percent dose differences across all contours were -0.62% ± 0.34%, 0.32% ± 0.23%, and -0.91% ± 0.98%, while average control limit ranges were 1.09% ± 0.77%, 3.69% ± 2.67%, 2.73% ± 1.86%, respectively. Averaging all contours and removing outliers, a 0% dose difference corresponded with a DLG value of 0.202 ± 0.019 cm and MLC-TF value of 0.020 ± 0.001 for Acuros and a DLG value of 0.135 ± 0.031 cm and MLC-TF value of 0.015 ± 0.001 for AAA. CONCLUSIONS Differences in mean dose and control limits between RPA and two separately commissioned TPSs were determined. With varying control limits and means, SPC provides a flexible and useful process quality assurance tool for monitoring a complex automated system such as the RPA.
Collapse
Affiliation(s)
- Hunter Mehrens
- Department of Radiation PhysicsThe University of Texas MD Anderson Cancer CenterHoustonTexasUSA,The University of Texas MD Anderson Graduate School of Biomedical ScienceHoustonTexasUSA
| | - Raphael Douglas
- Department of Radiation PhysicsThe University of Texas MD Anderson Cancer CenterHoustonTexasUSA
| | - Mary Gronberg
- Department of Radiation PhysicsThe University of Texas MD Anderson Cancer CenterHoustonTexasUSA,The University of Texas MD Anderson Graduate School of Biomedical ScienceHoustonTexasUSA
| | - Kelly Nealon
- Department of Radiation PhysicsThe University of Texas MD Anderson Cancer CenterHoustonTexasUSA,The University of Texas MD Anderson Graduate School of Biomedical ScienceHoustonTexasUSA
| | - Joy Zhang
- Department of Radiation PhysicsThe University of Texas MD Anderson Cancer CenterHoustonTexasUSA
| | - Laurence Court
- Department of Radiation PhysicsThe University of Texas MD Anderson Cancer CenterHoustonTexasUSA
| |
Collapse
|
7
|
Isaksson LJ, Summers P, Bhalerao A, Gandini S, Raimondi S, Pepa M, Zaffaroni M, Corrao G, Mazzola GC, Rotondi M, Lo Presti G, Haron Z, Alessi S, Pricolo P, Mistretta FA, Luzzago S, Cattani F, Musi G, De Cobelli O, Cremonesi M, Orecchia R, Marvaso G, Petralia G, Jereczek-Fossa BA. Quality assurance for automatically generated contours with additional deep learning. Insights Imaging 2022; 13:137. [PMID: 35976491 PMCID: PMC9385913 DOI: 10.1186/s13244-022-01276-7] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2022] [Accepted: 07/24/2022] [Indexed: 11/10/2022] Open
Abstract
OBJECTIVE Deploying an automatic segmentation model in practice should require rigorous quality assurance (QA) and continuous monitoring of the model's use and performance, particularly in high-stakes scenarios such as healthcare. Currently, however, tools to assist with QA for such models are not available to AI researchers. In this work, we build a deep learning model that estimates the quality of automatically generated contours. METHODS The model was trained to predict the segmentation quality by outputting an estimate of the Dice similarity coefficient given an image contour pair as input. Our dataset contained 60 axial T2-weighted MRI images of prostates with ground truth segmentations along with 80 automatically generated segmentation masks. The model we used was a 3D version of the EfficientDet architecture with a custom regression head. For validation, we used a fivefold cross-validation. To counteract the limitation of the small dataset, we used an extensive data augmentation scheme capable of producing virtually infinite training samples from a single ground truth label mask. In addition, we compared the results against a baseline model that only uses clinical variables for its predictions. RESULTS Our model achieved a mean absolute error of 0.020 ± 0.026 (2.2% mean percentage error) in estimating the Dice score, with a rank correlation of 0.42. Furthermore, the model managed to correctly identify incorrect segmentations (defined in terms of acceptable/unacceptable) 99.6% of the time. CONCLUSION We believe that the trained model can be used alongside automatic segmentation tools to ensure quality and thus allow intervention to prevent undesired segmentation behavior.
Collapse
Affiliation(s)
| | - Paul Summers
- Division of Radiology, IEO European Institute of Oncology IRCCS, Milan, Italy
| | - Abhir Bhalerao
- Department of Computer Science, University of Warwick, Coventry, Warwick, CV4 7AL, UK
| | - Sara Gandini
- Molecular and Pharmaco-Epidemiology Unit, Department of Experimental Oncology, IEO European Institute of Oncology IRCCS, Milan, Italy
| | - Sara Raimondi
- Molecular and Pharmaco-Epidemiology Unit, Department of Experimental Oncology, IEO European Institute of Oncology IRCCS, Milan, Italy
| | - Matteo Pepa
- Division of Radiation Oncology, IEO European Institute of Oncology IRCCS, Milan, Italy
| | - Mattia Zaffaroni
- Division of Radiation Oncology, IEO European Institute of Oncology IRCCS, Milan, Italy
| | - Giulia Corrao
- Division of Radiation Oncology, IEO European Institute of Oncology IRCCS, Milan, Italy.,Department of Oncology and Hemato-Oncology, University of Milan, Milan, Italy
| | - Giovanni Carlo Mazzola
- Division of Radiation Oncology, IEO European Institute of Oncology IRCCS, Milan, Italy.,Department of Oncology and Hemato-Oncology, University of Milan, Milan, Italy
| | - Marco Rotondi
- Division of Radiation Oncology, IEO European Institute of Oncology IRCCS, Milan, Italy.,Department of Oncology and Hemato-Oncology, University of Milan, Milan, Italy
| | - Giuliana Lo Presti
- Molecular and Pharmaco-Epidemiology Unit, Department of Experimental Oncology, IEO European Institute of Oncology IRCCS, Milan, Italy
| | - Zaharudin Haron
- Radiology Department, National Cancer Institute, Putrajaya, Malaysia
| | - Sara Alessi
- Division of Radiology, IEO European Institute of Oncology IRCCS, Milan, Italy
| | - Paola Pricolo
- Division of Radiology, IEO European Institute of Oncology IRCCS, Milan, Italy
| | | | - Stefano Luzzago
- Division of Urology, IEO European Institute of Oncology IRCCS, Milan, Italy
| | - Federica Cattani
- Medical Physics Unit, IEO European Institute of Oncology IRCCS, Milan, Italy
| | - Gennaro Musi
- Department of Oncology and Hemato-Oncology, University of Milan, Milan, Italy.,Division of Urology, IEO European Institute of Oncology IRCCS, Milan, Italy
| | - Ottavio De Cobelli
- Department of Oncology and Hemato-Oncology, University of Milan, Milan, Italy.,Division of Urology, IEO European Institute of Oncology IRCCS, Milan, Italy
| | - Marta Cremonesi
- Radiation Research Unit, IEO European Institute of Oncology IRCCS, Milan, Italy
| | - Roberto Orecchia
- Scientific Direction, IEO European Institute of Oncology IRCCS, Milan, Italy
| | - Giulia Marvaso
- Division of Radiation Oncology, IEO European Institute of Oncology IRCCS, Milan, Italy.,Department of Oncology and Hemato-Oncology, University of Milan, Milan, Italy
| | - Giuseppe Petralia
- Department of Oncology and Hemato-Oncology, University of Milan, Milan, Italy.,Precision Imaging and Research Unit, Department of Medical Imaging and Radiation Sciences, IEO European Institute of Oncology IRCCS, Milan, Italy
| | - Barbara Alicja Jereczek-Fossa
- Division of Radiation Oncology, IEO European Institute of Oncology IRCCS, Milan, Italy.,Department of Oncology and Hemato-Oncology, University of Milan, Milan, Italy
| |
Collapse
|
8
|
Rhee DJ, Jhingran A, Huang K, Netherton TJ, Fakie N, White I, Sherriff A, Cardenas CE, Zhang L, Prajapati S, Kry SF, Beadle BM, Shaw W, O'Reilly F, Parkes J, Burger H, Trauernicht C, Simonds H, Court LE. Clinical acceptability of fully automated external beam radiotherapy for cervical cancer with three different beam delivery techniques. Med Phys 2022; 49:5742-5751. [PMID: 35866442 PMCID: PMC9474595 DOI: 10.1002/mp.15868] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2021] [Revised: 06/16/2022] [Accepted: 07/12/2022] [Indexed: 12/02/2022] Open
Abstract
Purpose To fully automate CT‐based cervical cancer radiotherapy by automating contouring and planning for three different treatment techniques. Methods We automated three different radiotherapy planning techniques for locally advanced cervical cancer: 2D 4‐field‐box (4‐field‐box), 3D conformal radiotherapy (3D‐CRT), and volumetric modulated arc therapy (VMAT). These auto‐planning algorithms were combined with a previously developed auto‐contouring system. To improve the quality of the 4‐field‐box and 3D‐CRT plans, we used an in‐house, field‐in‐field (FIF) automation program. Thirty‐five plans were generated for each technique on CT scans from multiple institutions and evaluated by five experienced radiation oncologists from three different countries. Every plan was reviewed by two of the five radiation oncologists and scored using a 5‐point Likert scale. Results Overall, 87%, 99%, and 94% of the automatically generated plans were found to be clinically acceptable without modification for the 4‐field‐box, 3D‐CRT, and VMAT plans, respectively. Some customizations of the FIF configuration were necessary on the basis of radiation oncologist preference. Additionally, in some cases, it was necessary to renormalize the plan after it was generated to satisfy radiation oncologist preference. Conclusion Approximately, 90% of the automatically generated plans were clinically acceptable for all three planning techniques. This fully automated planning system has been implemented into the radiation planning assistant for further testing in resource‐constrained radiotherapy departments in low‐ and middle‐income countries.
Collapse
Affiliation(s)
- Dong Joo Rhee
- The University of Texas Graduate School of Biomedical Sciences at Houston, Houston, Texas, USA.,Department of Radiation Physics, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Anuja Jhingran
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Kai Huang
- The University of Texas Graduate School of Biomedical Sciences at Houston, Houston, Texas, USA.,Department of Radiation Physics, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Tucker J Netherton
- Department of Radiation Physics, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Nazia Fakie
- Division of Radiation Oncology and Medical Physics, University of Cape Town and Groote Schuur Hospital, Cape Town, South Africa
| | - Ingrid White
- Radiotherapy Department, Guy's and St Thomas' NHS Foundation Trust, London, United Kingdom
| | - Alicia Sherriff
- Department of Oncology, University of the Free State, Bloemfontein, South Africa
| | - Carlos E Cardenas
- Department of Radiation Oncology, The University of Alabama at Birmingham, Birmingham, Alabama, USA
| | - Lifei Zhang
- Department of Radiation Physics, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Surendra Prajapati
- Department of Radiation Physics, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Stephen F Kry
- Department of Radiation Physics, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Beth M Beadle
- Department of Radiation Oncology, Stanford University School of Medicine, Stanford, California, USA
| | - William Shaw
- Department of Medical Physics (G68), University of the Free State, Bloemfontein, South Africa
| | - Frederika O'Reilly
- Department of Medical Physics (G68), University of the Free State, Bloemfontein, South Africa
| | - Jeannette Parkes
- Division of Radiation Oncology and Medical Physics, University of Cape Town and Groote Schuur Hospital, Cape Town, South Africa
| | - Hester Burger
- Division of Radiation Oncology and Medical Physics, University of Cape Town and Groote Schuur Hospital, Cape Town, South Africa
| | - Chris Trauernicht
- Division of Medical Physics, Stellenbosch University, Tygerberg Academic Hospital, Cape Town, South Africa
| | - Hannah Simonds
- Division of Radiation Oncology, Stellenbosch University, Tygerberg Academic Hospital, Cape Town, South Africa
| | - Laurence E Court
- Department of Radiation Physics, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| |
Collapse
|
9
|
Nealon KA, Court LE, Douglas RJ, Zhang L, Han EY. Development and validation of a checklist for use with automatically generated radiotherapy plans. J Appl Clin Med Phys 2022; 23:e13694. [PMID: 35775105 PMCID: PMC9512344 DOI: 10.1002/acm2.13694] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2022] [Revised: 05/19/2022] [Accepted: 05/24/2022] [Indexed: 11/25/2022] Open
Abstract
Purpose To develop a checklist that improves the rate of error detection during the plan review of automatically generated radiotherapy plans. Methods A custom checklist was developed using guidance from American Association of Physicists in Medicine task groups 275 and 315 and the results of a failure modes and effects analysis of the Radiation Planning Assistant (RPA), an automated contouring and treatment planning tool. The preliminary checklist contained 90 review items for each automatically generated plan. In the first study, eight physicists were recruited from our institution who were familiar with the RPA. Each physicist reviewed 10 artificial intelligence‐generated resident treatment plans from the RPA for safety and plan quality, five of which contained errors. Physicists performed plan checks, recorded errors, and rated each plan's clinical acceptability. Following a 2‐week break, physicists reviewed 10 additional plans with a similar distribution of errors using our customized checklist. Participants then provided feedback on the usability of the checklist and it was modified accordingly. In a second study, this process was repeated with 14 senior medical physics residents who were randomly assigned to checklist or no checklist for their reviews. Each reviewed 10 plans, five of which contained errors, and completed the corresponding survey. Results In the first study, the checklist significantly improved the rate of error detection from 3.4 ± 1.1 to 4.4 ± 0.74 errors per participant without and with the checklist, respectively (p = 0.02). Error detection increased by 20% when the custom checklist was utilized. In the second study, 2.9 ± 0.84 and 3.5 ± 0.84 errors per participant were detected without and with the revised checklist, respectively (p = 0.08). Despite the lack of statistical significance for this cohort, error detection increased by 18% when the checklist was utilized. Conclusion Our results indicate that the use of a customized checklist when reviewing automated treatment plans will result in improved patient safety.
Collapse
Affiliation(s)
- Kelly A Nealon
- University of Texas MD Anderson UTHealth Graduate School of Biomedical Sciences, Houston, Texas, USA.,Department of Radiation Physics - Research, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Laurence E Court
- University of Texas MD Anderson UTHealth Graduate School of Biomedical Sciences, Houston, Texas, USA.,Department of Radiation Physics - Patient Care, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Raphael J Douglas
- Department of Radiation Physics - Research, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Lifei Zhang
- Department of Radiation Physics - Research, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Eun Young Han
- University of Texas MD Anderson UTHealth Graduate School of Biomedical Sciences, Houston, Texas, USA.,Department of Radiation Physics - Patient Care, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| |
Collapse
|
10
|
Modiri A, Vogelius I, Rechner LA, Nygård L, Bentzen SM, Specht L. Outcome-based multiobjective optimization of lymphoma radiation therapy plans. Br J Radiol 2021; 94:20210303. [PMID: 34541859 PMCID: PMC8553178 DOI: 10.1259/bjr.20210303] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/05/2021] [Revised: 09/02/2021] [Accepted: 09/06/2021] [Indexed: 02/04/2023] Open
Abstract
At its core, radiation therapy (RT) requires balancing therapeutic effects against risk of adverse events in cancer survivors. The radiation oncologist weighs numerous disease and patient-level factors when considering the expected risk-benefit ratio of combined treatment modalities. As part of this, RT plan optimization software is used to find a clinically acceptable RT plan delivering a prescribed dose to the target volume while respecting pre-defined radiation dose-volume constraints for selected organs at risk. The obvious limitation to the current approach is that it is virtually impossible to ensure the selected treatment plan could not be bettered by an alternative plan providing improved disease control and/or reduced risk of adverse events in this individual. Outcome-based optimization refers to a strategy where all planning objectives are defined by modeled estimates of a specific outcome's probability. Noting that various adverse events and disease control are generally incommensurable, leads to the concept of a Pareto-optimal plan: a plan where no single objective can be improved without degrading one or more of the remaining objectives. Further benefits of outcome-based multiobjective optimization are that quantitative estimates of risks and benefit are obtained as are the effects of choosing a different trade-off between competing objectives. Furthermore, patient-level risk factors and combined treatment modalities may be integrated directly into plan optimization. Here, we present this approach in the clinical setting of multimodality therapy for malignant lymphoma, a malignancy with marked heterogeneity in biology, target localization, and patient characteristics. We discuss future research priorities including the potential of artificial intelligence.
Collapse
Affiliation(s)
- Arezoo Modiri
- Department of Radiation Oncology, University of Maryland, School of Medicine, Baltimore, MD, USA
| | - Ivan Vogelius
- Department of Oncology, Section of Radiotherapy, Rigshospitalet, University of Copenhagen, Copenhagen, Denmark
| | - Laura Ann Rechner
- Department of Oncology, Section of Radiotherapy, Rigshospitalet, University of Copenhagen, Copenhagen, Denmark
| | - Lotte Nygård
- Department of Oncology, Section of Radiotherapy, Rigshospitalet, University of Copenhagen, Copenhagen, Denmark
| | - Søren M Bentzen
- Department of Epidemiology and Public Health, University of Maryland, School of Medicine, Baltimore, MD, USA
| | - Lena Specht
- Department of Oncology, Section of Radiotherapy, Rigshospitalet, University of Copenhagen, Copenhagen, Denmark
| |
Collapse
|
11
|
Zhou P, Li X, Zhou H, Fu X, Liu B, Zhang Y, Lin S, Pang H. Support Vector Machine Model Predicts Dose for Organs at Risk in High-Dose Rate Brachytherapy of Cervical Cancer. Front Oncol 2021; 11:619384. [PMID: 34336640 PMCID: PMC8319952 DOI: 10.3389/fonc.2021.619384] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/20/2020] [Accepted: 06/25/2021] [Indexed: 11/13/2022] Open
Abstract
Introduction This study aimed to establish a support vector machine (SVM) model to predict the dose for organs at risk (OARs) in intracavitary brachytherapy planning for cervical cancer with tandem and ovoid treatments. Methods Fifty patients with loco-regionally advanced cervical cancer treated with 200 CT-based tandem and ovoid brachytherapy plans were included. The brachytherapy plans were randomly divided into the training (N = 160) and verification groups (N = 40). The bladder, rectum, sigmoid colon, and small intestine were divided into sub-OARs. The SVM model was established using MATLAB software based on the sub-OAR volume to predict the bladder, rectum, sigmoid colon, and small intestine D 2 c m 3 . Model performance was quantified by mean squared error (MSE) and δ ( δ = | D 2 c m 3 / D prescription ( actual ) - D 2 c m 3 / D prescription ( predicted ) | ) . The goodness of fit of the model was quantified by the coefficient of determination (R2). The accuracy and validity of the SVM model were verified using the validation group. Results The D 2 c m 3 value of the bladder, rectum, sigmoid colon, and small intestine correlated with the volume of the corresponding sub-OARs in the training group. The mean squared error (MSE) in the SVM model training group was <0.05; the R2 of each OAR was >0.9. There was no significant difference between the D 2 c m 3 -predicted and actual values in the validation group (all P > 0.05): bladder δ = 0.024 ± 0.022, rectum δ = 0.026 ± 0.014, sigmoid colon δ = 0.035 ± 0.023, and small intestine δ = 0.032 ± 0.025. Conclusion The SVM model established in this study can effectively predict the D 2 c m 3 for the bladder, rectum, sigmoid colon, and small intestine in cervical cancer brachytherapy.
Collapse
Affiliation(s)
- Ping Zhou
- Department of Radiology, The Affiliated Hospital of Southwest Medical University, Luzhou, China
| | - Xiaojie Li
- Department of Oncology, The Affiliated Hospital of Southwest Medical University, Luzhou, China
| | - Hao Zhou
- Department of Nursing College, Southwest Medical University, Luzhou, China
| | - Xiao Fu
- Department of Nursing College, Southwest Medical University, Luzhou, China
| | - Bo Liu
- Department of Nursing College, Southwest Medical University, Luzhou, China
| | - Yu Zhang
- Department of Nursing College, Southwest Medical University, Luzhou, China
| | - Sheng Lin
- Department of Oncology, The Affiliated Hospital of Southwest Medical University, Luzhou, China
| | - Haowen Pang
- Department of Oncology, The Affiliated Hospital of Southwest Medical University, Luzhou, China
| |
Collapse
|
12
|
Critchfield LS, Visak J, Bernard ME, Randall ME, McGarry RC, Pokhrel D. Automation and integration of a novel restricted single-isocenter stereotactic body radiotherapy (a-RESIST) method for synchronous two lung lesions. J Appl Clin Med Phys 2021; 22:56-65. [PMID: 34032380 PMCID: PMC8292708 DOI: 10.1002/acm2.13259] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2021] [Revised: 03/19/2021] [Accepted: 03/29/2021] [Indexed: 11/12/2022] Open
Abstract
Synchronous treatment of two lung lesions using a single-isocenter volumetric modulated arc therapy (VMAT) stereotactic body radiation therapy (SBRT) plan can decrease treatment time and reduce the impact of intrafraction motion. However, alignment of both lesions on a single cone beam CT (CBCT) can prove difficult and may lead to setup errors and unacceptable target coverage loss. A Restricted Single-Isocenter Stereotactic Body Radiotherapy (RESIST) method was created to minimize setup uncertainties and provide treatment delivery flexibility. RESIST utilizes a single-isocenter placed at patient's midline and allows both lesions to be planned separately but treated in the same session. Herein is described a process of automation of this novel RESIST method. Automation of RESIST significantly reduced treatment planning time while maintaining the benefits of RESIST. To demonstrate feasibility, ten patients with two lung lesions previously treated with a single-isocenter clinical VMAT plan were replanned manually with RESIST (m-RESIST) and with automated RESIST (a-RESIST). a-RESIST method automatically sets isocenter, creates beam geometry, chooses appropriate dose calculation algorithms, and performs VMAT optimization using an in-house trained knowledge-based planning model for lung SBRT. Both m-RESIST and a-RESIST showed lower dose to normal tissues compared to manually planned clinical VMAT although a-RESIST provided slightly inferior, but still clinically acceptable, dose conformity and gradient indices. However, a-RESIST significantly reduced the treatment planning time to less than 20 min and provided a higher dose to the lung tumors. The a-RESIST method provides guidance for inexperienced planners by standardizing beam geometry and plan optimization using DVH estimates. It produces clinically acceptable two lesions VMAT lung SBRT plans efficiently. We have further validated a-RESIST on phantom measurement and independent pretreatment dose verification of another four selected 2-lesions lung SBRT patients and implemented clinically. Further development of a-RESIST for more than two lung lesions and refining this approach for extracranial oligometastastic abdominal/pelvic SBRT, including development of automated simulated collision detection algorithm, merits future investigation.
Collapse
Affiliation(s)
- Lana Sanford Critchfield
- Medical Physics Graduate ProgramDepartment of Radiation MedicineUniversity of KentuckyLexingtonKY40508USA
| | - Justin Visak
- Medical Physics Graduate ProgramDepartment of Radiation MedicineUniversity of KentuckyLexingtonKY40508USA
| | - Mark E Bernard
- Medical Physics Graduate ProgramDepartment of Radiation MedicineUniversity of KentuckyLexingtonKY40508USA
| | - Marcus E Randall
- Medical Physics Graduate ProgramDepartment of Radiation MedicineUniversity of KentuckyLexingtonKY40508USA
| | - Ronald C McGarry
- Medical Physics Graduate ProgramDepartment of Radiation MedicineUniversity of KentuckyLexingtonKY40508USA
| | - Damodar Pokhrel
- Medical Physics Graduate ProgramDepartment of Radiation MedicineUniversity of KentuckyLexingtonKY40508USA
| |
Collapse
|
13
|
Hernandez S, Sjogreen C, Gay SS, Nguyen C, Netherton T, Olanrewaju A, Zhang LJ, Rhee DJ, Méndez JD, Court LE, Cardenas CE. Development and dosimetric assessment of an automatic dental artifact classification tool to guide artifact management techniques in a fully automated treatment planning workflow. Comput Med Imaging Graph 2021; 90:101907. [PMID: 33845433 PMCID: PMC8180493 DOI: 10.1016/j.compmedimag.2021.101907] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2020] [Revised: 02/05/2021] [Accepted: 03/14/2021] [Indexed: 12/03/2022]
Abstract
Purpose: We conducted our study to develop a tool capable of automatically detecting dental artifacts in a CT scan on a slice-by-slice basis and to assess the dosimetric impact of implementing the tool into the Radiation Planning Assistant (RPA), a web-based platform designed to fully automate the radiation therapy treatment planning process. Methods: We developed an automatic dental artifact identification tool and assessed the dosimetric impact of its use in the RPA. Three users manually annotated 83,676 head-and-neck (HN) CT slices (549 patients). Majority-voting was applied to the individual annotations to determine the presence or absence of dental artifacts. The patients were divided into train, cross-validation, and test data sets (ratio: 3:1:1, respectively). A random subset of images without dental artifacts was used to balance classes (1:1) in the training data set. The Inception-V3 deep learning model was trained with the binary cross-entropy loss function. With use of this model, we automatically identified artifacts on 15 RPA HN plans on a slice-by-slice basis and investigated three dental artifact management methods applied before and after volumetric modulated arc therapy (VMAT) plan optimization. The resulting dose distributions and target coverage were quantified. Results: Per-slice accuracy, sensitivity, and specificity were 99 %, 91 %, and 99 %, respectively. The model identified all patients with artifacts. Small dosimetric differences in total plan dose were observed between the various density-override methods (±1 Gy). For the pre- and post-optimized plans, 90 % and 99 %, respectively, of dose comparisons resulted in normal structure dose differences of ±1 Gy. Differences in the volume of structures receiving 95 % of the prescribed dose (V95[%]) were ≤0.25 % for 100 % of plans. Conclusion: The dosimetric impact of applying dental artifact management before and after artifact plan optimization was minor. Our results suggest that not accounting for dental artifacts in the current RPA workflow (where only post-optimization dental artifact management is possible) may result in minor dosimetric differences. If RPA users choose to override CT densities as a solution to managing dental artifacts, our results suggest segmenting the volume of the artifact and overriding its density to water is a safe option.
Collapse
Affiliation(s)
- Soleil Hernandez
- The University of Texas MD Anderson Cancer Center Graduate School of Biomedical Sciences, Houston, TX, USA; The University of Texas MD Anderson Cancer Center, Department of Radiation Physics, Houston, TX, USA.
| | - Carlos Sjogreen
- The University of Texas MD Anderson Cancer Center Graduate School of Biomedical Sciences, Houston, TX, USA; The University of Texas MD Anderson Cancer Center, Department of Radiation Physics, Houston, TX, USA
| | - Skylar S Gay
- The University of Texas MD Anderson Cancer Center, Department of Radiation Physics, Houston, TX, USA
| | - Callistus Nguyen
- The University of Texas MD Anderson Cancer Center, Department of Radiation Physics, Houston, TX, USA
| | - Tucker Netherton
- The University of Texas MD Anderson Cancer Center Graduate School of Biomedical Sciences, Houston, TX, USA; The University of Texas MD Anderson Cancer Center, Department of Radiation Physics, Houston, TX, USA
| | - Adenike Olanrewaju
- The University of Texas MD Anderson Cancer Center, Department of Radiation Physics, Houston, TX, USA
| | - Lifei Joy Zhang
- The University of Texas MD Anderson Cancer Center, Department of Radiation Physics, Houston, TX, USA
| | - Dong Joo Rhee
- The University of Texas MD Anderson Cancer Center Graduate School of Biomedical Sciences, Houston, TX, USA; The University of Texas MD Anderson Cancer Center, Department of Radiation Physics, Houston, TX, USA
| | - José David Méndez
- The University of Texas MD Anderson Cancer Center, Department of Radiation Physics, Houston, TX, USA
| | - Laurence E Court
- The University of Texas MD Anderson Cancer Center Graduate School of Biomedical Sciences, Houston, TX, USA; The University of Texas MD Anderson Cancer Center, Department of Radiation Physics, Houston, TX, USA
| | - Carlos E Cardenas
- The University of Texas MD Anderson Cancer Center Graduate School of Biomedical Sciences, Houston, TX, USA; The University of Texas MD Anderson Cancer Center, Department of Radiation Physics, Houston, TX, USA
| |
Collapse
|
14
|
Huang K, Rhee DJ, Ger R, Layman R, Yang J, Cardenas CE, Court LE. Impact of slice thickness, pixel size, and CT dose on the performance of automatic contouring algorithms. J Appl Clin Med Phys 2021; 22:168-174. [PMID: 33779037 PMCID: PMC8130223 DOI: 10.1002/acm2.13207] [Citation(s) in RCA: 19] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/13/2020] [Revised: 12/12/2020] [Accepted: 01/30/2021] [Indexed: 12/25/2022] Open
Abstract
Purpose To investigate the impact of computed tomography (CT) image acquisition and reconstruction parameters, including slice thickness, pixel size, and dose, on automatic contouring algorithms. Methods Eleven scans from patients with head‐and‐neck cancer were reconstructed with varying slice thicknesses and pixel sizes. CT dose was varied by adding noise using low‐dose simulation software. The impact of these imaging parameters on two in‐house auto‐contouring algorithms, one convolutional neural network (CNN)‐based and one multiatlas‐based system (MACS) was investigated for 183 reconstructed scans. For each algorithm, auto‐contours for organs‐at‐risk were compared with auto‐contours from scans with 3 mm slice thickness, 0.977 mm pixel size, and 100% CT dose using Dice similarity coefficient (DSC), Hausdorff distance (HD), and mean surface distance (MSD). Results Increasing the slice thickness from baseline value of 3 mm gave a progressive reduction in DSC and an increase in HD and MSD on average for all structures. Reducing the CT dose only had a relatively minimal effect on DSC and HD. The rate of change with respect to dose for both auto‐contouring methods is approximately 0. Changes in pixel size had a small effect on DSC and HD for CNN‐based auto‐contouring with differences in DSC being within 0.07. Small structures had larger deviations from the baseline values than large structures for DSC. The relative differences in HD and MSD between the large and small structures were small. Conclusions Auto‐contours can deviate substantially with changes in CT acquisition and reconstruction parameters, especially slice thickness and pixel size. The CNN was less sensitive to changes in pixel size, and dose levels than the MACS. The results contraindicated more restrictive values for the parameters should be used than a typical imaging protocol for head‐and‐neck.
Collapse
Affiliation(s)
- Kai Huang
- The University of Texas MD Anderson Cancer Center UTHealth Graduate School of Biomedical Sciences, Houston, TX, USA.,Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Dong Joo Rhee
- The University of Texas MD Anderson Cancer Center UTHealth Graduate School of Biomedical Sciences, Houston, TX, USA.,Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Rachel Ger
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Rick Layman
- Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Jinzhong Yang
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Carlos E Cardenas
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Laurence E Court
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| |
Collapse
|
15
|
Maffei N, Manco L, Aluisio G, D'Angelo E, Ferrazza P, Vanoni V, Meduri B, Lohr F, Guidi G. Radiomics classifier to quantify automatic segmentation quality of cardiac sub-structures for radiotherapy treatment planning. Phys Med 2021; 83:278-286. [DOI: 10.1016/j.ejmp.2021.05.009] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/15/2020] [Revised: 04/30/2021] [Accepted: 05/03/2021] [Indexed: 12/24/2022] Open
|
16
|
Olanrewaju A, Court LE, Zhang L, Naidoo K, Burger H, Dalvie S, Wetter J, Parkes J, Trauernicht CJ, McCarroll RE, Cardenas C, Peterson CB, Benson KRK, du Toit M, van Reenen R, Beadle BM. Clinical Acceptability of Automated Radiation Treatment Planning for Head and Neck Cancer Using the Radiation Planning Assistant. Pract Radiat Oncol 2021; 11:177-184. [PMID: 33640315 DOI: 10.1016/j.prro.2020.12.003] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2020] [Revised: 11/25/2020] [Accepted: 12/08/2020] [Indexed: 11/29/2022]
Abstract
PURPOSE Radiation treatment planning for head and neck cancer is a complex process with much variability; automated treatment planning is a promising option to improve plan quality and efficiency. This study compared radiation plans generated from a fully automated radiation treatment planning system to plans generated manually that had been clinically approved and delivered. METHODS AND MATERIALS The study cohort consisted of 50 patients treated by a specialized head and neck cancer team at a tertiary care center. An automated radiation treatment planning system, the Radiation Planning Assistant, was used to create autoplans for all patients using their original, approved contours. Common dose-volume histogram (DVH) criteria were used to compare the quality of autoplans to the clinical plans. Fourteen radiation oncologists, each from a different institution, then reviewed and compared the autoplans and clinical plans in a blinded fashion. RESULTS Autoplans and clinical plans were very similar with regard to DVH metrics for coverage and critical structure constraints. Physician reviewers found both the clinical plans and autoplans acceptable for use; overall, 78% of the clinical plans and 88% of the autoplans were found to be usable as is (without any edits). When asked to choose which plan would be preferred for approval, 27% of physician reviewers selected the clinical plan, 47% selected the autoplan, 25% said both were equivalent, and 0% said neither. Hence, overall, 72% of physician reviewers believed the autoplan or either the clinical or autoplan was preferable. CONCLUSIONS Automated radiation treatment planning creates consistent, clinically acceptable treatment plans that meet DVH criteria and are found to be appropriate on physician review.
Collapse
Affiliation(s)
- Adenike Olanrewaju
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, Texas
| | - Laurence E Court
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, Texas
| | - Lifei Zhang
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, Texas
| | - Komeela Naidoo
- Department of Radiation Oncology, Stellenbosch University and Tygerberg Hospital, Cape Town, South Africa
| | - Hester Burger
- Department of Radiation Oncology, Groote Schuur Hospital and University of Cape Town, Cape Town, South Africa
| | - Sameera Dalvie
- Department of Radiation Oncology, Groote Schuur Hospital and University of Cape Town, Cape Town, South Africa
| | - Julie Wetter
- Department of Radiation Oncology, Groote Schuur Hospital and University of Cape Town, Cape Town, South Africa
| | - Jeannette Parkes
- Department of Radiation Oncology, Groote Schuur Hospital and University of Cape Town, Cape Town, South Africa
| | - Christoph J Trauernicht
- Department of Radiation Oncology, Stellenbosch University and Tygerberg Hospital, Cape Town, South Africa
| | - Rachel E McCarroll
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, Texas
| | - Carlos Cardenas
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, Texas
| | - Christine B Peterson
- Department of Biostatistics, University of Texas MD Anderson Cancer Center, Houston, Texas
| | - Kathryn R K Benson
- Department of Radiation Oncology, Stanford University, Stanford, California
| | - Monique du Toit
- Department of Radiation Oncology, Stellenbosch University and Tygerberg Hospital, Cape Town, South Africa
| | - Ricus van Reenen
- Department of Radiation Oncology, Stellenbosch University and Tygerberg Hospital, Cape Town, South Africa
| | - Beth M Beadle
- Department of Radiation Oncology, Stanford University, Stanford, California.
| |
Collapse
|
17
|
Netherton TJ, Rhee DJ, Cardenas CE, Chung C, Klopp AH, Peterson CB, Howell RM, Balter PA, Court LE. Evaluation of a multiview architecture for automatic vertebral labeling of palliative radiotherapy simulation CT images. Med Phys 2020; 47:5592-5608. [PMID: 33459402 PMCID: PMC7756475 DOI: 10.1002/mp.14415] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/10/2019] [Revised: 06/11/2020] [Accepted: 07/12/2020] [Indexed: 12/21/2022] Open
Abstract
PURPOSE The purpose of this work was to evaluate the performance of X-Net, a multiview deep learning architecture, to automatically label vertebral levels (S2-C1) in palliative radiotherapy simulation CT scans. METHODS For each patient CT scan, our automated approach 1) segmented spinal canal using a convolutional-neural network (CNN), 2) formed sagittal and coronal intensity projection pairs, 3) labeled vertebral levels with X-Net, and 4) detected irregular intervertebral spacing using an analytic methodology. The spinal canal CNN was trained via fivefold cross validation using 1,966 simulation CT scans and evaluated on 330 CT scans. After labeling vertebral levels (S2-C1) in 897 palliative radiotherapy simulation CT scans, a volume of interest surrounding the spinal canal in each patient's CT scan was converted into sagittal and coronal intensity projection image pairs. Then, intensity projection image pairs were augmented and used to train X-Net to automatically label vertebral levels using fivefold cross validation (n = 803). Prior to testing upon the final test set (n = 94), CT scans of patients with anatomical abnormalities, surgical implants, or other atypical features from the final test set were placed in an outlier group (n = 20), whereas those without these features were placed in a normative group (n = 74). The performance of X-Net, X-Net Ensemble, and another leading vertebral labeling architecture (Btrfly Net) was evaluated on both groups using identification rate, localization error, and other metrics. The performance of our approach was also evaluated on the MICCAI 2014 test dataset (n = 60). Finally, a method to detect irregular intervertebral spacing was created based on the rate of change in spacing between predicted vertebral body locations and was also evaluated using the final test set. Receiver operating characteristic analysis was used to investigate the performance of the method to detect irregular intervertebral spacing. RESULTS The spinal canal architecture yielded centroid coordinates spanning S2-C1 with submillimeter accuracy (mean ± standard deviation, 0.399 ± 0.299 mm; n = 330 patients) and was robust in the localization of spinal canal centroid to surgical implants and widespread metastases. Cross-validation testing of X-Net for vertebral labeling revealed that the deep learning model performance (F1 score, precision, and sensitivity) improved with CT scan length. The X-Net, X-Net Ensemble, and Btrfly Net mean identification rates and localization errors were 92.4% and 2.3 mm, 94.2% and 2.2 mm, and 90.5% and 3.4 mm, respectively, in the final test set and 96.7% and 2.2 mm, 96.9% and 2.0 mm, and 94.8% and 3.3 mm, respectively, within the normative group of the final test set. The X-Net Ensemble yielded the highest percentage of patients (94%) having all vertebral bodies identified correctly in the final test set when the three most inferior and superior vertebral bodies were excluded from the CT scan. The method used to detect labeling failures had 67% sensitivity and 95% specificity when combined with the X-Net Ensemble and flagged five of six patients with atypical vertebral counts (additional thoracic (T13), additional lumbar (L6) or only four lumbar vertebrae). Mean identification rate on the MICCAI 2014 dataset using an X-Net Ensemble was increased from 86.8% to 91.3% through the use of transfer learning and obtained state-of-the-art results for various regions of the spine. CONCLUSIONS We trained X-Net, our unique convolutional neural network, to automatically label vertebral levels from S2 to C1 on palliative radiotherapy CT images and found that an ensemble of X-Net models had high vertebral body identification rate (94.2%) and small localization errors (2.2 ± 1.8 mm). In addition, our transfer learning approach achieved state-of-the-art results on a well-known benchmark dataset with high identification rate (91.3%) and low localization error (3.3 mm ± 2.7 mm). When we pre-screened radiotherapy CT images for the presence of hardware, surgical implants, or other anatomic abnormalities prior to the use of X-Net, it labeled the spine correctly in more than 97% of patients and 94% of patients when scans were not prescreened. Automatically generated labels are robust to widespread vertebral metastases and surgical implants and our method to detect labeling failures based on neighborhood intervertebral spacing can reliably identify patients with an additional lumbar or thoracic vertebral body.
Collapse
Affiliation(s)
- Tucker J. Netherton
- Department of Radiation PhysicsThe University of Texas MD Anderson Cancer CenterHoustonTX77030USA
- The University of Texas MD Anderson Graduate School of Biomedical ScienceHoustonTX77030USA
| | - Dong Joo Rhee
- Department of Radiation PhysicsThe University of Texas MD Anderson Cancer CenterHoustonTX77030USA
- The University of Texas MD Anderson Graduate School of Biomedical ScienceHoustonTX77030USA
| | - Carlos E. Cardenas
- Department of Radiation PhysicsThe University of Texas MD Anderson Cancer CenterHoustonTX77030USA
| | - Caroline Chung
- Department of Radiation OncologyThe University of Texas MD Anderson Cancer CenterHoustonTX77030USA
| | - Ann H. Klopp
- Department of Radiation OncologyThe University of Texas MD Anderson Cancer CenterHoustonTX77030USA
| | - Christine B. Peterson
- Department of BiostatisticsThe University of Texas MD Anderson Cancer CenterHoustonTX77030USA
| | - Rebecca M. Howell
- Department of Radiation PhysicsThe University of Texas MD Anderson Cancer CenterHoustonTX77030USA
| | - Peter A. Balter
- Department of Radiation PhysicsThe University of Texas MD Anderson Cancer CenterHoustonTX77030USA
| | - Laurence E. Court
- Department of Radiation PhysicsThe University of Texas MD Anderson Cancer CenterHoustonTX77030USA
| |
Collapse
|
18
|
Rhee DJ, Jhingran A, Rigaud B, Netherton T, Cardenas CE, Zhang L, Vedam S, Kry S, Brock KK, Shaw W, O’Reilly F, Parkes J, Burger H, Fakie N, Trauernicht C, Simonds H, Court LE. Automatic contouring system for cervical cancer using convolutional neural networks. Med Phys 2020; 47:5648-5658. [PMID: 32964477 PMCID: PMC7756586 DOI: 10.1002/mp.14467] [Citation(s) in RCA: 19] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/29/2020] [Revised: 09/01/2020] [Accepted: 09/07/2020] [Indexed: 02/06/2023] Open
Abstract
PURPOSE To develop a tool for the automatic contouring of clinical treatment volumes (CTVs) and normal tissues for radiotherapy treatment planning in cervical cancer patients. METHODS An auto-contouring tool based on convolutional neural networks (CNN) was developed to delineate three cervical CTVs and 11 normal structures (seven OARs, four bony structures) in cervical cancer treatment for use with the Radiation Planning Assistant, a web-based automatic plan generation system. A total of 2254 retrospective clinical computed tomography (CT) scans from a single cancer center and 210 CT scans from a segmentation challenge were used to train and validate the CNN-based auto-contouring tool. The accuracy of the tool was evaluated by calculating the Sørensen-dice similarity coefficient (DSC) and mean surface and Hausdorff distances between the automatically generated contours and physician-drawn contours on 140 internal CT scans. A radiation oncologist scored the automatically generated contours on 30 external CT scans from three South African hospitals. RESULTS The average DSC, mean surface distance, and Hausdorff distance of our CNN-based tool were 0.86/0.19 cm/2.02 cm for the primary CTV, 0.81/0.21 cm/2.09 cm for the nodal CTV, 0.76/0.27 cm/2.00 cm for the PAN CTV, 0.89/0.11 cm/1.07 cm for the bladder, 0.81/0.18 cm/1.66 cm for the rectum, 0.90/0.06 cm/0.65 cm for the spinal cord, 0.94/0.06 cm/0.60 cm for the left femur, 0.93/0.07 cm/0.66 cm for the right femur, 0.94/0.08 cm/0.76 cm for the left kidney, 0.95/0.07 cm/0.84 cm for the right kidney, 0.93/0.05 cm/1.06 cm for the pelvic bone, 0.91/0.07 cm/1.25 cm for the sacrum, 0.91/0.07 cm/0.53 cm for the L4 vertebral body, and 0.90/0.08 cm/0.68 cm for the L5 vertebral bodies. On average, 80% of the CTVs, 97% of the organ at risk, and 98% of the bony structure contours in the external test dataset were clinically acceptable based on physician review. CONCLUSIONS Our CNN-based auto-contouring tool performed well on both internal and external datasets and had a high rate of clinical acceptability.
Collapse
Affiliation(s)
- Dong Joo Rhee
- MD Anderson UTHealth Graduate SchoolHoustonTXUSA
- Department of Radiation PhysicsDivision of Radiation OncologyThe University of Texas MD Anderson Cancer CenterHoustonTXUSA
| | - Anuja Jhingran
- Department of Radiation OncologyThe University of Texas MD Anderson Cancer CenterHoustonTXUSA
| | - Bastien Rigaud
- Department of Imaging PhysicsThe University of Texas MD Anderson Cancer CenterHoustonTXUSA
| | - Tucker Netherton
- MD Anderson UTHealth Graduate SchoolHoustonTXUSA
- Department of Radiation PhysicsDivision of Radiation OncologyThe University of Texas MD Anderson Cancer CenterHoustonTXUSA
| | - Carlos E. Cardenas
- Department of Radiation PhysicsDivision of Radiation OncologyThe University of Texas MD Anderson Cancer CenterHoustonTXUSA
| | - Lifei Zhang
- Department of Radiation PhysicsDivision of Radiation OncologyThe University of Texas MD Anderson Cancer CenterHoustonTXUSA
| | - Sastry Vedam
- Department of Radiation PhysicsDivision of Radiation OncologyThe University of Texas MD Anderson Cancer CenterHoustonTXUSA
| | - Stephen Kry
- Department of Radiation PhysicsDivision of Radiation OncologyThe University of Texas MD Anderson Cancer CenterHoustonTXUSA
| | - Kristy K. Brock
- Department of Imaging PhysicsThe University of Texas MD Anderson Cancer CenterHoustonTXUSA
| | - William Shaw
- Department of Medical Physics (G68)University of the Free StateBloemfonteinSouth Africa
| | - Frederika O’Reilly
- Department of Medical Physics (G68)University of the Free StateBloemfonteinSouth Africa
| | - Jeannette Parkes
- Division of Radiation Oncology and Medical PhysicsUniversity of Cape Town and Groote Schuur HospitalCape TownSouth Africa
| | - Hester Burger
- Division of Radiation Oncology and Medical PhysicsUniversity of Cape Town and Groote Schuur HospitalCape TownSouth Africa
| | - Nazia Fakie
- Division of Radiation Oncology and Medical PhysicsUniversity of Cape Town and Groote Schuur HospitalCape TownSouth Africa
| | - Chris Trauernicht
- Division of Medical PhysicsStellenbosch UniversityTygerberg Academic HospitalCape TownSouth Africa
| | - Hannah Simonds
- Division of Radiation OncologyStellenbosch UniversityTygerberg Academic HospitalCape TownSouth Africa
| | - Laurence E. Court
- Department of Radiation PhysicsDivision of Radiation OncologyThe University of Texas MD Anderson Cancer CenterHoustonTXUSA
| |
Collapse
|
19
|
Overview of artificial intelligence-based applications in radiotherapy: Recommendations for implementation and quality assurance. Radiother Oncol 2020; 153:55-66. [PMID: 32920005 DOI: 10.1016/j.radonc.2020.09.008] [Citation(s) in RCA: 133] [Impact Index Per Article: 33.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/03/2020] [Revised: 09/02/2020] [Accepted: 09/03/2020] [Indexed: 02/06/2023]
Abstract
Artificial Intelligence (AI) is currently being introduced into different domains, including medicine. Specifically in radiation oncology, machine learning models allow automation and optimization of the workflow. A lack of knowledge and interpretation of these AI models can hold back wide-spread and full deployment into clinical practice. To facilitate the integration of AI models in the radiotherapy workflow, generally applicable recommendations on implementation and quality assurance (QA) of AI models are presented. For commonly used applications in radiotherapy such as auto-segmentation, automated treatment planning and synthetic computed tomography (sCT) the basic concepts are discussed in depth. Emphasis is put on the commissioning, implementation and case-specific and routine QA of AI models needed for a methodical introduction in clinical practice.
Collapse
|
20
|
Kisling K, Cardenas C, Anderson BM, Zhang L, Jhingran A, Simonds H, Balter P, Howell RM, Schmeler K, Beadle BM, Court L. Automatic Verification of Beam Apertures for Cervical Cancer Radiation Therapy. Pract Radiat Oncol 2020; 10:e415-e424. [PMID: 32450365 PMCID: PMC8133770 DOI: 10.1016/j.prro.2020.05.001] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/22/2019] [Revised: 03/16/2020] [Accepted: 05/03/2020] [Indexed: 11/29/2022]
Abstract
PURPOSE Automated tools can help identify radiation treatment plans of unacceptable quality. To this end, we developed a quality verification technique to automatically verify the clinical acceptability of beam apertures for 4-field box treatments of patients with cervical cancer. By comparing the beam apertures to be used for treatment with a secondary set of beam apertures developed automatically, this quality verification technique can flag beam apertures that may need to be edited to be acceptable for treatment. METHODS AND MATERIALS The automated methodology for creating verification beam apertures uses a deep learning model trained on beam apertures and digitally reconstructed radiographs from 255 clinically acceptable planned treatments (as rated by physicians). These verification apertures were then compared with the treatment apertures using spatial comparison metrics to detect unacceptable treatment apertures. We tested the quality verification technique on beam apertures from 80 treatment plans. Each plan was rated by physicians, where 57 were rated clinically acceptable and 23 were rated clinically unacceptable. RESULTS Using various comparison metrics (the mean surface distance, Hausdorff distance, and Dice similarity coefficient) for the 2 sets of beam apertures, we found that treatment beam apertures rated acceptable had significantly better agreement with the verification beam apertures than those rated unacceptable (P < .01). Upon receiver operating characteristic analysis, we found the area under the curve for all metrics to be 0.89 to 0.95, which demonstrated the high sensitivity and specificity of our quality verification technique. CONCLUSIONS We found that our technique of automatically verifying the beam aperture is an effective tool for flagging potentially unacceptable beam apertures during the treatment plan review process. Accordingly, we will clinically deploy this quality verification technique as part of a fully automated treatment planning tool and automated plan quality assurance program.
Collapse
Affiliation(s)
- Kelly Kisling
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas
| | - Carlos Cardenas
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas
| | - Brian M Anderson
- Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas
| | - Lifei Zhang
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas
| | - Anuja Jhingran
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas
| | - Hannah Simonds
- Division of Radiation Oncology, Stellenbosch University and Tygerberg Hospital, Cape Town, South Africa
| | - Peter Balter
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas
| | - Rebecca M Howell
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas
| | - Kathleen Schmeler
- Gynecologic Oncology and Reproductive Medicine, The University of Texas MD Anderson Cancer Center, Houston, Texas
| | - Beth M Beadle
- Department of Radiation Oncology - Radiation Therapy, Stanford University, Stanford, California
| | - Laurence Court
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas.
| |
Collapse
|
21
|
Kisling K, Zhang L, Simonds H, Fakie N, Yang J, McCarroll R, Balter P, Burger H, Bogler O, Howell R, Schmeler K, Mejia M, Beadle BM, Jhingran A, Court L. Fully Automatic Treatment Planning for External-Beam Radiation Therapy of Locally Advanced Cervical Cancer: A Tool for Low-Resource Clinics. J Glob Oncol 2020; 5:1-9. [PMID: 30629457 PMCID: PMC6426517 DOI: 10.1200/jgo.18.00107] [Citation(s) in RCA: 21] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022] Open
Abstract
Purpose The purpose of this study was to validate a fully automatic treatment planning system for conventional radiotherapy of cervical cancer. This system was developed to mitigate staff shortages in low-resource clinics. Methods In collaboration with hospitals in South Africa and the United States, we have developed the Radiation Planning Assistant (RPA), which includes algorithms for automating every step of planning: delineating the body contour, detecting the marked isocenter, designing the treatment-beam apertures, and optimizing the beam weights to minimize dose heterogeneity. First, we validated the RPA retrospectively on 150 planning computed tomography (CT) scans. We then tested it remotely on 14 planning CT scans at two South African hospitals. Finally, automatically planned treatment beams were clinically deployed at our institution. Results The automatically and manually delineated body contours agreed well (median mean surface distance, 0.6 mm; range, 0.4 to 1.9 mm). The automatically and manually detected marked isocenters agreed well (mean difference, 1.1 mm; range, 0.1 to 2.9 mm). In validating the automatically designed beam apertures, two physicians, one from our institution and one from a South African partner institution, rated 91% and 88% of plans acceptable for treatment, respectively. The use of automatically optimized beam weights reduced the maximum dose significantly (median, −1.9%; P < .001). Of the 14 plans from South Africa, 100% were rated clinically acceptable. Automatically planned treatment beams have been used for 24 patients with cervical cancer by physicians at our institution, with edits as needed, and its use is ongoing. Conclusion We found that fully automatic treatment planning is effective for cervical cancer radiotherapy and may provide a reliable option for low-resource clinics. Prospective studies are ongoing in the United States and are planned with partner clinics.
Collapse
Affiliation(s)
- Kelly Kisling
- Kelly Kisling, Lifei Zhang, Jinzhong Yang, Rachel McCarroll, Peter Balter, Rebecca Howell, Kathleen Schmeler, Anuja Jhingran, and Laurence Court, The University of Texas MD Anderson Cancer Center, Houston, TX; Hannah Simonds, Stellenbosch University and Tygerberg Hospital; Nazia Fakie and Hester Burger, University of Cape Town and Groote Schuur Hospital, Cape Town, South Africa; Oliver Bogler, The University of New Mexico School of Medicine, Albuquerque, NM; Mike Mejia, University of Santo Tomas Hospital, Benavides Cancer Institute, Manila, Philippines; Beth M. Beadle, Stanford University, Stanford, CA
| | - Lifei Zhang
- Kelly Kisling, Lifei Zhang, Jinzhong Yang, Rachel McCarroll, Peter Balter, Rebecca Howell, Kathleen Schmeler, Anuja Jhingran, and Laurence Court, The University of Texas MD Anderson Cancer Center, Houston, TX; Hannah Simonds, Stellenbosch University and Tygerberg Hospital; Nazia Fakie and Hester Burger, University of Cape Town and Groote Schuur Hospital, Cape Town, South Africa; Oliver Bogler, The University of New Mexico School of Medicine, Albuquerque, NM; Mike Mejia, University of Santo Tomas Hospital, Benavides Cancer Institute, Manila, Philippines; Beth M. Beadle, Stanford University, Stanford, CA
| | - Hannah Simonds
- Kelly Kisling, Lifei Zhang, Jinzhong Yang, Rachel McCarroll, Peter Balter, Rebecca Howell, Kathleen Schmeler, Anuja Jhingran, and Laurence Court, The University of Texas MD Anderson Cancer Center, Houston, TX; Hannah Simonds, Stellenbosch University and Tygerberg Hospital; Nazia Fakie and Hester Burger, University of Cape Town and Groote Schuur Hospital, Cape Town, South Africa; Oliver Bogler, The University of New Mexico School of Medicine, Albuquerque, NM; Mike Mejia, University of Santo Tomas Hospital, Benavides Cancer Institute, Manila, Philippines; Beth M. Beadle, Stanford University, Stanford, CA
| | - Nazia Fakie
- Kelly Kisling, Lifei Zhang, Jinzhong Yang, Rachel McCarroll, Peter Balter, Rebecca Howell, Kathleen Schmeler, Anuja Jhingran, and Laurence Court, The University of Texas MD Anderson Cancer Center, Houston, TX; Hannah Simonds, Stellenbosch University and Tygerberg Hospital; Nazia Fakie and Hester Burger, University of Cape Town and Groote Schuur Hospital, Cape Town, South Africa; Oliver Bogler, The University of New Mexico School of Medicine, Albuquerque, NM; Mike Mejia, University of Santo Tomas Hospital, Benavides Cancer Institute, Manila, Philippines; Beth M. Beadle, Stanford University, Stanford, CA
| | - Jinzhong Yang
- Kelly Kisling, Lifei Zhang, Jinzhong Yang, Rachel McCarroll, Peter Balter, Rebecca Howell, Kathleen Schmeler, Anuja Jhingran, and Laurence Court, The University of Texas MD Anderson Cancer Center, Houston, TX; Hannah Simonds, Stellenbosch University and Tygerberg Hospital; Nazia Fakie and Hester Burger, University of Cape Town and Groote Schuur Hospital, Cape Town, South Africa; Oliver Bogler, The University of New Mexico School of Medicine, Albuquerque, NM; Mike Mejia, University of Santo Tomas Hospital, Benavides Cancer Institute, Manila, Philippines; Beth M. Beadle, Stanford University, Stanford, CA
| | - Rachel McCarroll
- Kelly Kisling, Lifei Zhang, Jinzhong Yang, Rachel McCarroll, Peter Balter, Rebecca Howell, Kathleen Schmeler, Anuja Jhingran, and Laurence Court, The University of Texas MD Anderson Cancer Center, Houston, TX; Hannah Simonds, Stellenbosch University and Tygerberg Hospital; Nazia Fakie and Hester Burger, University of Cape Town and Groote Schuur Hospital, Cape Town, South Africa; Oliver Bogler, The University of New Mexico School of Medicine, Albuquerque, NM; Mike Mejia, University of Santo Tomas Hospital, Benavides Cancer Institute, Manila, Philippines; Beth M. Beadle, Stanford University, Stanford, CA
| | - Peter Balter
- Kelly Kisling, Lifei Zhang, Jinzhong Yang, Rachel McCarroll, Peter Balter, Rebecca Howell, Kathleen Schmeler, Anuja Jhingran, and Laurence Court, The University of Texas MD Anderson Cancer Center, Houston, TX; Hannah Simonds, Stellenbosch University and Tygerberg Hospital; Nazia Fakie and Hester Burger, University of Cape Town and Groote Schuur Hospital, Cape Town, South Africa; Oliver Bogler, The University of New Mexico School of Medicine, Albuquerque, NM; Mike Mejia, University of Santo Tomas Hospital, Benavides Cancer Institute, Manila, Philippines; Beth M. Beadle, Stanford University, Stanford, CA
| | - Hester Burger
- Kelly Kisling, Lifei Zhang, Jinzhong Yang, Rachel McCarroll, Peter Balter, Rebecca Howell, Kathleen Schmeler, Anuja Jhingran, and Laurence Court, The University of Texas MD Anderson Cancer Center, Houston, TX; Hannah Simonds, Stellenbosch University and Tygerberg Hospital; Nazia Fakie and Hester Burger, University of Cape Town and Groote Schuur Hospital, Cape Town, South Africa; Oliver Bogler, The University of New Mexico School of Medicine, Albuquerque, NM; Mike Mejia, University of Santo Tomas Hospital, Benavides Cancer Institute, Manila, Philippines; Beth M. Beadle, Stanford University, Stanford, CA
| | - Oliver Bogler
- Kelly Kisling, Lifei Zhang, Jinzhong Yang, Rachel McCarroll, Peter Balter, Rebecca Howell, Kathleen Schmeler, Anuja Jhingran, and Laurence Court, The University of Texas MD Anderson Cancer Center, Houston, TX; Hannah Simonds, Stellenbosch University and Tygerberg Hospital; Nazia Fakie and Hester Burger, University of Cape Town and Groote Schuur Hospital, Cape Town, South Africa; Oliver Bogler, The University of New Mexico School of Medicine, Albuquerque, NM; Mike Mejia, University of Santo Tomas Hospital, Benavides Cancer Institute, Manila, Philippines; Beth M. Beadle, Stanford University, Stanford, CA
| | - Rebecca Howell
- Kelly Kisling, Lifei Zhang, Jinzhong Yang, Rachel McCarroll, Peter Balter, Rebecca Howell, Kathleen Schmeler, Anuja Jhingran, and Laurence Court, The University of Texas MD Anderson Cancer Center, Houston, TX; Hannah Simonds, Stellenbosch University and Tygerberg Hospital; Nazia Fakie and Hester Burger, University of Cape Town and Groote Schuur Hospital, Cape Town, South Africa; Oliver Bogler, The University of New Mexico School of Medicine, Albuquerque, NM; Mike Mejia, University of Santo Tomas Hospital, Benavides Cancer Institute, Manila, Philippines; Beth M. Beadle, Stanford University, Stanford, CA
| | - Kathleen Schmeler
- Kelly Kisling, Lifei Zhang, Jinzhong Yang, Rachel McCarroll, Peter Balter, Rebecca Howell, Kathleen Schmeler, Anuja Jhingran, and Laurence Court, The University of Texas MD Anderson Cancer Center, Houston, TX; Hannah Simonds, Stellenbosch University and Tygerberg Hospital; Nazia Fakie and Hester Burger, University of Cape Town and Groote Schuur Hospital, Cape Town, South Africa; Oliver Bogler, The University of New Mexico School of Medicine, Albuquerque, NM; Mike Mejia, University of Santo Tomas Hospital, Benavides Cancer Institute, Manila, Philippines; Beth M. Beadle, Stanford University, Stanford, CA
| | - Mike Mejia
- Kelly Kisling, Lifei Zhang, Jinzhong Yang, Rachel McCarroll, Peter Balter, Rebecca Howell, Kathleen Schmeler, Anuja Jhingran, and Laurence Court, The University of Texas MD Anderson Cancer Center, Houston, TX; Hannah Simonds, Stellenbosch University and Tygerberg Hospital; Nazia Fakie and Hester Burger, University of Cape Town and Groote Schuur Hospital, Cape Town, South Africa; Oliver Bogler, The University of New Mexico School of Medicine, Albuquerque, NM; Mike Mejia, University of Santo Tomas Hospital, Benavides Cancer Institute, Manila, Philippines; Beth M. Beadle, Stanford University, Stanford, CA
| | - Beth M Beadle
- Kelly Kisling, Lifei Zhang, Jinzhong Yang, Rachel McCarroll, Peter Balter, Rebecca Howell, Kathleen Schmeler, Anuja Jhingran, and Laurence Court, The University of Texas MD Anderson Cancer Center, Houston, TX; Hannah Simonds, Stellenbosch University and Tygerberg Hospital; Nazia Fakie and Hester Burger, University of Cape Town and Groote Schuur Hospital, Cape Town, South Africa; Oliver Bogler, The University of New Mexico School of Medicine, Albuquerque, NM; Mike Mejia, University of Santo Tomas Hospital, Benavides Cancer Institute, Manila, Philippines; Beth M. Beadle, Stanford University, Stanford, CA
| | - Anuja Jhingran
- Kelly Kisling, Lifei Zhang, Jinzhong Yang, Rachel McCarroll, Peter Balter, Rebecca Howell, Kathleen Schmeler, Anuja Jhingran, and Laurence Court, The University of Texas MD Anderson Cancer Center, Houston, TX; Hannah Simonds, Stellenbosch University and Tygerberg Hospital; Nazia Fakie and Hester Burger, University of Cape Town and Groote Schuur Hospital, Cape Town, South Africa; Oliver Bogler, The University of New Mexico School of Medicine, Albuquerque, NM; Mike Mejia, University of Santo Tomas Hospital, Benavides Cancer Institute, Manila, Philippines; Beth M. Beadle, Stanford University, Stanford, CA
| | - Laurence Court
- Kelly Kisling, Lifei Zhang, Jinzhong Yang, Rachel McCarroll, Peter Balter, Rebecca Howell, Kathleen Schmeler, Anuja Jhingran, and Laurence Court, The University of Texas MD Anderson Cancer Center, Houston, TX; Hannah Simonds, Stellenbosch University and Tygerberg Hospital; Nazia Fakie and Hester Burger, University of Cape Town and Groote Schuur Hospital, Cape Town, South Africa; Oliver Bogler, The University of New Mexico School of Medicine, Albuquerque, NM; Mike Mejia, University of Santo Tomas Hospital, Benavides Cancer Institute, Manila, Philippines; Beth M. Beadle, Stanford University, Stanford, CA
| |
Collapse
|
22
|
Rhee DJ, Cardenas CE, Elhalawani H, McCarroll R, Zhang L, Yang J, Garden AS, Peterson CB, Beadle BM, Court LE. Automatic detection of contouring errors using convolutional neural networks. Med Phys 2019; 46:5086-5097. [PMID: 31505046 PMCID: PMC6842055 DOI: 10.1002/mp.13814] [Citation(s) in RCA: 58] [Impact Index Per Article: 11.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2019] [Revised: 08/28/2019] [Accepted: 08/30/2019] [Indexed: 12/14/2022] Open
Abstract
PURPOSE To develop a head and neck normal structures autocontouring tool that could be used to automatically detect the errors in autocontours from a clinically validated autocontouring tool. METHODS An autocontouring tool based on convolutional neural networks (CNN) was developed for 16 normal structures of the head and neck and tested to identify the contour errors from a clinically validated multiatlas-based autocontouring system (MACS). The computed tomography (CT) scans and clinical contours from 3495 patients were semiautomatically curated and used to train and validate the CNN-based autocontouring tool. The final accuracy of the tool was evaluated by calculating the Sørensen-Dice similarity coefficients (DSC) and Hausdorff distances between the automatically generated contours and physician-drawn contours on 174 internal and 24 external CT scans. Lastly, the CNN-based tool was evaluated on 60 patients' CT scans to investigate the possibility to detect contouring failures. The contouring failures on these patients were classified as either minor or major errors. The criteria to detect contouring errors were determined by analyzing the DSC between the CNN- and MACS-based contours under two independent scenarios: (a) contours with minor errors are clinically acceptable and (b) contours with minor errors are clinically unacceptable. RESULTS The average DSC and Hausdorff distance of our CNN-based tool was 98.4%/1.23 cm for brain, 89.1%/0.42 cm for eyes, 86.8%/1.28 cm for mandible, 86.4%/0.88 cm for brainstem, 83.4%/0.71 cm for spinal cord, 82.7%/1.37 cm for parotids, 80.7%/1.08 cm for esophagus, 71.7%/0.39 cm for lenses, 68.6%/0.72 for optic nerves, 66.4%/0.46 cm for cochleas, and 40.7%/0.96 cm for optic chiasm. With the error detection tool, the proportions of the clinically unacceptable MACS contours that were correctly detected were 0.99/0.80 on average except for the optic chiasm, when contours with minor errors are clinically acceptable/unacceptable, respectively. The proportions of the clinically acceptable MACS contours that were correctly detected were 0.81/0.60 on average except for the optic chiasm, when contours with minor errors are clinically acceptable/unacceptable, respectively. CONCLUSION Our CNN-based autocontouring tool performed well on both the publically available and the internal datasets. Furthermore, our results show that CNN-based algorithms are able to identify ill-defined contours from a clinically validated and used multiatlas-based autocontouring tool. Therefore, our CNN-based tool can effectively perform automatic verification of MACS contours.
Collapse
Affiliation(s)
- Dong Joo Rhee
- The University of Texas Graduate School of Biomedical Sciences at HoustonHoustonTX77030USA
- Department of Radiation PhysicsDivision of Radiation OncologyThe University of Texas MD Anderson Cancer CenterHoustonTX77030USA
| | - Carlos E. Cardenas
- Department of Radiation PhysicsDivision of Radiation OncologyThe University of Texas MD Anderson Cancer CenterHoustonTX77030USA
| | - Hesham Elhalawani
- Department of Radiation OncologyDivision of Radiation OncologyThe University of Texas MD Anderson Cancer CenterHoustonTX77030USA
| | - Rachel McCarroll
- Department of Radiation OncologyThe University of Maryland Medical SystemBaltimoreMD21201USA
| | - Lifei Zhang
- Department of Radiation PhysicsDivision of Radiation OncologyThe University of Texas MD Anderson Cancer CenterHoustonTX77030USA
| | - Jinzhong Yang
- Department of Radiation PhysicsDivision of Radiation OncologyThe University of Texas MD Anderson Cancer CenterHoustonTX77030USA
| | - Adam S. Garden
- Department of Radiation OncologyDivision of Radiation OncologyThe University of Texas MD Anderson Cancer CenterHoustonTX77030USA
| | - Christine B. Peterson
- Department of BiostatisticsDivision of Basic SciencesThe University of Texas MD Anderson Cancer CenterHoustonTX77030USA
| | - Beth M. Beadle
- Department of Radiation OncologyStanford University School of MedicineStanfordCA94305USA
| | - Laurence E. Court
- Department of Radiation PhysicsDivision of Radiation OncologyThe University of Texas MD Anderson Cancer CenterHoustonTX77030USA
| |
Collapse
|
23
|
Kisling K, Zhang L, Shaitelman SF, Anderson D, Thebe T, Yang J, Balter PA, Howell RM, Jhingran A, Schmeler K, Simonds H, du Toit M, Trauernicht C, Burger H, Botha K, Joubert N, Beadle BM, Court L. Automated treatment planning of postmastectomy radiotherapy. Med Phys 2019; 46:3767-3775. [PMID: 31077593 PMCID: PMC6739169 DOI: 10.1002/mp.13586] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2019] [Revised: 05/01/2019] [Accepted: 05/05/2019] [Indexed: 11/23/2022] Open
Abstract
Purpose Breast cancer is the most common cancer in women globally and radiation therapy is a cornerstone of its treatment. However, there is an enormous shortage of radiotherapy staff, especially in low‐ and middle‐income countries. This shortage could be ameliorated through increased automation in the radiation treatment planning process, which may reduce the workload on radiotherapy staff and improve efficiency in preparing radiotherapy treatments for patients. To this end, we sought to create an automated treatment planning tool for postmastectomy radiotherapy (PMRT). Methods Algorithms to automate every step of PMRT planning were developed and integrated into a commercial treatment planning system. The only required inputs for automated PMRT planning are a planning computed tomography scan, a plan directive, and selection of the inferior border of the tangential fields. With no other human input, the planning tool automatically creates a treatment plan and presents it for review. The major automated steps are (a) segmentation of relevant structures (targets, normal tissues, and other planning structures), (b) setup of the beams (tangential fields matched with a supraclavicular field), and (c) optimization of the dose distribution by using a mix of high‐ and low‐energy photon beams and field‐in‐field modulation for the tangential fields. This automated PMRT planning tool was tested with ten computed tomography scans of patients with breast cancer who had received irradiation of the left chest wall. These plans were assessed quantitatively using their dose distributions and were reviewed by two physicians who rated them on a three‐tiered scale: use as is, minor changes, or major changes. The accuracy of the automated segmentation of the heart and ipsilateral lung was also assessed. Finally, a plan quality verification tool was tested to alert the user to any possible deviations in the quality of the automatically created treatment plans. Results The automatically created PMRT plans met the acceptable dose objectives, including target coverage, maximum plan dose, and dose to organs at risk, for all but one patient for whom the heart objectives were exceeded. Physicians accepted 50% of the treatment plans as is and required only minor changes for the remaining 50%, which included the one patient whose plan had a high heart dose. Furthermore, the automatically segmented contours of the heart and ipsilateral lung agreed well with manually edited contours. Finally, the automated plan quality verification tool detected 92% of the changes requested by physicians in this review. Conclusions We developed a new tool for automatically planning PMRT for breast cancer, including irradiation of the chest wall and ipsilateral lymph nodes (supraclavicular and level III axillary). In this initial testing, we found that the plans created by this tool are clinically viable, and the tool can alert the user to possible deviations in plan quality. The next step is to subject this tool to prospective testing, in which automatically planned treatments will be compared with manually planned treatments.
Collapse
Affiliation(s)
- Kelly Kisling
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, 77030, USA
| | - Lifei Zhang
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, 77030, USA
| | - Simona F Shaitelman
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, 77030, USA
| | - David Anderson
- Department of Radiation Oncology, University of Cape Town and Groote Schuur Hospital, Cape Town, 8000, South Africa
| | - Tselane Thebe
- Department of Radiation Oncology, University of Cape Town and Groote Schuur Hospital, Cape Town, 8000, South Africa
| | - Jinzhong Yang
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, 77030, USA
| | - Peter A Balter
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, 77030, USA
| | - Rebecca M Howell
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, 77030, USA
| | - Anuja Jhingran
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, 77030, USA
| | - Kathleen Schmeler
- Department of Gynecologic Oncology and Reproductive Medicine, The University of Texas MD Anderson Cancer Center, Houston, Texas, 77030, USA
| | - Hannah Simonds
- Division of Radiation Oncology, Stellenbosch University and Tygerberg Hospital, Cape Town, 7505, South Africa
| | - Monique du Toit
- Division of Medical Physics, Stellenbosch University and Tygerberg Hospital, Cape Town, 7505, South Africa
| | - Christoph Trauernicht
- Division of Medical Physics, Stellenbosch University and Tygerberg Hospital, Cape Town, 7505, South Africa
| | - Hester Burger
- Division of Medical Physics, University of Cape Town and Groote Schuur Hospital, Cape Town, 8000, South Africa
| | - Kobus Botha
- Division of Medical Physics, University of Cape Town and Groote Schuur Hospital, Cape Town, 8000, South Africa
| | - Nanette Joubert
- Division of Medical Physics, University of Cape Town and Groote Schuur Hospital, Cape Town, 8000, South Africa
| | - Beth M Beadle
- Department of Radiation Oncology, Stanford University, Stanford, CA, 94305, USA
| | - Laurence Court
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, 77030, USA
| |
Collapse
|
24
|
Kisling K, Johnson JL, Simonds H, Zhang L, Jhingran A, Beadle BM, Burger H, du Toit M, Joubert N, Makufa R, Shaw W, Trauernicht C, Balter P, Howell RM, Schmeler K, Court L. A risk assessment of automated treatment planning and recommendations for clinical deployment. Med Phys 2019; 46:2567-2574. [PMID: 31002389 PMCID: PMC6561826 DOI: 10.1002/mp.13552] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2019] [Revised: 04/04/2019] [Accepted: 04/05/2019] [Indexed: 12/20/2022] Open
Abstract
Purpose To assess the risk of failure of a recently developed automated treatment planning tool, the radiation planning assistant (RPA), and to determine the reduction in these risks with implementation of a quality assurance (QA) program specifically designed for the RPA. Methods We used failure mode and effects analysis (FMEA) to assess the risk of the RPA. The steps involved in the workflow of planning a four‐field box treatment of cervical cancer with the RPA were identified. Then, the potential failure modes at each step and their causes were identified and scored according to their likelihood of occurrence, severity, and likelihood of going undetected. Additionally, the impact of the components of the QA program on the detectability of the failure modes was assessed. The QA program was designed to supplement a clinic's standard QA processes and consisted of three components: (a) automatic, independent verification of the results of automated planning; (b) automatic comparison of treatment parameters to expected values; and (c) guided manual checks of the treatment plan. A risk priority number (RPN) was calculated for each potential failure mode with and without use of the QA program. Results In the RPA automated treatment planning workflow, we identified 68 potential failure modes with 113 causes. The average RPN was 91 without the QA program and 68 with the QA program (maximum RPNs were 504 and 315, respectively). The reduction in RPN was due to an improvement in the likelihood of detecting failures, resulting in lower detectability scores. The top‐ranked failure modes included incorrect identification of the marked isocenter, inappropriate beam aperture definition, incorrect entry of the prescription into the RPA plan directive, and lack of a comprehensive plan review by the physician. Conclusions Using FMEA, we assessed the risks in the clinical deployment of an automated treatment planning workflow and showed that a specialized QA program for the RPA, which included automatic QA techniques, improved the detectability of failures, reducing this risk. However, some residual risks persisted, which were similar to those found in manual treatment planning, and human error remained a major cause of potential failures. Through the risk analysis process, we identified three key aspects of safe deployment of automated planning: (a) user training on potential failure modes; (b) comprehensive manual plan review by physicians and physicists; and (c) automated QA of the treatment plan.
Collapse
Affiliation(s)
- Kelly Kisling
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, 77030, USA
| | - Jennifer L Johnson
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, 77030, USA
| | - Hannah Simonds
- Division of Radiation Oncology, Stellenbosch University and Tygerberg Hospital, Cape Town, 7505, South Africa
| | - Lifei Zhang
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, 77030, USA
| | - Anuja Jhingran
- Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, 77030, USA
| | - Beth M Beadle
- Department of Radiation Oncology - Radiation Therapy, Stanford University, Stanford, CA, 94305, USA
| | - Hester Burger
- Division of Medical Physics, University of Cape Town and Groote Schuur Hospital, Cape Town, 8000, South Africa
| | - Monique du Toit
- Division of Medical Physics, Stellenbosch University and Tygerberg Hospital, Cape Town, 7505, South Africa
| | - Nanette Joubert
- Division of Medical Physics, University of Cape Town and Groote Schuur Hospital, Cape Town, 8000, South Africa
| | - Remigio Makufa
- Department of Medical Physics, Gaborone Private Hospital, Gaborone, Botswana
| | - William Shaw
- Department of Medical Physics (G68), University of the Free State, Bloemfontein, 9301, South Africa
| | - Christoph Trauernicht
- Division of Medical Physics, Stellenbosch University and Tygerberg Hospital, Cape Town, 7505, South Africa
| | - Peter Balter
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, 77030, USA
| | - Rebecca M Howell
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, 77030, USA
| | - Kathleen Schmeler
- Department of Gynecologic Oncology and Reproductive Medicine, The University of Texas MD Anderson Cancer Center, Houston, TX, 77030, USA
| | - Laurence Court
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, 77030, USA
| |
Collapse
|
25
|
Fang R, Yang J, Du W, Court L. Automatic detection of graticule isocenter and scale from kV and MV images. J Appl Clin Med Phys 2019; 20:18-28. [PMID: 30843335 PMCID: PMC6448171 DOI: 10.1002/acm2.12558] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2018] [Revised: 02/04/2019] [Accepted: 02/12/2019] [Indexed: 11/11/2022] Open
Abstract
PURPOSE To automate the detection of isocenter and scale of the mechanical graticule on kilo-voltage (kV) or mega-voltage (MV) films or electronic portal imaging device (EPID) images. METHODS We developed a robust image processing approach to automatically detect isocenter and scale of mechanical graticule from digitized kV or MV films and EPID images. After a series of preprocessing steps applied to the digital images, a combination of Hough transform and Radon transform was performed to detect the graticule axes and isocenter. The magnification of the graticule was automatically detected by solving an optimization problem using golden section search and parabolic interpolation algorithm. Tick marks of the graticule were then determined by extending from isocenter along the graticule axes with multiples of the magnification value. This approach was validated using 23 kV films, 26 MV films, and 91 EPID images in different anatomical sites (head-and-neck, thorax, and pelvis). Accuracy was measured by comparing computer detected results with manually selected results. RESULTS The proposed approach was robust for kV and MV films of varying image quality. The isocenter was detected within 1 mm for 98% of the images. The exceptions were three kV films where the graticule was not actually visible. Of all images with correct isocenter detection, 99% had a magnification detection error less than 1% and tick mark detection error less than 1 mm, with the exception of 1 kV film (magnification error: 3.17%; tick mark error: 1.29 mm) and 1 MV film (magnification error: 0.45%; tick mark error: 1.11 mm). CONCLUSION We developed an approach to robustly and automatically detect graticule isocenter and scale from two-dimensionla (2D) kV and MV films. This is a first step toward automated treatment planning based on 2D x-ray images.
Collapse
Affiliation(s)
- Raymond Fang
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, USA.,Deparmtent of Physics and Astronomy, Rice University, Houston, TX, USA
| | - Jinzhong Yang
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Weiliang Du
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Laurence Court
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| |
Collapse
|
26
|
Horner M, Luke SM, Genc KO, Pietila TM, Cotton RT, Ache BA, Levine ZH, Townsend KC. Towards Estimating the Uncertainty Associated with Three-Dimensional Geometry Reconstructed from Medical Image Data. JOURNAL OF VERIFICATION, VALIDATION, AND UNCERTAINTY QUANTIFICATION 2019; 4:https://doi.org/10.1115/1.4045487. [PMID: 32856003 PMCID: PMC7448268] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
Patient-specific computational modeling is increasingly used to assist with visualization, planning, and execution of medical treatments. This trend is placing more reliance on medical imaging to provide accurate representations of anatomical structures. Digital image analysis is used to extract anatomical data for use in clinical assessment/planning. However, the presence of image artifacts, whether due to interactions between the physical object and the scanning modality or the scanning process, can degrade image accuracy. The process of extracting anatomical structures from the medical images introduces additional sources of variability, e.g., when thresholding or when eroding along apparent edges of biological structures. An estimate of the uncertainty associated with extracting anatomical data from medical images would therefore assist with assessing the reliability of patient-specific treatment plans. To this end, two image datasets were developed and analyzed using standard image analysis procedures. The first dataset was developed by performing a "virtual voxelization" of a CAD model of a sphere, representing the idealized scenario of no error in the image acquisition and reconstruction algorithms (i.e., a perfect scan). The second dataset was acquired by scanning three spherical balls using a laboratory-grade CT scanner. For the idealized sphere, the error in sphere diameter was less than or equal to 2% if 5 or more voxels were present across the diameter. The measurement error degraded to approximately 4% for a similar degree of voxelization of the physical phantom. The adaptation of established thresholding procedures to improve segmentation accuracy was also investigated.
Collapse
Affiliation(s)
| | | | | | | | | | | | - Zachary H Levine
- National Institute of Standards and Technology (NIST), Gaithersburg, MD
| | | |
Collapse
|
27
|
Cardenas CE, Anderson BM, Aristophanous M, Yang J, Rhee DJ, McCarroll RE, Mohamed ASR, Kamal M, Elgohari BA, Elhalawani HM, Fuller CD, Rao A, Garden AS, Court LE. Auto-delineation of oropharyngeal clinical target volumes using 3D convolutional neural networks. ACTA ACUST UNITED AC 2018; 63:215026. [DOI: 10.1088/1361-6560/aae8a9] [Citation(s) in RCA: 31] [Impact Index Per Article: 5.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/31/2022]
|