1
|
Trimpl MJ, Campbell S, Panakis N, Ajzensztejn D, Burke E, Ellis S, Johnstone P, Doyle E, Towers R, Higgins G, Bernard C, Hustinx R, Vallis KA, Stride EPJ, Gooding MJ. Deep learning-assisted interactive contouring of lung cancer: Impact on contouring time and consistency. Radiother Oncol 2024; 200:110500. [PMID: 39236985 DOI: 10.1016/j.radonc.2024.110500] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2023] [Revised: 07/24/2024] [Accepted: 08/19/2024] [Indexed: 09/07/2024]
Abstract
BACKGROUND AND PURPOSE To evaluate the impact of a deep learning (DL)-assisted interactive contouring tool on inter-observer variability and the time taken to complete tumour contouring. MATERIALS AND METHODS Nine clinicians contoured the gross tumour volume (GTV) using the PET-CT scans of 10 non-small cell lung cancer (NSCLC) patients, either using DL-assisted or manual contouring tools. After contouring a case using one contouring method, the same case was contoured one week later using the other method. The contours and time taken were compared. RESULTS Use of the DL-assisted tool led to a statistically significant decrease in active contouring time of 23 % relative to the standard manual segmentation method (p < 0.01). The mean observation time for all clinicians and cases made up nearly 60 % of interaction time for both contouring approaches. On average the time spent contouring per case was reduced from 22 min to 19 min when using the DL-assisted tool. Additionally, the DL-assisted tool reduced contour variability in the parts of tumour where clinicians tended to disagree the most, while the consensus contour was similar whichever of the two contouring approaches was used. CONCLUSIONS A DL-assisted interactive contouring approach decreased active contouring time and local inter-observer variability when used to delineate lung cancer GTVs compared to a standard manual method. Integration of this tool into the clinical workflow could assist clinicians in contouring tasks and improve contouring efficiency.
Collapse
Affiliation(s)
- Michael J Trimpl
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, UK; Department of Oncology, University of Oxford, Oxford, UK; Mirada Medical Ltd, Oxford, UK.
| | - Sorcha Campbell
- Edinburgh Cancer Centre, Western General Hospital, Edinburgh, UK.
| | - Niki Panakis
- Oxford University Hospitals NHS Foundation Trust, UK.
| | | | - Emma Burke
- Oxford University Hospitals NHS Foundation Trust, UK.
| | - Shawn Ellis
- Oxford University Hospitals NHS Foundation Trust, UK.
| | | | - Emma Doyle
- Edinburgh Cancer Centre, Western General Hospital, Edinburgh, UK.
| | | | | | | | | | | | - Eleanor P J Stride
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, UK.
| | - Mark J Gooding
- Mirada Medical Ltd, Oxford, UK; Division of Cancer Sciences, Faculty of Biology, Medicine and Health, The University of Manchester, Manchester, UK; Inpictura Ltd, Abingdon, UK.
| |
Collapse
|
2
|
Tong G, Jiang H, Shi T, Han XH, Yao YD. A Lightweight Network for Contextual and Morphological Awareness for Hepatic Vein Segmentation. IEEE J Biomed Health Inform 2023; 27:4878-4889. [PMID: 37585324 DOI: 10.1109/jbhi.2023.3305644] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/18/2023]
Abstract
Accurate segmentation of the hepatic vein can improve the precision of liver disease diagnosis and treatment. Since the hepatic venous system is a small target and sparsely distributed, with various and diverse morphology, data labeling is difficult. Therefore, automatic hepatic vein segmentation is extremely challenging. We propose a lightweight contextual and morphological awareness network and design a novel morphology aware module based on attention mechanism and a 3D reconstruction module. The morphology aware module can obtain the slice similarity awareness mapping, which can enhance the continuous area of the hepatic veins in two adjacent slices through attention weighting. The 3D reconstruction module connects the 2D encoder and the 3D decoder to obtain the learning ability of 3D context with a very small amount of parameters. Compared with other SOTA methods, using the proposed method demonstrates an enhancement in the dice coefficient with few parameters on the two datasets. A small number of parameters can reduce hardware requirements and potentially have stronger generalization, which is an advantage in clinical deployment.
Collapse
|
3
|
Mackay K, Bernstein D, Glocker B, Kamnitsas K, Taylor A. A Review of the Metrics Used to Assess Auto-Contouring Systems in Radiotherapy. Clin Oncol (R Coll Radiol) 2023; 35:354-369. [PMID: 36803407 DOI: 10.1016/j.clon.2023.01.016] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/25/2022] [Revised: 12/05/2022] [Accepted: 01/23/2023] [Indexed: 02/01/2023]
Abstract
Auto-contouring could revolutionise future planning of radiotherapy treatment. The lack of consensus on how to assess and validate auto-contouring systems currently limits clinical use. This review formally quantifies the assessment metrics used in studies published during one calendar year and assesses the need for standardised practice. A PubMed literature search was undertaken for papers evaluating radiotherapy auto-contouring published during 2021. Papers were assessed for types of metric and the methodology used to generate ground-truth comparators. Our PubMed search identified 212 studies, of which 117 met the criteria for clinical review. Geometric assessment metrics were used in 116 of 117 studies (99.1%). This includes the Dice Similarity Coefficient used in 113 (96.6%) studies. Clinically relevant metrics, such as qualitative, dosimetric and time-saving metrics, were less frequently used in 22 (18.8%), 27 (23.1%) and 18 (15.4%) of 117 studies, respectively. There was heterogeneity within each category of metric. Over 90 different names for geometric measures were used. Methods for qualitative assessment were different in all but two papers. Variation existed in the methods used to generate radiotherapy plans for dosimetric assessment. Consideration of editing time was only given in 11 (9.4%) papers. A single manual contour as a ground-truth comparator was used in 65 (55.6%) studies. Only 31 (26.5%) studies compared auto-contours to usual inter- and/or intra-observer variation. In conclusion, significant variation exists in how research papers currently assess the accuracy of automatically generated contours. Geometric measures are the most popular, however their clinical utility is unknown. There is heterogeneity in the methods used to perform clinical assessment. Considering the different stages of system implementation may provide a framework to decide the most appropriate metrics. This analysis supports the need for a consensus on the clinical implementation of auto-contouring.
Collapse
Affiliation(s)
- K Mackay
- The Institute of Cancer Research, London, UK; The Royal Marsden Hospital, London, UK.
| | - D Bernstein
- The Institute of Cancer Research, London, UK; The Royal Marsden Hospital, London, UK
| | - B Glocker
- Department of Computing, Imperial College London, South Kensington Campus, London, UK
| | - K Kamnitsas
- Department of Computing, Imperial College London, South Kensington Campus, London, UK; Department of Engineering Science, University of Oxford, Oxford, UK
| | - A Taylor
- The Institute of Cancer Research, London, UK; The Royal Marsden Hospital, London, UK
| |
Collapse
|
4
|
Zhang Y, Liang Y, Ding J, Amjad A, Paulson E, Ahunbay E, Hall WA, Erickson B, Li XA. A Prior Knowledge-Guided, Deep Learning-Based Semiautomatic Segmentation for Complex Anatomy on Magnetic Resonance Imaging. Int J Radiat Oncol Biol Phys 2022; 114:349-359. [PMID: 35667525 PMCID: PMC9639200 DOI: 10.1016/j.ijrobp.2022.05.039] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/09/2021] [Revised: 05/11/2022] [Accepted: 05/23/2022] [Indexed: 10/18/2022]
Abstract
PURPOSE Despite recent substantial improvement in autosegmentation using deep learning (DL) methods, labor-intensive and time-consuming slice-by-slice manual editing is often needed, particularly for complex anatomy (eg, abdominal organs). This work aimed to develop a fast, prior knowledge-guided DL semiautomatic segmentation (DL-SAS) method for complex structures on abdominal magnetic resonance imaging (MRI) scans. METHODS AND MATERIALS A novel application using contours on an adjacent slice as a prior knowledge informant in a 2-dimensional UNet DL model to guide autosegmentation for a subsequent slice was implemented for DL-SAS. A generalized, instead of organ-specific, DL-SAS model was trained and tested for abdominal organs on T2-weighted MRI scans collected from 75 patients (65 for training and 10 for testing). The DL-SAS model performance was compared with 3 common autocontouring methods (linear interpolation, rigid propagation, and a full 3-dimensional DL autosegmentation model trained with the same training data set) based on various quantitative metrics including the Dice similarity coefficient (DSC) and ratio of acceptable slices (ROA) using paired t tests. RESULTS For the 10 testing cases, the DL-SAS model performed best with the slice interval (SI) of 1, resulting in an average DSC of 0.93 ± 0.02, 0.92 ± 0.02, 0.91 ± 0.02, 0.88 ± 0.03, and 0.87 ± 0.02 for the large bowel, stomach, small bowel, duodenum, and pancreas, respectively. The performance decreased with increased SIs from the guidance slice. The DL-SAS method performed significantly better (P < .05) than the other 3 methods. The ROA values were in the range of 48% to 66% for all the organs with an SI of 1 for DL-SAS, higher than those for linear interpolation (31%-57% for an SI of 1) and DL auto-segmentation (16%-51%). CONCLUSIONS The developed DL-SAS model segmented complex abdominal structures on MRI with high accuracy and efficiency and may be implemented as an interactive manual contouring tool or a contour editing tool in conjunction with a full autosegmentation process, facilitating fast and accurate segmentation for MRI-guided online adaptive radiation therapy.
Collapse
Affiliation(s)
- Ying Zhang
- Department of Radiation Oncology, Medical College of Wisconsin, Milwaukee, Wisconsin
| | - Ying Liang
- Department of Radiation Oncology, Medical College of Wisconsin, Milwaukee, Wisconsin
| | - Jie Ding
- Department of Radiation Oncology, Medical College of Wisconsin, Milwaukee, Wisconsin
| | - Asma Amjad
- Department of Radiation Oncology, Medical College of Wisconsin, Milwaukee, Wisconsin
| | - Eric Paulson
- Department of Radiation Oncology, Medical College of Wisconsin, Milwaukee, Wisconsin
| | - Ergun Ahunbay
- Department of Radiation Oncology, Medical College of Wisconsin, Milwaukee, Wisconsin
| | - William A Hall
- Department of Radiation Oncology, Medical College of Wisconsin, Milwaukee, Wisconsin
| | - Beth Erickson
- Department of Radiation Oncology, Medical College of Wisconsin, Milwaukee, Wisconsin
| | - X Allen Li
- Department of Radiation Oncology, Medical College of Wisconsin, Milwaukee, Wisconsin.
| |
Collapse
|
5
|
Trimpl MJ, Primakov S, Lambin P, Stride EPJ, Vallis KA, Gooding MJ. Beyond automatic medical image segmentation-the spectrum between fully manual and fully automatic delineation. Phys Med Biol 2022; 67. [PMID: 35523158 DOI: 10.1088/1361-6560/ac6d9c] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/11/2022] [Accepted: 05/06/2022] [Indexed: 12/19/2022]
Abstract
Semi-automatic and fully automatic contouring tools have emerged as an alternative to fully manual segmentation to reduce time spent contouring and to increase contour quality and consistency. Particularly, fully automatic segmentation has seen exceptional improvements through the use of deep learning in recent years. These fully automatic methods may not require user interactions, but the resulting contours are often not suitable to be used in clinical practice without a review by the clinician. Furthermore, they need large amounts of labelled data to be available for training. This review presents alternatives to manual or fully automatic segmentation methods along the spectrum of variable user interactivity and data availability. The challenge lies to determine how much user interaction is necessary and how this user interaction can be used most effectively. While deep learning is already widely used for fully automatic tools, interactive methods are just at the starting point to be transformed by it. Interaction between clinician and machine, via artificial intelligence, can go both ways and this review will present the avenues that are being pursued to improve medical image segmentation.
Collapse
Affiliation(s)
- Michael J Trimpl
- Mirada Medical Ltd, Oxford, United Kingdom
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, United Kingdom
- Oxford Institute for Radiation Oncology, University of Oxford, Oxford, United Kingdom
| | - Sergey Primakov
- The D-Lab, Department of Precision Medicine, GROW-School for Oncology, Maastricht University, Maastricht, NL, The Netherlands
| | - Philippe Lambin
- The D-Lab, Department of Precision Medicine, GROW-School for Oncology, Maastricht University, Maastricht, NL, The Netherlands
| | - Eleanor P J Stride
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, United Kingdom
| | - Katherine A Vallis
- Oxford Institute for Radiation Oncology, University of Oxford, Oxford, United Kingdom
| | | |
Collapse
|
6
|
Li Z, Huang X, Zhang Z, Liu L, Wang F, Li S, Gao S, Xia J. Synthesis of magnetic resonance images from computed tomography data using convolutional neural network with contextual loss function. Quant Imaging Med Surg 2022; 12:3151-3169. [PMID: 35655819 PMCID: PMC9131350 DOI: 10.21037/qims-21-846] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2021] [Accepted: 02/23/2022] [Indexed: 12/26/2023]
Abstract
BACKGROUND Magnetic resonance imaging (MRI) images synthesized from computed tomography (CT) data can provide more detailed information on pathological structures than that of CT data alone; thus, the synthesis of MRI has received increased attention especially in medical scenarios where only CT images are available. A novel convolutional neural network (CNN) combined with a contextual loss function was proposed for synthesis of T1- and T2-weighted images (T1WI and T2WI) from CT data. METHODS A total of 5,053 and 5,081 slices of T1WI and T2WI, respectively were selected for the dataset of CT and MRI image pairs. Affine registration, image denoising, and contrast enhancement were done on the aforementioned multi-modality medical image dataset comprising T1WI, T2WI, and CT images of the brain. A deep CNN was then proposed by modifying the ResNet structure to constitute the encoder and decoder of U-Net, called double ResNet-U-Net (DRUNet). Three different loss functions were utilized to optimize the parameters of the proposed models: mean squared error (MSE) loss, binary crossentropy (BCE) loss, and contextual loss. Statistical analysis of the independent-sample t-test was conducted by comparing DRUNets with different loss functions and different network layers. RESULTS DRUNet-101 with contextual loss yielded higher values of peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), and Tenengrad function (i.e., 34.25±2.06, 0.97±0.03, and 17.03±2.75 for T1WI and 33.50±1.08, 0.98±0.05, and 19.76±3.54 for T2WI respectively). The results were statistically significant at P<0.001 with a narrow confidence interval of difference, indicating the superiority of DRUNet-101 with contextual loss. In addition, both image zooming and difference maps presented for the final synthetic MR images visually reflected the robustness of DRUNet-101 with contextual loss. The visualization of convolution filters and feature maps showed that the proposed model can generate synthetic MR images with high-frequency information. CONCLUSIONS The results demonstrated that DRUNet-101 with contextual loss function provided better high-frequency information in synthetic MR images compared with the other two functions. The proposed DRUNet model has a distinct advantage over previous models in terms of PSNR, SSIM, and Tenengrad score. Overall, DRUNet-101 with contextual loss is recommended for synthesizing MR images from CT scans.
Collapse
Affiliation(s)
- Zhaotong Li
- Institute of Medical Technology, Peking University Health Science Center, Beijing, China
- Institute of Medical Humanities, Peking University, Beijing, China
| | - Xinrui Huang
- Department of Biochemistry and Biophysics, School of Basic Medical Sciences, Peking University, Beijing, China
| | - Zeru Zhang
- Institute of Medical Technology, Peking University Health Science Center, Beijing, China
- Institute of Medical Humanities, Peking University, Beijing, China
| | - Liangyou Liu
- Institute of Medical Technology, Peking University Health Science Center, Beijing, China
- Institute of Medical Humanities, Peking University, Beijing, China
| | - Fei Wang
- Key Laboratory of Carcinogenesis and Translational Research (Ministry of Education/Beijing), Department of Radiation Oncology, Peking University Cancer Hospital & Institute, Beijing Cancer Hospital & Institute, Beijing, China
| | - Sha Li
- Key Laboratory of Carcinogenesis and Translational Research (Ministry of Education/Beijing), Department of Radiation Oncology, Peking University Cancer Hospital & Institute, Beijing Cancer Hospital & Institute, Beijing, China
| | - Song Gao
- Institute of Medical Technology, Peking University Health Science Center, Beijing, China
| | - Jun Xia
- Department of Radiology, The First Affiliated Hospital of Shenzhen University, Health Science Center, Shenzhen Second People’s Hospital, Shenzhen, China
| |
Collapse
|
7
|
Barragán-Montero A, Bibal A, Dastarac MH, Draguet C, Valdés G, Nguyen D, Willems S, Vandewinckele L, Holmström M, Löfman F, Souris K, Sterpin E, Lee JA. Towards a safe and efficient clinical implementation of machine learning in radiation oncology by exploring model interpretability, explainability and data-model dependency. Phys Med Biol 2022; 67:10.1088/1361-6560/ac678a. [PMID: 35421855 PMCID: PMC9870296 DOI: 10.1088/1361-6560/ac678a] [Citation(s) in RCA: 16] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2021] [Accepted: 04/14/2022] [Indexed: 01/26/2023]
Abstract
The interest in machine learning (ML) has grown tremendously in recent years, partly due to the performance leap that occurred with new techniques of deep learning, convolutional neural networks for images, increased computational power, and wider availability of large datasets. Most fields of medicine follow that popular trend and, notably, radiation oncology is one of those that are at the forefront, with already a long tradition in using digital images and fully computerized workflows. ML models are driven by data, and in contrast with many statistical or physical models, they can be very large and complex, with countless generic parameters. This inevitably raises two questions, namely, the tight dependence between the models and the datasets that feed them, and the interpretability of the models, which scales with its complexity. Any problems in the data used to train the model will be later reflected in their performance. This, together with the low interpretability of ML models, makes their implementation into the clinical workflow particularly difficult. Building tools for risk assessment and quality assurance of ML models must involve then two main points: interpretability and data-model dependency. After a joint introduction of both radiation oncology and ML, this paper reviews the main risks and current solutions when applying the latter to workflows in the former. Risks associated with data and models, as well as their interaction, are detailed. Next, the core concepts of interpretability, explainability, and data-model dependency are formally defined and illustrated with examples. Afterwards, a broad discussion goes through key applications of ML in workflows of radiation oncology as well as vendors' perspectives for the clinical implementation of ML.
Collapse
Affiliation(s)
- Ana Barragán-Montero
- Molecular Imaging, Radiation and Oncology (MIRO) Laboratory, Institut de Recherche Expérimentale et Clinique (IREC), UCLouvain, Belgium
| | - Adrien Bibal
- PReCISE, NaDI Institute, Faculty of Computer Science, UNamur and CENTAL, ILC, UCLouvain, Belgium
| | - Margerie Huet Dastarac
- Molecular Imaging, Radiation and Oncology (MIRO) Laboratory, Institut de Recherche Expérimentale et Clinique (IREC), UCLouvain, Belgium
| | - Camille Draguet
- Molecular Imaging, Radiation and Oncology (MIRO) Laboratory, Institut de Recherche Expérimentale et Clinique (IREC), UCLouvain, Belgium
- Department of Oncology, Laboratory of Experimental Radiotherapy, KU Leuven, Belgium
| | - Gilmer Valdés
- Department of Radiation Oncology, Department of Epidemiology and Biostatistics, University of California, San Francisco, United States of America
| | - Dan Nguyen
- Medical Artificial Intelligence and Automation (MAIA) Laboratory, Department of Radiation Oncology, UT Southwestern Medical Center, United States of America
| | - Siri Willems
- ESAT/PSI, KU Leuven Belgium & MIRC, UZ Leuven, Belgium
| | | | | | | | - Kevin Souris
- Molecular Imaging, Radiation and Oncology (MIRO) Laboratory, Institut de Recherche Expérimentale et Clinique (IREC), UCLouvain, Belgium
| | - Edmond Sterpin
- Molecular Imaging, Radiation and Oncology (MIRO) Laboratory, Institut de Recherche Expérimentale et Clinique (IREC), UCLouvain, Belgium
- Department of Oncology, Laboratory of Experimental Radiotherapy, KU Leuven, Belgium
| | - John A Lee
- Molecular Imaging, Radiation and Oncology (MIRO) Laboratory, Institut de Recherche Expérimentale et Clinique (IREC), UCLouvain, Belgium
| |
Collapse
|
8
|
Med PhysMEDICAL PHYSICS 2021; 48:7563. [PMID: 34390502 DOI: 10.1002/mp.15141] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022] Open
|