A deep registration method for accurate quantification of joint space narrowing progression in rheumatoid arthritis

Rheumatoid arthritis (RA) is a chronic autoimmune inflammatory disease that leads to progressive articular destruction and severe disability. Joint space narrowing (JSN) has been regarded as an important indicator for RA progression and has received significant attention. Radiology plays a crucial role in the diagnosis and monitoring of RA through the assessment of joint space. A new framework for monitoring joint space by quantifying joint space narrowing (JSN) progression through image registration in radiographic images has emerged as a promising research direction. This framework offers the advantage of high accuracy; however, challenges still exist in reducing mismatches and improving reliability. In this work, we utilize a deep intra-subject rigid registration network to automatically quantify JSN progression in the early stages of RA. In our experiments, the mean-square error of the Euclidean distance between the moving and fixed images was 0.0031, the standard deviation was 0.0661 mm and the mismatching rate was 0.48%. Our method achieves sub-pixel level accuracy, surpassing manual measurements significantly. The proposed method is robust to noise, rotation and scaling of joints. Moreover, it provides misalignment visualization, which can assist radiologists and rheumatologists in assessing the reliability of quantification, exhibiting potential for future clinical applications. As a result, we are optimistic that our proposed method will make a significant contribution to the automatic quantification of JSN progression in RA. Code is available at https://github.com/pokeblow/Deep-Registration-QJSN-Finger.git.

Detecting and quantifying spatial misalignment between longitudinal kilovoltage computed tomography (kVCT) scans of the head and neck by using convolutional neural networks (CNNs)

BACKGROUND

Adaptive radiotherapy (ART) aims to address anatomical modifications appearing during the treatment of patients by modifying the planning treatment according to the daily positioning image. Clinical implementation of ART relies on the quality of the deformable image registration (DIR) algorithms included in the ART workflow. To translate ART into clinical practice, automatic DIR assessment is needed.

OBJECTIVE

This article aims to estimate spatial misalignment between two head and neck kilovoltage computed tomography (kVCT) images by using two convolutional neural networks (CNNs).

METHODS

The first CNN quantifies misalignments between 0 mm and 15 mm and the second CNN detects and classifies misalignments into two classes (poor alignment and good alignment). Both networks take pairs of patches of 33x33x33 mm3 as inputs and use only the image intensity information. The training dataset was built by deforming kVCT images with basis splines (B-splines) to simulate DIR error maps. The test dataset was built using 2500 landmarks, consisting of hard and soft landmark tissues annotated by 6 clinicians at 10 locations.

RESULTS

The quantification CNN reaches a mean error of 1.26 mm (± 1.75 mm) on the landmark set which, depending on the location, has annotation errors between 1 mm and 2 mm. The errors obtained for the quantification network fit the computed interoperator error. The classification network achieves an overall accuracy of 79.32%, and although the classification network overdetects poor alignments, it performs well (i.e., it achieves a rate of 90.4%) in detecting poor alignments when given one.

CONCLUSION

The performances of the networks indicate the feasibility of using CNNs for an agnostic and generic approach to misalignment quantification and detection.

Estimating medical image registration error and confidence: A taxonomy and scoping review

Given that image registration is a fundamental and ubiquitous task in both clinical and research domains of the medical field, errors in registration can have serious consequences. Since such errors can mislead clinicians during image-guided therapies or bias the results of a downstream analysis, methods to estimate registration error are becoming more popular. To give structure to this new heterogenous field we developed a taxonomy and performed a scoping review of methods that quantitatively and automatically provide a dense estimation of registration error. The taxonomy breaks down error estimation methods into Approach (Image- or Transformation-based), Framework (Machine Learning or Direct) and Measurement (error or confidence) components. Following the PRISMA guidelines for scoping reviews, the 570 records found were reduced to twenty studies that met inclusion criteria, which were then reviewed according to the proposed taxonomy. Trends in the field, advantages and disadvantages of the methods, and potential sources of bias are also discussed. We provide suggestions for best practices and identify areas of future research.

Scalable quorum-based deep neural networks with adversarial learning for automated lung lobe segmentation in fast helical free-breathing CTs

PURPOSE

Fast helical free-breathing CT (FHFBCT) scans are widely used for 5DCT and 5D Cone Beam imaging protocols. For quantitative analysis of lung physiology and function, it is important to segment the lung lobes in these scans. Since the 5DCT protocols use up to 25 FHFBCT scans, it is important that this segmentation task be automated. In this paper, we present a deep neural network (DNN) framework for segmenting the lung lobes in near real time.

METHODS

A total of 22 patient datasets (550 3D CT scans) were used for the study. Each of the lung lobes was manually segmented and considered ground-truth. A supervised and constrained generative adversarial network (CGAN) was employed for learning each set of lobe segmentations for each patient with 12 patients designated for training data. The resulting generator DNNs represented the lobe segmentations for each training dataset. A quorum-based algorithm was then implemented to test validation data consisting of 10 separate patient datasets (250 3D CTs). Each of the DNNs predicted their corresponding lobes for the validation data, and equal weights were given to the 12 generator CGANs. The quorum process worked by selecting the weighted average result of all 12 CGAN results for each lobe.

RESULTS

When evaluated against ground-truth segmentations, the quorum-based lobe segmentation was observed to have average structural similarity index, normalized cross-correlation coefficient, and dice coefficient values of 0.929, 0.806, and 0.814, respectively, compared to values of 0.911, 0.698, and 0.696, respectively, using a conventional strategy.

CONCLUSION

The proposed quorum-based approach computed segmentations with clinically acceptable accuracy in near real time using a multi-GPU-based computing setup. This method is scalable as more patient-specific CGANs can be added to the quorum over time.

Machine learning applications in radiation oncology

Machine learning technology has a growing impact on radiation oncology with an increasing presence in research and industry. The prevalence of diverse data including 3D imaging and the 3D radiation dose delivery presents potential for future automation and scope for treatment improvements for cancer patients. Harnessing this potential requires standardization of tools and data, and focused collaboration between fields of expertise. The rapid advancement of radiation oncology treatment technologies presents opportunities for machine learning integration with investments targeted towards data quality, data extraction, software, and engagement with clinical expertise. In this review, we provide an overview of machine learning concepts before reviewing advances in applying machine learning to radiation oncology and integrating these techniques into the radiation oncology workflows. Several key areas are outlined in the radiation oncology workflow where machine learning has been applied and where it can have a significant impact in terms of efficiency, consistency in treatment and overall treatment outcomes. This review highlights that machine learning has key early applications in radiation oncology due to the repetitive nature of many tasks that also currently have human review. Standardized data management of routinely collected imaging and radiation dose data are also highlighted as enabling engagement in research utilizing machine learning and the ability integrate these technologies into clinical workflow to benefit patients. Physicists need to be part of the conversation to facilitate this technical integration.

Clinical use, challenges, and barriers to implementation of deformable image registration in radiotherapy - the need for guidance and QA tools

OBJECTIVE

The aim of this study was to evaluate the current status of the clinical use of deformable image registration (DIR) in radiotherapy and to gain an understanding of the challenges faced by centres in clinical implementation of DIR, including commissioning and quality assurance (QA), and to determine the barriers faced. The goal was to inform whether additional guidance and QA tools were needed.

METHODS

A survey focussed on clinical use, metrics used, how centres would like to use DIR in the future and challenges faced, was designed and sent to 71 radiotherapy centres in the UK. Data were gathered specifically on which centres we using DIR clinically, which applications were being used, what commissioning and QA tests were performed, and what barriers were preventing the integration of DIR into the clinical workflow. Centres that did not use DIR clinically were encouraged to fill in the survey and were asked if they have any future plans and in what timescale.

RESULTS

51 out of 71 (70%) radiotherapy centres responded. 47 centres reported access to a commercial software that could perform DIR. 20 centres already used DIR clinically, and 22 centres had plans to implement an application of DIR within 3 years of the survey. The most common clinical application of DIR was to propagate contours from one scan to another (19 centres). In each of the applications, the types of commissioning and QA tests performed varied depending on the type of application and between centres. Some of the key barriers were determining when a DIR was satisfactory including which metrics to use, and lack of resources.

CONCLUSION

The survey results highlighted that there is a need for additional guidelines, training, better tools for commissioning DIR software and for the QA of registration results, which should include developing or recommending which quantitative metrics to use.

ADVANCES IN KNOWLEDGE

This survey has given a useful picture of the clinical use and lack of use of DIR in UK radiotherapy centres. The survey provided useful insight into how centres commission and QA DIR applications, especially the variability among centres. It was also possible to highlight key barriers to implementation and determine factors that may help overcome this which include the need for additional guidance specific to different applications, better tools and metrics.

Deep learning in medical image registration: a review

This paper presents a review of deep learning (DL)-based medical image registration methods. We summarized the latest developments and applications of DL-based registration methods in the medical field. These methods were classified into seven categories according to their methods, functions and popularity. A detailed review of each category was presented, highlighting important contributions and identifying specific challenges. A short assessment was presented following the detailed review of each category to summarize its achievements and future potential. We provided a comprehensive comparison among DL-based methods for lung and brain registration using benchmark datasets. Lastly, we analyzed the statistics of all the cited works from various aspects, revealing the popularity and future trend of DL-based medical image registration.

Winter is over: The use of Artificial Intelligence to individualise radiation therapy for breast cancer

Artificial intelligence demonstrated its value for automated contouring of organs at risk and target volumes as well as for auto-planning of radiation dose distributions in terms of saving time, increasing consistency, and improving dose-volumes parameters. Future developments include incorporating dose/outcome data to optimise dose distributions with optimal coverage of the high-risk areas, while at the same time limiting doses to low-risk areas. An infinite gradient of volumes and doses to deliver spatially-adjusted radiation can be generated, allowing to avoid unnecessary radiation to organs at risk. Therefore, data about patient-, tumour-, and treatment-related factors have to be combined with dose distributions and outcome-containing databases.

Applications of Deep Learning to Neuro-Imaging Techniques

Many clinical applications based on deep learning and pertaining to radiology have been proposed and studied in radiology for classification, risk assessment, segmentation tasks, diagnosis, prognosis, and even prediction of therapy responses. There are many other innovative applications of AI in various technical aspects of medical imaging, particularly applied to the acquisition of images, ranging from removing image artifacts, normalizing/harmonizing images, improving image quality, lowering radiation and contrast dose, and shortening the duration of imaging studies. This article will address this topic and will seek to present an overview of deep learning applied to neuroimaging techniques.

Deformable image registration for radiation therapy: principle, methods, applications and evaluation

Background: Deformable image registration (DIR) is increasingly used in the field of radiation therapy (RT) to account for anatomical deformations. The aims of this paper are to describe the main applications of DIR in RT and discuss current DIR evaluation methods. Methods: Articles on DIR published from January 2000 to October 2018 were extracted from PubMed and Science Direct. Our search was restricted to articles that report data obtained from humans, were written in English, and address DIR methods for RT. A total of 207 articles were selected from among 2506 identified in the search process. Results: At planning, DIR is used for organ delineation using atlas-based segmentation, deformation-based planning target volume definition, functional planning and magnetic resonance imaging-based dose calculation. In image-guided RT, DIR is used for contour propagation and dose calculation on per-treatment imaging. DIR is also used to determine the accumulated dose from fraction to fraction in external beam RT and brachytherapy, both for dose reporting and adaptive RT. In the case of re-irradiation, DIR can be used to estimate the cumulated dose of the two irradiations. Finally, DIR can be used to predict toxicity in voxel-wise population analysis. However, the evaluation of DIR remains an open issue, especially when dealing with complex cases such as the disappearance of matter. To quantify DIR uncertainties, most evaluation methods are limited to geometry-based metrics. Software companies have now integrated DIR tools into treatment planning systems for clinical use, such as contour propagation and fraction dose accumulation. Conclusions: DIR is increasingly important in RT applications, from planning to toxicity prediction. DIR is routinely used to reduce the workload of contour propagation. However, its use for complex dosimetric applications must be carefully evaluated by combining quantitative and qualitative analyses.

A feasibility study on an automated method to generate patient-specific dose distributions for radiotherapy using deep learning

PURPOSE

To develop a method for predicting optimal dose distributions, given the planning image and segmented anatomy, by applying deep learning techniques to a database of previously optimized and approved Intensity-modulated radiation therapy treatment plans.

METHODS

Eighty cases of early-stage nasopharyngeal cancer (NPC) were included in the study. Seventy cases were chosen randomly as the training set and the remaining as the test set. The inputs were the images with structures, with each target and organs at risk (OARs) assigned a unique label. The outputs were dose maps, including coarse dose maps and converted fine dose maps (FDM) from convolution. Two types of input images with structures were used in the model building. One type of input included the images (with associated structures) without manipulation. The second type of input involved modifying the image gray label with information from radiation beam geometry. ResNet101 was chosen as the deep learning network for both. The accuracy of predicted dose distributions was evaluated against the corresponding dose as used in the clinic. A global three-dimensional gamma analysis was calculated for the evaluation.

RESULTS

The proposed model trained with the two different sets of input images and structures could both predict patient-specific dose distributions accurately. For the out-of-field dose distributions, the model obtained from the input with radiation geometry performed better (dose difference in %, 4.7 ± 6.1% vs 5.5 ± 7.9%, P < 0.05). The mean Gamma pass rates of dose distributions predicted with both types of input were comparable for most OARs (P > 0.05), except for the bilateral optic nerves and the optic chiasm.

CONCLUSIONS

The proposed system with radiation geometry added to the input is a promising method to generate patient-specific dose distributions for radiotherapy. It can be applied to obtain the dose distributions slice-by-slice for planning quality assurance and for guiding automated planning.

Deep learning in medical imaging and radiation therapy

The goals of this review paper on deep learning (DL) in medical imaging and radiation therapy are to (a) summarize what has been achieved to date; (b) identify common and unique challenges, and strategies that researchers have taken to address these challenges; and (c) identify some of the promising avenues for the future both in terms of applications as well as technical innovations. We introduce the general principles of DL and convolutional neural networks, survey five major areas of application of DL in medical imaging and radiation therapy, identify common themes, discuss methods for dataset expansion, and conclude by summarizing lessons learned, remaining challenges, and future directions.