Deep Adversarial Context-Aware Landmark Detection for Ultrasound Imaging

Influence of learned landmark correspondences on lung CT registration

BACKGROUND

Disease or injury may cause a change in the biomechanical properties of the lungs, which can alter lung function. Image registration can be used to measure lung ventilation and quantify volume change, which can be a useful diagnostic aid. However, lung registration is a challenging problem because of the variation in deformation along the lungs, sliding motion of the lungs along the ribs, and change in density.

PURPOSE

Landmark correspondences have been used to make deformable image registration robust to large displacements.

METHODS

To tackle the challenging task of intra-patient lung computed tomography (CT) registration, we extend the landmark correspondence prediction model deep convolutional neural network-Match by introducing a soft mask loss term to encourage landmark correspondences in specific regions and avoid the use of a mask during inference. To produce realistic deformations to train the landmark correspondence model, we use data-driven synthetic transformations. We study the influence of these learned landmark correspondences on lung CT registration by integrating them into intensity-based registration as a distance-based penalty.

RESULTS

Our results on the public thoracic CT dataset COPDgene show that using learned landmark correspondences as a soft constraint can reduce median registration error from approximately 5.46 to 4.08 mm compared to standard intensity-based registration, in the absence of lung masks.

CONCLUSIONS

We show that using landmark correspondences results in minor improvements in local alignment, while significantly improving global alignment.

Knee landmarks detection via deep learning for automatic imaging evaluation of trochlear dysplasia and patellar height

OBJECTIVES

To develop and validate a deep learning-based approach to automatically measure the patellofemoral instability (PFI) indices related to patellar height and trochlear dysplasia in knee magnetic resonance imaging (MRI) scans.

METHODS

A total of 763 knee MRI slices from 95 patients were included in the study, and 3393 anatomical landmarks were annotated for measuring sulcus angle (SA), trochlear facet asymmetry (TFA), trochlear groove depth (TGD) and lateral trochlear inclination (LTI) to assess trochlear dysplasia, and Insall-Salvati index (ISI), modified Insall-Salvati index (MISI), Caton Deschamps index (CDI) and patellotrochlear index (PTI) to assess patellar height. A U-Net based network was implemented to predict the landmarks' locations. The successful detection rate (SDR) and the mean absolute error (MAE) evaluation metrics were used to evaluate the performance of the network. The intraclass correlation coefficient (ICC) was also used to evaluate the reliability of the proposed framework to measure the mentioned PFI indices.

RESULTS

The developed models achieved good accuracy in predicting the landmarks' locations, with a maximum value for the MAE of 1.38 ± 0.76 mm. The results show that LTI, TGD, ISI, CDI and PTI can be measured with excellent reliability (ICC > 0.9), and SA, TFA and MISI can be measured with good reliability (ICC > 0.75), with the proposed framework.

CONCLUSIONS

This study proposes a reliable approach with promising applicability for automatic patellar height and trochlear dysplasia assessment, assisting the radiologists in their clinical practice.

CLINICAL RELEVANCE STATEMENT

The objective knee landmarks detection on MRI images provided by artificial intelligence may improve the reproducibility and reliability of the imaging evaluation of trochlear anatomy and patellar height, assisting radiologists in their clinical practice in the patellofemoral instability assessment.

KEY POINTS

• Imaging evaluation of patellofemoral instability is subjective and vulnerable to substantial intra and interobserver variability. • Patellar height and trochlear dysplasia are reliably assessed in MRI by means of artificial intelligence (AI). • The developed AI framework provides an objective evaluation of patellar height and trochlear dysplasia enhancing the clinical practice of the radiologists.

Generative Adversarial Network (Generative Artificial Intelligence) in Pediatric Radiology: A Systematic Review

Generative artificial intelligence, especially with regard to the generative adversarial network (GAN), is an important research area in radiology as evidenced by a number of literature reviews on the role of GAN in radiology published in the last few years. However, no review article about GAN in pediatric radiology has been published yet. The purpose of this paper is to systematically review applications of GAN in pediatric radiology, their performances, and methods for their performance evaluation. Electronic databases were used for a literature search on 6 April 2023. Thirty-seven papers met the selection criteria and were included. This review reveals that the GAN can be applied to magnetic resonance imaging, X-ray, computed tomography, ultrasound and positron emission tomography for image translation, segmentation, reconstruction, quality assessment, synthesis and data augmentation, and disease diagnosis. About 80% of the included studies compared their GAN model performances with those of other approaches and indicated that their GAN models outperformed the others by 0.1-158.6%. However, these study findings should be used with caution because of a number of methodological weaknesses. For future GAN studies, more robust methods will be essential for addressing these issues. Otherwise, this would affect the clinical adoption of the GAN-based applications in pediatric radiology and the potential advantages of GAN could not be realized widely.

Robust landmark-based brain shift correction with a Siamese neural network in ultrasound-guided brain tumor resection

PURPOSE

In brain tumor surgery, tissue shift (called brain shift) can move the surgical target and invalidate the surgical plan. A cost-effective and flexible tool, intra-operative ultrasound (iUS) with robust image registration algorithms can effectively track brain shift to ensure surgical outcomes and safety.

METHODS

We proposed to employ a Siamese neural network, which was first trained using natural images and fine-tuned with domain-specific data to automatically detect matching anatomical landmarks in iUS scans at different surgical stages. An efficient 2.5D approach and an iterative re-weighted least squares algorithm are utilized to perform landmark-based registration for brain shift correction. The proposed method is validated and compared against the state-of-the-art methods using the public BITE and RESECT datasets.

RESULTS

Registration of pre-resection iUS scans to during- and post-resection iUS images were executed. The results with the proposed method shows a significant improvement from the initial misalignment ([Formula: see text]) and the method is comparable to the state-of-the-art methods validated on the same datasets.

CONCLUSIONS

We have proposed a robust technique to efficiently detect matching landmarks in iUS and perform brain shift correction with excellent performance. It has the potential to improve the accuracy and safety of neurosurgery.

Improving GAN Learning Dynamics for Thyroid Nodule Segmentation

Thyroid nodules are lesions requiring diagnosis and follow-up. Tools for detecting and segmenting nodules can help physicians with this diagnosis. Besides immediate diagnosis, automated tools can also enable tracking of the probability of malignancy over time. This paper demonstrates a new algorithm for segmenting thyroid nodules in ultrasound images. The algorithm combines traditional supervised semantic segmentation with unsupervised learning using GANs. The hybrid approach has the potential to upgrade the semantic segmentation model's performance, but GANs have the well-known problems of unstable learning and mode collapse. To stabilize the training of the GAN model, we introduce the concept of closed-loop control of the gain on the loss output of the discriminator. We find gain control leads to smoother generator training and avoids the mode collapse that typically occurs when the discriminator learns too quickly relative to the generator. We also find that the combination of the supervised and unsupervised learning styles encourages both low-level accuracy and high-level consistency. As a test of the concept of controlled hybrid supervised and unsupervised semantic segmentation, we introduce a new model named the StableSeg GAN. The model uses DeeplabV3+ as the generator, Resnet18 as the discriminator, and uses PID control to stabilize the GAN learning process. The performance of the new model in terms of IoU is better than DeeplabV3+, with mean IoU of 81.26% on a challenging test set. The results of our thyroid nodule segmentation experiments show that StableSeg GANs have flexibility to segment nodules more accurately than either comparable supervised segmentation models or uncontrolled GANs.

Automatic Plane of Minimal Hiatal Dimensions Extraction From 3D Female Pelvic Floor Ultrasound

There is an increasing interest in the applications of 3D ultrasound imaging of the pelvic floor to improve the diagnosis, treatment, and surgical planning of female pelvic floor dysfunction (PFD). Pelvic floor biometrics are obtained on an oblique image plane known as the plane of minimal hiatal dimensions (PMHD). Identifying this plane requires the detection of two anatomical landmarks, the pubic symphysis and anorectal angle. The manual detection of the anatomical landmarks and the PMHD in 3D pelvic ultrasound requires expert knowledge of the pelvic floor anatomy, and is challenging, time-consuming, and subject to human error. These challenges have hindered the adoption of such quantitative analysis in the clinic. This work presents an automatic approach to identify the anatomical landmarks and extract the PMHD from 3D pelvic ultrasound volumes. To demonstrate clinical utility and a complete automated clinical task, an automatic segmentation of the levator-ani muscle on the extracted PMHD images was also performed. Experiments using 73 test images of patients during a pelvic muscle resting state showed that this algorithm has the capability to accurately identify the PMHD with an average Dice of 0.89 and an average mean boundary distance of 2.25mm. Further evaluation of the PMHD detection algorithm using 35 images of patients performing pelvic muscle contraction resulted in an average Dice of 0.88 and an average mean boundary distance of 2.75mm. This work had the potential to pave the way towards the adoption of ultrasound in the clinic and development of personalized treatment for PFD.

Attention-based generative adversarial network in medical imaging: A narrative review

As a popular probabilistic generative model, generative adversarial network (GAN) has been successfully used not only in natural image processing, but also in medical image analysis and computer-aided diagnosis. Despite the various advantages, the applications of GAN in medical image analysis face new challenges. The introduction of attention mechanisms, which resemble the human visual system that focuses on the task-related local image area for certain information extraction, has drawn increasing interest. Recently proposed transformer-based architectures that leverage self-attention mechanism encode long-range dependencies and learn representations that are highly expressive. This motivates us to summarize the applications of using transformer-based GAN for medical image analysis. We reviewed recent advances in techniques combining various attention modules with different adversarial training schemes, and their applications in medical segmentation, synthesis and detection. Several recent studies have shown that attention modules can be effectively incorporated into a GAN model in detecting lesion areas and extracting diagnosis-related feature information precisely, thus providing a useful tool for medical image processing and diagnosis. This review indicates that research on the medical imaging analysis of GAN and attention mechanisms is still at an early stage despite the great potential. We highlight the attention-based generative adversarial network is an efficient and promising computational model advancing future research and applications in medical image analysis.

Medical Robotics for Ultrasound Imaging: Current Systems and Future Trends

Purpose of Review

This review provides an overview of the most recent robotic ultrasound systems that have contemporary emerged over the past five years, highlighting their status and future directions. The systems are categorized based on their level of robot autonomy (LORA).

Recent Findings

Teleoperating systems show the highest level of technical maturity. Collaborative assisting and autonomous systems are still in the research phase, with a focus on ultrasound image processing and force adaptation strategies. However, missing key factors are clinical studies and appropriate safety strategies. Future research will likely focus on artificial intelligence and virtual/augmented reality to improve image understanding and ergonomics.

Summary

A review on robotic ultrasound systems is presented in which first technical specifications are outlined. Hereafter, the literature of the past five years is subdivided into teleoperation, collaborative assistance, or autonomous systems based on LORA. Finally, future trends for robotic ultrasound systems are reviewed with a focus on artificial intelligence and virtual/augmented reality.