Volumetric Ultrasound Imaging for the Whole Soft Tissue: Toward Enhanced Thyroid Disease Examination

OBJECTIVES

Ultrasound imaging (USI) is the gold standard in the clinical diagnosis of thyroid diseases. Compared with two-dimensional (2D) USI, three-dimensional (3D) USI could provide more structural information. However, the unstable pressure generated by the hand-hold ultrasound probe scanning can cause tissue deformation, especially in soft tissues such as the thyroid. The deformation is manifested as tissue structure being compressed in 2D USI, which results in structural discontinuity in 3D USI. Furthermore, multiple scans apply pressure in different directions to the tissue, which will cause relative displacement between the 3D images obtained from multiple thyroid scans.

METHODS

In this work, we proposed a framework to minimize the influence of the variation of pressure in thyroid 3D USI. To correct pressure artifacts in a single scanning sequence, an adaptive method to smooth the position of the 2D ultrasound (US) image sequence is adopted before performing volumetric reconstruction. To build a whole 3D US image including both sides of the thyroid gland, an iterative closest point (ICP) based registration pipeline is adopted to eliminate the relative displacement caused by different pressure directions.

RESULTS

Our proposed method was validated by in vivo experiments, including healthy volunteers and volunteers with thyroid nodules at different grading levels.

CONCLUSIONS

The thyroid gland and nodule are rendered intelligently in the whole scanning region to facilitate the observation of 3D USI results by the doctor. This work might make a positive contribution to the clinical diagnosis of diseases of the thyroid or other soft tissues.

Review of robotic systems for thoracoabdominal puncture interventional surgery

Cancer, with high morbidity and high mortality, is one of the major burdens threatening human health globally. Intervention procedures via percutaneous puncture have been widely used by physicians due to its minimally invasive surgical approach. However, traditional manual puncture intervention depends on personal experience and faces challenges in terms of precisely puncture, learning-curve, safety and efficacy. The development of puncture interventional surgery robotic (PISR) systems could alleviate the aforementioned problems to a certain extent. This paper attempts to review the current status and prospective of PISR systems for thoracic and abdominal application. In this review, the key technologies related to the robotics, including spatial registration, positioning navigation, puncture guidance feedback, respiratory motion compensation, and motion control, are discussed in detail.

Quantitative validation of two model-based methods for the correction of probe pressure deformation in ultrasound

PURPOSE

The acquisition of good quality ultrasound (US) images requires good acoustic coupling between the ultrasound probe and the patient's skin. In practice, this good coupling is achieved by the operator applying a force to the skin through the probe. This force induces a deformation of the tissues underlying the probe. The distorted images deteriorate the quality of the reconstructed 3D US image.

METHODS

In this work, we propose two methods to correct these deformations. These methods are based on the construction of a biomechanical model to predict the mechanical behavior of the imaged soft tissues. The originality of the methods is that they do not use external information (force or position value from sensors, or elasticity value from the literature). The model is parameterized thanks to the information contained in the image. This is allowed thanks to the optimization of two key parameters for the model which are the indentation d and the elasticity ratio α.

RESULTS

The validation is performed on real images acquired on a gelatin-based phantom using an ultrasound probe inducing an increasing vertical indentation using a step motor. The results showed a good correction of the two methods for indentations less than 4 mm. For larger indentations, one of the two methods (guided by the similarity score) provides a better quality of correction, presenting a Euclidean distance between the contours of the reference image and the corrected image of 0.71 mm.

CONCLUSION

The proposed methods ensured the correction of the deformed images induced by a linear probe pressure without using any information coming from sensors (force or position), or generic information about the mechanical parameters. The corrected images can be used to obtain a corrected 3D US image.

DefCor-Net: Physics-aware ultrasound deformation correction

The recovery of morphologically accurate anatomical images from deformed ones is challenging in ultrasound (US) image acquisition, but crucial to accurate and consistent diagnosis, particularly in the emerging field of computer-assisted diagnosis. This article presents a novel physics-aware deformation correction approach based on a coarse-to-fine, multi-scale deep neural network (DefCor-Net). To achieve pixel-wise performance, DefCor-Net incorporates biomedical knowledge by estimating pixel-wise stiffness online using a U-shaped feature extractor. The deformation field is then computed using polynomial regression by integrating the measured force applied by the US probe. Based on real-time estimation of pixel-by-pixel tissue properties, the learning-based approach enables the potential for anatomy-aware deformation correction. To demonstrate the effectiveness of the proposed DefCor-Net, images recorded at multiple locations on forearms and upper arms of six volunteers are used to train and validate DefCor-Net. The results demonstrate that DefCor-Net can significantly improve the accuracy of deformation correction to recover the original geometry (Dice Coefficient: from 14.3±20.9 to 82.6±12.1 when the force is 6N). Code:https://github.com/KarolineZhy/DefCorNet.

Autonomous Robotic Ultrasound Vascular Imaging System With Decoupled Control Strategy for External-Vision-Free Environments

OBJECTIVE

Ultrasound (US) probes scan over the surface of the human body to acquire US images in clinical vascular US diagnosis. However, due to the deformation and specificity of different human surfaces, the relationship between the scan trajectory of the skin and the internal tissues is not fully correlated, which poses a challenge for autonomous robotic US imaging in a dynamic and external-vision-free environment. Here, we propose a decoupled control strategy for autonomous robotic vascular US imaging in an environment without external vision.

METHODS

The proposed system is divided into outer-loop posture control and inner-loop orientation control, which are separately determined by a deep learning (DL) agent and a reinforcement learning (RL) agent. First, we use a weakly supervised US vessel segmentation network to estimate the probe orientation. In the outer loop control, we use a force-guided reinforcement learning agent to maintain a specific angle between the US probe and the skin in the dynamic imaging processes. Finally, the orientation and the posture are integrated to complete the imaging process.

RESULTS

Evaluation experiments on several volunteers showed that our RUS could autonomously perform vascular imaging in arms with different stiffness, curvature, and size without additional system adjustments. Furthermore, our system achieved reproducible imaging and reconstruction of dynamic targets without relying on vision-based surface information.

CONCLUSION AND SIGNIFICANCE

Our system and control strategy provides a novel framework for the application of US robots in complex and external-vision-free environments.

Robotic ultrasound imaging: State-of-the-art and future perspectives

Ultrasound (US) is one of the most widely used modalities for clinical intervention and diagnosis due to the merits of providing non-invasive, radiation-free, and real-time images. However, free-hand US examinations are highly operator-dependent. Robotic US System (RUSS) aims at overcoming this shortcoming by offering reproducibility, while also aiming at improving dexterity, and intelligent anatomy and disease-aware imaging. In addition to enhancing diagnostic outcomes, RUSS also holds the potential to provide medical interventions for populations suffering from the shortage of experienced sonographers. In this paper, we categorize RUSS as teleoperated or autonomous. Regarding teleoperated RUSS, we summarize their technical developments, and clinical evaluations, respectively. This survey then focuses on the review of recent work on autonomous robotic US imaging. We demonstrate that machine learning and artificial intelligence present the key techniques, which enable intelligent patient and process-specific, motion and deformation-aware robotic image acquisition. We also show that the research on artificial intelligence for autonomous RUSS has directed the research community toward understanding and modeling expert sonographers' semantic reasoning and action. Here, we call this process, the recovery of the "language of sonography". This side result of research on autonomous robotic US acquisitions could be considered as valuable and essential as the progress made in the robotic US examination itself. This article will provide both engineers and clinicians with a comprehensive understanding of RUSS by surveying underlying techniques. Additionally, we present the challenges that the scientific community needs to face in the coming years in order to achieve its ultimate goal of developing intelligent robotic sonographer colleagues. These colleagues are expected to be capable of collaborating with human sonographers in dynamic environments to enhance both diagnostic and intraoperative imaging.

Portable Device to Assist With Force Control in Ultrasound Acquisition

This study presents a portable device that ensures precise contact force between a subject and a probe to improve the stability and reproducibility of ultrasound (US) acquisition. The mechanical portion of the device includes a servo motor, gears, and a ball screw linear actuator; two photoelectric switches are used to limit the stroke. A combined force and position control system is developed, and a pressure threshold is introduced to reduce the chattering of the system so that it can be applied to US examinations of tissues of different stiffness levels. Force control experiments were conducted on the device, and the results showed that the device can overcome the chattering of a physician's hand and movement caused by a subject's respiration. Additionally, the stability of the US acquisition was substantially improved. Based on clinical trials on humans, this device was observed to improve the consistency of ultrasonic results and the repeatability of images, and it assisted sonographers with maintaining suitable contact force and improving imaging quality. The device can either be handheld by a physician or easily integrated with a manipulator as an autonomous robotic US acquisition device, thereby validating its potential for US applications.

On the importance of patient acceptance for medical robotic imaging

PURPOSE

Mutual acceptance is required for any human-to-human interaction. Therefore, one would assume that this also holds for robot-patient interactions. However, the medical robotic imaging field lacks research in the area of acceptance. This work, therefore, aims at analyzing the influence of robot-patient interactions on acceptance in an exemplary medical robotic imaging system.

METHODS

We designed an interactive human-robot scenario, including auditive and gestural cues, and compared this pipeline to a non-interactive scenario. Both scenarios were evaluated through a questionnaire to measure acceptance. Heart rate monitoring was also used to measure stress. The impact of the interaction was quantified in the use case of robotic ultrasound scanning of the neck.

RESULTS

We conducted the first user study on patient acceptance of robotic ultrasound. Results show that verbal interactions impacts trust more than gestural ones. Furthermore, through interaction, the robot is perceived to be friendlier. The heart rate data indicates that robot-patient interaction could reduce stress.

CONCLUSIONS

Robot-patient interactions are crucial for improving acceptance in medical robotic imaging systems. While verbal interaction is most important, the preferred interaction type and content are participant dependent. Heart rate values indicate that such interactions can also reduce stress. Overall, this initial work showed that interactions improve patient acceptance in medical robotic imaging, and other medical robot-patient systems can benefit from the design proposals to enhance acceptance in interactive scenarios.

Nondeformed Ultrasound Image Production Method for Ultrasound-Guided Radiotherapy

Purpose: During ultrasound (US)-guided radiotherapy, the tissue is deformed by probe pressure, and the US image is limited by changes in tissue and organ position and geometry when the US image is aligned with computed tomography (CT) image, leading to poor alignment. Accordingly, a pixel displacement-based nondeformed US image production method is proposed. Methods: The correction of US image deformation is achieved by calculating the pixel displacement of an image. The positioning CT image (CTstd) is used as the gold standard. The deformed US image (USdef) is inputted into the Harris algorithm to extract corner points for selecting feature points, and the displacement of adjacent pixels of feature points in the US video stream is calculated using the Lucas-Kanade optical flow algorithm. The moving least squares algorithm is used to correct USdef globally and locally in accordance with image pixel displacement to generate a nondeformed US image (USrev). In addition, USdef and USrev were separately aligned with CTstd to evaluate the improvement of alignment accuracy through deformation correction. Results: In the phantom experiment, the overall and local average correction errors of the US image under the optimal probe pressure were 1.0944 and 0.7388 mm, respectively, and the registration accuracy of USdef and USrev with CTstd was 0.6764 and 0.9016, respectively. During the volunteer experiment, the correction error of all 12 patients' data ranged from -1.7525 to 1.5685 mm, with a mean absolute error of 0.8612 mm. The improvement range of US and CT registration accuracy, before and after image deformation correction in the 12 patients evaluated by a normalized correlation coefficient, was 0.1232 to 0.2476. Conclusion: The pixel displacement-based deformation correction method can solve the limitation imposed by image deformation on image alignment in US-guided radiotherapy. Compared with USdef, the alignment results of USrev with CT were better.