Tele-operated robotic ultrasound system for medical diagnosis

An accurate paradigm for denoising degraded ultrasound images based on artificial intelligence systems

Ultrasound images are susceptible to various forms of quality degradation that negatively impact diagnosis. Common degradations include speckle noise, Gaussian noise, salt and pepper noise, and blurring. This research proposes an accurate ultrasound image denoising strategy based on firstly detecting the noise type, then, suitable denoising methods can be applied for each corruption. The technique depends on convolutional neural networks to categorize the type of noise affecting an input ultrasound image. Pre-trained convolutional neural network models including GoogleNet, VGG-19, AlexNet and AlexNet-support vector machine (SVM) are developed and trained to perform this classification. A dataset of 782 numerically generated ultrasound images across different diseases and noise types is utilized for model training and evaluation. Results show AlexNet-SVM achieves the highest accuracy of 99.2% in classifying noise types. The results indicate that, the present technique is considered one of the top-performing models is then applied to real ultrasound images with different noise corruptions to demonstrate efficacy of the proposed detect-then-denoise system. RESEARCH HIGHLIGHTS: Proposes an accurate ultrasound image denoising strategy based on detecting noise type first. Uses pre-trained convolutional neural networks to categorize noise type in input images. Evaluates GoogleNet, VGG-19, AlexNet, and AlexNet-support vector machine (SVM) models on a dataset of 782 synthetic ultrasound images. AlexNet-SVM achieves highest accuracy of 99.2% in classifying noise types. Demonstrates efficacy of the proposed detect-then-denoise system on real ultrasound images.

A review of robot-assisted ultrasound examination: Systems and technology

BACKGROUND

At present, the number and overall level of ultrasound (US) doctors cannot meet the medical needs, and the medical ultrasound robots will largely solve the shortage of medical resources.

METHODS

According to the degree of automation, the handheld, semi-automatic and automatic ultrasound examination robot systems are summarised. Ultrasound scanning path planning and robot control are the keys to ensure that the robot systems can obtain high-quality images. Therefore, the ultrasound scanning path planning and control methods are summarised. The research progress and future trends are discussed.

RESULTS

A variety of ultrasound robot systems have been applied to various medical works. With the continuous improvement of automation, the systems provide high-quality ultrasound images and image guidance for clinicians.

CONCLUSION

Although the development of medical ultrasound robot still faces challenges, with the continuous progress of robot technology and communication technology, medical ultrasound robot will have great development potential and broad application space.

Feasibility and Safety of Percutaneous Puncture Guided by a 5G-Based Telerobotic Ultrasound System: An Experimental Study

PURPOSE

To evaluate the feasibility and safety of percutaneous puncture guided by a 5th generation mobile communication technology (5G)-based telerobotic ultrasound system in phantom and animal experiments.

MATERIALS AND METHODS

In the phantom experiment, 10 simulated lesions were punctured, once at each of two angles for each lesion, under the guidance of a telerobotic ultrasound system and ultrasound-guided freehand puncture. Student's t test was used to compare the two methods in terms of puncture accuracy, total operation duration, and puncture duration. In the animal experiment, under the guidance of the telerobotic ultrasound system, an 18G puncture needle was used to puncture 3 target steel beads in the liver, right kidney, and right gluteal muscle, respectively. The animal experiment had no freehand ultrasound-guided control group. After puncture, a CT scan was performed to verify the position of the puncture needle in relation to the target, and the complications and puncture duration, etc., were recorded.

RESULTS

In the phantom experiment, the mean accuracies of puncture under telerobotic ultrasound guidance and conventional ultrasound guidance were 1.8 ± 0.3 mm and 1.6 ± 0.3 mm (P = 0.09), respectively; therefore, there was no significant difference in the accuracy of the two guide methods. In the animal experiment, the first-attempt puncture success (the needle tip close to the target) rate was 93%. Polypnea occurred during one puncture. No other intraoperative or postoperative complications were observed.

CONCLUSION

Puncture guided by a 5G-based telerobotic ultrasound system has shown good feasibility and safety in phantom and animal experiments.

MSK-TIM: A Telerobotic Ultrasound System for Assessing the Musculoskeletal System

The aim of this paper is to investigate technological advancements made to a robotic tele-ultrasound system for musculoskeletal imaging, the MSK-TIM (Musculoskeletal Telerobotic Imaging Machine). The hardware was enhanced with a force feedback sensor and a new controller was introduced. Software improvements were developed which allowed the operator to access ultrasound functions such as focus, depth, gain, zoom, color, and power Doppler controls. The device was equipped with Wi-Fi network capability which allowed the master and slave stations to be positioned in different locations. A trial assessing the system to scan the wrist was conducted with twelve participants, for a total of twenty-four arms. Both the participants and radiologist reported their experience. The images obtained were determined to be of satisfactory quality for diagnosis. The system improvements resulted in a better user and patient experience for the radiologist and participants. Latency with the VPN configuration was similar to the WLAN in our experiments. This research explores several technologies in medical telerobotics and provides insight into how they should be used in future. This study provides evidence to support larger-scale trials of the MSK-TIM for musculoskeletal imaging.

A 5G-based telerobotic ultrasound system provides qualified abdominal ultrasound services for patients on a rural island: a prospective and comparative study of 401 patients

PURPOSE

To explore the feasibility of a 5G-based telerobotic ultrasound (US) system for providing qualified abdominal US services on a rural island.

METHODS

This prospective study involved two medical centers (the tele-radiologist site's hospital and the patient site's hospital) separated by 72 km. Patients underwent 5G-based telerobotic US by tele-radiologists and conventional US by on-site radiologists from September 2020 to March 2021. The clinical feasibility and diagnostic performance of the 5G-based telerobotic abdominal US examination were assessed based on safety, duration, image quality, diagnostic findings, and questionnaires.

RESULTS

A total of 401 patients (217 women and 184 men; mean age, 54.96 ± 15.43 years) were enrolled. A total of 90.1% of patients indicated no discomfort with the telerobotic US examination. For the examination duration, telerobotic US took longer than conventional US (12.54 ± 3.20 min vs. 7.23 ± 2.10 min, p = 0.001). For image quality scores, the results of the two methods were similar (4.54 ± 0.63 vs. 4.57 ± 0.61, p = 0.112). No significant differences were found between the two methods in measurements for the aorta, portal vein, gallbladder, kidney (longitudinal diameter), prostate, and uterus; however, telerobotic US underestimated the transverse diameter of the kidney (p < 0.05). A total of 504 positive results, including 31 different diseases, were detected. Among them, 455 cases were identified by the two methods; 17 cases were identified by telerobotic US only; and 32 cases were identified by conventional US only. There was good consistency in the diagnosis of 29 types of disease between the two methods (κ = 0.773-1.000). Furthermore, more than 90% of patients accepted the telerobotic US examination and agreed to pay additional fees in future.

CONCLUSION

The 5G-based telerobotic US system can expand access to abdominal US services for patients in rural areas, thereby reducing health care disparities.

A Multimodal Point Cloud-Based Method for Tumor Localization in Robotic Ultrasound-Guided Radiotherapy

Objectives: Part of the tumor localization methods in radiotherapy have poor real-time performance and may generate additional radiation. We propose a multimodal point cloud-based method for tumor localization in robotic ultrasound-guided radiotherapy, which only irradiates computed tomography (CT) during radiotherapy planning to avoid additional radiation. Methods: The tumor position was determined using the CT point cloud, and the red green blue depth (RGBD) point cloud was used to determine body surface scanning location corresponding to the tumor location. The relationship between the CT point cloud and RGBD point cloud was established through multi-modal point cloud registration. The point cloud was then used for robot tumor localization through coordinate transformation between camera and robot. Results: The maximum mean absolute error of the tumor location in the X, Y, and Z directions of the robot coordinate system were 0.781, 1.334, and 1.490 mm, respectively. The average point-to-point translation mean absolute error between the actual and predicted positions of the localization points was 1.847 mm. The maximum error in the random positioning experiment was 1.77 mm. Conclusion: The proposed method is radiation free and has real-time performance, with tumor localization accuracy that meets the requirements of radiotherapy. The proposed method, which potentially reduces the risks associated with radiation exposure while ensuring efficient and accurate tumor localization, represents a promising advancement in the field of radiotherapy.

Remote ultrasound real‐time consultation and quality control system

Ultrasound examination is widely used, but there are problems such as difficulties in quality control and unbalanced distribution of ultrasound resources in primary hospitals. In this article, a novel remote ultrasound real‐time consultation and quality control system was developed. The system uses advanced information science technology, medical image acquisition equipment, audio and video coding and decoding technology, combined User Datagram Protocol (UDP) and Transmission Control Protocol (TCP) transfer protocols, server cluster design technology. It realizes remote real‐time medical consultation and quality control between the doctors in the consulation center and doctors in the ultrasound examination room. The image similarity, transmission delay, and efficiency of ultrasound treatment were tested. System test data show that the quality of ultrasound images transmitted to the consultation center is the same as the quality of the original ultrasound images, the distortion is less than 1%, and the audio and video transmission delay is less than 0.2 s. The system significantly improves the efficiency and quality of ultrasound diagnosis and treatment through its high interactivity, and can also help improve the level of ultrasound diagnosis and treatment in primary hospitals.

Tissue ultrasound imaging based on wavelet correlation analysis and pulse-inversion technique

BACKGROUND

Pulse-inversion-based tissue harmonic imaging has been utilized for many years because it can effectively eliminate the harmonic leakage and produce low side-lobe. However, the pulse inversion method is sensitive to imaging object movements, which may result in motion artifacts. Spatial resolution and contrast were limited.

OBJECTIVE

To improve ultrasound image quality by a new pulse-inversion-based tissue harmonic imaging technique.

METHODS

Continuous wavelet transform is applied to investigate the correlation between mother wavelet and the received echoes from two opposite pulses. To get a better correlation, a novel mother wavelet named 'tissue wavelet' is designed based on the Khokhlov-Zabolotskaya- Kuznetsov (KZK) wave equation. Radio frequency data were obtained from open Ultrasonix SonixTouch imaging system. Experiments were carried on ultrasonic tissue phantom, human carotid artery and human liver.

RESULTS

The average improvement of lateral spatial resolution is 49.52% compared to pulse-inversion-based tissue second-harmonic Imaging (PIHI). Contrast ratio (CR) and contrast-to-noise ratio (CNR) increased by 5.55 dB and 1.40 dB over PIHI. Tissue wavelet performs better than Mexh and Morl wavelet in lateral spatial resolution, CR, and CNR.

CONCLUSION

The proposed technique effectively improves the imaging quality in lateral spatial resolution, CR, and CNR.

Visualizing tactile feedback: an overview of current technologies with a focus on ultrasound elastography

Tissue elasticity remains an essential biomarker of health and is indicative of irregularities such as tumors or infection. The timely detection of such abnormalities is crucial for the prevention of disease progression and complications that arise from late-stage illnesses. However, at both the bedside and the operating table, there is a distinct lack of tactile feedback for deep-seated tissue. As surgical techniques advance toward remote or minimally invasive options to reduce infection risk and hasten healing time, surgeons lose the ability to manually palpate tissue. Furthermore, palpation of deep structures results in decreased accuracy, with the additional barrier of needing years of experience for adequate confidence of diagnoses. This review delves into the current modalities used to fulfill the clinical need of quantifying physical touch. It covers research efforts involving tactile sensing for remote or minimally invasive surgeries, as well as the potential of ultrasound elastography to further this field with non-invasive real-time imaging of the organ's biomechanical properties. Elastography monitors tissue response to acoustic or mechanical energy and reconstructs an image representative of the elastic profile in the region of interest. This intuitive visualization of tissue elasticity surpasses the tactile information provided by sensors currently used to augment or supplement manual palpation. Focusing on common ultrasound elastography modalities, we evaluate various sensing mechanisms used for measuring tactile information and describe their emerging use in clinical settings where palpation is insufficient or restricted. With the ongoing advancements in ultrasound technology, particularly the emergence of micromachined ultrasound transducers, these devices hold great potential in facilitating early detection of tissue abnormalities and providing an objective measure of patient health.

Visual Perception and Convolutional Neural Network-Based Robotic Autonomous Lung Ultrasound Scanning Localization System

Under the situation of severe COVID-19 epidemic, lung ultrasound (LUS) has been proved to be an effective and convenient method to diagnose and evaluate the extent of respiratory disease. However, the traditional clinical ultrasound (US) scanning requires doctors not only to be in close contact with patients but also to have rich experience. In order to alleviate the shortage of medical resources and reduce the work stress and risk of infection for doctors, we propose a visual perception and convolutional neural network (CNN)-based robotic autonomous LUS scanning localization system to realize scanned target recognition, probe pose solution and movement, and the acquisition of US images. The LUS scanned targets are identified through the target segmentation and localization algorithm based on the improved CNN, which is using the depth camera to collect the image information; furthermore, the method based on multiscale compensation normal vector is used to solve the attitude of the probe; finally, a position control strategy based on force feedback is designed to optimize the position and attitude of the probe, which can not only obtain high-quality US images but also ensure the safety of patients and the system. The results of human LUS scanning experiment verify the accuracy and feasibility of the system. The positioning accuracy of the scanned targets is 15.63 ± 0.18 mm, and the distance accuracy and rotation angle accuracy of the probe position calculation are 6.38 ± 0.25 mm and 8.60° ±2.29° , respectively. More importantly, the obtained high-quality US images can clearly capture the main pathological features of the lung. The system is expected to be applied in clinical practice.

Common Educational Teleoperation Platform for Robotics Utilizing Digital Twins

The erratic modern world introduces challenges to all sectors of societies and potentially introduces additional inequality. One possibility to decrease the educational inequality is to provide remote access to facilities that enable learning and training. A similar approach of remote resource usage can be utilized in resource-poor situations where the required equipment is available at other premises. The concept of Industry 5.0 (i5.0) focuses on a human-centric approach, enabling technologies to concentrate on human–machine interaction and emphasizing the importance of societal values. This paper introduces a novel robotics teleoperation platform supported by the i5.0. The platform reduces inequality and allows usage and learning of robotics remotely independently of time and location. The platform is based on digital twins with bi-directional data transmission between the physical and digital counterparts. The proposed system allows teleoperation, remote programming, and near real-time monitoring of controlled robots, robot time scheduling, and social interaction between users. The system design and implementation are described in detail, followed by experimental results.

Successful Use of a 5G-Based Robot-Assisted Remote Ultrasound System in a Care Center for Disabled Patients in Rural China

Background

Disability has become a global population health challenge. Due to difficulties in self-care or independent living, patients with disability mainly live in community-based care centers or institutions for long-term care. Nonetheless, these settings often lack basic medical resources, such as ultrasonography. Thus, remote ultrasonic robot technology for clinical applications across wide regions is imperative. To date, few experiences of remote diagnostic systems in rural care centers have been reported.

Objective

To assess the feasibility of a fifth-generation cellular technology (5G)-based robot-assisted remote ultrasound system in a care center for disabled patients in rural China.

Methods

Patients underwent remote robot-assisted and bedside ultrasound examinations of the liver, gallbladder, spleen, and kidneys. We compared the diagnostic consistency and differences between the two modalities and evaluated the examination duration, image quality, and safety.

Results

Forty-nine patients were included (21 men; mean age: 61.0 ± 19.0 [range: 19-91] years). Thirty-nine and ten had positive and negative results, respectively; 67 lesions were detected. Comparing the methods, 41 and 8 patients had consistent and inconsistent diagnoses, respectively. The McNemar and kappa values were 0.727 and 0.601, respectively. The mean duration of remote and bedside examinations was 12.2 ± 4.5 (range: 5-26) min and 7.5 ± 1.8 (range: 5-13) min (p < 0.001), respectively. The median image score for original images on the patient side and transmitted images on the doctor side was 5 points (interquartile range: [IQR]: 4.7-5.0) and 4.7 points (IQR: 4.5-5.0) (p = 0.176), respectively. No obvious complications from the examination were reported.

Conclusions

A 5G-based robot-assisted remote ultrasound system is feasible and has comparable diagnostic efficiency to traditional bedside ultrasound. This system may provide a unique solution for basic ultrasound diagnostic services in primary healthcare settings.

Clinical application of a 5G-based telerobotic ultrasound system for thyroid examination on a rural island: a prospective study

PURPOSE

To evaluate the feasibility of a 5G-based telerobotic ultrasound (US) system for thyroid examination on a rural island.

METHODS

From September 2020 to March 2021, this prospectively study enrolled a total of 139 patients (average age, 58.6 ± 12.7 years) included 33 males and 106 females, who underwent 5G-based telerobotic thyroid US examination by a tele-doctor at Shanghai Tenth People's Hospital and a conventional thyroid US examination at Chongming Second People's Hospital 84 km away. The clinical feasibility of 5G-based telerobotic US for thyroid examination were evaluated in terms of safety, duration, US image quality, diagnostic results, and questionnaire survey.

RESULTS

92.8% of patients had no examination-related complaints. The average duration of the 5G-based telerobotic US examination was similar as that of conventional US examination (5.57 ± 2.20 min vs. 5.23 ± 2.1 min, P = 0.164). The image quality of telerobotic US correlated well with that of conventional US (4.63 ± 0.60 vs. 4.65 ± 0.61, P = 0.102). There was no significant difference between two types of US examination methods for the diameter measurement of the thyroid, cervical lymph nodes, and thyroid nodules. Two lymphadenopathies and 20 diffuse thyroid diseases were detected in two types of US methods. 124 thyroid nodules were detected by telerobotic US and 127 thyroid nodules were detected by conventional US. Among them, 122 were the same thyroid nodules. In addition, there were good consistency in the US features (component, echogenicity, shape, and calcification) and ACR TI-RADS category of the same thyroid nodules between telerobotic and conventional US examinations (ICC = 0.788-0.863). 85.6% of patients accepted the telerobotic US, and 87.1% were willing to pay extra fee for the telerobotic US.

CONCLUSION

The 5G-based telerobotic US system can be a routine diagnostic tool for thyroid examination for patients on a rural island.

Dual Mode pHRI-teleHRI Control System with a Hybrid Admittance-Force Controller for Ultrasound Imaging

The COVID-19 pandemic has brought unprecedented extreme pressure on the medical system due to the physical distance policy, especially for procedures such as ultrasound (US) imaging, which are usually carried out in person. Tele-operation systems are a promising way to avoid physical human-robot interaction (pHRI). However, the system usually requires another robot on the remote doctor side to provide haptic feedback, which makes it expensive and complex. To reduce the cost and system complexity, in this paper, we present a low-cost, easy-to-use, dual-mode pHRI-teleHRI control system with a custom-designed hybrid admittance-force controller for US imaging. The proposed system requires only a tracking camera rather than a sophisticated robot on the remote side. An audio feedback is designed for replacing haptic feedback on the remote side, and its sufficiency is experimentally verified. The experimental results indicate that the designed hybrid controller can significantly improve the task performance in both modes. Furthermore, the proposed system enables the user to conduct US imaging while complying with the physical distance policy, and allows them to seamlessly switch modes from one to another in an online manner. The novel system can be easily adapted to other medical applications beyond the pandemic, such as tele-healthcare, palpation, and auscultation.