Robot-assisted ultrasound imaging: overview and development of a parallel telerobotic system

Elevation Resolution Enhancement Oriented 3D Ultrasound Imaging

Three-dimensional (3D) ultrasound imaging can be accomplished by reconstructing a sequence of two-dimensional (2D) ultrasound images. However, 2D ultrasound images usually suffer from low resolution in the elevation direction, thereby impacting the accuracy of 3D reconstructed results. The lateral resolution of 2D ultrasound is known to significantly exceed the elevation resolution. By combining scanning sequences acquired from orthogonal directions, the effects of poor elevation resolution can be mitigated through a composite reconstructing process. Moreover, capturing ultrasound images from multiple perspectives necessitates a precise probe positioning method with a wide angle of coverage. Optical tracking is popularly used for probe positioning for its high accuracy and environment-robustness. In this paper, a novel large-angle accurate optical positioning method is used for enhancing resolution in 3D ultrasound imaging through orthogonal-view scanning and composite reconstruction. Experiments on two phantoms proved that our method could significantly improve reconstruction accuracy in the elevation direction of the probe compared with single-angle parallel scanning. The results indicate that our method holds the potential to improve current 3D ultrasound imaging techniques.

A fully autonomous robotic ultrasound system for thyroid scanning

The current thyroid ultrasound relies heavily on the experience and skills of the sonographer and the expertise of the radiologist, and the process is physically and cognitively exhausting. In this paper, we report a fully autonomous robotic ultrasound system, which is able to scan thyroid regions without human assistance and identify malignant nod- ules. In this system, human skeleton point recognition, reinforcement learning, and force feedback are used to deal with the difficulties in locating thyroid targets. The orientation of the ultrasound probe is adjusted dynamically via Bayesian optimization. Experimental results on human participants demonstrated that this system can perform high-quality ultrasound scans, close to manual scans obtained by clinicians. Additionally, it has the potential to detect thyroid nodules and provide data on nodule characteristics for American College of Radiology Thyroid Imaging Reporting and Data System (ACR TI-RADS) calculation.

Progress in the Application of Portable Ultrasound Combined with Artificial Intelligence in Pre-Hospital Emergency and Disaster Sites

With the miniaturization of ultrasound and the development of artificial intelligence, its application in disaster scenes and pre-hospital emergency care has become more and more common. This study summarizes the literature on portable ultrasound in pre-hospital emergency and disaster scene treatment in the past decade and reviews the development and application of portable ultrasound. Portable ultrasound diagnostic equipment can be used to diagnose abdominal bleeding, limb fracture, hemopneumothorax, pericardial effusion, etc., based on which trauma can be diagnosed pre-hospital and provide guiding suggestions for the next triage and rescue; in early rescue, portable ultrasound can guide emergency operations, such as tracheal intubation, pericardial cavity puncture, and thoracic and abdominal puncture as well as improve the accuracy and timeliness of operation techniques. In addition, with the development of artificial intelligence (AI), AI-assisted diagnosis can improve the diagnosis level of ultrasound at disaster sites. The portable ultrasound diagnosis system equipped with an AI robotic arm can maximize the pre-screening classification and fast and concise diagnosis and treatment of batch casualties, thus providing a reliable basis for batch casualty classification and evacuation at disaster accident sites.

Miniaturized Endoscopic 2D US Transducer for Volumetric Ultrasound Imaging of the Auditory System

OBJECTIVE

In this paper, we focus on the carrying out and validation of minimally invasive three-dimensional (3D) ultrasound (US) imaging of the auditory system, which is based on a new miniaturized endoscopic 2D US transducer.

METHODS

This unique probe consists of a 18 MHz 24 elements curved array transducer with a distal diameter of 4 mm so it can be inserted into the external auditory canal. Typical acquisition is achieved by rotating such a transducer around its own axis using a robotic platform. Reconstruction of a US volume from the set of acquired B-scans during the rotation is then performed using scan-conversion. The accuracy of the reconstruction procedure is evaluated using a dedicated phantom that includes a set of wires as reference geometry.

RESULTS

Twelve acquisitions obtained from different probe poses are compared to a micro-computed tomographic model of the phantom, leading to a maximum error of 0.20 mm. Additionally, acquisitions with a cadaveric head highlight the clinical applicability of this set up. Structures of the auditory system such as the ossicles and the round window can be identified from the obtained 3D volumes.

CONCLUSION

These results confirm that our technique enables the accurate imaging of the middle and inner ears without having to deteriorate the surrounding bone.

SIGNIFICANCE

Since US is a real-time, wide available and non-ionizing imaging modality, our acquisition setup could facilitate the minimally invasive diagnosis and surgical navigation for otology in a fast, cost-effective and safe way.

Toward Ergonomic Design in Ultrasound Scanning: Strategies to Mitigate Injurious Forces

ABSTRACT

Sonographers experience stress-related musculoskeletal injuries at alarming rates. The magnitude of the problem has resulted in substantial economic and human costs. Early efforts, such as the transposition of office ergonomic models to the ultrasound suite, have been only marginally successful because of the vast difference between clerical and sonography work demands. In addition, although the issue has attracted the attention of professional sonographer societies, equipment manufacturers, and governmental agencies, such as the OSHA (Occupational Safety and Health), the causative mechanisms by which sonographers are injured remain inadequately addressed. A definitive ergonomic solution has been elusive.With the help of occupational therapists and biomechanical engineers, we have developed a number of possible ergonomic modifications of ultrasound equipment. We explain the rationale behind these approaches to transducer design and assess the strengths and deficiencies of each.

Remote Ultrasound Scan Procedures with Medical Robots: Towards New Perspectives between Medicine and Engineering

Background

This review explores state-of-the-art teleoperated robots for medical ultrasound scan procedures, providing a comprehensive look including the recent trends arising from the COVID-19 pandemic.

Methods

Physicians' experience is included to indicate the importance of their role in the design of improved medical robots. From this perspective, novel classes of equipment for remote diagnostics based on medical robotics are discussed in terms of innovative engineering technologies.

Results

Relevant literature is reviewed under the system engineering point of view, organizing the discussion on the basis of the main technological focus of each contribution.

Conclusions

This contribution is aimed at stimulating new research to obtain faster results on teleoperated robotics for ultrasound diagnostics in response to the high demand raised by the ongoing pandemic.

Preliminary examination of the potential of robot-assisted sonography - An ergonomic tool for obstetric sonographers

The aim of this study was to explore the ergonomic challenges, the needs and reservations related to robot-assisted ultrasound for obstetric sonographers and thereby to provide information for the design of robotic solutions. A mixed-method design was used, where data from the obstetric sonographers and their immediate managers from 18 out of a Danish total of 20 obstetric departments was collected. The data was collected through a survey and interviews. 98.1% of the obstetric sonographers experienced ache, pain or discomfort related to scans. The most frequent cause for the sonographers' ergonomic challenges were the patients' physique (93,52%) and the need to obtain good image quality (83,33%). These reasons are non-controllable parameters for the obstetric sonographers and requires a solution, which ergonomically supports the sonographers in these situations. All of the interviewed obstetric sonographers (n = 8) and immediate managers (n = 3) claimed they were interested in testing a solution based on robot-assisted ultrasound.

Telerobotic Sonography for Remote Diagnostic Imaging: Narrative Review of Current Developments and Clinical Applications

Access to sonographers and sonologists is limited in many communities around the world. Telerobotic sonography (robotic ultrasound) is a new technology to increase access to sonography, providing sonographers and sonologists the ability to manipulate an ultrasound probe from a distant location and remotely perform ultrasound examinations. This narrative review discusses the development of telerobotic ultrasound systems, clinical studies evaluating the feasibility and diagnostic accuracy of telerobotic sonography, and emerging use of telerobotic sonography in clinical settings. Telerobotic sonography provides an opportunity to provide real-time ultrasound examinations to underserviced rural and remote communities to increase equity in the delivery of diagnostic imaging.

Medical Robotics for Ultrasound Imaging: Current Systems and Future Trends

Purpose of Review

This review provides an overview of the most recent robotic ultrasound systems that have contemporary emerged over the past five years, highlighting their status and future directions. The systems are categorized based on their level of robot autonomy (LORA).

Recent Findings

Teleoperating systems show the highest level of technical maturity. Collaborative assisting and autonomous systems are still in the research phase, with a focus on ultrasound image processing and force adaptation strategies. However, missing key factors are clinical studies and appropriate safety strategies. Future research will likely focus on artificial intelligence and virtual/augmented reality to improve image understanding and ergonomics.

Summary

A review on robotic ultrasound systems is presented in which first technical specifications are outlined. Hereafter, the literature of the past five years is subdivided into teleoperation, collaborative assistance, or autonomous systems based on LORA. Finally, future trends for robotic ultrasound systems are reviewed with a focus on artificial intelligence and virtual/augmented reality.

Cardiac ultrasonography over 4G wireless networks using a tele-operated robot

This Letter proposes an end-to-end mobile tele-echography platform using a portable robot for remote cardiac ultrasonography. Performance evaluation investigates the capacity of long-term evolution (LTE) wireless networks to facilitate responsive robot tele-manipulation and real-time ultrasound video streaming that qualifies for clinical practice. Within this context, a thorough video coding standards comparison for cardiac ultrasound applications is performed, using a data set of ten ultrasound videos. Both objective and subjective (clinical) video quality assessment demonstrate that H.264/AVC and high efficiency video coding standards can achieve diagnostically-lossless video quality at bitrates well within the LTE supported data rates. Most importantly, reduced latencies experienced throughout the live tele-echography sessions allow the medical expert to remotely operate the robot in a responsive manner, using the wirelessly communicated cardiac ultrasound video to reach a diagnosis. Based on preliminary results documented in this Letter, the proposed robotised tele-echography platform can provide for reliable, remote diagnosis, achieving comparable quality of experience levels with in-hospital ultrasound examinations.

Medical telerobotic systems: current status and future trends

Teleoperated medical robotic systems allow procedures such as surgeries, treatments, and diagnoses to be conducted across short or long distances while utilizing wired and/or wireless communication networks. This study presents a systematic review of the relevant literature between the years 2004 and 2015, focusing on medical teleoperated robotic systems which have witnessed tremendous growth over the examined period. A thorough insight of telerobotics systems discussing design concepts, enabling technologies (namely robotic manipulation, telecommunications, and vision systems), and potential applications in clinical practice is provided, while existing limitations and future trends are also highlighted. A representative paradigm of the short-distance case is the da Vinci Surgical System which is described in order to highlight relevant issues. The long-distance telerobotics concept is exemplified through a case study on diagnostic ultrasound scanning. Moreover, the present review provides a classification into short- and long-distance telerobotic systems, depending on the distance from which they are operated. Telerobotic systems are further categorized with respect to their application field. For the reviewed systems are also examined their engineering characteristics and the employed robotics technology. The current status of the field, its significance, the potential, as well as the challenges that lie ahead are thoroughly discussed.

Robot-assisted automatic ultrasound calibration

PURPOSE

Ultrasound (US) calibration is the process of determining the unknown transformation from a coordinate frame such as the robot's tooltip to the US image frame and is a necessary task for any robotic or tracked US system. US calibration requires submillimeter-range accuracy for most applications, but it is a time-consuming and repetitive task. We provide a new framework for automatic US calibration with robot assistance and without the need for temporal calibration.

METHOD

US calibration based on active echo (AE) phantom was previously proposed, and its superiority over conventional cross-wire phantom-based calibration was shown. In this work, we use AE to guide the robotic arm motion through the process of data collection; we combine the capability of the AE point to localize itself in the frame of the US image with the automatic motion of the robotic arm to provide a framework for calibrating the arm to the US image automatically.

RESULTS

We demonstrated the efficacy of the automated method compared to the manual method through experiments. To highlight the necessity of frequent ultrasound calibration, it is demonstrated that the calibration precision changed from 1.67 to 3.20 mm if the data collection is not repeated after a dismounting/mounting of the probe holder. In a large data set experiment, similar reconstruction precision of automatic and manual data collection was observed, while the time was reduced by 58 %. In addition, we compared ten automatic calibrations with ten manual ones, each performed in 15 min, and showed that all the automatic ones could converge in the case of setting the initial matrix as identity, while this was not achieved by manual data sets. Given the same initial matrix, the repeatability of the automatic was [0.46, 0.34, 0.80, 0.47] versus [0.42, 0.51, 0.98, 1.15] mm in the manual case for the US image four corners.

CONCLUSIONS

The submillimeter accuracy requirement of US calibration makes frequent data collections unavoidable. We proposed an automated calibration setup and showed feasibility by implementing it for a robot tooltip to US image calibration. The automated method showed a similar reconstruction precision as well as repeatability compared to the manual method, while the time consumed for data collection was reduced. The automatic method also reduces the burden of data collection for the user. Thus, the automated method can be a viable solution for applications that require frequent calibrations.

System Integration and Preliminary In-Vivo Experiments of a Robot for Ultrasound Guidance and Monitoring during Radiotherapy

We are developing a cooperatively-controlled robot system in which a clinician and robot share control of a 3D ultrasound (US) probe. The goals of the system are to provide guidance for patient setup and real-time target monitoring during fractionated radiotherapy. Currently, there is limited use of realtime US image feedback during radiotherapy for lower abdominal organs and it has not yet been clinically applied for upper abdominal organs. One challenge is that placing an US probe on the patient produces tissue deformation around the target organ, leading to displacement of the target. Our solution is to perform treatment planning on the deformed organ and then to reproduce this deformation during radiotherapy. We therefore introduce a robot system to hold the US probe on the patient. In order to create a consistent deformation, the system records the robot position, contact force, and reference US image during simulation and then introduces virtual constraints (soft virtual fixtures) to guide the clinician to correctly place the probe during the fractionated treatments. Because the robot is under-actuated (5 motorized and 6 passive degrees-of-freedom), the guidance also involves a graphical user interface (adjustment GUI) to achieve the desired probe orientation. This paper presents the integrated system, a proposed clinical workflow, the results of an initial in-vivo canine study with a 3-DOF robot, and the results of phantom experiments with an improved 5-DOF robotic system. The results suggest that the guidance may enable the clinician to more consistently and accurately place the US probe.