Remote control of a robotic prosthesis arm with six-degree-of-freedom for ultrasonic scanning and three-dimensional imaging

MSK-TIM: A Telerobotic Ultrasound System for Assessing the Musculoskeletal System

The aim of this paper is to investigate technological advancements made to a robotic tele-ultrasound system for musculoskeletal imaging, the MSK-TIM (Musculoskeletal Telerobotic Imaging Machine). The hardware was enhanced with a force feedback sensor and a new controller was introduced. Software improvements were developed which allowed the operator to access ultrasound functions such as focus, depth, gain, zoom, color, and power Doppler controls. The device was equipped with Wi-Fi network capability which allowed the master and slave stations to be positioned in different locations. A trial assessing the system to scan the wrist was conducted with twelve participants, for a total of twenty-four arms. Both the participants and radiologist reported their experience. The images obtained were determined to be of satisfactory quality for diagnosis. The system improvements resulted in a better user and patient experience for the radiologist and participants. Latency with the VPN configuration was similar to the WLAN in our experiments. This research explores several technologies in medical telerobotics and provides insight into how they should be used in future. This study provides evidence to support larger-scale trials of the MSK-TIM for musculoskeletal imaging.

A review on tele-manipulators for remote diagnostic procedures and surgery

With modern medicine and healthcare services improving in leaps and bounds, the integration of telemedicine has helped in expanding these specialised healthcare services to remote locations. Healthcare telerobotic systems form a component of telemedicine, which allows medical intervention from a distance. It has been nearly 40 years since a robotic technology, PUMA 560, was introduced to perform a stereotaxic biopsy in the brain. The use of telemanipulators for remote surgical procedures began around 1995, with the Aesop, the Zeus, and the da Vinci robotic surgery systems. Since then, the utilisation of robots has steadily increased in diverse healthcare disciplines, from clinical diagnosis to telesurgery. The telemanipulator system functions in a master–slave protocol mode, with the doctor operating the master system, aided by audio-visual and haptic feedback. Based on the control commands from the master, the slave system, a remote manipulator, interacts directly with the patient. It eliminates the requirement for the doctor to be physically present in the spatial vicinity of the patient by virtually bringing expert-guided medical services to them. Post the Covid-19 pandemic, an exponential surge in the utilisation of telerobotic systems has been observed. This study aims to present an organised review of the state-of-the-art telemanipulators used for remote diagnostic procedures and surgeries, highlighting their challenges and scope for future research and development.

Current Applications of Robot-Assisted Ultrasound Examination

Despite advances in miniaturization and automation, the need for expert acquisition of a full echocardiogram, including Doppler, has restricted access in remote areas. Recent developments in robotics, teleoperation, and upgraded telecommunications infrastructure may provide a solution to this deficiency. Robot-assisted teleoperated ultrasound examination can aid medical diagnosis in remote locations and may improve health inequalities between rural and urban settings. This review aimed to analyze the status of teleoperated robotic systems for ultrasound examinations, evaluate clinical and preclinical applications, identify limitations, and outline future directions for clinical use. Overall, robot-assisted teleoperated ultrasound is feasible and safe in the reported clinical and preclinical studies, with the robots able to follow the hand movements performed by sonographers and researchers from a distance or in local networks. Moreover, multiple types of ultrasound examinations have been performed in remote areas, with a high success rate nearly comparable to that of conventional sonography. The studies showed that although a low-bandwidth link can be used to control a robot, the bandwidth requirements for real-time transmission of video and ultrasound images are significantly higher. Furthermore, if haptic feedback is implemented, the bandwidth requirements are increased. Haptically enabled systems that improve robotic control are necessary for accelerating the introduction to clinical use. Haptic feedback and enhanced front-end interface control for remote users are vital aspects required for clinical application. The incorporation of artificial intelligence through either aiding in window acquisition (knowledge of anatomical landmarks to adjust scanning planes) or through measurement and disease identification is yet to be researched. However, it has the potential to lead to dramatic advances. A new generation of robots is in development, and several projects in the preclinical stage reveal a promising future to overcome the shortage of health professionals in remote areas.

A multimodal user interface for touchless control of robotic ultrasound

PURPOSE

Past research contained the investigation and development of robotic ultrasound. In this context, interfaces which allow for interaction with the robotic system are of paramount importance. Few researchers have addressed the issue of developing non-tactile interaction approaches, although they could be beneficial for maintaining sterility during medical procedures. Interaction could be supported by multimodality, which has the potential to enable intuitive and natural interaction. To assess the feasibility of multimodal interaction for non-tactile control of a co-located robotic ultrasound system, a novel human-robot interaction concept was developed.

METHODS

The medical use case of needle-based interventions under hybrid computed tomography and ultrasound imaging was analyzed by interviewing four radiologists. From the resulting workflow, interaction tasks were derived which include human-robot interaction. Based on this, characteristics of a multimodal, touchless human-robot interface were elaborated, suitable interaction modalities were identified, and a corresponding interface was developed, which was thereafter evaluated in a user study with eight participants.

RESULTS

The implemented interface includes voice commands, combined with hand gesture control for discrete control and navigation interaction of the robotic US probe, respectively. The interaction concept was evaluated by the users in the form of a quantitative questionnaire with a average usability. Qualitative analysis of interview results revealed user satisfaction with the implemented interaction methods and potential improvements to the system.

CONCLUSION

A multimodal, touchless interaction concept for a robotic US for the use case of needle-based procedures in interventional radiology was developed, incorporating combined voice and hand gesture control. Future steps will include the integration of a solution for the missing haptic feedback and the evaluation of its clinical suitability.

Telerobotic Sonography for Remote Diagnostic Imaging: Narrative Review of Current Developments and Clinical Applications

Access to sonographers and sonologists is limited in many communities around the world. Telerobotic sonography (robotic ultrasound) is a new technology to increase access to sonography, providing sonographers and sonologists the ability to manipulate an ultrasound probe from a distant location and remotely perform ultrasound examinations. This narrative review discusses the development of telerobotic ultrasound systems, clinical studies evaluating the feasibility and diagnostic accuracy of telerobotic sonography, and emerging use of telerobotic sonography in clinical settings. Telerobotic sonography provides an opportunity to provide real-time ultrasound examinations to underserviced rural and remote communities to increase equity in the delivery of diagnostic imaging.

Performance Enhanced Ultrasound Probe Tracking With a Hemispherical Marker Rigid Body

Among tracking techniques applied in the 3-D freehand ultrasound (US), the camera-based tracking method is relatively mature and reliable. However, constrained by manufactured marker rigid bodies, the US probe is usually limited to operate within a narrow rotational range before occlusion issues affect accurate and robust tracking performance. Thus, this study proposed a hemispherical marker rigid body to hold passive noncoplanar markers so that the markers could be identified by the camera, mitigating self-occlusion. The enlarged rotational range provides greater freedom for sonographers while performing examinations. The single-axis rotational and translational tracking performances of the system, equipped with the newly designed marker rigid body, were investigated and evaluated. Tracking with the designed marker rigid body achieved high tracking accuracy with 0.57° for the single-axis rotation and 0.01 mm for the single-axis translation for sensor distance between 1.5 and 2 m. In addition to maintaining high accuracy, the system also possessed an enhanced ability to capture over 99.76% of the motion data in the experiments. The results demonstrated that with the designed marker rigid body, the missing data were remarkably reduced from over 15% to less than 0.5%, which enables interpolation in the data postprocessing. An imaging test was further conducted, and the volume reconstruction of a four-month fetal phantom was demonstrated using the motion data obtained from the tracking system.

An Improved Sensing Method of a Robotic Ultrasound System for Real-Time Force and Angle Calibration

An ultrasonic examination is a clinically universal and safe examination method, and with the development of telemedicine and precision medicine, the robotic ultrasound system (RUS) integrated with a robotic arm and ultrasound imaging system receives increasing attention. As the RUS requires precision and reproducibility, it is important to monitor the real-time calibration of the RUS during examination, especially the angle of the probe for image detection and its force on the surface. Additionally, to speed up the integration of the RUS and the current medical ultrasound system (US), the current RUSs mostly use a self-designed fixture to connect the probe to the arm. If the fixture has inconsistencies, it may cause an operating error. In order to improve its resilience, this study proposed an improved sensing method for real-time force and angle calibration. Based on multichannel pressure sensors, an inertial measurement unit (IMU), and a novel sensing structure, the ultrasonic probe and robotic arm could be simply and rapidly combined, which rendered real-time force and angle calibration at a low cost. The experimental results show that the average success rate of the downforce position identification achieved was 88.2%. The phantom experiment indicated that the method could assist the RUS in the real-time calibration of both force and angle during an examination.

Medical Robotics for Ultrasound Imaging: Current Systems and Future Trends

Purpose of Review

This review provides an overview of the most recent robotic ultrasound systems that have contemporary emerged over the past five years, highlighting their status and future directions. The systems are categorized based on their level of robot autonomy (LORA).

Recent Findings

Teleoperating systems show the highest level of technical maturity. Collaborative assisting and autonomous systems are still in the research phase, with a focus on ultrasound image processing and force adaptation strategies. However, missing key factors are clinical studies and appropriate safety strategies. Future research will likely focus on artificial intelligence and virtual/augmented reality to improve image understanding and ergonomics.

Summary

A review on robotic ultrasound systems is presented in which first technical specifications are outlined. Hereafter, the literature of the past five years is subdivided into teleoperation, collaborative assistance, or autonomous systems based on LORA. Finally, future trends for robotic ultrasound systems are reviewed with a focus on artificial intelligence and virtual/augmented reality.