Deep learning robotic guidance for autonomous vascular access

Artificial intelligence in interventional radiology: state of the art

Artificial intelligence (AI) has demonstrated great potential in a wide variety of applications in interventional radiology (IR). Support for decision-making and outcome prediction, new functions and improvements in fluoroscopy, ultrasound, computed tomography, and magnetic resonance imaging, specifically in the field of IR, have all been investigated. Furthermore, AI represents a significant boost for fusion imaging and simulated reality, robotics, touchless software interactions, and virtual biopsy. The procedural nature, heterogeneity, and lack of standardisation slow down the process of adoption of AI in IR. Research in AI is in its early stages as current literature is based on pilot or proof of concept studies. The full range of possibilities is yet to be explored.Relevance statement Exploring AI's transformative potential, this article assesses its current applications and challenges in IR, offering insights into decision support and outcome prediction, imaging enhancements, robotics, and touchless interactions, shaping the future of patient care.Key points• AI adoption in IR is more complex compared to diagnostic radiology.• Current literature about AI in IR is in its early stages.• AI has the potential to revolutionise every aspect of IR.

Thermal and Magnetic Dual-Responsive Catheter-Assisted Shape Memory Microrobots for Multistage Vascular Embolization

Catheters navigating through complex vessels, such as sharp turns or multiple U-turns, remain challenging for vascular embolization. Here, we propose a novel multistage vascular embolization strategy for hard-to-reach vessels that releases untethered swimming shape-memory magnetic microrobots (SMMs) from the prior catheter to the vessel bifurcation. SMMs, made of organo-gel with magnetic particles, ensure biocompatibility, radiopacity, thrombosis, and fast thermal and magnetic responses. An SMM is initially a linear shape with a 0.5-mm diameter at 20 °C inserted in a catheter. It transforms into a predetermined helix within 2 s at 38 °C blood temperature after being pushed out of the catheter into the blood. SMMs enable agile swimming in confined and tortuous vessels and can swim upstream using helical propulsion with rotating magnetic fields. Moreover, we validated this multistage vascular embolization in living rabbits, completing 100-cm travel and renal artery embolization in 2 min. After 4 weeks, the SMMs maintained the embolic position, and the kidney volume decreased by 36%.

AI co-pilot bronchoscope robot

The unequal distribution of medical resources and scarcity of experienced practitioners confine access to bronchoscopy primarily to well-equipped hospitals in developed regions, contributing to the unavailability of bronchoscopic services in underdeveloped areas. Here, we present an artificial intelligence (AI) co-pilot bronchoscope robot that empowers novice doctors to conduct lung examinations as safely and adeptly as experienced colleagues. The system features a user-friendly, plug-and-play catheter, devised for robot-assisted steering, facilitating access to bronchi beyond the fifth generation in average adult patients. Drawing upon historical bronchoscopic videos and expert imitation, our AI-human shared control algorithm enables novice doctors to achieve safe steering in the lung, mitigating misoperations. Both in vitro and in vivo results underscore that our system equips novice doctors with the skills to perform lung examinations as expertly as seasoned practitioners. This study offers innovative strategies to address the pressing issue of medical resource disparities through AI assistance.

Visualization of Subcutaneous Blood Vessels Based on Hyperspectral Imaging and Three-Wavelength Index Images

Blood vessel visualization technology allows nursing staff to transition from traditional palpation or touch to locate the subcutaneous blood vessels to visualized localization by providing a clear visual aid for performing various medical procedures accurately and efficiently involving blood vessels; this can further improve the first-attempt puncture success rate for nursing staff and reduce the pain of patients. We propose a novel technique for hyperspectral visualization of blood vessels in human skin. An experiment with six participants with different skin types, race, and nationality backgrounds is described. A mere separation of spectral layers for different skin types is shown to be insufficient. The use of three-wavelength indices in imaging has shown a significant improvement in the quality of results compared to using only two-wavelength indices. This improvement can be attributed to an increase in the contrast ratio, which can be as high as 25%. We propose and implement a technique for finding new index formulae based on an exhaustive search and a binary blood-vessel image obtained through an expert assessment. As a result of the search, a novel index formula was deduced, allowing high-contrast blood vessel images to be generated for any skin type.

Expert-Guided Knowledge Distillation for Semi-Supervised Vessel Segmentation

In medical image analysis, blood vessel segmentation is of considerable clinical value for diagnosis and surgery. The predicaments of complex vascular structures obstruct the development of the field. Despite many algorithms have emerged to get off the tight corners, they rely excessively on careful annotations for tubular vessel extraction. A practical solution is to excavate the feature information distribution from unlabeled data. This work proposes a novel semi-supervised vessel segmentation framework, named EXP-Net, to navigate through finite annotations. Based on the training mechanism of the Mean Teacher model, we innovatively engage an expert network in EXP-Net to enhance knowledge distillation. The expert network comprises knowledge and connectivity enhancement modules, which are respectively in charge of modeling feature relationships from global and detailed perspectives. In particular, the knowledge enhancement module leverages the vision transformer to highlight the long-range dependencies among multi-level token components; the connectivity enhancement module maximizes the properties of topology and geometry by skeletonizing the vessel in a non-parametric manner. The key components are dedicated to the conditions of weak vessel connectivity and poor pixel contrast. Extensive evaluations show that our EXP-Net achieves state-of-the-art performance on subcutaneous vessel, retinal vessel, and coronary artery segmentations.

Autonomous Robotic Ultrasound Vascular Imaging System With Decoupled Control Strategy for External-Vision-Free Environments

OBJECTIVE

Ultrasound (US) probes scan over the surface of the human body to acquire US images in clinical vascular US diagnosis. However, due to the deformation and specificity of different human surfaces, the relationship between the scan trajectory of the skin and the internal tissues is not fully correlated, which poses a challenge for autonomous robotic US imaging in a dynamic and external-vision-free environment. Here, we propose a decoupled control strategy for autonomous robotic vascular US imaging in an environment without external vision.

METHODS

The proposed system is divided into outer-loop posture control and inner-loop orientation control, which are separately determined by a deep learning (DL) agent and a reinforcement learning (RL) agent. First, we use a weakly supervised US vessel segmentation network to estimate the probe orientation. In the outer loop control, we use a force-guided reinforcement learning agent to maintain a specific angle between the US probe and the skin in the dynamic imaging processes. Finally, the orientation and the posture are integrated to complete the imaging process.

RESULTS

Evaluation experiments on several volunteers showed that our RUS could autonomously perform vascular imaging in arms with different stiffness, curvature, and size without additional system adjustments. Furthermore, our system achieved reproducible imaging and reconstruction of dynamic targets without relying on vision-based surface information.

CONCLUSION AND SIGNIFICANCE

Our system and control strategy provides a novel framework for the application of US robots in complex and external-vision-free environments.

Autonomous medical needle steering in vivo

The use of needles to access sites within organs is fundamental to many interventional medical procedures both for diagnosis and treatment. Safely and accurately navigating a needle through living tissue to a target is currently often challenging or infeasible because of the presence of anatomical obstacles, high levels of uncertainty, and natural tissue motion. Medical robots capable of automating needle-based procedures have the potential to overcome these challenges and enable enhanced patient care and safety. However, autonomous navigation of a needle around obstacles to a predefined target in vivo has not been shown. Here, we introduce a medical robot that autonomously navigates a needle through living tissue around anatomical obstacles to a target in vivo. Our system leverages a laser-patterned highly flexible steerable needle capable of maneuvering along curvilinear trajectories. The autonomous robot accounts for anatomical obstacles, uncertainty in tissue/needle interaction, and respiratory motion using replanning, control, and safe insertion time windows. We applied the system to lung biopsy, which is critical for diagnosing lung cancer, the leading cause of cancer-related deaths in the United States. We demonstrated successful performance of our system in multiple in vivo porcine studies achieving targeting errors less than the radius of clinically relevant lung nodules. We also demonstrated that our approach offers greater accuracy compared with a standard manual bronchoscopy technique. Our results show the feasibility and advantage of deploying autonomous steerable needle robots in living tissue and how these systems can extend the current capabilities of physicians to further improve patient care.

SCANet: A Unified Semi-Supervised Learning Framework for Vessel Segmentation

Automatic subcutaneous vessel imaging with near-infrared (NIR) optical apparatus can promote the accuracy of locating blood vessels, thus significantly contributing to clinical venipuncture research. Though deep learning models have achieved remarkable success in medical image segmentation, they still struggle in the subfield of subcutaneous vessel segmentation due to the scarcity and low-quality of annotated data. To relieve it, this work presents a novel semi-supervised learning framework, SCANet, that achieves accurate vessel segmentation through an alternate training strategy. The SCANet is composed of a multi-scale recurrent neural network that embeds coarse-to-fine features and two auxiliary branches, a consistency decoder and an adversarial learning branch, responsible for strengthening fine-grained details and eliminating differences between ground-truths and predictions, respectively. Equipped with a novel semi-supervised alternate training strategy, the three components work collaboratively, enabling SCANet to accurately segment vessel regions with only a handful of labeled data and abounding unlabeled data. Moreover, to mitigate the shortage of annotated data in this field, we provide a new subcutaneous vessel dataset, VESSEL-NIR. Extensive experiments on a wide variety of tasks, including the segmentation of subcutaneous vessels, retinal vessels, and skin lesions, well demonstrate the superiority and generality of our approach.

A Novel Repetition Frequency-Based DNA Encoding Scheme to Predict Human and Mouse DNA Enhancers with Deep Learning

Recent studies have shown that DNA enhancers have an important role in the regulation of gene expression. They are responsible for different important biological elements and processes such as development, homeostasis, and embryogenesis. However, experimental prediction of these DNA enhancers is time-consuming and costly as it requires laboratory work. Therefore, researchers started to look for alternative ways and started to apply computation-based deep learning algorithms to this field. Yet, the inconsistency and unsuccessful prediction performance of computational-based approaches among various cell lines led to the investigation of these approaches as well. Therefore, in this study, a novel DNA encoding scheme was proposed, and solutions were sought to the problems mentioned and DNA enhancers were predicted with BiLSTM. The study consisted of four different stages for two scenarios. In the first stage, DNA enhancer data were obtained. In the second stage, DNA sequences were converted to numerical representations by both the proposed encoding scheme and various DNA encoding schemes including EIIP, integer number, and atomic number. In the third stage, the BiLSTM model was designed, and the data were classified. In the final stage, the performance of DNA encoding schemes was determined by accuracy, precision, recall, F1-score, CSI, MCC, G-mean, Kappa coefficient, and AUC scores. In the first scenario, it was determined whether the DNA enhancers belonged to humans or mice. As a result of the prediction process, the highest performance was achieved with the proposed DNA encoding scheme, and an accuracy of 92.16% and an AUC score of 0.85 were calculated, respectively. The closest accuracy score to the proposed scheme was obtained with the EIIP DNA encoding scheme and the result was observed as 89.14%. The AUC score of this scheme was measured as 0.87. Among the remaining DNA encoding schemes, the atomic number showed an accuracy score of 86.61%, while this rate decreased to 76.96% with the integer scheme. The AUC values of these schemes were 0.84 and 0.82, respectively. In the second scenario, it was determined whether there was a DNA enhancer and, if so, it was decided to which species this enhancer belonged. In this scenario, the highest accuracy score was obtained with the proposed DNA encoding scheme and the result was 84.59%. Moreover, the AUC score of the proposed scheme was determined as 0.92. EIIP and integer DNA encoding schemes showed accuracy scores of 77.80% and 73.68%, respectively, while their AUC scores were close to 0.90. The most ineffective prediction was performed with the atomic number and the accuracy score of this scheme was calculated as 68.27%. Finally, the AUC score of this scheme was 0.81. At the end of the study, it was observed that the proposed DNA encoding scheme was successful and effective in predicting DNA enhancers.

When neuro-robots go wrong: A review

Neuro-robots are a class of autonomous machines that, in their architecture, mimic aspects of the human brain and cognition. As such, they represent unique artifacts created by humans based on human understanding of healthy human brains. European Union's Convention on Roboethics 2025 states that the design of all robots (including neuro-robots) must include provisions for the complete traceability of the robots' actions, analogous to an aircraft's flight data recorder. At the same time, one can anticipate rising instances of neuro-robotic failure, as they operate on imperfect data in real environments, and the underlying AI behind such neuro-robots has yet to achieve explainability. This paper reviews the trajectory of the technology used in neuro-robots and accompanying failures. The failures demand an explanation. While drawing on existing explainable AI research, we argue explainability in AI limits the same in neuro-robots. In order to make robots more explainable, we suggest potential pathways for future research.

Current Applications of Robot-Assisted Ultrasound Examination

Despite advances in miniaturization and automation, the need for expert acquisition of a full echocardiogram, including Doppler, has restricted access in remote areas. Recent developments in robotics, teleoperation, and upgraded telecommunications infrastructure may provide a solution to this deficiency. Robot-assisted teleoperated ultrasound examination can aid medical diagnosis in remote locations and may improve health inequalities between rural and urban settings. This review aimed to analyze the status of teleoperated robotic systems for ultrasound examinations, evaluate clinical and preclinical applications, identify limitations, and outline future directions for clinical use. Overall, robot-assisted teleoperated ultrasound is feasible and safe in the reported clinical and preclinical studies, with the robots able to follow the hand movements performed by sonographers and researchers from a distance or in local networks. Moreover, multiple types of ultrasound examinations have been performed in remote areas, with a high success rate nearly comparable to that of conventional sonography. The studies showed that although a low-bandwidth link can be used to control a robot, the bandwidth requirements for real-time transmission of video and ultrasound images are significantly higher. Furthermore, if haptic feedback is implemented, the bandwidth requirements are increased. Haptically enabled systems that improve robotic control are necessary for accelerating the introduction to clinical use. Haptic feedback and enhanced front-end interface control for remote users are vital aspects required for clinical application. The incorporation of artificial intelligence through either aiding in window acquisition (knowledge of anatomical landmarks to adjust scanning planes) or through measurement and disease identification is yet to be researched. However, it has the potential to lead to dramatic advances. A new generation of robots is in development, and several projects in the preclinical stage reveal a promising future to overcome the shortage of health professionals in remote areas.

Human Health during Space Travel: State-of-the-Art Review

The field of human space travel is in the midst of a dramatic revolution. Upcoming missions are looking to push the boundaries of space travel, with plans to travel for longer distances and durations than ever before. Both the National Aeronautics and Space Administration (NASA) and several commercial space companies (e.g., Blue Origin, SpaceX, Virgin Galactic) have already started the process of preparing for long-distance, long-duration space exploration and currently plan to explore inner solar planets (e.g., Mars) by the 2030s. With the emergence of space tourism, space travel has materialized as a potential new, exciting frontier of business, hospitality, medicine, and technology in the coming years. However, current evidence regarding human health in space is very limited, particularly pertaining to short-term and long-term space travel. This review synthesizes developments across the continuum of space health including prior studies and unpublished data from NASA related to each individual organ system, and medical screening prior to space travel. We categorized the extraterrestrial environment into exogenous (e.g., space radiation and microgravity) and endogenous processes (e.g., alteration of humans' natural circadian rhythm and mental health due to confinement, isolation, immobilization, and lack of social interaction) and their various effects on human health. The aim of this review is to explore the potential health challenges associated with space travel and how they may be overcome in order to enable new paradigms for space health, as well as the use of emerging Artificial Intelligence based (AI) technology to propel future space health research.

Puncture site decision method for venipuncture robot based on near-infrared vision and multiobjective optimization

Venipuncture robots have superior perception and stability to humans and are expected to replace manual venipuncture. However, their use is greatly restricted because they cannot make decisions regarding the puncture sites. Thus, this study presents a multi-information fusion method for determining puncture sites for venipuncture robots to improve their autonomy in the case of limited resources. Here, numerous images have been gathered and processed to establish an image dataset of human forearms for training the U-Net with the soft attention mechanism (SAU-Net) for vein segmentation. Then, the veins are segmented from the images, feature information is extracted based on near-infrared vision, and a multiobjective optimization model for puncture site decision is provided by considering the depth, diameter, curvature, and length of the vein to determine the optimal puncture site. Experiments demonstrate that the method achieves a segmentation accuracy of 91.2% and a vein extraction rate of 86.7% while achieving the Pareto solution set (average time: 1.458 s) and optimal results for each vessel. Finally, a near-infrared camera is applied to the venipuncture robot to segment veins and determine puncture sites in real time, with the results transmitted back to the robot for an attitude adjustment. Consequently, this method can enhance the autonomy of venipuncture robots if implemented dramatically.

Is artificial intelligence capable of generating hospital discharge summaries from inpatient records?

Medical professionals have been burdened by clerical work, and artificial intelligence may efficiently support physicians by generating clinical summaries. However, whether hospital discharge summaries can be generated automatically from inpatient records stored in electronic health records remains unclear. Therefore, this study investigated the sources of information in discharge summaries. First, the discharge summaries were automatically split into fine-grained segments, such as those representing medical expressions, using a machine learning model from a previous study. Second, these segments in the discharge summaries that did not originate from inpatient records were filtered out. This was performed by calculating the n-gram overlap between inpatient records and discharge summaries. The final source origin decision was made manually. Finally, to reveal the specific sources (e.g., referral documents, prescriptions, and physician's memory) from which the segments originated, they were manually classified by consulting medical professionals. For further and deeper analysis, this study designed and annotated clinical role labels that represent the subjectivity of the expressions and builds a machine learning model to assign them automatically. The analysis results revealed the following: First, 39% of the information in the discharge summary originated from external sources other than inpatient records. Second, patient's past clinical records constituted 43%, and patient referral documents constituted 18% of the expressions derived from external sources. Third, 11% of the missing information was not derived from any documents. These are possibly derived from physicians' memories or reasoning. According to these results, end-to-end summarization using machine learning is considered infeasible. Machine summarization with an assisted post-editing process is the best fit for this problem domain.

Prediction of viral-host interactions of COVID-19 by computational methods

Experimental approaches are currently used to determine viral-host interactions, but these approaches are both time-consuming and costly. For these reasons, computational-based approaches are recommended. In this study, using computational-based approaches, viral-host interactions of SARS-CoV-2 virus and human proteins were predicted. The study consists of four different stages; in the first stage viral and host protein sequences were obtained. In the second stage, protein sequences were converted into numerical expressions by various protein mapping methods. These methods are entropy-based, AVL-tree, FIBHASH, binary encoding, CPNR, PAM250, BLOSUM62, Atchley factors, Meiler parameters, EIIP, AESNN1, Miyazawa energies, Micheletti potentials, Z-scale, and hydrophobicity. In the third stage, a deep learning model was designed and BiLSTM was used for this. In the last stage, the protein sequences were classified, and the viral-host interactions were predicted. The performances of protein mapping methods were determined by accuracy, F1-score, specificity, sensitivity, and AUC scores. According to the classification results, the best classification process was obtained by the entropy-based method. With this method, 94.74% accuracy, and 0.95 AUC score were calculated. Then, the most successful classification process was performed with the Z-scale and 91.23% accuracy, and 0.96 AUC score were obtained. Although other protein mapping methods are not as efficient as Z-scale and entropy-based methods, they have achieved successful classification. AVL-tree, FIBHASH, binary encoding, CPNR, PAM250, BLOSUM62, Atchley factors, Meiler parameters and AESNN1 methods showed over 80% accuracy, F1-score, and AUC score. Accuracy scores of EIIP, Miyazawa energies, Micheletti potentials and hydrophobicity methods remained below 80%. When the results were examined in general, it was observed that the computational approaches were successful in predicting viral-host interactions between SARS-CoV-2 virus and human proteins.

Adaptive wireless millirobotic locomotion into distal vasculature

Microcatheters have enabled diverse minimally invasive endovascular operations and notable health benefits compared with open surgeries. However, with tortuous routes far from the arterial puncture site, the distal vascular regions remain challenging for safe catheter access. Therefore, we propose a wireless stent-shaped magnetic soft robot to be deployed, actively navigated, used for medical functions, and retrieved in the example M4 segment of the middle cerebral artery. We investigate shape-adaptively controlled locomotion in phantoms emulating the physiological conditions here, where the lumen diameter shrinks from 1.5 mm to 1 mm, the radius of curvature of the tortuous lumen gets as small as 3 mm, the lumen bifurcation angle goes up to 120°, and the pulsatile flow speed reaches up to 26 cm/s. The robot can also withstand the flow when the magnetic actuation is turned off. These locomotion capabilities are confirmed in porcine arteries ex vivo. Furthermore, variants of the robot could release the tissue plasminogen activator on-demand locally for thrombolysis and function as flow diverters, initiating promising therapies towards acute ischemic stroke, aneurysm, arteriovenous malformation, dural arteriovenous fistulas, and brain tumors. These functions should facilitate the robot's usage in new distal endovascular operations.

Macrophage-compatible magnetic achiral nanorobots fabricated by electron beam lithography

With the development and progress of nanotechnology, the prospect of using nanorobots to achieve targeted drug delivery is becoming possible. Although nanorobots can potentially improve nano-drug delivery systems, there remains a significant challenge to fabricating magnetically controllable nanorobots with a size suitable for drug delivery in complex in vivo environments. Most of the current research focused on the preparation and functionalization of microscale and milliscale robots due to the relative difficulties in fabricating nanoscale robots. To address this problem and move towards in vivo applications, this study uses electron beam lithography to fabricate achiral planar L-shaped nanorobots that are biocompatible with immune cells. Their minimal planar geometry enabled nanolithography to fabricate nanorobots with a minimum feature size down to 400 nm. Using an integrated imaging and control system, the locomotive behavior of the L-shaped nanorobots in a fluidic environment was studied by examining their velocity profiles and trajectories. Furthermore, the nanorobots exhibit excellent cell compatibility with various types of cells, including macrophage cells. Finally, the long-term cell culture medium immersion test demonstrated that the L-shaped nanorobots have robust stability. This work will demonstrate the potential to use these nanorobots to operate in vivo without triggering immune cell responses.

Design and Evaluation of a Handheld Robotic Device for Peripheral Catheterization

Medical robots provide enhanced dexterity, vision, and safety for a broad range of procedures. In this article, we present a handheld, robotic device capable of performing peripheral catheter insertions with high accuracy and repeatability. The device utilizes a combination of ultrasound imaging, miniaturized robotics, and machine learning to safely and efficiently introduce a catheter sheath into a peripheral blood vessel. Here, we present the mechanical design and experimental validation of the device, known as VeniBot. Additionally, we present results on our ultrasound deep learning algorithm for vessel segmentation, and performance on tissue-mimicking phantom models that simulate difficult peripheral catheter placement. Overall, the device achieved first-attempt success rates of 97 ± 4% for vessel punctures and 89 ± 7% for sheath cannulations on the tissue mimicking models (n = 240). The results from these studies demonstrate the viability of a handheld device for performing semi-automated peripheral catheterization. In the future, the use of this device has the potential to improve clinical workflow and reduce patient discomfort by assuring a safe and efficient procedure.

A review on quantum computing and deep learning algorithms and their applications

In this paper, we describe a review concerning the Quantum Computing (QC) and Deep Learning (DL) areas and their applications in Computational Intelligence (CI). Quantum algorithms (QAs), engage the rules of quantum mechanics to solve problems using quantum information, where the quantum information is concerning the state of a quantum system, which can be manipulated using quantum information algorithms and other processing techniques. Nowadays, many QAs have been proposed, whose general conclusion is that using the effects of quantum mechanics results in a significant speedup (exponential, polynomial, super polynomial) over the traditional algorithms. This implies that some complex problems currently intractable with traditional algorithms can be solved with QA. On the other hand, DL algorithms offer what is known as machine learning techniques. DL is concerned with teaching a computer to filter inputs through layers to learn how to predict and classify information. Observations can be in the form of plain text, images, or sound. The inspiration for deep learning is the way that the human brain filters information. Therefore, in this research, we analyzed these two areas to observe the most relevant works and applications developed by the researchers in the world.

Scaphoid Fracture Detection by Using Convolutional Neural Network

Scaphoid fractures frequently appear in injury radiograph, but approximately 20% are occult. While there are few studies in the fracture detection of X-ray scaphoid images, their effectiveness is insignificant in detecting the scaphoid fractures. Traditional image processing technology had been applied to segment interesting areas of X-ray images, but it always suffered from the requirements of manual intervention and a large amount of computational time. To date, the models of convolutional neural networks have been widely applied to medical image recognition; thus, this study proposed a two-stage convolutional neural network to detect scaphoid fractures. In the first stage, the scaphoid bone is separated from the X-ray image using the Faster R-CNN network. The second stage uses the ResNet model as the backbone for feature extraction, and uses the feature pyramid network and the convolutional block attention module to develop the detection and classification models for scaphoid fractures. Various metrics such as recall, precision, sensitivity, specificity, accuracy, and the area under the receiver operating characteristic curve (AUC) are used to evaluate our proposed method’s performance. The scaphoid bone detection achieved an accuracy of 99.70%. The results of scaphoid fracture detection with the rotational bounding box revealed a recall of 0.789, precision of 0.894, accuracy of 0.853, sensitivity of 0.789, specificity of 0.90, and AUC of 0.920. The resulting scaphoid fracture classification had the following performances: recall of 0.735, precision of 0.898, accuracy of 0.829, sensitivity of 0.735, specificity of 0.920, and AUC of 0.917. According to the experimental results, we found that the proposed method can provide effective references for measuring scaphoid fractures. It has a high potential to consider the solution of detection of scaphoid fractures. In the future, the integration of images of the anterior–posterior and lateral views of each participant to develop more powerful convolutional neural networks for fracture detection by X-ray radiograph is probably important to research.

Development of Stereo NIR-II Fluorescence Imaging System for 3D Tumor Vasculature in Small Animals

Near-infrared-II (NIR-II, 1000–1700 nm) fluorescence imaging boasts high spatial resolution and deep tissue penetration due to low light scattering, reduced photon absorption, and low tissue autofluorescence. NIR-II biological imaging is applied mainly in the noninvasive visualization of blood vessels and tumors in deep tissue. In the study, a stereo NIR-II fluorescence imaging system was developed for acquiring three-dimension (3D) images on tumor vasculature in real-time, on top of the development of fluorescent semiconducting polymer dots (IR-TPE Pdots) with ultra-bright NIR-II fluorescence (1000–1400 nm) and high stability to perform long-term fluorescence imaging. The NIR-II imaging system only consists of one InGaAs camera and a moving stage to simulate left-eye view and right-eye view for the construction of 3D in-depth blood vessel images. The system was validated with blood vessel phantom of tumor-bearing mice and was applied successfully in obtaining 3D blood vessel images with 0.6 mm- and 5 mm-depth resolution and 0.15 mm spatial resolution. The NIR-II stereo vision provides precise 3D information on the tumor microenvironment and blood vessel path.

Unbox the black-box for the medical explainable AI via multi-modal and multi-centre data fusion: A mini-review, two showcases and beyond

Explainable Artificial Intelligence (XAI) is an emerging research topic of machine learning aimed at unboxing how AI systems' black-box choices are made. This research field inspects the measures and models involved in decision-making and seeks solutions to explain them explicitly. Many of the machine learning algorithms cannot manifest how and why a decision has been cast. This is particularly true of the most popular deep neural network approaches currently in use. Consequently, our confidence in AI systems can be hindered by the lack of explainability in these black-box models. The XAI becomes more and more crucial for deep learning powered applications, especially for medical and healthcare studies, although in general these deep neural networks can return an arresting dividend in performance. The insufficient explainability and transparency in most existing AI systems can be one of the major reasons that successful implementation and integration of AI tools into routine clinical practice are uncommon. In this study, we first surveyed the current progress of XAI and in particular its advances in healthcare applications. We then introduced our solutions for XAI leveraging multi-modal and multi-centre data fusion, and subsequently validated in two showcases following real clinical scenarios. Comprehensive quantitative and qualitative analyses can prove the efficacy of our proposed XAI solutions, from which we can envisage successful applications in a broader range of clinical questions.

AI-Enabled, Ultrasound-Guided Handheld Robotic Device for Femoral Vascular Access

Hemorrhage is a leading cause of trauma death, particularly in prehospital environments when evacuation is delayed. Obtaining central vascular access to a deep artery or vein is important for administration of emergency drugs and analgesics, and rapid replacement of blood volume, as well as invasive sensing and emerging life-saving interventions. However, central access is normally performed by highly experienced critical care physicians in a hospital setting. We developed a handheld AI-enabled interventional device, AI-GUIDE (Artificial Intelligence Guided Ultrasound Interventional Device), capable of directing users with no ultrasound or interventional expertise to catheterize a deep blood vessel, with an initial focus on the femoral vein. AI-GUIDE integrates with widely available commercial portable ultrasound systems and guides a user in ultrasound probe localization, venous puncture-point localization, and needle insertion. The system performs vascular puncture robotically and incorporates a preloaded guidewire to facilitate the Seldinger technique of catheter insertion. Results from tissue-mimicking phantom and porcine studies under normotensive and hypotensive conditions provide evidence of the technique's robustness, with key performance metrics in a live porcine model including: a mean time to acquire femoral vein insertion point of 53 ± 36 s (5 users with varying experience, in 20 trials), a total time to insert catheter of 80 ± 30 s (1 user, in 6 trials), and a mean number of 1.1 (normotensive, 39 trials) and 1.3 (hypotensive, 55 trials) needle insertion attempts (1 user). These performance metrics in a porcine model are consistent with those for experienced medical providers performing central vascular access on humans in a hospital.

Robotics and artificial intelligence in healthcare during COVID-19 pandemic: A systematic review

The outbreak of the COVID-19 pandemic is unarguably the biggest catastrophe of the 21st century, probably the most significant global crisis after the second world war. The rapid spreading capability of the virus has compelled the world population to maintain strict preventive measures. The outrage of the virus has rampaged through the healthcare sector tremendously. This pandemic created a huge demand for necessary healthcare equipment, medicines along with the requirement for advanced robotics and artificial intelligence-based applications. The intelligent robot systems have great potential to render service in diagnosis, risk assessment, monitoring, telehealthcare, disinfection, and several other operations during this pandemic which has helped reduce the workload of the frontline workers remarkably. The long-awaited vaccine discovery of this deadly virus has also been greatly accelerated with AI-empowered tools. In addition to that, many robotics and Robotics Process Automation platforms have substantially facilitated the distribution of the vaccine in many arrangements pertaining to it. These forefront technologies have also aided in giving comfort to the people dealing with less addressed mental health complicacies. This paper investigates the use of robotics and artificial intelligence-based technologies and their applications in healthcare to fight against the COVID-19 pandemic. A systematic search following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) method is conducted to accumulate such literature, and an extensive review on 147 selected records is performed.

Automatic and accurate needle detection in 2D ultrasound during robot-assisted needle insertion process

PURPOSE

Robot-assisted needle insertion guided by 2D ultrasound (US) can effectively improve the accuracy and success rate of clinical puncture. To this end, automatic and accurate needle-tracking methods are important for monitoring puncture processes, avoiding the needle deviating from the intended path, and reducing the risk of injury to surrounding tissues. This work aims to develop a framework for automatic and accurate detection of an inserted needle in 2D US image during the insertion process.

METHODS

We propose a novel convolutional neural network architecture comprising of a two-channel encoder and single-channel decoder for needle segmentation using needle motion information extracted from two adjacent US image frames. Based on the novel network, we further propose an automatic needle detection framework. According to the prediction result of the previous frame, a region of interest of the needle in the US image was extracted and fed into the proposed network to achieve finer and faster continuous needle localization.

RESULTS

The performance of our method was evaluated based on 1000 pairs of US images extracted from robot-assisted needle insertions on freshly excised bovine and porcine tissues. The needle segmentation network achieved 99.7% accuracy, 86.2% precision, 89.1% recall, and an F₁-score of 0.87. The needle detection framework successfully localized the needle with a mean tip error of 0.45 ± 0.33 mm and a mean orientation error of 0.42° ± 0.34° and achieved a total processing time of 50 ms per image.

CONCLUSION

The proposed framework demonstrated the capability to realize robust, accurate, and real-time needle localization during robot-assisted needle insertion processes. It has a promising application in tracking the needle and ensuring the safety of robotic-assisted automatic puncture during challenging US-guided minimally invasive procedures.

Use of Artificial Intelligence in Non-Oncologic Interventional Radiology: Current State and Future Directions

The future of radiology is disproportionately linked to the applications of artificial intelligence (AI). Recent exponential advancements in AI are already beginning to augment the clinical practice of radiology. Driven by a paucity of review articles in the area, this article aims to discuss applications of AI in non-oncologic IR across procedural planning, execution, and follow-up along with a discussion on the future directions of the field. Applications in vascular imaging, radiomics, touchless software interactions, robotics, natural language processing, post-procedural outcome prediction, device navigation, and image acquisition are included. Familiarity with AI study analysis will help open the current 'black box' of AI research and help bridge the gap between the research laboratory and clinical practice.

Telerobotic Operation of Intensive Care Unit Ventilators

Since the first reports of a novel coronavirus (SARS-CoV-2) in December 2019, over 33 million people have been infected worldwide and approximately 1 million people worldwide have died from the disease caused by this virus, COVID-19. In the United States alone, there have been approximately 7 million cases and over 200,000 deaths. This outbreak has placed an enormous strain on healthcare systems and workers. Severe cases require hospital care, and 8.5% of patients require mechanical ventilation in an intensive care unit (ICU). One major challenge is the necessity for clinical care personnel to don and doff cumbersome personal protective equipment (PPE) in order to enter an ICU unit to make simple adjustments to ventilator settings. Although future ventilators and other ICU equipment may be controllable remotely through computer networks, the enormous installed base of existing ventilators do not have this capability. This paper reports the development of a simple, low cost telerobotic system that permits adjustment of ventilator settings from outside the ICU. The system consists of a small Cartesian robot capable of operating a ventilator touch screen with camera vision control via a wirelessly connected tablet master device located outside the room. Engineering system tests demonstrated that the open-loop mechanical repeatability of the device was 7.5 mm, and that the average positioning error of the robotic finger under visual servoing control was 5.94 mm. Successful usability tests in a simulated ICU environment were carried out and are reported. In addition to enabling a significant reduction in PPE consumption, the prototype system has been shown in a preliminary evaluation to significantly reduce the total time required for a respiratory therapist to perform typical setting adjustments on a commercial ventilator, including donning and doffing PPE, from 271 to 109 s.

Coronary vessel detection methods for organ-mounted robots

BACKGROUND

HeartLander is a tethered robot walker that utilizes suction to adhere to the beating heart. HeartLander can be used for minimally invasive administration of cardiac medications or ablation of tissue. In order to administer injections safely, HeartLander must avoid coronary vasculature.

METHODS

Doppler ultrasound signals were recorded using a custom-made cardiac phantom and used to classify different coronary vessel properties. The classification was performed by two machine learning algorithms, the support vector machines and a deep convolutional neural network. These algorithms were then validated in animal trials.

RESULTS

Accuracy of identifying vessels above turbulent flow reached greater than 92% in phantom trials and greater than 98% in animal trials.

CONCLUSIONS

Through the use of two machine learning algorithms, HeartLander has shown the ability to identify different sized vasculature proximally above turbulent flow. These results indicate that it is feasible to use Doppler ultrasound to identify and avoid coronary vasculature during cardiac interventions using HeartLander.

Progress in robotics for combating infectious diseases

The world was unprepared for the COVID-19 pandemic, and recovery is likely to be a long process. Robots have long been heralded to take on dangerous, dull, and dirty jobs, often in environments that are unsuitable for humans. Could robots be used to fight future pandemics? We review the fundamental requirements for robotics for infectious disease management and outline how robotic technologies can be used in different scenarios, including disease prevention and monitoring, clinical care, laboratory automation, logistics, and maintenance of socioeconomic activities. We also address some of the open challenges for developing advanced robots that are application oriented, reliable, safe, and rapidly deployable when needed. Last, we look at the ethical use of robots and call for globally sustained efforts in order for robots to be ready for future outbreaks.

Conflicting roles for humans in learning health systems and AI-enabled healthcare

The goals of learning health systems (LHS) and of AI in medicine overlap in many respects. Both require significant improvements in data sharing and IT infrastructure, aim to provide more personalized care for patients, and strive to break down traditional barriers between research and care. However, the defining features of LHS and AI diverge when it comes to the people involved in medicine, both patients and providers. LHS aim to enhance physician-patient relationships while developments in AI emphasize a physicianless experience. LHS also encourage better coordination of specialists across the health system, but AI aims to replace many specialists with technology and algorithms. This paper argues that these points of conflict may require a reconsideration of the role of humans in medical decision making. Although it is currently unclear to what extent machines will replace humans in healthcare, the parallel development of LHS and AI raises important questions about the exact role for humans within AI-enabled healthcare.

Autonomous Robot for Removing Superficial Traumatic Blood

Objective: To remove blood from an incision and find the incision spot is a key task during surgery, or else over discharge of blood will endanger a patient's life. However, the repetitive manual blood removal involves plenty of workload contributing fatigue of surgeons. Thus, it is valuable to design a robotic system which can automatically remove blood on the incision surface. Methods: In this paper, we design a robotic system to fulfill the surgical task of the blood removal. The system consists of a pair of dual cameras, a 6-DoF robotic arm, an aspirator whose handle is fixed to a robotic arm, and a pump connected to the aspirator. Further, a path-planning algorithm is designed to generate a path, which the aspirator tip should follow to remove blood. Results: In a group of simulating bleeding experiments on ex vivo porcine tissue, the contour of the blood region is detected, and the reconstructed spatial coordinates of the detected blood contour is obtained afterward. The BRR robot cleans thoroughly the blood running out the incision. Conclusions: This study contributes the first result on designing an autonomous blood removal medical robot. The skill of the surgical blood removal operation, which is manually operated by surgeons nowadays, is alternatively grasped by the proposed BRR medical robot.

A hybrid and scalable brain-inspired robotic platform

Recent years have witnessed tremendous progress of intelligent robots brought about by mimicking human intelligence. However, current robots are still far from being able to handle multiple tasks in a dynamic environment as efficiently as humans. To cope with complexity and variability, further progress toward scalability and adaptability are essential for intelligent robots. Here, we report a brain-inspired robotic platform implemented by an unmanned bicycle that exhibits scalability of network scale, quantity and diversity to handle the changing needs of different scenarios. The platform adopts rich coding schemes and a trainable and scalable neural state machine, enabling flexible cooperation of hybrid networks. In addition, an embedded system is developed using a cross-paradigm neuromorphic chip to facilitate the implementation of diverse neural networks in spike or non-spike form. The platform achieved various real-time tasks concurrently in different real-world scenarios, providing a new pathway to enhance robots' intelligence.

Translational biomarkers in the era of precision medicine

In this chapter we discuss the past, present and future of clinical biomarker development. We explore the advent of new technologies, paving the way in which health, medicine and disease is understood. This review includes the identification of physicochemical assays, current regulations, the development and reproducibility of clinical trials, as well as, the revolution of omics technologies and state-of-the-art integration and analysis approaches.