Fully Convolutional Regression Network for Accurate Detection of Measurement Points. In: Cardoso MJ, Arbel T, Carneiro G, Syeda-mahmood T, Tavares JMR, Moradi M, Bradley A, Greenspan H, Papa JP, Madabhushi A, Nascimento JC, Cardoso JS, Belagiannis V, Lu Z, editors. Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. Cham: Springer International Publishing; 2017. pp. 258-66. [DOI: 10.1007/978-3-319-67558-9_30]

Deep learning for automated left ventricular outflow tract diameter measurements in 2D echocardiography

BACKGROUND

Measurement of the left ventricular outflow tract diameter (LVOTd) in echocardiography is a common source of error when used to calculate the stroke volume. The aim of this study is to assess whether a deep learning (DL) model, trained on a clinical echocardiographic dataset, can perform automatic LVOTd measurements on par with expert cardiologists.

METHODS

Data consisted of 649 consecutive transthoracic echocardiographic examinations of patients with coronary artery disease admitted to a university hospital. 1304 LVOTd measurements in the parasternal long axis (PLAX) and zoomed parasternal long axis views (ZPLAX) were collected, with each patient having 1-6 measurements per examination. Data quality control was performed by an expert cardiologist, and spatial geometry data was preserved for each LVOTd measurement to convert DL predictions into metric units. A convolutional neural network based on the U-Net was used as the DL model.

RESULTS

The mean absolute LVOTd error was 1.04 (95% confidence interval [CI] 0.90-1.19) mm for DL predictions on the test set. The mean relative LVOTd errors across all data subgroups ranged from 3.8 to 5.1% for the test set. Generally, the DL model had superior performance on the ZPLAX view compared to the PLAX view. DL model precision for patients with repeated LVOTd measurements had a mean coefficient of variation of 2.2 (95% CI 1.6-2.7) %, which was comparable to the clinicians for the test set.

CONCLUSION

DL for automatic LVOTd measurements in PLAX and ZPLAX views is feasible when trained on a limited clinical dataset. While the DL predicted LVOTd measurements were within the expected range of clinical inter-observer variability, the robustness of the DL model requires validation on independent datasets. Future experiments using temporal information and anatomical constraints could improve valvular identification and reduce outliers, which are challenges that must be addressed before clinical utilization.

Point detection through multi-instance deep heatmap regression for sutures in endoscopy

Purpose:

Mitral valve repair is a complex minimally invasive surgery of the heart valve. In this context, suture detection from endoscopic images is a highly relevant task that provides quantitative information to analyse suturing patterns, assess prosthetic configurations and produce augmented reality visualisations. Facial or anatomical landmark detection tasks typically contain a fixed number of landmarks, and use regression or fixed heatmap-based approaches to localize the landmarks. However in endoscopy, there are a varying number of sutures in every image, and the sutures may occur at any location in the annulus, as they are not semantically unique.

Method:

In this work, we formulate the suture detection task as a multi-instance deep heatmap regression problem, to identify entry and exit points of sutures. We extend our previous work, and introduce the novel use of a 2D Gaussian layer followed by a differentiable 2D spatial Soft-Argmax layer to function as a local non-maximum suppression.

Results:

We present extensive experiments with multiple heatmap distribution functions and two variants of the proposed model. In the intra-operative domain, Variant 1 showed a mean \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$F_1$$\end{document}F1 of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$+ 0.0422$$\end{document}+0.0422 over the baseline. Similarly, in the simulator domain, Variant 1 showed a mean \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$F_1$$\end{document}F1 of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$+ 0.0865$$\end{document}+0.0865 over the baseline.

Conclusion:

The proposed model shows an improvement over the baseline in the intra-operative and the simulator domains. The data is made publicly available within the scope of the MICCAI AdaptOR2021 Challenge https://adaptor2021.github.io/, and the code at https://github.com/Cardio-AI/suture-detection-pytorch/.

Ultrasound-on-chip platform for medical imaging, analysis, and collective intelligence

Affordable hand-held ultrasound is transforming health care as a diagnostic tool with the potential to be as ubiquitous as the stethoscope. Here, we present a platform for advancing diagnostic care consisting of an ultrasound-on-chip probe, leveraging state-of-the-art silicon-based semiconductor foundries, paired with a mobile device and artificial-intelligence–guided image interpretation and cloud interconnectivity. Demonstrations across key organs and modes illustrate the imaging capabilities. Presentations of automated guidance for untrained ultrasound users show the potential for further broadening accessibility and utility.

Over the past half-century, ultrasound imaging has become a key technology for assessing an ever-widening range of medical conditions at all stages of life. Despite ultrasound’s proven value, expensive systems that require domain expertise in image acquisition and interpretation have limited its broad adoption. The proliferation of portable and low-cost ultrasound imaging can improve global health and also enable broad clinical and academic studies with great impact on the fields of medicine. Here, we describe the design of a complete ultrasound-on-chip, the first to be cleared by the Food and Drug Administration for 13 indications, comprising a two-dimensional array of silicon-based microelectromechanical systems (MEMS) ultrasonic sensors directly integrated into complementary metal–oxide–semiconductor-based control and processing electronics to enable an inexpensive whole-body imaging probe. The fabrication and design of the transducer array with on-chip analog and digital circuits, having an operating power consumption of 3 W or less, are described, in which approximately 9,000 seven-level feedback-based pulsers are individually addressable to each MEMS element and more than 11,000 amplifiers, more than 1,100 analog-to-digital converters, and more than 1 trillion operations per second are implemented. We quantify the measured performance and the ability to image areas of the body that traditionally takes three separate probes. Additionally, two applications of this platform are described—augmented reality assistance that guides the user in the acquisition of diagnostic-quality images of the heart and algorithms that automate the measurement of cardiac ejection fraction, an indicator of heart health.

Artificial intelligence in detection and segmentation of internal auditory canal and its nerves using deep learning techniques

PURPOSE

Artificial intelligence (AI) in medical imaging is a burgeoning topic that involves the interpretation of complex image structures. The recent advancements in deep learning techniques increase the computational powers to extract vital features without human intervention. The automatic detection and segmentation of subtle tissue such as the internal auditory canal (IAC) and its nerves is a challenging task, and it can be improved using deep learning techniques.

METHODS

The main scope of this research is to present an automatic method to detect and segment the IAC and its nerves like the facial nerve, cochlear nerve, inferior vestibular nerve, and superior vestibular nerve. To address this issue, we propose a Mask R-CNN approach driven with U-net to detect and segment the IAC and its nerves. The Mask R-CNN with its backbone network of the RESNET50 model learns a background-based localization policy to produce an actual bounding box of the IAC. Furthermore, the U-net segments the structure related information of IAC and its nerves by learning its features.

RESULTS

The proposed method was experimented on clinical datasets of 50 different patients including adults and children. The localization of IAC using Mask R-CNN was evaluated using Intersection of Union (IoU), and segmentation of IAC and its nerves was evaluated using Dice similarity coefficient.

CONCLUSIONS

The localization result shows that mean IoU of RESNET50, RESNET101 are 0.79 and 0.74, respectively. The Dice similarity coefficient of IAC and its nerves using region growing, PSO and U-net method scored 92%, 94%, and 96%, respectively. The result shows that the proposed method outperform better in localization and segmentation of IAC and its nerves. Thus, AI aids the radiologists in making the right decisions as the localization and segmentation of IAC is accurate.

VOLUMETRIC LANDMARK DETECTION WITH A MULTI-SCALE SHIFT EQUIVARIANT NEURAL NETWORK

Deep neural networks yield promising results in a wide range of computer vision applications, including landmark detection. A major challenge for accurate anatomical landmark detection in volumetric images such as clinical CT scans is that large-scale data often constrain the capacity of the employed neural network architecture due to GPU memory limitations, which in turn can limit the precision of the output. We propose a multi-scale, end-to-end deep learning method that achieves fast and memory-efficient landmark detection in 3D images. Our architecture consists of blocks of shift-equivariant networks, each of which performs landmark detection at a different spatial scale. These blocks are connected from coarse to fine-scale, with differentiable resampling layers, so that all levels can be trained together. We also present a noise injection strategy that increases the robustness of the model and allows us to quantify uncertainty at test time. We evaluate our method for carotid artery bifurcations detection on 263 CT volumes and achieve a better than state-of-the-art accuracy with mean Euclidean distance error of 2.81mm.