Fully automated 3D segmentation and separation of multiple cervical vertebrae in CT images using a 2D convolutional neural network

Automated vertebral bone mineral density measurement with phantomless internal calibration in chest LDCT scans using deep learning

OBJECTIVE

To develop and evaluate a fully automated method based on deep learning and phantomless internal calibration for bone mineral density (BMD) measurement and opportunistic low BMD (osteopenia and osteoporosis) screening using chest low-dose CT (LDCT) scans.

METHODS

A total of 1175 individuals were enrolled in this study, who underwent both chest LDCT and BMD examinations with quantitative computed tomography (QCT), by two different CT scanners (Siemens and GE). Two convolutional neural network (CNN) models were employed for vertebral body segmentation and labeling, respectively. A histogram technique was applied for vertebral BMD calculation using paraspinal muscle and surrounding fat as references. 195 cases (by Siemens scanner) as fitting cohort were used to build the calibration function. 698 cases as validation cohort I (VCI, by Siemens scanner) and 282 cases as validation cohort II (VCII, by GE scanner) were performed to evaluate the performance of the proposed method, with QCT as the standard for analysis.

RESULTS

The average BMDs from the proposed method were strongly correlated with QCT (in VCI: r = 0.896, in VCII: r = 0.956, p < 0.001). Bland-Altman analysis showed a small mean difference of 1.1 mg/cm3, and large interindividual differences as seen by wide 95% limits of agreement (-29.9 to +32.0 mg/cm3) in VCI. The proposed method measured BMDs were higher than QCT measured BMDs in VCII (mean difference = 15.3 mg/cm3, p < 0.001). Osteoporosis and low BMD were diagnosed by proposed method with AUCs of 0.876 and 0.903 in VCI, 0.731 and 0.794 in VCII, respectively. The AUCs of the proposed method were increased to over 0.920 in both VCI and VCII after adjusting the cut-off.

CONCLUSION

Without manual selection of the region of interest of body tissues, the proposed method based on deep learning and phantomless internal calibration has the potential for preliminary screening of patients with low BMD using chest LDCT scans. However, the agreement between the proposed method and QCT is insufficient to allow them to be used interchangeably in BMD measurement.

ADVANCES IN KNOWLEDGE

This study proposed an automated vertebral BMD measurement method based on deep learning and phantomless internal calibration with paraspinal muscle and fat as reference.

Dimensionality Reduction Hybrid U-Net for Brain Extraction in Magnetic Resonance Imaging

In various applications, such as disease diagnosis, surgical navigation, human brain atlas analysis, and other neuroimage processing scenarios, brain extraction is typically regarded as the initial stage in MRI image processing. Whole-brain semantic segmentation algorithms, such as U-Net, have demonstrated the ability to achieve relatively satisfactory results even with a limited number of training samples. In order to enhance the precision of brain semantic segmentation, various frameworks have been developed, including 3D U-Net, slice U-Net, and auto-context U-Net. However, the processing methods employed in these models are relatively complex when applied to 3D data models. In this article, we aim to reduce the complexity of the model while maintaining appropriate performance. As an initial step to enhance segmentation accuracy, the preprocessing extraction of full-scale information from magnetic resonance images is performed with a cluster tool. Subsequently, three multi-input hybrid U-Net model frameworks are tested and compared. Finally, we propose utilizing a fusion of two-dimensional segmentation outcomes from different planes to attain improved results. The performance of the proposed framework was tested using publicly accessible benchmark datasets, namely LPBA40, in which we obtained Dice overlap coefficients of 98.05%. Improvement was achieved via our algorithm against several previous studies.

Deep Learning-Based Medical Images Segmentation of Musculoskeletal Anatomical Structures: A Survey of Bottlenecks and Strategies

By leveraging the recent development of artificial intelligence algorithms, several medical sectors have benefited from using automatic segmentation tools from bioimaging to segment anatomical structures. Segmentation of the musculoskeletal system is key for studying alterations in anatomical tissue and supporting medical interventions. The clinical use of such tools requires an understanding of the proper method for interpreting data and evaluating their performance. The current systematic review aims to present the common bottlenecks for musculoskeletal structures analysis (e.g., small sample size, data inhomogeneity) and the related strategies utilized by different authors. A search was performed using the PUBMED database with the following keywords: deep learning, musculoskeletal system, segmentation. A total of 140 articles published up until February 2022 were obtained and analyzed according to the PRISMA framework in terms of anatomical structures, bioimaging techniques, pre/post-processing operations, training/validation/testing subset creation, network architecture, loss functions, performance indicators and so on. Several common trends emerged from this survey; however, the different methods need to be compared and discussed based on each specific case study (anatomical region, medical imaging acquisition setting, study population, etc.). These findings can be used to guide clinicians (as end users) to better understand the potential benefits and limitations of these tools.

A review of advances in image-guided orthopedic surgery

Orthopedic surgery remains technically demanding due to the complex anatomical structures and cumbersome surgical procedures. The introduction of image-guided orthopedic surgery (IGOS) has significantly decreased the surgical risk and improved the operation results. This review focuses on the application of recent advances in artificial intelligence (AI), deep learning (DL), augmented reality (AR) and robotics in image-guided spine surgery, joint arthroplasty, fracture reduction and bone tumor resection. For the pre-operative stage, key technologies of AI and DL based medical image segmentation, 3D visualization and surgical planning procedures are systematically reviewed. For the intra-operative stage, the development of novel image registration, surgical tool calibration and real-time navigation are reviewed. Furthermore, the combination of the surgical navigation system with AR and robotic technology is also discussed. Finally, the current issues and prospects of the IGOS system are discussed, with the goal of establishing a reference and providing guidance for surgeons, engineers, and researchers involved in the research and development of this area.

Automatic Segmentation of Lumbar Spine MRI Images Based on Improved Attention U-Net

Lumbar spine segmentation is important to help doctors diagnose lumbar disc herniation (LDH) and patients' rehabilitation treatment. In order to accurately segment the lumbar spine, a lumbar spine image segmentation algorithm based on improved Attention U-Net is proposed. The algorithm is based on Attention U-Net, the attention module based on multilevel feature map fusion is adopted, two residual modules are introduced instead of the original convolution blocks. a hybrid loss function is used for prediction during the training process, and finally, the image superposition process is realized. In this experiment, we expanded 420 lumbar MRI images of 180 patients to 1000 images and trained them by different algorithms, respectively, and accuracy, recall, and Dice similarity coefficient metrics were used to analyze these algorithms. The results show that compared with SVM, FCN, R-CNN, U-Net, and Attention U-Net models, the improved model achieved better results in all three evaluations, with 95.50%, 94.53%, and 95.01%, respectively, which proves the better performance of the proposed method for segmentation in lumbar disc and caudal vertebrae.

Current development and prospects of deep learning in spine image analysis: a literature review

BACKGROUND AND OBJECTIVE

As the spine is pivotal in the support and protection of human bodies, much attention is given to the understanding of spinal diseases. Quick, accurate, and automatic analysis of a spine image greatly enhances the efficiency with which spine conditions can be diagnosed. Deep learning (DL) is a representative artificial intelligence technology that has made encouraging progress in the last 6 years. However, it is still difficult for clinicians and technicians to fully understand this rapidly evolving field due to the diversity of applications, network structures, and evaluation criteria. This study aimed to provide clinicians and technicians with a comprehensive understanding of the development and prospects of DL spine image analysis by reviewing published literature.

METHODS

A systematic literature search was conducted in the PubMed and Web of Science databases using the keywords "deep learning" and "spine". Date ranges used to conduct the search were from 1 January, 2015 to 20 March, 2021. A total of 79 English articles were reviewed.

KEY CONTENT AND FINDINGS

The DL technology has been applied extensively to the segmentation, detection, diagnosis, and quantitative evaluation of spine images. It uses static or dynamic image information, as well as local or non-local information. The high accuracy of analysis is comparable to that achieved manually by doctors. However, further exploration is needed in terms of data sharing, functional information, and network interpretability.

CONCLUSIONS

The DL technique is a powerful method for spine image analysis. We believe that, with the joint efforts of researchers and clinicians, intelligent, interpretable, and reliable DL spine analysis methods will be widely applied in clinical practice in the future.

AM-UNet: automated mini 3D end-to-end U-net based network for brain claustrum segmentation

Recent advances in deep learning (DL) have provided promising solutions to medical image segmentation. Among existing segmentation approaches, the U-Net-based methods have been used widely. However, very few U-Net-based studies have been conducted on automatic segmentation of the human brain claustrum (CL). The CL segmentation is challenging due to its thin, sheet-like structure, heterogeneity of its image modalities and formats, imperfect labels, and data imbalance. We propose an automatic optimized U-Net-based 3D segmentation model, called AM-UNet, designed as an end-to-end process of the pre and post-process techniques and a U-Net model for CL segmentation. It is a lightweight and scalable solution which has achieved the state-of-the-art accuracy for automatic CL segmentation on 3D magnetic resonance images (MRI). On the T1/T2 combined MRI CL dataset, AM-UNet has obtained excellent results, including Dice, Intersection over Union (IoU), and Intraclass Correlation Coefficient (ICC) scores of 82%, 70%, and 90%, respectively. We have conducted the comparative evaluation of AM-UNet with other pre-existing models for segmentation on the MRI CL dataset. As a result, medical experts confirmed the superiority of the proposed AM-UNet model for automatic CL segmentation. The source code and model of the AM-UNet project is publicly available on GitHub: https://github.com/AhmedAlbishri/AM-UNET.

Fully automated segmentation of lumbar bone marrow in sagittal, high-resolution T1-weighted magnetic resonance images using 2D U-NET

BACKGROUND

We investigated a 2-dimensional (2D) U-Net model to delineate lumbar bone marrow (BM) using a high resolution T1-weighted magnetic resonance imaging.

METHOD

Healthy controls (n = 44, 836 images) and patients with hematologic diseases (n = 56, 1064 images) received MRI of the lumbar spines. Lumbar BM on each image was manually delineated by an experienced radiologist as a ground-truth. The 2D U-Net models were trained using a healthy lumbar BM only, diseased BM only, and using healthy and diseased BM combined, respectively. The models were validated using healthy and diseased subjects, separately. A repeated-measures analysis of variance was performed to compare segmentation accuracies with 2 validation cohorts among U-Net trained with healthy subjects (UNET_HC), U-Net trained with diseased subjects (UNET_HD), U-Net trained with all subjects including both healthy and diseased subjects (UNET_HCHD), and 3-dimensional Grow-Cut algorithm (3DGC).

RESULTS

When validated with the healthy subjects, UNET_HC, UNET_HD, UNET_HCHD and 3DGC achieved the mean and standard deviation of the Dice Similarity Coefficient (DSC) of 0.9415 ± 0.07056, 0.9583 ± 0.05146, 0.9602 ± 0.0486 and 0.9139 ± 0.2039, respectively. When validated with the diseased subjects, DSCs of UNET_HC, UNET_HD, UNET_HCHD and 3DGC were 0.8303 ± 0.1073, 0.9502 ± 0.0217, 0.9502 ± 0.0217 and 0.8886 ± 0.2179, respectively. The U-Net models segmented BM better than the semi-automatic 3DGC (P < 0.0001), and UNET_HD produced better results than UNET_HC (P < 0.0001).

CONCLUSIONS

We successfully constructed a fully automatic lumbar BM segmentation model for a high-resolution T1-weighted MRI using U-Net, which outperformed most of the previously reported approaches and the existing semi-automatic algorithm.

Local Style Preservation in Improved GAN-Driven Synthetic Image Generation for Endoscopic Tool Segmentation

Accurate semantic image segmentation from medical imaging can enable intelligent vision-based assistance in robot-assisted minimally invasive surgery. The human body and surgical procedures are highly dynamic. While machine-vision presents a promising approach, sufficiently large training image sets for robust performance are either costly or unavailable. This work examines three novel generative adversarial network (GAN) methods of providing usable synthetic tool images using only surgical background images and a few real tool images. The best of these three novel approaches generates realistic tool textures while preserving local background content by incorporating both a style preservation and a content loss component into the proposed multi-level loss function. The approach is quantitatively evaluated, and results suggest that the synthetically generated training tool images enhance UNet tool segmentation performance. More specifically, with a random set of 100 cadaver and live endoscopic images from the University of Washington Sinus Dataset, the UNet trained with synthetically generated images using the presented method resulted in 35.7% and 30.6% improvement over using purely real images in mean Dice coefficient and Intersection over Union scores, respectively. This study is promising towards the use of more widely available and routine screening endoscopy to preoperatively generate synthetic training tool images for intraoperative UNet tool segmentation.

3D vertebrae labeling in spine CT: an accurate, memory-efficient (Ortho2D) framework

Purpose.Accurate localization and labeling of vertebrae in computed tomography (CT) is an important step toward more quantitative, automated diagnostic analysis and surgical planning. In this paper, we present a framework (called Ortho2D) for vertebral labeling in CT in a manner that is accurate and memory-efficient.Methods. Ortho2D uses two independent faster R-convolutional neural network networks to detect and classify vertebrae in orthogonal (sagittal and coronal) CT slices. The 2D detections are clustered in 3D to localize vertebrae centroids in the volumetric CT and classify the region (cervical, thoracic, lumbar, or sacral) and vertebral level. A post-process sorting method incorporates the confidence in network output to refine classifications and reduce outliers. Ortho2D was evaluated on a publicly available dataset containing 302 normal and pathological spine CT images with and without surgical instrumentation. Labeling accuracy and memory requirements were assessed in comparison to other recently reported methods. The memory efficiency of Ortho2D permitted extension to high-resolution CT to investigate the potential for further boosts to labeling performance.Results. Ortho2D achieved overall vertebrae detection accuracy of 97.1%, region identification accuracy of 94.3%, and individual vertebral level identification accuracy of 91.0%. The framework achieved 95.8% and 83.6% level identification accuracy in images without and with surgical instrumentation, respectively. Ortho2D met or exceeded the performance of previously reported 2D and 3D labeling methods and reduced memory consumption by a factor of ∼50 (at 1 mm voxel size) compared to a 3D U-Net, allowing extension to higher resolution datasets than normally afforded. The accuracy of level identification increased from 80.1% (for standard/low resolution CT) to 95.1% (for high-resolution CT).Conclusions. The Ortho2D method achieved vertebrae labeling performance that is comparable to other recently reported methods with significant reduction in memory consumption, permitting further performance boosts via application to high-resolution CT.

Segmentation and Identification of Vertebrae in CT Scans Using CNN, k-Means Clustering and k-NN

The accurate segmentation and identification of vertebrae presents the foundations for spine analysis including fractures, malfunctions and other visual insights. The large-scale vertebrae segmentation challenge (VerSe), organized as a competition at the Medical Image Computing and Computer Assisted Intervention (MICCAI), is aimed at vertebrae segmentation and labeling. In this paper, we propose a framework that addresses the tasks of vertebrae segmentation and identification by exploiting both deep learning and classical machine learning methodologies. The proposed solution comprises two phases: a binary fully automated segmentation of the whole spine, which exploits a 3D convolutional neural network, and a semi-automated procedure that allows locating vertebrae centroids using traditional machine learning algorithms. Unlike other approaches, the proposed method comes with the added advantage of no requirement for single vertebrae-level annotations to be trained. A dataset of 214 CT scans has been extracted from VerSe’20 challenge data, for training, validating and testing the proposed approach. In addition, to evaluate the robustness of the segmentation and labeling algorithms, 12 CT scans from subjects affected by severe, moderate and mild scoliosis have been collected from a local medical clinic. On the designated test set from Verse’20 data, the binary spine segmentation stage allowed to obtain a binary Dice coefficient of 89.17%, whilst the vertebrae identification one reached an average multi-class Dice coefficient of 90.09%. In order to ensure the reproducibility of the algorithms hereby developed, the code has been made publicly available.

Augmented Reality in Orthopedic Surgery Is Emerging from Proof of Concept Towards Clinical Studies: a Literature Review Explaining the Technology and Current State of the Art

PURPOSE OF REVIEW

Augmented reality (AR) is becoming increasingly popular in modern-day medicine. Computer-driven tools are progressively integrated into clinical and surgical procedures. The purpose of this review was to provide a comprehensive overview of the current technology and its challenges based on recent literature mainly focusing on clinical, cadaver, and innovative sawbone studies in the field of orthopedic surgery. The most relevant literature was selected according to clinical and innovational relevance and is summarized.

RECENT FINDINGS

Augmented reality applications in orthopedic surgery are increasingly reported. In this review, we summarize basic principles of AR including data preparation, visualization, and registration/tracking and present recently published clinical applications in the area of spine, osteotomies, arthroplasty, trauma, and orthopedic oncology. Higher accuracy in surgical execution, reduction of radiation exposure, and decreased surgery time are major findings presented in the literature. In light of the tremendous progress of technological developments in modern-day medicine and emerging numbers of research groups working on the implementation of AR in routine clinical procedures, we expect the AR technology soon to be implemented as standard devices in orthopedic surgery.

A Review on the Use of Artificial Intelligence in Spinal Diseases

Artificial neural networks (ANNs) have been used in a wide variety of real-world applications and it emerges as a promising field across various branches of medicine. This review aims to identify the role of ANNs in spinal diseases. Literature were searched from electronic databases of Scopus and Medline from 1993 to 2020 with English publications reported on the application of ANNs in spinal diseases. The search strategy was set as the combinations of the following keywords: “artificial neural networks,” “spine,” “back pain,” “prognosis,” “grading,” “classification,” “prediction,” “segmentation,” “biomechanics,” “deep learning,” and “imaging.” The main findings of the included studies were summarized, with an emphasis on the recent advances in spinal diseases and its application in the diagnostic and prognostic procedures. According to the search strategy, a set of 3,653 articles were retrieved from Medline and Scopus databases. After careful evaluation of the abstracts, the full texts of 89 eligible papers were further examined, of which 79 articles satisfied the inclusion criteria of this review. Our review indicates several applications of ANNs in the management of spinal diseases including (1) diagnosis and assessment of spinal disease progression in the patients with low back pain, perioperative complications, and readmission rate following spine surgery; (2) enhancement of the clinically relevant information extracted from radiographic images to predict Pfirrmann grades, Modic changes, and spinal stenosis grades on magnetic resonance images automatically; (3) prediction of outcomes in lumbar spinal stenosis, lumbar disc herniation and patient-reported outcomes in lumbar fusion surgery, and preoperative planning and intraoperative assistance; and (4) its application in the biomechanical assessment of spinal diseases. The evidence suggests that ANNs can be successfully used for optimizing the diagnosis, prognosis and outcome prediction in spinal diseases. Therefore, incorporation of ANNs into spine clinical practice may improve clinical decision making.