Semantic segmentation of the multiform proximal femur and femoral head bones with the deep convolutional neural networks in low quality MRI sections acquired in different MRI protocols

Fully automatic system to detect and segment the proximal femur in pelvic radiographic images for Legg-Calvé-Perthes disease

This study aimed to develop a method using computer vision techniques to accurately detect and delineate the proximal femur in radiographs of Legg-Calvé-Perthes disease (LCPD) patients. Currently, evaluating femoral head deformity, a crucial predictor of LCPD outcomes, relies on unreliable categorical and qualitative classifications. To address this limitation, we employed the pretrained object detection model YOLOv5 to detect the proximal femur on over 2000 radiographs, including images of shoulders and chests, to enhance robustness and generalizability. Subsequently, we utilized the U-Net convolutional neural network architecture for image segmentation of the proximal femur in more than 800 manually annotated images of stage IV LCPD. The results demonstrate outstanding performance, with the object detection model achieving high accuracy (mean average precision of 0.99) and the segmentation model attaining an accuracy score of 91%, dice coefficient of 0.75, and binary IoU score of 0.85 on the held-out test set. The proposed fully automatic proximal femur detection and segmentation system offers a promising approach to accurately detect and delineate the proximal femoral bone contour in radiographic images, which is essential for further image analysis in LCPD patients. Clinical significance: This study highlights the potential of computer vision techniques for enhancing the reliability of Legg-Calvé-Perthes disease staging and outcome prediction.

Automatic Segmentation of the Paediatric Femoral Head

Developmental dysplasia of the hip (DDH) is a developmental deformity occurring in 0.1-3.4% of infants. Timely surgical intervention can ameliorate the condition in stable hips and reduce future cases of osteoarthritis and total hip replacement. However, current definitions of DDH are subjective, and thus would benefit from a more objective and reliable assessment metric. Since the shape of the femoral head and its congruence with the acetabulum are disrupted by DDH, analysis of the femoral head could potentially play a role in the development of novel objective morphological metric for stable DDH. Therefore, this paper aimed to segment the paediatric femoral head in stable hips from radiographs, which has not been attempted before in the chosen focus age group (1-16 years) where the pelvis and hip joint undergo significant development. Two techniques were compared against a baseline U-Net: data augmentation and region-of-interest (ROI) networks. Four models were developed either without, with just one, or with both techniques. Evaluated using tenfold cross-validation, the U-Net trained with both techniques achieved the best results, with a Dice Similarity Coefficient (DSC) of 0.951±0.037 (mean ± standard deviation, calculated with 720 images). Future work will use this segmentation algorithm to accurately characterise hip joint morphology and estimate the benefit of early surgical intervention in DDH.

An AI based classifier model for lateral pillar classification of Legg-Calve-Perthes

We intended to compare the doctors with a convolutional neural network (CNN) that we had trained using our own unique method for the Lateral Pillar Classification (LPC) of Legg-Calve-Perthes Disease (LCPD). Thousands of training data sets are frequently required for artificial intelligence (AI) applications in medicine. Since we did not have enough real patient radiographs to train a CNN, we devised a novel method to obtain them. We trained the CNN model with the data we created by modifying the normal hip radiographs. No real patient radiographs were ever used during the training phase. We tested the CNN model on 81 hips with LCPD. Firstly, we detected the interobserver reliability of the whole system and then the reliability of CNN alone. Second, the consensus list was used to compare the results of 11 doctors and the CNN model. Percentage agreement and interobserver analysis revealed that CNN had good reliability (ICC = 0.868). CNN has achieved a 76.54% classification performance and outperformed 9 out of 11 doctors. The CNN, which we trained with the aforementioned method, can now provide better results than doctors. In the future, as training data evolves and improves, we anticipate that AI will perform significantly better than physicians.

A review of advances in image-guided orthopedic surgery

Orthopedic surgery remains technically demanding due to the complex anatomical structures and cumbersome surgical procedures. The introduction of image-guided orthopedic surgery (IGOS) has significantly decreased the surgical risk and improved the operation results. This review focuses on the application of recent advances in artificial intelligence (AI), deep learning (DL), augmented reality (AR) and robotics in image-guided spine surgery, joint arthroplasty, fracture reduction and bone tumor resection. For the pre-operative stage, key technologies of AI and DL based medical image segmentation, 3D visualization and surgical planning procedures are systematically reviewed. For the intra-operative stage, the development of novel image registration, surgical tool calibration and real-time navigation are reviewed. Furthermore, the combination of the surgical navigation system with AR and robotic technology is also discussed. Finally, the current issues and prospects of the IGOS system are discussed, with the goal of establishing a reference and providing guidance for surgeons, engineers, and researchers involved in the research and development of this area.

Development and external validation of automated detection, classification, and localization of ankle fractures: inside the black box of a convolutional neural network (CNN)

Purpose

Convolutional neural networks (CNNs) are increasingly being developed for automated fracture detection in orthopaedic trauma surgery. Studies to date, however, are limited to providing classification based on the entire image—and only produce heatmaps for approximate fracture localization instead of delineating exact fracture morphology. Therefore, we aimed to answer (1) what is the performance of a CNN that detects, classifies, localizes, and segments an ankle fracture, and (2) would this be externally valid?

Methods

The training set included 326 isolated fibula fractures and 423 non-fracture radiographs. The Detectron2 implementation of the Mask R-CNN was trained with labelled and annotated radiographs. The internal validation (or ‘test set’) and external validation sets consisted of 300 and 334 radiographs, respectively. Consensus agreement between three experienced fellowship-trained trauma surgeons was defined as the ground truth label. Diagnostic accuracy and area under the receiver operator characteristic curve (AUC) were used to assess classification performance. The Intersection over Union (IoU) was used to quantify accuracy of the segmentation predictions by the CNN, where a value of 0.5 is generally considered an adequate segmentation.

Results

The final CNN was able to classify fibula fractures according to four classes (Danis-Weber A, B, C and No Fracture) with AUC values ranging from 0.93 to 0.99. Diagnostic accuracy was 89% on the test set with average sensitivity of 89% and specificity of 96%. External validity was 89–90% accurate on a set of radiographs from a different hospital. Accuracies/AUCs observed were 100/0.99 for the ‘No Fracture’ class, 92/0.99 for ‘Weber B’, 88/0.93 for ‘Weber C’, and 76/0.97 for ‘Weber A’. For the fracture bounding box prediction by the CNN, a mean IoU of 0.65 (SD ± 0.16) was observed. The fracture segmentation predictions by the CNN resulted in a mean IoU of 0.47 (SD ± 0.17).

Conclusions

This study presents a look into the ‘black box’ of CNNs and represents the first automated delineation (segmentation) of fracture lines on (ankle) radiographs. The AUC values presented in this paper indicate good discriminatory capability of the CNN and substantiate further study of CNNs in detecting and classifying ankle fractures.

Level of evidence

II, Diagnostic imaging study.

Automated 3D Analysis of Clinical Magnetic Resonance Images Demonstrates Significant Reductions in Cam Morphology Following Arthroscopic Intervention in Contrast to Physiotherapy

Purpose

To obtain automated measurements of cam volume, surface area, and height from baseline (preintervention) and 12-month magnetic resonance (MR) images acquired from male and female patients allocated to physiotherapy (PT) or arthroscopic surgery (AS) management for femoroacetabular impingement (FAI) in the Australian FASHIoN trial.

Methods

An automated segmentation pipeline (CamMorph) was used to obtain cam morphology data from three-dimensional (3D) MR hip examinations in FAI patients classified with mild, moderate, or major cam volumes. Pairwise comparisons between baseline and 12-month cam volume, surface area, and height data were performed within the PT and AS patient groups using paired t-tests or Wilcoxon signed-rank tests.

Results

A total of 43 patients were included with 15 PT patients (9 males, 6 females) and 28 AS patients (18 males, 10 females) for premanagement and postmanagement cam morphology assessments. Within the PT male and female patient groups, there were no significant differences between baseline and 12-month mean cam volume (male: 1269 vs 1288 mm³, t[16] = −0.39; female: 545 vs 550 mm,³t[10] = −0.78), surface area (male: 1525 vs 1491 mm², t[16] = 0.92; female: 885 vs 925 mm,²t[10] = −0.78), maximum height (male: 4.36 vs 4.32 mm, t[16] = 0.34; female: 3.05 vs 2.96 mm, t[10] = 1.05) and average height (male: 2.18 vs 2.18 mm, t[16] = 0.22; female: 1.4 vs 1.43 mm, t[10] = −0.38). In contrast, within the AS male and female patient groups, there were significant differences between baseline and 12-month cam volume (male: 1343 vs 718 mm³, W = 0.0; female: 499 vs 240 mm³, t[18] = 2.89), surface area (male: 1520 vs 1031 mm², t(34) = 6.48; female: 782 vs 483 mm², t(18) = 3.02), maximum-height (male: 4.3 vs 3.42 mm, W = 13.5; female: 2.85 vs 2.24 mm, t(18) = 3.04) and average height (male: 2.17 vs 1.52 mm, W = 3.0; female: 1.4 vs 0.94 mm, W = 3.0). In AS patients, 3D bone models provided good visualization of cam bone mass removal postostectomy.

Conclusions

Automated measurement of cam morphology from baseline (preintervention) and 12-month MR images demonstrated that the cam volume, surface area, maximum-height, and average height were significantly smaller in AS patients following ostectomy, whereas there were no significant differences in these cam measures in PT patients from the Australian FASHIoN study.

Level of Evidence

Level II, cohort study.

The Progress of Medical Image Semantic Segmentation Methods for Application in COVID-19 Detection

Image medical semantic segmentation has been employed in various areas, including medical imaging, computer vision, and intelligent transportation. In this study, the method of semantic segmenting images is split into two sections: the method of the deep neural network and previous traditional method. The traditional method and the published dataset for segmentation are reviewed in the first step. The presented aspects, including all-convolution network, sampling methods, FCN connector with CRF methods, extended convolutional neural network methods, improvements in network structure, pyramid methods, multistage and multifeature methods, supervised methods, semiregulatory methods, and nonregulatory methods, are then thoroughly explored in current methods based on the deep neural network. Finally, a general conclusion on the use of developed advances based on deep neural network concepts in semantic segmentation is presented.

Panoptic Segmentation on Panoramic Radiographs: Deep Learning-Based Segmentation of Various Structures Including Maxillary Sinus and Mandibular Canal

Panoramic radiographs, also known as orthopantomograms, are routinely used in most dental clinics. However, it has been difficult to develop an automated method that detects the various structures present in these radiographs. One of the main reasons for this is that structures of various sizes and shapes are collectively shown in the image. In order to solve this problem, the recently proposed concept of panoptic segmentation, which integrates instance segmentation and semantic segmentation, was applied to panoramic radiographs. A state-of-the-art deep neural network model designed for panoptic segmentation was trained to segment the maxillary sinus, maxilla, mandible, mandibular canal, normal teeth, treated teeth, and dental implants on panoramic radiographs. Unlike conventional semantic segmentation, each object in the tooth and implant classes was individually classified. For evaluation, the panoptic quality, segmentation quality, recognition quality, intersection over union (IoU), and instance-level IoU were calculated. The evaluation and visualization results showed that the deep learning-based artificial intelligence model can perform panoptic segmentation of images, including those of the maxillary sinus and mandibular canal, on panoramic radiographs. This automatic machine learning method might assist dental practitioners to set up treatment plans and diagnose oral and maxillofacial diseases.

Deep learning in electron microscopy

Deep learning is transforming most areas of science and technology, including electron microscopy. This review paper offers a practical perspective aimed at developers with limited familiarity. For context, we review popular applications of deep learning in electron microscopy. Following, we discuss hardware and software needed to get started with deep learning and interface with electron microscopes. We then review neural network components, popular architectures, and their optimization. Finally, we discuss future directions of deep learning in electron microscopy.

A novel approach for computerized quantitative image analysis of proximal femur bone shape deformities based on the hip joint symmetry

As a result of most of the bone disorders seen in hip joints, shape deformities occur in the structural form of the hip joint components. Image-based quantitative analysis and assessment of these deformities in bone shapes are very important for the evaluation, treatment, and prognosis of the various hip joint bone disorders. In this article, a novel approach for the image-based computerized quantitative analysis of proximal femur shape deformities is presented. In the proposed approach, shape deformities of the pathological proximal femurs were quantified over the contralateral healthy proximal femur shape structure of the same patient in 2D by taking the hip joint symmetry property of human anatomy into consideration. It is based on the idea that if the right and left proximal femurs in bilateral hip joints are highly symmetrical and also if one of the proximal femurs is healthy and the contralateral one is pathological, the non-overlapping bone shape regions can represent the deformities in pathological proximal femurs when both proximal femurs are registered to overlap each other. In the methodological process of the proposed study, a set of image preprocessing operations was primarily performed on the raw magnetic resonance imaging (MRI) data. Then, the segmented proximal femurs in bilateral hip joint images were automatically aligned with the Iterative Closest Point (ICP) rigid registration method. Following the registration, a set of image postprocessing operations was performed on the images of proximal femurs aligned. In the quantification phase, the bone shape deformities in pathological proximal femurs were quantified simply in terms of the mismatching area in 2D by measuring a shape variation index representing the total bone shape deformity ratio. To evaluate the proposed quantitative shape analysis approach, bilateral hip joints in a total of 13 coronal MRI sections of 13 patients with Legg-Calve-Perthes disease (LCPD) were used. Experimental studies have shown that the proposed approach has quite promising results in the quantitative representation of the pathological proximal femur shape deformities. Furthermore, consistent results have been observed for the Waldenström classification stages of the disease. The shape deformity ratios in pathological proximal femurs were quantified as 9.44% (±1.40), 18.38% (±6.30), 24.73% (±12.42), and 27.66% (±10.41), respectively for the Initial, Fragmentation, Reossification, and Remodelling stages of LCPD with the quantification error rates of 0.29% (±0.16), 0.58% (±0.71), 1.12% (±0.82), and 0.80% (±0.98). Additionally, a mean error rate of 0.65% (±0.68) was observed for the quantified shape deformity ratios of all samples.