A Comprehensive Survey on Bone Segmentation Techniques in Knee Osteoarthritis Research: From Conventional Methods to Deep Learning

Survival analysis on subchondral bone length for total knee replacement

OBJECTIVE

Use subchondral bone length (SBL), a new MRI-derived measure that reflects the extent of cartilage loss and bone flattening, to predict the risk of progression to total knee replacement (TKR).

METHODS

We employed baseline MRI data from the Osteoarthritis Initiative (OAI), focusing on 760 men and 1214 women with bone marrow lesions (BMLs) and joint space narrowing (JSN) scores, to predict the progression to TKR. To minimize bias from analyzing both knees of a participant, only the knee with a higher Kellgren-Lawrence (KL) grade was considered, given its greater potential need for TKR. We utilized the Kaplan-Meier survival curves and Cox proportional hazards models, incorporating raw and normalized values of SBL, JSN, and BML as predictors. The study included subgroup analyses for different demographics and clinical characteristics, using models for raw and normalized SBL (merged, femoral, tibial), BML (merged, femoral, tibial), and JSN (medial and lateral compartments). Model performance was evaluated using the time-dependent area under the curve (AUC), Brier score, and Concordance index to gauge accuracy, calibration, and discriminatory power. Knee joint and region-level analyses were conducted to determine the effectiveness of SBL, JSN, and BML in predicting TKR risk.

RESULTS

The SBL model, incorporating data from both the femur and tibia, demonstrated a predictive capacity for TKR that closely matched the performance of the BML score and the JSN grade. The Concordance index of the SBL model was 0.764, closely mirroring the BML's 0.759 and slightly below JSN's 0.788. The Brier score for the SBL model stood at 0.069, showing comparability with BML's 0.073 and a minor difference from JSN's 0.067. Regarding the AUC, the SBL model achieved 0.803, nearly identical to BML's 0.802 and slightly lower than JSN's 0.827.

CONCLUSION

SBL's capacity to predict the risk of progression to TKR highlights its potential as an effective imaging biomarker for knee osteoarthritis.

SSDL-an automated semi-supervised deep learning approach for patient-specific 3D reconstruction of proximal femur from QCT images

Deep Learning (DL) techniques have recently been used in medical image segmentation and the reconstruction of 3D anatomies of a human body. In this work, we propose a semi-supervised DL (SSDL) approach utilizing a CNN-based 3D U-Net model for femur segmentation from sparsely annotated quantitative computed tomography (QCT) slices. Specifically, QCT slices at the proximal end of the femur forming ball and socket joint with acetabulum were annotated for precise segmentation, where a segmenting binary mask was generated using a 3D U-Net model to segment the femur accurately. A total of 5474 QCT slices were considered for training among which 2316 slices were annotated. 3D femurs were further reconstructed from segmented slices employing polynomial spline interpolation. Both qualitative and quantitative performance of segmentation and 3D reconstruction were satisfactory with more than 90% accuracy achieved for all of the standard performance metrics considered. The spatial overlap index and reproducibility validation metric for segmentation-Dice Similarity Coefficient was 91.8% for unseen patients and 99.2% for validated patients. An average relative error of 12.02% and 10.75% for volume and surface area, respectively, were computed for 3D reconstructed femurs. The proposed approach demonstrates its effectiveness in accurately segmenting and reconstructing 3D femur from QCT slices.

Construction of multi-scale feature fusion segmentation model of MRI knee images based on dual attention mechanism weighted aggregation

BACKGROUND

Early diagnosis of knee osteoarthritis is an important area of research in the field of clinical medicine. Due to the complexity in the MRI imaging sequences and the diverse structure of cartilage, there are many challenges in the segmentation of knee bone and cartilage. Relevant studies have conducted semantic fusion processing through splicing or summing forms, which results in reduced resolution and the accumulation of redundant information.

OBJECTIVE

This study was envisaged to construct an MRI image segmentation model to improve the diagnostic efficiency and accuracy of different grade knee osteoarthritis by adopting the Dual Attention and Multi-scale Feature Fusion Segmentation network (DA-MFFSnet).

METHODS

The feature information of different scales was fused through the Multi-scale Attention Downsample module to extract more accurate feature information, and the Global Attention Upsample module weighted lower-level feature information to reduce the loss of key information.

RESULTS

The collected MRI knee images were screened and labeled, and the study results showed that the segmentation effect of DA-MFFSNet model was closer to that of the manually labeled images. The mean intersection over union, the dice similarity coefficient and the volumetric overlap error was 92.74%, 91.08% and 7.44%, respectively, and the accuracy of the differential diagnosis of knee osteoarthritis was 84.42%.

CONCLUSIONS

The model exhibited better stability and classification effect. Our results indicated that the Dual Attention and Multi-scale Feature Fusion Segmentation model can improve the segmentation effect of MRI knee images in mild and medium knee osteoarthritis, thereby offering an important clinical value and improving the accuracy of the clinical diagnosis.

Automatic segmentation of inconstant fractured fragments for tibia/fibula from CT images using deep learning

Orthopaedic surgeons need to correctly identify bone fragments using 2D/3D CT images before trauma surgery. Advances in deep learning technology provide good insights into trauma surgery over manual diagnosis. This study demonstrates the application of the DeepLab v3+ -based deep learning model for the automatic segmentation of fragments of the fractured tibia and fibula from CT images and the results of the evaluation of the performance of the automatic segmentation. The deep learning model, which was trained using over 11 million images, showed good performance with a global accuracy of 98.92%, a weighted intersection over the union of 0.9841, and a mean boundary F1 score of 0.8921. Moreover, deep learning performed 5-8 times faster than the experts' recognition performed manually, which is comparatively inefficient, with almost the same significance. This study will play an important role in preoperative surgical planning for trauma surgery with convenience and speed.

Artificial intelligence in knee osteoarthritis: A comprehensive review for 2022

Objective

The aim of this literature review is to yield a comprehensive and exhaustive overview of the existing evidence and up-to-date applications of artificial intelligence for knee osteoarthritis.

Methods

A literature review was performed by using PubMed, Google Scholar, and IEEE databases for articles published in peer-reviewed journals in 2022. The articles focusing on the use of artificial intelligence in diagnosis and prognosis of knee osteoarthritis and accelerating the image acquisition were selected. For each selected study, the code availability, considered number of patients and knees, imaging type, covariates, grading type of osteoarthritis, models, validation approaches, objectives, and results were reviewed.

Results

395 articles were screened, and 35 of them were reviewed. Eight articles were based on diagnosis, six on prognosis prediction, three on classification, three on accelerated image acquisition, and 15 on segmentation of knee osteoarthritis. 57% of the articles used MRI, 26% radiography, 6% MRI together with radiography, 6% ultrasonography, and 6% only clinical data. 23% of the articles made the computer codes available for their study, and 26% used clinical data. External validation and nested cross-validation were used in 17% and 14% of articles, respectively.

Conclusions

The use of artificial intelligence provided a promising potential to enhance the detection and management of knee osteoarthritis. Translating the developed models into clinics is still in the early stages of development. The translation of artificial intelligence models is expected to be further examined in prospective studies to support clinicians in improving routine healthcare practice.

Hybrid Techniques of X-ray Analysis to Predict Knee Osteoarthritis Grades Based on Fusion Features of CNN and Handcrafted

Knee osteoarthritis (KOA) is a chronic disease that impedes movement, especially in the elderly, affecting more than 5% of people worldwide. KOA goes through many stages, from the mild grade that can be treated to the severe grade in which the knee must be replaced. Therefore, early diagnosis of KOA is essential to avoid its development to the advanced stages. X-rays are one of the vital techniques for the early detection of knee infections, which requires highly experienced doctors and radiologists to distinguish Kellgren-Lawrence (KL) grading. Thus, artificial intelligence techniques solve the shortcomings of manual diagnosis. This study developed three methodologies for the X-ray analysis of both the Osteoporosis Initiative (OAI) and Rani Channamma University (RCU) datasets for diagnosing KOA and discrimination between KL grades. In all methodologies, the Principal Component Analysis (PCA) algorithm was applied after the CNN models to delete the unimportant and redundant features and keep the essential features. The first methodology for analyzing x-rays and diagnosing the degree of knee inflammation uses the VGG-19 -FFNN and ResNet-101 -FFNN systems. The second methodology of X-ray analysis and diagnosis of KOA grade by Feed Forward Neural Network (FFNN) is based on the combined features of VGG-19 and ResNet-101 before and after PCA. The third methodology for X-ray analysis and diagnosis of KOA grade by FFNN is based on the fusion features of VGG-19 and handcrafted features, and fusion features of ResNet-101 and handcrafted features. For an OAI dataset with fusion features of VGG-19 and handcrafted features, FFNN obtained an AUC of 99.25%, an accuracy of 99.1%, a sensitivity of 98.81%, a specificity of 100%, and a precision of 98.24%. For the RCU dataset with the fusion features of VGG-19 and the handcrafted features, FFNN obtained an AUC of 99.07%, an accuracy of 98.20%, a sensitivity of 98.16%, a specificity of 99.73%, and a precision of 98.08%.

Auto-segmentation of the tibia and femur from knee MR images via deep learning and its application to cartilage strain and recovery

The ability to efficiently and reproducibly generate subject-specific 3D models of bone and soft tissue is important to many areas of musculoskeletal research. However, methodologies requiring such models have largely been limited by lengthy manual segmentation times. Recently, machine learning, and more specifically, convolutional neural networks, have shown potential to alleviate this bottleneck in research throughput. Thus, the purpose of this work was to develop a modified version of the convolutional neural network architecture U-Net to automate segmentation of the tibia and femur from double echo steady state knee magnetic resonance (MR) images. Our model was trained on a dataset of over 4,000 MR images from 34 subjects, segmented by three experienced researchers, and reviewed by a musculoskeletal radiologist. For our validation and testing sets, we achieved dice coefficients of 0.985 and 0.984, respectively. As further testing, we applied our trained model to a prior study of tibial cartilage strain and recovery. In this analysis, across all subjects, there were no statistically significant differences in cartilage strain between the machine learning and ground truth bone models, with a mean difference of 0.2 ± 0.7 % (mean ± 95 % confidence interval). This difference is within the measurement resolution of previous cartilage strain studies from our lab using manual segmentation. In summary, we successfully trained, validated, and tested a machine learning model capable of segmenting MR images of the knee, achieving results that are comparable to trained human segmenters.

Open-source pipeline for automatic segmentation and microstructural analysis of murine knee subchondral bone

μCT images are commonly analysed to assess changes in bone density and microstructure in preclinical murine models. Several platforms provide automated analysis of bone microstructural parameters from volumetric regions of interest (ROI). However, segmentation of the regions of subchondral bone to create the volumetric ROIs remains a manual and time-consuming task. This study aimed to develop an automated end-to-end pipeline, combining segmentation and microstructural analysis, to evaluate subchondral bone in the mouse proximal knee.

METHODS

A segmented dataset of μCT scans from 62 knees (healthy and arthritic) from 10-week male C57BL/6 mice was used to train a U-Net type architecture to automate segmentation of the subchondral trabecular bone. These segmentations were used in tandem with the original scans as input for microstructural analysis along with thresholded trabecular bone. Manually and U-Net segmented ROIs were fed into two available pipelines for microstructural analysis: the ITKBoneMorphometry library and CTan (SKYSCAN). Outcome parameters were compared between pipelines, including: bone volume (BV), total volume (TV), BV/TV, trabecular number (TbN), trabecular thickness (TbTh), trabecular separation (TbSp), and bone surface density (BSBV).

RESULTS

There was good agreement for all bone measures comparing the manual and U-Net pipelines utilizing ITK (R = 0.88-0.98) and CTAn (R = 0.91-0.98). ITK and CTAn showed good agreement for BV, TV, BV/TV, TbTh and BSBV (R = 0.9-0.98). However, limited agreement was seen between TbN (R = 0.73) and TbSb (R = 0.59) due to methodological differences in how spacing is evaluated. Microstructural parameters generated from manual and automatic segmentations showed high correlation across all measures. Using the CTAn pipeline yielded strong R² values (0.83-0.96) and very strong agreement based on ICC (0.90-0.98). The ITK pipeline yielded similarly high R² values (0.91-0.96, except for TbN (0.77)), and ICC values (0.88-0.98). The automated segmentations yield lower average values for BV, TV and BV/TV (ranging from 14 % to 6.3 %), but differences were not found to be influenced by the mean ROI values.

CONCLUSIONS

This integrated pipeline seamlessly automated both segmentation and quantification of the proximal tibia subchondral bone microstructure. This automated pipeline allows the analysis of large volumes of data, and its open-source nature may enable the standardization of microstructural analysis of trabecular bone across different research groups.

Pairwise attention-enhanced adversarial model for automatic bone segmentation in CT images

Objective. Bone segmentation is a critical step in screw placement navigation. Although the deep learning methods have promoted the rapid development for bone segmentation, the local bone separation is still challenging due to irregular shapes and similar representational features.Approach. In this paper, we proposed the pairwise attention-enhanced adversarial model (Pair-SegAM) for automatic bone segmentation in computed tomography images, which includes the two parts of the segmentation model and discriminator. Considering that the distributions of the predictions from the segmentation model contains complicated semantics, we improve the discriminator to strengthen the awareness ability of the target region, improving the parsing of semantic information features. The Pair-SegAM has a pairwise structure, which uses two calculation mechanics to set up pairwise attention maps, then we utilize the semantic fusion to filter unstable regions. Therefore, the improved discriminator provides more refinement information to capture the bone outline, thus effectively enhancing the segmentation models for bone segmentation.Main results. To test the Pair-SegAM, we selected the two bone datasets for assessment. We evaluated our method against several bone segmentation models and latest adversarial models on the both datasets. The experimental results prove that our method not only exhibits superior bone segmentation performance, but also states effective generalization.Significance. Our method provides a more efficient segmentation of specific bones and has the potential to be extended to other semantic segmentation domains.

Performance and Robustness of Regional Image Segmentation Driven by Selected Evolutionary and Genetic Algorithms: Study on MR Articular Cartilage Images

The analysis and segmentation of articular cartilage magnetic resonance (MR) images belongs to one of the most commonly routine tasks in diagnostics of the musculoskeletal system of the knee area. Conventional regional segmentation methods, which are based either on the histogram partitioning (e.g., Otsu method) or clustering methods (e.g., K-means), have been frequently used for the task of regional segmentation. Such methods are well known as fast and well working in the environment, where cartilage image features are reliably recognizable. The well-known fact is that the performance of these methods is prone to the image noise and artefacts. In this context, regional segmentation strategies, driven by either genetic algorithms or selected evolutionary computing strategies, have the potential to overcome these traditional methods such as Otsu thresholding or K-means in the context of their performance. These optimization strategies consecutively generate a pyramid of a possible set of histogram thresholds, of which the quality is evaluated by using the fitness function based on Kapur's entropy maximization to find the most optimal combination of thresholds for articular cartilage segmentation. On the other hand, such optimization strategies are often computationally demanding, which is a limitation of using such methods for a stack of MR images. In this study, we publish a comprehensive analysis of the optimization methods based on fuzzy soft segmentation, driven by artificial bee colony (ABC), particle swarm optimization (PSO), Darwinian particle swarm optimization (DPSO), and a genetic algorithm for an optimal thresholding selection against the routine segmentations Otsu and K-means for analysis and the features extraction of articular cartilage from MR images. This study objectively analyzes the performance of the segmentation strategies upon variable noise with dynamic intensities to report a segmentation's robustness in various image conditions for a various number of segmentation classes (4, 7, and 10), cartilage features (area, perimeter, and skeleton) extraction preciseness against the routine segmentation strategies, and lastly the computing time, which represents an important factor of segmentation performance. We use the same settings on individual optimization strategies: 100 iterations and 50 population. This study suggests that the combination of fuzzy thresholding with an ABC algorithm gives the best performance in the comparison with other methods as from the view of the segmentation influence of additive dynamic noise influence, also for cartilage features extraction. On the other hand, using genetic algorithms for cartilage segmentation in some cases does not give a good performance. In most cases, the analyzed optimization strategies significantly overcome the routine segmentation methods except for the computing time, which is normally lower for the routine algorithms. We also publish statistical tests of significance, showing differences in the performance of individual optimization strategies against Otsu and K-means method. Lastly, as a part of this study, we publish a software environment, integrating all the methods from this study.

Recognition of Knee Osteoarthritis (KOA) Using YOLOv2 and Classification Based on Convolutional Neural Network

Knee osteoarthritis (KOA) is one of the deadliest forms of arthritis. If not treated at an early stage, it may lead to knee replacement. That is why early diagnosis of KOA is necessary for better treatment. Manually KOA detection is a time-consuming and error-prone task. Computerized methods play a vital role in accurate and speedy detection. Therefore, the classification and localization of the KOA method are proposed in this work using radiographic images. The two-dimensional radiograph images are converted into three-dimensional and LBP features are extracted having the dimension of N × 59 out of which the best features of N × 55 are selected using PCA. The deep features are also extracted using Alex-Net and Dark-net-53 with the dimensions of N × 1024 and N × 4096, respectively, where N represents the number of images. Then, N × 1000 features are selected individually from both models using PCA. Finally, the extracted features are fused serially with the dimension of N × 2055 and passed to the classifiers on a 10-fold cross-validation that provides an accuracy of 90.6% for the classification of KOA grades. The localization model is proposed with the combination of an open exchange neural network (ONNX) and YOLOv2 that is trained on the selected hyper-parameters. The proposed model provides 0.98 mAP for the localization of classified images. The experimental analysis proves that the presented framework provides better results as compared to existing works.

Federated Learning-Based Detection of Invasive Carcinoma of No Special Type with Histopathological Images

Invasive carcinoma of no special type (IC-NST) is known to be one of the most prevalent kinds of breast cancer, hence the growing research interest in studying automated systems that can detect the presence of breast tumors and appropriately classify them into subtypes. Machine learning (ML) and, more specifically, deep learning (DL) techniques have been used to approach this problem. However, such techniques usually require massive amounts of data to obtain competitive results. This requirement makes their application in specific areas such as health problematic as privacy concerns regarding the release of patients’ data publicly result in a limited number of publicly available datasets for the research community. This paper proposes an approach that leverages federated learning (FL) to securely train mathematical models over multiple clients with local IC-NST images partitioned from the breast histopathology image (BHI) dataset to obtain a global model. First, we used residual neural networks for automatic feature extraction. Then, we proposed a second network consisting of Gabor kernels to extract another set of features from the IC-NST dataset. After that, we performed a late fusion of the two sets of features and passed the output through a custom classifier. Experiments were conducted for the federated learning (FL) and centralized learning (CL) scenarios, and the results were compared. Competitive results were obtained, indicating the positive prospects of adopting FL for IC-NST detection. Additionally, fusing the Gabor features with the residual neural network features resulted in the best performance in terms of accuracy, F1 score, and area under the receiver operation curve (AUC-ROC). The models show good generalization by performing well on another domain dataset, the breast cancer histopathological (BreakHis) image dataset. Our method also outperformed other methods from the literature.

Diagnostics of Articular Cartilage Damage Based on Generated Acoustic Signals Using ANN—Part II: Patellofemoral Joint

Cartilage loss due to osteoarthritis (OA) in the patellofemoral joint provokes pain, stiffness, and restriction of joint motion, which strongly reduces quality of life. Early diagnosis is essential for prolonging painless joint function. Vibroarthrography (VAG) has been proposed in the literature as a safe, noninvasive, and reproducible tool for cartilage evaluation. Until now, however, there have been no strict protocols for VAG acquisition especially in regard to differences between the patellofemoral and tibiofemoral joints. The purpose of this study was to evaluate the proposed examination and acquisition protocol for the patellofemoral joint, as well as to determine the optimal examination protocol to obtain the best diagnostic results. Thirty-four patients scheduled for knee surgery due to cartilage lesions were enrolled in the study and compared with 33 healthy individuals in the control group. VAG acquisition was performed prior to surgery, and cartilage status was evaluated during the surgery as a reference point. Both closed (CKC) and open (OKC) kinetic chains were assessed during VAG. The selection of the optimal signal measures was performed using a neighborhood component analysis (NCA) algorithm. The classification was performed using multilayer perceptron (MLP) and radial basis function (RBF) neural networks. The classification using artificial neural networks was performed for three variants: I. open kinetic chain, II. closed kinetic chain, and III. open and closed kinetic chain. The highest diagnostic accuracy was obtained for variants I and II for the RBF 9-35-2 and MLP 10-16-2 networks, respectively, achieving a classification accuracy of 98.53, a sensitivity of 0.958, and a specificity of 1. For variant III, a diagnostic accuracy of 97.79 was obtained with a sensitivity and specificity of 0.978 for MLP 8-3-2. This indicates a possible simplification of the examination protocol to single kinetic chain analyses.