Discriminative Regularized Auto-Encoder for Early Detection of Knee OsteoArthritis: Data from the Osteoarthritis Initiative

Knee Osteoarthritis SCAENet: Adaptive Knee Osteoarthritis Severity Assessment Using Spatial Separable Convolution with Attention-Based Ensemble Networks with Hybrid Optimization Strategy

Osteoarthritis (OA) of the knee is a chronic state that significantly lowers the quality of life for its patients. Early detection and lifetime monitoring of the progression of OA are necessary for preventive therapy. In the course of therapy, the Kellgren and Lawrence (KL) assessment model categorizes the rigidity of OA. Deep techniques have recently been used to increase the precision and effectiveness of OA severity assessments. The training process is compromised by low-confidence samples, which are less accurate than normal ones. In this work, a deep learning-based knee osteoarthritis severity assessment model is recommended to accurately identify the condition in patients. The phases of the designed model are data collection, feature extraction, and prediction. At first, the images are generally gathered from online resources. The gathered images are given into the feature extraction phase. A new model is implemented to predict knee osteoarthritis named Spatial Separable Convolution with Attention-based Ensemble Networks (SCAENet), which includes feature extraction, stacked target-based feature pool generation, and knee osteoarthritis prediction. The feature extraction is done using ResNet, Visual Geometry Group (VGG16), and DenseNet. The stacked target-based feature pool is obtained from the SCAENet. Hence, the stacked target-based feature pool is obtained by the Hybridization of Equilibrium Slime Mould with Bald Eagle Search Optimization (HESM-BESO). Here, the knee osteoarthritis's severity prediction is performed using the dimensional convolutional neural network (1DCNN) technique. The designed SCAENet model is validated with other conventional methods to show high performance.

A semi-supervised multiview-MRI network for the detection of Knee Osteoarthritis

Knee OsteoArthritis (OA) is a prevalent chronic condition, affecting a significant proportion of the global population. Detecting knee OA is crucial as the degeneration of the knee joint is irreversible. In this paper, we introduce a semi-supervised multi-view framework and a 3D CNN model for detecting knee OA using 3D Magnetic Resonance Imaging (MRI) scans. We introduce a semi-supervised learning approach combining labeled and unlabeled data to improve the performance and generalizability of the proposed model. Experimental results show the efficacy of our proposed approach in detecting knee OA from 3D MRI scans using a large cohort of 4297 subjects. An ablation study was conducted to investigate the contributions of various components of the proposed model, providing insights into the optimal design of the model. Our results indicate the potential of the proposed approach to improve the accuracy and efficiency of OA diagnosis. The proposed framework reported an AUC of 93.20% for the detection of knee OA.

DC-AAE: Dual channel adversarial autoencoder with multitask learning for KL-grade classification in knee radiographs

Knee osteoarthritis (OA) is a frequent musculoskeletal disorder that leads to physical disability in older adults. Manual OA assessment is performed via visual inspection, which is highly subjective as it suffers from moderate to high inter-observer variability. Many deep learning-based techniques have been proposed to address this issue. However, owing to the limited amount of labelled data, all existing solutions have limitations in terms of performance or the number of classes. This paper proposes a novel fully automatic Kellgren and Lawrence (KL) grade classification scheme in knee radiographs. We developed a semi-supervised multi-task learning-based approach that enables the exploitation of additional unlabelled data in an unsupervised as well as supervised manner. Specifically, we propose a dual-channel adversarial autoencoder, which is first trained in an unsupervised manner for reconstruction tasks only. To exploit the additional data in a supervised way, we propose a multi-task learning framework by introducing an auxiliary task. In particular, we use leg side identification as an auxiliary task, which allows the use of more datasets, e.g., CHECK dataset. The work demonstrates that the utilization of additional data can improve the primary task of KL-grade classification for which only limited labelled data is available. This semi-supervised learning essentially helps to improve the feature learning ability of our framework, which leads to improved performance for KL-grade classification. We rigorously evaluated our proposed model on the two largest publicly available datasets for various aspects, i.e., overall performance, the effect of additional unlabelled samples and auxiliary tasks, robustness analysis, and ablation study. The proposed model achieved the accuracy, precision, recall, and F1 score of 75.53%, 74.1%, 78.51%, and 75.34%, respectively. Furthermore, the experimental results show that the suggested model not only achieves state-of-the-art performance on two publicly available datasets but also exhibits remarkable robustness.

A discriminative shape-texture convolutional neural network for early diagnosis of knee osteoarthritis from X-ray images

Knee Osteoarthritis (OA) is one of the most common causes of physical disability worldwide associated with a significant personal and socioeconomic burden. Deep Learning approaches based on Convolutional Neural Networks (CNNs) achieved remarkable improvements in knee OA detection. Despite this success, the problem of early knee OA diagnosis from plain radiographs remains a challenging task. This is due to the high similarity between the X-ray images of OA and non-OA subjects and the disappearance of texture information regarding bone microarchitecture changes in the top layers during the learning process of the CNN models. To address these issues, we propose a Discriminative Shape-Texture Convolutional Neural Network (DST-CNN), which automatically diagnoses early knee OA from X-ray images. The proposed model incorporates a discriminative loss to improve class separability and deal with high inter-class similarities. In addition, a new Gram Matrix Descriptor (GMD) block is embedded in the CNN architecture to compute texture features from several intermediate layers and combine them with the shape features in the top layers. We show that merging texture features with deep ones leads to better prediction of the early stages of OA. Comprehensive experimental results on two large public databases, Osteoarthritis Initiative (OAI) and Multicenter Osteoarthritis Study (MOST) demonstrate the potential of the proposed network. Ablation studies and visualizations are provided for a detailed understanding of our proposed approach.

Knee Osteoarthritis Detection and Severity Classification Using Residual Neural Networks on Preprocessed X-ray Images

One of the most common and challenging medical conditions to deal with in old-aged people is the occurrence of knee osteoarthritis (KOA). Manual diagnosis of this disease involves observing X-ray images of the knee area and classifying it under five grades using the Kellgren-Lawrence (KL) system. This requires the physician's expertise, suitable experience, and a lot of time, and even after that the diagnosis can be prone to errors. Therefore, researchers in the ML/DL domain have employed the capabilities of deep neural network (DNN) models to identify and classify KOA images in an automated, faster, and accurate manner. To this end, we propose the application of six pretrained DNN models, namely, VGG16, VGG19, ResNet101, MobileNetV2, InceptionResNetV2, and DenseNet121 for KOA diagnosis using images obtained from the Osteoarthritis Initiative (OAI) dataset. More specifically, we perform two types of classification, namely, a binary classification, which detects the presence or absence of KOA and secondly, classifying the severity of KOA in a three-class classification. For a comparative analysis, we experiment on three datasets (Dataset I, Dataset II, and Dataset III) with five, two, and three classes of KOA images, respectively. We achieved maximum classification accuracies of 69%, 83%, and 89%, respectively, with the ResNet101 DNN model. Our results show an improved performance from the existing work in the literature.

A joint multi-modal learning method for early-stage knee osteoarthritis disease classification

Osteoarthritis (OA) is a progressive and chronic disease. Identifying the early stages of OA disease is important for the treatment and care of patients. However, most state-of-the-art methods only use single-modal data to predict disease status, so that these methods usually ignore complementary information in multi-modal data. In this study, we develop an integrated multi-modal learning method (MMLM) that uses an interpretable strategy to select and fuse clinical, imaging, and demographic features to classify the grade of early-stage knee OA disease. MMLM applies XGboost and ResNet50 to extract two heterogeneous features from the clinical data and imaging data, respectively. And then we integrate these extracted features with demographic data. To avoid the negative effects of redundant features in a direct integration of multiple features, we propose a L1-norm-based optimization method (MMLM) to regularize the inter-correlations among the multiple features. MMLM was assessed using the Osteoarthritis Initiative (OAI) data set with machine learning classifiers. Extensive experiments demonstrate that MMLM improves the performance of the classifiers. Furthermore, a visual analysis of the important features in the multimodal data verified the relations among the modalities when classifying the grade of knee OA disease.

IJES-OA Net: A residual neural network to classify knee osteoarthritis from radiographic images based on the edges of the intra-joint spaces

Among the musculoskeletal disorders in the world, osteoarthritis is the most common, affecting most of the body joints, especially the knee. Clinical radiographic imaging methods are commonly used to diagnose osteoarthritis thanks to their cheapness and availability. Due to the low quality and indiscernibility of these images, however, accurate osteoarthritis diagnosis has always faced inaccuracies, such as the wrong diagnosis. One of the osteoarthritis hallmarks is joint space narrowing. Thus, its degree and severity can be determined relatively by assessing the space between the bones in the joint. Therefore, in this research, a deep residual neural network, termed IJES-OA Net, is presented to automatically grade (classify) the severity of knee osteoarthritis via radiographs. This is achieved by tuning it in a way to have it focused on the distance of the edges of the bones inside the knee joint. Experimental results which are conducted on MOST (for training) and OAI (for validation and testing) datasets show that the IJES-OA Net achieves high average accuracy as well as average precision (80.23% and 0.802, respectively) while having less complexity compared to other methods. Additionally, the resulting attention maps from IJES-OA Net are accurate enough that increase experts' reliance on the provided results.

Variational autoencoder-based estimation of chronological age and changes in morphological features of teeth

This study led to the development of a variational autoencoder (VAE) for estimating the chronological age of subjects using feature values extracted from their teeth. Further, it determined how given teeth images affected the estimation accuracy. The developed VAE was trained with the first molar and canine tooth images, and a parallel VAE structure was further constructed to extract common features shared by the two types of teeth more effectively. The encoder of the VAE was combined with a regression model to estimate the age. To determine which parts of the tooth images were more or less important when estimating age, a method of visualizing the obtained regression coefficient using the decoder of the VAE was developed. The developed age estimation model was trained using data from 910 individuals aged 10-79. This model showed a median absolute error (MAE) of 6.99 years, demonstrating its ability to estimate age accurately. Furthermore, this method of visualizing the influence of particular parts of tooth images on the accuracy of age estimation using a decoder is expected to provide novel insights for future research on explainable artificial intelligence.

Prediction of knee osteoarthritis progression using radiological descriptors obtained from bone texture analysis and Siamese neural networks: data from OAI and MOST cohorts

BACKGROUND

Trabecular bone texture (TBT) analysis has been identified as an imaging biomarker that provides information on trabecular bone changes due to knee osteoarthritis (KOA). In parallel with the improvement in medical imaging technologies, machine learning methods have received growing interest in the scientific osteoarthritis community to potentially provide clinicians with prognostic data from conventional knee X-ray datasets, in particular from the Osteoarthritis Initiative (OAI) and the Multicenter Osteoarthritis Study (MOST) cohorts.

PATIENTS AND METHODS

This study included 1888 patients from OAI and 683 patients from MOST cohorts. Radiographs were automatically segmented to determine 16 regions of interest. Patients with an early stage of OA risk, with Kellgren and Lawrence (KL) grade of 1 < KL < 4, were selected. The definition of OA progression was an increase in the OARSI medial joint space narrowing (mJSN) grades over 48 months in OAI and 60 months in MOST. The performance of the TBT-CNN model was evaluated and compared to well-known prediction models using logistic regression.

RESULTS

The TBT-CNN model was predictive of the JSN progression with an area under the curve (AUC) up to 0.75 in OAI and 0.81 in MOST. The predictive ability of the TBT-CNN model was invariant with respect to the acquisition modality or image quality. The prediction models performed significantly better with estimated KL (KLprob) grades than those provided by radiologists. TBT-based models significantly outperformed KLprob-based models in MOST and provided similar performances in OAI. In addition, the combined model, when trained in one cohort, was able to predict OA progression in the other cohort.

CONCLUSION

The proposed combined model provides a good performance in the prediction of mJSN over 4 to 6 years in patients with relevant KOA. Furthermore, the current study presents an important contribution in showing that TBT-based OA prediction models can work with different databases.

Osteoarthritis year in review 2021: imaging

PURPOSE

To provide a narrative review of original articles on imaging of osteoarthritis (OA) published between January 1, 2020 and March 31, 2021, with a special focus on imaging of inflammation, imaging of bone, cartilage and bone-cartilage interactions, imaging of peri-articular tissues, imaging scoring methods for OA, and artificial intelligence (AI) applied to OA imaging.

METHODS

The Embase, Pubmed, Medline, Cochrane databases were searched for original research articles in the English language on human, in vivo, imaging of OA published between January 1, 2020 and March 31, 2021. Search terms related to osteoarthritis combined with all imaging modalities and artificial intelligence were applied. A selection of articles reporting on one of the focus topics was discussed further.

RESULTS

The search resulted in 651 articles, of which 214 were deemed relevant to human OA imaging. Among the articles included, the knee joint (69%) and magnetic resonance imaging (MRI) (52%) were the predominant anatomical area and imaging modality studied. There were also a substantial number of papers (n = 46) reporting on AI applications in the field of OA imaging.

CONCLUSION

Imaging continues to play an important role in the assessment of OA. Recent advances in OA imaging include quantitative, non-contrast, and hybrid imaging techniques for improved characterization of multiple tissue processes in OA. In addition, an increasing effort in AI techniques is undertaken to enhance OA imaging acquisition and analysis.

Emergence of Deep Learning in Knee Osteoarthritis Diagnosis

Osteoarthritis (OA), especially knee OA, is the most common form of arthritis, causing significant disability in patients worldwide. Manual diagnosis, segmentation, and annotations of knee joints remain as the popular method to diagnose OA in clinical practices, although they are tedious and greatly subject to user variation. Therefore, to overcome the limitations of the commonly used method as above, numerous deep learning approaches, especially the convolutional neural network (CNN), have been developed to improve the clinical workflow efficiency. Medical imaging processes, especially those that produce 3-dimensional (3D) images such as MRI, possess ability to reveal hidden structures in a volumetric view. Acknowledging that changes in a knee joint is a 3D complexity, 3D CNN has been employed to analyse the joint problem for a more accurate diagnosis in the recent years. In this review, we provide a broad overview on the current 2D and 3D CNN approaches in the OA research field. We reviewed 74 studies related to classification and segmentation of knee osteoarthritis from the Web of Science database and discussed the various state-of-the-art deep learning approaches proposed. We highlighted the potential and possibility of 3D CNN in the knee osteoarthritis field. We concluded by discussing the possible challenges faced as well as the potential advancements in adopting 3D CNNs in this field.

Towards novel osteoarthritis biomarkers: Multi-criteria evaluation of 46,996 segmented knee MRI data from the Osteoarthritis Initiative

Convolutional neural networks (CNNs) are the state-of-the-art for automated assessment of knee osteoarthritis (KOA) from medical image data. However, these methods lack interpretability, mainly focus on image texture, and cannot completely grasp the analyzed anatomies' shapes. In this study we assess the informative value of quantitative features derived from segmentations in order to assess their potential as an alternative or extension to CNN-based approaches regarding multiple aspects of KOA. Six anatomical structures around the knee (femoral and tibial bones, femoral and tibial cartilages, and both menisci) are segmented in 46,996 MRI scans. Based on these segmentations, quantitative features are computed, i.e., measurements such as cartilage volume, meniscal extrusion and tibial coverage, as well as geometric features based on a statistical shape encoding of the anatomies. The feature quality is assessed by investigating their association to the Kellgren-Lawrence grade (KLG), joint space narrowing (JSN), incident KOA, and total knee replacement (TKR). Using gold standard labels from the Osteoarthritis Initiative database the balanced accuracy (BA), the area under the Receiver Operating Characteristic curve (AUC), and weighted kappa statistics are evaluated. Features based on shape encodings of femur, tibia, and menisci plus the performed measurements showed most potential as KOA biomarkers. Differentiation between non-arthritic and severely arthritic knees yielded BAs of up to 99%, 84% were achieved for diagnosis of early KOA. Weighted kappa values of 0.73, 0.72, and 0.78 were achieved for classification of the grade of medial JSN, lateral JSN, and KLG, respectively. The AUC was 0.61 and 0.76 for prediction of incident KOA and TKR within one year, respectively. Quantitative features from automated segmentations provide novel biomarkers for KLG and JSN classification and show potential for incident KOA and TKR prediction. The validity of these features should be further evaluated, especially as extensions of CNN-based approaches. To foster such developments we make all segmentations publicly available together with this publication.

Trabecular bone texture analysis of conventional radiographs in the assessment of knee osteoarthritis: review and viewpoint

BACKGROUND

Trabecular bone texture analysis (TBTA) has been identified as an imaging biomarker that provides information on trabecular bone changes due to knee osteoarthritis (KOA). Consequently, it is important to conduct a comprehensive review that would permit a better understanding of this unfamiliar image analysis technique in the area of KOA research. We examined how TBTA, conducted on knee radiographs, is associated to (i) KOA incidence and progression, (ii) total knee arthroplasty, and (iii) KOA treatment responses. The primary aims of this study are twofold: to provide (i) a narrative review of the studies conducted on radiographic KOA using TBTA, and (ii) a viewpoint on future research priorities.

METHOD

Literature searches were performed in the PubMed electronic database. Studies published between June 1991 and March 2020 and related to traditional and fractal image analysis of trabecular bone texture (TBT) on knee radiographs were identified.

RESULTS

The search resulted in 219 papers. After title and abstract scanning, 39 studies were found eligible and then classified in accordance to six criteria: cross-sectional evaluation of osteoarthritis and non-osteoarthritis knees, understanding of bone microarchitecture, prediction of KOA progression, KOA incidence, and total knee arthroplasty and association with treatment response. Numerous studies have reported the relevance of TBTA as a potential bioimaging marker in the prediction of KOA incidence and progression. However, only a few studies have focused on the association of TBTA with both OA treatment responses and the prediction of knee joint replacement.

CONCLUSION

Clear evidence of biological plausibility for TBTA in KOA is already established. The review confirms the consistent association between TBT and important KOA endpoints such as KOA radiographic incidence and progression. TBTA could provide markers for enrichment of clinical trials enhancing the screening of KOA progressors. Major advances were made towards a fully automated assessment of KOA.

Learning from Highly Confident Samples for Automatic Knee Osteoarthritis Severity Assessment: Data from the Osteoarthritis Initiative

Knee osteoarthritis (OA) is a chronic disease that considerably reduces patients' life quality. Preventive therapies require early detection and lifetime monitor of OA progression. In the clinical environment, the severity of OA is classified by Kellgren and Lawrence (KL) grading system, ranging from KL-0 to KL-4. Recently, deep learning methods were applied to OA severity assessment to improve the accuracy and efficiency. Researchers fine-tuned convolution neural networks (CNN) on the OA dataset and built end-to-end approaches. However, this task is still challenging due to the ambiguity between adjacent grading, especially in early-stage OA. Low confident samples, which are less representative than the typical ones, undermine the training process. Targeting the uncertainty in the OA dataset, we propose a novel learning scheme that dynamically separates the data into two sets according to their reliability. Besides, we design a hybrid loss function to help CNN learn from the two sets accordingly. With the proposed approach, we emphasize the typical samples and control the impacts of low confident cases. Experiments are conducted in a five-fold manner. Our method achieves a mean accuracy of 70.13\% on the five-class OA assessment task, which outperforms all other start-of-art methods. Despite that early-stage OA detection still benefits from the human intervention of lesion region selection, our approach achieves superior performance on the KL-0 vs. KL-2 task. Moreover, we design an experiment to validate large-scale automatic data refining during training. The result verifies the ability of characterizing low confidence samples by our approach. Dataset used in this paper was obtained from the osteoarthritis Initiative.