Applying cascaded convolutional neural network design further enhances automatic scoring of arthritis disease activity on ultrasound images from rheumatoid arthritis patients

Advancing precision rheumatology: applications of machine learning for rheumatoid arthritis management

Rheumatoid arthritis (RA) is an autoimmune disease causing progressive joint damage. Early diagnosis and treatment is critical, but remains challenging due to RA complexity and heterogeneity. Machine learning (ML) techniques may enhance RA management by identifying patterns within multidimensional biomedical data to improve classification, diagnosis, and treatment predictions. In this review, we summarize the applications of ML for RA management. Emerging studies or applications have developed diagnostic and predictive models for RA that utilize a variety of data modalities, including electronic health records, imaging, and multi-omics data. High-performance supervised learning models have demonstrated an Area Under the Curve (AUC) exceeding 0.85, which is used for identifying RA patients and predicting treatment responses. Unsupervised learning has revealed potential RA subtypes. Ongoing research is integrating multimodal data with deep learning to further improve performance. However, key challenges remain regarding model overfitting, generalizability, validation in clinical settings, and interpretability. Small sample sizes and lack of diverse population testing risks overestimating model performance. Prospective studies evaluating real-world clinical utility are lacking. Enhancing model interpretability is critical for clinician acceptance. In summary, while ML shows promise for transforming RA management through earlier diagnosis and optimized treatment, larger scale multisite data, prospective clinical validation of interpretable models, and testing across diverse populations is still needed. As these gaps are addressed, ML may pave the way towards precision medicine in RA.

Deep learning driven diagnosis of malignant soft tissue tumors based on dual-modal ultrasound images and clinical indexes

Background

Soft tissue tumors (STTs) are benign or malignant superficial neoplasms arising from soft tissues throughout the body with versatile pathological types. Although Ultrasonography (US) is one of the most common imaging tools to diagnose malignant STTs, it still has several drawbacks in STT diagnosis that need improving.

Objectives

The study aims to establish this deep learning (DL) driven Artificial intelligence (AI) system for predicting malignant STTs based on US images and clinical indexes of the patients.

Methods

We retrospectively enrolled 271 malignant and 462 benign masses to build the AI system using 5-fold validation. A prospective dataset of 44 malignant masses and 101 benign masses was used to validate the accuracy of system. A multi-data fusion convolutional neural network, named ultrasound clinical soft tissue tumor net (UC-STTNet), was developed to combine gray scale and color Doppler US images and clinic features for malignant STTs diagnosis. Six radiologists (R1-R6) with three experience levels were invited for reader study.

Results

The AI system achieved an area under receiver operating curve (AUC) value of 0.89 in the retrospective dataset. The diagnostic performance of the AI system was higher than that of one of the senior radiologists (AUC of AI vs R2: 0.89 vs. 0.84, p=0.022) and all of the intermediate and junior radiologists (AUC of AI vs R3, R4, R5, R6: 0.89 vs 0.75, 0.81, 0.80, 0.63; p <0.01). The AI system also achieved an AUC of 0.85 in the prospective dataset. With the assistance of the system, the diagnostic performances and inter-observer agreement of the radiologists was improved (AUC of R3, R5, R6: 0.75 to 0.83, 0.80 to 0.85, 0.63 to 0.69; p<0.01).

Conclusion

The AI system could be a useful tool in diagnosing malignant STTs, and could also help radiologists improve diagnostic performance.

Artificial intelligence model for segmentation and severity scoring of osteophytes in hand osteoarthritis on ultrasound images

Objective

To develop an artificial intelligence (AI) model able to perform both segmentation of hand joint ultrasound images for osteophytes, bone, and synovium and perform osteophyte severity scoring following the EULAR-OMERACT grading system (EOGS) for hand osteoarthritis (OA).

Methods

One hundred sixty patients with pain or reduced function of the hands were included. Ultrasound images of the metacarpophalangeal (MCP), proximal interphalangeal (PIP), distal interphalangeal (DIP), and first carpometacarpal (CMC1) joints were then manually segmented for bone, synovium and osteophytes and scored from 0 to 3 according to the EOGS for OA. Data was divided into a training, validation, and test set. The AI model was trained on the training data to perform bone, synovium, and osteophyte identification on the images. Based on the manually performed image segmentation, an AI was trained to classify the severity of osteophytes according to EOGS from 0 to 3. Percent Exact Agreement (PEA) and Percent Close Agreement (PCA) were assessed on individual joints and overall. PCA allows a difference of one EOGS grade between doctor assessment and AI.

Results

A total of 4615 ultrasound images were used for AI development and testing. The developed AI model scored on the test set for the MCP joints a PEA of 76% and PCA of 97%; for PIP, a PEA of 70% and PCA of 97%; for DIP, a PEA of 59% and PCA of 94%, and CMC a PEA of 50% and PCA of 82%. Combining all joints, we found a PEA between AI and doctor assessments of 68% and a PCA of 95%.

Conclusion

The developed AI model can perform joint ultrasound image segmentation and severity scoring of osteophytes, according to the EOGS. As proof of concept, this first version of the AI model is successful, as the agreement performance is slightly higher than previously found agreements between experts when assessing osteophytes on hand OA ultrasound images. The segmentation of the image makes the AI explainable to the doctor, who can immediately see why the AI applies a given score. Future validation in hand OA cohorts is necessary though.

Deep learning-based automatic scoring models for the disease activity of rheumatoid arthritis based on multimodal ultrasound images

OBJECTIVES

We aimed to investigate the value of deep learning (DL) models based on multimodal ultrasonographic (US) images to quantify RA activity.

METHODS

Static greyscale (SGS), dynamic greyscale (DGS), static power Doppler (SPD) and dynamic power Doppler (DPD) US images were collected and evaluated by two expert radiologists according to the EULAR-OMERACT Synovitis Scoring system. Four DL models were developed based on the ResNet-type structure, evaluated on two separate test cohorts, and finally compared with the performance of 12 radiologists with different levels of experience.

RESULTS

In total, 1244 images were used for the model training, and 152 and 354 for testing (cohort 1 and 2, respectively). The best-performing models for the scores of 0/1/2/3 were the DPD, SGS, DGS and SPD models, respectively (Area Under the receiver operating characteristic Curve [AUC] = 0.87/0.95/0.74/0.95; no significant differences). All the DL models provided results comparable to the experienced radiologists on a per-image basis (intraclass correlation coefficient: 0.239-0.756, P < 0.05). The SPD model performed better than the SGS one on test cohort 1 (score of 0/2/3: AUC = 0.82/0.67/0.95 vs 0.66/0.66/0.75, respectively) and test cohort 2 (score of 0: AUC = 0.89 vs 0.81). The dynamic DL models performed better than the static ones in most of the scoring processes and were more accurate than the most of senior radiologists, especially the DPD model.

CONCLUSION

DL models based on multimodal US images allow a quantitative and objective assessment of RA activity. Dynamic DL models in particular have potential value in assisting radiologists to improve the accuracy of RA US-based grading.

Deep learning in rheumatological image interpretation

Artificial intelligence techniques, specifically deep learning, have already affected daily life in a wide range of areas. Likewise, initial applications have been explored in rheumatology. Deep learning might not easily surpass the accuracy of classic techniques when performing classification or regression on low-dimensional numerical data. With images as input, however, deep learning has become so successful that it has already outperformed the majority of conventional image-processing techniques developed during the past 50 years. As with any new imaging technology, rheumatologists and radiologists need to consider adapting their arsenal of diagnostic, prognostic and monitoring tools, and even their clinical role and collaborations. This adaptation requires a basic understanding of the technical background of deep learning, to efficiently utilize its benefits but also to recognize its drawbacks and pitfalls, as blindly relying on deep learning might be at odds with its capabilities. To facilitate such an understanding, it is necessary to provide an overview of deep-learning techniques for automatic image analysis in detecting, quantifying, predicting and monitoring rheumatic diseases, and of currently published deep-learning applications in radiological imaging for rheumatology, with critical assessment of possible limitations, errors and confounders, and conceivable consequences for rheumatologists and radiologists in clinical practice.

Computer vision detects inflammatory arthritis in standardized smartphone photographs in an Indian patient cohort

Introduction

Computer vision extracts meaning from pixelated images and holds promise in automating various clinical tasks. Convolutional neural networks (CNNs), a deep learning network used therein, have shown promise in analyzing X-ray images and joint photographs. We studied the performance of a CNN on standardized smartphone photographs in detecting inflammation in three hand joints and compared it to a rheumatologist's diagnosis.

Methods

We enrolled 100 consecutive patients with inflammatory arthritis with an onset period of less than 2 years, excluding those with deformities. Each patient was examined by a rheumatologist, and the presence of synovitis in each joint was recorded. Hand photographs were taken in a standardized manner, anonymized, and cropped to include joints of interest. A ResNet-101 backbone modified for two class outputs (inflamed or not) was used for training. We also tested a hue-augmented dataset. We reported accuracy, sensitivity, and specificity for three joints: wrist, index finger proximal interphalangeal (IFPIP), and middle finger proximal interphalangeal (MFPIP), taking the rheumatologist's opinion as the gold standard.

Results

The cohort consisted of 100 individuals, of which 22 of them were men, with a mean age of 49.7 (SD 12.9) years. The majority of the cohort (n = 68, 68%) had rheumatoid arthritis. The wrist (125/200, 62.5%), MFPIP (94/200, 47%), and IFPIP (83/200, 41.5%) were the three most commonly inflamed joints. The CNN achieved the highest accuracy, sensitivity, and specificity in detecting synovitis in the MFPIP (83, 77, and 88%, respectively), followed by the IFPIP (74, 74, and 75%, respectively) and the wrist (62, 90, and 21%, respectively).

Discussion

We have demonstrated that computer vision was able to detect inflammation in three joints of the hand with reasonable accuracy on standardized photographs despite a small dataset. Feature engineering was not required, and the CNN worked despite a diversity in clinical diagnosis. Larger datasets are likely to improve accuracy and help explain the basis of classification. These data suggest a potential use of computer vision in screening and follow-up of inflammatory arthritis.

Automatic evaluation of atlantoaxial subluxation in rheumatoid arthritis by a deep learning model

BACKGROUND

This work aims to develop a deep learning model, assessing atlantoaxial subluxation (AAS) in rheumatoid arthritis (RA), which can often be ambiguous in clinical practice.

METHODS

We collected 4691 X-ray images of the cervical spine of the 906 patients with RA. Among these images, 3480 were used for training the deep learning model, 803 were used for validating the model during the training process, and the remaining 408 were used for testing the performance of the trained model. The two-dimensional key points' detection model of Deep High-Resolution Representation Learning for Human Pose Estimation was adopted as the base convolutional neural network model. The model inferred four coordinates to calculate the atlantodental interval (ADI) and space available for the spinal cord (SAC). Finally, these values were compared with those by clinicians to evaluate the performance of the model.

RESULTS

Among the 408 cervical images for testing the performance, the trained model correctly identified the four coordinates in 99.5% of the dataset. The values of ADI and SAC were positively correlated among the model and two clinicians. The sensitivity of AAS diagnosis with ADI or SAC by the model was 0.86 and 0.97 respectively. The specificity of that was 0.57 and 0.5 respectively.

CONCLUSIONS

We present the development of a deep learning model for the evaluation of cervical lesions of patients with RA. The model was demonstrably shown to be useful for quantitative evaluation.

High Frequency of Osteophytes Detected by High-Resolution Ultrasound at the Finger Joints of Asymptomatic Factory Workers

Hand Osteoarthritis (HOA) is a frequently occurring musculoskeletal disease that impacts health. Diagnostic criteria often incorporate osteophytes documented through imaging procedures. Radiographic imaging is considered the gold standard; however, more sensitive and safer methods like ultrasound imaging are becoming increasingly important. We conducted a population-based cross-sectional study to examine the prevalence, grade, and pattern of osteophytes using high-resolution ultrasound investigation. Factory workers were recruited on-site for the study. Each participant had 26 finger joints examined using ultrasonography to grade the occurrence of osteophytes on a semi-quantitative scale ranging from 0-3, where higher scores indicate larger osteophytes. A total of 427 participants (mean age 53.5 years, range 20-79 years) were included, resulting in 11,000 joints scored. At least one osteophyte was found in 4546 out of 11,000 (41.3%) joints or in 426 out of 427 (99.8%) participants, but only 5.0% (553) of the joints showed grade 2 or 3 osteophytes. The total osteophyte sum score increased by 0.18 per year as age increased (p < 0.001). The distal interphalangeal joints were the most commonly affected, with 61%, followed by the proximal interphalangeal joints with 48%, carpometacarpal joint 1 with 39%, and metacarpophalangeal joints with 16%. There was no observed impact of gender or workload. In conclusion, ultrasound imaging proves to be a practical screening tool for osteophytes and HOA. Grade 1 osteophytes are often detected in the working population through ultrasound assessments and their incidence increases with age. The occurrence of grade 2 or 3 osteophytes is less frequent and indicates the clinical presence of HOA. Subsequent evaluations are imperative to ascertain the predictive significance of early osteophytes.

Understanding the role and adoption of artificial intelligence techniques in rheumatology research: An in-depth review of the literature

The major and upward trend in the number of published research related to rheumatic and musculoskeletal diseases, in which artificial intelligence plays a key role, has exhibited the interest of rheumatology researchers in using these techniques to answer their research questions. In this review, we analyse the original research articles that combine both worlds in a five- year period (2017-2021). In contrast to other published papers on the same topic, we first studied the review and recommendation articles that were published during that period, including up to October 2022, as well as the publication trends. Secondly, we review the published research articles and classify them into one of the following categories: disease identification and prediction, disease classification, patient stratification and disease subtype identification, disease progression and activity, treatment response, and predictors of outcomes. Thirdly, we provide a table with illustrative studies in which artificial intelligence techniques have played a central role in more than twenty rheumatic and musculoskeletal diseases. Finally, the findings of the research articles, in terms of disease and/or data science techniques employed, are highlighted in a discussion. Therefore, the present review aims to characterise how researchers are applying data science techniques in the rheumatology medical field. The most immediate conclusions that can be drawn from this work are: multiple and novel data science techniques have been used in a wide range of rheumatic and musculoskeletal diseases including rare diseases; the sample size and the data type used are heterogeneous, and new technical approaches are expected to arrive in the short-middle term.

Artificial Intelligence in Rheumatoid Arthritis: Current Status and Future Perspectives: A State-of-the-Art Review

Investigation of the potential applications of artificial intelligence (AI), including machine learning (ML) and deep learning (DL) techniques, is an exponentially growing field in medicine and healthcare. These methods can be critical in providing high-quality care to patients with chronic rheumatological diseases lacking an optimal treatment, like rheumatoid arthritis (RA), which is the second most prevalent autoimmune disease. Herein, following reviewing the basic concepts of AI, we summarize the advances in its applications in RA clinical practice and research. We provide directions for future investigations in this field after reviewing the current knowledge gaps and technical and ethical challenges in applying AI. Automated models have been largely used to improve RA diagnosis since the early 2000s, and they have used a wide variety of techniques, e.g., support vector machine, random forest, and artificial neural networks. AI algorithms can facilitate screening and identification of susceptible groups, diagnosis using omics, imaging, clinical, and sensor data, patient detection within electronic health record (EHR), i.e., phenotyping, treatment response assessment, monitoring disease course, determining prognosis, novel drug discovery, and enhancing basic science research. They can also aid in risk assessment for incidence of comorbidities, e.g., cardiovascular diseases, in patients with RA. However, the proposed models may vary significantly in their performance and reliability. Despite the promising results achieved by AI models in enhancing early diagnosis and management of patients with RA, they are not fully ready to be incorporated into clinical practice. Future investigations are required to ensure development of reliable and generalizable algorithms while they carefully look for any potential source of bias or misconduct. We showed that a growing body of evidence supports the potential role of AI in revolutionizing screening, diagnosis, and management of patients with RA. However, multiple obstacles hinder clinical applications of AI models. Incorporating the machine and/or deep learning algorithms into real-world settings would be a key step in the progress of AI in medicine.

RATING: Medical knowledge-guided rheumatoid arthritis assessment from multimodal ultrasound images via deep learning

Multimodal ultrasound has demonstrated its power in the clinical assessment of rheumatoid arthritis (RA). However, for radiologists, it requires strong experience. In this paper, we propose a rheumatoid arthritis knowledge guided (RATING) system that automatically scores the RA activity and generates interpretable features to assist radiologists' decision-making based on deep learning. RATING leverages the complementary advantages of multimodal ultrasound images and solves the limited training data problem with self-supervised pretraining. RATING outperforms all of the existing methods, achieving an accuracy of 86.1% on a prospective test dataset and 85.0% on an external test dataset. A reader study demonstrates that the RATING system improves the average accuracy of 10 radiologists from 41.4% to 64.0%. As an assistive tool, not only can RATING indicate the possible lesions and enhance the diagnostic performance with multimodal ultrasound but it can also enlighten the road to human-machine collaboration in healthcare.

•

RATING is a medical knowledge-guided deep learning system for RA assessment

•

It leverages diagnostic paradigm and experience to enhance the robustness

•

Self-supervised pretraining ensures reliability even with limited training data

•

A clinical reader study demonstrates its effectiveness in assisting RA assessment

Rheumatoid arthritis (RA) has detrimental outcomes, including increased disability and mortality. To enhance the clinical assessment of RA, we propose a rheumatoid arthritis knowledge guided (RATING) system for scoring RA activity from multimodal ultrasound images. It combines the knowledge of clinical diagnosis with deep learning, serving as an example of designing deep learning systems for handling real clinical problems. We further integrated the system into the clinical decision-making process via human-machine collaboration and demonstrated significant improvements in assessment performance. We expect that our research will illuminate the road to human-machine collaboration and help transform clinical diagnostics and precision medicine in a wider range of biomedical research.

Ultrasound joint examination by an automated system versus by a rheumatologist: from a patient perspective

Background

The Arthritis Ultrasound Robot (ARTHUR) is an automated system for ultrasound scanning of the joints of both hands and wrists, with subsequent disease activity scoring using artificial intelligence. The objective was to describe the patient’s perspective of being examined by ARTHUR, compared to an ultrasound examination by a rheumatologist. Further, to register any safety issues with the use of ARTHUR.

Methods

Twenty-five patients with rheumatoid arthritis (RA) had both hands and wrists examined by ultrasound, first by a rheumatologist and subsequently by ARTHUR. Patient-reported outcomes (PROs) were obtained after the examination by the rheumatologist and by ARTHUR. PROs regarding pain, discomfort and overall experience were collected, including willingness to be examined again by ARTHUR as part of future clinical follow-up. All ARTHUR examinations were observed for safety issues.

Results

There was no difference in pain or discomfort between the examination by a rheumatologist and by ARTHUR (p = 0.29 and p = 0.20, respectively). The overall experience of ARTHUR was described as very good or good by 92% (n = 23), with no difference compared to the examination by the rheumatologist (p = 0.50). All (n = 25) patients were willing to be examined by ARTHUR again, and 92% (n = 23) would accept ARTHUR as a regular part of their RA clinical follow up. No safety issues were registered.

Conclusions

Joint ultrasound examination by ARTHUR was safe and well-received, with no difference in PRO components compared to ultrasound examination by a rheumatologist. Fully automated systems for RA disease activity assessment could be important in future strategies for managing RA patients.

Trial registration: The study was evaluated by the regional ethics committee (ID: S-20200145), which ruled it was not a clinical trial necessary for their approval. It was a quality assessment project, as there was no intervention to the patient. The study was hereafter submitted and registered to Odense University Hospital, Region of Southern Denmark as a quality assessment project and approved (ID: 20/55294).

Dorsal Finger Fold Recognition by Convolutional Neural Networks for the Detection and Monitoring of Joint Swelling in Patients with Rheumatoid Arthritis

Digital biomarkers such as wearables are of increasing interest in monitoring rheumatic diseases, but they usually lack disease specificity. In this study, we apply convolutional neural networks (CNN) to real-world hand photographs in order to automatically detect, extract, and analyse dorsal finger fold lines as a correlate of proximal interphalangeal (PIP) joint swelling in patients with rheumatoid arthritis (RA). Hand photographs of RA patients were taken by a smartphone camera in a standardized manner. Overall, 190 PIP joints were categorized as either swollen or not swollen based on clinical judgement and ultrasound. Images were automatically preprocessed by cropping PIP joints and extracting dorsal finger folds. Subsequently, metrical analysis of dorsal finger folds was performed, and a CNN was trained to classify the dorsal finger lines into swollen versus non-swollen joints. Representative horizontal finger folds were also quantified in a subset of patients before and after resolution of PIP swelling and in patients with disease flares. In swollen joints, the number of automatically extracted deep skinfold imprints was significantly reduced compared to non-swollen joints (1.3, SD 0.8 vs. 3.3, SD 0.49, p < 0.01). The joint diameter/deep skinfold length ratio was significantly higher in swollen (4.1, SD 1.4) versus non-swollen joints (2.1, SD 0.6, p < 0.01). The CNN model successfully differentiated swollen from non-swollen joints based on finger fold patterns with a validation accuracy of 0.84, a sensitivity of 88%, and a specificity of 75%. A heatmap of the original images obtained by an extraction algorithm confirmed finger folds as the region of interest for correct classification. After significant response to disease-modifying antirheumatic drug ± corticosteroid therapy, longitudinal metrical analysis of eight representative deep finger folds showed a decrease in the mean diameter/finger fold length (finger fold index, FFI) from 3.03 (SD 0.68) to 2.08 (SD 0.57). Conversely, the FFI increased in patients with disease flares. In conclusion, automated preprocessing and the application of CNN algorithms in combination with longitudinal metrical analysis of dorsal finger fold patterns extracted from real-world hand photos might serve as a digital biomarker in RA.

Deep Learning-Based Classification of Inflammatory Arthritis by Identification of Joint Shape Patterns—How Neural Networks Can Tell Us Where to “Deep Dive” Clinically

Objective:

We investigated whether a neural network based on the shape of joints can differentiate between rheumatoid arthritis (RA), psoriatic arthritis (PsA), and healthy controls (HC), which class patients with undifferentiated arthritis (UA) are assigned to, and whether this neural network is able to identify disease-specific regions in joints.

Methods

We trained a novel neural network on 3D articular bone shapes of hand joints of RA and PsA patients as well as HC. Bone shapes were created from high-resolution peripheral-computed-tomography (HR-pQCT) data of the second metacarpal bone head. Heat maps of critical spots were generated using GradCAM. After training, we fed shape patterns of UA into the neural network to classify them into RA, PsA, or HC.

Results

Hand bone shapes from 932 HR-pQCT scans of 617 patients were available. The network could differentiate the classes with an area-under-receiver-operator-curve of 82% for HC, 75% for RA, and 68% for PsA. Heat maps identified anatomical regions such as bare area or ligament attachments prone to erosions and bony spurs. When feeding UA data into the neural network, 86% were classified as “RA,” 11% as “PsA,” and 3% as “HC” based on the joint shape.

Conclusion

We investigated neural networks to differentiate the shape of joints of RA, PsA, and HC and extracted disease-specific characteristics as heat maps on 3D joint shapes that can be utilized in clinical routine examination using ultrasound. Finally, unspecific diseases such as UA could be grouped using the trained network based on joint shape.

Development of a convolutional neural network for the identification and the measurement of the median nerve on ultrasound images acquired at carpal tunnel level

Background

Deep learning applied to ultrasound (US) can provide a feedback to the sonographer about the correct identification of scanned tissues and allows for faster and standardized measurements. The most frequently adopted parameter for US diagnosis of carpal tunnel syndrome is the increasing of the cross-sectional area (CSA) of the median nerve. Our aim was to develop a deep learning algorithm, relying on convolutional neural networks (CNNs), for the localization and segmentation of the median nerve and the automatic measurement of its CSA on US images acquired at the proximal inlet of the carpal tunnel.

Methods

Consecutive patients with rheumatic and musculoskeletal disorders were recruited. Transverse US images were acquired at the carpal tunnel inlet, and the CSA was manually measured. Anatomical variants were registered. The dataset consisted of 246 images (157 for training, 40 for validation, and 49 for testing) from 103 patients each associated with manual annotations of the nerve boundary. A Mask R-CNN, state-of-the-art CNN for image semantic segmentation, was trained on this dataset to accurately localize and segment the median nerve section. To evaluate the performances on the testing set, precision (Prec), recall (Rec), mean average precision (mAP), and Dice similarity coefficient (DSC) were computed. A sub-analysis excluding anatomical variants was performed. The CSA was automatically measured by the algorithm.

Results

The algorithm correctly identified the median nerve in 41/49 images (83.7%) and in 41/43 images (95.3%) excluding anatomical variants. The following metrics were obtained (with and without anatomical variants, respectively): Prec 0.86 ± 0.33 and 0.96 ± 0.18, Rec 0.88 ± 0.33 and 0.98 ± 0.15, mAP 0.88 ± 0.33 and 0.98 ± 0.15, and DSC 0.86 ± 0.19 and 0.88 ± 0.19. The agreement between the algorithm and the sonographer CSA measurements was excellent [ICC 0.97 (0.94–0.98)].

Conclusions

The developed algorithm has shown excellent performances, especially if excluding anatomical variants. Future research should aim at expanding the US image dataset including a wider spectrum of normal anatomy and pathology. This deep learning approach has shown very high potentiality for a fully automatic support for US assessment of carpal tunnel syndrome.

Artificial Intelligence for Ultrasound Informative Image Selection of Metacarpal Head Cartilage. A Pilot Study

Objectives: This study aims to develop an automatic deep-learning algorithm, which is based on Convolutional Neural Networks (CNNs), for ultrasound informative-image selection of hyaline cartilage at metacarpal head level. The algorithm performance and that of three beginner sonographers were compared with an expert assessment, which was considered the gold standard.

Methods: The study was divided into two steps. In the first one, an automatic deep-learning algorithm for image selection was developed using 1,600 ultrasound (US) images of the metacarpal head cartilage (MHC) acquired in 40 healthy subjects using a very high-frequency probe (up to 22 MHz). The algorithm task was to identify US images defined informative as they show enough information to fulfill the Outcome Measure in Rheumatology US definition of healthy hyaline cartilage. The algorithm relied on VGG16 CNN, which was fine-tuned to classify US images in informative and non-informative ones. A repeated leave-four-subject out cross-validation was performed using the expert sonographer assessment as gold-standard. In the second step, the expert assessed the algorithm and the beginner sonographers' ability to obtain US informative images of the MHC.

Results: The VGG16 CNN showed excellent performance in the first step, with a mean area (AUC) under the receiver operating characteristic curve, computed among the 10 models obtained from cross-validation, of 0.99 ± 0.01. The model that reached the best AUC on the testing set, which we named “MHC identifier 1,” was then evaluated by the expert sonographer. The agreement between the algorithm, and the expert sonographer was almost perfect [Cohen's kappa: 0.84 (95% confidence interval: 0.71–0.98)], whereas the agreement between the expert and the beginner sonographers using conventional assessment was moderate [Cohen's kappa: 0.63 (95% confidence interval: 0.49–0.76)]. The conventional obtainment of US images by beginner sonographers required 6.0 ± 1.0 min, whereas US videoclip acquisition by a beginner sonographer lasted only 2.0 ± 0.8 min.

Conclusion: This study paves the way for the automatic identification of informative US images for assessing MHC. This may redefine the US reliability in the evaluation of MHC integrity, especially in terms of intrareader reliability and may support beginner sonographers during US training.