Imaging in Short Stature and Bone Age Estimation

Clinical application of artificial intelligence in longitudinal image analysis of bone age among GHD patients

OBJECTIVE

This study aims to explore the clinical value of artificial intelligence (AI)-assisted bone age assessment (BAA) among children with growth hormone deficiency (GHD).

METHODS

A total of 290 bone age (BA) radiographs were collected from 52 children who participated in the study at Sun Yat-sen Memorial Hospital between January 2016 and August 2017. Senior pediatric endocrinologists independently evaluated BA according to the China 05 (CH05) method, and their consistent results were regarded as the gold standard (GS). Meanwhile, two junior pediatric endocrinologists were asked to assessed BA both with and without assistance from the AI-based BA evaluation system. Six months later, around 20% of the images assessed by the junior pediatric endocrinologists were randomly selected to be re-evaluated with the same procedure half a year later. Root mean square error (RMSE), mean absolute error (MAE), accuracy, and Bland-Altman plots were used to compare differences in BA. The intra-class correlation coefficient (ICC) and one-way repeated ANOVA were used to assess inter- and intra-observer variabilities in BAA. A boxplot of BA evaluated by different raters during the course of treatment and a mixed linear model were used to illustrate inter-rater effect over time.

RESULTS

A total of 52 children with GHD were included, with mean chronological age and BA by GS of 6.64 ± 2.49 and 5.85 ± 2.30 years at baseline, respectively. After incorporating AI assistance, the performance of the junior pediatric endocrinologists improved (P < 0.001), with MAE and RMSE both decreased by more than 1.65 years (Rater 1: ΔMAE = 1.780, ΔRMSE = 1.655; Rater 2: ΔMAE = 1.794, ΔRMSE = 1.719), and accuracy increasing from approximately 10% to over 91%. The ICC also increased from 0.951 to 0.990. During GHD treatment (at baseline, 6-, 12-, 18-, and 24-months), the difference decreased sharply when AI was applied. Furthermore, a significant inter-rater effect (P = 0.002) also vanished upon AI involvement.

CONCLUSION

AI-assisted interpretation of BA can improve accuracy and decrease variability in results among junior pediatric endocrinologists in longitudinal cohort studies, which shows potential for further clinical application.

Artificial Intelligence-Assisted Bone Age Assessment to Improve the Accuracy and Consistency of Physicians With Different Levels of Experience

BACKGROUND

The accuracy and consistency of bone age assessments (BAA) using standard methods can vary with physicians' level of experience.

METHODS

To assess the impact of information from an artificial intelligence (AI) deep learning convolutional neural network (CNN) model on BAA, specialists with different levels of experience (junior, mid-level, and senior) assessed radiographs from 316 children aged 4-18 years that had been randomly divided into two equal sets-group A and group B. Bone age (BA) was assessed independently by each specialist without additional information (group A) and with information from the model (group B). With the mean assessment of four experts as the reference standard, mean absolute error (MAE), and intraclass correlation coefficient (ICC) were calculated to evaluate accuracy and consistency. Individual assessments of 13 bones (radius, ulna, and short bones) were also compared between group A and group B with the rank-sum test.

RESULTS

The accuracies of senior, mid-level, and junior physicians were significantly better (all P < 0.001) with AI assistance (MAEs 0.325, 0.344, and 0.370, respectively) than without AI assistance (MAEs 0.403, 0.469, and 0.755, respectively). Moreover, for senior, mid-level, and junior physicians, consistency was significantly higher (all P < 0.001) with AI assistance (ICCs 0.996, 0.996, and 0.992, respectively) than without AI assistance (ICCs 0.987, 0.989, and 0.941, respectively). For all levels of experience, accuracy with AI assistance was significantly better than accuracy without AI assistance for assessments of the first and fifth proximal phalanges.

CONCLUSIONS

Information from an AI model improves both the accuracy and the consistency of bone age assessments for physicians of all levels of experience. The first and fifth proximal phalanges are difficult to assess, and they should be paid more attention.