SMANet: multi-region ensemble of convolutional neural network model for skeletal maturity assessment

Artificial intelligence model system for bone age assessment of preschool children

BACKGROUD

Our study aimed to assess the impact of inter- and intra-observer variations when utilizing an artificial intelligence (AI) system for bone age assessment (BAA) of preschool children.

METHODS

A retrospective study was conducted involving a total sample of 53 female individuals and 41 male individuals aged 3-6 years in China. Radiographs were assessed by four mid-level radiology reviewers using the TW3 and RUS-CHN methods. Bone age (BA) was analyzed in two separate situations, with/without the assistance of AI. Following a 4-week wash-out period, radiographs were reevaluated in the same manner. Accuracy metrics, the correlation coefficient (ICC)and Bland-Altman plots were employed.

RESULTS

The accuracy of BAA by the reviewers was significantly improved with AI. The results of RMSE and MAE decreased in both methods (p < 0.001). When comparing inter-observer agreement in both methods and intra-observer reproducibility in two interpretations, the ICC results were improved with AI. The ICC values increased in both two interpretations for both methods and exceeded 0.99 with AI.

CONCLUSION

In the assessment of BA for preschool children, AI was found to be capable of reducing inter-observer variability and enhancing intra-observer reproducibility, which can be considered an important tool for clinical work by radiologists.

IMPACT

The RUS-CHN method is a special bone age method devised to be suitable for Chinese children. The preschool stage is a critical phase for children, marked by a high degree of variability that renders BA prediction challenging. The accuracy of BAA by the reviewers can be significantly improved with the aid of an AI model system. This study is the first to assess the impact of inter- and intra-observer variations when utilizing an AI model system for BAA of preschool children using both the TW3 and RUS-CHN methods.

Validation of an established TW3 artificial intelligence bone age assessment system: a prospective, multicenter, confirmatory study

Background

In 2020, our center established a Tanner-Whitehouse 3 (TW3) artificial intelligence (AI) system using a convolutional neural network (CNN), which was built upon 9059 radiographs. However, the system, upon which our study is based, lacked a gold standard for comparison and had not undergone thorough evaluation in different working environments.

Methods

To further verify the applicability of the AI system in clinical bone age assessment (BAA) and to enhance the accuracy and homogeneity of BAA, a prospective multi-center validation was conducted. This study utilized 744 left-hand radiographs of patients, ranging from 1 to 20 years of age, with 378 boys and 366 girls. These radiographs were obtained from nine different children's hospitals between August and December 2020. The BAAs were performed using the TW3 AI system and were also reviewed by experienced reviewers. Bone age accuracy within 1 year, root mean square error (RMSE), and mean absolute error (MAE) were statistically calculated to evaluate the accuracy. Kappa test and Bland-Altman (B-A) plot were conducted to measure the diagnostic consistency.

Results

The system exhibited a high level of performance, producing results that closely aligned with those of the reviewers. It achieved a RMSE of 0.52 years and an accuracy of 94.55% for the radius, ulna, and short bones series. When assessing the carpal series of bones, the system achieved a RMSE of 0.85 years and an accuracy of 80.38%. Overall, the system displayed satisfactory accuracy and RMSE, particularly in patients over 7 years old. The system excelled in evaluating the carpal bone age of patients aged 1-6. Both the Kappa test and B-A plot demonstrated substantial consistency between the system and the reviewers, although the model encountered challenges in consistently distinguishing specific bones, such as the capitate. Furthermore, the system's performance proved acceptable across different genders and age groups, as well as radiography instruments.

Conclusions

In this multi-center validation, the system showcased its potential to enhance the efficiency and consistency of healthy delivery, ultimately resulting in improved patient outcomes and reduced healthcare costs.

A real-time automated bone age assessment system based on the RUS-CHN method

Background

Bone age is the age of skeletal development and is a direct indicator of physical growth and development in children. Most bone age assessment (BAA) systems use direct regression with the entire hand bone map or first segmenting the region of interest (ROI) using the clinical a priori method and then deriving the bone age based on the characteristics of the ROI, which takes more time and requires more computation.

Materials and methods

Key bone grades and locations were determined using three real-time target detection models and Key Bone Search (KBS) post-processing using the RUS-CHN approach, and then the age of the bones was predicted using a Lightgbm regression model. Intersection over Union (IOU) was used to evaluate the precision of the key bone locations, while the mean absolute error (MAE), the root mean square error (RMSE), and the root mean squared percentage error (RMSPE) were used to evaluate the discrepancy between predicted and true bone age. The model was finally transformed into an Open Neural Network Exchange (ONNX) model and tested for inference speed on the GPU (RTX 3060).

Results

The three real-time models achieved good results with an average (IOU) of no less than 0.9 in all key bones. The most accurate outcomes for the inference results utilizing KBS were a MAE of 0.35 years, a RMSE of 0.46 years, and a RMSPE of 0.11. Using the GPU RTX3060 for inference, the critical bone level and position inference time was 26 ms. The bone age inference time was 2 ms.

Conclusions

We developed an automated end-to-end BAA system that is based on real-time target detection, obtaining key bone developmental grade and location in a single pass with the aid of KBS, and using Lightgbm to obtain bone age, capable of outputting results in real-time with good accuracy and stability, and able to be used without hand-shaped segmentation. The BAA system automatically implements the entire process of the RUS-CHN method and outputs information on the location and developmental grade of the 13 key bones of the RUS-CHN method along with the bone age to assist the physician in making judgments, making full use of clinical a priori knowledge.