Development and validation of deep learning algorithms for scoliosis screening using back images

Applications of artificial intelligence for adolescent idiopathic scoliosis: mapping the evidence

PURPOSE

Adolescent idiopathic scoliosis (AIS) is a common spinal deformity with varying progression, complicating treatment decisions. Artificial intelligence (AI) and machine learning (ML) are increasingly prominent in orthopedic care, aiding in diagnosis, risk-stratification, and treatment guidance. This scoping review outlines AI applications in AIS.

METHODS

This study followed PRISMA-ScR guidelines and included articles that reported the development, use, or validation of AI models for treating, diagnosing, or predicting clinical outcomes in AIS.

RESULTS

40 full-text articles were included, with most studies published in the last 5 years (77.5%). Common ML techniques were convolutional neural networks (55%), decision trees and random forests (15%), and artificial neural networks (15%). Most AI applications in AIS were for imaging analysis (25/40; 62.5%), focusing on automatic measurement of Cobb angle, and axial vertebral rotation (13/25; 52%) and curve classification/severity (13/25; 52%). Prediction was the second most common application (15/40; 37.5%), with studies predicting curve progression (9/15; 60%), and Cobb angles (9/15; 60%). Only 15 studies (37.5%) reported clinical implementation guidelines for AI in AIS management. 52.5% of studies reported model accuracy, with an average of 85.4%.

CONCLUSION

This review highlights the applications of AI in AIS care, notably including automatic radiographic analysis, curve type classification, prediction of curve progression, and AIS diagnosis. However, the current lack of clear clinical implementation guidelines, model transparency, and external validation of studied models limits clinician trust and the generalizability and applicability of AI in AIS management.

Deep learning prediction of curve severity from rasterstereographic back images in adolescent idiopathic scoliosis

PURPOSE

Radiation-free systems based on dorsal surface topography can potentially represent an alternative to radiographic examination for early screening of scoliosis, based on the ability of recognizing the presence of deformity or classifying its severity. This study aims to assess the effectiveness of a deep learning model based on convolutional neural networks in directly predicting the Cobb angle from rasterstereographic images of the back surface in subjects with adolescent idiopathic scoliosis.

METHODS

Two datasets, comprising a total of 900 individuals, were utilized for model training (720 samples) and testing (180). Rasterstereographic scans were performed using the Formetric4D device. The true Cobb angle was obtained from radiographic examination. The best model configuration was identified by comparing different network architectures and hyperparameters through cross-validation in the training set. The performance of the developed model in predicting the Cobb angle was assessed on the test set. The accuracy in classifying scoliosis severity (non-scoliotic, mild, and moderate category) based on Cobb angle was evaluated as well.

RESULTS

The mean absolute error in predicting the Cobb angle was 6.1° ± 5.0°. Moderate correlation (r = 0.68) and a root-mean-square error of 8° between the predicted and true values was reported. The overall accuracy in classifying scoliosis severity was 59%.

CONCLUSION

Despite some improvement over previous approaches that relied on spine shape reconstruction, the performance of the present fully automatic application is below that of radiographic evaluation performed by human operators. The study confirms that rasterstereography cannot be considered a valid non-invasive alternative to radiographic examination for clinical purposes.

A New Method for Scoliosis Screening Incorporating Deep Learning With Back Images

STUDY DESIGN

Retrospective observational study.

OBJECTIVES

Scoliosis is commonly observed in adolescents, with a world0wide prevalence of 0.5%. It is prone to be overlooked by parents during its early stages, as it often lacks overt characteristics. As a result, many individuals are not aware that they may have scoliosis until the symptoms become quite severe, significantly affecting the physical and mental well-being of patients. Traditional screening methods for scoliosis demand significant physician effort and require unnecessary radiography exposure; thus, implementing large-scale screening is challenging. The application of deep learning algorithms has the potential to reduce unnecessary radiation risks as well as the costs of scoliosis screening.

METHODS

The data of 247 scoliosis patients observed between 2008 and 2021 were used for training. The dataset included frontal, lateral, and back upright images as well as X-ray images obtained during the same period. We proposed and validated deep learning algorithms for automated scoliosis screening using upright back images. The overall process involved the localization of the back region of interest (ROI), spinal region segmentation, and Cobb angle measurements.

RESULTS

The results indicated that the accuracy of the Cobb angle measurement was superior to that of the traditional human visual recognition method, providing a concise and convenient scoliosis screening capability without causing any harm to the human body.

CONCLUSIONS

The method was automated, accurate, concise, and convenient. It is potentially applicable to a wide range of screening methods for the detection of early scoliosis.

Conditional generative adversarial network-assisted system for radiation-free evaluation of scoliosis using a single smartphone photograph: a model development and validation study

Background

Adolescent idiopathic scoliosis (AIS) is the most common spinal disorder in children, characterized by insidious onset and rapid progression, which can lead to severe consequences if not detected in a timely manner. Currently, the diagnosis of AIS primarily relies on X-ray imaging. However, due to limitations in healthcare access and concerns over radiation exposure, this diagnostic method cannot be widely adopted. Therefore, we have developed and validated a screening system using deep learning technology, capable of generating virtual X-ray images (VXI) from two-dimensional Red Green Blue (2D-RGB) images captured by a smartphone or camera to assist spine surgeons in the rapid, accurate, and non-invasive assessment of AIS.

Methods

We included 2397 patients with AIS and 48 potential patients with AIS who visited four medical institutions in mainland China from June 11th 2014 to November 28th 2023. Participants data included standing full-spine X-ray images captured by radiology technicians and 2D-RGB images taken by spine surgeons using a camera. We developed a deep learning model based on conditional generative adversarial networks (cGAN) called Swin-pix2pix to generate VXI on retrospective training (n = 1842) and validation (n = 100) dataset, then validated the performance of VXI in quantifying the curve type and severity of AIS on retrospective internal (n = 100), external (n = 135), and prospective test datasets (n = 268). The prospective test dataset included 268 participants treated in Nanjing, China, from April 19th, 2023, to November 28th, 2023, comprising 220 patients with AIS and 48 potential patients with AIS. Their data underwent strict quality control to ensure optimal data quality and consistency.

Findings

Our Swin-pix2pix model generated realistic VXI, with the mean absolute error (MAE) for predicting the main and secondary Cobb angles of AIS significantly lower than other baseline cGAN models, at 3.2° and 3.1° on prospective test dataset. The diagnostic accuracy for scoliosis severity grading exceeded that of two spine surgery experts, with accuracy of 0.93 (95% CI [0.91, 0.95]) in main curve and 0.89 (95% CI [0.87, 0.91]) in secondary curve. For main curve position and curve classification, the predictive accuracy of the Swin-pix2pix model also surpassed that of the baseline cGAN models, with accuracy of 0.93 (95% CI [0.90, 0.95]) for thoracic curve and 0.97 (95% CI [0.96, 0.98]), achieving satisfactory results on three external datasets as well.

Interpretation

Our developed Swin-pix2pix model holds promise for using a single photo taken with a smartphone or camera to rapidly assess AIS curve type and severity without radiation, enabling large-scale screening. However, limited data quality and quantity, a homogeneous participant population, and rotational errors during imaging may affect the applicability and accuracy of the system, requiring further improvement in the future.

Funding

National Key R&D Program of China, Natural Science Foundation of Jiangsu Province, China Postdoctoral Science Foundation, Nanjing Medical Science and Technology Development Foundation, Jiangsu Provincial Key Research and Development Program, and Jiangsu Provincial Medical Innovation Centre of Orthopedic Surgery.

Heatmap-Based Active Shape Model for Landmark Detection in Lumbar X-ray Images

Medical staff inspect lumbar X-ray images to diagnose lumbar spine diseases, and the analysis process is currently automated using deep-learning techniques. The detection of landmarks is necessary in the automatic process of localizing the position and identifying the morphological features of the vertebrae. However, detection errors may occur owing to the noise and ambiguity of images, as well as individual variations in the shape of the lumbar vertebrae. This study proposes a method to improve the robustness of landmark detection results. This method assumes that landmarks are detected by a convolutional neural network-based two-step model consisting of Pose-Net and M-Net. The model generates a heatmap response to indicate the probable landmark positions. The proposed method then corrects the landmark positions using the heatmap response and active shape model, which employs statistical information on the landmark distribution. Experiments were conducted using 3600 lumbar X-ray images, and the results showed that the landmark detection error was reduced by the proposed method. The average value of maximum errors decreased by 5.58% after applying the proposed method, which combines the outstanding image analysis capabilities of deep learning with statistical shape constraints on landmark distribution. The proposed method could also be easily integrated with other techniques to increase the robustness of landmark detection results such as CoordConv layers and non-directional part affinity field. This resulted in a further enhancement in the landmark detection performance. These advantages can improve the reliability of automatic systems used to inspect lumbar X-ray images. This will benefit both patients and medical staff by reducing medical expenses and increasing diagnostic efficiency.

Explainable Deep-Learning-Based Gait Analysis of Hip-Knee Cyclogram for the Prediction of Adolescent Idiopathic Scoliosis Progression

Accurate prediction of scoliotic curve progression is crucial for guiding treatment decisions in adolescent idiopathic scoliosis (AIS). Traditional methods of assessing the likelihood of AIS progression are limited by variability and rely on static measurements. This study developed and validated machine learning models for classifying progressive and non-progressive scoliotic curves based on gait analysis using wearable inertial sensors. Gait data from 38 AIS patients were collected using seven inertial measurement unit (IMU) sensors, and hip-knee (HK) cyclograms representing inter-joint coordination were generated. Various machine learning algorithms, including support vector machine (SVM), random forest (RF), and novel deep convolutional neural network (DCNN) models utilizing multi-plane HK cyclograms, were developed and evaluated using 10-fold cross-validation. The DCNN model incorporating multi-plane HK cyclograms and clinical factors achieved an accuracy of 92% in predicting curve progression, outperforming SVM (55% accuracy) and RF (52% accuracy) models using handcrafted gait features. Gradient-based class activation mapping revealed that the DCNN model focused on the swing phase of the gait cycle to make predictions. This study demonstrates the potential of deep learning techniques, and DCNNs in particular, in accurately classifying scoliotic curve progression using gait data from wearable IMU sensors.

Development and validation of an artificial intelligence model to accurately predict spinopelvic parameters

OBJECTIVE

Achieving appropriate spinopelvic alignment has been shown to be associated with improved clinical symptoms. However, measurement of spinopelvic radiographic parameters is time-intensive and interobserver reliability is a concern. Automated measurement tools have the promise of rapid and consistent measurements, but existing tools are still limited to some degree by manual user-entry requirements. This study presents a novel artificial intelligence (AI) tool called SpinePose that automatically predicts spinopelvic parameters with high accuracy without the need for manual entry.

METHODS

SpinePose was trained and validated on 761 sagittal whole-spine radiographs to predict the sagittal vertical axis (SVA), pelvic tilt (PT), pelvic incidence (PI), sacral slope (SS), lumbar lordosis (LL), T1 pelvic angle (T1PA), and L1 pelvic angle (L1PA). A separate test set of 40 radiographs was labeled by four reviewers, including fellowship-trained spine surgeons and a fellowship-trained radiologist with neuroradiology subspecialty certification. Median errors relative to the most senior reviewer were calculated to determine model accuracy on test images. Intraclass correlation coefficients (ICCs) were used to assess interrater reliability.

RESULTS

SpinePose exhibited the following median (interquartile range) parameter errors: SVA 2.2 mm (2.3 mm) (p = 0.93), PT 1.3° (1.2°) (p = 0.48), SS 1.7° (2.2°) (p = 0.64), PI 2.2° (2.1°) (p = 0.24), LL 2.6° (4.0°) (p = 0.89), T1PA 1.1° (0.9°) (p = 0.42), and L1PA 1.4° (1.6°) (p = 0.49). Model predictions also exhibited excellent reliability at all parameters (ICC 0.91-1.0).

CONCLUSIONS

SpinePose accurately predicted spinopelvic parameters with excellent reliability comparable to that of fellowship-trained spine surgeons and neuroradiologists. Utilization of predictive AI tools in spinal imaging can substantially aid in patient selection and surgical planning.

Deep Learning-Based Prediction Model for the Cobb Angle in Adolescent Idiopathic Scoliosis Patients

Scoliosis, characterized by spine deformity, is most common in adolescent idiopathic scoliosis (AIS). Manual Cobb angle measurement limitations underscore the need for automated tools. This study employed a vertebral landmark extraction method and Feedforward Neural Network (FNN) to predict scoliosis progression in 79 AIS patients. The novel intervertebral angles matrix format showcased results. The mean absolute error for the intervertebral angle progression was 1.5 degrees, while the Pearson correlation of the predicted Cobb angles was 0.86. The accuracy in classifying Cobb angles (<15°, 15-25°, 25-35°, 35-45°, >45°) was 0.85, with 0.65 sensitivity and 0.91 specificity. The FNN demonstrated superior accuracy, sensitivity, and specificity, aiding in tailored treatments for potential scoliosis progression. Addressing FNNs' over-fitting issue through strategies like "dropout" or regularization could further enhance their performance. This study presents a promising step towards automated scoliosis diagnosis and prognosis.

Deep learning system for screening AIDS-related cytomegalovirus retinitis with ultra-wide-field fundus images

Background

Ophthalmological screening for cytomegalovirus retinitis (CMVR) for HIV/AIDS patients is important to prevent lifelong blindness. Previous studies have shown good properties of automated CMVR screening using digital fundus images. However, the application of a deep learning (DL) system to CMVR with ultra-wide-field (UWF) fundus images has not been studied, and the feasibility and efficiency of this method are uncertain.

Methods

In this study, we developed, internally validated, externally validated, and prospectively validated a DL system to detect AIDS-related from UWF fundus images from different clinical datasets. We independently used the InceptionResnetV2 network to develop and internally validate a DL system for identifying active CMVR, inactive CMVR, and non-CMVR in 6960 UWF fundus images from 862 AIDS patients and validated the system in a prospective and an external validation data set using the area under the curve (AUC), accuracy, sensitivity, and specificity. A heat map identified the most important area (lesions) used by the DL system for differentiating CMVR.

Results

The DL system showed AUCs of 0.945 (95 % confidence interval [CI]: 0.929, 0.962), 0.964 (95 % CI: 0.870, 0.999) and 0.968 (95 % CI: 0.860, 1.000) for detecting active CMVR from non-CMVR and 0.923 (95 % CI: 0.908, 0.938), 0.902 (0.857, 0.948) and 0.884 (0.851, 0.917) for detecting active CMVR from non-CMVR in the internal cross-validation, external validation, and prospective validation, respectively. Deep learning performed promisingly in screening CMVR. It also showed the ability to differentiate active CMVR from non-CMVR and inactive CMVR as well as to identify active CMVR and inactive CMVR from non-CMVR (all AUCs in the three independent data sets >0.900). The heat maps successfully highlighted lesion locations.

Conclusions

Our UWF fundus image-based DL system showed reliable performance for screening AIDS-related CMVR showing its potential for screening CMVR in HIV/AIDS patients, especially in the absence of ophthalmic resources.

The Application of Surface Electromyography Technology in Evaluating Paraspinal Muscle Function

Surface electromyography (sEMG) has emerged as a valuable tool for assessing muscle activity in various clinical and research settings. This review focuses on the application of sEMG specifically in the context of paraspinal muscles. The paraspinal muscles play a critical role in providing stability and facilitating movement of the spine. Dysfunctions or alterations in paraspinal muscle activity can lead to various musculoskeletal disorders and spinal pathologies. Therefore, understanding and quantifying paraspinal muscle activity is crucial for accurate diagnosis, treatment planning, and monitoring therapeutic interventions. This review discusses the clinical applications of sEMG in paraspinal muscles, including the assessment of low back pain, spinal disorders, and rehabilitation interventions. It explores how sEMG can aid in diagnosing the potential causes of low back pain and monitoring the effectiveness of physical therapy, spinal manipulative therapy, and exercise protocols. It also discusses emerging technologies and advancements in sEMG techniques that aim to enhance the accuracy and reliability of paraspinal muscle assessment. In summary, the application of sEMG in paraspinal muscles provides valuable insights into muscle function, dysfunction, and therapeutic interventions. By examining the literature on sEMG in paraspinal muscles, this review offers a comprehensive understanding of the current state of research, identifies knowledge gaps, and suggests future directions for optimizing the use of sEMG in assessing paraspinal muscle activity.

Adolescent idiopathic scoliosis: treatment outcomes, quality of life and implications for practice

Adolescent idiopathic scoliosis (AIS) is the most common spinal disorder among children and adolescents, with most cases being diagnosed around puberty. While the majority of people with AIS do not undergo treatment, a small but significant number are treated, depending on the extent of their spinal curvature. Treatment typically involves bracing, which requires substantial adherence, and/or surgery, which is invasive and permanent. Furthermore, decisions about treatment often need to be made at a critical stage of the person's development. This article examines the evidence on AIS and its treatment, synthesising the current literature and drawing from the authors' empirical work to explore the clinical outcomes of bracing and surgery, as well as the longer-term effects on people's quality of life. Drawing from this evidence, the authors provide guidance for nurses and healthcare professionals who care for people with AIS.

Deep learning-based identification of spine growth potential on EOS radiographs

OBJECTIVES

To develop an automatic computer-based method that can help clinicians in assessing spine growth potential based on EOS radiographs.

METHODS

We developed a deep learning-based (DL) algorithm that can mimic the human judgment process to automatically determine spine growth potential and the Risser sign based on full-length spine EOS radiographs. A total of 3383 EOS cases were collected and used for the training and test of the algorithm. Subsequently, the completed DL algorithm underwent clinical validation on an additional 440 cases and was compared to the evaluations of four clinicians.

RESULTS

Regarding the Risser sign, the weighted kappa value of our DL algorithm was 0.933, while that of the four clinicians ranged from 0.909 to 0.930. In the assessment of spine growth potential, the kappa value of our DL algorithm was 0.944, while the kappa values of the four clinicians were 0.916, 0.934, 0.911, and 0.920, respectively. Furthermore, our DL algorithm obtained a slightly higher accuracy (0.973) and Youden index (0.952) compared to the best values achieved by the four clinicians. In addition, the speed of our DL algorithm was 15.2 ± 0.3 s/40 cases, much faster than the inference speeds of the clinicians, ranging from 177.2 ± 28.0 s/40 cases to 241.2 ± 64.1 s/40 cases.

CONCLUSIONS

Our algorithm demonstrated comparable or even better performance compared to clinicians in assessing spine growth potential. This stable, efficient, and convenient algorithm seems to be a promising approach to assist doctors in clinical practice and deserves further study.

CLINICAL RELEVANCE STATEMENT

This method has the ability to quickly ascertain the spine growth potential based on EOS radiographs, and it holds promise to provide assistance to busy doctors in certain clinical scenarios.

KEY POINTS

• In the clinic, there is no available computer-based method that can automatically assess spine growth potential. • We developed a deep learning-based method that could automatically ascertain spine growth potential. • Compared with the results of the clinicians, our algorithm got comparable results.

Cost-effectiveness and cost-utility of a digital technology-driven hierarchical healthcare screening pattern in China

Utilization of digital technologies for cataract screening in primary care is a potential solution for addressing the dilemma between the growing aging population and unequally distributed resources. Here, we propose a digital technology-driven hierarchical screening (DH screening) pattern implemented in China to promote the equity and accessibility of healthcare. It consists of home-based mobile artificial intelligence (AI) screening, community-based AI diagnosis, and referral to hospitals. We utilize decision-analytic Markov models to evaluate the cost-effectiveness and cost-utility of different cataract screening strategies (no screening, telescreening, AI screening and DH screening). A simulated cohort of 100,000 individuals from age 50 is built through a total of 30 1-year Markov cycles. The primary outcomes are incremental cost-effectiveness ratio and incremental cost-utility ratio. The results show that DH screening dominates no screening, telescreening and AI screening in urban and rural China. Annual DH screening emerges as the most economically effective strategy with 341 (338 to 344) and 1326 (1312 to 1340) years of blindness avoided compared with telescreening, and 37 (35 to 39) and 140 (131 to 148) years compared with AI screening in urban and rural settings, respectively. The findings remain robust across all sensitivity analyses conducted. Here, we report that DH screening is cost-effective in urban and rural China, and the annual screening proves to be the most cost-effective option, providing an economic rationale for policymakers promoting public eye health in low- and middle-income countries.

Monitoring of Curve Progression in Patients with Adolescent Idiopathic Scoliosis Using 3-D Ultrasound

OBJECTIVE

The aim of the work described here was to determine whether 3-D ultrasound can provide results comparable to those of conventional X-ray examination in assessing curve progression in patients with adolescent idiopathic scoliosis (AIS).

METHODS

One hundred thirty-six participants with AIS (42 males and 94 females; age range: 10-18 y, mean age: 14.1 ± 1.9 y) with scoliosis of different severity (Cobb angle range: 10º- 85º, mean: of 24.3 ± 14.4º) were included. Each participant underwent biplanar low-dose X-ray EOS and 3-D ultrasound system scanning with the same posture on the same date. Participants underwent the second assessment at routine clinical follow-up. Manual measurements of scoliotic curvature on ultrasound coronal projection images and posterior-anterior radiographs were expressed as the ultrasound curve angle (UCA) and radiographic Cobb angle (RCA), respectively. RCA and UCA increments ≥5º represented a scoliosis progression detected by X-ray assessment and 3-D ultrasound assessment, respectively.

RESULTS

The sensitivity and specificity of UCA measurement in detecting scoliosis progression were 0.93 and 0.90, respectively. The negative likelihood ratio of the diagnostic test for scoliosis progression by the 3-D ultrasound imaging system was 0.08.

CONCLUSION

The 3-D ultrasound imaging method is a valid technique for detecting coronal curve progression as compared with conventional radiography in follow-up of AIS. Substituting conventional radiography with 3-D ultrasound is effective in reducing the radiation dose to which AIS patients are exposed during their follow-up examinations.

Promises, Pitfalls, and Clinical Applications of Artificial Intelligence in Pediatrics

Artificial intelligence (AI) broadly describes a branch of computer science focused on developing machines capable of performing tasks typically associated with human intelligence. Those who connect AI with the world of science fiction may meet its growing rise with hesitancy or outright skepticism. However, AI is becoming increasingly pervasive in our society, from algorithms helping to sift through airline fares to substituting words in emails and SMS text messages based on user choices. Data collection is ongoing and is being leveraged by software platforms to analyze patterns and make predictions across multiple industries. Health care is gradually becoming part of this technological transformation, as advancements in computational power and storage converge with the rapid expansion of digitized medical information. Given the growing and inevitable integration of AI into health care systems, it is our viewpoint that pediatricians urgently require training and orientation to the uses, promises, and pitfalls of AI in medicine. AI is unlikely to solve the full array of complex challenges confronting pediatricians today; however, if used responsibly, it holds great potential to improve many aspects of care for providers, children, and families. Our aim in this viewpoint is to provide clinicians with a targeted introduction to the field of AI in pediatrics, including key promises, pitfalls, and clinical applications, so they can play a more active role in shaping the future impact of AI in medicine.

Scoliosis school screening of 139,922 multi-ethnic children in Dali, southwestern China: A large epidemiological study

Idiopathic scoliosis (IS) primarily impacts adolescents and requires early intervention to prevent deformity. Early diagnosis and prediction of spine curvature in children could be aided by school scoliosis screening (SSS). In the Dali Bai Autonomous Prefecture, SSS, including 139,922 children from 18 ethnic groups in 8 counties ranging in age from 6 to 18, was carried out. A medical team conducted the screening with inspection, Adam's test, and angles of trunk rotation (ATR). The overall prevalence of suspected scoliosis was 2.37%, with girls (2.5%) more affected than boys (2.0%). Using penalized regression analysis of LASSO, the variable-selection process was conducted to determine the final regression model. The results showed that age, gender, height, BMI, altitude, latitude, ethnicity, and county were all influencing variables for suspected scoliosis, according to the adjusted final model of multi-factor regression analysis. These results provide substantial information and suggestions for preventative and person-centered healthcare interventions for IS.

Spine Deformity Assessment for Scoliosis Diagnostics Utilizing Image Processing Techniques: A Systematic Review

Spinal deformity refers to a range of disorders that are defined by anomalous curvature of the spine and may be classified as scoliosis, hypo/hyperlordosis, or hypo/hyperkyphosis. Among these, scoliosis stands out as the most common type of spinal deformity in human beings, and it can be distinguished by abnormal lateral spine curvature accompanied by axial rotation. Accurate identification of spinal deformity is crucial for a person’s diagnosis, and numerous assessment methods have been developed by researchers. Therefore, the present study aims to systematically review the recent works on spinal deformity assessment for scoliosis diagnosis utilizing image processing techniques. To gather relevant studies, a search strategy was conducted on three electronic databases (Scopus, ScienceDirect, and PubMed) between 2012 and 2022 using specific keywords and focusing on scoliosis cases. A total of 17 papers fully satisfied the established criteria and were extensively evaluated. Despite variations in methodological designs across the studies, all reviewed articles obtained quality ratings higher than satisfactory. Various diagnostic approaches have been employed, including artificial intelligence mechanisms, image processing, and scoliosis diagnosis systems. These approaches have the potential to save time and, more significantly, can reduce the incidence of human error. While all assessment methods have potential in scoliosis diagnosis, they possess several limitations that can be ameliorated in forthcoming studies. Therefore, the findings of this study may serve as guidelines for the development of a more accurate spinal deformity assessment method that can aid medical personnel in the real diagnosis of scoliosis.

The use of deep learning in medical imaging to improve spine care: A scoping review of current literature and clinical applications

Background

Artificial intelligence is a revolutionary technology that promises to assist clinicians in improving patient care. In radiology, deep learning (DL) is widely used in clinical decision aids due to its ability to analyze complex patterns and images. It allows for rapid, enhanced data, and imaging analysis, from diagnosis to outcome prediction. The purpose of this study was to evaluate the current literature and clinical utilization of DL in spine imaging.

Methods

This study is a scoping review and utilized the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodology to review the scientific literature from 2012 to 2021. A search in PubMed, Web of Science, Embased, and IEEE Xplore databases with syntax specific for DL and medical imaging in spine care applications was conducted to collect all original publications on the subject. Specific data was extracted from the available literature, including algorithm application, algorithms tested, database type and size, algorithm training method, and outcome of interest.

Results

A total of 365 studies (total sample of 232,394 patients) were included and grouped into 4 general applications: diagnostic tools, clinical decision support tools, automated clinical/instrumentation assessment, and clinical outcome prediction. Notable disparities exist in the selected algorithms and the training across multiple disparate databases. The most frequently used algorithms were U-Net and ResNet. A DL model was developed and validated in 92% of included studies, while a pre-existing DL model was investigated in 8%. Of all developed models, only 15% of them have been externally validated.

Conclusions

Based on this scoping review, DL in spine imaging is used in a broad range of clinical applications, particularly for diagnosing spinal conditions. There is a wide variety of DL algorithms, database characteristics, and training methods. Future studies should focus on external validation of existing models before bringing them into clinical use.

Is Shoulder Imbalance a Useful Parameter in the Screening of Idiopathic Scoliosis? A Preliminary Study

Objective  The present study aims to analyze the use of shoulder imbalance as a parameter for scoliosis screening as well as its relationship with other parameters of physical examination. Methods  This study assesses a smartphone application that analyzes several parameters of the physical examination in adolescent idiopathic scoliosis. Medical and non-medical examiners applied the screening tool in students in a public school and in a private sports club. After data collection, interobserver correlation was done to verify shoulder imbalance and to compare shoulder imbalance with Adam's bending test and with trunk rotation. Results  Eighty-nine participants were examined, 18 of whom were women and 71 of whom were men. Two subjects were excluded from the analysis. The mean age of subjects from the public school was 11.30 years and, for those from the sports club, it was 11.92 years. The examiners had poor-to-slight interobserver concordance on shoulder asymmetry in the anterior and posterior view. No significant statistical correlation was found between shoulder asymmetry and positive Adam's forward bending test. Conclusion  Our preliminary study shows that the shoulder asymmetry has a poor correlation with the Adam's forward bending test and measuring trunk rotation using a scoliometer. Therefore, the use of shoulder imbalance might not be useful for idiopathic scoliosis screening. Level of Evidence III; Diagnostic Study.

Deep Learning Model to Classify and Monitor Idiopathic Scoliosis in Adolescents Using a Single Smartphone Photograph

Importance

Adolescent idiopathic scoliosis (AIS) is the most common pediatric spinal disorder. Routine physical examinations by trained personnel are critical to diagnose severity and monitor curve progression in AIS. In the presence of concerning malformation, radiographs are necessary for diagnosis or follow-up, guiding further management, such as bracing correction for moderate malformation and spine surgery for severe malformation. If left unattended, progressive deterioration occurs in two-thirds of patients, leading to significant health concerns for growing children.

Objective

To assess the ability of an open platform application (app) using a validated deep learning model to classify AIS severity and curve type, as well as identify progression.

Design, Setting, and Participants

This diagnostic study was performed with data from radiographs and smartphone photographs of the backs of adolescent patients at spine clinics. The ScolioNets deep learning model was developed and validated in a prospective training cohort, then incorporated and tested in the AlignProCARE open platform app in 2022. Ground truths (GTs) included severity, curve type, and progression as manually annotated by 2 experienced spine specialists based on the radiographic examinations of the participants' spines. The GTs and app results were blindly compared with another 2 spine surgeons' assessments of unclothed back appearance. Data were analyzed from October 2022 to February 2023.

Exposure

Acquisitions of unclothed back photographs using a mobile app.

Main Outcomes and Measures

Outcomes of interest were classification of AIS severity and progression. Quantitative statistical analyses were performed to assess the performance of the deep learning model in classifying the deformity as well as in distinguishing progression during 6-month follow-up.

Results

The training data set consisted of 1780 patients (1295 [72.8%] female; mean [SD] age, 14.3 [3.3] years), and the prospective testing data sets consisted of 378 patients (279 [73.8%] female; mean [SD] age, 14.3 [3.8] years) and 376 follow-ups (294 [78.2%] female; mean [SD] age, 15.6 [2.9] years). The model recommended follow-up with an area under receiver operating characteristic curve (AUC) of 0.839 (95% CI, 0.789-0.882) and considering surgery with an AUC of 0.902 (95% CI, 0.859-0.936), while showing good ability to distinguish among thoracic (AUC, 0.777 [95% CI, 0.745-0.808]), thoracolumbar or lumbar (AUC, 0.760 [95% CI, 0.727-0.791]), or mixed (AUC, 0.860 [95% CI, 0.834-0.887]) curve types. For follow-ups, the model distinguished participants with or without curve progression with an AUC of 0.757 (95% CI, 0.630-0.858). Compared with both surgeons, the model could recognize severities and curve types with a higher sensitivity (eg, sensitivity for recommending follow-up: model, 84.88% [95% CI, 75.54%-91.70%]; senior surgeon, 44.19%; junior surgeon, 62.79%) and negative predictive values (NPVs; eg, NPV for recommending follow-up: model, 89.22% [95% CI, 84.25%-93.70%]; senior surgeon, 71.76%; junior surgeon, 79.35%). For distinguishing curve progression, the sensitivity and NPV were comparable with the senior surgeons (sensitivity, 63.33% [95% CI, 43.86%-80.87%] vs 77.42%; NPV, 68.57% [95% CI, 56.78%-78.37%] vs 72.00%). The junior surgeon reported an inability to identify curve types and progression by observing the unclothed back alone.

Conclusions

This diagnostic study of adolescent patients screened for AIS found that the deep learning app had the potential for out-of-hospital accessible and radiation-free management of children with scoliosis, with comparable performance as spine surgeons experienced in AIS management.

A Neural Network Approach for Inertial Measurement Unit-Based Estimation of Three-Dimensional Spinal Curvature

The spine is an important part of the human body. Thus, its curvature and shape are closely monitored, and treatment is required if abnormalities are detected. However, the current method of spinal examination mostly relies on two-dimensional static imaging, which does not provide real-time information on dynamic spinal behaviour. Therefore, this study explored an easier and more efficient method based on machine learning and sensors to determine the curvature of the spine. Fifteen participants were recruited and performed tests to generate data for training a neural network. This estimated the spinal curvature from the readings of three inertial measurement units and had an average absolute error of 0.261161 cm.

Radiograph-comparable image synthesis for spine alignment analysis using deep learning with prospective clinical validation

Background

Adolescent idiopathic scoliosis (AIS) is the most common type of spinal disorder affecting children. Clinical screening and diagnosis require physical and radiographic examinations, which are either subjective or increase radiation exposure. We therefore developed and validated a radiation-free portable system and device utilising light-based depth sensing and deep learning technologies to analyse AIS by landmark detection and image synthesis.

Methods

Consecutive patients with AIS attending two local scoliosis clinics in Hong Kong between October 9, 2019, and May 21, 2022, were recruited. Patients were excluded if they had psychological and/or systematic neural disorders that could influence the compliance of the study and/or the mobility of the patients. For each participant, a Red Green Blue-Depth (RGBD) image of the nude back was collected using our in-house radiation-free device. Manually labelled landmarks and alignment parameters by our spine surgeons were considered as the ground truth (GT). Images from training and internal validation cohorts (n = 1936) were used to develop the deep learning models. The model was then prospectively validated on another cohort (n = 302) which was collected in Hong Kong and had the same demographic properties as the training cohort. We evaluated the prediction accuracy of the model on nude back landmark detection as well as the performance on radiograph-comparable image (RCI) synthesis. The obtained RCIs contain sufficient anatomical information that can quantify disease severities and curve types.

Findings

Our model had a consistently high accuracy in predicting the nude back anatomical landmarks with a less than 4-pixel error regarding the mean Euclidian and Manhattan distance. The synthesized RCI for AIS severity classification achieved a sensitivity and negative predictive value of over 0.909 and 0.933, and the performance for curve type classification was 0.974 and 0.908, with spine specialists' manual assessment results on real radiographs as GT. The estimated Cobb angle from synthesized RCIs had a strong correlation with the GT angles (R² = 0.984, p < 0.001).

Interpretation

The radiation-free medical device powered by depth sensing and deep learning techniques can provide instantaneous and harmless spine alignment analysis which has the potential for integration into routine screening for adolescents.

Funding

Innovation and Technology Fund (MRP/038/20X), Health Services Research Fund (HMRF) 08192266.

Screening of adolescent idiopathic scoliosis using generative adversarial network (GAN) inversion method in chest radiographs

OBJECTIVE

Conventional computer-aided diagnosis using convolutional neural networks (CNN) has limitations in detecting sensitive changes and determining accurate decision boundaries in spectral and structural diseases such as scoliosis. We devised a new method to detect and diagnose adolescent idiopathic scoliosis in chest X-rays (CXRs) employing the latent space's discriminative ability in the generative adversarial network (GAN) and a simple multi-layer perceptron (MLP) to screen adolescent idiopathic scoliosis CXRs.

MATERIALS AND METHODS

Our model was trained and validated in a two-step manner. First, we trained a GAN using CXRs with various scoliosis severities and utilized the trained network as a feature extractor using the GAN inversion method. Second, we classified each vector from the latent space using a simple MLP.

RESULTS

The 2-layer MLP exhibited the best classification in the ablation study. With this model, the area under the receiver operating characteristic (AUROC) curves were 0.850 in the internal and 0.847 in the external datasets. Furthermore, when the sensitivity was fixed at 0.9, the model's specificity was 0.697 in the internal and 0.646 in the external datasets.

CONCLUSION

We developed a classifier for Adolescent idiopathic scoliosis (AIS) through generative representation learning. Our model shows good AUROC under screening chest radiographs in both the internal and external datasets. Our model has learned the spectral severity of AIS, enabling it to generate normal images even when trained solely on scoliosis radiographs.

Prediction of Cobb Angle Using Deep Learning Algorithm with Three-Dimensional Depth Sensor Considering the Influence of Garment in Idiopathic Scoliosis

Adolescent idiopathic scoliosis (AIS) is the most common pediatric spinal deformity. Early detection of deformity and timely intervention, such as brace treatment, can help inhibit progressive changes. A three-dimensional (3D) depth-sensor imaging system with a convolutional neural network was previously developed to predict the Cobb angle. The purpose of the present study was to (1) evaluate the performance of the deep learning algorithm (DLA) in predicting the Cobb angle and (2) assess the predictive ability depending on the presence or absence of clothing in a prospective analysis. We included 100 subjects with suspected AIS. The correlation coefficient between the actual and predicted Cobb angles was 0.87, and the mean absolute error and root mean square error were 4.7° and 6.0°, respectively, for Adam's forward bending without underwear. There were no significant differences in the correlation coefficients between the groups with and without underwear in the forward-bending posture. The performance of the DLA with a 3D depth sensor was validated using an independent external validation dataset. Because the psychological burden of children and adolescents on naked body imaging is an unignorable problem, scoliosis examination with underwear is a valuable alternative in clinics or schools.

Development of a machine learning algorithm to identify total and reverse shoulder arthroplasty implants from X-ray images

Introduction

Demand for total shoulder arthroplasty (TSA) has risen significantly and is projected to continue growing. From 2012 to 2017, the incidence of reverse total shoulder arthroplasty (rTSA) rose from 7.3 cases per 100,000 to 19.3 per 100,000. Anatomical TSA saw a growth from 9.5 cases per 100,000 to 12.5 per 100,000. Failure to identify implants in a timely manner can increase operative time, cost and risk of complications. Several machine learning models have been developed to perform medical image analysis. However, they have not been widely applied in shoulder surgery. The authors developed a machine learning model to identify shoulder implant manufacturers and type from anterior-posterior X-ray images.

Methods

The model deployed was a convolutional neural network (CNN), which has been widely used in computer vision tasks. 696 radiographs were obtained from a single institution. 70% were used to train the model, while evaluation was done on 30%.

Results

On the evaluation set, the model performed with an overall accuracy of 93.9% with positive predictive value, sensitivity and F-1 scores of 94% across 10 different implant types (4 reverse, 6 anatomical). Average identification time was 0.110 s per implant.

Conclusion

This proof of concept study demonstrates that machine learning can assist with preoperative planning and improve cost-efficiency in shoulder surgery.

Diagnosis of Scoliosis Using Chest Radiographs with a Semi-Supervised Generative Adversarial Network

Purpose

To develop and validate a deep learning-based screening tool for the early diagnosis of scoliosis using chest radiographs with a semi-supervised generative adversarial network (GAN).

Materials and Methods

Using a semi-supervised learning framework with a GAN, a screening tool for diagnosing scoliosis was developed and validated through the chest PA radiographs of patients at two different tertiary hospitals. Our proposed method used training GAN with mild to severe scoliosis only in a semi-supervised manner, as an upstream task to learn scoliosis representations and a downstream task to perform simple classification for differentiating between normal and scoliosis states sensitively.

Results

The area under the receiver operating characteristic curve, negative predictive value (NPV), positive predictive value, sensitivity, and specificity were 0.856, 0.950, 0.579, 0.985, and 0.285, respectively.

Conclusion

Our deep learning-based artificial intelligence software in a semi-supervised manner achieved excellent performance in diagnosing scoliosis using the chest PA radiographs of young individuals; thus, it could be used as a screening tool with high NPV and sensitivity and reduce the burden on radiologists for diagnosing scoliosis through health screening chest radiographs.

The current and future roles of artificial intelligence in pediatric radiology

Artificial intelligence (AI) is a broad and complicated concept that has begun to affect many areas of medicine, perhaps none so much as radiology. While pediatric radiology has been less affected than other radiology subspecialties, there are some well-developed and some nascent applications within the field. This review focuses on the use of AI within pediatric radiology for image interpretation, with descriptive summaries of the literature to date. We highlight common features that enable successful application of the technology, along with some of the limitations that can inhibit the development of this field. We present some ideas for further research in this area and challenges that must be overcome, with an understanding that technology often advances in unpredictable ways.

Current and emerging artificial intelligence applications for pediatric musculoskeletal radiology

Artificial intelligence (AI) is playing an ever-increasing role in radiology (more so in the adult world than in pediatrics), to the extent that there are unfounded fears it will completely take over the role of the radiologist. In relation to musculoskeletal applications of AI in pediatric radiology, we are far from the time when AI will replace radiologists; even for the commonest application (bone age assessment), AI is more often employed in an AI-assist mode rather than an AI-replace or AI-extend mode. AI for bone age assessment has been in clinical use for more than a decade and is the area in which most research has been conducted. Most other potential indications in children (such as appendicular and vertebral fracture detection) remain largely in the research domain. This article reviews the areas in which AI is most prominent in relation to the pediatric musculoskeletal system, briefly summarizing the current literature and highlighting areas for future research. Pediatric radiologists are encouraged to participate as members of the research teams conducting pediatric radiology artificial intelligence research.

Current Applications of Machine Learning in Spine: From Clinical View

STUDY DESIGN

Narrative review.

OBJECTIVES

This review aims to present current applications of machine learning (ML) in spine domain to clinicians.

METHODS

We conducted a comprehensive PubMed search of peer-reviewed articles that were published between 2006 and 2020 using terms (spine, spinal, lumbar, cervical, thoracic, machine learning) to examine ML in spine. Then exclude research of other domain, case report, review or meta-analysis, and which without available abstract or full text.

RESULTS

Total 1738 articles were retrieved from database, and 292 studies were finally included. Key findings of current applications were compiled and summarized in this review. Main clinical applications of those techniques including image processing, diagnosis, decision supporting, operative assistance, rehabilitation, surgery outcomes, complications, hospitalization and cost.

CONCLUSIONS

ML had achieved excellent performance and hold immense potential in spine. ML could help clinical staff to improve medical level, enhance work efficiency, and reduce adverse events. However more randomized controlled trials and improvement of interpretability are essential to clinicians accepting models' assistance in real work.

Deep Learning Multi-label Tongue Image Analysis and Its Application in a Population Undergoing Routine Medical Checkup

Background

Research on intelligent tongue diagnosis is a main direction in the modernization of tongue diagnosis technology. Identification of tongue shape and texture features is a difficult task for tongue diagnosis in traditional Chinese medicine (TCM). This study aimed to explore the application of deep learning techniques in tongue image analyses.

Methods

A total of 8676 tongue images were annotated by clinical experts, into seven categories, including the fissured tongue, tooth-marked tongue, stasis tongue, spotted tongue, greasy coating, peeled coating, and rotten coating. Based on the labeled tongue images, the deep learning model faster region-based convolutional neural networks (Faster R-CNN) was utilized to classify tongue images. Four performance indices, i.e., accuracy, recall, precision, and F1-score, were selected to evaluate the model. Also, we applied it to analyze tongue image features of 3601 medical checkup participants in order to explore gender and age factors and the correlations among tongue features in diseases through complex networks.

Results

The average accuracy, recall, precision, and F1-score of our model achieved 90.67%, 91.25%, 99.28%, and 95.00%, respectively. Over the tongue images from the medical checkup population, the model Faster R-CNN detected 41.49% fissured tongue images, 37.16% tooth-marked tongue images, 29.66% greasy coating images, 18.66% spotted tongue images, 9.97% stasis tongue images, 3.97% peeled coating images, and 1.22% rotten coating images. There were significant differences in the incidence of the fissured tongue, tooth-marked tongue, spotted tongue, and greasy coating among age and gender. Complex networks revealed that fissured tongue and tooth-marked were closely related to hypertension, dyslipidemia, overweight and nonalcoholic fatty liver disease (NAFLD), and a greasy coating tongue was associated with hypertension and overweight.

Conclusion

The model Faster R-CNN shows good performance in the tongue image classification. And we have preliminarily revealed the relationship between tongue features and gender, age, and metabolic diseases in a medical checkup population.

Conformity assessment of a computer vision-based posture analysis system for the screening of postural deformation

Background

This study evaluates the conformity of using a computer vision-based posture analysis system as a screening assessment for postural deformity detection in the spine that is easily applicable to clinical practice.

Methods

One hundred forty participants were enrolled for screening of the postural deformation. Factors that determine the presence or absence of spinal deformation, such as shoulder height difference (SHD), pelvic height difference (PHD), and leg length mismatch (LLD), were used as parameters for the clinical decision support system (CDSS) using a commercial computer vision-based posture analysis system. For conformity analysis, the probability of postural deformation provided by CDSS, the Cobb angle, the PHD, and the SHD was compared and analyzed between the system and radiographic parameters. A principal component analysis (PCA) of the CDSS and correlation analysis were conducted.

Results

The Cobb angles of the 140 participants ranged from 0° to 61°, with an average of 6.16° ± 8.50°. The postural deformation of CDSS showed 94% conformity correlated with radiographic assessment. The conformity assessment results were more accurate in the participants of postural deformation with normal (0–9°) and mild (10–25°) ranges of scoliosis. The referenced SHD and the SHD of the CDSS showed statistical significance (p < 0.001) on a paired t-test. SHD and PHD for PCA were the predominant factors (PC1 SHD for 79.97%, PC2 PHD for 19.86%).

Conclusion

The CDSS showed 94% conformity for the screening of postural spinal deformity. The main factors determining diagnostic suitability were two main variables: SHD and PHD. In conclusion, a computer vision-based posture analysis system can be utilized as a safe, efficient, and convenient CDSS for early diagnosis of spinal posture deformation, including scoliosis.

Current development and prospects of deep learning in spine image analysis: a literature review

BACKGROUND AND OBJECTIVE

As the spine is pivotal in the support and protection of human bodies, much attention is given to the understanding of spinal diseases. Quick, accurate, and automatic analysis of a spine image greatly enhances the efficiency with which spine conditions can be diagnosed. Deep learning (DL) is a representative artificial intelligence technology that has made encouraging progress in the last 6 years. However, it is still difficult for clinicians and technicians to fully understand this rapidly evolving field due to the diversity of applications, network structures, and evaluation criteria. This study aimed to provide clinicians and technicians with a comprehensive understanding of the development and prospects of DL spine image analysis by reviewing published literature.

METHODS

A systematic literature search was conducted in the PubMed and Web of Science databases using the keywords "deep learning" and "spine". Date ranges used to conduct the search were from 1 January, 2015 to 20 March, 2021. A total of 79 English articles were reviewed.

KEY CONTENT AND FINDINGS

The DL technology has been applied extensively to the segmentation, detection, diagnosis, and quantitative evaluation of spine images. It uses static or dynamic image information, as well as local or non-local information. The high accuracy of analysis is comparable to that achieved manually by doctors. However, further exploration is needed in terms of data sharing, functional information, and network interpretability.

CONCLUSIONS

The DL technique is a powerful method for spine image analysis. We believe that, with the joint efforts of researchers and clinicians, intelligent, interpretable, and reliable DL spine analysis methods will be widely applied in clinical practice in the future.

Using deep transfer learning to detect scoliosis and spondylolisthesis from x-ray images

Recent years have witnessed wider prevalence of vertebral column pathologies due to lifestyle changes, sedentary behaviors, or injuries. Spondylolisthesis and scoliosis are two of the most common ailments with an incidence of 5% and 3% in the United States population, respectively. Both of these abnormalities can affect children at a young age and, if left untreated, can progress into severe pain. Moreover, severe scoliosis can even lead to lung and heart problems. Thus, early diagnosis can make it easier to apply remedies/interventions and prevent further disease progression. Current diagnosis methods are based on visual inspection by physicians of radiographs and/or calculation of certain angles (e.g., Cobb angle). Traditional artificial intelligence-based diagnosis systems utilized these parameters to perform automated classification, which enabled fast and easy diagnosis supporting tools. However, they still require the specialists to perform error-prone tedious measurements. To this end, automated measurement tools were proposed based on processing techniques of X-ray images. In this paper, we utilize advances in deep transfer learning to diagnose spondylolisthesis and scoliosis from X-ray images without the need for any measurements. We collected raw data from real X-ray images of 338 subjects (i.e., 188 scoliosis, 79 spondylolisthesis, and 71 healthy). Deep transfer learning models were developed to perform three-class classification as well as pair-wise binary classifications among the three classes. The highest mean accuracy and maximum accuracy for three-class classification was 96.73% and 98.02%, respectively. Regarding pair-wise binary classification, high accuracy values were achieved for most of the models (i.e., > 98%). These results and other performance metrics reflect a robust ability to diagnose the subjects’ vertebral column disorders from standard X-ray images. The current study provides a supporting tool that can reasonably help the physicians make the correct early diagnosis with less effort and errors, and reduce the need for surgical interventions.

A Review of the Methods on Cobb Angle Measurements for Spinal Curvature

Scoliosis is a common disease of the spine and requires regular monitoring due to its progressive properties. A preferred indicator to assess scoliosis is by the Cobb angle, which is currently measured either manually by the relevant medical staff or semi-automatically, aided by a computer. These methods are not only labor-intensive but also vary in precision by the inter-observer and intra-observer. Therefore, a reliable and convenient method is urgently needed. With the development of computer vision and deep learning, it is possible to automatically calculate the Cobb angles by processing X-ray or CT/MR/US images. In this paper, the research progress of Cobb angle measurement in recent years is reviewed from the perspectives of computer vision and deep learning. By comparing the measurement effects of typical methods, their advantages and disadvantages are analyzed. Finally, the key issues and their development trends are also discussed.

Machine Learning Applications of Surgical Imaging for the Diagnosis and Treatment of Spine Disorders: Current State of the Art

Recent developments in machine learning (ML) methods demonstrate unparalleled potential for application in the spine. The ability for ML to provide diagnostic faculty, produce novel insights from existing capabilities, and augment or accelerate elements of surgical planning and decision making at levels equivalent or superior to humans will tremendously benefit spine surgeons and patients alike. In this review, we aim to provide a clinically relevant outline of ML-based technology in the contexts of spinal deformity, degeneration, and trauma, as well as an overview of commercial-level and precommercial-level surgical assist systems and decisional support tools. Furthermore, we briefly discuss potential applications of generative networks before highlighting some of the limitations of ML applications. We conclude that ML in spine imaging represents a significant addition to the neurosurgeon's armamentarium-it has the capacity to directly address and manifest clinical needs and improve diagnostic and procedural quality and safety-but is yet subject to challenges that must be addressed before widespread implementation.

Electromyographic Analysis of Paraspinal Muscles of Scoliosis Patients Using Machine Learning Approaches

A large number of studies have used electromyography (EMG) to measure the paraspinal muscle activity of adolescents with idiopathic scoliosis. However, investigations on the features of these muscles are very limited even though the information is useful for evaluating the effectiveness of various types of interventions, such as scoliosis-specific exercises. The aim of this cross-sectional study is to investigate the characteristics of participants with imbalanced muscle activity and the relationships among 13 features (physical features and EMG signal value). A total of 106 participants (69% with scoliosis; 78% female; 9–30 years old) are involved in this study. Their basic profile information is obtained, and the surface EMG signals of the upper trapezius, latissimus dorsi, and erector spinae (thoracic and erector spinae) lumbar muscles are tested in the static (sitting) and dynamic (prone extension position) conditions. Then, two machine learning approaches and an importance analysis are used. About 30% of the participants in this study find that balancing their paraspinal muscle activity during sitting is challenging. The most interesting finding is that the dynamic asymmetry of the erector spinae (lumbar) group of muscles is an important (third in importance) predictor of scoliosis aside from the angle of trunk rotation and height of the subject.

Prognosis Prediction of Uveal Melanoma After Plaque Brachytherapy Based on Ultrasound With Machine Learning

INTRODUCTION

Uveal melanoma (UM) is the most common intraocular malignancy in adults. Plaque brachytherapy remains the dominant eyeball-conserving therapy for UM. Tumor regression in UM after plaque brachytherapy has been reported as a valuable prognostic factor. The present study aimed to develop an accurate machine-learning model to predict the 4-year risk of metastasis and death in UM based on ocular ultrasound data.

MATERIAL AND METHODS

A total of 454 patients with UM were enrolled in this retrospective, single-center study. All patients were followed up for at least 4 years after plaque brachytherapy and underwent ophthalmologic evaluations before the therapy. B-scan ultrasonography was used to measure the basal diameters and thickness of tumors preoperatively and postoperatively. Random Forest (RF) algorithm was used to construct two prediction models: whether a patient will survive for more than 4 years and whether the tumor will develop metastasis within 4 years after treatment.

RESULTS

Our predictive model achieved an area under the receiver operating characteristic curve (AUC) of 0.708 for predicting death using only a one-time follow-up record. Including the data from two additional follow-ups increased the AUC of the model to 0.883. We attained AUCs of 0.730 and 0.846 with data from one and three-time follow-up, respectively, for predicting metastasis. The model found that the amount of postoperative follow-up data significantly improved death and metastasis prediction accuracy. Furthermore, we divided tumor treatment response into four patterns. The D(decrease)/S(stable) patterns are associated with a significantly better prognosis than the I(increase)/O(other) patterns.

CONCLUSIONS

The present study developed an RF model to predict the risk of metastasis and death from UM within 4 years based on ultrasound follow-up records following plaque brachytherapy. We intend to further validate our model in prospective datasets, enabling us to implement timely and efficient treatments.

Feature Importance Analysis for Postural Deformity Detection System Using Explainable Predictive Modeling Technique

This study aimed to analyze feature importance by applying explainable artificial intelligence (XAI) to postural deformity parameters extracted from a computer vision-based posture analysis system (CVPAS). Overall, 140 participants were screened for CVPAS and enrolled. The main data analyzed were shoulder height difference (SHD), wrist height difference (WHD), and pelvic height difference (PHD) extracted using a CVPAS. Standing X-ray imaging and radiographic assessments were performed. Predictive modeling was implemented with XGBoost, random forest regressor, and logistic regression using XAI techniques for global and local feature analyses. Correlation analysis was performed between radiographic assessment and AI evaluation for PHD, SHD, and Cobb angle. Main global features affecting scoliosis were analyzed in the order of importance for PHD (0.18) and ankle height difference (0.06) in predictive modeling. Outstanding local features were PHD, WHD, and KHD that predominantly contributed to the increase in the probability of scoliosis, and the prediction probability of scoliosis was 94%. When the PHD was >3 mm, the probability of scoliosis increased sharply to 85.3%. The paired t-test result for AI and radiographic assessments showed that the SHD, Cobb angle, and scoliosis probability were significant (p < 0.05). Feature importance analysis using XAI to postural deformity parameters extracted from a CVPAS is a useful clinical decision support system for the early detection of posture deformities. PHD was a major parameter for both global and local analyses, and 3 mm was a threshold for significantly increasing the probability of local interpretation of each participant and the prediction of postural deformation, which leads to the prediction of participant-specific scoliosis.

A Survey of Methods and Technologies Used for Diagnosis of Scoliosis

The purpose of this article is to present diagnostic methods used in the diagnosis of scoliosis in the form of a brief review. This article aims to point out the advantages of select methods. This article focuses on general issues without elaborating on problems strictly related to physiotherapy and treatment methods, which may be the subject of further discussions. By outlining and categorizing each method, we summarize relevant publications that may not only help introduce other researchers to the field but also be a valuable source for studying existing methods, developing new ones or choosing evaluation strategies.

Deep learning-based facial image analysis in medical research: a systematic review protocol

INTRODUCTION

Deep learning techniques are gaining momentum in medical research. Evidence shows that deep learning has advantages over humans in image identification and classification, such as facial image analysis in detecting people's medical conditions. While positive findings are available, little is known about the state-of-the-art of deep learning-based facial image analysis in the medical context. For the consideration of patients' welfare and the development of the practice, a timely understanding of the challenges and opportunities faced by research on deep-learning-based facial image analysis is needed. To address this gap, we aim to conduct a systematic review to identify the characteristics and effects of deep learning-based facial image analysis in medical research. Insights gained from this systematic review will provide a much-needed understanding of the characteristics, challenges, as well as opportunities in deep learning-based facial image analysis applied in the contexts of disease detection, diagnosis and prognosis.

METHODS

Databases including PubMed, PsycINFO, CINAHL, IEEEXplore and Scopus will be searched for relevant studies published in English in September, 2021. Titles, abstracts and full-text articles will be screened to identify eligible articles. A manual search of the reference lists of the included articles will also be conducted. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses framework was adopted to guide the systematic review process. Two reviewers will independently examine the citations and select studies for inclusion. Discrepancies will be resolved by group discussions till a consensus is reached. Data will be extracted based on the research objective and selection criteria adopted in this study.

ETHICS AND DISSEMINATION

As the study is a protocol for a systematic review, ethical approval is not required. The study findings will be disseminated via peer-reviewed publications and conference presentations.

PROSPERO REGISTRATION NUMBER

CRD42020196473.

AI-Model for Identifying Pathologic Myopia Based on Deep Learning Algorithms of Myopic Maculopathy Classification and "Plus" Lesion Detection in Fundus Images

Background: Pathologic myopia (PM) associated with myopic maculopathy (MM) and “Plus” lesions is a major cause of irreversible visual impairment worldwide. Therefore, we aimed to develop a series of deep learning algorithms and artificial intelligence (AI)–models for automatic PM identification, MM classification, and “Plus” lesion detection based on retinal fundus images.

Materials and Methods: Consecutive 37,659 retinal fundus images from 32,419 patients were collected. After excluding 5,649 ungradable images, a total dataset of 32,010 color retinal fundus images was manually graded for training and cross-validation according to the META-PM classification. We also retrospectively recruited 1,000 images from 732 patients from the three other hospitals in Zhejiang Province, serving as the external validation dataset. The area under the receiver operating characteristic curve (AUC), sensitivity, specificity, accuracy, and quadratic-weighted kappa score were calculated to evaluate the classification algorithms. The precision, recall, and F1-score were calculated to evaluate the object detection algorithms. The performance of all the algorithms was compared with the experts’ performance. To better understand the algorithms and clarify the direction of optimization, misclassification and visualization heatmap analyses were performed.

Results: In five-fold cross-validation, algorithm I achieved robust performance, with accuracy = 97.36% (95% CI: 0.9697, 0.9775), AUC = 0.995 (95% CI: 0.9933, 0.9967), sensitivity = 93.92% (95% CI: 0.9333, 0.9451), and specificity = 98.19% (95% CI: 0.9787, 0.9852). The macro-AUC, accuracy, and quadratic-weighted kappa were 0.979, 96.74% (95% CI: 0.963, 0.9718), and 0.988 (95% CI: 0.986, 0.990) for algorithm II. Algorithm III achieved an accuracy of 0.9703 to 0.9941 for classifying the “Plus” lesions and an F1-score of 0.6855 to 0.8890 for detecting and localizing lesions. The performance metrics in external validation dataset were comparable to those of the experts and were slightly inferior to those of cross-validation.

Conclusion: Our algorithms and AI-models were confirmed to achieve robust performance in real-world conditions. The application of our algorithms and AI-models has promise for facilitating clinical diagnosis and healthcare screening for PM on a large scale.

Validation of the Relationship Between Iris Color and Uveal Melanoma Using Artificial Intelligence With Multiple Paths in a Large Chinese Population

Previous studies have shown that light iris color is a predisposing factor for the development of uveal melanoma (UM) in a population of Caucasian ancestry. However, in all these studies, a remarkably low percentage of patients have brown eyes, so we applied deep learning methods to investigate the correlation between iris color and the prevalence of UM in the Chinese population. All anterior segment photos were automatically segmented with U-NET, and only the iris regions were retained. Then the iris was analyzed with machine learning methods (random forests and convolutional neural networks) to obtain the corresponding iris color spectra (classification probability). We obtained satisfactory segmentation results with high consistency with those from experts. The iris color spectrum is consistent with the raters’ view, but there is no significant correlation with UM incidence.

The measurement of Cobb angle based on spine X-ray images using multi-scale convolutional neural network

Adolescent idiopathic scoliosis (AIS) is a structural spinal deformity mainly in the coronal plane and is among the most frequent deformities in children, adolescents, and young adults, with an overall prevalence of 0.47-5.2%. The Cobb angle is an objective measure to determine the progression of deformity and plays a critical role in the planning of surgical treatment. However, existing studies suggested that Cobb angle measurement is susceptible to inter- and intra-observer variability, as well as a high variability in the definition of the end vertebra. In this study, we proposed an automatic method for the spine vertebrae segmentation using Deeplab V3+, a powerful tool that has shown success in the image segmentation of other anatomical regions but spine, and Cobb angle measurement. The segmentation performance was compared to existing mainstay neural networks. Compared to U-Net, Residual U-Net and Dilated U-Net, our method using Deeplab V3+ showed the best performance in the Dice Similarity Coefficient (DSC), accuracy, sensitivity and Jaccard Index. An excellent correlation in the final Cobb angle calculation was achieved between the smallest distance point (SDP) method and two experts (> 0.95), with a small error in the angle estimation compared (MAE < 3°). The proposed method could provide a potential tool for the automatic estimation of the Cobb angle to improve the efficiency and accuracy of the treatment workflow for AIS.

Computerized assisted evaluation system for canine cardiomegaly via key points detection with deep learning

Cardiomegaly is the main imaging finding for canine heart diseases. There are many advances in the field of medical diagnosing based on imaging with deep learning for human being. However there are also increasing realization of the potential of using deep learning in veterinary medicine. We reported a clinically applicable assisted platform for diagnosing the canine cardiomegaly with deep learning. VHS (vertebral heart score) is a measuring method used for the heart size of a dog. The concrete value of VHS is calculated with the relative position of 16 key points detected by the system, and this result is then combined with VHS reference range of all dog breeds to assist in the evaluation of the canine cardiomegaly. We adopted HRNet (high resolution network) to detect 16 key points (12 and four key points located on vertebra and heart respectively) in 2274 lateral X-ray images (training and validation datasets) of dogs, the model was then used to detect the key points in external testing dataset (396 images), the AP (average performance) for key point detection reach 86.4 %. Then we applied an additional post processing procedure to correct the output of HRNets so that the AP reaches 90.9 %. This result signifies that this system can effectively assist the evaluation of canine cardiomegaly in a real clinical scenario.

Detecting pathological features and predicting fracture risk from dual-energy X-ray absorptiometry images using deep learning

Dual-energy X-ray absorptiometry (DXA) is the gold standard imaging method for diagnosing osteoporosis in clinical practice. The DXA images are commonly used to estimate a numerical value for bone mineral density (BMD), which decreases in osteoporosis. Low BMD is a known risk factor for osteoporotic fractures. In this study, we used deep learning to identify lumbar scoliosis and structural abnormalities that potentially affect BMD but are often neglected in lumbar spine DXA analysis. In addition, we tested the approach's ability to predict fractures using only DXA images. A dataset of 2949 images gathered by Kuopio Osteoporosis Risk Factor and Prevention Study was used to train a convolutional neural network (CNN) for classification. The model was able to classify scoliosis with an AUC of 0.96 and structural abnormalities causing unreliable BMD measurement with an AUC of 0.91. It predicted fractures occurring within 5 years from the lumbar spine DXA scan with an AUC of 0.63, meeting the predictive performance of combined BMD measurements from the lumbar spine and hip. In an independent test set of 574 clinical patients, AUC for lumbar scoliosis was 0.93 and AUC for unreliable BMD measurements was 0.94. In each classification task, neural network visualizations indicated the model's predictive strategy. We conclude that deep learning could complement the well established DXA method for osteoporosis diagnostics by analyzing incidental findings and image reliability, and improve its predictive ability in the future.

An algorithm for using deep learning convolutional neural networks with three dimensional depth sensor imaging in scoliosis detection

BACKGROUND CONTEXT

Timely intervention in growing individuals, such as brace treatment, relies on early detection of adolescent idiopathic scoliosis (AIS). To this end, several screening methods have been implemented. However, these methods have limitations in predicting the Cobb angle.

PURPOSE

This study aimed to evaluate the performance of a three-dimensional depth sensor imaging system with a deep learning algorithm, in predicting the Cobb angle in AIS.

STUDY DESIGN

Retrospective analysis of prospectively collected, consecutive, nonrandomized series of patients at five scoliosis centers in Japan.

PATIENT SAMPLE

One hundred and-sixty human subjects suspected to have AIS were included.

OUTCOME MEASURES

Patient demographics, radiographic measurements, and predicted Cobb angle derived from the deep learning algorithm were the outcome measures for this study.

METHODS

One hundred and sixty data files were shuffled into five datasets with 32 data files at random (dataset 1, 2, 3, 4, and 5) and five-fold cross validation was performed. The relationships between the actual and predicted Cobb angles were calculated using Pearson's correlation coefficient analyses. The prediction performances of the network models were evaluated using mean absolute error and root mean square error between the actual and predicted Cobb angles. The shuffling into five datasets and five-fold cross validation was conducted ten times. There were no study-specific biases related to conflicts of interest.

RESULTS

The correlation between the actual and the mean predicted Cobb angles was 0.91. The mean absolute error and root mean square error were 4.0° and 5.4°, respectively. The accuracy of the mean predicted Cobb angle was 94% for identifying a Cobb angle of ≥10° and 89% for that of ≥20°.

CONCLUSIONS

The three-dimensional depth sensor imaging system with its newly innovated convolutional neural network for regression is objective and has significant ability to predict the Cobb angle in children and adolescents. This system is expected to be used for screening scoliosis in clinics or physical examination at schools.

Automatically Diagnosing Disk Bulge and Disk Herniation With Lumbar Magnetic Resonance Images by Using Deep Convolutional Neural Networks: Method Development Study

Background

Disk herniation and disk bulge are two common disorders of lumbar intervertebral disks (IVDs) that often result in numbness, pain in the lower limbs, and lower back pain. Magnetic resonance (MR) imaging is one of the most efficient techniques for detecting lumbar diseases and is widely used for making clinical diagnoses at hospitals. However, there is a lack of efficient tools for effectively interpreting massive amounts of MR images to meet the requirements of many radiologists.

Objective

The aim of this study was to present an automatic system for diagnosing disk bulge and herniation that saves time and can effectively and significantly reduce the workload of radiologists.

Methods

The diagnosis of lumbar vertebral disorders is highly dependent on medical images. Therefore, we chose the two most common diseases—disk bulge and herniation—as research subjects. This study is mainly about identifying the position of IVDs (lumbar vertebra [L] 1 to L2, L2-L3, L3-L4, L4-L5, and L5 to sacral vertebra [S] 1) by analyzing the geometrical relationship between sagittal and axial images and classifying axial lumbar disk MR images via deep convolutional neural networks.

Results

This system involved 4 steps. In the first step, it automatically located vertebral bodies (including the L1, L2, L3, L4, L5, and S1) in sagittal images by using the faster region-based convolutional neural network, and our fourfold cross-validation showed 100% accuracy. In the second step, it spontaneously identified the corresponding disk in each axial lumbar disk MR image with 100% accuracy. In the third step, the accuracy for automatically locating the intervertebral disk region of interest in axial MR images was 100%. In the fourth step, the 3-class classification (normal disk, disk bulge, and disk herniation) accuracies for the L1-L2, L2-L3, L3-L4, L4-L5, and L5-S1 IVDs were 92.7%, 84.4%, 92.1%, 90.4%, and 84.2%, respectively.

Conclusions

The automatic diagnosis system was successfully built, and it could classify images of normal disks, disk bulge, and disk herniation. This system provided a web-based test for interpreting lumbar disk MR images that could significantly improve diagnostic efficiency and standardized diagnosis reports. This system can also be used to detect other lumbar abnormalities and cervical spondylosis.

Classification of neurofibromatosis-related dystrophic or nondystrophic scoliosis based on image features using Bilateral CNN

PURPOSE

We developed a system that can automatically classify cases of scoliosis secondary to neurofibromatosis type 1 (NF1-S) using deep learning algorithms (DLAs) and improve the accuracy and effectiveness of classification, thereby assisting surgeons with the auxiliary diagnosis.

METHODS

Comprehensive experiments in NF1 classification were performed based on a dataset consisting 211 NF1-S (131 dystrophic and 80 nondystrophic NF1-S) patients. Additionally, 100 congenital scoliosis (CS), 100 adolescent idiopathic scoliosis (AIS) patients, and 114 normal controls were used for experiments in primary classification. For identification of NF1-S with nondystrophic or dystrophic curves, we devised a novel network (i.e., Bilateral convolutional neural network [CNN]) utilizing a bilinear-like operation to discover the similar interest features between whole spine AP and lateral x-ray images. The performance of Bilateral CNN was compared with spine surgeons, conventional DLAs (i.e., VGG-16, ResNet-50, and Bilinear CNN [BCNN]), recently proposed DLAs (i.e., ShuffleNet, MobileNet, and EfficientNet), and Two-path BCNN which was the extension of BCNN using AP and lateral x-ray images as inputs.

RESULTS

In NF1 classification, our proposed Bilateral CNN with 80.36% accuracy outperformed the other seven DLAs ranging from 61.90% to 76.19% with fivefold cross-validation. It also outperformed the spine surgeons (with an average accuracy of 77.5% for the senior surgeons and 65.0% for the junior surgeons). Our method is highly generalizable due to the proposed methodology and data augmentation. Furthermore, the heatmaps extracted by Bilateral CNN showed curve pattern and morphology of ribs and vertebrae contributing most to the classification results. In primary classification, our proposed method with an accuracy of 87.92% also outperformed all the other methods with varied accuracies between 52.58% and 83.35% with fivefold cross-validation.

CONCLUSIONS

The proposed Bilateral CNN can automatically capture representative features for classifying NF1-S utilizing AP and lateral x-ray images, leading to a relatively good performance. Moreover, the proposed method can identify other spine deformities for auxiliary diagnosis.

Developing of a Mathematical Model to Perform Measurements of Axial Vertebral Rotation on Computer-Aided and Automated Diagnosis Systems, Using Raimondi's Method

Introduction

Axial vertebral rotation (AVR) is a basic parameter in the study of idiopathic scoliosis and on physical two-dimensional images. Raimondi's tables are the most used method in the quantification of AVR. The development of computing technologies has enabled the creation of computer-aided or automated diagnosis systems (CADx) with which measurement on medical images can be carried out more quickly, simply, and with less intra and interobserver variabilities than manual methods. Although there are several publications dealing with the measurement of AVR in CADx systems, none of them provides information on the equation or algorithm used for the measurement applying Raimondi's method. Goal. The aim of this work is to perform a mathematical modelling of the data contained in Raimondi's tables that enable the Raimondi method to be used in digital medical images more precisely and in a more exact manner.

Methods

Data from Raimondi's tables were tabulated on a first step. After this, each column of Raimondi's tables containing values corresponding to vertebral body width (D) were adjusted to a curve determined by AVR = f (d). Third, representative values of each rotation divided by D were obtained through the equation of each column D. In a fourth step, a regression line was fitted to the data in each row, and from its equation, the mean value of the D/d distribution is calculated (value corresponding to the central column, D = 45). Finally, a curve was adjusted to the obtained data using the least squares method. Summary and Conclusion. Our mathematical equation allows the Raimondi method to be used in digital images of any format in a more accurate and simplified approach. This equation can be easily and freely implemented in any CADx system to quantify AVR, providing a more precise use of Raimondi's method, as well as being used in traditional manual measurement as it is performed with Raimondi tables.