Deep learning approach for automatic landmark detection and alignment analysis in whole-spine lateral radiographs

Heatmap-Based Active Shape Model for Landmark Detection in Lumbar X-ray Images

Medical staff inspect lumbar X-ray images to diagnose lumbar spine diseases, and the analysis process is currently automated using deep-learning techniques. The detection of landmarks is necessary in the automatic process of localizing the position and identifying the morphological features of the vertebrae. However, detection errors may occur owing to the noise and ambiguity of images, as well as individual variations in the shape of the lumbar vertebrae. This study proposes a method to improve the robustness of landmark detection results. This method assumes that landmarks are detected by a convolutional neural network-based two-step model consisting of Pose-Net and M-Net. The model generates a heatmap response to indicate the probable landmark positions. The proposed method then corrects the landmark positions using the heatmap response and active shape model, which employs statistical information on the landmark distribution. Experiments were conducted using 3600 lumbar X-ray images, and the results showed that the landmark detection error was reduced by the proposed method. The average value of maximum errors decreased by 5.58% after applying the proposed method, which combines the outstanding image analysis capabilities of deep learning with statistical shape constraints on landmark distribution. The proposed method could also be easily integrated with other techniques to increase the robustness of landmark detection results such as CoordConv layers and non-directional part affinity field. This resulted in a further enhancement in the landmark detection performance. These advantages can improve the reliability of automatic systems used to inspect lumbar X-ray images. This will benefit both patients and medical staff by reducing medical expenses and increasing diagnostic efficiency.

Artificial Intelligence Assistance for the Measurement of Full Alignment Parameters in Whole-Spine Lateral Radiographs

BACKGROUND

Measuring spinal alignment with radiological parameters is essential in patients with spinal conditions likely to be treated surgically. These evaluations are not usually included in the radiological report. As a result, spinal surgeons commonly perform the measurement, which is time-consuming and subject to errors. We aim to develop a fully automated artificial intelligence (AI) tool to assist in measuring alignment parameters in whole-spine lateral radiograph (WSL X-rays).

METHODS

We developed a tool called Vertebrai that automatically calculates the global spinal parameters (GSPs): Pelvic incidence, sacral slope, pelvic tilt, L1-L4 angle, L4-S1 lumbo-pelvic angle, T1 pelvic angle, sagittal vertical axis, cervical lordosis, C1-C2 lordosis, lumbar lordosis, mid-thoracic kyphosis, proximal thoracic kyphosis, global thoracic kyphosis, T1 slope, C2-C7 plummet, spino-sacral angle, C7 tilt, global tilt, spinopelvic tilt, and hip odontoid axis. We assessed human-AI interaction instead of AI performance alone. We compared the time to measure GSP and inter-rater agreement with and without AI assistance. Two institutional datasets were created with 2267 multilabel images for classification and 784 WSL X-rays with reference standard landmark labeled by spinal surgeons.

RESULTS

Vertebrai significantly reduced the measurement time comparing spine surgeons with AI assistance and the AI algorithm alone, without human intervention (3 minutes vs. 0.26 minutes; P < 0.05). Vertebrai achieved an average accuracy of 83% in detecting abnormal alignment values, with the sacral slope parameter exhibiting the lowest accuracy at 61.5% and spinopelvic tilt demonstrating the highest accuracy at 100%. Intraclass correlation analysis revealed a high level of correlation and consistency in the global alignment parameters.

CONCLUSIONS

Vertebrai's measurements can accurately detect alignment parameters, making it a promising tool for measuring GSP automatically.

Artificial Intelligence in Spinal Imaging and Patient Care: A Review of Recent Advances

Artificial intelligence (AI) is transforming spinal imaging and patient care through automated analysis and enhanced decision-making. This review presents a clinical task-based evaluation, highlighting the specific impact of AI techniques on different aspects of spinal imaging and patient care. We first discuss how AI can potentially improve image quality through techniques like denoising or artifact reduction. We then explore how AI enables efficient quantification of anatomical measurements, spinal curvature parameters, vertebral segmentation, and disc grading. This facilitates objective, accurate interpretation and diagnosis. AI models now reliably detect key spinal pathologies, achieving expert-level performance in tasks like identifying fractures, stenosis, infections, and tumors. Beyond diagnosis, AI also assists surgical planning via synthetic computed tomography generation, augmented reality systems, and robotic guidance. Furthermore, AI image analysis combined with clinical data enables personalized predictions to guide treatment decisions, such as forecasting spine surgery outcomes. However, challenges still need to be addressed in implementing AI clinically, including model interpretability, generalizability, and data limitations. Multicenter collaboration using large, diverse datasets is critical to advance the field further. While adoption barriers persist, AI presents a transformative opportunity to revolutionize spinal imaging workflows, empowering clinicians to translate data into actionable insights for improved patient care.

Deep Learning Method for Precise Landmark Identification and Structural Assessment of Whole-Spine Radiographs

This study measured parameters automatically by marking the point for measuring each parameter on whole-spine radiographs. Between January 2020 and December 2021, 1017 sequential lateral whole-spine radiographs were retrospectively obtained. Of these, 819 and 198 were used for training and testing the performance of the landmark detection model, respectively. To objectively evaluate the program's performance, 690 whole-spine radiographs from four other institutions were used for external validation. The combined dataset comprised radiographs from 857 female and 850 male patients (average age 42.2 ± 27.3 years; range 20-85 years). The landmark localizer showed the highest accuracy in identifying cervical landmarks (median error 1.5-2.4 mm), followed by lumbosacral landmarks (median error 2.1-3.0 mm). However, thoracic landmarks displayed larger localization errors (median 2.4-4.3 mm), indicating slightly reduced precision compared with the cervical and lumbosacral regions. The agreement between the deep learning model and two experts was good to excellent, with intraclass correlation coefficient values >0.88. The deep learning model also performed well on the external validation set. There were no statistical differences between datasets in all parameters, suggesting that the performance of the artificial intelligence model created was excellent. The proposed automatic alignment analysis system identified anatomical landmarks and positions of the spine with high precision and generated various radiograph imaging parameters that had a good correlation with manual measurements.

A Multi-Element Identification System Based on Deep Learning for the Visual Field of Percutaneous Endoscopic Spine Surgery

Background

Lumbar disc herniation is a common degenerative lumbar disease with an increasing incidence. Percutaneous endoscopic lumbar discectomy can treat lumbar disc herniation safely and effectively with a minimally invasive procedure. However, the learning curve of this technology is steep, which means that initial learners are often not sufficiently proficient in endoscopic operations, which can easily lead to iatrogenic damage. At present, the application of computer deep learning technology to clinical diagnosis, treatment, and surgical navigation has achieved satisfactory results.

Purpose

The objective of our team is to develop a multi-element identification system for the visual field of endoscopic spine surgery using deep learning algorithms and to evaluate the feasibility of this system.

Method

We established an image database by collecting surgical videos of 48 patients diagnosed with lumbar disc herniation, which was labeled by two spinal surgeons. We selected 6000 images of the visual field of percutaneous endoscopic spine surgery (including various tissue structures and surgical instruments), divided into the training data, validation data, and test data according to 2:1:2. We developed convolutional neural network models based on instance segmentation-Solov2, CondInst, Mask R-CNN and Yolact, and set the four network model backbone as ResNet101 and ResNet50 respectively. Mean average precision (mAP) and frames per second (FPS) were used to measure the performance of each model for classification, localization and recognition in real time, and AP (average) is used to evaluate how easily an element is detected by neural networks based on computer deep learning.

Result

Comprehensively comparing mAP and FSP of each model for bounding box test and segmentation task for the test set of images, we found that Solov2 (ResNet101) (mAP = 73.5%, FPS = 28.9), Mask R-CNN (ResNet101) (mAP = 72.8%, FPS = 28.5) models are the most stable, with higher precision and faster image processing speed. Combining the average precision of the elements in the bounding box test and segmentation tasks in each network, the AP(average) was highest for tool 3 (bbox-0.85, segm-0.89) and lowest for tool 5 (bbox-0.63, segm-0.72) in the instrumentation, whereas in the anatomical tissue elements, the fibrosus annulus (bbox-0.68, segm-0.69) and ligamentum flavum (bbox-0.65, segm-0.62) had higher AP(average),while extra-dural fat (bbox-0.42, segm-0.44) was lowest.

Conclusion

Our team has developed a multi-element identification system for the visual field of percutaneous endoscopic spine surgery adapted to the interlaminar and foraminal approaches, which can identify and track anatomical tissue (nerve, ligamentum flavum, nucleus pulposus, etc.) and surgical instruments (endoscopic forceps, an high-speed diamond burr, etc.), which can be used in the future as a virtual educational tool or applied to the intraoperative real-time assistance system for spinal endoscopic operation.

Reference Data for Diagnosis of Spondylolisthesis and Disc Space Narrowing Based on NHANES-II X-rays

Robust reference data, representing a large and diverse population, are needed to objectively classify measurements of spondylolisthesis and disc space narrowing as normal or abnormal. The reference data should be open access to drive standardization across technology developers. The large collection of radiographs from the 2nd National Health and Nutrition Examination Survey was used to establish reference data. A pipeline of neural networks and coded logic was used to place landmarks on the corners of all vertebrae, and these landmarks were used to calculate multiple disc space metrics. Descriptive statistics for nine SPO and disc metrics were tabulated and used to identify normal discs, and data for only the normal discs were used to arrive at reference data. A spondylolisthesis index was developed that accounts for important variables. These reference data facilitate simplified and standardized reporting of multiple intervertebral disc metrics.

Multicentric development and validation of a multi-scale and multi-task deep learning model for comprehensive lower extremity alignment analysis

Osteoarthritis of the knee, a widespread cause of knee disability, is commonly treated in orthopedics due to its rising prevalence. Lower extremity misalignment, pivotal in knee injury etiology and management, necessitates comprehensive mechanical alignment evaluation via frequently-requested weight-bearing long leg radiographs (LLR). Despite LLR's routine use, current analysis techniques are error-prone and time-consuming. To address this, we conducted a multicentric study to develop and validate a deep learning (DL) model for fully automated leg alignment assessment on anterior-posterior LLR, targeting enhanced reliability and efficiency. The DL model, developed using 594 patients' LLR and a 60%/10%/30% data split for training, validation, and testing, executed alignment analyses via a multi-step process, employing a detection network and nine specialized networks. It was designed to assess all vital anatomical and mechanical parameters for standard clinical leg deformity analysis and preoperative planning. Accuracy, reliability, and assessment duration were compared with three specialized orthopedic surgeons across two distinct institutional datasets (136 and 143 radiographs). The algorithm exhibited equivalent performance to the surgeons in terms of alignment accuracy (DL: 0.21 ± 0.18°to 1.06 ± 1.3°vs. OS: 0.21 ± 0.16°to 1.72 ± 1.96°), interrater reliability (ICC DL: 0.90 ± 0.05 to 1.0 ± 0.0 vs. ICC OS: 0.90 ± 0.03 to 1.0 ± 0.0), and clinically acceptable accuracy (DL: 53.9%-100% vs OS 30.8%-100%). Further, automated analysis significantly reduced analysis time compared to manual annotation (DL: 22 ± 0.6 s vs. OS; 101.7 ± 7 s, p ≤ 0.01). By demonstrating that our algorithm not only matches the precision of expert surgeons but also significantly outpaces them in both speed and consistency of measurements, our research underscores a pivotal advancement in harnessing AI to enhance clinical efficiency and decision-making in orthopaedics.

Landet: an efficient physics-informed deep learning approach for automatic detection of anatomical landmarks and measurement of spinopelvic alignment

PURPOSE

An efficient physics-informed deep learning approach for extracting spinopelvic measures from X-ray images is introduced and its performance is evaluated against manual annotations.

METHODS

Two datasets, comprising a total of 1470 images, were collected to evaluate the model's performance. We propose a novel method of detecting landmarks as objects, incorporating their relationships as constraints (LanDet). Using this approach, we trained our deep learning model to extract five spine and pelvis measures: Sacrum Slope (SS), Pelvic Tilt (PT), Pelvic Incidence (PI), Lumbar Lordosis (LL), and Sagittal Vertical Axis (SVA). The results were compared to manually labelled test dataset (GT) as well as measures annotated separately by three surgeons.

RESULTS

The LanDet model was evaluated on the two datasets separately and on an extended dataset combining both. The final accuracy for each measure is reported in terms of Mean Absolute Error (MAE), Standard Deviation (SD), and R Pearson correlation coefficient as follows: [ S S ∘ : 3.7 ( 2.7 ) , R = 0.89 ] ,[ P T ∘ : 1.3 ( 1.1 ) , R = 0.98 ] , [ P I ∘ : 4.2 ( 3.1 ) , R = 0.93 ] , [ L L ∘ : 5.1 ( 6.4 ) , R = 0.83 ] , [ S V A ( m m ) : 2.1 ( 1.9 ) , R = 0.96 ] . To assess model reliability and compare it against surgeons, the intraclass correlation coefficient (ICC) metric is used. The model demonstrated better consistency with surgeons with all values over 0.88 compared to what was previously reported in the literature.

CONCLUSION

The LanDet model exhibits competitive performance compared to existing literature. The effectiveness of the physics-informed constraint method, utilized in our landmark detection as object algorithm, is highlighted. Furthermore, we addressed the limitations of heatmap-based methods for anatomical landmark detection and tackled issues related to mis-identifying of similar or adjacent landmarks instead of intended landmark using this novel approach.

A quantitative evaluation of the deep learning model of segmentation and measurement of cervical spine MRI in healthy adults

PURPOSE

To evaluate the 3D U-Net model for automatic segmentation and measurement of cervical spine structures using magnetic resonance (MR) images of healthy adults.

MATERIALS AND METHODS

MR images of the cervical spine from 160 healthy adults were collected retrospectively. A previously constructed deep-learning model was used to automatically segment anatomical structures. Segmentation and localization results were checked by experienced radiologists. Pearson's correlation analyses were conducted to examine relationships between patient and image parameters.

RESULTS

No measurement was significantly correlated with age or sex. The mean values of the areas of the subarachnoid space and spinal cord from the C2/3 (cervical spine 2-3) to C6/7 intervertebral disc levels were 102.85-358.12 mm2 and 53.71-110.32 mm2 , respectively. The ratios of the areas of the spinal cord to the subarachnoid space were 0.25-0.68. The transverse and anterior-posterior diameters of the subarachnoid space were 14.77-26.56 mm and 7.38-17.58 mm, respectively. The transverse and anterior-posterior diameters of the spinal cord were 9.11-16.02 mm and 5.47-10.12 mm, respectively.

CONCLUSION

A deep learning model based on 3D U-Net automatically segmented and performed measurements on cervical spine MR images from healthy adults, paving the way for quantitative diagnosis models for spinal cord diseases.

Inadequate Annotation and Its Impact on Pelvic Tilt Measurement in Clinical Practice

BACKGROUND

Accurate pre-surgical templating of the pelvic tilt (PT) angle is essential for hip and spine surgeries, yet the reliability of PT annotations is often compromised by human error, inherent subjectivity, and variations in radiographic quality. This study aims to identify challenges leading to inadequate annotations at a landmark dimension and evaluating their impact on PT.

METHODS

We retrospectively collected 115 consecutive sagittal radiographs for the measurement of PT based on two definitions: the anterior pelvic plane and a line connecting the femoral head's centre to the sacral plate's midpoint. Five annotators engaged in the measurement, followed by a secondary review to assess the adequacy of the annotations across all the annotators.

RESULTS

The outcomes indicated that over 60% images had at least one landmark considered inadequate by the majority of the reviewers, with poor image quality, outliers, and unrecognized anomalies being the primary causes. Such inadequacies led to discrepancies in the PT measurements, ranging from -2° to 2°.

CONCLUSION

This study highlights that landmarks annotated from clear anatomical references were more reliable than those estimated. It also underscores the prevalence of suboptimal annotations in PT measurements, which extends beyond the scope of traditional statistical analysis and could result in significant deviations in individual cases, potentially impacting clinical outcomes.

Landmark annotation through feature combinations: a comparative study on cephalometric images with in-depth analysis of model's explainability

OBJECTIVES

The objectives of this study are to explore and evaluate the automation of anatomical landmark localization in cephalometric images using machine learning techniques, with a focus on feature extraction and combinations, contextual analysis, and model interpretability through Shapley Additive exPlanations (SHAP) values.

METHODS

We conducted extensive experimentation on a private dataset of 300 lateral cephalograms to thoroughly study the annotation results obtained using pixel feature descriptors including raw pixel, gradient magnitude, gradient direction, and histogram-oriented gradient (HOG) values. The study includes evaluation and comparison of these feature descriptions calculated at different contexts namely local, pyramid, and global. The feature descriptor obtained using individual combinations is used to discern between landmark and nonlandmark pixels using classification method. Additionally, this study addresses the opacity of LGBM ensemble tree models across landmarks, introducing SHAP values to enhance interpretability.

RESULTS

The performance of feature combinations was assessed using metrics like mean radial error, standard deviation, success detection rate (SDR) (2 mm), and test time. Remarkably, among all the combinations explored, both the HOG and gradient direction operations demonstrated significant performance across all context combinations. At the contextual level, the global texture outperformed the others, although it came with the trade-off of increased test time. The HOG in the local context emerged as the top performer with an SDR of 75.84% compared to others.

CONCLUSIONS

The presented analysis enhances the understanding of the significance of different features and their combinations in the realm of landmark annotation but also paves the way for further exploration of landmark-specific feature combination methods, facilitated by explainability.

Novel AI-Based Algorithm for the Automated Measurement of Cervical Sagittal Balance Parameters. A Validation Study on Pre- and Postoperative Radiographs of 129 Patients

STUDY DESIGN

Retrospective, mono-centric cohort research study.

OBJECTIVES

The analysis of cervical sagittal balance parameters is essential for preoperative planning and dependent on the physician's experience. A fully automated artificial intelligence-based algorithm could contribute to an objective analysis and save time. Therefore, this algorithm should be validated in this study.

METHODS

Two surgeons measured C2-C7 lordosis, C1-C7 Sagittal Vertical Axis (SVA), C2-C7-SVA, C7-slope and T1-slope in pre- and postoperative lateral cervical X-rays of 129 patients undergoing anterior cervical surgery. All parameters were measured twice by surgeons and compared to the measurements by the AI algorithm consisting of 4 deep convolutional neural networks. Agreement between raters was quantified, among other metrics, by mean errors and single measure intraclass correlation coefficients for absolute agreement.

RESULTS

ICC-values for intra- (range: .92-1.0) and inter-rater (.91-1.0) reliability reflect excellent agreement between human raters. The AI-algorithm could determine all parameters with excellent ICC-values (preop:0.80-1.0; postop:0.86-.99). For a comparison between the AI algorithm and 1 surgeon, mean errors were smallest for C1-C7 SVA (preop: -.3 mm (95% CI:-.6 to -.1 mm), post: .3 mm (.0-.7 mm)) and largest for C2-C7 lordosis (preop:-2.2° (-2.9 to -1.6°), postop: 2.3°(-3.0 to -1.7°)). The automatic measurement was possible in 99% and 98% of pre- and postoperative images for all parameters except T1 slope, which had a detection rate of 48% and 51% in pre- and postoperative images.

CONCLUSION

This study validates that an AI-algorithm can reliably measure cervical sagittal balance parameters automatically in patients suffering from degenerative spinal diseases. It may simplify manual measurements and autonomously analyze large-scale datasets. Further studies are required to validate the algorithm on a larger and more diverse patient cohort.

Deep learning algorithm for fully automated measurement of sagittal balance in adult spinal deformity

AIM

Deep learning (DL) algorithms can be used for automated analysis of medical imaging. The aim of this study was to assess the accuracy of an innovative, fully automated DL algorithm for analysis of sagittal balance in adult spinal deformity (ASD).

MATERIAL AND METHODS

Sagittal balance (sacral slope, pelvic tilt, pelvic incidence, lumbar lordosis and sagittal vertical axis) was evaluated in 141 preoperative and postoperative radiographs of patients with ASD. The DL, landmark-based measurements, were compared with the ground truth values from validated manual measurements.

RESULTS

The DL algorithm showed an excellent consistency with the ground truth measurements. The intra-class correlation coefficient between the DL and ground truth measurements was 0.71-0.99 for preoperative and 0.72-0.96 for postoperative measurements. The DL detection rate was 91.5% and 84% for preoperative and postoperative images, respectively.

CONCLUSION

This is the first study evaluating a complete automated DL algorithm for analysis of sagittal balance with high accuracy for all evaluated parameters. The excellent accuracy in the challenging pathology of ASD with long construct instrumentation demonstrates the eligibility and possibility for implementation in clinical routine.

Deep learning system for automated detection of posterior ligamentous complex injury in patients with thoracolumbar fracture on MRI

This study aimed to develop a deep learning (DL) algorithm for automated detection and localization of posterior ligamentous complex (PLC) injury in patients with acute thoracolumbar (TL) fracture on magnetic resonance imaging (MRI) and evaluate its diagnostic performance. In this retrospective multicenter study, using midline sagittal T2-weighted image with fracture (± PLC injury), a training dataset and internal and external validation sets of 300, 100, and 100 patients, were constructed with equal numbers of injured and normal PLCs. The DL algorithm was developed through two steps (Attention U-net and Inception-ResNet-V2). We evaluate the diagnostic performance for PLC injury between the DL algorithm and radiologists with different levels of experience. The area under the curves (AUCs) generated by the DL algorithm were 0.928, 0.916 for internal and external validations, and by two radiologists for observer performance test were 0.930, 0.830, respectively. Although no significant difference was found in diagnosing PLC injury between the DL algorithm and radiologists, the DL algorithm exhibited a trend of higher AUC than the radiology trainee. Notably, the radiology trainee's diagnostic performance significantly improved with DL algorithm assistance. Therefore, the DL algorithm exhibited high diagnostic performance in detecting PLC injuries in acute TL fractures.

Machine Learning-Based Measurement of Regional and Global Spinal Parameters Using the Concept of Incidence Angle of Inflection Points

This study delves into the application of convolutional neural networks (CNNs) in evaluating spinal sagittal alignment, introducing the innovative concept of incidence angles of inflection points (IAIPs) as intuitive parameters to capture the interplay between pelvic and spinal alignment. Pioneering the fusion of IAIPs with machine learning for sagittal alignment analysis, this research scrutinized whole-spine lateral radiographs from hundreds of patients who visited a single institution, utilizing high-quality images for parameter assessments. Noteworthy findings revealed robust success rates for certain parameters, including pelvic and C2 incidence angles, but comparatively lower rates for sacral slope and L1 incidence. The proposed CNN-based machine learning method demonstrated remarkable efficiency, achieving an impressive 80 percent detection rate for various spinal angles, such as lumbar lordosis and thoracic kyphosis, with a precise error threshold of 3.5°. Further bolstering the study's credibility, measurements derived from the novel formula closely aligned with those directly extracted from the CNN model. In conclusion, this research underscores the utility of the CNN-based deep learning algorithm in delivering precise measurements of spinal sagittal parameters, and highlights the potential for integrating machine learning with the IAIP concept for comprehensive data accumulation in the domain of sagittal spinal alignment analysis, thus advancing our understanding of spinal health.

AI-Driven Segmentation and Automated Analysis of the Whole Sagittal Spine from X-ray Images for Spinopelvic Parameter Evaluation

Spinal-pelvic parameters are utilized in orthopedics for assessing patients' curvature and body alignment in diagnosing, treating, and planning surgeries for spinal and pelvic disorders. Segmenting and autodetecting the whole spine from lateral radiographs is challenging. Recent efforts have employed deep learning techniques to automate the segmentation and analysis of whole-spine lateral radiographs. This study aims to develop an artificial intelligence (AI)-based deep learning approach for the automated segmentation, alignment, and measurement of spinal-pelvic parameters through whole-spine lateral radiographs. We conducted the study on 932 annotated images from various spinal pathologies. Using a deep learning (DL) model, anatomical landmarks of the cervical, thoracic, lumbar vertebrae, sacrum, and femoral head were automatically distinguished. The algorithm was designed to measure 13 radiographic alignment and spinal-pelvic parameters from the whole-spine lateral radiographs. Training data comprised 748 digital radiographic (DR) X-ray images, while 90 X-ray images were used for validation. Another set of 90 X-ray images served as the test set. Inter-rater reliability between orthopedic spine specialists, orthopedic residents, and the DL model was evaluated using the intraclass correlation coefficient (ICC). The segmentation accuracy for anatomical landmarks was within an acceptable range (median error: 1.7-4.1 mm). The inter-rater reliability between the proposed DL model and individual experts was fair to good for measurements of spinal curvature characteristics (all ICC values > 0.62). The developed DL model in this study demonstrated good levels of inter-rater reliability for predicting anatomical landmark positions and measuring radiographic alignment and spinal-pelvic parameters. Automated segmentation and analysis of whole-spine lateral radiographs using deep learning offers a promising tool to enhance accuracy and efficiency in orthopedic diagnostics and treatments.

Automatic Calculation of Cervical Spine Parameters Using Deep Learning: Development and Validation on an External Dataset

STUDY DESIGN

Retrospective data analysis.

OBJECTIVES

This study aims to develop a deep learning model for the automatic calculation of some important spine parameters from lateral cervical radiographs.

METHODS

We collected two datasets from two different institutions. The first dataset of 1498 images was used to train and optimize the model to find the best hyperparameters while the second dataset of 79 images was used as an external validation set to evaluate the robustness and generalizability of our model. The performance of the model was assessed by calculating the median absolute errors between the model prediction and the ground truth for the following parameters: T1 slope, C7 slope, C2-C7 angle, C2-C6 angle, Sagittal Vertical Axis (SVA), C0-C2, Redlund-Johnell distance (RJD), the cranial tilting (CT) and the craniocervical angle (CCA).

RESULTS

Regarding the angles, we found median errors of 1.66° (SD 2.46°), 1.56° (1.95°), 2.46° (SD 2.55), 1.85° (SD 3.93°), 1.25° (SD 1.83°), .29° (SD .31°) and .67° (SD .77°) for T1 slope, C7 slope, C2-C7, C2-C6, C0-C2, CT, and CCA respectively. As concerns the distances, we found median errors of .55 mm (SD .47 mm) and .47 mm (.62 mm) for SVA and RJD respectively.

CONCLUSIONS

In this work, we developed a model that was able to accurately predict cervical spine parameters from lateral cervical radiographs. In particular, the performances on the external validation set demonstrate the robustness and the high degree of generalizability of our model on images acquired in a different institution.

Performance evaluation of a deep learning-based cascaded HRNet model for automatic measurement of X-ray imaging parameters of lumbar sagittal curvature

PURPOSE

To develop a deep learning-based cascaded HRNet model, in order to automatically measure X-ray imaging parameters of lumbar sagittal curvature and to evaluate its prediction performance.

METHODS

A total of 3730 lumbar lateral digital radiography (DR) images were collected from picture archiving and communication system (PACS). Among them, 3150 images were randomly selected as the training dataset and validation dataset, and 580 images as the test dataset. The landmarks of the lumbar curve index (LCI), lumbar lordosis angle (LLA), sacral slope (SS), lumbar lordosis index (LLI), and the posterior edge tangent angle of the vertebral body (PTA) were identified and marked. The measured results of landmarks on the test dataset were compared with the mean values of manual measurement as the reference standard. Percentage of correct key-points (PCK), intra-class correlation coefficient (ICC), Pearson correlation coefficient (r), mean absolute error (MAE), mean square error (MSE), root-mean-square error (RMSE), and Bland-Altman plot were used to evaluate the performance of the cascade HRNet model.

RESULTS

The PCK of the cascaded HRNet model was 97.9-100% in the 3 mm distance threshold. The mean differences between the reference standard and the predicted values for LCI, LLA, SS, LLI, and PTA were 0.43 mm, 0.99°, 1.11°, 0.01 mm, and 0.23°, respectively. There were strong correlation and consistency of the five parameters between the cascaded HRNet model and manual measurements (ICC = 0.989-0.999, R = 0.991-0.999, MAE = 0.63-1.65, MSE = 0.61-4.06, RMSE = 0.78-2.01).

CONCLUSION

The cascaded HRNet model based on deep learning algorithm could accurately identify the sagittal curvature-related landmarks on lateral lumbar DR images and automatically measure the relevant parameters, which is of great significance in clinical application.

Automatic Spine Segmentation and Parameter Measurement for Radiological Analysis of Whole-Spine Lateral Radiographs Using Deep Learning and Computer Vision

Radiographic examination is essential for diagnosing spinal disorders, and the measurement of spino-pelvic parameters provides important information for the diagnosis and treatment planning of spinal sagittal deformities. While manual measurement methods are the golden standard for measuring parameters, they can be time consuming, inefficient, and rater dependent. Previous studies that have used automatic measurement methods to alleviate the downsides of manual measurements showed low accuracy or could not be applied to general films. We propose a pipeline for automated measurement of spinal parameters by combining a Mask R-CNN model for spine segmentation with computer vision algorithms. This pipeline can be incorporated into clinical workflows to provide clinical utility in diagnosis and treatment planning. A total of 1807 lateral radiographs were used for the training (n = 1607) and validation (n = 200) of the spine segmentation model. An additional 200 radiographs, which were also used for validation, were examined by three surgeons to evaluate the performance of the pipeline. Parameters automatically measured by the algorithm in the test set were statistically compared to parameters measured manually by the three surgeons. The Mask R-CNN model achieved an average precision at 50% intersection over union (AP50) of 96.2% and a Dice score of 92.6% for the spine segmentation task in the test set. The mean absolute error values of the spino-pelvic parameters measurement results were within the range of 0.4° (pelvic tilt) to 3.0° (lumbar lordosis, pelvic incidence), and the standard error of estimate was within the range of 0.5° (pelvic tilt) to 4.0° (pelvic incidence). The intraclass correlation coefficient values ranged from 0.86 (sacral slope) to 0.99 (pelvic tilt, sagittal vertical axis).

An automated landmark method to describe geometric changes in the human mandible during growth

OBJECTIVE

The principal aim of this study was to assess an automatic landmarking approach to human mandibles based on the atlas method. The secondary aim was to identify the areas of greatest variation in the mandibles of middle-aged to older adults.

DESIGN

Our sample consisted of 160 mandibles from computed tomography scans of 80 men and 80 women aged between 40 and 79 years. Eleven anatomical landmarks were placed manually on mandibles. The automated landmarking through point cloud alignment and correspondence (ALPACA) method implemented in 3D Slicer was used to automatically place landmarks to all meshes. Euclidean distances, normalized centroid size, and Procrustes ANOVA were calculated for both methods. A pseudo-landmarks approach was followed using ALPACA to identify areas of changes among our sample.

RESULTS

The ALPACA method showed significant differences in Euclidean distances for all landmarks compared to the manual method. A mean Euclidean distance of 1.7 mm was found for the ALPACA method and 0.99 mm for the manual method. Both methods found that sex, age, and size had a significant effect on mandibular shape. The greatest variations were observed in the condyle, ramus, and symphysis regions.

CONCLUSION

The results obtained using the ALPACA method are acceptable and promising. This approach can automatically place landmarks with an average accuracy of less than 2 mm, which may be sufficient in most anthropometric analyses. In the light of our results, however, odontological application such as occlusal analysis is not recommended.

Feature-correlation-aware history-preserving-sparse-coding framework for automatic vertebra recognition

Automatic vertebra recognition from magnetic resonance imaging (MRI) is of significance in disease diagnosis and surgical treatment of spinal patients. Although modern methods have achieved remarkable progress, vertebra recognition still faces two challenges in practice: (1) Vertebral appearance challenge: The vertebral repetitive nature causes similar appearance among different vertebrae, while pathological variation causes different appearance among the same vertebrae; (2) Field of view (FOV) challenge: The FOVs of the input MRI images are unpredictable, which exacerbates the appearance challenge because there may be no specific-appearing vertebrae to assist recognition. In this paper, we propose a Feature-cOrrelation-aware history-pReserving-sparse-Coding framEwork (FORCE) to extract highly discriminative features and alleviate these challenges. FORCE is a recognition framework with two elaborated modules: (1) A feature similarity regularization (FSR) module to constrain the features of the vertebrae with the same label (but potentially with different appearances) to be closer in the latent feature space in an Eigenmap-based regularization manner. (2) A cumulative sparse representation (CSR) module to achieve feed-forward sparse coding while preventing historical features from being erased, which leverages both the intrinsic advantages of sparse codes and the historical features for obtaining more discriminative sparse codes encoding each vertebra. These two modules are embedded into the vertebra recognition framework in a plug-and-play manner to improve feature discrimination. FORCE is trained and evaluated on a challenging dataset containing 600 MRI images. The evaluation results show that FORCE achieves high performance in vertebra recognition and outperforms other state-of-the-art methods.

Reliability Analysis of Vertebral Landmark Labelling on Lumbar Spine X-ray Images

Vertebral landmark labelling on X-ray images is important for objective and quantitative diagnosis. Most studies related to the reliability of labelling focus on the Cobb angle, and it is difficult to find studies describing landmark point locations. Since points are the most fundamental geometric feature that can generate lines and angles, the assessment of landmark point locations is essential. The aim of this study is to provide a reliability analysis of landmark points and vertebral endplate lines with a large number of lumbar spine X-ray images. A total of 1000 pairs of anteroposterior and lateral view lumbar spine images were prepared, and 12 manual medicine experts participated in the labelling process as raters. A standard operating procedure (SOP) was proposed by consensus of the raters based on manual medicine and provided guidelines for reducing sources of error in landmark labelling. High intraclass correlation coefficients ranging from 0.934 to 0.991 verified the reliability of the labelling process using the proposed SOP. We also presented means and standard deviations of measurement errors, which could be a valuable reference for evaluating both automated landmark detection algorithms and manual labelling by experts.

Emerging trends and research foci of deep learning in spine: bibliometric and visualization study

As the cognition of spine develops, deep learning (DL) emerges as a powerful tool with tremendous potential for advancing research in this field. To provide a comprehensive overview of DL-spine research, our study utilized bibliometric and visual methods to retrieve relevant articles from the Web of Science database. VOSviewer and CiteSpace were primarily used for literature measurement and knowledge graph analysis. A total of 273 studies focusing on deep learning in the spine, with a combined total of 2302 citations, were retrieved. Additionally, the overall number of articles published on this topic demonstrated a continuous upward trend. China was the country with the highest number of publications, whereas the USA had the most citations. The two most prominent journals were "European Spine Journal" and "Medical Image Analysis," and the most involved research area was Radiology Nuclear Medicine Medical Imaging. VOSviewer identified three visually distinct clusters: "segmentation," "area," and "neural network." Meanwhile, CiteSpace highlighted "magnetic resonance image" and "lumbar" as the keywords with the longest usage, and "agreement" and "automated detection" as the most commonly used keywords. Although the application of DL in spine is still in its infancy, its future is promising. Intercontinental cooperation, extensive application, and more interpretable algorithms will invigorate DL in the field of spine.

Automatic deep learning-based assessment of spinopelvic coronal and sagittal alignment

PURPOSE

The purpose of this study was to evaluate an artificial intelligence (AI) solution for estimating coronal and sagittal spinopelvic alignment on conventional uniplanar two-dimensional whole-spine radiograph.

MATERIAL AND METHODS

This retrospective observational study included 100 patients (35 men, 65 women) with a median age of 14 years (IQR: 13, 15.25; age range: 3-64 years) who underwent conventional uniplanar two-dimensional whole-spine radiograph in standing position between January and July 2022. Ten most commonly used spinopelvic coronal and sagittal parameters were retrospectively measured without AI by a junior radiologist and approved or adjusted by a senior musculoskeletal radiologist to reach final measurements. Final measurements were used as the ground truth to assess AI performance for each parameter. AI performances were estimated using mean absolute errors (MAE), intraclass correlation coefficient (ICCs), and accuracy for selected clinically relevant thresholds. Readers visually classified AI outputs to assess reliability at a patient-level.

RESULTS

AI solution showed excellent consistency without bias in coronal (ICCs ≥ 0.95; MAE ≤ 2.9° or 1.97 mm) and sagittal (ICCs ≥ 0.85; MAE ≤ 4.4° or 2.7 mm) spinopelvic evaluation, except for kyphosis (ICC = 0.58; MAE = 8.7°). AI accuracy to classify low Cobb angle, severe scoliosis or frontal pelvic asymmetry was 91% (95% CI: 85-96), 99% (95% CI: 97-100) and 94% (95% CI: 89-98), respectively. Overall, AI provided reliable measurements in 72/100 patients (72%).

CONCLUSION

The AI solution used in this study for combined coronal and sagittal spinopelvic balance assessment provides results consistent with those of a senior musculoskeletal radiologist, and shows potential benefit for reducing workload in future routine implementation.

Deep Learning for Automatic Hyoid Tracking in Videofluoroscopic Swallow Studies

The hyoid bone excursion is one of the most important gauges of larynx elevation in swallowing, contributing to airway protection and bolus passage into the esophagus. However, the implications of various parameters of hyoid bone excursion, such as the horizontal or vertical displacement and velocity, remain elusive and raise the need for a tool providing automatic kinematics analysis. Several conventional and deep learning-based models have been applied automatically to track the hyoid bone, but previous methods either require partial manual localization or do not transform the trajectory by anatomic axis. This work describes a convolutional neural network-based algorithm featuring fully automatic hyoid bone localization and tracking and spine axis determination. The algorithm automatically estimates the hyoid bone trajectory and calculates several physical quantities, including the average velocity and displacement in horizontal or vertical anatomic axis. The model was trained in a dataset of 365 videos of videofluoroscopic swallowing from 189 patients in a tertiary medical center and tested using 44 videos from 44 patients with different dysphagia etiologies. The algorithm showed high detection rates for the hyoid bone. The results showed excellent inter-rater reliability for hyoid bone detection, good-to-excellent inter-rater reliability for calculating the maximal displacement and the average velocity of the hyoid bone in horizontal or vertical directions, and moderate-to-good reliability in calculating the average velocity in horizontal direction. The proposed algorithm allows for complete automatic kinematic analysis of hyoid bone excursion, providing a versatile tool with high potential for clinical applications.

Radiographic findings involved in knee osteoarthritis progression are associated with pain symptom frequency and baseline disease severity: a population-level analysis using deep learning

PURPOSE

To (1) develop a deep-learning (DL) algorithm capable of producing limb-length and knee-alignment measurements, and (2) determine the association between limb-length discrepancy (LLD), coronal-plane alignment, osteoarthritis (OA) severity, and patient-reported knee pain.

METHODS

A multicenter, prospective patient cohort from the Osteoarthritis Initiative between 2004 and 2015 with full-limb standing radiographs at 12 month follow-up was included. A convolutional neural network was developed to automate measurements of the hip-knee-ankle (HKA) angle, femur, and tibia lengths, and LLD. At 12 month follow-up, patients reported their frequency of knee pain since enrollment and current level of knee pain.

RESULTS

A total of 1011 patients (2022 knees, 52.3% female) with an average age of 61.2 ± 9.0 years were included. The algorithm performed 12,312 measurements in 5.4 h. ICC values of HKA and LLD ranged between 0.87 and 1.00 when compared against trained radiologist measurements. Knees producing pain most days of the month were significantly more varus (mean HKA:- 3.9° ± 2.8°) or valgus (mean HKA:2.8° ± 2.3°) compared to knees that did not produce any pain (p < 0.05). In varus knees, those producing pain on most days were part of the shorter limb compared to nonpainful knees (p < 0.05). Baseline Kellgren-Lawrence grade was significantly associated with HKA magnitude, LLD, and pain frequency at 12 month follow-up (p < 0.05 all).

CONCLUSION

A higher frequency of knee pain was associated with more severe coronal plane deformity, with valgus deviation being one degree less than varus on average, suggesting that the knee tolerates less valgus deformation before symptoms become more consistent. Knee pain frequency was also associated with greater LLD and baseline KL grade, suggesting an association between radiographically apparent joint degeneration and pain frequency.

LEVEL OF EVIDENCE

IV case series.

Automated measurement of spine indices on axial MR images for lumbar spinal stenosis diagnosis using segmentation-guided regression network

PURPOSE

Automated measurement of spine indices on axial magnetic resonance (MR) images plays a significant role in lumbar spinal stenosis diagnosis. Existing direct spine indices measurement approaches fail to explicitly focus on the task-specific region or feature channel with the additional information for guiding. We aim to achieve accurate spine indices measurement by introducing the guidance of the segmentation task.

METHODS

In this paper, we propose a segmentation-guided regression network (SGRNet) to achieve automated spine indices measurement. SGRNet consists of a segmentation path for generating the spine segmentation prediction and a regression path for producing spine indices estimation. The segmentation path is a U-Net-like network which includes a segmentation encoder and a decoder which generates multilevel segmentation features and segmentation prediction. The proposed segmentation-guided attention module (SGAM) in the regression encoder extracts the attention-aware regression feature under the guidance of the segmentation feature. Based on the attention-aware regression feature, a fully connected layer is utilized to output the accurate spine indices estimation.

RESULTS

Experiments on the open-access Lumbar Spine MRI data set show that SGRNet achieves state-of-the-art performance with a mean absolute error of 0.49 mm and mean Pearson correlation coefficient of 0.956 for four indices estimation.

CONCLUSIONS

The proposed SGAM in SGRNet is capable of improving the performance of spine indices measurement by focusing on the task-specific region and feature channel under the guidance of the segmentation task.

Part Affinity Fields and CoordConv for Detecting Landmarks of Lumbar Vertebrae and Sacrum in X-ray Images

With the prevalence of degenerative diseases due to the increase in the aging population, we have encountered many spine-related disorders. Since the spine is a crucial part of the body, fast and accurate diagnosis is critically important. Generally, clinicians use X-ray images to diagnose the spine, but X-ray images are commonly occluded by the shadows of some bones, making it hard to identify the whole spine. Therefore, recently, various deep-learning-based spinal X-ray image analysis approaches have been proposed to help diagnose the spine. However, these approaches did not consider the characteristics of frequent occlusion in the X-ray image and the properties of the vertebra shape. Therefore, based on the X-ray image properties and vertebra shape, we present a novel landmark detection network specialized in lumbar X-ray images. The proposed network consists of two stages: The first step detects the centers of the lumbar vertebrae and the upper end plate of the first sacral vertebra (S1), and the second step detects the four corner points of each lumbar vertebra and two corner points of S1 from the image obtained in the first step. We used random spine cutout augmentation in the first step to robustify the network against the commonly obscured X-ray images. Furthermore, in the second step, we used CoordConv to make the network recognize the location distribution of landmarks and part affinity fields to understand the morphological features of the vertebrae, resulting in more accurate landmark detection. The proposed network was evaluated using 304 X-ray images, and it achieved 98.02% accuracy in center detection and 8.34% relative distance error in corner detection. This indicates that our network can detect spinal landmarks reliably enough to support radiologists in analyzing the lumbar X-ray images.

Comparison of Concordance between Chuna Manual Therapy Diagnostic Methods (Palpation, X-ray, Artificial Intelligence Program) in Lumbar Spine: An Exploratory, Cross-Sectional Clinical Study

Before Chuna manual therapy (CMT), a manual therapy applied in Korean medicine, CMT spinal diagnosis using palpation or X-ray is performed. However, studies on the inter-rater concordance of CMT diagnostic methods, concordance among diagnostic methods, and standard CMT diagnostic methods are scarce. Moreover, no clinical studies have used artificial intelligence (AI) programs for X-ray image-based CMT diagnosis. Therefore, this study sought a feasible and standard CMT spinal diagnostic method and explored the clinical applicability of the CMT-AI program. One hundred participants were recruited, and the concordance within and among different diagnostic modalities was analyzed by dividing them into manual diagnosis (MD), X-ray image-based diagnosis (XRD) by experts and non-experts, and XRD using a CMT-AI program by non-experts. Regarding intra-group concordance, XRD by experts showed the highest concordance (used as a gold standard when comparing inter-group concordance), followed by XRD using the AI program, XRD by non-experts, and then MD. Comparing diagnostic results between the groups, concordance with the gold standard was the highest for XRD using the AI program, followed by XRD by non-experts, and MD. Therefore, XRD is a more reasonable CMT diagnostic method than MD. Furthermore, the clinical applicability of the CMT-AI program is high.

Sagittal intervertebral rotational motion: a deep learning-based measurement on flexion–neutral–extension cervical lateral radiographs

Background

The analysis of sagittal intervertebral rotational motion (SIRM) can provide important information for the evaluation of cervical diseases. Deep learning has been widely used in spinal parameter measurements, however, there are few investigations on spinal motion analysis. The purpose of this study is to develop a deep learning-based model for fully automated measurement of SIRM based on flexion–neutral–extension cervical lateral radiographs and to evaluate its applicability for the flexion–extension (F/E), flexion–neutral (F/N), and neutral–extension (N/E) motion analysis.

Methods

A total of 2796 flexion, neutral, and extension cervical lateral radiographs from 932 patients were analyzed. Radiographs from 100 patients were randomly selected as the test set, and those from the remaining 832 patients were used for training and validation. Landmarks were annotated for measuring SIRM at five segments from C2/3 to C6/7 on F/E, F/N, and N/E motion. High-Resolution Net (HRNet) was used as the main structure to train the landmark detection network. Landmark performance was assessed according to the percentage of correct key points (PCK) and mean of the percentage of correct key points (MPCK). Measurement performance was evaluated by intra-class correlation coefficient (ICC), Pearson correlation coefficient, mean absolute error (MAE), root mean square error (RMSE), and Bland-Altman plots.

Results

At a 2-mm distance threshold, the PCK for the model ranged from 94 to 100%. Compared with the reference standards, the model showed high accuracy for SIRM measurements for all segments on F/E and F/N motion. On N/E motion, the model provided reliable measurements from C3/4 to C6/7, but not C2/3. Compared with the radiologists’ measurements, the model showed similar performance to the radiologists.

Conclusions

The developed model can automatically measure SIRM on flexion–neutral–extension cervical lateral radiographs and showed comparable performance with radiologists. It may provide rapid, accurate, and comprehensive information for cervical motion analysis.

The application of artificial intelligence in spine surgery

Due to its obvious advantages in processing big data and image information, the combination of artificial intelligence and medical care may profoundly change medical practice and promote the gradual transition from traditional clinical care to precision medicine mode. In this artical, we reviewed the relevant literatures and found that artificial intelligence was widely used in spine surgery. The application scenarios included etiology, diagnosis, treatment, postoperative prognosis and decision support systems of spinal diseases. The shift to artificial intelligence model in medicine constantly improved the level of doctors' diagnosis and treatment and the development of orthopedics.

Automatic recognition of whole-spine sagittal alignment and curvature analysis through a deep learning technique

PURPOSE

Artificial intelligence based on deep learning (DL) approaches enables the automatic recognition of anatomic landmarks and subsequent estimation of various spinopelvic parameters. The locations of inflection points (IPs) and apices (APs) in whole-spine lateral radiographs could be mathematically determined by a fully automatic spinal sagittal curvature analysis system.

METHODS

We developed a DL model for automatic spinal curvature analysis of whole-spine lateral plain radiographs by using 1800 annotated images of various spinal disease etiologies. The DL model comprised a landmark localizer to detect 25 vertebral landmarks and a numerical algorithm for the generation of an individualized spinal sagittal curvature. The characteristics of the spinal curvature, including the IPs, APs, and curvature angle, could thus be analyzed using mathematical definitions. The localization error of each landmark was calculated from the predictions of 300 test images to evaluate the performance of the landmark localizer. The interrater reliability among a senior orthopedic surgeon, a radiologist, and the DL model was assessed using the intraclass correlation coefficient (ICC).

RESULTS

The accuracy of the landmark localizer was within an acceptable range (median error: 1.7-4.1 mm), and the interrater reliabilities between the proposed DL model and each expert were good to excellent (all ICCs > 0.85) for the measurement of spinal curvature characteristics.

CONCLUSION

The interrater reliability between the proposed DL model and human experts was good to excellent in predicting the locations of IPs, APs, and curvature angles. Future applications should be explored to validate this system and improve its clinical efficiency.

Spinopelvic measurements of sagittal balance with deep learning: systematic review and critical evaluation

PURPOSE

To summarize and critically evaluate the existing studies for spinopelvic measurements of sagittal balance that are based on deep learning (DL).

METHODS

Three databases (PubMed, WoS and Scopus) were queried for records using keywords related to DL and measurement of sagittal balance. After screening the resulting 529 records that were augmented with specific web search, 34 studies published between 2017 and 2022 were included in the final review, and evaluated from the perspective of the observed sagittal spinopelvic parameters, properties of spine image datasets, applied DL methodology and resulting measurement performance.

RESULTS

Studies reported DL measurement of up to 18 different spinopelvic parameters, but the actual number depended on the image field of view. Image datasets were composed of lateral lumbar spine and whole spine X-rays, biplanar whole spine X-rays and lumbar spine magnetic resonance cross sections, and were increasing in size or enriched by augmentation techniques. Spinopelvic parameter measurement was approached either by landmark detection or structure segmentation, and U-Net was the most frequently applied DL architecture. The latest DL methods achieved excellent performance in terms of mean absolute error against reference manual measurements (~ 2° or ~ 1 mm).

CONCLUSION

Although the application of relatively complex DL architectures resulted in an improved measurement accuracy of sagittal spinopelvic parameters, future methods should focus on multi-institution and multi-observer analyses as well as uncertainty estimation and error handling implementations for integration into the clinical workflow. Further advances will enhance the predictive analytics of DL methods for spinopelvic parameter measurement.

LEVEL OF EVIDENCE I

Diagnostic: individual cross-sectional studies with the consistently applied reference standard and blinding.

Deep Learning Algorithms Improve the Detection of Subtle Lisfranc Malalignments on Weightbearing Radiographs

BACKGROUND

Detection of Lisfranc malalignment leading to the instability of the joint, particularly in subtle cases, has been a concern for foot and ankle care providers. X-ray radiographs are the mainstay in the diagnosis of these injuries; thus, improving the performance of clinicians in interpreting radiographs can noticeably affect the quality of health care in these patients. Here we assessed the performance of deep learning algorithms on weightbearing radiographs for detection of Lisfranc joint malalignment in patients with Lisfranc instability.

METHODS

In a retrospective study, 640 patients with Lisfranc malalignment leading to instability were recruited plus 640 individuals with uninjured feet and healthy Lisfranc joint as the control group. All radiographs were screened by orthopaedic surgeons. Two deep learning models were trained, validated, and tested (in a ratio 80:10:10) using a single-view (anteroposterior) and 3-view (anteroposterior, lateral, oblique) radiographs. The performances of the models were reported as sensitivity, specificity, positive and negative predictive values, accuracy, F score, and area under the curve (AUC).

RESULTS

No significant differences were observed between the patients and the controls regarding age, gender, race, and body mass index. The best deep learning algorithm outperformed our human interpreters (<1% vs ~10% misdiagnosis), 94.8% sensitivity, 96.9% specificity, 98.6% accuracy, 95.8% F score, and 99.4% AUC.

CONCLUSION

Deep learning methods have shown promising potential in acting as an assistant interpreter of radiographic images in patients with Lisfranc malalignment. Developing these algorithms can hasten and improve the accuracy of diagnosis and reduce further costs and burdens on the patients and health care system.

LEVEL OF EVIDENCE

Level III, case-control Machine Learning study.

A fresh look at spinal alignment and deformities: Automated analysis of a large database of 9832 biplanar radiographs

We developed and used a deep learning tool to process biplanar radiographs of 9,832 non-surgical patients suffering from spinal deformities, with the aim of reporting the statistical distribution of radiological parameters describing the spinal shape and the correlations and interdependencies between them. An existing tool able to automatically perform a three-dimensional reconstruction of the thoracolumbar spine has been improved and used to analyze a large set of biplanar radiographs of the trunk. For all patients, the following parameters were calculated: spinopelvic parameters; lumbar lordosis; mismatch between pelvic incidence and lumbar lordosis; thoracic kyphosis; maximal coronal Cobb angle; sagittal vertical axis; T1-pelvic angle; maximal vertebral rotation in the transverse plane. The radiological parameters describing the sagittal alignment were found to be highly interrelated with each other, as well as dependent on age, while sex had relatively minor but statistically significant importance. Lumbar lordosis was associated with thoracic kyphosis, pelvic incidence and sagittal vertical axis. The pelvic incidence-lumbar lordosis mismatch was found to be dependent on the pelvic incidence and on age. Scoliosis had a distinct association with the sagittal alignment in adolescent and adult subjects. The deep learning-based tool allowed for the analysis of a large imaging database which would not be reasonably feasible if performed by human operators. The large set of results will be valuable to trigger new research questions in the field of spinal deformities, as well as to challenge the current knowledge.

Deformable 3D-2D image registration and analysis of global spinal alignment in long-length intraoperative spine imaging

BACKGROUND

Spinal deformation during surgical intervention (caused by patient positioning and/or correction of malalignment) confounds conventional navigation due to assumptions of rigid transformation. Moreover, the ability to accurately quantify spinal alignment in the operating room would provide assessment of the surgical product via metrics that correlate with clinical outcome.

PURPOSE

A method for deformable 3D-2D registration of preoperative CT to intraoperative long-length tomosynthesis images is reported for accurate 3D evaluation of device placement in the presence of spinal deformation and automated evaluation of global spinal alignment (GSA).

METHODS

Long-length tomosynthesis ("Long Film", LF) images were acquired using an O-arm™ imaging system (Medtronic, Minneapolis USA). A deformable 3D-2D patient registration was developed using multi-scale masking (proceeding from the full-length image to local subvolumes about each vertebra) to transform vertebral labels and planning information from preoperative CT to the LF images. Automatic measurement of GSA [Main Thoracic Kyphosis (MThK) and Lumbar Lordosis (LL)] was obtained using a spline fit to registered labels. The "Known-Component Registration" (KC-Reg) method for device registration was adapted to the multi-scale process for 3D device localization from orthogonal LF images. The multi-scale framework was evaluated using a deformable spine phantom in which pedicle screws were inserted, and deformations were induced over a range in LL ∼25-80°. Further validation was carried out in a cadaver study with implanted pedicle screws and a similar range of spinal deformation. The accuracy of patient and device registration was evaluated in terms of 3D translational error and target registration error (TRE), respectively, and the accuracy of automatic GSA measurements were compared to manual annotation.

RESULTS

Phantom studies demonstrated accurate registration via the multi-scale framework for all vertebral levels in both the neutral and deformed spine: median (interquartile range, IQR) patient registration error was 1.1 mm (0.7-1.9 mm IQR). Automatic measures of MThK and LL agreed with manual delineation within -1.1° ± 2.2° and 0.7° ± 2.0° (mean and standard deviation), respectively. Device registration error was 0.7 mm (0.4-1.0 mm IQR) at the screw tip and 0.9° (1.0°-1.5°) about the screw trajectory. Deformable 3D-2D registration significantly outperformed conventional rigid registration (p < 0.05), which exhibited device registration error of 2.1 mm (0.8-4.1 mm) and 4.1° (1.2°-9.5°). Cadaver studies verified performance under realistic conditions, demonstrating patient registration error of 1.6 mm (0.9-2.1 mm); MThK within -4.2° ± 6.8° and LL within 1.7° ± 3.5°; and device registration error of 0.8 mm (0.5-1.9 mm) and 0.7° (0.4°-1.2°) for the multi-scale deformable method, compared to 2.5 mm (1.0-7.9 mm) and 2.3° (1.6°-8.1°) for rigid registration (p < 0.05).

CONCLUSION

The deformable 3D-2D registration framework leverages long-length intraoperative imaging to achieve accurate patient and device registration over extended lengths of the spine (up to 64 cm) even with strong anatomical deformation. The method offers a new means for quantitative validation of spinal correction (intraoperative GSA measurement) and 3D verification of device placement in comparison to preoperative images and planning data. This article is protected by copyright. All rights reserved.

Study on Automatic Multi-Classification of Spine Based on Deep Learning and Postoperative Infection Screening

The preoperative qualitative and hierarchical diagnosis of intervertebral foramen stenosis is very important for clinicians to explore the effect of multimodal analgesia nursing on pain control after spinal fusion and to formulate treatment strategies and patients' health recovery. However, there are still many problems in this aspect, and there is a lack of relevant research and effective methods to assist clinicians in diagnosis. Therefore, to improve the accuracy of computer-aided diagnosis of intervertebral foramen stenosis and the work efficiency of doctors, a deep learning automatic grading algorithm of intervertebral foramen stenosis image is proposed in this study. The image of intervertebral foramen was extracted from the MRI image of sagittal spine, and the image was preprocessed. 86 patients with spinal fusion treated in our hospital, specifically from May 2018 to May 2020, were randomly divided into the control group (routine analgesic nursing) and the multimodal group (multimodal analgesic nursing), with 43 cases in each group. The pain control effect and satisfaction of the two groups were observed. The results after multimodal analgesia nursing show that the VASs of the multimodal group at different time points were significantly lower than those of the control group (P < 0.05); the satisfaction score of pain control in the multimodal group was significantly higher than that in the control group (P < 0.05). Multimodal analgesia nursing for patients undergoing spinal fusion can effectively reduce the degree of postoperative pain and improve the effect of pain control and satisfaction with pain control, which is worthy of promotion.

Evaluation of Deep Learning-Based Automated Detection of Primary Spine Tumors on MRI Using the Turing Test

Background

Recently, the Turing test has been used to investigate whether machines have intelligence similar to humans. Our study aimed to assess the ability of an artificial intelligence (AI) system for spine tumor detection using the Turing test.

Methods

Our retrospective study data included 12179 images from 321 patients for developing AI detection systems and 6635 images from 187 patients for the Turing test. We utilized a deep learning-based tumor detection system with Faster R-CNN architecture, which generates region proposals by Region Proposal Network in the first stage and corrects the position and the size of the bounding box of the lesion area in the second stage. Each choice question featured four bounding boxes enclosing an identical tumor. Three were detected by the proposed deep learning model, whereas the other was annotated by a doctor; the results were shown to six doctors as respondents. If the respondent did not correctly identify the image annotated by a human, his answer was considered a misclassification. If all misclassification rates were &gt;30%, the respondents were considered unable to distinguish the AI-detected tumor from the human-annotated one, which indicated that the AI system passed the Turing test.

Results

The average misclassification rates in the Turing test were 51.2% (95% CI: 45.7%–57.5%) in the axial view (maximum of 62%, minimum of 44%) and 44.5% (95% CI: 38.2%–51.8%) in the sagittal view (maximum of 59%, minimum of 36%). The misclassification rates of all six respondents were &gt;30%; therefore, our AI system passed the Turing test.

Conclusion

Our proposed intelligent spine tumor detection system has a similar detection ability to annotation doctors and may be an efficient tool to assist radiologists or orthopedists in primary spine tumor detection.

μMatch: 3D Shape Correspondence for Biological Image Data

Modern microscopy technologies allow imaging biological objects in 3D over a wide range of spatial and temporal scales, opening the way for a quantitative assessment of morphology. However, establishing a correspondence between objects to be compared, a first necessary step of most shape analysis workflows, remains challenging for soft-tissue objects without striking features allowing them to be landmarked. To address this issue, we introduce the μMatch 3D shape correspondence pipeline. μMatch implements a state-of-the-art correspondence algorithm initially developed for computer graphics and packages it in a streamlined pipeline including tools to carry out all steps from input data pre-processing to classical shape analysis routines. Importantly, μMatch does not require any landmarks on the object surface and establishes correspondence in a fully automated manner. Our open-source method is implemented in Python and can be used to process collections of objects described as triangular meshes. We quantitatively assess the validity of μMatch relying on a well-known benchmark dataset and further demonstrate its reliability by reproducing published results previously obtained through manual landmarking.

Virtual Surgical Planning: Modeling from the Present to the Future

Virtual surgery planning is a non-invasive procedure, which uses digital clinical data for diagnostic, procedure selection and treatment planning purposes, including the forecast of potential outcomes. The technique begins with 3D data acquisition, using various methods, which may or may not utilize ionizing radiation, such as 3D stereophotogrammetry, 3D cone-beam CT scans, etc. Regardless of the imaging technique selected, landmark selection, whether it is manual or automated, is the key to transforming clinical data into objects that can be interrogated in virtual space. As a prerequisite, the data require alignment and correspondence such that pre- and post-operative configurations can be compared in real and statistical shape space. In addition, these data permit predictive modeling, using either model-based, data-based or hybrid modeling. These approaches provide perspectives for the development of customized surgical procedures and medical devices with accuracy, precision and intelligence. Therefore, this review briefly summarizes the current state of virtual surgery planning.

Deep learning approach for guiding three-dimensional computed tomography reconstruction of lower limbs for robotically-assisted total knee arthroplasty

BACKGROUND

Robotic-assisted total knee arthroplasty (TKA) was performed to promote the accuracy of bone resection and mechanical alignment. Among these TKA system procedures, 3D reconstruction of CT data of lower limbs consumes significant manpower. Artificial intelligence (AI) algorithms applying deep learning has been proved efficient in automated identification and visual processing.

METHODS

CT data of a total of 200 lower limbs scanning were used for AI-based 3D model construction and CT data of 20 lower limbs scanning were utilised for verification.

RESULTS

We showed that the performance of an AI-guided 3D reconstruction of CT data of lower limbs for robotic-assisted TKA was similar to that of the operator-based approach. The time of 3D lower limb model construction using AI was 4.7 min. AI-based 3D models can be used for surgical planning.

CONCLUSION

AI was used for the first time to guide the 3D reconstruction of CT data of lower limbs for facilitating robotic-assisted TKA. Incorporation of AI in 3D model reconstruction before TKA might reduce the workload of radiologists.