Robotic Assistance and Intervention in Spine Surgery

Vertebrae Labeling via End-to-End Integral Regression Localization and Multi-Label Classification Network

Accurate identification and localization of the vertebrae in CT scans is a critical and standard pre-processing step for clinical spinal diagnosis and treatment. Existing methods are mainly based on the integration of multiple neural networks, and most of them use heatmaps to locate the vertebrae's centroid. However, the process of obtaining vertebrae's centroid coordinates using heatmaps is non-differentiable, so it is impossible to train the network to label the vertebrae directly. Therefore, for end-to-end differential training of vertebrae coordinates on CT scans, a robust and accurate automatic vertebral labeling algorithm is proposed in this study. First, a novel end-to-end integral regression localization and multi-label classification network is developed, which can capture multi-scale features and also utilize the residual module and skip connection to fuse the multi-level features. Second, to solve the problem that the process of finding coordinates is non-differentiable and the spatial structure of location being destroyed, an integral regression module is used in the localization network. It combines the advantages of heatmaps representation and direct regression coordinates to achieve end-to-end training and can be compatible with any key point detection methods of medical images based on heatmaps. Finally, multi-label classification of vertebrae is carried out to improve the identification rate, which uses bidirectional long short-term memory (Bi-LSTM) online to enhance the learning of long contextual information of vertebrae. The proposed method is evaluated on a challenging data set, and the results are significantly better than state-of-the-art methods (identification rate is 91.1% and the mean localization error is 2.2 mm). The method is evaluated on a new CT data set, and the results show that our method has good generalization.

3D vertebrae labeling in spine CT: an accurate, memory-efficient (Ortho2D) framework

Purpose.Accurate localization and labeling of vertebrae in computed tomography (CT) is an important step toward more quantitative, automated diagnostic analysis and surgical planning. In this paper, we present a framework (called Ortho2D) for vertebral labeling in CT in a manner that is accurate and memory-efficient.Methods. Ortho2D uses two independent faster R-convolutional neural network networks to detect and classify vertebrae in orthogonal (sagittal and coronal) CT slices. The 2D detections are clustered in 3D to localize vertebrae centroids in the volumetric CT and classify the region (cervical, thoracic, lumbar, or sacral) and vertebral level. A post-process sorting method incorporates the confidence in network output to refine classifications and reduce outliers. Ortho2D was evaluated on a publicly available dataset containing 302 normal and pathological spine CT images with and without surgical instrumentation. Labeling accuracy and memory requirements were assessed in comparison to other recently reported methods. The memory efficiency of Ortho2D permitted extension to high-resolution CT to investigate the potential for further boosts to labeling performance.Results. Ortho2D achieved overall vertebrae detection accuracy of 97.1%, region identification accuracy of 94.3%, and individual vertebral level identification accuracy of 91.0%. The framework achieved 95.8% and 83.6% level identification accuracy in images without and with surgical instrumentation, respectively. Ortho2D met or exceeded the performance of previously reported 2D and 3D labeling methods and reduced memory consumption by a factor of ∼50 (at 1 mm voxel size) compared to a 3D U-Net, allowing extension to higher resolution datasets than normally afforded. The accuracy of level identification increased from 80.1% (for standard/low resolution CT) to 95.1% (for high-resolution CT).Conclusions. The Ortho2D method achieved vertebrae labeling performance that is comparable to other recently reported methods with significant reduction in memory consumption, permitting further performance boosts via application to high-resolution CT.

Joint Vertebrae Identification and Localization in Spinal CT Images by Combining Short- and Long-Range Contextual Information

Automatic vertebrae identification and localization from arbitrary computed tomography (CT) images is challenging. Vertebrae usually share similar morphological appearance. Because of pathology and the arbitrary field-of-view of CT scans, one can hardly rely on the existence of some anchor vertebrae or parametric methods to model the appearance and shape. To solve the problem, we argue that: 1) one should make use of the short-range contextual information, such as the presence of some nearby organs (if any), to roughly estimate the target vertebrae; and 2) due to the unique anatomic structure of the spine column, vertebrae have fixed sequential order, which provides the important long-range contextual information to further calibrate the results. We propose a robust and efficient vertebrae identification and localization system that can inherently learn to incorporate both the short- and long-range contextual information in a supervised manner. To this end, we develop a multi-task 3-D fully convolutional neural network to effectively extract the short-range contextual information around the target vertebrae. For the long-range contextual information, we propose a multi-task bidirectional recurrent neural network to encode the spatial and contextual information among the vertebrae of the visible spine column. We demonstrate the effectiveness of the proposed approach on a challenging data set, and the experimental results show that our approach outperforms the state-of-the-art methods by a significant margin.