Long-length tomosynthesis and 3D-2D registration for intraoperative assessment of spine instrumentation

Deformable registration of preoperative MR and intraoperative long-length tomosynthesis images for guidance of spine surgery via image synthesis

PURPOSE

Improved integration and use of preoperative imaging during surgery hold significant potential for enhancing treatment planning and instrument guidance through surgical navigation. Despite its prevalent use in diagnostic settings, MR imaging is rarely used for navigation in spine surgery. This study aims to leverage MR imaging for intraoperative visualization of spine anatomy, particularly in cases where CT imaging is unavailable or when minimizing radiation exposure is essential, such as in pediatric surgery.

METHODS

This work presents a method for deformable 3D-2D registration of preoperative MR images with a novel intraoperative long-length tomosynthesis imaging modality (viz., Long-Film [LF]). A conditional generative adversarial network is used to translate MR images to an intermediate bone image suitable for registration, followed by a model-based 3D-2D registration algorithm to deformably map the synthesized images to LF images. The algorithm's performance was evaluated on cadaveric specimens with implanted markers and controlled deformation, and in clinical images of patients undergoing spine surgery as part of a large-scale clinical study on LF imaging.

RESULTS

The proposed method yielded a median 2D projection distance error of 2.0 mm (interquartile range [IQR]: 1.1-3.3 mm) and a 3D target registration error of 1.5 mm (IQR: 0.8-2.1 mm) in cadaver studies. Notably, the multi-scale approach exhibited significantly higher accuracy compared to rigid solutions and effectively managed the challenges posed by piecewise rigid spine deformation. The robustness and consistency of the method were evaluated on clinical images, yielding no outliers on vertebrae without surgical instrumentation and 3% outliers on vertebrae with instrumentation.

CONCLUSIONS

This work constitutes the first reported approach for deformable MR to LF registration based on deep image synthesis. The proposed framework provides access to the preoperative annotations and planning information during surgery and enables surgical navigation within the context of MR images and/or dual-plane LF images.

Facial augmented reality based on hierarchical optimization of similarity aspect graph

BACKGROUND

The existing face matching method requires a point cloud to be drawn on the real face for registration, which results in low registration accuracy due to the irregular deformation of the patient's skin that makes the point cloud have many outlier points.

METHODS

This work proposes a non-contact pose estimation method based on similarity aspect graph hierarchical optimization. The proposed method constructs a distance-weighted and triangular-constrained similarity measure to describe the similarity between views by automatically identifying the 2D and 3D feature points of the face. A mutual similarity clustering method is proposed to construct a hierarchical aspect graph with 3D pose as nodes. A Monte Carlo tree search strategy is used to search the hierarchical aspect graph for determining the optimal pose of the facial 3D model, so as to realize the accurate registration of the facial 3D model and the real face.

RESULTS

The proposed method was used to conduct accuracy verification experiments on the phantoms and volunteers, which were compared with four advanced pose calibration methods. The proposed method obtained average fusion errors of 1.13 ± 0.20 mm and 0.92 ± 0.08 mm in head phantom and volunteer experiments, respectively, which exhibits the best fusion performance among all comparison methods.

CONCLUSIONS

Our experiments proved the effectiveness of the proposed pose estimation method in facial augmented reality.

Comparison of 3D Bone Position Estimation Using QR Code and Metal Bead Markers

To improve the accuracy of a 3D bone position estimation system that displays 3D images in response to changes in the position of fluoroscopic images, modified markers using quick response (QR) codes were developed. The aims of this study were to assess the accuracy of the estimated bone position on 3D images with reference to QR code markers on fluoroscopic images and to compare its accuracy with metal bead markers. Bone positions were estimated from reference points on a fluoroscopic image compared with those on a 3D image. The positional relationships of QR code and metal bead markers on the fluoroscopic image were compared with those on the 3D image in order to establish whether a 3D image may be drawn by tracking positional changes in radius models. Differences were investigated by comparing the distance between markers on the fluoroscopic image and that on the 3D image, which was projected on the monitor. The error ratio, which was defined as the difference in the measurement between the fluoroscopic and 3D images divided by the fluoroscopic measurement, was compared between QR code and metal bead markers. Error ratios for the QR code markers were 5.0 ± 2.0%, 6.4 ± 7.6%, and 1.0 ± 0.8% in the anterior–posterior view, ulnar side lateral view, and posterior–anterior view, respectively. Error ratios for the metal bead markers were 1.3 ± 1.7%, 13.8 ± 14.5%, and 4.7 ± 5.7% in the anterior–posterior view, ulnar side lateral view, and posterior–anterior view, respectively. The error ratio for the metal bead markers was smaller in the initial position (p < 0.01). However, the error ratios for the QR code markers were smaller in the lateral position and the posterior–anterior position (p < 0.05). In QR code marker tracking, tracking was successful even with discontinuous images. The accuracy of a 3D bone position estimation was increased by using the QR code marker system. QR code marker tracking facilitates real-time comparisons of dynamic changes in preoperative 3D and intraoperative fluoroscopic images.

Multi-perspective region-based CNNs for vertebrae labeling in intraoperative long-length images

PURPOSE

Effective aggregation of intraoperative x-ray images that capture the patient anatomy from multiple view-angles has the potential to enable and improve automated image analysis that can be readily performed during surgery. We present multi-perspective region-based neural networks that leverage knowledge of the imaging geometry for automatic vertebrae labeling in Long-Film images - a novel tomographic imaging modality with an extended field-of-view for spine imaging.

METHOD

A multi-perspective network architecture was designed to exploit small view-angle disparities produced by a multi-slot collimator and consolidate information from overlapping image regions. A second network incorporates large view-angle disparities to jointly perform labeling on images from multiple views (viz., AP and lateral). A recurrent module incorporates contextual information and enforce anatomical order for the detected vertebrae. The three modules are combined to form the multi-view multi-slot (MVMS) network for labeling vertebrae using images from all available perspectives. The network was trained on images synthesized from 297 CT images and tested on 50 AP and 50 lateral Long-Film images acquired from 13 cadaveric specimens. Labeling performance of the multi-perspective networks was evaluated with respect to the number of vertebrae appearances and presence of surgical instrumentation.

RESULTS

The MVMS network achieved an F1 score of >96% and an average vertebral localization error of 3.3 mm, with 88.3% labeling accuracy on both AP and lateral images - (15.5% and 35.0% higher than conventional Faster R-CNN on AP and lateral views, respectively). Aggregation of multiple appearances of the same vertebra using the multi-slot network significantly improved the labeling accuracy (p < 0.05). Using the multi-view network, labeling accuracy on the more challenging lateral views was improved to the same level as that of the AP views. The approach demonstrated robustness to the presence of surgical instrumentation, commonly encountered in intraoperative images, and achieved comparable performance in images with and without instrumentation (88.9% vs. 91.2% labeling accuracy).

CONCLUSION

The MVMS network demonstrated effective multi-perspective aggregation, providing means for accurate, automated vertebrae labeling during spine surgery. The algorithms may be generalized to other imaging tasks and modalities that involve multiple views with view-angle disparities (e.g., bi-plane radiography). Predicted labels can help avoid adverse events during surgery (e.g., wrong-level surgery), establish correspondence with labels in preoperative modalities to facilitate image registration, and enable automated measurement of spinal alignment metrics for intraoperative assessment of spinal curvature.

Extended Intraoperative Longitudinal 3-Dimensional Cone Beam Computed Tomography Imaging With a Continuous Multi-Turn Reverse Helical Scan

OBJECTIVES

Cone beam computed tomography (CBCT) imaging is becoming an indispensable intraoperative tool; however, the current field of view prevents visualization of long anatomical sites, limiting clinical utility. Here, we demonstrate the longitudinal extension of the intraoperative CBCT field of view using a multi-turn reverse helical scan and assess potential clinical utility in interventional procedures.

MATERIALS AND METHODS

A fixed-room robotic CBCT imaging system, with additional real-time control, was used to implement a multi-turn reverse helical scan. The scan consists of C-arm rotation, through a series of clockwise and anticlockwise rotations, combined with simultaneous programmed table translation. The motion properties and geometric accuracy of the multi-turn reverse helical imaging trajectory were examined using a simple geometric phantom. To assess potential clinical utility, a pedicle screw posterior fixation procedure in the thoracic spine from T1 to T12 was performed on an ovine cadaver. The multi-turn reverse helical scan was used to provide postoperative assessment of the screw insertion via cortical breach grading and mean screw angle error measurements (axial and sagittal) from 2 observers. For all screw angle measurements, the intraclass correlation coefficient was calculated to determine observer reliability.

RESULTS

The multi-turn reverse helical scans took 100 seconds to complete and increased the longitudinal coverage by 370% from 17 cm to 80 cm. Geometric accuracy was examined by comparing the measured to actual dimensions (0.2 ± 0.1 mm) and angles (0.2 ± 0.1 degrees) of a simple geometric phantom, indicating that the multi-turn reverse helical scan provided submillimeter and degree accuracy with no distortion. During the pedicle screw procedure in an ovine cadaver, the multi-turn reverse helical scan identified 4 cortical breaches, confirmed via the postoperative CT scan. Directly comparing the screw insertion angles (n = 22) measured in the postoperative multi-turn reverse helical and CT scans revealed an average difference of 3.3 ± 2.6 degrees in axial angle and 1.9 ± 1.5 degrees in the sagittal angle from 2 expert observers. The intraclass correlation coefficient was above 0.900 for all measurements (axial and sagittal) across all scan types (conventional CT, multi-turn reverse helical, and conventional CBCT), indicating excellent reliability between observers.

CONCLUSIONS

Extended longitudinal field-of-view intraoperative 3-dimensional imaging with a multi-turn reverse helical scan is feasible on a clinical robotic CBCT imaging system, enabling long anatomical sites to be visualized in a single image, including in the presence of metal hardware.

An Experimental Study of a 3D Bone Position Estimation System Based on Fluoroscopic Images

To compare a 3D preoperative planning image and fluoroscopic image, a 3D bone position estimation system that displays 3D images in response to changes in the position of fluoroscopic images was developed. The objective of the present study was to evaluate the accuracy of the estimated position of 3D bone images with reference to fluoroscopic images. Bone positions were estimated from reference points on a fluoroscopic image compared with those on a 3D image. The four reference markers positional relationships on the fluoroscopic image were compared with those on the 3D image to evaluate whether a 3D image may be drawn by tracking positional changes in the radius model. Intra-class correlations coefficients for reference marker distances between the fluoroscopic image and 3D image were 0.98–0.99. Average differences between measured values on the fluoroscopic image and 3D bone image for each marker corresponding to the direction of the bone model were 1.1 ± 0.7 mm, 2.4 ± 1.8 mm, 1.4 ± 0.8 mm, and 2.0 ± 1.6 mm in the anterior-posterior view, ulnar side lateral view, posterior-anterior view, and radial side lateral view, respectively. Marker positions were more accurate in the anterior-posterior and posterior-anterior views than in the radial and ulnar side lateral views. This system helps in real-time comparison of dynamic changes in preoperative 3D and intraoperative fluoroscopy images.

Novel 2D long film imaging utility to avoid wrong level spinal surgery

Wrong-level spinal surgery (WLSS) can lead to increased morbidity, cost, and worse long-term outcomes. Current intraoperative localization methods rely on counting spinal levels from a known reference location using fluoroscopy. Miscounting from a reference is an intraoperative error that leads to WLSS, especially for those with anatomical variations. The problem is exacerbated when fluoroscopy is not able to produce images with the clarity needed to confidently count levels, a prevalent issue for obese patients.

A new feature called the “2D Long Film'' is available for the Medtronic (Minneapolis, MN) O-arm Surgical Imaging System. Using this novel technology and standard fluoroscopy, this study reports the imaging of two obese adult female patients with a body mass index of 36.9 and 42.0 undergoing transforaminal thoracic interbody fusion. Fluoroscopy images of obese patients are difficult to capture for two reasons: increased scatter and restricted field of view. This report demonstrates that 2D Long Film can improve both these issues for obese patients in need of thoracic localization. The 2D Long Film captures existing instrumentation, localization needles, and the vertebral levels in a clear single image.

We display the differences between standard fluoroscopy and the 2D Long Film for thoracic level localization, demonstrating a potential new standard of care and better visualization, leading to a less challenging vertebrae localization process, potentially mitigating WLSS risk. The quality of this new 2D Long Film feature could also reduce time in the operating room and the necessity of other visualization methods.

Deformable 3D-2D image registration and analysis of global spinal alignment in long-length intraoperative spine imaging

BACKGROUND

Spinal deformation during surgical intervention (caused by patient positioning and/or correction of malalignment) confounds conventional navigation due to assumptions of rigid transformation. Moreover, the ability to accurately quantify spinal alignment in the operating room would provide assessment of the surgical product via metrics that correlate with clinical outcome.

PURPOSE

A method for deformable 3D-2D registration of preoperative CT to intraoperative long-length tomosynthesis images is reported for accurate 3D evaluation of device placement in the presence of spinal deformation and automated evaluation of global spinal alignment (GSA).

METHODS

Long-length tomosynthesis ("Long Film", LF) images were acquired using an O-arm™ imaging system (Medtronic, Minneapolis USA). A deformable 3D-2D patient registration was developed using multi-scale masking (proceeding from the full-length image to local subvolumes about each vertebra) to transform vertebral labels and planning information from preoperative CT to the LF images. Automatic measurement of GSA [Main Thoracic Kyphosis (MThK) and Lumbar Lordosis (LL)] was obtained using a spline fit to registered labels. The "Known-Component Registration" (KC-Reg) method for device registration was adapted to the multi-scale process for 3D device localization from orthogonal LF images. The multi-scale framework was evaluated using a deformable spine phantom in which pedicle screws were inserted, and deformations were induced over a range in LL ∼25-80°. Further validation was carried out in a cadaver study with implanted pedicle screws and a similar range of spinal deformation. The accuracy of patient and device registration was evaluated in terms of 3D translational error and target registration error (TRE), respectively, and the accuracy of automatic GSA measurements were compared to manual annotation.

RESULTS

Phantom studies demonstrated accurate registration via the multi-scale framework for all vertebral levels in both the neutral and deformed spine: median (interquartile range, IQR) patient registration error was 1.1 mm (0.7-1.9 mm IQR). Automatic measures of MThK and LL agreed with manual delineation within -1.1° ± 2.2° and 0.7° ± 2.0° (mean and standard deviation), respectively. Device registration error was 0.7 mm (0.4-1.0 mm IQR) at the screw tip and 0.9° (1.0°-1.5°) about the screw trajectory. Deformable 3D-2D registration significantly outperformed conventional rigid registration (p < 0.05), which exhibited device registration error of 2.1 mm (0.8-4.1 mm) and 4.1° (1.2°-9.5°). Cadaver studies verified performance under realistic conditions, demonstrating patient registration error of 1.6 mm (0.9-2.1 mm); MThK within -4.2° ± 6.8° and LL within 1.7° ± 3.5°; and device registration error of 0.8 mm (0.5-1.9 mm) and 0.7° (0.4°-1.2°) for the multi-scale deformable method, compared to 2.5 mm (1.0-7.9 mm) and 2.3° (1.6°-8.1°) for rigid registration (p < 0.05).

CONCLUSION

The deformable 3D-2D registration framework leverages long-length intraoperative imaging to achieve accurate patient and device registration over extended lengths of the spine (up to 64 cm) even with strong anatomical deformation. The method offers a new means for quantitative validation of spinal correction (intraoperative GSA measurement) and 3D verification of device placement in comparison to preoperative images and planning data. This article is protected by copyright. All rights reserved.

Slot-scan dual-energy bone densitometry using motorized X-ray systems

PURPOSE

We investigate the feasibility of slot-scan dual-energy (DE) bone densitometry on motorized radiographic equipment. This approach will enable fast quantitative measurements of areal bone mineral density (aBMD) for opportunistic evaluation of osteoporosis.

METHODS

We investigated DE slot-scan protocols to obtain aBMD measurements at the lumbar spine (L-spine) and hip using a motorized x-ray platform capable of synchronized translation of the x-ray source and flat-panel detector (FPD). The slot dimension was 5 × 20 cm² . The DE slot views were processed as follows: (1) convolution kernel-based scatter correction, (2) unfiltered backprojection to tile the slots into long-length radiographs, and (3) projection-domain DE decomposition, consisting of an initial adipose-water decomposition in a bone-free region followed by water-CaHA decomposition with adjustment for adipose content. The accuracy and reproducibility of slot-scan aBMD measurements were investigated using a high-fidelity simulator of a robotic x-ray system (Siemens Multitom Rax) in a total of 48 body phantom realizations: four average bone density settings (cortical bone mass fraction: 10-40%), four body sizes (waist circumference, WC = 70-106 cm), and three lateral shifts of the body within the slot field of view (FOV) (centered and ±1 cm off-center). Experimental validations included: (1) x-ray test-bench feasibility study of adipose-water decomposition and (2) initial demonstration of slot-scan DE bone densitometry on the robotic x-ray system using the European Spine Phantom (ESP) with added attenuation (polymethyl methacrylate [PMMA] slabs) ranging 2 to 6 cm thick.

RESULTS

For the L-spine, the mean aBMD error across all WC settings ranged from 0.08 g/cm² for phantoms with average cortical bone fraction w_cortical  = 10% to ∼0.01 g/cm² for phantoms with w_cortical  = 40%. The L-spine aBMD measurements were fairly robust to changes in body size and positioning, e.g., coefficient of variation (CV) for L1 with w_cortical  = 30% was ∼0.034 for various WC and ∼0.02 for an obese patient (WC = 106 cm) changing lateral shift. For the hip, the mean aBMD error across all phantom configurations was about 0.07 g/cm² for a centered patient. The reproducibility of hip aBMD was slightly worse than in the L-spine (e.g., in the femoral neck, the CV with respect to changing WC was ∼0.13 for phantom realizations with w_cortical  = 30%) due to more challenging scatter estimation in the presence of an air-tissue interface within the slot FOV. The aBMD of the hip was therefore sensitive to lateral positioning of the patient, especially for obese patients: e.g., the CV with respect to patient lateral shift for femoral neck with WC = 106 cm and w_cortical  = 30% was 0.14. Empirical evaluations confirmed substantial reduction in aBMD errors with the proposed adipose estimation procedure and demonstrated robust aBMD measurements on the robotic x-ray system, with aBMD errors of ∼0.1 g/cm² across all three simulated ESP vertebrae and all added PMMA attenuator settings.

CONCLUSIONS

We demonstrated that accurate aBMD measurements can be obtained on a motorized FPD-based x-ray system using DE slot-scans with kernel-based scatter correction, backprojection-based slot view tiling, and DE decomposition with adipose correction.

Intraoperative cone-beam and slot-beam CT: 3D image quality and dose with a slot collimator on the O-arm imaging system

PURPOSE

To characterize the 3D imaging performance and radiation dose for a prototype slot-beam configuration on an intraoperative O-arm™ Surgical Imaging System (Medtronic Inc., Littleton, MA) and identify potential improvements in soft-tissue image quality for surgical interventions.

METHODS

A slot collimator was integrated with the O-arm^™ system for slot-beam axial CT. The collimator can be automatically actuated to provide 1.2° slot-beam longitudinal collimation. Cone-beam and slot-beam configurations were investigated with and without an antiscatter grid (12:1 grid ratio, 60 lines/cm). Dose, scatter, image noise, and soft-tissue contrast resolution were evaluated in quantitative phantoms for head and body configurations over a range of exposure levels (beam energy and mAs), with reconstruction performed via filtered-backprojection. Qualitative imaging performance across various anatomical sites and imaging tasks was assessed with anthropomorphic head, abdomen, and pelvis phantoms.

RESULTS

The dose for a slot-beam scan varied from 0.02-0.06 mGy/mAs for head protocols to 0.01-0.03 mGy/mAs for body protocols, yielding dose reduction by ∼1/5 to 1/3 compared to cone-beam, owing to beam collimation and reduced x-ray scatter. The slot-beam provided an ∼6-7× reduction in scatter-to-primary ratio (SPR) compared to the cone-beam, yielding SPR ∼20-80% for head and body without the grid and ∼7-30% with the grid. Compared to cone-beam scans at equivalent dose, slot-beam images exhibited an ∼2.5× increase in soft-tissue contrast-to-noise ratio (CNR) for both grid and gridless configurations. For slot-beam scans, a further ∼10-30% improvement in CNR was achieved when the grid was removed. Slot-beam imaging could benefit certain interventional scenarios in which improved visualization of soft tissues is required within a fairly narrow longitudinal region of interest ( ± 7 mm in z )--for example, checking the completeness of tumor resection, preservation of adjacent anatomy, or detection of complications (e.g., hemorrhage). While preserving existing capabilities for fluoroscopy and cone-beam CT, slot-beam scanning could enhance the utility of intraoperative imaging and provide a useful mode for safety and validation checks in image-guided surgery.

CONCLUSIONS

The 3D imaging performance and dose of a prototype slot-beam CT configuration on the O-arm^™ system was investigated. Substantial improvements in soft-tissue image quality and reduction in radiation dose are evident with the slot-beam configuration due to reduced x-ray scatter.