The distribution of target registration error in rigid-body point-based registration

Pre-operative lung ablation prediction using deep learning

OBJECTIVE

Microwave lung ablation (MWA) is a minimally invasive and inexpensive alternative cancer treatment for patients who are not candidates for surgery/radiotherapy. However, a major challenge for MWA is its relatively high tumor recurrence rates, due to incomplete treatment as a result of inaccurate planning. We introduce a patient-specific, deep-learning model to accurately predict post-treatment ablation zones to aid planning and enable effective treatments.

MATERIALS AND METHODS

Our IRB-approved retrospective study consisted of ablations with a single applicator/burn/vendor between 01/2015 and 01/2019. The input data included pre-procedure computerized tomography (CT), ablation power/time, and applicator position. The ground truth ablation zone was segmented from follow-up CT post-treatment. Novel deformable image registration optimized for ablation scans and an applicator-centric co-ordinate system for data analysis were applied. Our prediction model was based on the U-net architecture. The registrations were evaluated using target registration error (TRE) and predictions using Bland-Altman plots, Dice co-efficient, precision, and recall, compared against the applicator vendor's estimates.

RESULTS

The data included 113 unique ablations from 72 patients (median age 57, interquartile range (IQR) (49-67); 41 women). We obtained a TRE ≤ 2 mm on 52 ablations. Our prediction had no bias from ground truth ablation volumes (p = 0.169) unlike the vendor's estimate (p < 0.001) and had smaller limits of agreement (p < 0.001). An 11% improvement was achieved in the Dice score. The ability to account for patient-specific in-vivo anatomical effects due to vessels, chest wall, heart, lung boundaries, and fissures was shown.

CONCLUSIONS

We demonstrated a patient-specific deep-learning model to predict the ablation treatment effect prior to the procedure, with the potential for improved planning, achieving complete treatments, and reduce tumor recurrence.

CLINICAL RELEVANCE STATEMENT

Our method addresses the current lack of reliable tools to estimate ablation extents, required for ensuring successful ablation treatments. The potential clinical implications include improved treatment planning, ensuring complete treatments, and reducing tumor recurrence.

Development of real-time navigation system for laparoscopic hepatectomy using magnetic micro sensor

BACKGROUND

We report a new real-time navigation system for laparoscopic hepatectomy (LH), which resembles a car navigation system.

MATERIAL AND METHODS

Virtual three-dimensional liver and body images were reconstructed using the "New-VES" system, which worked as roadmap during surgery. Several points of the patient's body were registered in virtual images using a magnetic position sensor (MPS). A magnetic transmitter, corresponding to an artificial satellite, was placed about 40 cm above the patient's body. Another MPS, corresponding to a GPS antenna, was fixed on the handling part of the laparoscope. Fiducial registration error (FRE, an error between real and virtual lengths) was utilized to evaluate the accuracy of this system.

RESULTS

Twenty-one patients underwent LH with this system. Mean FRE of the initial five patients was 17.7 mm. Mean FRE of eight patients in whom MDCT was taken using radiological markers for registration of body parts as first improvement, was reduced to 10.2 mm (p = .014). As second improvement, a new MPS as an intraoperative body position sensor was fixed on the right-sided chest wall for automatic correction of postural gap. The preoperative and postoperative mean FREs of 8 patients with both improvements were 11.1 mm and 10.1 mm (p = .250).

CONCLUSIONS

Our system may provide a promising option that virtually guides LH.

Non-invasive quantitative assessment of induced component displacement can safely and accurately diagnose tibial component loosening in patients: A prospective diagnostic study

PURPOSE

Aseptic loosening often requires major, expensive and invasive revision surgery. Current diagnostic modalities merely show indirect signs of loosening. A recent proof of concept study proposed a non-invasive technique for the quantitative and visual assessment of implant movement as a diagnostic aid for tibial component loosening. The primary research question addressed is whether this novel diagnostic modality can safely and effectively aid the diagnosis of aseptic loosening.

METHODS

This clinical study included patients suspected of aseptic total knee arthroplasty (TKA) loosening listed for revision surgery and asymptomatic patients. Safety was evaluated using a numerical rating scale (NRS) for discomfort and by registration of adverse events. Feasibility was assessed by recording the duration and ease of the procedure. Intra- and interrater reliability were evaluated. In symptomatic patients, diagnostic accuracy metrics were evaluated with intra-operative assessment as a reference test.

RESULTS

In total, 34 symptomatic and 38 asymptomatic knees with a TKA were analysed. The median NRS for discomfort during loading was 6 (interquartile range [IQR]: 3.75-7.00) in symptomatic patients and 2 (IQR: 1.00-3.00) in asymptomatic patients. No adverse events were reported. The majority of users found the use of the loading device easy. The median time spent in the computed tomography room was 9 min (IQR: 8.00-11.00). Excellent to good intra- and interrater reliabilities were achieved. Diagnostic accuracy analysis resulted in a sensitivity of 0.91 (95% confidence interval [CI]: 0.72-0.97) and a specificity of 0.72 (95% CI: 0.43-0.90).

CONCLUSIONS

The proposed diagnostic method is safe, feasible, reliable and accurate in aiding the diagnosis of aseptic tibial component loosening.

LEVEL OF EVIDENCE

Level II.

Robot-assisted augmented reality surgical navigation based on optical tracking for mandibular reconstruction surgery

PURPOSE

This work proposes a robot-assisted augmented reality (AR) surgical navigation system for mandibular reconstruction. The system accurately superimposes the preoperative osteotomy plan of the mandible and fibula into a real scene. It assists the doctor in osteotomy quickly and safely under the guidance of the robotic arm.

METHODS

The proposed system mainly consists of two modules: the AR guidance module of the mandible and fibula and the robot navigation module. In the AR guidance module, we propose an AR calibration method based on the spatial registration of the image tracking marker to superimpose the virtual models of the mandible and fibula into the real scene. In the robot navigation module, the posture of the robotic arm is first calibrated under the tracking of the optical tracking system. The robotic arm can then be positioned at the planned osteotomy after the registration of the computed tomography image and the patient position. The combined guidance of AR and robotic arm can enhance the safety and precision of the surgery.

RESULTS

The effectiveness of the proposed system was quantitatively assessed on cadavers. In the AR guidance module, osteotomies of the mandible and fibula achieved mean errors of 1.61 ± 0.62 and 1.08 ± 0.28 mm, respectively. The mean reconstruction error of the mandible was 1.36 ± 0.22 mm. In the AR-robot guidance module, the mean osteotomy errors of the mandible and fibula were 1.47 ± 0.46 and 0.98 ± 0.24 mm, respectively. The mean reconstruction error of the mandible was 1.20 ± 0.36 mm.

CONCLUSIONS

The cadaveric experiments of 12 fibulas and six mandibles demonstrate the proposed system's effectiveness and potential clinical value in reconstructing the mandibular defect with a free fibular flap.

Design and Simulate Intracranial Support to Guide Maxillo Surgery: A Study Based on Bioengineering

BACKGROUND

Intraoperative navigation allows for the creation of a real-time relationship between the anatomy imagined during diagnosis/planning and the site of surgical interest. This procedure takes place by identifying and registering trustworthy anatomical markers on planning images and using a point locator during the operation. The locator is calibrated in the workspace by placing a Dynamic Reference Frame (DRF) sensor.

OBJECTIVE

This study aims to calculate the localization accuracy of an electromagnetic locator of neuro-maxillofacial surgery, moving the standard sensor position to a different position more suitable for maxillofacial surgery.

MATERIALS AND METHODS

The upper dental arch was chosen as an alternative fixed point for the positioning of the sensor. The prototype of a bite support device was designed and generated via 3D printing. CT images of a skull phantom with 10 anatomical landmarks were acquired. The testing procedure consisted of 10 measurements for each position of the sensor: precisely 10 measurements with the sensor placed on the forehead and 10 measurements with the sensor placed on the bite support device. It also evaluated the localization error by comparing the two procedures.

RESULTS

The localization error, when the sensor was placed on the bite support device, was lower in the sphere located on the temporal bone. It was the same in the spheres located on the maxillary bone. The test analysis of the data of the new device showed that it is reliable; the tests are reproducible and can be considered as accurate as the traditional ones. In addition, the sensor mounted on this device has proven to be slightly superior in terms of accuracy and accuracy in areas such as the middle third of the face and jaw.

DISCUSSION AND CONCLUSION

The realization of the bite support device allowed the sensor to change position concerning its natural site. This procedure allows us to explore structures, such as the frontal site, which were initially difficult to approach with neuronavigation and improves the approach to midface structures, already studied with neuronavigation. The new calibration, with the position of the sensor on the support device in the same reference points sphere, highlighted the reduction in the location error. We can say that the support proposed in this study lays the foundations for a new navigation approach for patients in maxillofacial surgery, by changing the position of the sensor. It has strong points in improving the localization error for some reference points without determining disadvantages both in the calibration and in the surgical impediment.

Load-induced deformation of the tibia and its effect on implant loosening detection

CT imaging under external valgus and varus loading conditions and consecutive image analysis can be used to detect tibial implant loosening after total knee arthroplasty. However, the applied load causes the tibia to deform, which could result in an overestimation of implant displacement. This research evaluates the extent of tibia deformation and its effect on measuring implant displacement. Ten cadaver specimen with TKA were CT-scanned under valgus/varus loading (20 Nm), first implanted without bone cement fixation (mimicking a loose implant) and subsequently with bone cement fixation (mimicking a fixed implant). By means of image analysis, three relative displacements were assessed: (1) between the proximal and distal tibia (measure of deformation), (2) between the implant and the whole tibia (including potential deformation effect) and (3) between the implant and the proximal tibia (reduced deformation effect). Relative displacements were quantified in terms of translations along, and rotations about the axes of a local coordinate system. As a measure of deformation, the proximal tibia moved relative to the distal tibia by, on average 1.27 mm (± 0.50 mm) and 0.64° (± 0.25°). Deformation caused an overestimation of implant displacement in the cemented implant. The implant displaced with respect to the whole tibia by 0.45 mm (± 0.22 mm) and 0.79° (± 0.38°). Relative to the proximal tibia, the implant moved by 0.23 mm (± 0.10 mm) and 0.62° (± 0.34°). The differentiation between loose and fixed implants improved when tibia deformation was compensated for by using the proximal tibia rather than the whole tibia.

Performance assessment of surgical tracking systems based on statistical process control and longitudinal QA

A system for performance assessment and quality assurance (QA) of surgical trackers is reported based on principles of geometric accuracy and statistical process control (SPC) for routine longitudinal testing. A simple QA test phantom was designed, where the number and distribution of registration fiducials was determined drawing from analytical models for target registration error (TRE). A tracker testbed was configured with open-source software for measurement of a TRE-based accuracy metric ε and Jitter (J ). Six trackers were tested: 2 electromagnetic (EM - Aurora); and 4 infrared (IR - 1 Spectra, 1 Vega, and 2 Vicra) - all NDI (Waterloo, ON). Phase I SPC analysis of Shewhart mean (x ¯ ) and standard deviation (s ) determined system control limits. Phase II involved weekly QA of each system for up to 32 weeks and identified Pass, Note, Alert, and Failure action rules. The process permitted QA in <1 min. Phase I control limits were established for all trackers: EM trackers exhibited higher upper control limits than IR trackers in ε (EM: x ¯ ε   ∼ 2.8-3.3 mm, IR: x ¯ ε   ∼ 1.6-2.0 mm) and Jitter (EM: x ¯ jitter   ∼ 0.30-0.33 mm, IR: x ¯ jitter   ∼ 0.08-0.10 mm), and older trackers showed evidence of degradation - e.g. higher Jitter for the older Vicra (p-value < .05). Phase II longitudinal tests yielded 676 outcomes in which a total of 4 Failures were noted - 3 resolved by intervention (metal interference for EM trackers) - and 1 owing to restrictive control limits for a new system (Vega). Weekly tests also yielded 40 Notes and 16 Alerts - each spontaneously resolved in subsequent monitoring.

Generating Patient-Specific Acoustic Simulations for Transcranial Focused Ultrasound Procedures Based on Optical Tracking Information

Optical tracking is a real-time transducer positioning method for transcranial focused ultrasound (tFUS) procedures, but the predicted focus from optical tracking typically does not incorporate subject-specific skull information. Acoustic simulations can estimate the pressure field when propagating through the cranium but rely on accurately replicating the positioning of the transducer and skull in a simulated space. Here, we develop and characterize the accuracy of a workflow that creates simulation grids based on optical tracking information in a neuronavigated phantom with and without transmission through an ex vivo skull cap. The software pipeline could replicate the geometry of the tFUS procedure within the limits of the optical tracking system (transcranial target registration error (TRE): 3.9 ± 0.7 mm). The simulated focus and the free-field focus predicted by optical tracking had low Euclidean distance errors of 0.5±0.1 and 1.2±0.4 mm for phantom and skull cap, respectively, and some skull-specific effects were captured by the simulation. However, the TRE of simulation informed by optical tracking was 4.6±0.2, which is as large or greater than the focal spot size used by many tFUS systems. By updating the position of the transducer using the original TRE offset, we reduced the simulated TRE to 1.1 ± 0.4 mm. Our study describes a software pipeline for treatment planning, evaluates its accuracy, and demonstrates an approach using MR-acoustic radiation force imaging as a method to improve dosimetry. Overall, our software pipeline helps estimate acoustic exposure, and our study highlights the need for image feedback to increase the accuracy of tFUS dosimetry.

Intraoperative laparoscopic photoacoustic image guidance system in the da Vinci surgical system

This paper describes a framework allowing intraoperative photoacoustic (PA) imaging integrated into minimally invasive surgical systems. PA is an emerging imaging modality that combines the high penetration of ultrasound (US) imaging with high optical contrast. With PA imaging, a surgical robot can provide intraoperative neurovascular guidance to the operating physician, alerting them of the presence of vital substrate anatomy invisible to the naked eye, preventing complications such as hemorrhage and paralysis. Our proposed framework is designed to work with the da Vinci surgical system: real-time PA images produced by the framework are superimposed on the endoscopic video feed with an augmented reality overlay, thus enabling intuitive three-dimensional localization of critical anatomy. To evaluate the accuracy of the proposed framework, we first conducted experimental studies in a phantom with known geometry, which revealed a volumetric reconstruction error of 1.20 ± 0.71 mm. We also conducted an ex vivo study by embedding blood-filled tubes into chicken breast, demonstrating the successful real-time PA-augmented vessel visualization onto the endoscopic view. These results suggest that the proposed framework could provide anatomical and functional feedback to surgeons and it has the potential to be incorporated into robot-assisted minimally invasive surgical procedures.

Magnetic resonance imaging datasets with anatomical fiducials for quality control and registration

Tools available for reproducible, quantitative assessment of brain correspondence have been limited. We previously validated the anatomical fiducial (AFID) placement protocol for point-based assessment of image registration with millimetric (mm) accuracy. In this data descriptor, we release curated AFID placements for some of the most commonly used structural magnetic resonance imaging datasets and templates. The release of our accurate placements allows for rapid quality control of image registration, teaching neuroanatomy, and clinical applications such as disease diagnosis and surgical targeting. We release placements on individual subjects from four datasets (N = 132 subjects for a total of 15,232 fiducials) and 14 brain templates (4,288 fiducials), totalling more than 300 human rater hours of annotation. We also validate human rater accuracy of released placements to be within 1 - 2 mm (using more than 45,000 Euclidean distances), consistent with prior studies. Our data is compliant with the Brain Imaging Data Structure allowing for facile incorporation into neuroimaging analysis pipelines.

Reference-free Bayesian model for pointing errors of typein neurosurgical planning

PURPOSE

Many neurosurgical planning tasks rely on identifying points of interest in volumetric images. Often, these points require significant expertise to identify correctly as, in some cases, they are not visible but instead inferred by the clinician. This leads to a high degree of variability between annotators selecting these points. In particular, errors of type are when the experts fundamentally select different points rather than the same point with some inaccuracy. This complicates research as their mean may not reflect any of the experts' intentions nor the ground truth.

METHODS

We present a regularised Bayesian model for measuring errors of type in pointing tasks. This model is reference-free; in that it does not require a priori knowledge of the ground truth point but instead works on the basis of the level of consensus between multiple annotators. We apply this model to simulated data and clinical data from transcranial magnetic stimulation for chronic pain.

RESULTS

Our model estimates the probabilities of selecting the correct point in the range of 82.6[Formula: see text]88.6% with uncertainties in the range of 2.8[Formula: see text]4.0%. This agrees with the literature where ground truth points are known. The uncertainty has not previously been explored in the literature and gives an indication of the dataset's strength.

CONCLUSIONS

Our reference-free Bayesian framework easily models errors of type in pointing tasks. It allows for clinical studies to be performed with a limited number of annotators where the ground truth is not immediately known, which can be applied widely for better understanding human errors in neurosurgical planning.

Spatial Positioning of Immune Hotspots Reflects the Interplay between B and T Cells in Lung Squamous Cell Carcinoma

Beyond tertiary lymphoid structures, a significant number of immune-rich areas without germinal center-like structures are observed in non-small cell lung cancer. Here, we integrated transcriptomic data and digital pathology images to study the prognostic implications, spatial locations, and constitution of immune rich areas (immune hotspots) in a cohort of 935 patients with lung cancer from The Cancer Genome Atlas. A high intratumoral immune hotspot score, which measures the proportion of immune hotspots interfacing with tumor islands, was correlated with poor overall survival in lung squamous cell carcinoma but not in lung adenocarcinoma. Lung squamous cell carcinomas with high intratumoral immune hotspot scores were characterized by consistent upregulation of B-cell signatures. Spatial statistical analyses conducted on serial multiplex IHC slides further revealed that only 4.87% of peritumoral immune hotspots and 0.26% of intratumoral immune hotspots were tertiary lymphoid structures. Significantly lower densities of CD20+CXCR5+ and CD79b+ B cells and less diverse immune cell interactions were found in intratumoral immune hotspots compared with peritumoral immune hotspots. Furthermore, there was a negative correlation between the percentages of CD8+ T cells and T regulatory cells in intratumoral but not in peritumoral immune hotspots, with tertiary lymphoid structures excluded. These findings suggest that the intratumoral immune hotspots reflect an immunosuppressive niche compared with peritumoral immune hotspots, independent of the distribution of tertiary lymphoid structures. A balance toward increased intratumoral immune hotspots is indicative of a compromised antitumor immune response and poor outcome in lung squamous cell carcinoma.

SIGNIFICANCE

Intratumoral immune hotspots beyond tertiary lymphoid structures reflect an immunosuppressive microenvironment, different from peritumoral immune hotspots, warranting further study in the context of immunotherapies.

A trial to visualize perforators images from CTA with a tablet device: experience of operating on minipigs

A reliable method for precise perforator mapping can be extremely valuable in perforator flap surgery. In this study, we attempted to map perforator location using 3-dimensional computed tomography angiography (CTA), a newly developed application, and a tablet device. Preliminary examinations to test the device were conducted in mini-pigs. We used 5 female mini-pigs. Preoperative imaging of the vasculature was undertaken with CTA in the prone position, following Iopamidol (200 ml) injection via the internal jugular vein. Prior to the examination, we placed round markers on the backs of the mini-pigs. To assess accuracy, we compared the perforator positions acquired with an optical position measurement device with the perforator positions acquired with the tablet device. Furthermore, we compared the perforator positions with the tablet navigation device, which we measured directly. We measured 12 perforators with the optical position measurement device. The mean difference was 10 mm (minimum, 2 mm; maximum, 20 mm). We measured these perforators with the tablet navigation device. The mean difference was 5.4 mm (minimum, 0 mm; maximum, 20 mm). The perforator flaps were elevated safely. The perforator flaps could be elevated safely using our device, as the mean difference was only 10 mm, which is acceptable for navigating perforator flap operations. Pig backs are triangular in shape; therefore, we were unable to place markers on the contralateral side. Thus, for clinical applications of the device, we should determine the ideal marker locations.

Modeling of Respiratory Motion to Support the Minimally Invasive Destruction of Liver Tumors

OBJECTIVE

Respiratory movements are a significant factor that may hinder the use of image navigation systems during minimally invasive procedures used to destroy focal lesions in the liver. This article aims to present a method of estimating the displacement of the target point due to respiratory movements during the procedure, working in real time.

METHOD

The real-time method using skin markers and non-rigid registration algorithms has been implemented and tested for various classes of transformation. The method was validated using clinical data from 21 patients diagnosed with liver tumors. For each patient, each marker was treated as a target and the remaining markers as target position predictors, resulting in 162 configurations and 1095 respiratory cycles analyzed. In addition, the possibility of estimating the respiratory phase signal directly from intraoperative US images and the possibility of synchronization with the 4D CT respiratory sequence are also presented, based on ten patients.

RESULTS

The median value of the target registration error (TRE) was 3.47 for the non-rigid registration method using the combination of rigid transformation and elastic body spline curves, and an adaptation of the assessing quality using image registration circuits (AQUIRC) method. The average maximum distance was 3.4 (minimum: 1.6, maximum 6.8) mm.

CONCLUSIONS

The proposed method obtained promising real-time TRE values. It also allowed for the estimation of the TRE at a given geometric margin level to determine the estimated target position. Directions for further quantitative research and the practical possibility of combining both methods are also presented.

Deep learning facilitates fully automated brain image registration of optoacoustic tomography and magnetic resonance imaging

Multispectral optoacoustic tomography (MSOT) is an emerging optical imaging method providing multiplex molecular and functional information from the rodent brain. It can be greatly augmented by magnetic resonance imaging (MRI) which offers excellent soft-tissue contrast and high-resolution brain anatomy. Nevertheless, registration of MSOT-MRI images remains challenging, chiefly due to the entirely different image contrast rendered by these two modalities. Previously reported registration algorithms mostly relied on manual user-dependent brain segmentation, which compromised data interpretation and quantification. Here we propose a fully automated registration method for MSOT-MRI multimodal imaging empowered by deep learning. The automated workflow includes neural network-based image segmentation to generate suitable masks, which are subsequently registered using an additional neural network. The performance of the algorithm is showcased with datasets acquired by cross-sectional MSOT and high-field MRI preclinical scanners. The automated registration method is further validated with manual and half-automated registration, demonstrating its robustness and accuracy.

Ultrasound-based navigated pedicle screw insertion without intraoperative radiation: feasibility study on porcine cadavers

BACKGROUND

Navigation systems for spinal fusion surgery rely on intraoperative computed tomography (CT) or fluoroscopy imaging. Both expose patient, surgeons and operating room staff to significant amounts of radiation. Alternative methods involving intraoperative ultrasound (iUS) imaging have recently shown promise for image-to-patient registration. Yet, the feasibility and safety of iUS navigation in spinal fusion have not been demonstrated.

PURPOSE

To evaluate the accuracy of pedicle screw insertion in lumbar and thoracolumbar spinal fusion using a fully automated iUS navigation system.

STUDY DESIGN

Prospective porcine cadaver study.

METHODS

Five porcine cadavers were used to instrument the lumbar and thoracolumbar spine using posterior open surgery. During the procedure, iUS images were acquired and used to establish automatic registration between the anatomy and preoperative CT images. Navigation was performed with the preoperative CT using tracked instruments. The accuracy of the system was measured as the distance of manually collected points to the preoperative CT vertebral surface and compared against fiducial-based registration. A postoperative CT was acquired, and screw placements were manually verified. We report breach rates, as well as axial and sagittal screw deviations.

RESULTS

A total of 56 screws were inserted (5.50 mm diameter n=50, and 6.50 mm diameter n=6). Fifty-two screws were inserted safely without breach. Four screws (7.14%) presented a medial breach with an average deviation of 1.35±0.37 mm (all <2 mm). Two breaches were caused by 6.50 mm diameter screws, and two by 5.50 mm screws. For vertebrae instrumented with 5.50 mm screws, the average axial diameter of the pedicle was 9.29 mm leaving a 1.89 mm margin in the left and right pedicle. For vertebrae instrumented with 6.50 mm screws, the average axial diameter of the pedicle was 8.99 mm leaving a 1.24 mm error margin in the left and right pedicle. The average distance to the vertebral surface was 0.96 mm using iUS registration and 0.97 mm using fiducial-based registration.

CONCLUSIONS

We successfully implanted all pedicle screws in the thoracolumbar spine using the ultrasound-based navigation system. All breaches recorded were minor (<2 mm) and the breach rate (7.14%) was comparable to existing literature. More investigation is needed to evaluate consistency, reproducibility, and performance in surgical context.

CLINICAL SIGNIFICANCE

Intraoperative US-based navigation is feasible and practical for pedicle screw insertion in a porcine model. It might be used as a low-cost and radiation-free alternative to intraoperative CT and fluoroscopy in the future.

Estimating medical image registration error and confidence: A taxonomy and scoping review

Given that image registration is a fundamental and ubiquitous task in both clinical and research domains of the medical field, errors in registration can have serious consequences. Since such errors can mislead clinicians during image-guided therapies or bias the results of a downstream analysis, methods to estimate registration error are becoming more popular. To give structure to this new heterogenous field we developed a taxonomy and performed a scoping review of methods that quantitatively and automatically provide a dense estimation of registration error. The taxonomy breaks down error estimation methods into Approach (Image- or Transformation-based), Framework (Machine Learning or Direct) and Measurement (error or confidence) components. Following the PRISMA guidelines for scoping reviews, the 570 records found were reduced to twenty studies that met inclusion criteria, which were then reviewed according to the proposed taxonomy. Trends in the field, advantages and disadvantages of the methods, and potential sources of bias are also discussed. We provide suggestions for best practices and identify areas of future research.

Accuracy evaluation of 3D-printed noninvasive adhesive marker for dynamic navigation implant surgery in a maxillary edentulous model: An in vitro study

Dynamic computer-aided implant surgery (DCAIS) can improve dental implantation accuracy and reduce surgical risks. In the registration procedure of DCAIS, the type and the number of registration markers significantly impact the accuracy of DCAIS. One problem of DCAIS in clinical application is that only invasive screw markers can be used for implantation in edentulous patients. It could cause additional trauma, scar formation and usually increase patient discomfort. In this experiment, a personalized 3D-printed edentulous maxillary model was used for simulating clinical situations, and a 3D-printed noninvasive adhesive marker (3D-PNAM) was designed to figure out the above problem. In this research, six target screws were implanted into the model's maxillary alveolar ridge as targets for accuracy analysis. This study used target registration error (TRE) as an index to evaluate the accuracy of invasive screw makers and noninvasive adhesive markers. Results showed that 3D-PNAMs had the same accuracy as screw markers, and placing at least six registration markers in the maxilla was needed for good registration accuracy. The registration markers should be further improved and designed according to application areas' clinical needs and anatomical characteristics in future clinical studies.

A hybrid registration method using the mandibular bone surface for electromagnetic navigation in mandibular surgery

PURPOSE

To utilize navigated mandibular (reconstructive) surgery, accurate registration of the preoperative CT scan with the actual patient in the operating room (OR) is required. In this phantom study, the feasibility of a noninvasive hybrid registration method is assessed. This method consists of a point registration with anatomic landmarks for initialization and a surface registration using the bare mandibular bone surface for optimization.

METHODS

Three mandible phantoms with reference notches on two osteotomy planes were 3D printed. An electromagnetic tracking system in combination with 3D Slicer software was used for navigation. Different configurations, i.e., different surface point areas and number and configuration of surface points, were tested with a dentate phantom (A) in a metal-free environment. To simulate the intraoperative environment and different anatomies, the registration procedure was also performed with an OR bed using the dentate phantom and two (partially) edentulous phantoms with atypical anatomy (B and C). The accuracy of the registration was calculated using the notches on the osteotomy planes and was expressed as the target registration error (TRE). TRE values of less than 2.0 mm were considered as clinically acceptable.

RESULTS

In all experiments, the mean TRE was less than 2.0 mm. No differences were found using different surface point areas or number or configurations of surface points. Registration accuracy in the simulated intraoperative setting was-mean (SD)-0.96 (0.22), 0.93 (0.26), and 1.50 (0.28) mm for phantom A, phantom B, and phantom C.

CONCLUSION

Hybrid registration is a noninvasive method that requires only a small area of the bare mandibular bone surface to obtain high accuracy in phantom setting. Future studies should test this method in clinical setting during actual surgery.

Comparative Analysis of Optoelectronic Accuracy in the Laboratory Setting Versus Clinical Operative Environment: A Systematic Review

STUDY DESIGN

Systematic review.

OBJECTIVES

The optoelectronic camera source and data interpolation process serve as the foundation for navigational integrity in robotic-assisted surgical platforms. The current systematic review serves to provide a basis for the numerical disparity observed when comparing the intrinsic accuracy of optoelectronic cameras versus accuracy in the laboratory setting and clinical operative environments.

METHODS

Review of the PubMed and Cochrane Library research databases was performed. The exhaustive literature compilation obtained was then vetted to reduce redundancies and categorized into topics of intrinsic accuracy, registration accuracy, musculoskeletal kinematic platforms, and clinical operative platforms.

RESULTS

A total of 465 references were vetted and 137 comprise the basis for the current analysis. Regardless of application, the common denominators affecting overall optoelectronic accuracy are intrinsic accuracy, registration accuracy, and application accuracy. Intrinsic accuracy equaled or was less than 0.1 mm translation and 0.1 degrees rotation per fiducial. Controlled laboratory platforms reported 0.1 to 0.5 mm translation and 0.1 to 1.0 degrees rotation per array. Accuracy in robotic-assisted spinal surgery reported 1.5 to 6.0 mm translation and 1.5 to 5.0 degrees rotation when comparing planned to final implant position.

CONCLUSIONS

Navigational integrity and maintenance of fidelity of optoelectronic data is the cornerstone of robotic-assisted spinal surgery. Transitioning from controlled laboratory to clinical operative environments requires an increased number of steps in the optoelectronic kinematic chain and error potential. Diligence in planning, fiducial positioning, system registration and intra-operative workflow have the potential to improve accuracy and decrease disparity between planned and final implant position.

Reliability and accuracy of dynamic navigation for zygomatic implant placement

Objectives

To assess the accuracy of a real‐time dynamic navigation system applied in zygomatic implant (ZI) surgery and summarize device‐related negative events and their management.

Material and methods

Patients who presented with severely maxillary atrophy or maxillary defects and received dynamic navigation‐supported ZI surgery were included. The deviations of entry, exit, and angle were measured after image data fusion. A linear mixed‐effects model was used. Statistical significance was defined as p < .05. Device‐related negative events and their management were also recorded and analyzed.

Results

Two hundred and thirty‐one zygomatic implants (ZIs) with navigation‐guided placement were planned in 74 consecutive patients between Jan 2015 and Aug 2020. Among them, 71 patients with 221 ZIs received navigation‐guided surgery finally. The deviations in entry, exit, and angle were 1.57 ± 0.71 mm, 2.1 ± 0.94 mm and 2.68 ± 1.25 degrees, respectively. Significant differences were found in entry and exit deviation according to the number of ZIs in the zygomata (p = .03 and .00, respectively). Patients with atrophic maxillary or maxillary defects showed a significant difference in exit deviation (p = .01). A total of 28 device‐related negative events occurred, and one resulted in 2 ZI failures due to implant malposition. The overall survival rate of ZIs was 98.64%, and the mean follow‐up time was 24.11 months (Standard Deviation [SD]: 12.62).

Conclusions

The navigation‐supported ZI implantation is an accurate and reliable surgical approach. However, relevant technical negative events in the navigation process are worthy of attention.

Automatic Registration and Error Color Maps to Improve Accuracy for Navigated Bone Tumor Surgery Using Intraoperative Cone-Beam CT

Computer-assisted surgery (CAS) can improve surgical precision in orthopaedic oncology. Accurate alignment of the patient’s imaging coordinates with the anatomy, known as registration, is one of the most challenging aspects of CAS and can be associated with substantial error. Using intraoperative, on-the-table, cone-beam computed tomography (CBCT), we performed a pilot clinical study to validate a method for automatic intraoperative registration.

Accuracy of Robotic-Assisted Spinal Surgery-Comparison to TJR Robotics, da Vinci Robotics, and Optoelectronic Laboratory Robotics

BACKGROUND

The optoelectronic camera source and data interpolation serve as the foundation for navigational integrity in the robotic-assisted surgical platform. The objective of the current systematic review serves to provide a basis for the numerical disparity that exists when comparing the intrinsic accuracy of optoelectronic cameras: accuracy observed in the laboratory setting versus accuracy in the clinical operative environment. It is postulated that there exists a greater number of connections in the optoelectronic kinematic chain when analyzing the clinical operative environment to the laboratory setting. This increase in data interpolation, coupled with intraoperative workflow challenges, reduces the degree of accuracy based on surgical application and to that observed in controlled musculoskeletal kinematic laboratory investigations.

METHODS

Review of the PubMed and Cochrane Library research databases was performed. The exhaustive literature compilation obtained was then vetted to reduce redundancies and categorized into topics of intrinsic optoelectronic accuracy, registration accuracy, musculoskeletal kinematic platforms, and clinical operative platforms.

RESULTS

A total of 147 references make up the basis for the current analysis. Regardless of application, the common denominators affecting overall optoelectronic accuracy are intrinsic accuracy, registration accuracy, and application accuracy. Intrinsic accuracy of optoelectronic tracking equaled or was less than 0.1 mm of translation and 0.1° of rotation per fiducial. Controlled laboratory platforms reported 0.1 to 0.5 mm of translation and 0.1°-1.0° of rotation per array. There is a huge falloff in clinical applications: accuracy in robotic-assisted spinal surgery reported 1.5 to 6.0 mm of translation and 1.5° to 5.0° of rotation when comparing planned to final implant position. Total Joint Robotics and da Vinci urologic robotics computed accuracy, as predicted, lies between these two extremes-1.02 mm for da Vinci and 2 mm for MAKO.

CONCLUSIONS

Navigational integrity and maintenance of fidelity of optoelectronic data is the cornerstone of robotic-assisted spinal surgery. Transitioning from controlled laboratory to clinical operative environments requires an increased number of steps in the optoelectronic kinematic chain and error potential. Diligence in planning, fiducial positioning, system registration, and intraoperative workflow have the potential to improve accuracy and decrease disparity between planned and final implant position. The key determining factors limiting navigation resolution accuracy are highlighted by this Cochrane research analysis.

Transcranial focused ultrasound modulates cortical and thalamic motor activity in awake sheep

Transcranial application of pulsed low-intensity focused ultrasound (FUS) modulates the excitability of region-specific brain areas, and anesthetic confounders on brain activity warrant the evaluation of the technique in awake animals. We examined the neuromodulatory effects of FUS in unanesthetized sheep by developing a custom-fit headgear capable of reproducibly placing an acoustic focus on the unilateral motor cortex (M1) and corresponding thalamic area. The efferent responses to sonication, based on the acoustic parameters previously identified in anesthetized sheep, were measured using electromyography (EMG) from both hind limbs across three experimental conditions: on-target sonication, off-target sonication, and without sonication. Excitatory sonication yielded greater amplitude of EMG signals obtained from the hind limb contralateral to sonication than that from the ipsilateral limb. Spurious appearance of motion-related EMG signals limited the amount of analyzed data (~ 10% selection of acquired data) during excitatory sonication, and the averaged EMG response rates elicited by the M1 and thalamic stimulations were 7.5 ± 1.4% and 6.7 ± 1.5%, respectively. Suppressive sonication, while sheep walked on the treadmill, temporarily reduced the EMG amplitude from the limb contralateral to sonication. No significant change was found in the EMG amplitudes during the off-target sonication. Behavioral observation throughout the study and histological analysis showed no sign of brain tissue damage caused by the acoustic stimulation. Marginal response rates observed during excitatory sonication call for technical refinement to reduce motion artifacts during EMG acquisitions as well as acoustic aberration correction schemes to improve spatial accuracy of sonication. Yet, our results indicate that low-intensity FUS modulated the excitability of regional brain tissues reversibly and safely in awake sheep, supporting its potential in theragnostic applications.

Registration-free workflow for electromagnetic and optical navigation in orbital and craniofacial surgery

The accuracy of intra-operative navigation is largely dependent on the intra-operative registration procedure. Next to accuracy, important factors to consider for the registration procedure are invasiveness, time consumption, logistical demands, user-dependency, compatibility and radiation exposure. In this study, a workflow is presented that eliminates the need for a registration procedure altogether: registration-free navigation. In the workflow, the maxillary dental model is fused to the pre-operative imaging data using commercially available virtual planning software. A virtual Dynamic Reference Frame on a splint is designed on the patient’s fused maxillary dentition: during surgery, the splint containing the reference frame is positioned on the patient’s dentition. This alleviates the need for any registration procedure, since the position of the reference frame is known from the design. The accuracy of the workflow was evaluated in a cadaver set-up, and compared to bone-anchored fiducial, virtual splint and surface-based registration. The results showed that accuracy of the workflow was greatly dependent on tracking technique used: the workflow was the most accurate with electromagnetic tracking, but the least accurate with optical tracking. Although this method offers a time-efficient, non-invasive, radiation-free automatic alternative for registration, clinical implementation is hampered by the unexplained differences in accuracy between tracking techniques.

New Navigation Approaches for Endoscopic Lateral Skull Base Surgery

Image-guided navigation is well established for surgery of the brain and anterior skull base. Although navigation workstations have been used widely by neurosurgeons and rhinologists for decades, utilization in the lateral skull base (LSB) has been less due to stricter requirements for overall accuracy less than 1 mm in this region. Endoscopic approaches to the LSB facilitate minimally invasive surgeries with less morbidity, yet there are risks of injury to critical structures. With improvements in technology over the years, image-guided navigation for endoscopic LSB surgery can reduce operative time, optimize exposure for surgical corridors, and increase safety in difficult cases.

Automated Implant Resizing for Single-Stage Cranioplasty

Patient-specific customized cranial implants (CCIs) are designed to fill the bony voids in the cranial and craniofacial skeleton. The current clinical approach during single-stage cranioplasty involves a surgeon modifying an oversized CCI to fit a patient's skull defect. The manual process, however, can be imprecise and time-consuming. This paper presents an automated surgical workflow with a robotic workstation for intraoperative CCI modification that provides higher resizing accuracy compared to the manual approach. We proposed a 2-scan method for intraoperative patient-to-CT registration using reattachable fiducial markers to address the registration issue caused by the clinical draping requirement. First, the draped defected skull was 3D scanned and registered to the CT space using our proposed 2-scan registration method. Next, our algorithm generates a robot cutting toolpath based on the 3D defect model. The robot then performs automatic 3D scanning to localize the implant and resizes the implant to match the cranial defect. We evaluated the implant resizing accuracy of the proposed paradigm against the resizing accuracy of the manual approach by an expert surgeon on two plastic skulls and two cadavers. The evaluation results showed that our system was able to decrease the bone gap distance by more than 60% and 30% on plastic skulls and cadavers respectively compared to the manual approach, indicating lower risk of post-surgical complication and better aesthetic restoration.

A Cost Function for the Uncertainty of Matching Point Distribution on Image Registration

Computing the homography matrix using the known matching points is a key step in computer vision for image registration. In practice, the number, accuracy, and distribution of the known matching points can affect the uncertainty of the homography matrix. This study mainly focuses on the effect of matching point distribution on image registration. First, horizontal dilution of precision (HDOP) is derived to measure the influence of the distribution of known points on fixed point position accuracy on the image. The quantization function, which is the average of the center points’ HDOP* of the overlapping region, is then constructed to measure the uncertainty of matching distribution. Finally, the experiments in the field of image registration are performed to verify the proposed function. We test the consistency of the relationship between the proposed function and the average of symmetric transfer errors. Consequently, the proposed function is appropriate for measuring the uncertainty of matching point distribution on image registration.

Development and in vivo assessment of a novel MRI-compatible headframe system for the ovine animal model

BACKGROUND

The brain of sheep has primarily been used in neuroscience as an animal model because of its similarity to the human brain, in particular if compared to other models such as the lissencephalic rodent brain. Their brain size also makes sheep an ideal model for the development of neurosurgical techniques using conventional clinical CT/MRI scanners and stereotactic systems for neurosurgery.

METHODS

In this study, we present the design and validation of a new CT/MRI compatible head frame for the ovine model and software, with its assessment under two real clinical scenarios.

RESULTS

Ex-vivo and in vivo trial results report an average linear displacement of the ovine head frame during conventional surgical procedures of 0.81 mm for ex-vivo trials and 0.68 mm for in vivo tests, respectively.

CONCLUSIONS

These trial results demonstrate the robustness of the head frame system and its suitability to be employed within a real clinical setting.

Automated fiducial marker detection and localization in volumetric computed tomography images: a three-step hybrid approach with deep learning

Purpose: Automating fiducial detection and localization in the patient’s pre-operative images can lead to better registration accuracy, reduced human errors, and shorter intervention time. Most current approaches are optimized for a single marker type, mainly spherical adhesive markers. A fully automated algorithm is proposed and evaluated for screw and spherical titanium fiducials, typically used in high-accurate frameless surgical navigation.

Approach: The algorithm builds on previous approaches with morphological functions and pose estimation algorithms. A 3D convolutional neural network (CNN) is proposed for the fiducial classification task and evaluated for both traditional closed-set and emerging open-set classifiers. A proposed digital ground-truth experiment, with cone-beam computed tomography (CBCT) imaging software, is performed to determine the localization accuracy of the algorithm. The localized fiducial positions in the CBCT images by the presented algorithm were compared to the actual known positions in the virtual phantom models. The difference represents the fiducial localization error (FLE).

Results: A total of 241 screws, 151 spherical fiducials, and 1550 other structures are identified with the best true positive rate 95.9% for screw and 99.3% for spherical fiducials at 8.7% and 3.4% false positive rate, respectively. The best achieved FLE mean and its standard deviation for a screw and spherical marker are 58 (14) and 14 (6)  μm, respectively.

Conclusions: Accurate marker detection and localization were achieved, with spherical fiducials being superior to screws. Large marker volume and smaller voxel size yield significantly smaller FLEs. Attenuating noise by mesh smoothing has a minor effect on FLE. Future work will focus on expanding the CNN for image segmentation.

Strain Energy Decay Predicts Elastic Registration Accuracy From Intraoperative Data Constraints

Image-guided intervention for soft tissue organs depends on the accuracy of deformable registration methods to achieve effective results. While registration techniques based on elastic theory are prevalent, no methods yet exist that can prospectively estimate registration uncertainty to regulate sources and mitigate consequences of localization error in deforming organs. This paper introduces registration uncertainty metrics based on dispersion of strain energy from boundary constraints to predict the proportion of target registration error (TRE) remaining after nonrigid elastic registration. These uncertainty metrics depend on the spatial distribution of intraoperative constraints provided to registration with relation to patient-specific organ geometry. Predictive linear and bivariate gamma models are fit and cross-validated using an existing dataset of 6291 simulated registration examples, plus 699 novel simulated registrations withheld for independent validation. Average uncertainty and average proportion of TRE remaining after elastic registration are strongly correlated ( r = 0.78 ), with mean absolute difference in predicted TRE equivalent to 0.9 ± 0.6 mm (cross-validation) and 0.9 ± 0.5 mm (independent validation). Spatial uncertainty maps also permit localized TRE estimates accurate to an equivalent of 3.0 ± 3.1 mm (cross-validation) and 1.6 ± 1.2 mm (independent validation). Additional clinical evaluation of vascular features yields localized TRE estimates accurate to 3.4 ± 3.2 mm. This work formalizes a lower bound for the inherent uncertainty of nonrigid elastic registrations given coverage of intraoperative data constraints, and demonstrates a relation to TRE that can be predictively leveraged to inform data collection and provide a measure of registration confidence for elastic methods.

Registration of magnetic resonance and computed tomography images in patients with oral squamous cell carcinoma for three-dimensional virtual planning of mandibular resection and reconstruction

The aim of this study was to evaluate and present an automated method for registration of magnetic resonance imaging (MRI) and computed tomography (CT) or cone beam CT (CBCT) images of the mandibular region for patients with oral squamous cell carcinoma (OSCC). Registered MRI and (CB)CT could facilitate the three-dimensional virtual planning of surgical guides employed for resection and reconstruction in patients with OSCC with mandibular invasion. MRI and (CB)CT images were collected retrospectively from 19 patients. MRI images were aligned with (CB)CT images employing a rigid registration approach (stage 1), a rigid registration approach using a mandibular mask (stage 2), and two non-rigid registration approaches (stage 3). Registration accuracy was quantified by the mean target registration error (mTRE), calculated over a set of landmarks annotated by two observers. Stage 2 achieved the best registration result, with an mTRE of 2.5±0.7mm, which was comparable to the inter- and intra-observer variabilities of landmark placement in MRI. Stage 2 was significantly better aligned compared to all approaches in stage 3. In conclusion, this study demonstrated that rigid registration with the use of a mask is an appropriate image registration method for aligning MRI and (CB)CT images of the mandibular region in patients with OSCC.

Left Atrial Enhancement Correlates With Myocardial Conduction Velocity in Patients With Persistent Atrial Fibrillation

BACKGROUND

Conduction velocity (CV) heterogeneity and myocardial fibrosis both promote re-entry, but the relationship between fibrosis as determined by left atrial (LA) late-gadolinium enhanced cardiac magnetic resonance imaging (LGE-CMRI) and CV remains uncertain.

OBJECTIVE

Although average CV has been shown to correlate with regional LGE-CMRI in patients with persistent AF, we test the hypothesis that a localized relationship exists to underpin LGE-CMRI as a minimally invasive tool to map myocardial conduction properties for risk stratification and treatment guidance.

METHOD

3D LA electroanatomic maps during LA pacing were acquired from eight patients with persistent AF following electrical cardioversion. Local CVs were computed using triads of concurrently acquired electrograms and were co-registered to allow correlation with LA wall intensities obtained from LGE-CMRI, quantified using normalized intensity (NI) and image intensity ratio (IIR). Association was evaluated using multilevel linear regression.

RESULTS

An association between CV and LGE-CMRI intensity was observed at scales comparable to the size of a mapping electrode: -0.11 m/s per unit increase in NI (P < 0.001) and -0.96 m/s per unit increase in IIR (P < 0.001). The magnitude of this change decreased with larger measurement area. Reproducibility of the association was observed with NI, but not with IIR.

CONCLUSION

At clinically relevant spatial scales, comparable to area of a mapping catheter electrode, LGE-CMRI correlates with CV. Measurement scale is important in accurately quantifying the association of CV and LGE-CMRI intensity. Importantly, NI, but not IIR, accounts for changes in the dynamic range of CMRI and enables quantitative reproducibility of the association.

A feasibility study to estimate optimal rigid-body registration using combinatorial rigid registration optimization (CORRO)

Purpose

Clinical image pairs provide the most realistic test data for image registration evaluation. However, the optimal registration is unknown. Using combinatorial rigid registration optimization (CORRO) we demonstrate a method to estimate the optimal alignment for rigid‐registration of clinical image pairs.

Methods

Expert selected landmark pairs were selected for each CT/CBCT image pair for six cases representing head and neck, thoracic, and pelvic anatomic regions. Combination subsets of a k number of landmark pairs (k‐combination set) were generated without repeat to form a large set of k‐combination sets (k‐set) for k = 4,8,12. The rigid transformation between the image pairs was calculated for each k‐combination set. The mean and standard deviation of these transformations were used to derive final registration for each k‐set.

Results

The standard deviation of registration output decreased as the k‐size increased for all cases. The joint entropy evaluated for each k‐set of each case was smaller than those from two commercially available registration programs indicating a stronger correlation between the image pair after CORRO was used. A joint histogram plot of all three algorithms showed high correlation between them. As further proof of the efficacy of CORRO the joint entropy of each member of 30 000 k‐combination sets in k = 4 were calculated for one of the thoracic cases. The minimum joint entropy was found to exist at the estimated mean of registration indicating CORRO converges to the optimal rigid‐registration results.

Conclusions

We have developed a methodology called CORRO that allows us to estimate optimal alignment for rigid‐registration of clinical image pairs using a large set landmark point. The results for the rigid‐body registration have been shown to be comparable to results from commercially available algorithms for all six cases. CORRO can serve as an excellent tool that can be used to test and validate rigid registration algorithms.

Ultrasound 3D reconstruction of malignant masses in robotic-assisted partial nephrectomy using the PAF rail system: a comparison study

Purpose

In robotic-assisted partial nephrectomy (RAPN), the use of intraoperative ultrasound (IOUS) helps to localise and outline the tumours as well as the blood vessels within the kidney. The aim of this work is to evaluate the use of the pneumatically attachable flexible (PAF) rail system for US 3D reconstruction of malignant masses in RAPN. The PAF rail system is a novel device developed and previously presented by the authors to enable track-guided US scanning.

Methods

We present a comparison study between US 3D reconstruction of masses based on: the da Vinci Surgical System kinematics, single- and stereo-camera tracking of visual markers embedded on the probe. An US-realistic kidney phantom embedding a mass is used for testing. A new design for the US probe attachment to enhance the performance of the kinematic approach is presented. A feature extraction algorithm is proposed to detect the margins of the targeted mass in US images.

Results

To evaluate the performance of the investigated approaches the resulting 3D reconstructions have been compared to a CT scan of the phantom. The data collected indicates that single camera reconstruction outperformed the other approaches, reconstructing with a sub-millimetre accuracy the targeted mass.

Conclusions

This work demonstrates that the PAF rail system provides a reliable platform to enable accurate US 3D reconstruction of masses in RAPN procedures. The proposed system has also the potential to be employed in other surgical procedures such as hepatectomy or laparoscopic liver resection.

Stereotactic Navigation for Rectal Surgery: Comparison of 3-Dimensional C-Arm-Based Registration to Paired-Point Registration

BACKGROUND

Stereotactic navigation enables surgeons to use the preoperative CT or MRI images as a real-time "navigation map." Although stereotactic navigation has been established in neurosurgery and orthopedic surgery, whether this technology is applicable to GI tract surgery remains challenging because of tissue deformation and organ motion. A critical component of this technology is the registration that links the patient's actual body to the preoperative imaging data.

OBJECTIVE

The objective was to assess the applicability of stereotactic navigation in rectal surgery, focusing on the registration method.

DESIGN

This study was based on a prospective case series.

SETTING

The study was conducted in a single university hospital.

PATIENTS

Four patients who underwent laparoscopic rectal surgery were included.

INTERVENTIONS

Paired-point registration was performed for 2 cases, whereas 3-dimensional C-arm-based registration was performed for the other 2 cases. In addition, 3-dimensional C-arm-based registration was performed twice during the operation.

MAIN OUTCOME MEASURE

Navigation accuracy was evaluated by measuring target registration error at 8 anatomical landmarks.

RESULTS

Target registration error of the 3-dimensional C-arm-based registration group was significantly smaller than that of the paired-point registration group (median, 19.5 mm vs 54.1 mm; p < 0.001). In particular, the error of Z-axis (cranial-to-caudal direction) was significantly smaller in 3-dimensional C-arm-based registration (median, 12.4 mm vs 48.8 mm; p < 0.001). In one case in the 3-dimensional C-arm-based registration group, target registration error of the second registration became significantly smaller than that of the first registration (p = 0.008).

LIMITATIONS

This was an observational study with small sample size.

CONCLUSION

Three-dimensional C-arm-based registration could be performed with the patient in a lithotomy position with head down and lateral tilt without being affected by positional changes. Three-dimensional C-arm-based registration resulted in significantly higher navigation accuracy than paired-point registration, and its accuracy could be further improved by intraoperative re-registration.

respiTrack: Patient-specific real-time respiratory tumor motion prediction using magnetic tracking

Purpose

An intraoperative real-time respiratory tumor motion prediction system with magnetic tracking technology is presented. Based on respiratory movements in different body regions, it provides patient and single/multiple tumor-specific prediction that facilitates the guiding of treatments.

Methods

A custom-built phantom patient model replicates the respiratory cycles similar to a human body, while the custom-built sensor holder concept is applied on the patient’s surface to find optimum sensor number and their best possible placement locations to use in real-time surgical navigation and motion prediction of internal tumors. Automatic marker localization applied to patient’s 4D-CT data, feature selection and Gaussian process regression algorithms enable off-line prediction in the preoperative phase to increase the accuracy of real-time prediction.

Results

Two evaluation methods with three different registration patterns (at fully/half inhaled and fully exhaled positions) were used quantitatively at all internal target positions in phantom: The statical method evaluates the accuracy by stopping simulated breathing and dynamic with continued breathing patterns. The overall root mean square error (RMS) for both methods was between \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$0.32\pm 0.06~\hbox {mm}$$\end{document}0.32±0.06mm and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$3.71\pm 0.79~\hbox {mm}$$\end{document}3.71±0.79mm. The overall registration RMS error was \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$0.6\pm 0.4~\hbox {mm}$$\end{document}0.6±0.4mm. The best prediction errors were observed by registrations at half inhaled positions with minimum \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$0.27\pm 0.02~\hbox {mm}$$\end{document}0.27±0.02mm, maximum \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$2.90\pm 0.72~\hbox {mm}$$\end{document}2.90±0.72mm. The resulting accuracy satisfies most radiotherapy treatments or surgeries, e.g., for lung, liver, prostate and spine.

Conclusion

The built system is proposed to predict respiratory motions of internal structures in the body while the patient is breathing freely during treatment. The custom-built sensor holders are compatible with magnetic tracking. Our presented approach reduces known technological and human limitations of commonly used methods for physicians and patients.

Augmented reality navigation for liver resection with a stereoscopic laparoscope

OBJECTIVE

Understanding the three-dimensional (3D) spatial position and orientation of vessels and tumor(s) is vital in laparoscopic liver resection procedures. Augmented reality (AR) techniques can help surgeons see the patient's internal anatomy in conjunction with laparoscopic video images.

METHOD

In this paper, we present an AR-assisted navigation system for liver resection based on a rigid stereoscopic laparoscope. The stereo image pairs from the laparoscope are used by an unsupervised convolutional network (CNN) framework to estimate depth and generate an intraoperative 3D liver surface. Meanwhile, 3D models of the patient's surgical field are segmented from preoperative CT images using V-Net architecture for volumetric image data in an end-to-end predictive style. A globally optimal iterative closest point (Go-ICP) algorithm is adopted to register the pre- and intraoperative models into a unified coordinate space; then, the preoperative 3D models are superimposed on the live laparoscopic images to provide the surgeon with detailed information about the subsurface of the patient's anatomy, including tumors, their resection margins and vessels.

RESULTS

The proposed navigation system is tested on four laboratory ex vivo porcine livers and five operating theatre in vivo porcine experiments to validate its accuracy. The ex vivo and in vivo reprojection errors (RPE) are 6.04 ± 1.85 mm and 8.73 ± 2.43 mm, respectively.

CONCLUSION AND SIGNIFICANCE

Both the qualitative and quantitative results indicate that our AR-assisted navigation system shows promise and has the potential to be highly useful in clinical practice.

Evaluation of 3D ultrasound for image guidance

PURPOSE

In this paper we compared two different 3D ultrasound (US) modes (3D free-hand mode and 3D wobbler mode) to see which is more suitable to perform the 3D-US/3D-US registration for clinical guidance applications. The typical errors with respect to their impact on the final localization error were evaluated step by step.

METHODS

Multi-point target and Hand-eye calibration methods were used for 3D US calibration together with a newly designed multi-cone phantom. Pointer based and image based methods were used for 2D US calibration. The calibration target error was computed by using a different multi-cone phantom. An egg-shaped phantom was used as ground truth to compare distortions for both 3D modes along with the measurements of the volume. Finally, we compared 3D ultrasound images acquired by 3D wobbler mode and 3D free-hand mode with respect to their 3D-US/3D-US registration accuracy using both, phantom and patient data. A theoretical step by step error analysis was performed and compared to empirical data.

RESULTS

Target registration errors based on the calibration with the 3D Multi-point and 2D pointer/image method have been found to be comparable (∼1mm). They both outperformed the 3D Hand-eye method (error >2mm). Volume measurements with the 3D free-hand mode were closest to the ground truth (around 6% error compared to 9% with the 3D wobbler mode). Additional scans on phantoms showed a 3D-US/3D-US registration error below 1 mm for both, the 3D free-hand mode and the 3D wobbler mode, respectively. Results with patient data showed greater error with the 3D free-hand mode (6.50mm - 13.37mm) than with the 3D wobbler mode (2.99 ± 1.54 mm). All the measured errors were found to be in accordance to their theoretical upper bounds.

CONCLUSION

While both 3D volume methods showed comparable results with respect to 3D-US/3D-US registration for phantom images, for patient data registrations the 3D wobbler mode is superior to the 3D free-hand mode. The effect of all error sources could be estimated by theoretical derivations.

Optimization Model for the Distribution of Fiducial Markers in Liver Intervention

The distribution of fiducial markers is one of the main factors affected the accuracy of optical navigation system. However, many studies have been focused on improving the fiducial registration accuracy or the target registration accuracy, but few solutions involve optimization model for the distribution of fiducial markers. In this paper, we propose an optimization model for the distribution of fiducial markers to improve the optical navigation accuracy. The strategy of optimization model is reducing the distribution from three dimensional to two dimensional to obtain the 2D optimal distribution by using optimization algorithm in terms of the marker number and the expectation equation of target registration error (TRE), and then extend the 2D optimal distribution in two dimensional to three dimensional to calculate the optimal distribution according to the distance parameter and the expectation equation of TRE. The results of the experiments show that the averaged TRE for the human phantom is approximately 1.00 mm by applying the proposed optimization model, and the averaged TRE for the abdominal phantom is 0.59 mm. The experimental results of liver simulator model and ex-vivo porcine liver model show that the proposed optimization model can be effectively applied in liver intervention.

Evaluating an Image-Guided Operating Room with Cone Beam CT for Skull Base Surgery

Importance  Skull base surgery requires precise preoperative assessment and intraoperative management of the patient. Surgical navigation is routinely used for complex skull base cases; however, the image guidance is commonly based on preoperative scans alone. Objective  The primary objective of this study was to assess the image quality of intraoperative cone-beam computed tomography (CBCT) within anatomical landmarks used in sinus and skull base surgery. The secondary objective was to assess the registration error of a surgical navigation system based on intraoperative CBCT. Design  Present study is a retrospective case series of image quality after intraoperative cone beam CT. Setting  The study was conducted at Toronto General Hospital and Princess Margaret Cancer Centre, University Health Network, Toronto. Participants  A total of 46 intraoperative scans (34 patients, 21 skull base, 13 head and neck) were studied. Main Outcome and Measures  Thirty anatomical landmarks (vascular, soft tissue, and bony) within the sinuses and anterior skull base were evaluated for general image quality characteristics: (1) bony detail visualization; (2) soft-tissue visualization; (3) vascular visualization; and (4) freedom from artifacts (e.g., metal). Levels of intravenous (IV) contrast enhancement were quantified in Hounsfield's units (HU). Standard paired-point registration between imaging and tracker coordinates was performed using 6 to 8 skin fiducial markers and the corresponding fiducial registration error (FRE) was measured. Results  Median score for bony detail on CBCT was 5, remaining at 5 after administration of IV contrast. Median soft-tissue score was 2 for both pre- and postcontrast. Median vascular score was 1 precontrast and 3 postcontrast. Median score for artifacts on CBCT were 2 for both pre-and postcontrast, and metal objects were noted to be the most significant source of artifact. Intraoperative CBCT allowed preresection images and immediate postresection images to be available to the skull base surgeon. There was a significant improvement in mean (standard deviation [SD]) CT intensity in the left carotid artery postcontrast 334 HU (67 HU) ( p  < 10 ^-10 ). The mean FRE was 1.8 mm (0.45 mm). Conclusion  Intraoperative CBCT in complex skull base procedures provides high-resolution bony detail allowing immediate assessment of complex resections. The use of IV contrast with CBCT improves the visualization of vasculature. Image-guidance based on CBCT yields registration errors consistent with standard techniques.

A Real-Time 3D Laparoscopic Imaging System: Design, Method, and Validation

OBJECTIVE

This paper aims to propose a 3D laparoscopic imaging system that can realize dense 3D reconstruction in real time.

METHODS

Based on the active stereo technique which yields high-density, accurate and robust 3D reconstruction by combining structured light and stereo vision, we design a laparoscopic system consisting of two image feedback channels and one pattern projection channel. Remote high-speed image acquisition and pattern generation lay the foundation for the real-time dense 3D surface reconstruction and enable the miniaturization of the laparoscopic probe. To enhance the reconstruction efficiency and accuracy, we propose a novel active stereo method by which the dense 3D point cloud is obtained using only five patterns, while most existing multiple-shot structured light techniques require [Formula: see text] patterns. In our method, dual-frequency phase-shifting fringes are utilized to uniquely encode the pixels of the measured targets, and a dual-codeword matching scheme is developed to simplify the matching procedure and achieve high-precision reconstruction.

RESULTS

Compared with the existing structured light techniques, the proposed method shows better real-time efficiency and accuracy in both quantitative and qualitative ways. Ex-vivo experiments demonstrate the robustness of the proposed method to different biological organs and the effectiveness to lesions and deformations of the organs. Feasibility of the proposed system for real-time dense 3D reconstruction is verified in dynamic experiments. According to the experimental results, the system acquires 3D point clouds with a speed of 12 frames per second. Each frame contains more than 40,000 points, and the average errors tested on standard objects are less than 0.2 mm.

SIGNIFICANCE

This paper provides a new real-time dense 3D reconstruction method for 3D laparoscopic imaging. The established prototype system has shown good performance in reconstructing surface of biological tissues.

Simultaneous Optimization of Patient-Image Registration and Hand-Eye Calibration for Accurate Augmented Reality in Surgery

OBJECTIVE

Augmented reality (AR) navigation using a position sensor in endoscopic surgeries relies on the quality of patient-image registration and hand-eye calibration. Conventional methods collect the necessary data to compute two output transformation matrices separately. However, the AR display setting during surgery generally differs from that during preoperative processes. Although conventional methods can identify optimal solutions under initial conditions, AR display errors are unavoidable during surgery owing to the inherent computational complexity of AR processes, such as error accumulation over successive matrix multiplications, and tracking errors of position sensor.

METHODS

We propose the simultaneous optimization of patient-image registration and hand-eye calibration in an AR environment before surgery. The relationship between the endoscope and a virtual object to overlay is first calculated using an endoscopic image, which also functions as a reference during optimization. After including the tracking information from the position sensor, patient-image registration and hand-eye calibration are optimized in terms of least-squares.

RESULTS

Experiments with synthetic data verify that the proposed method is less sensitive to computation and tracking errors. A phantom experiment with a position sensor is also conducted. The accuracy of the proposed method is significantly higher than that of the conventional method.

CONCLUSION

The AR accuracy of the proposed method is compared with those of the conventional ones, and the superiority of the proposed method is verified.

SIGNIFICANCE

This study demonstrates that the proposed method exhibits substantial potential for improving AR navigation accuracy.

Image Overlay Surgery Based on Augmented Reality: A Systematic Review

Augmented Reality (AR) applied to surgical guidance is gaining relevance in clinical practice. AR-based image overlay surgery (i.e. the accurate overlay of patient-specific virtual images onto the body surface) helps surgeons to transfer image data produced during the planning of the surgery (e.g. the correct resection margins of tissue flaps) to the operating room, thus increasing accuracy and reducing surgery times. We systematically reviewed 76 studies published between 2004 and August 2018 to explore which existing tracking and registration methods and technologies allow healthcare professionals and researchers to develop and implement these systems in-house. Most studies used non-invasive markers to automatically track a patient's position, as well as customised algorithms, tracking libraries or software development kits (SDKs) to compute the registration between patient-specific 3D models and the patient's body surface. Few studies combined the use of holographic headsets, SDKs and user-friendly game engines, and described portable and wearable systems that combine tracking, registration, hands-free navigation and direct visibility of the surgical site. Most accuracy tests included a low number of subjects and/or measurements and did not normally explore how these systems affect surgery times and success rates. We highlight the need for more procedure-specific experiments with a sufficient number of subjects and measurements and including data about surgical outcomes and patients' recovery. Validation of systems combining the use of holographic headsets, SDKs and game engines is especially interesting as this approach facilitates an easy development of mobile AR applications and thus the implementation of AR-based image overlay surgery in clinical practice.

Surgineering: curriculum concept for experiential learning in upper-level biomedical engineering

PURPOSE

A strong foundation in the fundamental principles of medical intervention combined with genuine exposure to real clinical systems and procedures will improve engineering students' capability for informed innovation on clinical problems. To help build such a foundation, a new course (dubbed Surgineering) was developed to convey fundamental principles of surgery, interventional radiology (IR), and radiation therapy, with an emphasis on experiential learning, hands-on with real clinical systems, exposure to clinicians, and visits to real operating theaters. The concept, structure, and outcomes of the course of the first run of the first semester of the course are described.

METHOD

The course included six segments spanning fundamental concepts and cutting-edge approaches in a spectrum of surgical specialties, body and neurological IR, and radiation therapy. Each class involved a minimum of didactic content and an emphasis on hands-on experience with instrumentation, equipment, surgical approaches, anatomical models, dissection, and visits to clinical theaters. Outcomes on the quality of the course and areas for continuing improvement were assessed by student surveys (5-point Likert scores and word-cloud representations of free response) as well as feedback from clinical collaborators.

RESULT

Surveys assessed four key areas of feedback on the course and were analyzed quantitatively and in word-cloud representations of: (1) best aspects (hands-on experience with surgeons); (2) worst aspects (quizzes and reading materials); (3) areas for improvement (projects, quizzes, and background on anatomy); and (4) what prospective students should know (a lot background reading for every class). Five-point Likert scores from survey respondents (16/19 students) indicated: overall quality of the course 4.63 ± 0.72 (median 5.00); instructor teaching effectiveness 4.06 ± 1.06 (median 4.00); intellectual challenge 4.19 ± 0.40 (median 4.00); and workload somewhat heavier (62.5%) compared to other courses. Novel elements of the course included the opportunity to engage with clinical faculty and participate in realistic laboratory exercises, work with clinical instruments and equipment, and visit real operating theaters. An additional measure of the success of the course was evidenced by surveys and a strong escalation in enrollment in the following year.

CONCLUSIONS

The Surgineering course presents an important addition to upper-level engineering curricula and a valuable opportunity for engineering students to gain hands-on experience and interaction with clinical experts. Close partnership with clinical faculty was essential to the schedule and logistics of the course as well as to the continuity of concepts delivered over the semester. The knowledge and experience gained provides stronger foundation for identification of un-met clinical needs and ideation of new engineering approaches in medicine. The course also provides a valuable prerequisite to higher-level coursework in systems engineering, human factors, and data science applied to medicine.