Self-calibrating 3D-ultrasound-based bone registration for minimally invasive orthopedic surgery

A Method to Track 3D Knee Kinematics by Multi-Channel 3D-Tracked A-Mode Ultrasound

This paper introduces a method for measuring 3D tibiofemoral kinematics using a multi-channel A-mode ultrasound system under dynamic conditions. The proposed system consists of a multi-channel A-mode ultrasound system integrated with a conventional motion capture system (i.e., optical tracking system). This approach allows for the non-invasive and non-radiative quantification of the tibiofemoral joint's six degrees of freedom (DOF). We demonstrated the feasibility and accuracy of this method in the cadaveric experiment. The knee joint's motions were mimicked by manually manipulating the leg through multiple motion cycles from flexion to extension. To measure it, six custom ultrasound holders, equipped with a total of 30 A-mode ultrasound transducers and 18 optical markers, were mounted on various anatomical regions of the lower extremity of the specimen. During experiments, 3D-tracked intra-cortical bone pins were inserted into the femur and tibia to measure the ground truth of tibiofemoral kinematics. The results were compared with the tibiofemoral kinematics derived from the proposed ultrasound system. The results showed an average rotational error of 1.51 ± 1.13° and a translational error of 3.14 ± 1.72 mm for the ultrasound-derived kinematics, compared to the ground truth. In conclusion, this multi-channel A-mode ultrasound system demonstrated a great potential of effectively measuring tibiofemoral kinematics during dynamic motions. Its improved accuracy, nature of non-invasiveness, and lack of radiation exposure make this method a promising alternative to incorporate into gait analysis and prosthetic kinematic measurements later.

Coplanarity Constrained Ultrasound Probe Calibration Based on N-Wire Phantom

OBJECTIVE

N-wire phantom-based ultrasound probe calibration has been used widely in many freehand tracked ultrasound imaging systems. The calibration matrix is obtained by registering the coplanar point cloud in ultrasound space and non-coplanar point cloud in tracking sensor space based on the least squares method. This method is sensitive to outliers and loses the coplanar information of the fiducial points. In this article, we describe a coplanarity-constrained calibration algorithm focusing on these issues.

METHODS

We verified that the out-of-plane error along the oblique wire in the N-wire phantom followed a normal distribution and used it to remove the experimental outliers and fit the plane with the Levenberg-Marquardt algorithm. Then, we projected the points to the plane along the oblique wire. Coplanarity-constrained point cloud registration was used to calculate the transformation matrix.

RESULTS

Compared with the other two commonly used methods, our method had the best calibration precision and achieved 25% and 36% improvement of the mean calibration accuracy than the closed-form solution and in-plane error method respectively at depth 16. Experiments at different depths revealed that our algorithm had better performance in our setup.

CONCLUSION

Our proposed coplanarity-constrained calibration algorithm achieved significant improvement in both precision and accuracy compared with existing algorithms with the same N-wire phantom. It is expected that calibration accuracy will improve when the algorithm is applied to all other N-wire phantom-based calibration procedures.

Feasibility of tracked ultrasound registration for pelvic-abdominal tumor navigation: a patient study

PURPOSE

Surgical navigation techniques can guide surgeons in localizing pelvic-abdominal malignancies. For abdominal navigation, accurate patient registration is crucial and is generally performed using an intra-operative cone-beam CT (CBCT). However, this method causes 15-min surgical preparation workflow interruption and radiation exposure, and more importantly, it cannot be repeated during surgery to compensate for large patient movement. As an alternative, the accuracy and feasibility of tracked ultrasound (US) registration are assessed in this patient study.

METHODS

Patients scheduled for surgical navigation during laparotomy of pelvic-abdominal malignancies were prospectively included. In the operating room, two percutaneous tracked US scans of the pelvic bone were acquired: one in supine and one in Trendelenburg patient position. Postoperatively, the bone surface was semiautomatically segmented from US images and registered to the bone surface on the preoperative CT scan. The US registration accuracy was computed using the CBCT registration as a reference and acquisition times were compared. Additionally, both US measurements were compared to quantify the registration error caused by patient movement into Trendelenburg.

RESULTS

In total, 18 patients were included and analyzed. US registration resulted in a mean surface registration error of 1.2 ± 0.2 mm and a mean target registration error of 3.3 ± 1.4 mm. US acquisitions were 4 × faster than the CBCT scans (two-sample t-test P < 0.05) and could even be performed during standard patient preparation before skin incision. Patient repositioning in Trendelenburg caused a mean target registration error of 7.7 ± 3.3 mm, mainly in cranial direction.

CONCLUSION

US registration based on the pelvic bone is accurate, fast and feasible for surgical navigation. Further optimization of the bone segmentation algorithm will allow for real-time registration in the clinical workflow. In the end, this would allow intra-operative US registration to correct for large patient movement.

TRIAL REGISTRATION

This study is registered in ClinicalTrials.gov (NCT05637359).

Automatic spatial calibration of freehand ultrasound probe with a multilayer N-wire phantom

The classic N-wire phantom has been widely used in the calibration of freehand ultrasound probes. One of the main challenges of the phantom is accurately identifying N-fiducials in ultrasound images, especially with multiple N-wire structures. In this study, a method using a multilayer N-wire phantom for the automatic spatial calibration of ultrasound images is proposed. All dots in the ultrasound image are segmented, scored, and classified according to the unique geometric features of the multilayer N-wire phantom. A recognition method for identifying N-fiducials from the dots is proposed for calibrating the spatial transformation of the ultrasound probe. At depths of 9, 11, 13, and 15 cm, the reconstruction error of 50 points is 1.24 ± 0.16, 1.09 ± 0.06, 0.95 ± 0.08, 1.02 ± 0.05 mm, respectively. The reconstruction mockup test shows that the distance accuracy is 1.11 ± 0.82 mm at a depth of 15 cm.

Annotation-guided encoder-decoder network for bone extraction in ultrasound-assisted orthopedic surgery

The patients and surgeons are usually exposed in massive ionizing radiation during fluoroscopy-based navigation orthopedic surgery. Comparatively, ultrasound-assisted orthopedic surgery could not only decrease the risk of radiation but also provide rich navigation information. However, due to the artifacts in ultrasound images, the extraction of bone structure from ultrasound sequences can be a particularly difficult task, which leads to some major challenges in ultrasound-assisted orthopedic navigation. In this paper, we propose an annotation-guided encoder-decoder network (AGN) to extract bone structure from the radiation-free ultrasound sequences. Specifically, the variability of the ultrasound probe's pose leads to the change of the ultrasound frame during the acquisition of ultrasound sequences. Therefore, a feature alignment module deployed in the AGN model is used to achieve reliable matching across ultrasound frames. Moreover, inspired by the interactive ultrasound analysis, where user annotated foreground information can help target extraction, our AGN model incorporates the annotation information obtained by Siamese networks. Experimental results validated that the AGN model not only produced better bone surface extraction than state-of-the-art methods (IOU: 0.92 versus. 0.88), but also achieved almost real-time extraction with the speed about 15 frames per second. In addition, the acquired bone surface further provided radiation-free 3D intraoperative bone structure for the intuitive navigation of orthopedic surgery.

A CNN-based method to reconstruct 3-D spine surfaces from US images in vivo

Three-dimensional (3-D) reconstruction of the spine surface is of strong clinical relevance for the diagnosis and prognosis of spine disorders and intra-operative image guidance. In this paper, we report a new technique to reconstruct lumbar spine surfaces in 3-D from non-invasive ultrasound (US) images acquired in free-hand mode. US images randomly sampled from in vivo scans of 9 rabbits were used to train a U-net convolutional neural network (CNN). More specifically, a late fusion (LF)-based U-net trained jointly on B-mode and shadow-enhanced B-mode images was generated by fusing two individual U-nets and expanding the set of trainable parameters to around twice the capacity of a basic U-net. This U-net was then applied to predict spine surface labels in in vivo images obtained from another rabbit, which were then used for 3-D spine surface reconstruction. The underlying pose of the transducer during the scan was estimated by registering stacks of US images to a geometrical model derived from corresponding CT data and used to align detected surface points. Final performance of the reconstruction method was assessed by computing the mean absolute error (MAE) between pairs of spine surface points detected from US and CT and by counting the total number of surface points detected from US. Comparison was made between the LF-based U-net and a previously developed phase symmetry (PS)-based method. Using the LF-based U-net, the averaged number of US surface points across the lumbar region increased by 21.61% and MAE reduced by 26.28% relative to the PS-based method. The overall MAE (in mm) was 0.24±0.29. Based on these results, we conclude that: 1) the proposed U-net can detect the spine posterior arch with low MAE and large number of US surface points and 2) the newly proposed reconstruction framework may complement and, under certain circumstances, be used without the aid of an external tracking system in intra-operative spine applications.

Identification of ultrasound imaging markers to quantify long bone regeneration in a segmental tibial defect sheep model in vivo

The healing of large bone defects has been investigated for decades due to its complexity and clinical relevance. Ultrasound (US) methods have shown promise in monitoring bone healing, but no quantitative method to assess regenerated bone morphology in US images has been presented yet. In this study, we investigate new US morphometric parameters to quantify bone regeneration in vivo. A segmental tibial defect was surgically created and stabilized in a sheep animal model. US and computed tomography (CT) imaging data were collected two months post-surgery. New bone was assessed, reconstructed and quantified from the US and CT data using 3 morphometric parameters: the new-bone bulk (NBB), new-bone surface (NBS) and new-bone contact (NBC). The distance (mm) between surface reconstructions from repeated US was \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$0.49\pm 0.30$$\end{document}0.49±0.30 and from US and CT was \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$0.89\pm 0.49$$\end{document}0.89±0.49. In the mid-shaft of the defected tibia, US measurements of NBB, NBS and NBC were significantly higher than the corresponding CT measurements (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p < 0.001$$\end{document}p<0.001). Based on our results, we conclude that US may complement CT to reconstruct and quantify bone regrowth, especially in its early stages.

The state-of-the-art in ultrasound-guided spine interventions

During the last two decades, intra-operative ultrasound (iUS) imaging has been employed for various surgical procedures of the spine, including spinal fusion and needle injections. Accurate and efficient registration of pre-operative computed tomography or magnetic resonance images with iUS images are key elements in the success of iUS-based spine navigation. While widely investigated in research, iUS-based spine navigation has not yet been established in the clinic. This is due to several factors including the lack of a standard methodology for the assessment of accuracy, robustness, reliability, and usability of the registration method. To address these issues, we present a systematic review of the state-of-the-art techniques for iUS-guided registration in spinal image-guided surgery (IGS). The review follows a new taxonomy based on the four steps involved in the surgical workflow that include pre-processing, registration initialization, estimation of the required patient to image transformation, and a visualization process. We provide a detailed analysis of the measurements in terms of accuracy, robustness, reliability, and usability that need to be met during the evaluation of a spinal IGS framework. Although this review is focused on spinal navigation, we expect similar evaluation criteria to be relevant for other IGS applications.

Real-time, image-based slice-to-volume registration for ultrasound-guided spinal intervention

Real-time fusion of magnetic resonance (MR) and ultrasound (US) images could facilitate safe and accurate needle placement in spinal interventions. We develop an entirely image-based registration method (independent of or complementary to surgical trackers) that includes an efficient US probe pose initialization algorithm. The registration enables the simultaneous display of 2D ultrasound image slices relative to 3D pre-procedure MR images for navigation. A dictionary-based 3D-2D pose initialization algorithm was developed in which likely probe positions are predefined in a dictionary with feature encoding by Haar wavelet filters. Feature vectors representing the 2D US image are computed by scaling and translating multiple Haar basis filters to capture scale, location, and relative intensity patterns of distinct anatomical features. Following pose initialization, fast 3D-2D registration was performed by optimizing normalized cross-correlation between intra- and pre-procedure images using Powell's method. Experiments were performed using a lumbar puncture phantom and a fresh cadaver specimen presenting realistic image quality in spinal US imaging. Accuracy was quantified by comparing registration transforms to ground truth motion imparted by a computer-controlled motion system and calculating target registration error (TRE) in anatomical landmarks. Initialization using a 315-length feature vector yielded median translation accuracy of 2.7 mm (3.4 mm interquartile range, IQR) in the phantom and 2.1 mm (2.5 mm IQR) in the cadaver. By comparison, storing the entire image set in the dictionary and optimizing correlation yielded a comparable median accuracy of 2.1 mm (2.8 mm IQR) in the phantom and 2.9 mm (3.5 mm IQR) in the cadaver. However, the dictionary-based method reduced memory requirements by 47×  compared to storing the entire image set. The overall 3D error after registration measured using 3D landmarks was 3.2 mm (1.8 mm IQR) mm in the phantom and 3.0 mm (2.3 mm IQR) mm in the cadaver. The system was implemented in a 3D Slicer interface to facilitate translation to clinical studies. Haar feature based initialization provided accuracy and robustness at a level that was sufficient for real-time registration using an entirely image-based method for ultrasound navigation. Such an approach could improve the accuracy and safety of spinal interventions in broad utilization, since it is entirely software-based and can operate free from the cost and workflow requirements of surgical trackers.

3D Ultrasound for Orthopedic Interventions

Ultrasound is a real-time, non-radiation-based imaging modality with an ability to acquire two-dimensional (2D) and three-dimensional (3D) data. Due to these capabilities, research has been carried out in order to incorporate it as an intraoperative imaging modality for various orthopedic surgery procedures. However, high levels of noise, different imaging artifacts, and bone surfaces appearing blurred with several mm in thickness have prohibited the widespread use of ultrasound as a standard of care imaging modality in orthopedics. In this chapter, we provided a detailed overview of numerous applications of 3D ultrasound in the domain of orthopedic surgery. Specifically, we discuss the advantages and disadvantages of methods proposed for segmentation and enhancement of bone ultrasound data and the successful application of these methods in clinical domain. Finally, a number of challenges are identified which need to be overcome in order for ultrasound to become a preferred imaging modality in orthopedics.

Three-Dimensional Registration of Freehand-Tracked Ultrasound to CT Images of the Talocrural Joint

A rigid surface⁻volume registration scheme is presented in this study to register computed tomography (CT) and free-hand tracked ultrasound (US) images of the talocrural joint. Prior to registration, bone surfaces expected to be visible in US are extracted from the CT volume and bone contours in 2D US data are enhanced based on monogenic signal representation of 2D US images. A 3D monogenic signal data is reconstructed from the 2D data using the position of the US probe recorded with an optical tracking system. When registering the surface extracted from the CT scan to the monogenic signal feature volume, six transformation parameters are estimated so as to optimize the sum of monogenic signal features over the transformed surface. The robustness of the registration algorithm was tested on a dataset collected from 12 cadaveric ankles. The proposed method was used in a clinical case study to investigate the potential of US imaging for pre-operative planning of arthroscopic access to talar (osteo)chondral defects (OCDs). The results suggest that registrations with a registration error of 2 mm and less is achievable, and US has the potential to be used in assessment of an OCD' arthroscopic accessibility, given the fact that 51% of the talar surface could be visualized.

Fast and automatic bone segmentation and registration of 3D ultrasound to CT for the full pelvic anatomy: a comparative study

PURPOSE

Ultrasound (US) is a safer alternative to X-rays for bone imaging, and its popularity for orthopedic surgical navigation is growing. Routine use of intraoperative US for navigation requires fast, accurate and automatic alignment of tracked US to preoperative computed tomography (CT) patient models. Our group previously investigated image segmentation and registration to align untracked US to CT of only the partial pelvic anatomy. In this paper, we extend this to study the performance of these previously published techniques over the full pelvis in a tracked framework, to characterize their suitability in more realistic scenarios, along with an additional simplified segmentation method and similarity metric for registration.

METHOD

We evaluated phase symmetry segmentation, and Gaussian mixture model (GMM) and coherent point drift (CPD) registration methods on a pelvic phantom augmented with human soft tissue images. Additionally, we proposed and evaluated a simplified 3D bone segmentation algorithm we call Shadow-Peak (SP), which uses acoustic shadowing and peak intensities to detect bone surfaces. We paired this with a registration pipeline that optimizes the normalized cross-correlation (NCC) between distance maps of the segmented US-CT images.

RESULTS

SP segmentation combined with the proposed NCC registration successfully aligned tracked US volumes to the preoperative CT model in all trials, in contrast to the other techniques. SP with NCC achieved a median target registration error (TRE) of 2.44 mm (maximum 4.06 mm), when imaging all three anterior pelvic structures, and a mean runtime of 27.3 s. SP segmentation with CPD registration was the next most accurate combination: median TRE of 3.19 mm (maximum 6.07 mm), though a much faster runtime of 4.2 s.

CONCLUSION

We demonstrate an accurate, automatic image processing pipeline for intraoperative alignment of US-CT over the full pelvis and compare its performance with the state-of-the-art methods. The proposed methods are amenable to clinical implementation due to their high accuracy on realistic data and acceptably low runtimes.

Measuring relative positions and orientations of the tibia with respect to the femur using one-channel 3D-tracked A-mode ultrasound tracking system: A cadaveric study

The purpose of this study is to investigate the technical feasibility of measuring relative positions and orientations of the tibia with respect to the femur in an in-vitro experiment by using a 3D-tracked A-mode ultrasound system and to determine its accuracy of angular and translational measurements. As A-mode ultrasound is capable of detecting bone surface through soft tissue in a non-invasive manner, the combination of a single A-mode ultrasound transducer with an optical motion tracking system provides the possibility for digitizing the 3D locations of bony points at different anatomical regions on the thigh and the shank. After measuring bony points over a large area of both the femur and tibia, the bone models of the femur and tibia that were segmented from CT or MRI images were registered to the corresponding bony points. Then the relative position of the tibia with respect to the femur could be obtained and the angular and translational components could also be measured. A cadaveric experiment was conducted to assess its accuracy compared to the reference measurement obtained by optical markers fixed to intra-cortical bone pins placed in the femur and tibia. The results showed that the ultrasound system could achieve 0.49 ± 0.83°, 0.85 ± 1.86° and 1.85 ± 2.78° (mean ± standard deviation) errors for Flexion-Extension, Adduction-Abduction and External-Internal rotations, respectively, and -2.22 ± 3.62 mm, -2.80 ± 2.35 mm and -1.44 ± 2.90 mm errors for Anterior-Posterior, Proximal-Distal and Lateral-Medial translations, respectively. It was concluded that this technique is feasible and facilitates the integration of arrays of A-mode ultrasound transducers with an optical motion tracking system for non-invasive dynamic tibiofemoral kinematics measurement.

Bone surface enhancement in ultrasound images using a new Doppler-based acquisition/processing method

Ultrasound (US) imaging has long been considered as a potential aid in orthopedic surgeries. US technologies are safe, portable and do not use radiations. This would make them a desirable tool for real-time assessment of fractures and to monitor fracture healing. However, image quality of US imaging methods in bone applications is limited by speckle, attenuation, shadow, multiple reflections and other imaging artifacts. While bone surfaces typically appear in US images as somewhat 'brighter' than soft tissue, they are often not easily distinguishable from the surrounding tissue. Therefore, US imaging methods aimed at segmenting bone surfaces need enhancement in image contrast prior to segmentation to improve the quality of the detected bone surface. In this paper, we present a novel acquisition/processing technique for bone surface enhancement in US images. Inspired by elastography and Doppler imaging methods, this technique takes advantage of the difference between the mechanical and acoustic properties of bones and those of soft tissues to make the bone surface more easily distinguishable in US images. The objective of this technique is to facilitate US-based bone segmentation methods and improve the accuracy of their outcomes. The newly proposed technique is tested both in in vitro and in vivo experiments. The results of these preliminary experiments suggest that the use of the proposed technique has the potential to significantly enhance the detectability of bone surfaces in noisy ultrasound images.

Phantomless Auto-Calibration and Online Calibration Assessment for a Tracked Freehand 2-D Ultrasound Probe

This paper presents a method for automatically calibrating and assessing the calibration quality of an externally tracked 2-D ultrasound (US) probe by scanning arbitrary, natural tissues, as opposed a specialized calibration phantom as is the typical practice. A generative topic model quantifies the posterior probability of calibration parameters conditioned on local 2-D image features arising from a generic underlying substrate. Auto-calibration is achieved by identifying the maximum a-posteriori image-to-probe transform, and calibration quality is assessed online in terms of the posterior probability of the current image-to-probe transform. Both are closely linked to the 3-D point reconstruction error (PRE) in aligning feature observations arising from the same underlying physical structure in different US images. The method is of practical importance in that it operates simply by scanning arbitrary textured echogenic structures, e.g., in-vivo tissues in the context of the US-guided procedures, without requiring specialized calibration procedures or equipment. Observed data take the form of local scale-invariant features that can be extracted and fit to the model in near real-time. Experiments demonstrate the method on a public data set of in vivo human brain scans of 14 unique subjects acquired in the context of neurosurgery. Online calibration assessment can be performed at approximately 3 Hz for the US images of pixels. Auto-calibration achieves an internal mean PRE of 1.2 mm and a discrepancy of [2 mm, 6 mm] in comparison to the calibration via a standard phantom-based method.

Real-time Triple-modal Photoacoustic, Ultrasound, and Magnetic Resonance Fusion Imaging of Humans

Imaging that fuses multiple modes has become a useful tool for diagnosis and therapeutic monitoring. As a next step, real-time fusion imaging has attracted interest as for a tool to guide surgery. One widespread fusion imaging technique in surgery combines real-time ultrasound (US) imaging and pre-acquired magnetic resonance (MR) imaging. However, US imaging visualizes only structural information with relatively low contrast. Here, we present a photoacoustic (PA), US, and MR fusion imaging system which integrates a clinical PA and US imaging system with an optical tracking-based navigation sub-system. Through co-registration of pre-acquired MR and real-time PA/US images, overlaid PA, US, and MR images can be concurrently displayed in real time. We successfully acquired fusion images from a phantom and a blood vessel in a human forearm. This fusion imaging can complementarily delineate the morphological and vascular structure of tissues with good contrast and sensitivity, has a well-established user interface, and can be flexibly integrated with clinical environments. As a novel fusion imaging, the proposed triple-mode imaging can provide comprehensive image guidance in real time, and can potentially assist various surgeries.

Ultrasound imaging and segmentation of bone surfaces: A review

Due to its real-time, non-radiation based three-dimensional (3D) imaging capabilities, ultrasound (US) has been incorporated into various orthopedic procedures. However, imaging artifacts, low signal-to-noise ratio (SNR) and bone boundaries appearing several mm in thickness make the analysis of US data difficult. This paper provides a review about the state-of-the-art bone segmentation and enhancement methods developed for two-dimensional (2D) and 3D US data. First, an overview for the appearance of bone surface response in B-mode data is presented. Then, classification of the proposed techniques in terms of the image information being used is provided. Specifically, the focus is given on segmentation and enhancement of B-mode US data. The review is concluded by discussing future directions of research and additional challenges which need to be overcome in order to make this imaging modality more successful in orthopedics.

Enhancement of bone shadow region using local phase-based ultrasound transmission maps

PURPOSE

Ultrasound is increasingly being employed in different orthopedic procedures as an imaging modality for real-time guidance. Nevertheless, low signal-to-noise-ratio and different imaging artifacts continue to hamper the success of ultrasound-based procedures. Bone shadow region is an important feature indicating the presence of bone/tissue interface in the acquired ultrasound data. Enhancement and automatic detection of this region could improve the sensitivity of ultrasound for imaging bone and result in improved guidance for various orthopedic procedures.

METHODS

In this work, a method is introduced for the enhancement of bone shadow regions from B-mode ultrasound data. The method is based on the combination of three different image phase features: local phase tensor, local weighted mean phase angle, and local phase energy. The combined local phase image features are used as an input to an [Formula: see text] norm-based contextual regularization method which emphasizes uncertainty in the shadow regions. The enhanced bone shadow images are automatically segmented and compared against expert segmentation.

RESULTS

Qualitative and quantitative validation was performed on 100 in vivo US scans obtained from five subjects by scanning femur and vertebrae bones. Validation against expert segmentation achieved a mean dice similarity coefficient of 0.88.

CONCLUSIONS

The encouraging results obtained in this initial study suggest that the proposed method is promising enough for further evaluation. The calculated bone shadow maps could be incorporated into different ultrasound bone segmentation and registration approaches as an additional feature.

Strain-Initialized Robust Bone Surface Detection in 3-D Ultrasound

Three-dimensional ultrasound has been increasingly considered as a safe radiation-free alternative to radiation-based fluoroscopic imaging for surgical guidance during computer-assisted orthopedic interventions, but because ultrasound images contain significant artifacts, it is challenging to automatically extract bone surfaces from these images. We propose an effective way to extract 3-D bone surfaces using a surface growing approach that is seeded from 2-D bone contours. The initial 2-D bone contours are estimated from a combination of ultrasound strain images and envelope power images. Novel features of the proposed method include: (i) improvement of a previously reported 2-D strain imaging-based bone segmentation method by incorporation of a depth-dependent cumulative power of the envelope into the elastographic data; (ii) incorporation of an echo decorrelation measure-based weight to fuse the strain and envelope maps; (iii) use of local statistics of the bone surface candidate points to detect the presence of any bone discontinuity; and (iv) an extension of our 2-D bone contour into a 3-D bone surface by use of an effective surface growing approach. Our new method produced average improvements in the mean absolute error of 18% and 23%, respectively, on 2-D and 3-D experimental phantom data, compared with those of two state-of-the-art bone segmentation methods. Validation on 2-D and 3-D clinical in vivo data also reveals, respectively, an average improvement in the mean absolute fitting error of 55% and an 18-fold improvement in the computation time.

Non-ionizing real-time ultrasonography in implant and oral surgery: A feasibility study

PURPOSE

Ultrasound imaging has potential to complement radiographic imaging modalities in implant and oral surgery given that it is non-ionizing and provides instantaneous images of anatomical structures. For application in oral and dental imaging, its qualities are dependent on its ability to accurately capture these complex structures. Therefore, the aim of this feasibility study was to investigate ultrasound to image soft tissue, hard tissue surface topography and specific vital structures.

MATERIAL AND METHODS

A clinical ultrasound scanner, paired with two 14-MHz transducers of different sizes (one for extraoral and the other for intraoral scans), was used to scan the following structures on a fresh cadaver: (i) the facial bone surface and soft tissue of maxillary anterior teeth, (ii) the greater palatine foramen; (iii) the mental foramen and (iv) the lingual nerve. Multiple measurements relevant to these structures were made on the ultrasound images and compared to those on cone-beam computed tomography (CBCT) scans and/or direct measurements.

RESULTS

Ultrasound imaging could delineate hard tissue surfaces, including enamel, root dentin and bone as well as soft tissue with high resolution (110 μm wavelength). The greater palatine foramen, mental foramen and lingual nerve were clearly shown in ultrasound images. Merging ultrasound and CBCT images demonstrated overall spatial accuracy of ultrasound images, which was corroborated by data gathered from direct measurements.

CONCLUSION

For the first time, this study provides proof-of-concept evidence that ultrasound can be a real-time and non-invasive alternative for the evaluation of oral and dental anatomical structures relevant for implant and oral surgery.

User-friendly freehand ultrasound calibration using Lego bricks and automatic registration

PURPOSE

As an inexpensive, noninvasive, and portable clinical imaging modality, ultrasound (US) has been widely employed in many interventional procedures for monitoring potential tissue deformation, surgical tool placement, and locating surgical targets. The application requires the spatial mapping between 2D US images and 3D coordinates of the patient. Although positions of the devices (i.e., ultrasound transducer) and the patient can be easily recorded by a motion tracking system, the spatial relationship between the US image and the tracker attached to the US transducer needs to be estimated through an US calibration procedure. Previously, various calibration techniques have been proposed, where a spatial transformation is computed to match the coordinates of corresponding features in a physical phantom and those seen in the US scans. However, most of these methods are difficult to use for novel users.

METHODS

We proposed an ultrasound calibration method by constructing a phantom from simple Lego bricks and applying an automated multi-slice 2D-3D registration scheme without volumetric reconstruction. The method was validated for its calibration accuracy and reproducibility.

RESULTS

Our method yields a calibration accuracy of [Formula: see text] mm and a calibration reproducibility of 1.29 mm.

CONCLUSION

We have proposed a robust, inexpensive, and easy-to-use ultrasound calibration method.

Minimally invasive registration for computer-assisted orthopedic surgery: combining tracked ultrasound and bone surface points via the P-IMLOP algorithm

PURPOSE

We present a registration method for computer-assisted total hip replacement (THR) surgery, which we demonstrate to improve the state of the art by both reducing the invasiveness of current methods and increasing registration accuracy. A critical element of computer-guided procedures is the determination of the spatial correspondence between the patient and a computational model of patient anatomy. The current method for establishing this correspondence in robot-assisted THR is to register points intraoperatively sampled by a tracked pointer from the exposed proximal femur and, via auxiliary incisions, from the distal femur.

METHODS

In this paper, we demonstrate a noninvasive technique for sampling points on the distal femur using tracked B-mode ultrasound imaging and present a new algorithm for registering these data called Projected Iterative Most-Likely Oriented Point (P-IMLOP). Points and normal orientations of the distal bone surface are segmented from ultrasound images and registered to the patient model along with points sampled from the exposed proximal femur via a tracked pointer.

RESULTS

The proposed approach is evaluated using a bone- and tissue-mimicking leg phantom constructed to enable accurate assessment of experimental registration accuracy with respect to a CT-image-based model of the phantom. These experiments demonstrate that localization of the femur shaft is greatly improved by tracked ultrasound. The experiments further demonstrate that, for ultrasound-based data, the P-IMLOP algorithm significantly improves registration accuracy compared to the standard ICP algorithm.

CONCLUSION

Registration via tracked ultrasound and the P-IMLOP algorithm has high potential to reduce the invasiveness and improve the registration accuracy of computer-assisted orthopedic procedures.

Automatic bone detection and soft tissue aware ultrasound-CT registration for computer-aided orthopedic surgery

PURPOSE

The transfer of preoperative CT data into the tracking system coordinates within an operating room is of high interest for computer-aided orthopedic surgery. In this work, we introduce a solution for intra-operative ultrasound-CT registration of bones.

METHODS

We have developed methods for fully automatic real-time bone detection in ultrasound images and global automatic registration to CT. The bone detection algorithm uses a novel bone-specific feature descriptor and was thoroughly evaluated on both in-vivo and ex-vivo data. A global optimization strategy aligns the bone surface, followed by a soft tissue aware intensity-based registration to provide higher local registration accuracy.

RESULTS

We evaluated the system on femur, tibia and fibula anatomy in a cadaver study with human legs, where magnetically tracked bone markers were implanted to yield ground truth information. An overall median system error of 3.7 mm was achieved on 11 datasets.

CONCLUSION

Global and fully automatic registration of bones aquired with ultrasound to CT is feasible, with bone detection and tracking operating in real time for immediate feedback to the surgeon.

Local phase tensor features for 3-D ultrasound to statistical shape+pose spine model registration

Most conventional spine interventions are performed under X-ray fluoroscopy guidance. In recent years, there has been a growing interest to develop nonionizing imaging alternatives to guide these procedures. Ultrasound guidance has emerged as a leading alternative. However, a challenging problem is automatic identification of the spinal anatomy in ultrasound data. In this paper, we propose a local phase-based bone feature enhancement technique that can robustly identify the spine surface in ultrasound images. The local phase information is obtained using a gradient energy tensor filter. This information is used to construct local phase tensors in ultrasound images, which highlight the spine surface. We show that our proposed approach results in a more distinct enhancement of the bone surfaces compared to recently proposed techniques based on monogenic scale-space filters and logarithmic Gabor filters. We also demonstrate that registration accuracy of a statistical shape+pose model of the spine to 3-D ultrasound images can be significantly improved, using the proposed method, compared to those obtained using monogenic scale-space filters and logarithmic Gabor filters.

A novel registration method for computer-assisted total knee arthroplasty using a patient-specific registration guide

Total knee arthroplasty (TKA) is a surgical method for replacing a degenerated or diseased knee joint that can no longer perform daily functions with an artificial knee implant. In TKA, the artificial knee implant should be inserted such that it aligns well with the mechanical axis of the leg. Thus, precise bone cutting is essential. To improve TKA outcomes, a registration process is performed to locate the predetermined bone cutting area by calculating the position and posture of the femur and tibia. In this article, we propose a patient-specific registration guide that is able to significantly reduce registration time and effort without loss of accuracy. Furthermore, the patient-specific registration guide can be implemented with real-time registration, allowing continuous surgical information to be provided without the insertion of any tracking devices. The precision and accuracy of the proposed registration guide were confirmed through animal tests with a digitizer, stereo camera, and linear motion generator. The error of our registration method, including measurement and guide attachment errors, reached a maximum of 0.321 mm for one pair of cow legs.

Ultrasound bone detection using patient-specific CT prior

Registration of pre-operative CT datasets to intra-operative 3D freehand ultrasound has been of high interest for computer assisted orthopedic surgery. Feature-based registration relies on an accurate detection of the bone surface in the B-mode ultrasound images. In this work we present a fully automatic bone detection approach for US. The pre-operative CT is utilized to create a patient-specific bone model for our joint detection-registration framework. The model provides a geometric constraint for accurate and robust detection. Simultaneously to the detection, our method yields a close estimate of the rigid transformation from US to CT, which can be used as an initialization for further refinement through sophisticated intensity-/feature-based registration methods. We evaluated our approach on datasets of the human femur acquired in a cadaver study and demonstrate a mean bone detection error of below 0.4 mm.

A bounded iterative closest point method for minimally invasive registration of the femur

This article describes a novel method for image-based, minimally invasive registration of the femur, for application to computer-assisted unicompartmental knee arthroplasty. The method is adapted from the well-known iterative closest point algorithm. By utilising an estimate of the hip centre on both the preoperative model and intraoperative patient anatomy, the proposed 'bounded' iterative closest point algorithm robustly produces accurate varus-valgus and anterior-posterior femoral alignment with minimal distal access requirements. Similar to the original iterative closest point implementation, the bounded iterative closest point algorithm converges monotonically to the closest minimum, and the presented case includes a common method for global minimum identification. The bounded iterative closest point method has shown to have exceptional resistance to noise during feature acquisition through simulations and in vitro plastic bone trials, where its performance is compared to a standard form of the iterative closest point algorithm.

Ultrasound confidence maps using random walks

Advances in ultrasound system development have led to a substantial improvement of image quality and to an increased use of ultrasound in clinical practice. Nevertheless, ultrasound attenuation and shadowing artifacts cannot be entirely avoided and continue to challenge medical image computing algorithms. We introduce a method for estimating a per-pixel confidence in the information depicted by ultrasound images, referred to as an ultrasound confidence map, which emphasizes uncertainty in attenuated and/or shadow regions. Our main novelty is the modeling of the confidence estimation problem within a random walks framework by taking into account ultrasound specific constraints. The solution to the random walks equilibrium problem is global and takes the entire image content into account. As a result, our method is applicable to a variety of ultrasound image acquisition setups. We demonstrate the applicability of our confidence maps for ultrasound shadow detection, 3D freehand ultrasound reconstruction, and multi-modal image registration.

Automated intraoperative calibration for prostate cancer brachytherapy

PURPOSE

Prostate cancer brachytherapy relies on an accurate spatial registration between the implant needles and the TRUS image, called "calibration". The authors propose a new device and a fast, automatic method to calibrate the brachytherapy system in the operating room, with instant error feedback.

METHODS

A device was CAD-designed and precision-engineered, which mechanically couples a calibration phantom with an exact replica of the standard brachytherapy template. From real-time TRUS images acquired from the calibration device and processed by the calibration system, the coordinate transformation between the brachytherapy template and the TRUS images was computed automatically. The system instantly generated a report of the target reconstruction accuracy based on the current calibration outcome.

RESULTS

Four types of validation tests were conducted. First, 50 independent, real-time calibration trials yielded an average of 0.57 ± 0.13 mm line reconstruction error (LRE) relative to ground truth. Second, the averaged LRE was 0.37 ± 0.25 mm relative to ground truth in tests with six different commercial TRUS scanners operating at similar imaging settings. Furthermore, testing with five different commercial stepper systems yielded an average of 0.29 ± 0.16 mm LRE relative to ground truth. Finally, the system achieved an average of 0.56 ± 0.27 mm target registration error (TRE) relative to ground truth in needle insertion tests through the template in a water tank.

CONCLUSIONS

The proposed automatic, intraoperative calibration system for prostate cancer brachytherapy has achieved high accuracy, precision, and robustness.

Automatic bone localization and fracture detection from volumetric ultrasound images using 3-D local phase features

This article presents a novel method for bone segmentation from three-dimensional (3-D) ultrasound images that derives intensity-invariant 3-D local image phase measures that are then employed for extracting ridge-like features similar to those that occur at soft tissue/bone interfaces. The main contributions in this article include: (1) the extension of our previously proposed phase-symmetry-based bone surface extraction from two-dimensional (2-D) to 3-D images using 3-D Log-Gabor filters; (2) the design of a new framework for accuracy evaluation based on using computed tomography as a gold standard that allows the assessment of surface localization accuracy across the entire 3-D surface; (3) the quantitative validation of accuracy of our 3-D phase-processing approach on both intact and fractured bone surfaces using phantoms and ex vivo 3-D ultrasound scans; and (4) the qualitative validation obtained by scanning emergency room patients with distal radius and pelvis fractures. We show a 41% improvement in surface localization error over the previous 2-D phase symmetry method. The results demonstrate clearly visible segmentations of bone surfaces with a localization accuracy of <0.6 mm and mean errors in estimating fracture displacements below 0.6 mm. The results show that the proposed method is successful even for situations when the bone surface response is weak due to shadowing from muscle and fascia interfaces above the bone, which is a situation where the 2-D method fails.

A motion constrained cross-wire phantom for tracked 2D ultrasound calibration

PURPOSE

Ultrasound-guided 3D interventions require calibration to relate real-time 2D images with the position and orientation of the ultrasound probe. Capturing several images of a single fixed point from different viewpoints is a simple and commonly used approach, but it is cumbersome and tedious. A new phantom for calibration was designed, built and tested to simplify this process.

METHODS

A mechanical phantom that restricts the motion of the ultrasound probe was designed such that the ultrasound image always captures a designated fixed point. Software was implemented which computes calibration parameters. Although the software provides no scientific novelty, it is required to demonstrate the proof of concept and to assess the accuracy and precision of the calibration phantom. The software also illustrates how the phantom enables the fixed point to be located automatically, both in tracker device coordinates and in image pixel coordinates.

RESULTS

The phantom was used to capture several hundred images of a single fixed point in less than 1 min, with different probe positions and orientations around the fixed point and with the single fixed point located in different parts of the ultrasound image. It would not be feasible to capture the same number of images by manual alignment of the probe with the fixed point.

CONCLUSION

Images for single fixed point calibration can be captured easily and quickly with a new calibration phantom. Since a larger number of images can be used to compute the required parameters, the calibration robustness is increased.

Compensation of sound speed deviations in 3-D B-mode ultrasound for intraoperative determination of the anterior pelvic plane

An accurate determination of the pelvic orientation is inevitable for the correct cup prosthesis placement of navigated total hip arthroplasties. Conventionally, this step is accomplished by percutaneous palpation of anatomic landmarks. Sterility issues and an increased landmark localization error for obese patients lead to the application of B-mode ultrasound imaging in the field of computer-assisted orthopedic surgery. Many approaches have been proposed in the literature to replace the percutaneous digitization by 3-D B-mode ultrasound imaging. However, the correct depth localization of the pelvic landmarks could be significantly affected by the acoustic properties of the penetrated tissues. Imprecise depth estimation could lead to a miscalculation of the pelvic orientation and subsequently to a misalignment of the acetabular cup implant. But so far, no solution has been presented, which compensates for acoustic property differences for correct depth estimation. In this paper, we present a novel approach to determine pelvic orientation from ultrasound images by applying a hierarchical registration scheme based on patch statistical shape models to compensate for differences in speed of sound. The method was validated based on plastic bones and a cadaveric specimen.

Automatic adaptive parameterization in local phase feature-based bone segmentation in ultrasound

Intensity-invariant local phase features based on Log-Gabor filters have been recently shown to produce highly accurate localizations of bone surfaces from three-dimensional (3-D) ultrasound. A key challenge, however, remains in the proper selection of filter parameters, whose values have so far been chosen empirically and kept fixed for a given image. Since Log-Gabor filter responses widely change when varying the filter parameters, actual parameter selection can significantly affect the quality of extracted features. This article presents a novel method for contextual parameter selection that autonomously adapts to image content. Our technique automatically selects the scale, bandwidth and orientation parameters of Log-Gabor filters for optimizing local phase symmetry. The proposed approach incorporates principle curvature computed from the Hessian matrix and directional filter banks in a phase scale-space framework. Evaluations performed on carefully designed in vitro experiments demonstrate 35% improvement in accuracy of bone surface localization compared with empirically-set parameterization results. Results from a pilot in vivo study on human subjects, scanned in the operating room, show similar improvements.

Integration of model-based weighting into an ICP variant to account for measurement errors in intra-operative A-Mode ultrasound-based registration

This paper addresses error modeling in A-Mode ultrasound- (US-) based registration and integration of model-based weighting into the Random-ICP (R-ICP) algorithm. The R-ICP is a variant of the Iterative Closest Point (ICP) algorithm, and it was suggested for surface-based registration using A-Mode US in the context of skull surgery. In that application area the R-ICP could yield high accuracy even in case of a small number of data points and a very inaccurate user-interactive pre-registration. However, it cannot cope with unequal point uncertainty, which is an important drawback in the context of hip surgery: Uncertainty about the average speed of sound is an error source, whose impact on the registration accuracy increases with the thickness of the scanned soft tissue. It can, therefore, lead to considerable localization errors if a thick soft tissue layer is scanned, and it might vary a lot from data point to data point as the soft tissue thickness is inhomogeneous. The present work investigates how to account for this error source considering also other error sources such as the establishment of point correspondences. Simulation results show that registration accuracy can be substantially improved when model-based weighting is integrated into the R-ICP.

Three-dimensional ultrasound imaging

This review is about the development of three-dimensional (3D) ultrasonic medical imaging, how it works, and where its future lies. It assumes knowledge of two-dimensional (2D) ultrasound, which is covered elsewhere in this issue. The three main ways in which 3D ultrasound may be acquired are described: the mechanically swept 3D probe, the 2D transducer array that can acquire intrinsically 3D data, and the freehand 3D ultrasound. This provides an appreciation of the constraints implicit in each of these approaches together with their strengths and weaknesses. Then some of the techniques that are used for processing the 3D data and the way this can lead to information of clinical value are discussed. A table is provided to show the range of clinical applications reported in the literature. Finally, the discussion relating to the technology and its clinical applications to explain why 3D ultrasound has been relatively slow to be adopted in routine clinics is drawn together and the issues that will govern its development in the future explored.

Rapid dynamic image registration of the beating heart for diagnosis and surgical navigation

Dynamic cardiac magnetic resonance imaging (MR) and computed tomography (CT) provide cardiologists and cardiac surgeons with high-quality 4-D images for diagnosis and therapy, yet the effective use of these high-quality anatomical models remains a challenge. Ultrasound (US) is a flexible imaging tool, but the US images produced are often difficult to interpret unless they are placed within their proper 3-D anatomical context. The ability to correlate real-time 3-D US volumes (RT3D US) with dynamic MR/CT images would offer a significant contribution to improve the quality of cardiac procedures. In this paper, we present a rapid two-step method for registering RT3D US to high-quality dynamic 3-D MR/CT images of the beating heart. This technique overcomes some major limitations of image registration (such as the correct registration result not necessarily occurring at the maximum of the mutual information (MI) metric) using the MI metric. We demonstrate the effectiveness of our method in a dynamic heart phantom (DHP) study and a human subject study. The achieved mean target registration error of CT+US images in the phantom study is 2.59 mm. Validation using human MR/US volumes shows a target registration error of 1.76 mm. We anticipate that this technique will substantially improve the quality of cardiac diagnosis and therapies.

Bone surface localization in ultrasound using image phase-based features

Current practice in orthopedic surgery relies on intraoperative fluoroscopy as the main imaging modality for localization and visualization of bone tissue, fractures, implants and surgical tool positions. Ultrasound (US) has recently emerged as a potential nonionizing imaging alternative that promises safer operation while remaining relatively cheap and widely available. US images, however, often depict bone structures poorly, making automatic, accurate and robust localization of bone surfaces quite challenging. In this paper, we present a novel technique for automatic bone surface localization in US that uses local phase image information to derive symmetry-based features corresponding to tissue/bone interfaces through the use of 2-D Log-Gabor filters. We validate the performance of the proposed approach quantitatively using realistic phantom and in vitro experiments as well as qualitatively on in vivo data. Results demonstrate that the proposed technique detects bone surfaces with a localization mean error below 0.40 mm. Furthermore, small gaps between bone fragments can be detected with fracture displacement mean error below 0.33 mm for vertical misalignments, and 0.47 mm for horizontal misalignments.

Dynamic 2D ultrasound and 3D CT image registration of the beating heart

Two-dimensional ultrasound (US) is widely used in minimally invasive cardiac procedures due to its convenience of use and noninvasive nature. However, the low quality of US images often limits their utility as a means for guiding procedures, since it is often difficult to relate the images to their anatomical context. To improve the interpretability of the US images while maintaining US as a flexible anatomical and functional real-time imaging modality, we describe a multimodality image navigation system that integrates 2D US images with their 3D context by registering them to high quality preoperative models based on magnetic resonance imaging (MRI) or computed tomography (CT) images. The mapping from such a model to the patient is completed using spatial and temporal registrations. Spatial registration is performed by a two-step rapid registration method that first approximately aligns the two images as a starting point to an automatic registration procedure. Temporal alignment is performed with the aid of electrocardiograph (ECG) signals and a latency compensation method. Registration accuracy is measured by calculating the TRE. Results show that the error between the US and preoperative images of a beating heart phantom is 1.7 +/-0.4 mm, with a similar performance being observed in in vivo animal experiments.

A real-time freehand ultrasound calibration system with automatic accuracy feedback and control

This article describes a fully automatic, real-time, freehand ultrasound calibration system. The system was designed to be simple and sterilizable, intended for operating-room usage. The calibration system employed an automatic-error-retrieval and accuracy-control mechanism based on a set of ground-truth data. Extensive validations were conducted on a data set of 10,000 images in 50 independent calibration trials to thoroughly investigate the accuracy, robustness, and performance of the calibration system. On average, the calibration accuracy (measured in three-dimensional reconstruction error against a known ground truth) of all 50 trials was 0.66 mm. In addition, the calibration errors converged to submillimeter in 98% of all trials within 12.5 s on average. Overall, the calibration system was able to consistently, efficiently and robustly achieve high calibration accuracy with real-time performance.