Registration of 3D freehand ultrasound to a bone model for orthopedic procedures of the forearm

Anatomy-aware computed tomography-to-ultrasound spine registration

BACKGROUND

Ultrasound (US) has demonstrated to be an effective guidance technique for lumbar spine injections, enabling precise needle placement without exposing the surgeon or the patient to ionizing radiation. However, noise and acoustic shadowing artifacts make US data interpretation challenging. To mitigate these problems, many authors suggested using computed tomography (CT)-to-US registration to align the spine in pre-operative CT to intra-operative US data, thus providing localization of spinal landmarks.

PURPOSE

In this paper, we propose a deep learning (DL) pipeline for CT-to-US registration and address the problem of a need for annotated medical data for network training. Firstly, we design a data generation method to generate paired CT-US data where the spine is deformed in a physically consistent manner. Secondly, we train a point cloud (PC) registration network using anatomy-aware losses to enforce anatomically consistent predictions.

METHODS

Our proposed pipeline relies on training the network on realistic generated data. In our data generation method, we model the properties of the joints and disks between vertebrae based on biomechanical measurements in previous studies. We simulate the supine and prone position deformation by applying forces on the spine models. We choose the spine models from 35 patients in VerSe dataset. Each spine is deformed 10 times to create a noise-free data with ground-truth segmentation at hand. In our experiments, we use one-leave-out cross-validation strategy to measure the performance and the stability of the proposed method. For each experiment, we choose generated PCs from three spines as the test set. From the remaining, data from 3 spines act as the validation set and we use the rest of the data for training the algorithm. To train our network, we introduce anatomy-aware losses and constraints on the movement to match the physics of the spine, namely, rigidity loss and bio-mechanical loss. We define rigidity loss based on the fact that each vertebra can only transform rigidly while the disks and the surrounding tissue are deformable. Second, by using bio-mechanical loss we stop the network from inferring extreme movements by penalizing the force needed to get to a certain pose.

RESULTS

To validate the effectiveness of our fully automated data generation pipeline, we qualitatively assess the fidelity of the generated data. This assessment involves verifying the realism of the spinal deformation and subsequently confirming the plausibility of the simulated ultrasound images. Next, we demonstrate that the introduction of the anatomy-aware losses brings us closer to state-of-the-art (SOTA) and yields a reduction of 0.25 mm in terms of target registration error (TRE) compared to using only mean squared error (MSE) loss on the generated dataset. Furthermore, by using the proposed losses, the rigidity loss in inference decreases which shows that the inferred deformation respects the rigidity of the vertebrae and only introduces deformations in the soft tissue area to compensate the difference to the target PC. We also show that our results are close to the SOTA for the simulated US dataset with TRE of 3.89 mm and 3.63 mm for the proposed method and SOTA respectively. In addition, we show that our method is more robust against errors in the initialization in comparison to SOTA and significantly achieves better results (TRE of 4.88 mm compared to 5.66 mm) in this experiment.

CONCLUSIONS

In conclusion, we present a pipeline for spine CT-to-US registration and explore the potential benefits of utilizing anatomy-aware losses to enhance registration results. Additionally, we propose a fully automatic method to synthesize paired CT-US data with physically consistent deformations, which offers the opportunity to generate extensive datasets for network training. The generated dataset and the source code for data generation and registration pipeline can be accessed via https://github.com/mfazampour/medphys_ct_us_registration.

Tracked ultrasound registration for intraoperative navigation during pediatric bone tumor resections with soft tissue components: a porcine cadaver study

PURPOSE

Resection of pediatric osteosarcoma in the extremities with soft tissue involvement presents surgical challenges due to difficult visualization and palpation of the tumor. Therefore, an adequate image-guided surgery (IGS) system is required for more accurate tumor resection. The use of a 3D model in combination with intraoperative tracked ultrasound (iUS) may enhance surgical decision making. This study evaluates the clinical feasibility of iUS as a surgical tool using a porcine cadaver model.

METHODS

First, a 3D model of the porcine lower limb was created based on preoperative scans. Second, the bone surface of the tibia was automatically detected with an iUS by a sweep on the skin. The bone surface of the preoperative 3D model was then matched with the bone surface detected by the iUS. Ten artificial targets were used to calculate the target registration error (TRE). Intraoperative performance of iUS IGS was evaluated by six pediatric surgeons and two pediatric oncologic orthopedists. Finally, user experience was assessed with a post-procedural questionnaire.

RESULTS

Eight registration procedures were performed with a mean TRE of 6.78 ± 1.33 mm. The surgeons agreed about the willingness for clinical implementation in their current clinical practice. They mentioned the additional clinical value of iUS in combination with the 3D model for the localization of the soft tissue components of the tumor. The concept of the proposed IGS system is considered feasible by the clinical panel, but the large TRE and degree of automation need to be addressed in further work.

CONCLUSION

The participating pediatric surgeons and orthopedists were convinced of the clinical value of the interaction between the iUS and the 3D model. Further research is required to improve the surgical accuracy and degree of automation of iUS-based registration systems for the surgical management of pediatric osteosarcoma.

Three-dimensional ultrasound image reconstruction based on 3D-ResNet in the musculoskeletal system using a 1D probe:ex vivoandin vivofeasibility studies

Objective. Three-dimensional (3D) ultrasound (US) is needed to provide sonographers with a more intuitive panoramic view of the complex anatomical structure, especially the musculoskeletal system. In actual scanning, sonographers may perform fast scanning using a one-dimensional (1D) array probe .at random angles to gain rapid feedback, which leads to a large US image interval and missing regions in the reconstructed volume.Approach.In this study, a 3D residual network (3D-ResNet) modified by a 3D global residual branch (3D-GRB) and two 3D local residual branches (3D-LRBs) was proposed to retain detail and reconstruct high-quality 3D US volumes with high efficiency using only sparse two-dimensional (2D) US images. The feasibility and performance of the proposed algorithm were evaluated onex vivoandin vivosets.Main results. High-quality 3D US volumes in the fingers, radial and ulnar bones, and metacarpophalangeal joints were obtained by the 3D-ResNet, respectively. Their axial, coronal, and sagittal slices exhibited rich texture and speckle details. Compared with kernel regression, voxel nearest-neighborhood, squared distance weighted methods, and a 3D convolution neural network in the ablation study, the mean peak-signal-to-noise ratio and mean structure similarity of the 3D-ResNet were up to 28.53 ± 1.29 dB and 0.98 ± 0.01, respectively, and the corresponding mean absolute error dropped to 0.023 ± 0.003 with a better resolution gain of 1.22 ± 0.19 and shorter reconstruction time.Significance.These results illustrate that the proposed algorithm can rapidly reconstruct high-quality 3D US volumes in the musculoskeletal system in cases of a large amount of data loss. This suggests that the proposed algorithm has the potential to provide rapid feedback and precise analysis of stereoscopic details in complex and meticulous musculoskeletal system scanning with a less limited scanning speed and pose variations for the 1D array probe.

A review of advances in image-guided orthopedic surgery

Orthopedic surgery remains technically demanding due to the complex anatomical structures and cumbersome surgical procedures. The introduction of image-guided orthopedic surgery (IGOS) has significantly decreased the surgical risk and improved the operation results. This review focuses on the application of recent advances in artificial intelligence (AI), deep learning (DL), augmented reality (AR) and robotics in image-guided spine surgery, joint arthroplasty, fracture reduction and bone tumor resection. For the pre-operative stage, key technologies of AI and DL based medical image segmentation, 3D visualization and surgical planning procedures are systematically reviewed. For the intra-operative stage, the development of novel image registration, surgical tool calibration and real-time navigation are reviewed. Furthermore, the combination of the surgical navigation system with AR and robotic technology is also discussed. Finally, the current issues and prospects of the IGOS system are discussed, with the goal of establishing a reference and providing guidance for surgeons, engineers, and researchers involved in the research and development of this area.

Anisotropic computational modelling of bony structures from CT data: An almost automatic procedure

BACKGROUND AND OBJECTIVE

The use of modelling techniques that combine CT data and bone tissue micromechanics is spreading in computational biomechanics. Finite Element models show great potential in surgical planning of intervention and in prediction of stress and strain fields through a non-invasive method. The main challenge pertains to the reliable characterization of bone mechanical behaviour. An almost automatic procedure is here defined, which provides computational models of bony structures considering the actual anisotropy of bone tissue response. The innovative aspect resides on the automatic detection of the directions of anisotropy as the eigenvectors of a three-dimensional distribution matrix of HU values.

METHODS

The procedure combines CT data and micromechanics modelling techniques. Regarding a specific location, the procedure reports both the orthotropic elastic constants, by the analysis of the local HU value, and the anisotropic material directions, by the analysis of the HU values distribution around the specific location.

RESULTS

The procedure returns the distribution of bone tissue orthotropic elasticity tensor. The procedure proves to correctly respect the differentiation between cortical and trabecular bone. Principal directions show to be consistent with experimental data from ultrasound measurements. Regarding the material mapping from voxel to FE model, the developed strategies show to be reliable, leading to marginal errors (lower than 10%) for most of CT voxels (more than 90%). The computational analyses of typical structural loading conditions lead to strain values that are comparable with results from strain gauges experimentations. The development and the exploitation of FE models of different bony structures allow assessing the reliability of the procedure for cortical bone.

CONCLUSIONS

The results highlight the potentialities of the procedure in providing accurate patient-specific biomechanical models of bony structures starting from CT data. The accuracy and the automatism of the procedure are important factors for the development of real time clinical tools. The main limitations of this work remain the not fully automatism and the reliability assessment, which is based mainly on cortical bone regions only.

Ultrasound Bone Segmentation: A Scoping Review of Techniques and Validation Practices

Ultrasound bone segmentation is an important yet challenging task for many clinical applications. Several works have emerged attempting to improve and automate bone segmentation, which has led to a variety of computational techniques, validation practices and applied clinical scenarios. We characterize this exciting and growing body of research by reviewing published ultrasound bone segmentation techniques. We review 56 articles in detail and categorize and discuss the image analysis techniques that have been used for bone segmentation. We highlight the general trends of this field in terms of clinical motivation, image analysis techniques, ultrasound modalities and the types of validation practices used to quantify segmentation performance. Finally, we present an outlook on promising areas of research based on the unaddressed needs for solving ultrasound bone segmentation.