Ultrasound Bone Segmentation: A Scoping Review of Techniques and Validation Practices

Ultrasound-based 3D bone modelling in computer assisted orthopedic surgery - a review and future challenges

Computer-assisted orthopedic surgery requires precise representations of bone surfaces. To date, computed tomography constitutes the gold standard, but comes with a number of limitations, including costs, radiation and availability. Ultrasound has potential to become an alternative to computed tomography, yet suffers from low image quality and limited field-of-view. These shortcomings may be addressed by a fully automatic segmentation and model-based completion of 3D bone surfaces from ultrasound images. This survey summarizes the state-of-the-art in this field by introducing employed algorithms, and determining challenges and trends. For segmentation, a clear trend toward machine learning-based algorithms can be observed. For 3D bone model completion however, none of the published methods involve machine learning. Furthermore, data sets and metrics are identified as weak spots in current research, preventing development and evaluation of models that generalize well.

Hybrid CNN-transformer network for interactive learning of challenging musculoskeletal images

BACKGROUND AND OBJECTIVES

Segmentation of regions of interest (ROIs) such as tumors and bones plays an essential role in the analysis of musculoskeletal (MSK) images. Segmentation results can help with orthopaedic surgeons in surgical outcomes assessment and patient's gait cycle simulation. Deep learning-based automatic segmentation methods, particularly those using fully convolutional networks (FCNs), are considered as the state-of-the-art. However, in scenarios where the training data is insufficient to account for all the variations in ROIs, these methods struggle to segment the challenging ROIs that with less common image characteristics. Such characteristics might include low contrast to the background, inhomogeneous textures, and fuzzy boundaries.

METHODS

we propose a hybrid convolutional neural network - transformer network (HCTN) for semi-automatic segmentation to overcome the limitations of segmenting challenging MSK images. Specifically, we propose to fuse user-inputs (manual, e.g., mouse clicks) with high-level semantic image features derived from the neural network (automatic) where the user-inputs are used in an interactive training for uncommon image characteristics. In addition, we propose to leverage the transformer network (TN) - a deep learning model designed for handling sequence data, in together with features derived from FCNs for segmentation; this addresses the limitation of FCNs that can only operate on small kernels, which tends to dismiss global context and only focus on local patterns.

RESULTS

We purposely selected three MSK imaging datasets covering a variety of structures to evaluate the generalizability of the proposed method. Our semi-automatic HCTN method achieved a dice coefficient score (DSC) of 88.46 ± 9.41 for segmenting the soft-tissue sarcoma tumors from magnetic resonance (MR) images, 73.32 ± 11.97 for segmenting the osteosarcoma tumors from MR images and 93.93 ± 1.84 for segmenting the clavicle bones from chest radiographs. When compared to the current state-of-the-art automatic segmentation method, our HCTN method is 11.7%, 19.11% and 7.36% higher in DSC on the three datasets, respectively.

CONCLUSION

Our experimental results demonstrate that HCTN achieved more generalizable results than the current methods, especially with challenging MSK studies.

The Big Bang of Deep Learning in Ultrasound-Guided Surgery: A Review

Ultrasound (US) imaging is a paramount modality in many image-guided surgeries and percutaneous interventions, thanks to its high portability, temporal resolution, and cost-efficiency. However, due to its imaging principles, the US is often noisy and difficult to interpret. Appropriate image processing can greatly enhance the applicability of the imaging modality in clinical practice. Compared with the classic iterative optimization and machine learning (ML) approach, deep learning (DL) algorithms have shown great performance in terms of accuracy and efficiency for US processing. In this work, we conduct a comprehensive review on deep-learning algorithms in the applications of US-guided interventions, summarize the current trends, and suggest future directions on the topic.

Knee Bone Models From Ultrasound

The number of total knee arthroplasties performed worldwide is on the rise. Patient-specific planning and implants may improve surgical outcomes but require 3-D models of the bones involved. Ultrasound (US) may become a cheap and nonharmful imaging modality if the shortcomings of segmentation techniques in terms of automation, accuracy, and robustness are overcome; furthermore, any kind of US-based bone reconstruction must involve some kind of model completion to handle occluded areas, for example, the frontal femur. A fully automatic and robust processing pipeline is proposed, generating full bone models from 3-D freehand US scanning. A convolutional neural network (CNN) is combined with a statistical shape model (SSM) to segment and extrapolate the bone surface. We evaluate the method in vivo on ten subjects, comparing the US-based model to a magnetic resonance imaging (MRI) reference. The partial freehand 3-D record of the femur and tibia bones deviate by 0.7-0.8 mm from the MRI reference. After completion, the full bone model shows an average submillimetric error in the case of the femur and 1.24 mm in the case of the tibia. Processing of the images is performed in real time, and the final model fitting step is computed in less than a minute. It took an average of 22 min for a full record per subject.

Frequency-domain full-waveform inversion-based musculoskeletal ultrasound computed tomography

Recently, full-waveform inversion (FWI) has become a promising tool for ultrasound computed tomography (USCT). However, as a computationally intensive technique, FWI suffers from computational burden, especially in conventional time-domain full-waveform inversion (TDFWI). On the contrary, frequency-domain full-waveform inversion (FDFWI) provides a relatively high computational efficiency as the propagation of discrete frequencies is much cheaper than full time-domain modeling. FDFWI has already been applied in soft tissue imaging, such as breast, but for the musculoskeletal model with high impedance contrast between hard and soft tissues, there is still a lack of an effective source estimation method. In this paper, a water-referenced data calibration method is proposed to address the source estimation challenge in the presence of bones, which achieves consistency between the measured and simulated data before the FDFWI procedure. To avoid the cycle-skipping local minimum effect and facilitate the algorithm convergence, a starting frequency criterion for musculoskeletal FDFWI is further proposed. The feasibility of the proposed method is demonstrated by numerical studies on retrieving the anatomies of the leg models and different musculoskeletal lesions. The study extends the advanced FDFWI method to the musculoskeletal system and provides an alternative solution for musculoskeletal USCT imaging with high computational efficiency.

Accurate and robust registration method for computer-assisted high tibial osteotomy surgery

PURPOSE

Computer-assisted high tibial osteotomy (HTO) is a frequently used treatment technique for lower extremity orthopedics, and its small incision and low exposure area are major limitations in tibial registration. This work combines skin surface features and gives a suitable registration algorithm based on Iterative Closest Points (ICP) algorithm to improve registration results. Furthermore, the precision, stability and efficiency of the described method is evaluated.

METHODS

After the initialization stage, the bone surface and skin surface data are combined to construct registration features. Then, a steepest perturbation search method is performed after the ICP algorithm (SPS-ICP) to obtain the optimal transformation through several iterations. Finally, the registration result is evaluated by establishing ground-truth through manual landmarks.

RESULTS

Phantom experiments including simulated human tissue show that the proximal fiducial registration error (FRE) of our method can reach 0.80 ± 0.30 mm (mean ± SD) with an overall rotational error < 1° and translational error < 1.5 mm. Furthermore, it remains stable when the point set is sparse. The average registration time is less than 40 s to ensure the high efficiency of surgical operation.

CONCLUSIONS

The approach fully describes a well-defined framework without additional imaging acquisition equipment for Computer-assisted HTO. By the experiment on the basis of a phantom with simulated soft tissue, the proposed method enables the accurate and robust registration of the tibia, and its computation time meets the demands of surgery.

Anterior pelvic plane estimation for total hip arthroplasty using a joint ultrasound and statistical shape model based approach

Orienting properly the prosthetic cup in total hip arthroplasty is key to ensure the postoperative stability. Several navigation solutions have been developed to assist surgeons in orienting the cup regarding the anterior pelvic plane (APP), defined by both anterior superior iliac spines (ASIS) and the pubic symphysis. However acquiring the APP when the patient is ready for surgery, i.e., mainly in lateral decubitus, is difficult due to the contralateral ASIS being against the operating table. We propose a method to determine the APP from both (1) alternative anatomical landmarks which are easy to acquire with a navigated ultrasound probe and (2) a Statistical Shape Model (SSM) of the pelvis. After creating a pelvic SSM from 40 data, a SSM-based morphometric analysis has been carried out to identify the best anatomical landmarks allowing the easy determination of the APP. The proposed method has then been assessed with both in silico and in vivo experiments on respectively forty synthetic data, and five healthy volunteers. The in silico experiment shows the feasibility to determine the APP with an average error of 4.7^∘ by only acquiring the iliac crest, the anterior superior iliac spine, the anterior inferior iliac spine, and the pubic symphysis. The average in vivo error using the ultrasound modality was 7.3^∘ with an estimated impact on both the cup anteversion and inclination of 4.0^∘ and 1.7^∘ respectively. The proposed method shows promising results that could allow the determination of the APP in lateral decubitus with a clinically acceptable impact on the computation of the cup orientation.

An Investigation of Medial Tibial Component Overhang in Unicompartmental and Total Knee Replacements Using Ultrasound in the Outpatient Department

Tibial component overhang is known to be a contributor to worse outcomes in knee arthroplasty. The aim of this study is to investigate the presence of tibial component overhang, and whether overhang correlates to a higher local tenderness in both medial unicompartmental and total knee replacements. Also, to determine if a rotational projection phenomenon is presented with radiographs when investigating tibial component overhang. A prospective study, including 64 participants, was performed, where ultrasound measurements were compared with postoperative radiographs. Local tenderness was measured as a pressure pain threshold, determined at 3 months postoperatively using algometry. Sixty-two of sixty-four patients had an underdiagnosed medial overhang on radiographs, with a mean difference of 2.4 mm between radiographs and ultrasound (p < 0.001), presenting a rotational projection phenomenon. When comparing sites with ultrasound measured overhang to sites without overhang measured by ultrasound, a higher local tenderness was observed (p < 0.001). A positive linear correlation was found between patients' overhang and local tenderness (r = 0.2; p = 0.045). Subgroup analysis of medial overhang showed significantly higher tenderness than all other locations. No significant differences were seen for lateral overhang. An apparent rotational projection phenomenon of overhang on radiographs was seen, and a linear association between overhang and local tenderness was demonstrated. This study warrants the use of ultrasound when a surgeon is presented with a patient with postoperative medial tenderness, but no overhang can be seen on radiographs. It should also raise awareness of implant selection and positioning during surgery, especially avoiding the overhang to be localized directly medially.

Joint Spine Segmentation and Noise Removal From Ultrasound Volume Projection Images With Selective Feature Sharing

Volume Projection Imaging from ultrasound data is a promising technique to visualize spine features and diagnose Adolescent Idiopathic Scoliosis. In this paper, we present a novel multi-task framework to reduce the scan noise in volume projection images and to segment different spine features simultaneously, which provides an appealing alternative for intelligent scoliosis assessment in clinical applications. Our proposed framework consists of two streams: i) A noise removal stream based on generative adversarial networks, which aims to achieve effective scan noise removal in a weakly-supervised manner, i.e., without paired noisy-clean samples for learning; ii) A spine segmentation stream, which aims to predict accurate bone masks. To establish the interaction between these two tasks, we propose a selective feature-sharing strategy to transfer only the beneficial features, while filtering out the useless or harmful information. We evaluate our proposed framework on both scan noise removal and spine segmentation tasks. The experimental results demonstrate that our proposed method achieves promising performance on both tasks, which provides an appealing approach to facilitating clinical diagnosis.

A Review of the Methods on Cobb Angle Measurements for Spinal Curvature

Scoliosis is a common disease of the spine and requires regular monitoring due to its progressive properties. A preferred indicator to assess scoliosis is by the Cobb angle, which is currently measured either manually by the relevant medical staff or semi-automatically, aided by a computer. These methods are not only labor-intensive but also vary in precision by the inter-observer and intra-observer. Therefore, a reliable and convenient method is urgently needed. With the development of computer vision and deep learning, it is possible to automatically calculate the Cobb angles by processing X-ray or CT/MR/US images. In this paper, the research progress of Cobb angle measurement in recent years is reviewed from the perspectives of computer vision and deep learning. By comparing the measurement effects of typical methods, their advantages and disadvantages are analyzed. Finally, the key issues and their development trends are also discussed.

Can uncertainty estimation predict segmentation performance in ultrasound bone imaging?

PURPOSE

Segmenting bone surfaces in ultrasound (US) is a fundamental step in US-based computer-assisted orthopaedic surgeries. Neural network-based segmentation techniques are a natural choice for this, given promising results in related tasks. However, to gain widespread use, we must be able to know how much to trust segmentation networks during clinical deployment when ground-truth data is unavailable.

METHODS

We investigated alternative ways to measure the uncertainty of trained networks by implementing a baseline U-Net trained on a large dataset, together with three uncertainty estimation modifications: Monte Carlo dropout, test time augmentation, and ensemble learning. We measured the segmentation performance, calibration quality, and the ability to predict segmentation performance on test data. We further investigated the effect of data quality on these measures.

RESULTS

Overall, we found that ensemble learning with binary cross-entropy (BCE) loss achieved the best segmentation performance (mean Dice: 0.75-0.78 and RMS distance: 0.62-0.86mm) and the lowest calibration errors (mean: 0.22-0.28%). In contrast to previous studies of area or volumetric segmentation, we found that the resulting uncertainty measures are not reliable proxies for surface segmentation performance.

CONCLUSION

Our experiments indicate that a significant performance and confidence calibration boost can be achieved with ensemble learning and BCE loss, as tested on 13,687 US images containing various anatomies and imaging parameters. However, these techniques do not allow us to reliably predict future segmentation performance. The results of this study can be used to improve the calibration and performance of US segmentation networks.

Dual-task ultrasound spine transverse vertebrae segmentation network with contour regularization

3D ultrasound imaging has become one of the common diagnosis ways to assess scoliosis since it is radiation-free, real-time, and low-cost. Spine curvature angle measurement is an important step to assess scoliosis precisely. One way to calculate the angle is using the vertebrae features of the 2-D coronal images to identify the most tilted vertebrae. To do the measurement, the segmentation of the transverse vertebrae is an important step. In this paper, we propose a dual-task ultrasound transverse vertebrae segmentation network (D-TVNet) based on U-Net. First, we arrange an auxiliary shape regularization network to learn the contour segmentation of the bones. It improves the boundary segmentation and anti-interference ability of the U-Net by fusing some of the features of the auxiliary task and the main task. Then, we introduce the atrous spatial pyramid pooling (ASPP) module to the end of the down-sampling stage of the main task stream to improve the relative feature extraction ability. To further improve the boundary segmentation, we extendedly fuse the down-sampling output features of the auxiliary network in the ASPP. The experiment results show that the proposed D-TVNet achieves the best dice score of 86.68% and the mean dice score of 86.17% based on cross-validation, which is an improvement of 5.17% over the baseline U-Net. An automatic ultrasound spine bone segmentation network with promising results has been achieved.

3-D Ultrasound Imaging Reliability of Measuring Dysplasia Metrics in Infants

Developmental dysplasia of the hip is a hip abnormality that ranges from mild acetabular dysplasia to irreducible femoral head dislocations. While 2-D B-mode ultrasound (US)-based dysplasia metrics or disease metrics are currently used clinically to diagnose developmental dysplasia of the hip, such estimates suffer from high inter-exam variability. In this work, we propose and evaluate 3-D US-derived dysplasia metrics that are automatically computed and demonstrate that these automatically derived dysplasia metrics are considerably more reproducible. The key features of our automatic method are (i) a random forest-based learning technique to remove regions across the coronal axis that do not contain bone structures necessary for dysplasia-metric extraction, thereby reducing outliers; (ii) a bone segmentation method that uses rotation-invariant and intensity-invariant filters, thus remaining robust to signal dropout and varying bone morphology; (iii) a novel slice-based learning and 3-D reconstruction strategy to estimate a probability map of the hypoechoic femoral head in the US volume; and (iv) formulae for calculating the 3-D US-derived dysplasia metrics. We validate our proposed method on real clinical data acquired from 40 infant hip examinations. Results show a considerable (around 70%) reduction in variability in two key 3-D US-derived dysplasia metrics compared with their 2-D counterparts.

An efficient end-to-end CNN for segmentation of bone surfaces from ultrasound

The application of ultrasound (US) imaging in orthopedic surgery has always been a research direction. However, the various problems of US imaging hinder the development of computer assisted orthopedic surgery guided by US. US bone segmentation has been an important yet challenging task for many clinical applications. We propose a new end-to-end fully convolution network called BoneNet for real-time and accurate segmentation of bone surface from US image. The BoneNet employs the squeeze-and-excitation residual to realize a robust feature learning. In order to speed up the segmentation, we reduce the convolution kernel and used depth-wise separable convolution to reduce network parameters. In addition, we assessed the impact of different normalization operations and loss functions on network performance. Finally, we realize the comparison of the segmentation performance and generalization ability of the existing real-time US bone surface segmentation network under the unified dataset. We achieved an average Dice coefficient of 93.03 % on segmentation performance test, and 91.25 % on the generalization ability test. The results show that our proposed method ensures the real-time performance and achieves significant improvements in accuracy, which substantially outperformed the state-of-the-art.