Fracture reduction planning and guidance in orthopaedic trauma surgery via multi-body image registration

A bidirectional framework for fracture simulation and deformation-based restoration prediction in pelvic fracture surgical planning

Pelvic fracture is a severe trauma with life-threatening implications. Surgical reduction is essential for restoring the anatomical structure and functional integrity of the pelvis, requiring accurate preoperative planning. However, the complexity of pelvic fractures and limited data availability necessitate labor-intensive manual corrections in a clinical setting. We describe in this paper a novel bidirectional framework for automatic pelvic fracture surgical planning based on fracture simulation and structure restoration. Our fracture simulation method accounts for patient-specific pelvic structures, bone density information, and the randomness of fractures, enabling the generation of various types of fracture cases from healthy pelvises. Based on these features and on adversarial learning, we develop a novel structure restoration network to predict the deformation mapping in CT images before and after a fracture for the precise structural reconstruction of any fracture. Furthermore, a self-supervised strategy based on pelvic anatomical symmetry priors is developed to optimize the details of the restored pelvic structure. Finally, the restored pelvis is used as a template to generate a surgical reduction plan in which the fragments are repositioned in an efficient jigsaw puzzle registration manner. Extensive experiments on simulated and clinical datasets, including scans with metal artifacts, show that our method achieves good accuracy and robustness: a mean SSIM of 90.7% for restorations, with translational errors of 2.88 mm and rotational errors of 3.18°for reductions in real datasets. Our method takes 52.9 s to complete the surgical planning in the phantom study, representing a significant acceleration compared to standard clinical workflows. Our method may facilitate effective surgical planning for pelvic fractures tailored to individual patients in clinical settings.

PelviNet: A Collaborative Multi-agent Convolutional Network for Enhanced Pelvic Image Registration

PelviNet introduces a groundbreaking multi-agent convolutional network architecture tailored for enhancing pelvic image registration. This innovative framework leverages shared convolutional layers, enabling synchronized learning among agents and ensuring an exhaustive analysis of intricate 3D pelvic structures. The architecture combines max pooling, parametric ReLU activations, and agent-specific layers to optimize both individual and collective decision-making processes. A communication mechanism efficiently aggregates outputs from these shared layers, enabling agents to make well-informed decisions by harnessing combined intelligence. PelviNet's evaluation centers on both quantitative accuracy metrics and visual representations to elucidate agents' performance in pinpointing optimal landmarks. Empirical results demonstrate PelviNet's superiority over traditional methods, achieving an average image-wise error of 2.8 mm, a subject-wise error of 3.2 mm, and a mean Euclidean distance error of 3.0 mm. These quantitative results highlight the model's efficiency and precision in landmark identification, crucial for medical contexts such as radiation therapy, where exact landmark identification significantly influences treatment outcomes. By reliably identifying critical structures, PelviNet advances pelvic image analysis and offers potential enhancements for broader medical imaging applications, marking a significant step forward in computational healthcare.

Fragment distance-guided dual-stream learning for automatic pelvic fracture segmentation

Pelvic fracture is a complex and severe injury. Accurate diagnosis and treatment planning require the segmentation of the pelvic structure and the fractured fragments from preoperative CT scans. However, this segmentation is a challenging task, as the fragments from a pelvic fracture typically exhibit considerable variability and irregularity in the morphologies, locations, and quantities. In this study, we propose a novel dual-stream learning framework for the automatic segmentation and category labeling of pelvic fractures. Our method uniquely identifies pelvic fracture fragments in various quantities and locations using a dual-branch architecture that leverages distance learning from bone fragments. Moreover, we develop a multi-size feature fusion module that adaptively aggregates features from diverse receptive fields tailored to targets of different sizes and shapes, thus boosting segmentation performance. Extensive experiments on three pelvic fracture datasets from different medical centers demonstrated the accuracy and generalizability of the proposed method. It achieves a mean Dice coefficient and mean Sensitivity of 0.935±0.068 and 0.929±0.058 in the dataset FracCLINIC, and 0.955±0.072 and 0.912±0.125 in the dataset FracSegData, which are superior than other comparing methods. Our method optimizes the process of pelvic fracture segmentation, potentially serving as an effective tool for preoperative planning in the clinical management of pelvic fractures.

Clinical validation of artificial intelligence-based preoperative virtual reduction for Neer 3- or 4-part proximal humerus fractures

BACKGROUND

If reduction images of fractures can be provided in advance with artificial-intelligence (AI)-based technology, it can assist with preoperative surgical planning. Recently, we developed the AI-based preoperative virtual reduction model for orthopedic trauma, which can provide an automatic segmentation and reduction of fractured fragments. The purpose of this study was to validate a quality of reduction model of Neer 3- or 4-part proximal humerus fractures established by AI-based technology.

METHODS

To develop the AI-based preoperative virtual reduction model, deep learning performed the segmentation of fracture fragments, and a Monte Carlo simulation completed the virtual reduction to determine the best model. A total of 20 pre/postoperative three-dimensional computed tomography (CT) scans of proximal humerus fracture were prepared. The preoperative CT scans were employed as the input of AI-based automated reduction (AI-R) to deduce the reduction models of fracture fragments, meanwhile, the manual reduction (MR) was conducted using the same CT images. Dice similarity coefficient (DSC) and intersection over union (IoU) between the reduction model from the AI-R/MR and postoperative CT scans were evaluated. Working times were compared between the two groups. Clinical validity agreement (CVA) and reduction quality score (RQS) were investigated for clinical validation outcomes by 20 orthopedic surgeons.

RESULTS

The mean DSC and IoU were better when using AI-R that when using MR (0.78 ± 0.13 vs. 0.69 ± 0.16, p < 0.001 and 0.65 ± 0.16 vs. 0.55 ± 0.18, p < 0.001, respectively). The working time of AI-R was, on average, 1.41% of that of MR. The mean CVA of all cases was 81%±14.7% (AI-R, 82.25%±14.27%; MR, 76.75%±14.17%, p = 0.06). The mean RQS was significantly higher when AI-R compared with MR was used (91.47 ± 1.12 vs. 89.30 ± 1.62, p = 0.045).

CONCLUSION

The AI-based preoperative virtual reduction model showed good performance in the reduction model in proximal humerus fractures with faster working times. Beyond diagnosis, classification, and outcome prediction, the AI-based technology can change the paradigm of preoperative surgical planning in orthopedic surgery.

LEVEL OF EVIDENCE

Level IV.

Experimental research based on robot-assisted surgery: Lower limb fracture reduction surgery planning navigation system

Background and Aims

Lower extremity fracture reduction surgery is a key step in the treatment of lower extremity fractures. How to ensure high precision of fracture reduction while reducing secondary trauma during reduction is a difficult problem in current surgery.

Methods

First, segmentation and three-dimensional reconstruction are performed based on fracture computed tomography images. A cross-sectional point cloud extraction algorithm based on the normal filtering of the long axis of the bone is designed to obtain the cross-sectional point clouds of the distal bone and the proximal bone, and the optimal reset target pose of the broken bone is obtained by using the iterative closest point algorithm. Then, the optimal reset sequence of reset parameters was determined, combined with the broken bone collision detection algorithm, a surgical planning algorithm for lower limb fracture reset was proposed, which can effectively reduce the reset force while ensuring the accuracy of the reset process without collision.

Results

The average error of the reduction of the model bone was within 1.0 mm. The reduction operation using the planning and navigation system of lower extremity fracture reduction surgery can effectively reduce the reduction force. At the same time, it can better ensure the smooth change of the reduction force.

Conclusion

Planning and navigation system of lower extremity fracture reduction surgery is feasible and effective.

Fast X-ray/CT image registration based on perspective projection triangular features

X-ray/CT image registration plays a pivotal role in enhancing surgical navigation success rates. However, challenges stemming from sparse and noisy X-ray image features, coupled with the complexities of multi-parameter optimization, impose limitations on existing methods in terms of registration accuracy and efficiency. In response, this paper presents an innovative approach-a fast X-ray/CT image registration method based on perspective projection triangular features(F-PPTF). By leveraging the conformal nature of perspective projection, the proposed method constructs perspective projection triangular features with rotation, translation, and scale invariance using point feature descriptors. Diverging from multi-parameter iterative optimization techniques, this approach achieves the decoupling of the six transformation parameters. This decoupling simplifies computational intricacies, thereby facilitating swift registration. Experimental evaluations conducted on synthetic and real X-ray images reveal an average rotational absolute error of 0.41°, an average translational absolute error of 1.16 mm, and an average registration time of 14.89 s. In comparison to conventional registration methodologies, the method presented in this paper demonstrates pronounced superiority in terms of both registration accuracy and efficiency, thereby exhibiting heightened potential for broader applicability.

An automatic reduction method of 3D bone fragments based on a novel section contour point descriptor

Comminuted fractures are orthopedic traumas with greater surgical difficulty. In clinical treatment, a great challenge is precise reduction of multiple broken bone fragments; Another great challenge is personalized and precise internal fixation after reduction. For these two issues, we designed an automated method framework for precise reduction and internal fixation of comminuted fractures. First, the Gaussian mixture model (GMM) is used to distinguish section points and noise points in a broken bone model; Second, ellipse fitting is carried out to achieve section points matching and a descriptor is proposed to describe the section features; Then, the Convolution Auto-Encoder (CAE) and genetic algorithm are used to extract feature vectors; Finally, after broken bone models registration, internal fixed plate can be reconstructed. Three verification experiments for comminuted bone fracture show this method has high accuracy and good efficiency. It can provide support for minimally invasive treatment for comminuted fractures.

Automatic segmentation of inconstant fractured fragments for tibia/fibula from CT images using deep learning

Orthopaedic surgeons need to correctly identify bone fragments using 2D/3D CT images before trauma surgery. Advances in deep learning technology provide good insights into trauma surgery over manual diagnosis. This study demonstrates the application of the DeepLab v3+ -based deep learning model for the automatic segmentation of fragments of the fractured tibia and fibula from CT images and the results of the evaluation of the performance of the automatic segmentation. The deep learning model, which was trained using over 11 million images, showed good performance with a global accuracy of 98.92%, a weighted intersection over the union of 0.9841, and a mean boundary F1 score of 0.8921. Moreover, deep learning performed 5-8 times faster than the experts' recognition performed manually, which is comparatively inefficient, with almost the same significance. This study will play an important role in preoperative surgical planning for trauma surgery with convenience and speed.

Two-Stage Structure-Focused Contrastive Learning for Automatic Identification and Localization of Complex Pelvic Fractures

Pelvic fracture is a severe trauma with a high rate of morbidity and mortality. Accurate and automatic diagnosis and surgical planning of pelvic fracture require effective identification and localization of the fracture zones. This is a challenging task due to the complexity of pelvic fractures, which often exhibit multiple fragments and sites, large fragment size differences, and irregular morphology. We have developed a novel two-stage method for the automatic identification and localization of complex pelvic fractures. Our method is unique in that it allows to combine the symmetry properties of the pelvic anatomy and capture the symmetric feature differences caused by the fracture on both the left and right sides, thereby overcoming the limitations of existing methods which consider only image or geometric features. It implements supervised contrastive learning with a novel Siamese deep neural network, which consists of two weight-shared branches with a structural attention mechanism, to minimize the confusion of local complex structures of the pelvic bones with the fracture zones. A structure-focused attention (SFA) module is designed to capture the spatial structural features and enhances the recognition ability of fracture zones. Comprehensive experiments on 103 clinical CT scans from the publicly available dataset CTPelvic1K show that our method achieves a mean accuracy and sensitivity of 0.92 and 0.93, which are superior to those reported with three SOTA contrastive learning methods and five advanced classification networks, demonstrating the effectiveness of identifying and localizing various types of complex pelvic fractures from clinical CT images.

Semi-Supervised Medical Image Segmentation Guided by Bi-Directional Constrained Dual-Task Consistency

BACKGROUND

Medical image processing tasks represented by multi-object segmentation are of great significance for surgical planning, robot-assisted surgery, and surgical safety. However, the exceptionally low contrast among tissues and limited available annotated data makes developing an automatic segmentation algorithm for pelvic CT challenging.

METHODS

A bi-direction constrained dual-task consistency model named PICT is proposed to improve segmentation quality by leveraging free unlabeled data. First, to learn more unmarked data features, it encourages the model prediction of the interpolated image to be consistent with the interpolation of the model prediction at the pixel, model, and data levels. Moreover, to constrain the error prediction of interpolation interference, PICT designs an auxiliary pseudo-supervision task that focuses on the underlying information of non-interpolation data. Finally, an effective loss algorithm for both consistency tasks is designed to ensure the complementary manner and produce more reliable predictions.

RESULTS

Quantitative experiments show that the proposed PICT achieves 87.18%, 96.42%, and 79.41% mean DSC score on ACDC, CTPelvic1k, and the individual Multi-tissue Pelvis dataset with gains of around 0.8%, 0.5%, and 1% compared to the state-of-the-art semi-supervised method. Compared to the baseline supervised method, the PICT brings over 3-9% improvements.

CONCLUSIONS

The developed PICT model can effectively leverage unlabeled data to improve segmentation quality of low contrast medical images. The segmentation result could improve the precision of surgical path planning and provide input for robot-assisted surgery.

Multi-Body 3D-2D Registration for Robot-Assisted Joint Reduction: Preclinical Evaluation in the Ankle Syndesmosis

Purpose

Existing methods to improve the accuracy of tibiofibular joint reduction present workflow challenges, high radiation exposure, and a lack of accuracy and precision, leading to poor surgical outcomes. To address these limitations, we propose a method to perform robot-assisted joint reduction using intraoperative imaging to align the dislocated fibula to a target pose relative to the tibia.

Methods

The approach (1) localizes the robot via 3D-2D registration of a custom plate adapter attached to its end effector, (2) localizes the tibia and fibula using multi-body 3D-2D registration, and (3) drives the robot to reduce the dislocated fibula according to the target plan. The custom robot adapter was designed to interface directly with the fibular plate while presenting radiographic features to aid registration. Registration accuracy was evaluated on a cadaveric ankle specimen, and the feasibility of robotic guidance was assessed by manipulating a dislocated fibula in a cadaver ankle.

Results

Using standard AP and mortise radiographic views registration errors were measured to be less than 1 mm and 1° for the robot adapter and the ankle bones. Experiments in a cadaveric specimen revealed up to 4 mm deviations from the intended path, which was reduced to <2 mm using corrective actions guided by intraoperative imaging and 3D-2D registration.

Conclusions

Preclinical studies suggest that significant robot flex and tibial motion occur during fibula manipulation, motivating the use of the proposed method to dynamically correct the robot trajectory. Accurate robot registration was achieved via the use of fiducials embedded within the custom design. Future work will evaluate the approach on a custom radiolucent robot design currently under construction and verify the solution on additional cadaveric specimens.

Intelligent robot-assisted minimally invasive reduction system for reduction of unstable pelvic fractures

OBJECTIVE

Currently, minimally invasive internal fixation is recommended for the surgical treatment of unstable pelvic fractures. The premise and difficulty of minimally invasive internal fixation are minimally invasive reduction of fractures. This review aimed to investigate the indications, surgical strategy and techniques, safety, and efficacy of intelligent robot-assisted fracture reduction (RAFR) system of pelvic ring injuries.

METHODS

This retrospective study reviewed a case series from March 2021 to November 2021. A total of 22 patients with unstable pelvic fracture injuries underwent minimally invasive internal fixations. All pelvic ring fractures were reduced with our intelligent RAFR system. The robot system intelligently designs the optimal position and reduction path based on the patient's preoperative 3D CT. During the operation, the three-dimensional visualization of the fracture is realized through image registration, and the Robot completes the automatic reduction of the fracture. The global 3D point cloud error between the preoperative planning results and the actual postoperative reduction results was calculated. The postoperative reduction results of residual displacement were graded by the Matta Criteria.

RESULTS

Minimally invasive closed reduction procedures were completed in all 22 cases with our RAFR system. The average global 3D point cloud reduction error between the preoperative planning results and the actual postoperative reduction results was 3.41mm±1.83mm. The mean residual displacement was 4.61mm±3.29mm. Given the Matta criteria, 16 cases were excellent, five were good, and one was fair, with an excellent and good rate of 95.5%.

CONCLUSION

Our new pelvic fracture reduction robot system can complete intelligent and minimally invasive fracture reduction for most patients with unstable pelvic fractures. The system has intelligent reduction position and path planning and realizes stable pelvis control through a unique holding arm and a robotic arm. The operation process will not cause additional damage to the patient, which fully meets the clinical requirements. Our study demonstrated the safety and effectiveness of our robotic reduction system and its applicability and usability in clinical practice, thus paving the way towards Robot minimally invasive pelvic fracture surgeries.

A review of advances in image-guided orthopedic surgery

Orthopedic surgery remains technically demanding due to the complex anatomical structures and cumbersome surgical procedures. The introduction of image-guided orthopedic surgery (IGOS) has significantly decreased the surgical risk and improved the operation results. This review focuses on the application of recent advances in artificial intelligence (AI), deep learning (DL), augmented reality (AR) and robotics in image-guided spine surgery, joint arthroplasty, fracture reduction and bone tumor resection. For the pre-operative stage, key technologies of AI and DL based medical image segmentation, 3D visualization and surgical planning procedures are systematically reviewed. For the intra-operative stage, the development of novel image registration, surgical tool calibration and real-time navigation are reviewed. Furthermore, the combination of the surgical navigation system with AR and robotic technology is also discussed. Finally, the current issues and prospects of the IGOS system are discussed, with the goal of establishing a reference and providing guidance for surgeons, engineers, and researchers involved in the research and development of this area.

Computer-assisted preoperative planning of bone fracture fixation surgery: A state-of-the-art review

Background: Bone fracture fixation surgery is one of the most commonly performed surgical procedures in the orthopedic field. However, fracture healing complications occur frequently, and the choice of the most optimal surgical approach often remains challenging. In the last years, computational tools have been developed with the aim to assist preoperative planning procedures of bone fracture fixation surgery. Objectives: The aims of this review are 1) to provide a comprehensive overview of the state-of-the-art in computer-assisted preoperative planning of bone fracture fixation surgery, 2) to assess the clinical feasibility of the existing virtual planning approaches, and 3) to assess their clinical efficacy in terms of clinical outcomes as compared to conventional planning methods. Methods: A literature search was performed in the MEDLINE-PubMed, Ovid-EMBASE, Ovid-EMCARE, Web of Science, and Cochrane libraries to identify articles reporting on the clinical use of computer-assisted preoperative planning of bone fracture fixation. Results: 79 articles were included to provide an overview of the state-of-the art in virtual planning. While patient-specific geometrical model construction, virtual bone fracture reduction, and virtual fixation planning are routinely applied in virtual planning, biomechanical analysis is rarely included in the planning framework. 21 of the included studies were used to assess the feasibility and efficacy of computer-assisted planning methods. The reported total mean planning duration ranged from 22 to 258 min in different studies. Computer-assisted planning resulted in reduced operation time (Standardized Mean Difference (SMD): -2.19; 95% Confidence Interval (CI): -2.87, -1.50), less blood loss (SMD: -1.99; 95% CI: -2.75, -1.24), decreased frequency of fluoroscopy (SMD: -2.18; 95% CI: -2.74, -1.61), shortened fracture healing times (SMD: -0.51; 95% CI: -0.97, -0.05) and less postoperative complications (Risk Ratio (RR): 0.64, 95% CI: 0.46, 0.90). No significant differences were found in hospitalization duration. Some studies reported improvements in reduction quality and functional outcomes but these results were not pooled for meta-analysis, since the reported outcome measures were too heterogeneous. Conclusion: Current computer-assisted planning approaches are feasible to be used in clinical practice and have been shown to improve clinical outcomes. Including biomechanical analysis into the framework has the potential to further improve clinical outcome.

An Experimental Study of a 3D Bone Position Estimation System Based on Fluoroscopic Images

To compare a 3D preoperative planning image and fluoroscopic image, a 3D bone position estimation system that displays 3D images in response to changes in the position of fluoroscopic images was developed. The objective of the present study was to evaluate the accuracy of the estimated position of 3D bone images with reference to fluoroscopic images. Bone positions were estimated from reference points on a fluoroscopic image compared with those on a 3D image. The four reference markers positional relationships on the fluoroscopic image were compared with those on the 3D image to evaluate whether a 3D image may be drawn by tracking positional changes in the radius model. Intra-class correlations coefficients for reference marker distances between the fluoroscopic image and 3D image were 0.98–0.99. Average differences between measured values on the fluoroscopic image and 3D bone image for each marker corresponding to the direction of the bone model were 1.1 ± 0.7 mm, 2.4 ± 1.8 mm, 1.4 ± 0.8 mm, and 2.0 ± 1.6 mm in the anterior-posterior view, ulnar side lateral view, posterior-anterior view, and radial side lateral view, respectively. Marker positions were more accurate in the anterior-posterior and posterior-anterior views than in the radial and ulnar side lateral views. This system helps in real-time comparison of dynamic changes in preoperative 3D and intraoperative fluoroscopy images.

The Impact of Machine Learning on 2D/3D Registration for Image-Guided Interventions: A Systematic Review and Perspective

Image-based navigation is widely considered the next frontier of minimally invasive surgery. It is believed that image-based navigation will increase the access to reproducible, safe, and high-precision surgery as it may then be performed at acceptable costs and effort. This is because image-based techniques avoid the need of specialized equipment and seamlessly integrate with contemporary workflows. Furthermore, it is expected that image-based navigation techniques will play a major role in enabling mixed reality environments, as well as autonomous and robot-assisted workflows. A critical component of image guidance is 2D/3D registration, a technique to estimate the spatial relationships between 3D structures, e.g., preoperative volumetric imagery or models of surgical instruments, and 2D images thereof, such as intraoperative X-ray fluoroscopy or endoscopy. While image-based 2D/3D registration is a mature technique, its transition from the bench to the bedside has been restrained by well-known challenges, including brittleness with respect to optimization objective, hyperparameter selection, and initialization, difficulties in dealing with inconsistencies or multiple objects, and limited single-view performance. One reason these challenges persist today is that analytical solutions are likely inadequate considering the complexity, variability, and high-dimensionality of generic 2D/3D registration problems. The recent advent of machine learning-based approaches to imaging problems that, rather than specifying the desired functional mapping, approximate it using highly expressive parametric models holds promise for solving some of the notorious challenges in 2D/3D registration. In this manuscript, we review the impact of machine learning on 2D/3D registration to systematically summarize the recent advances made by introduction of this novel technology. Grounded in these insights, we then offer our perspective on the most pressing needs, significant open problems, and possible next steps.

Drill-mounted video guidance for orthopaedic trauma surgery

Purpose: Percutaneous fracture fixation is a challenging procedure that requires accurate interpretation of fluoroscopic images to insert guidewires through narrow bone corridors. We present a guidance system with a video camera mounted onboard the surgical drill to achieve real-time augmentation of the drill trajectory in fluoroscopy and/or CT. Approach: The camera was mounted on the drill and calibrated with respect to the drill axis. Markers identifiable in both video and fluoroscopy are placed about the surgical field and co-registered by feature correspondences. If available, a preoperative CT can also be co-registered by 3D-2D image registration. Real-time guidance is achieved by virtual overlay of the registered drill axis on fluoroscopy or in CT. Performance was evaluated in terms of target registration error (TRE), conformance within clinically relevant pelvic bone corridors, and runtime. Results: Registration of the drill axis to fluoroscopy demonstrated median TRE of 0.9 mm and 2.0 deg when solved with two views (e.g., anteroposterior and lateral) and five markers visible in both video and fluoroscopy-more than sufficient to provide Kirschner wire (K-wire) conformance within common pelvic bone corridors. Registration accuracy was reduced when solved with a single fluoroscopic view ( TRE = 3.4    mm and 2.7 deg) but was also sufficient for K-wire conformance within pelvic bone corridors. Registration was robust with as few as four markers visible within the field of view. Runtime of the initial implementation allowed fluoroscopy overlay and/or 3D CT navigation with freehand manipulation of the drill up to 10    frames / s . Conclusions: A drill-mounted video guidance system was developed to assist with K-wire placement. Overall workflow is compatible with fluoroscopically guided orthopaedic trauma surgery and does not require markers to be placed in preoperative CT. The initial prototype demonstrates accuracy and runtime that could improve the accuracy of K-wire placement, motivating future work for translation to clinical studies.