Automated Registration of 3-D Knee Implant Models to Fluoroscopic Images Using Lipschitzian Optimization

A novel post-processing technique for correcting symmetric implant ambiguity in measuring total knee arthroplasty kinematics from single-plane fluoroscopy

Recent advancements in computer vision and machine learning enable autonomous measurement of total knee arthroplasty kinematics through single-plane fluoroscopy. However, symmetric components present challenges in optimization routines, causing "symmetry traps" and ambiguous poses. Achieving clinically robust kinematics measurement requires addressing this issue. We devised an algorithm that converts a "true" pose to its corresponding "symmetry trap" orientation. From a dataset of nearly 13,000 human supervised kinematics, this algorithm constructs an augmented dataset of "true" and "symmetry trap" kinematics, used to train eight classification machine learning algorithms. The outputs from the highest-performing algorithm classify kinematics sequences as 'obviously true' or 'potentially ambiguous.' We construct a spline through 'obviously true' poses, and 'ambiguous' poses are compared to the spline to determine correct orientation. The machine learning algorithms achieved 88-94% accuracy on our internal test set and 91-93% on our external test set. Applying our spline algorithm to kinematics sequences yielded 91.1% accuracy, 94% specificity, but 67% sensitivity. The accuracy of standard ML algorithms for implants within 5 degrees of a pure-lateral view was 71%, rising to 88% beyond 5 degrees. This pioneering study systematizes addressing model-image registration issues with symmetric tibial implants. High accuracy suggests potential use of ML algorithms to mitigate shape-ambiguity errors in pose measurements from single-plane fluoroscopy. Our results also suggest an imaging protocol for measuring kinematics that favors more oblique viewing angles, which could further disambiguate "true" and "symmetry trap" poses.

Scan-Free and Fully Automatic Tracking of Native Knee Anatomy from Dynamic Stereo-Radiography with Statistical Shape and Intensity Models

Kinematic tracking of native anatomy from stereo-radiography provides a quantitative basis for evaluating human movement. Conventional tracking procedures require significant manual effort and call for acquisition and annotation of subject-specific volumetric medical images. The current work introduces a framework for fully automatic tracking of native knee anatomy from dynamic stereo-radiography which forgoes reliance on volumetric scans. The method consists of three computational steps. First, captured radiographs are annotated with segmentation maps and anatomic landmarks using a convolutional neural network. Next, a non-convex polynomial optimization problem formulated from annotated landmarks is solved to acquire preliminary anatomy and pose estimates. Finally, a global optimization routine is performed for concurrent refinement of anatomy and pose. An objective function is maximized which quantifies similarities between masked radiographs and digitally reconstructed radiographs produced from statistical shape and intensity models. The proposed framework was evaluated against manually tracked trials comprising dynamic activities, and additional frames capturing a static knee phantom. Experiments revealed anatomic surface errors routinely below 1.0 mm in both evaluation cohorts. Median absolute errors of individual bone pose estimates were below 1.0∘ or mm for 15 out of 18 degrees of freedom in both evaluation cohorts. Results indicate that accurate pose estimation of native anatomy from stereo-radiography may be performed with significantly reduced manual effort, and without reliance on volumetric scans.

Joint Track Machine Learning: An Autonomous Method of Measuring Total Knee Arthroplasty Kinematics From Single-Plane X-Ray Images

BACKGROUND

Dynamic radiographic measurements of 3-dimensional (3-D) total knee arthroplasty (TKA) kinematics have provided important information for implant design and surgical technique for over 30 years. However, current methods of measuring TKA kinematics are too cumbersome, inaccurate, or time-consuming for practical clinical application. Even state-of-the-art techniques require human-supervision to obtain clinically reliable kinematics. Eliminating human supervision could potentially make this technology practical for clinical use.

METHODS

We demonstrate a fully autonomous pipeline for quantifying 3D-TKA kinematics from single-plane radiographic imaging. First, a convolutional neural network (CNN) segmented the femoral and tibial implants from the image. Second, those segmented images were compared to precomputed shape libraries for initial pose estimates. Lastly, a numerical optimization routine aligned 3D implant contours and fluoroscopic images to obtain the final implant poses.

RESULTS

The autonomous technique reliably produces kinematic measurements comparable to human-supervised measures, with root-mean-squared differences of less than 0.7 mm and 4° for our test data, and 0.8 mm and 1.7° for external validation studies.

CONCLUSION

A fully autonomous method to measure 3D-TKA kinematics from single-plane radiographic images produces results equivalent to a human-supervised method, and may soon make it practical to perform these measurements in a clinical setting.

Fully automatic tracking of native glenohumeral kinematics from stereo-radiography

The current work introduces a system for fully automatic tracking of native glenohumeral kinematics in stereo-radiography sequences. The proposed method first applies convolutional neural networks to obtain segmentation and semantic key point predictions in biplanar radiograph frames. Preliminary bone pose estimates are computed by solving a non-convex optimization problem with semidefinite relaxations to register digitized bone landmarks to semantic key points. Initial poses are then refined by registering computed tomography-based digitally reconstructed radiographs to captured scenes, which are masked by segmentation maps to isolate the shoulder joint. A particular neural net architecture which exploits subject-specific geometry is also introduced to improve segmentation predictions and increase robustness of subsequent pose estimates. The method is evaluated by comparing predicted glenohumeral kinematics to manually tracked values from 17 trials capturing 4 dynamic activities. Median orientation differences between predicted and ground truth poses were 1.7∘ and 8.6∘ for the scapula and humerus, respectively. Joint-level kinematics differences were less than 2∘ in 65%, 13%, and 63% of frames for XYZ orientation DoFs based on Euler angle decompositions. Automation of kinematic tracking can increase scalability of tracking workflows in research, clinical, or surgical applications.

Validation of a machine learning technique for segmentation and pose estimation in single plane fluoroscopy

Kinematics of total knee replacements (TKR) play an important role in assessing the success of a procedure and would be a valuable addition to clinical practice; however, measuring TKR kinematics is time consuming and labour intensive. Recently, an automatic single-plane fluoroscopic method utilizing machine learning has been developed to facilitate a quick and simple process for measuring TKR kinematics. This study aimed to validate the new automatic single-plane technique using biplanar radiostereometric analysis (RSA) as the gold standard. Twenty-four knees were imaged at various angles of flexion in a dedicated RSA lab and 113 image pairs were obtained. Only the lateral RSA images were used for the automatic single-plane technique to simulate single-plane fluoroscopy. Two networks helped automate the kinematics measurement process, one segmented implant components and the other generated an initial pose estimate for the optimization algorithm. Kinematics obtained via the automatic single plane and manual biplane techniques were compared using root-mean-square error and Bland-Altman plots. Two observers measured the kinematics using the automated technique and results were compared with assess reproducibility. Root-mean-square errors were 0.8 mm for anterior-posterior translation, 0.5 mm for superior-inferior translation, 2.6 mm for medial-lateral translation, 1.0° for flexion-extension, 1.2° for abduction-adduction, and 1.7° for internal-external rotation. Reproducibility, reported as root-mean-square errors between operator measurements, was submillimeter for in-plane translations and below 2° for all rotations. Clinical Significance: The advantages of the automated single plane technique should aid in the kinematic measurement process and help researchers and clinicians perform TKR kinematic analyses.

Personalised pose estimation from single-plane moving fluoroscope images using deep convolutional neural networks

Measuring joint kinematics is a key requirement for a plethora of biomechanical research and applications. While x-ray based systems avoid the soft-tissue artefacts arising in skin-based measurement systems, extracting the object’s pose (translation and rotation) from the x-ray images is a time-consuming and expensive task. Based on about 106’000 annotated images of knee implants, collected over the last decade with our moving fluoroscope during activities of daily living, we trained a deep-learning model to automatically estimate the 6D poses for the femoral and tibial implant components. By pretraining a single stage of our architecture using renderings of the implant geometries, our approach offers personalised predictions of the implant poses, even for unseen subjects. Our approach predicted the pose of both implant components better than about 0.75 mm (in-plane translation), 25 mm (out-of-plane translation), and 2° (all Euler-angle rotations) over 50% of the test samples. When evaluating over 90% of test samples, which included heavy occlusions and low contrast images, translation performance was better than 1.5 mm (in-plane) and 30 mm (out-of-plane), while rotations were predicted better than 3−4°. Importantly, this approach now allows for pose estimation in a fully automated manner.

Automatic tracking of healthy joint kinematics from stereo-radiography sequences

Kinematic tracking of healthy joints in radiography sequences is frequently performed by maximizing similarities between computed perspective projections of 3D computer models and corresponding objects' appearances in radiographic images. Significant human effort associated with manual tracking presents a major bottleneck in biomechanics research methods and limits the scale of target applications. The current work introduces a method for fully-automatic tracking of tibiofemoral and patellofemoral kinematics in stereo-radiography sequences for subjects performing dynamic activities. The proposed method involves the application of convolutional neural networks for annotating radiographs and a multi-stage optimization pipeline for estimating bone pose based on information provided by neural net predictions. Predicted kinematics are evaluated by comparing against manually-tracked trends across 20 distinct trials. Median absolute differences below 1.5 millimeters or degrees for 6 tibiofemoral and 3 patellofemoral degrees of freedom demonstrate the utility of our approach, which improves upon previous semi-automatic methods by enabling end-to-end automation. Implementation of a fully-automatic pipeline for kinematic tracking will benefit evaluation of human movement by enabling large-scale studies of healthy knee kinematics.

Kinematic Accuracy in Tracking Total Wrist Arthroplasty with Biplane Videoradiography using a CT-generated Model

Total Wrist Arthroplasty (TWA) for improving the functionality of severe wrist joint pathology has not had the same success, in parameters such as motion restoration and implant survival, as hip, knee, and shoulder arthroplasty. These other arthroplasties have been studied extensively, including the use of biplane videoradiography (BVR) that has allowed investigators to study the in-vivo motion of the total joint replacement during dynamic activities. The wrist has not been a previous focus, and utilization of BVR for wrist arthroplasty presents unique challenges due to the design characteristics of TWAs. Accordingly, the aims of this study were 1) to develop a methodology for generating TWA component models for use in BVR, and 2) to evaluate the accuracy of model-image registration in a single cadaveric model. A model of the carpal component was constructed from a CT scan, and a model of the radial component was generated from a surface scanner. BVR was acquired for three anatomical tasks from a cadaver specimen. Optical motion capture was used as the gold standard. BVR's bias in flexion/extension, radial/ulnar deviation, and pronosupination was less than 0.3°, 0.5°, and 0.6°. Translation bias was less than 0.2 mm with a standard deviation of less than 0.4 mm. This BVR technique achieved a kinematic accuracy comparable to previous studies on other total joint replacements. BVR's application to the study of TWA function in patients could advance the understanding of TWA and thus the implant's success.