Rigorous geometric self-calibrating bundle adjustment for a dual fluoroscopic imaging system

A Rigorous 2D-3D Registration Method for a High-Speed Bi-Planar Videoradiography Imaging System

High-speed biplanar videoradiography can derive the dynamic bony translations and rotations required for joint cartilage contact mechanics to provide insights into the mechanical processes and mechanisms of joint degeneration or pathology. A key challenge is the accurate registration of 3D bone models (from MRI or CT scans) with 2D X-ray image pairs. Marker-based or model-based 2D-3D registration can be performed. The former has higher registration accuracy owing to corresponding marker pairs. The latter avoids bead implantation and uses radiograph intensity or features. A rigorous new method based on projection strategy and least-squares estimation that can be used for both methods is proposed and validated by a 3D-printed bone with implanted beads. The results show that it can achieve greater marker-based registration accuracy than the state-of-the-art RSA method. Model-based registration achieved a 3D reconstruction accuracy of 0.79 mm. Systematic offsets between detected edges in the radiographs and their actual position were observed and modeled to improve the reconstruction accuracy to 0.56 mm (tibia) and 0.64 mm (femur). This method is demonstrated on in vivo data, achieving a registration precision of 0.68 mm (tibia) and 0.60 mm (femur). The proposed method allows the determination of accurate 3D kinematic parameters that can be used to calculate joint cartilage contact mechanics.

Enhancing Single-Plane Fluoroscopy: A Self-Calibrating Bundle Adjustment for Distortion Modeling

Single-plane fluoroscopy systems with image intensifiers remain commonly employed in a clinical setting. The imagery they capture is vulnerable to several types of geometric distortions introduced by the system's components and their assembly as well as interactions with the local and global magnetic fields. In this study, the application of a self-calibrating bundle adjustment is investigated as a method to correct geometric distortions in single-plane fluoroscopic imaging systems. The resulting calibrated imagery is then applied in the quantitative analysis of diaphragmatic motion and potential diagnostic applications to hemidiaphragm paralysis. The calibrated imagery is further explored and discussed in its potential impact on areas of surgical navigation. This work was accomplished through the application of a controlled experiment with three separate Philips Easy Diagnost R/F Systems. A highly redundant (~2500 to 3500 degrees-of-freedom) and geometrically strong network of 18 to 22 images of a low-cost target field was collected. The target field comprised 121 pre-surveyed tantalum beads embedded on a 25.4 mm × 25.4 mm acrylic base plate. The modeling process resulted in the estimation of five to eight distortion coefficients, depending on the system. The addition of these terms resulted in 83-85% improvement in terms of image point precision (model fit) and 85-95% improvement in 3D object reconstruction accuracy after calibration. This study demonstrates significant potential in enhancing the accuracy and reliability of fluoroscopic imaging, thereby improving the overall quality and effectiveness of medical diagnostics and treatments.

Comparison of Dynamic Knee Contact Mechanics with T2 Imaging in Different Ages of Healthy Participants

Aging is a known risk factor for Osteoarthritis (OA), however, relations between cartilage composition and aging remain largely unknown in understanding human OA. T2 imaging provides an approach to assess cartilage composition. Whether these T2 relaxation times in the joint contact region change with time during gait remain unexplored. The study purpose was to demonstrate a methodology for linking dynamic joint contact mechanics to cartilage composition as measured by T2 relaxometry. T2 relaxation times for unloaded cartilage were measured in a 3T General Electric magnetic resonance (MR) scanner in this preliminary study. High-speed biplanar video-radiography (HSBV) was captured for five 20-30-year-old and five 50-60-year-old participants with asymptomatic knees. By mapping the T2 cartilages to the dynamic contact regions, T2 values were averaged over the contact area at each measurement within the gait cycle. T2 values demonstrated a functional relationship across the gait cycle. There were no statistically significant differences between 20- and 30-year-old and 50-60-year-old participant T2 values at first force peak of the gait cycle in the medial femur (p = 1.00, U = 12) or in the medial tibia (p = 0.31, U = 7). In the medial and lateral femur in swing phase, the joint moved from a region of high T2 values at 75% of gait to a minimum at 85-95% of swing. The lateral femur and tibia demonstrated similar patterns to the medial compartments but were less pronounced. This research advances understanding of the linkage between cartilage contact and cartilage composition. The change from a high T2 value at ~ 75% of gait to a lower value near the initiation of terminal swing (90% gait) indicates that there are changes to T2 averages corresponding to changes in the contact region across the gait cycle. No differences were found between age groups for healthy participants. These preliminary findings provide interesting insights into the cartilage composition corresponding to dynamic cyclic motion and inform mechanisms of osteoarthritis.

The influence of smoothing techniques on the accuracy of the reference finite helical axis when applied to 2D-3D registrations

Highspeed Biplanar Videoradiography (HSBV) permits recording of 3D bone movements with sub-millimeter precision. 2D-3D registrations are performed to quantify bone movements, providing a series of affine transformation matrices (ATMs). These registrations may result in alignment errors that produce inaccurate kinematics. Smoothing techniques can be applied to the ATMs to reduce these inaccuracies. Which techniques are best for this application remain unknown. The purpose of this study was to investigate the performance of six smoothing techniques on ATMs obtained from HSBV. Performance was assessed by measuring the accuracy of three reference finite helical axis (rFHA) measures during a turntable rotation: orientation, dispersion, and rotation speed difference (RSD = rFHA RS-turntable RS). A 3D printed femur and tibia were mounted to the turntable and rotations recorded with HSBV. The rFHA was calculated for the bones using each smoothing technique and ranked using a Friedman test. The relative percent change to the unsmoothed data was reported. A spline filter with outlier removal (SPOUT) was ranked the best technique, producing the most accurate RSDs for the femur (-79.64%) and tibia (-70.59%). SPOUT was the top performing smoothing technique. Further investigations using SPOUT are required for in-vivo human movements.

Extrapolating distortion correction with local measurements for space-based multi-module splicing large-format infrared cameras

Conventional distortion correction methods with the classical models, including radial, decentering, and thin prism distortions and with the interpolation template, depend heavily on the evenly distributed measurement data on the entire focal plane. However, owing to the restricted cubage of the vacuum tank and the large size of the assembled camera, there is no more extra space for the amounted large-format camera to adjust with the 2D turntable during laboratory vacuum experiment, which, accordingly, makes the collected measurement points gathered in just one module of the focal plane and eventually results in poor correction accuracy of the mentioned approaches. Here, in terms of the problems above, an extrapolating distortion correction method with local measurements for space-based multi-module splicing large-format infrared cameras was proposed in this paper. Benefiting from the polynomial model not being affected by the distribution of data, a third-order polynomial model adopted for distortion correction is solved by using local measurements and extrapolated reasonably, which guarantees the global camera calibration. Experimental results show that the mean distortion error can be corrected within 0.5 pixels. This method overcoming the deficiency of local test points can effectively improve the correction accuracy of the large-format camera and provide a new idea for global high-precision calibration of on-orbit payloads based on local measurements.

Creep behavior of human knee joint determined with high-speed biplanar video-radiography and finite element simulation

Creep and relaxation of knee cartilage and meniscus have been extensively studied at the tissue level with constitutive laws well established. At the joint level, however, both experimental and model studies have been focused on either elastic or kinematic responses of the knee, where the time-dependent response is typically neglected for simplicity. The objectives of this study were to quantify the in-vivo creep behavior of human knee joints produced by the cartilaginous tissues and to use the relevant data to validate a previously proposed poromechanical model. Two participants with no history of leg injury volunteered for 3T magnetic resonance imaging (MRI) of their unloaded right knees and for biplanar video-radiography (BVR) of the same knees during standing on an instrumented treadmill for 10 min. Approximately 550 temporal data points were obtained for the in-vivo displacement of the right femur relative to the tibia of the knee. Models of the bones and soft tissues were derived from the MRI. The bone models were used to reconstruct the 3D bone kinematics measured using BVR. Ground reaction forces were simultaneously recorded for the right leg, which were used as input for the subject-specific finite element knee models. Cartilaginous tissues were modeled as fluid-saturated fibril-reinforced materials. In-vivo creep of the knee was experimentally observed for both participants, i.e., the joint displacement increased with time while the reaction forces at the foot were approximately constant. The creep displacements obtained from the finite element models compared well with the experimental data when the tissue properties were calibrated (Pearson correlation coefficient = 0.99). The results showed the capacity of the poromechanical knee model to capture the creep response of the joint. The combined experimental and model study may be used to understand the fluid-pressure load support and contact mechanics of the joint using material properties calibrated from the displacement data, which enhance the fidelity of model results.

Robust Self-Supervised Learning of Deterministic Errors in Single-Plane (Monoplanar) and Dual-Plane (Biplanar) X-Ray Fluoroscopy

Fluoroscopic imaging that captures X-ray images at video framerates is advantageous for guiding catheter insertions by vascular surgeons and interventional radiologists. Visualizing the dynamical movements non-invasively allows complex surgical procedures to be performed with less trauma to the patient. To improve surgical precision, endovascular procedures can benefit from more accurate fluoroscopy data via calibration. This paper presents a robust self-calibration algorithm suitable for single-plane and dual-plane fluoroscopy. A three-dimensional (3D) target field was imaged by the fluoroscope in a strong geometric network configuration. The unknown 3D positions of targets and the fluoroscope pose were estimated simultaneously by maximizing the likelihood of the Student-t probability distribution function. A smoothed k-nearest-neighbour (kNN) regression is then used to model the deterministic component of the image reprojection error of the robust bundle adjustment. The Maximum Likelihood Estimation step and the kNN regression step are then repeated iteratively until convergence. Four different error modeling schemes were compared while varying the quantity of training images. It was found that using a smoothed kNN regression can automatically model the systematic errors in fluoroscopy with similar accuracy as a human expert using a small training dataset. When all training images were used, the 3D mapping error was reduced from 0.61-0.83 mm to 0.04 mm post-calibration (94.2-95.7% improvement), and the 2D reprojection error was reduced from 1.17-1.31 to 0.20-0.21 pixels (83.2-83.8% improvement). When using biplanar fluoroscopy, the 3D measurement accuracy of the system improved from 0.60 mm to 0.32 mm (47.2% improvement).

Toward an End-to-End Calibration for Mobile C-Arm in Combination with a Depth Sensor for Surgical Augmented Reality Applications

C-arm X-ray imaging is commonly applied in operating rooms for guiding orthopedic surgeries. Augmented Reality (AR) with C-arm X-ray images during surgery is an efficient way to facilitate procedures for surgeons. However, the accurate calibration process for surgical AR based on C-arm is essential and still challenging due to the limitations of C-arm imaging systems, such as instability of C-arm calibration parameters and the narrow field of view. We extend existing methods using a depth camera and propose a new calibration procedure consisting of calibration of the C-arm imaging system, and 3D/2D calibration of an RGB-D camera and C-arm system with a new method to achieve reliable data and promising accuracy and, at the same time, consistent with standard surgical protocols. For the calibration procedure, we apply bundle adjustment equations with a 3D designed Lego multi-modal phantom, in contrast to the previous methods in which planar calibration phantoms were applied. By using our method, the visualization of the X-ray image upon the 3D data was done, and the achieved mean overlay error was 1.03 mm. The evaluations showed that the proposed calibration procedure provided promising accuracy for AR surgeries and it improved the flexibility and robustness of existing C-arm calibration methods for surgical augmented reality (using C-arm and RGB-D sensor). Moreover, the results showed the efficiency of our method to compensate for the effects of the C-arm movement on calibration parameters. It was shown that the obtained overlay error was improved for the non-zero rotation movement of C-arm by using a virtual detector.

Accurate Calibration Scheme for a Multi-Camera Mobile Mapping System

Mobile mapping systems (MMS) are increasingly used for many photogrammetric and computer vision applications, especially encouraged by the fast and accurate geospatial data generation. The accuracy of point position in an MMS is mainly dependent on the quality of calibration, accuracy of sensor synchronization, accuracy of georeferencing and stability of geometric configuration of space intersections. In this study, we focus on multi-camera calibration (interior and relative orientation parameter estimation) and MMS calibration (mounting parameter estimation). The objective of this study was to develop a practical scheme for rigorous and accurate system calibration of a photogrammetric mapping station equipped with a multi-projective camera (MPC) and a global navigation satellite system (GNSS) and inertial measurement unit (IMU) for direct georeferencing. The proposed technique is comprised of two steps. Firstly, interior orientation parameters of each individual camera in an MPC and the relative orientation parameters of each cameras of the MPC with respect to the first camera are estimated. In the second step the offset and misalignment between MPC and GNSS/IMU are estimated. The global accuracy of the proposed method was assessed using independent check points. A correspondence map for a panorama is introduced that provides metric information. Our results highlight that the proposed calibration scheme reaches centimeter-level global accuracy for 3D point positioning. This level of global accuracy demonstrates the feasibility of the proposed technique and has the potential to fit accurate mapping purposes.

Advancing quantitative techniques to improve understanding of the skeletal structure-function relationship

Although all functional movement arises from the interplay between the neurological, skeletal, and muscular systems, it is the skeletal system that forms the basic framework for functional movement. Central to understanding human neuromuscular development, along with the genesis of musculoskeletal pathologies, is quantifying how the human skeletal system adapts and mal-adapts to its mechanical environment. Advancing this understanding is hampered by an inability to directly and non-invasively measure in vivo strains, stresses, and forces on bone. Thus, we traditionally have turned to animal models to garner such information. These models enable direct in vivo measures that are not available for human subjects, providing information in regards to both skeletal adaptation and the interplay between the skeletal and muscular systems. Recently, there has been an explosion of new imaging and modeling techniques providing non-invasive, in vivo measures and estimates of skeletal form and function that have long been missing. Combining multiple modalities and techniques has proven to be one of our most valuable resources in enhancing our understanding of the form-function relationship of the human skeletal, muscular, and neurological systems. Thus, to continue advancing our knowledge of the structural-functional relationship, validation of current tools is needed, while development is required to limit the deficiencies in these tools and develop new ones.

Validating Dual Fluoroscopy System Capabilities for Determining In-Vivo Knee Joint Soft Tissue Deformation: A Strategy for Registration Error Management

Knee osteoarthritis (OA) causes structural and mechanical changes within tibiofemoral (TF) cartilage affecting tissue load deformation behavior. Quantifying in-vivo TF soft tissue deformations in healthy and early OA may provide a novel biomechanical marker, sensitive to alterations occurring prior to radiographic change. Dual Fluoroscopy (DF) allows accurate in-vivo TF soft tissue deformation assessment but requires validation. In-vivo healthy and early OA TF cartilage deforms 0.3-1.2mm during static standing full body-weight loading. Our aim was to establish minimum detectable displacement (MDD) for femoral translation in a DF system using a marker-based and markerless approach with variable image intensifier magnifications. An instrumented frame allowed controlled femur specimen translations. Bone positions were reconstructed from DF data using centroids of affixed steel beads (marker-based) and 2D-3D bone feature registration (markerless). Statistical analyses included independent samples t-tests and reliability analysis. Markerless measurements by three trained operators had large variations making it prudent to have an appropriate error management strategy when performing 2D-3D registration. Marker-based MDD improved with image resolution and was 0.05 mm at 3.2 LP/mm (LP: line pairs). Markerless MDD at 3.2 LP/mm was 0.08 mm. Average femur and tibia 2D-3D registrations yielded excellent reliability (84.4%). Therefore, DF images acquired at resolution greater than 3.2 LP/mm would be capable for determining accurate and reliable in-vivo healthy and early OA TF soft tissue deformation. This study provides a registration error management strategy for in-vivo TF soft tissue deformation assessment that could be applied for future clinical applications to establish non-invasive biomechanical markers for early OA diagnosis.