3D Multi-Modality Medical Imaging: Combining Anatomical and Infrared Thermal Images for 3D Reconstruction

US & MR/CT Image Fusion with Markerless Skin Registration: A Proof of Concept

This paper presents an innovative automatic fusion imaging system that combines 3D CT/MR images with real-time ultrasound acquisition. The system eliminates the need for external physical markers and complex training, making image fusion feasible for physicians with different experience levels. The integrated system involves a portable 3D camera for patient-specific surface acquisition, an electromagnetic tracking system, and US components. The fusion algorithm comprises two main parts: skin segmentation and rigid co-registration, both integrated into the US machine. The co-registration aligns the surface extracted from CT/MR images with the 3D surface acquired by the camera, facilitating rapid and effective fusion. Experimental tests in different settings, validate the system's accuracy, computational efficiency, noise robustness, and operator independence.

Novel Entropy for Enhanced Thermal Imaging and Uncertainty Quantification

This paper addresses the critical need for precise thermal modeling in electronics, where temperature significantly impacts system reliability. We emphasize the necessity of accurate temperature measurement and uncertainty quantification in thermal imaging, a vital tool across multiple industries. Current mathematical models and uncertainty measures, such as Rényi and Shannon entropies, are inadequate for the detailed informational content required in thermal images. Our work introduces a novel entropy that effectively captures the informational content of thermal images by combining local and global data, surpassing existing metrics. Validated by rigorous experimentation, this method enhances thermal images' reliability and information preservation. We also present two enhancement frameworks that integrate an optimized genetic algorithm and image fusion techniques, improving image quality by reducing artifacts and enhancing contrast. These advancements offer significant contributions to thermal imaging and uncertainty quantification, with broad applications in various sectors.

Feasibility evaluation of radiotherapy positioning system guided by augmented reality and point cloud registration

PURPOSE

To develop a radiotherapy positioning system based on Point Cloud Registration (PCR) and Augmented Reality (AR), and to verify its feasibility.

METHODS

The optimal steps of PCR were investigated, and virtual positioning experiments were designed to evaluate its accuracy and speed. AR was implemented by Unity 3D and Vuforia for initial position correction, and PCR for precision registration, to achieve the proposed radiotherapy positioning system. Feasibility of the proposed system was evaluated through phantom positioning tests as well as real human positioning tests. Real human tests involved breath-holding positioning and free-breathing positioning tests. Evaluation metrics included 6 Degree of Freedom (DOF) deviations and Distance (D) errors. Additionally, the interaction between CBCT and the proposed system was envisaged through CBCT and optical cross-source PCR.

RESULTS

Point-to-plane iterative Closest Point (ICP), statistical filtering, uniform down-sampling, and optimal sampling ratio were determined for PCR procedure. In virtual positioning tests, a single registration took only 0.111 s, and the average D error for 15 patients was 0.015 ± 0.029 mm., Errors of phantom tests were at the sub-millimeter level, with an average D error of 0.6 ± 0.2 mm. In the real human positioning tests, the average accuracy of breath-holding positioning was still at the sub-millimeter level, where the errors of X, Y and Z axes were 0.59 ± 0.12 mm, 0.54 ± 0.12 mm, and 0.52 ± 0.09 mm, and the average D error was 0.96 ± 0.22 mm. In the free-breathing positioning, the average errors in X, Y, and Z axes were still less than 1 mm. Although the mean D error was 1.93 ± 0.36 mm, it still falls within a clinically acceptable error margin.

CONCLUSION

The AR and PCR-guided radiotherapy positioning system enables markerless, radiation-free and high-accuracy positioning, which is feasible in real-world scenarios.

Use of Infrared Thermography in Medical Diagnosis, Screening, and Disease Monitoring: A Scoping Review

Thermography provides non-invasive, radiation-free diagnostic imaging. Despite the extensive literature on medical thermography, a comprehensive overview of current applications is lacking. Hence, the aim of this scoping review is to identify the medical applications of passive infrared thermography and to catalogue the technical and environmental modalities. The diagnostic performance of thermography and the existence of specific reference data are evaluated, and research gaps and future tasks identified. The entire review process followed the Joanna Briggs Institute (JBI) approach and the results are reported according to PRISMA-ScR guidelines. The scoping review protocol is registered at the Open Science Framework (OSF). PubMed, CENTRAL, Embase, Web of Science, OpenGrey, OSF, and PROSPERO were searched using pretested search strategies based on the Population, Concept, Context (PCC) approach. According to the eligibility criteria, references were screened by two researchers independently. Seventy-two research articles were identified describing screening, diagnostic, or monitoring studies investigating the potential of thermography in a total of 17,314 participants within 38 different health conditions across 13 therapeutic areas. The use of several camera models from various manufacturers is described. These and other facts and figures are compiled and presented in a detailed, descriptive tabular and visual format. Thermography offers promising diagnostic capabilities, alone or in addition to conventional methods.

Visual Sensing and Depth Perception for Welding Robots and Their Industrial Applications

With the rapid development of vision sensing, artificial intelligence, and robotics technology, one of the challenges we face is installing more advanced vision sensors on welding robots to achieve intelligent welding manufacturing and obtain high-quality welding components. Depth perception is one of the bottlenecks in the development of welding sensors. This review provides an assessment of active and passive sensing methods for depth perception and classifies and elaborates on the depth perception mechanisms based on monocular vision, binocular vision, and multi-view vision. It explores the principles and means of using deep learning for depth perception in robotic welding processes. Further, the application of welding robot visual perception in different industrial scenarios is summarized. Finally, the problems and countermeasures of welding robot visual perception technology are analyzed, and developments for the future are proposed. This review has analyzed a total of 2662 articles and cited 152 as references. The potential future research topics are suggested to include deep learning for object detection and recognition, transfer deep learning for welding robot adaptation, developing multi-modal sensor fusion, integrating models and hardware, and performing a comprehensive requirement analysis and system evaluation in collaboration with welding experts to design a multi-modal sensor fusion architecture.

A Virtual, 3D Multimodal Approach to Victim and Crime Scene Reconstruction

In the last two decades, forensic pathology and crime scene investigations have seen a rapid increase in examination tools due to the implementation of several imaging techniques, e.g., CT and MR scanning, surface scanning and photogrammetry. These tools encompass relatively simple visualization tools to powerful instruments for performing virtual 3D crime scene reconstructions. A multi-modality and multiscale approach to a crime scene, where 3D models of victims and the crime scene are combined, offers several advantages. A permanent documentation of all evidence in a single 3D environment can be used during the investigation phases (e.g., for testing hypotheses) or during the court procedures (e.g., to visualize the scene and the victim in a more intuitive manner). Advanced computational approaches to understand what might have happened during a crime can also be applied by, e.g., performing a virtual animation of the victim in the actual context, which can provide important information about possible dynamics during the event. Here, we present an overview of the different techniques and modalities used in forensic pathology in conjunction with crime scene investigations. Based on our experiences, the advantages and challenges of an image-based multi-modality approach will be discussed, including how their use may introduce new visualization modalities in court, e.g., virtual reality (VR) and 3D printing. Finally, considerations about future directions in research will be mentioned.

Research on 3D Reconstruction of Binocular Vision Based on Thermal Infrared

Thermal infrared imaging is less affected by lighting conditions and smoke compared to visible light imaging. However, thermal infrared images often have lower resolution and lack rich texture details, making them unsuitable for stereo matching and 3D reconstruction. To enhance the quality of infrared stereo imaging, we propose an advanced stereo matching algorithm. Firstly, the images undergo preprocessing using a non-local mean noise reduction algorithm to remove thermal noise and achieve a smoother result. Subsequently, we perform camera calibration using a custom-made chessboard calibration board and Zhang's camera calibration method to obtain accurate camera parameters. Finally, the disparity map is generated using the SGBM (semi-global block matching) algorithm based on the weighted least squares method, enabling the 3D point cloud reconstruction of the object. The experimental results demonstrate that the proposed algorithm performs well in objects with sufficient thermal contrast and relatively simple scenes. The proposed algorithm reduces the average error value by 10.9 mm and the absolute value of the average error by 1.07% when compared with the traditional SGBM algorithm, resulting in improved stereo matching accuracy for thermal infrared imaging. While ensuring accuracy, our proposed algorithm achieves the stereo reconstruction of the object with a good visual effect, thereby holding high practical value.

A Workflow for Studying the Stump-Socket Interface in Persons with Transtibial Amputation through 3D Thermographic Mapping

The design and fitting of prosthetic sockets can significantly affect the acceptance of an artificial limb by persons with lower limb amputations. Clinical fitting is typically an iterative process, which requires patients' feedback and professional assessment. When feedback is unreliable due to the patient's physical or psychological conditions, quantitative measures can support decision-making. Specifically, monitoring the skin temperature of the residual limb can provide valuable information regarding unwanted mechanical stresses and reduced vascularization, which can lead to inflammation, skin sores and ulcerations. Multiple 2D images to examine a real-life 3D limb can be cumbersome and might only offer a partial assessment of critical areas. To overcome these issues, we developed a workflow for integrating thermographic information on the 3D scan of a residual limb, with intrinsic reconstruction quality measures. Specifically, workflow allows us to calculate a 3D thermal map of the skin of the stump at rest and after walking, and summarize this information with a single 3D differential map. The workflow was tested on a person with transtibial amputation, with a reconstruction accuracy lower than 3 mm, which is adequate for socket adaptation. We expect the workflow to improve socket acceptance and patients' quality of life.

Three-Dimensional Immersion Scanning Technique: A Scalable Low-Cost Solution for 3D Scanning Using Water-Based Fluid

Three-dimensional scanning technology has been traditionally used in the medical and engineering industries, but these scanners can be expensive or limited in their capabilities. This research aimed to develop low-cost 3D scanning using rotation and immersion in a water-based fluid. This technique uses a reconstruction approach similar to CT scanners but with significantly less instrumentation and cost than traditional CT scanners or other optical scanning techniques. The setup consisted of a container filled with a mixture of water and Xanthan gum. The object to be scanned was submerged at various rotation angles. A stepper motor slide with a needle was used to measure the fluid level increment as the object being scanned was submerged into the container. The results showed that the 3D scanning using immersion in a water-based fluid was feasible and could be adapted to a wide range of object sizes. The technique produced reconstructed images of objects with gaps or irregularly shaped openings in a low-cost fashion. A 3D printed model with a width of 30.7200 ± 0.2388 mm and height of 31.6800 ± 0.3445 mm was compared to its scan to evaluate the precision of the technique. Its width/height ratio (0.9697 ± 0.0084) overlaps the margin of error of the width/height ratio of the reconstructed image (0.9649 ± 0.0191), showing statistical similarities. The signal-to-noise ratio was calculated at around 6 dB. Suggestions for future work are made to improve the parameters of this promising, low-cost technique.