Simple and Rapid Creation of Customized 3-dimensional Printed Bolus Using iPhone X True Depth Camera

Smartphone applications for facial scanning: A technical and scoping review

INTRODUCTION

Facial scanning through smartphone scanning applications (SSA) is increasingly being used for medical applications as cost-effective, chairside method. However, clinical validation is lacking. This review aims to address: (1) Which SSA could perform facial scanning? (2) Which SSA can be clinically used? (3) Which SSA have been reported and scientifically validated for medical applications?

METHODS

Technical search for SSA designed for face or object scanning was conducted on Google, Apple App Store, and Google Play Store from August 2022 to December 2023. Literature search was performed on PubMed, Cochrane, EMBASE, MEDLINE, Scopus, IEEE Xplore, ACM Digital Library, Clinicaltrials.gov, ICTRP (WHO) and preprints up to 2023. Eligibility criteria included English-written scientific articles incorporating at least one SSA for clinical purposes. SSA selection and data extraction were executed by one reviewer, validated by second, with third reviewer being consulted for discordances.

RESULTS

Sixty-three applications designed for three-dimensional object scanning were retrieved, with 52 currently offering facial scanning capabilities. Fifty-six scientific articles, comprising two case reports, 16 proof-of-concepts and 38 experimental studies were analysed. Thirteen applications (123D Catch, 3D Creator, Bellus 3D Dental Pro, Bellus 3D Face app, Bellus 3D Face Maker, Capture, Heges, Metascan, Polycam, Scandy Pro, Scaniverse, Tap tap tap and Trnio) were reported in literature for digital workflow integration, comparison or proof-of-concept studies.

CONCLUSION

Fifty-two SSA can perform facial scanning currently and can be used clinically, offering cost-effectiveness, portability and user-friendliness. Although clinical validation is crucial, only 13 SSA were scientifically validated, underlying awareness of potential pitfalls and limitations.

An evaluation of consumer smartphones for generating bolus and surface mould applicators for radiation oncology

BACKGROUND

The use of Computed Tomography (CT) imaging data to create 3D printable patient-specific devices for radiation oncology purposes is already well established in the literature and has shown to have superior conformity than conventional methods. Using non-ionizing radiation imaging techniques such as photogrammetry or laser scanners in-lieu of a CT scanner presents many desirable benefits including reduced imaging dose and fabrication of the device can be completed prior to simulation. With recent advancements in smartphone-based technology, photographic and LiDAR-based technologies are more readily available than ever before and to a high level of quality. As a result, these non-ionizing radiation imaging methods are now able to generate patient-specific devices that can be acceptable for clinical use.

PURPOSE

In this work, we aim to determine if smartphones can be used by radiation oncologists or other radiation oncology staff to generate bolus or brachytherapy surface moulds instead of conventional CT with equivalent or comparable accuracy.

METHODS

This work involved two separate studies: a phantom and participant study. For the phantom study, a RANDO anthropomorphic phantom (limited to the nose region) was used to generate 3D models based on three different imaging techniques: conventional CT, photogrammetry & LiDAR which were both acquired on a smartphone. Virtual boli were designed in Blender and 3D printed from PLA plastic material. The conformity of each printed boli was assessed by measuring the air gap volume and approximate thickness between the phantom & bolus acquired together on a CT. For the participant study, photographs, and a LiDAR scan of four volunteers were captured using an iPhone 13 Pro™ to assess their feasibility for generating human models. Each virtual 3D model was visually assessed to identify any issues in their reconstruction. The LiDAR models were registered to the photogrammetry models where a distance to agreement analysis was performed to assess their level of similarity. Additionally, a 3D virtual bolus was designed and printed using ABS material from all models to assess their conformity onto the participants skin surface using a verbal feedback method.

RESULTS

The photogrammetry derived bolus showed comparable conformity to the CT derived bolus while the LiDAR derived bolus showed poorer conformity as shown by their respective air gap volume and thickness measurements. The reconstruction quality of both the photogrammetry and LiDAR models of the volunteers was inadequate in regions of facial hair and occlusion, which may lead to clinically unacceptable patient-specific device that are created from these areas. All participants found the photogrammetry 3D printed bolus to conform to their nose region with minimal room to move while three of the four participants found the LiDAR was acceptable and could be positioned comfortably over their entire nose.

CONCLUSIONS

Smartphone-based photogrammetry and LiDAR software show great potential for future use in generating 3D reference models for radiation oncology purposes. Further investigations into whether they can be used to fabricate clinically acceptable patient-specific devices on a larger and more diverse cohort of participants and anatomical locations is required for a thorough validation of their clinical usefulness.

Synthetic cranial MRI from 3D optical surface scans using deep learning for radiation therapy treatment planning

BACKGROUND

Optical scanning technologies are increasingly being utilised to supplement treatment workflows in radiation oncology, such as surface-guided radiotherapy or 3D printing custom bolus. One limitation of optical scanning devices is the absence of internal anatomical information of the patient being scanned. As a result, conventional radiation therapy treatment planning using this imaging modality is not feasible. Deep learning is useful for automating various manual tasks in radiation oncology, most notably, organ segmentation and treatment planning. Deep learning models have also been used to transform MRI datasets into synthetic CT datasets, facilitating the development of MRI-only radiation therapy planning.

AIMS

To train a pix2pix generative adversarial network to transform 3D optical scan data into estimated MRI datasets for a given patient to provide additional anatomical data for a select few radiation therapy treatment sites. The proposed network may provide useful anatomical information for treatment planning of surface mould brachytherapy, total body irradiation, and total skin electron therapy, for example, without delivering any imaging dose.

METHODS

A 2D pix2pix GAN was trained on 15,000 axial MRI slices of healthy adult brains paired with corresponding external mask slices. The model was validated on a further 5000 previously unseen external mask slices. The predictions were compared with the "ground-truth" MRI slices using the multi-scale structural similarity index (MSSI) metric. A certified neuro-radiologist was subsequently consulted to provide an independent review of the model's performance in terms of anatomical accuracy and consistency. The network was then applied to a 3D photogrammetry scan of a test subject to demonstrate the feasibility of this novel technique.

RESULTS

The trained pix2pix network predicted MRI slices with a mean MSSI of 0.831 ± 0.057 for the 5000 validation images indicating that it is possible to estimate a significant proportion of a patient's gross cranial anatomy from a patient's exterior contour. When independently reviewed by a certified neuro-radiologist, the model's performance was described as "quite amazing, but there are limitations in the regions where there is wide variation within the normal population." When the trained network was applied to a 3D model of a human subject acquired using optical photogrammetry, the network could estimate the corresponding MRI volume for that subject with good qualitative accuracy. However, a ground-truth MRI baseline was not available for quantitative comparison.

CONCLUSIONS

A deep learning model was developed, to transform 3D optical scan data of a patient into an estimated MRI volume, potentially increasing the usefulness of optical scanning in radiation therapy planning. This work has demonstrated that much of the human cranial anatomy can be predicted from the external shape of the head and may provide an additional source of valuable imaging data. Further research is required to investigate the feasibility of this approach for use in a clinical setting and further improve the model's accuracy.

Use of Facial Morphology to Determine Nutritional Status in Older Adults: Opportunities and Challenges

Undiagnosed malnutrition is a significant problem in high-income countries, which can reduce the quality of life of many individuals, particularly of older adults. Moreover, it can also inflate the costs of existing health care systems because of the many metabolic complications that it can cause. The current methods for assessing malnutrition can be cumbersome. A trained practitioner must be present to conduct an assessment, or patients must travel to facilities with specialized equipment to obtain their measurements. Therefore, digital health care is a possible way of closing this gap as it is rapidly gaining traction as a scalable means of improving efficiency in the health care system. It allows for the remote monitoring of nutritional status without requiring the physical presence of practitioners or the use of advanced medical equipment. As such, there is an increasing interest in expanding the range of digital applications to facilitate remote monitoring and management of health issues. In this study, we discuss the feasibility of a novel digital remote method for diagnosing malnutrition using facial morphometrics. Many malnutrition screening assessments include subjective assessments of the head and the face. Facial appearance is often used by clinicians as the first point of qualitative indication of health status. Hence, there may be merit in quantifying these subtle but observable changes using facial morphometrics. Modern advancements in artificial intelligence, data science, sensors, and computing technologies allow facial features to be accurately digitized, which could potentially allow these previously intuitive assessments to be quantified. This study aims to stimulate further discussion and discourse on how this emerging technology can be used to provide real-time access to nutritional status. The use of facial morphometrics extends the use of currently available technology and may provide a scalable, easily deployable solution for nutritional status to be monitored in real time. This will enable clinicians and dietitians to keep track of patients remotely and provide the necessary intervention measures as required, as well as providing health care institutions and policy makers with essential information that can be used to inform and enable targeted public health approaches within affected populations.

Reduced Calibration Strategy Using a Basketball for RGB-D Cameras

RGB-D cameras produce depth and color information commonly used in the 3D reconstruction and vision computer areas. Different cameras with the same model usually produce images with different calibration errors. The color and depth layer usually requires calibration to minimize alignment errors, adjust precision, and improve data quality in general. Standard calibration protocols for RGB-D cameras require a controlled environment to allow operators to take many RGB and depth pair images as an input for calibration frameworks making the calibration protocol challenging to implement without ideal conditions and the operator experience. In this work, we proposed a novel strategy that simplifies the calibration protocol by requiring fewer images than other methods. Our strategy uses an ordinary object, a know-size basketball, as a ground truth sphere geometry during the calibration. Our experiments show comparable results requiring fewer images and non-ideal scene conditions than a reference method to align color and depth image layers.

Evaluation of optical 3D scanning system for radiotherapy use

INTRODUCTION

Optical three-dimensional scanning devices can produce geometrically accurate, high-resolution models of patients suitable for clinical use. This article describes the use of a metrology-grade structured light scanner for the design and production of radiotherapy medical devices and synthetic water-equivalent computer tomography images.

METHODS

Following commissioning of the device by scanning objects of known properties, 173 scans were performed on 26 volunteers, with observations of subjects and operators collected.

RESULTS

The fit of devices produced using these scans was assessed, and a workflow for the design of complex devices using a treatment planning system was identified.

CONCLUSIONS

Recommendations are provided on the use of the device within a radiation oncology department.

A dosimetric comparison of CT- and photogrammetry- generated 3D printed HDR brachytherapy surface applicators

In this study, we investigate whether an acceptable dosimetric plan can be obtained for a brachytherapy surface applicator designed using photogrammetry and compare the plan quality to a CT-derived applicator. The nose region of a RANDO anthropomorphic phantom was selected as the treatment site due to its high curvature. Photographs were captured using a Nikon D5600 DSLR camera and reconstructed using Agisoft Metashape while CT data was obtained using a Canon Aquillion scanner. Virtual surface applicators were designed in Blender and printed with PLA plastic. Treatment plans with a prescription dose of 3.85 Gy × 10 fractions with 100% dose to PTV on the bridge of the nose at 2 mm depth were generated separately using AcurosBV in the Varian BrachyVision TPS. PTV D_98%, D_90% and V_100%, and OAR D_0.1cc, D_2cc and V_50% dose metrics and dwell times were evaluated, with the applicator fit assessed by air-gap volume measurements. Both types of surface applicators were printed with minimal defects and visually fitted well to the target area. The measured air-gap volume between the photogrammetry applicator and phantom surface was 44% larger than the CT-designed applicator, with a mean air gap thickness of 3.24 and 2.88 mm, respectively. The largest difference in the dose metric observed for the PTV and OAR was the PTV V_100% of - 1.27% and skin D_0.1cc of - 0.28%. PTV D_98% and D_90% and OAR D_2cc and V_50% for the photogrammetry based plan were all within 0.5% of the CT based plan. Total dwell times were also within 5%. A 3D printed surface applicator for the nose was successfully constructed using photogrammetry techniques. Although it produced a larger air gap between the surface applicator and phantom surface, a clinically acceptable dose plan was created with similar PTV and OAR dose metrics to the CT-designed applicator. Additional future work is required to comprehensively evaluate its suitability in a clinically environment.

DeepWL: Robust EPID based Winston-Lutz analysis using deep learning, synthetic image generation and optical path-tracing

Radiation therapy requires clinical linear accelerators to be mechanically and dosimetrically calibrated to a high standard. One important quality assurance test is the Winston-Lutz test which localises the radiation isocentre of the linac. In the current work we demonstrate a novel method of analysing EPID based Winston-Lutz QA images using a deep learning model trained only on synthetic image data. In addition, we propose a novel method of generating the synthetic WL images and associated 'ground-truth' masks using an optical path-tracing engine to 'fake' mega-voltage EPID images. The model called DeepWL was trained on 1500 synthetic WL images using data augmentation techniques for 180 epochs. The model was built using Keras with a TensorFlow backend on an Intel Core i5-6500T CPU and trained in approximately 15 h. DeepWL was shown to produce ball bearing and multi-leaf collimator field segmentations with a mean dice coefficient of 0.964 and 0.994 respectively on previously unseen synthetic testing data. When DeepWL was applied to WL data measured on an EPID, the predicted mean displacements were shown to be statistically similar to the Canny Edge detection method. However, the DeepWL predictions for the ball bearing locations were shown to correlate better with manual annotations compared with the Canny edge detection algorithm. DeepWL was demonstrated to analyse Winston-Lutz images with an accuracy suitable for routine linac quality assurance with some statistical evidence that it may outperform Canny Edge detection methods in terms of segmentation robustness and the resultant displacement predictions.

Evaluation of camera settings for photogrammetric reconstruction of humanoid phantoms for EBRT bolus and HDR surface brachytherapy applications

The fabrication of brachytherapy surface moulds is considered laborious and time consuming that often result in repeated attempts due to incorrect catheter positioning or the presence of air gaps. 3-dimensional printing using low-cost and reliable materials has allowed the rapid creation of patient-specific surface mould applicators to be achieved using patient imaging data obtained via CT scan. In this study we investigate whether an alternative approach using photogrammetry techniques can improve this process and how camera settings and object texture affect the reconstructions. Two humanoid phantoms, an anthropomorphic RANDO phantom and a Laerdal Little Anne CPR training manikin were used in this study. Both were imaged using a Nikon D5600 DSLR and Nokia 3.1 smartphone camera and reconstructed using Agisoft Metashape software. CT scans of both phantoms were taken as references for comparing the photogrammetry reconstructions. Models were reconstructed from different photo sets and assessed by distance to agreement with the CT models. Both phantoms were effectively reconstructed for most experiments. Increasing the number of photos used produced the better reconstructions while in general, reconstructions using video data were poor. The two phantoms were reconstructed at a similar quality. Background light that caused undesirable reflections significantly reduced reconstruction quality. Applying a non-reflective tape to the affected regions provided a suitable method for reducing their effects. Photogrammetry techniques were effectively able to reconstruct 3-dimensional models of both phantom. The camera settings and lighting did have a profound effect on the reconstruction quality and should be chosen appropriately depending on the scene.