A novel Cerenkov luminescence tomography approach using multilayer fully connected neural network

3D-deep optical learning: a multimodal and multitask reconstruction framework for optical molecular tomography

Optical molecular tomography (OMT) is an emerging imaging technique. To date, the poor universality of reconstruction algorithms based on deep learning for various imaged objects and optical probes limits the development and application of OMT. In this study, based on a new mapping representation, a multimodal and multitask reconstruction framework-3D deep optical learning (3DOL), was presented to overcome the limitations of OMT in universality by decomposing it into two tasks, optical field recovery and luminous source reconstruction. Specifically, slices of the original anatomy (provided by computed tomography) and boundary optical measurement of imaged objects serve as inputs of a recurrent convolutional neural network encoded parallel to extract multimodal features, and 2D information from a few axial planes within the samples is explicitly incorporated, which enables 3DOL to recognize different imaged objects. Subsequently, the optical field is recovered under the constraint of the object geometry, and then the luminous source is segmented by a learnable Laplace operator from the recovered optical field, which obtains stable and high-quality reconstruction results with extremely few parameters. This strategy enable 3DOL to better understand the relationship between the boundary optical measurement, optical field, and luminous source to improve 3DOL's ability to work in a wide range of spectra. The results of numerical simulations, physical phantoms, and in vivo experiments demonstrate that 3DOL is a compatible deep-learning approach to tomographic imaging diverse objects. Moreover, the fully trained 3DOL under specific wavelengths can be generalized to other spectra in the 620-900 nm NIR-I window.

K-CapsNet: K-Nearest Neighbor Based Convolution Capsule Network for Cerenkov Luminescence Tomography Reconstruction

Cerenkov luminescence tomography (CLT) has received significant attention as a promising imaging modality that can display the three-dimensional (3D) distribution of radioactive probes. However, the reconstruction of CLT suffers from severe ill-posed problem. It is difficult for traditional model-based method to obtain satisfactory result. Recently, deep learning-based method have shown great potential for accurate and efficient CLT reconstruction. In this study, a KNN-based convolution capsule network, named K-CapsNet, is proposed for cerenkov luminescence tomography. In K-CapsNet, the surface photon intensity is encoded in capsule form. The KNN-based convolution and K-means clustering are proposed for efficient encoding. Numerical simulation experiments have been carried out to verify the performance of K-CapsNet, and the results show that it performs superior in source localization and morphological restoration compared with existing methods.

FISTA-NET: Deep Algorithm Unrolling for Cerenkov luminescence tomography

Cerenkov luminescence tomography (CLT) is a highly sensitive and promising imaging technique that can be used to reconstruct the three-dimensional distribution of radioactive probes in living animals. However, the accuracy of CLT reconstruction is limited by the simplified radiative transfer equation and ill-conditioned inverse problem. To address this issue, we propose a model-based deep learning network that combines the neural network with a model-based approach to enhance the performance of CLT reconstruction. The Fast Iterative Shrinkage Thresholding Algorithm (FISTA), a traditional model-based approach, is expanded into a deep network (known as FISTA-NET). Each layer in the network represents an iteration of the algorithm steps, and connecting these layers can form a deep neural network. In addition, different from the traditional FISTA, the key parameters in FISTA, such as gradient step size and threshold value, can be learned through training data without manual production. To evaluate the performance of FISTA-NET, numerical simulation experiments were conducted, which demonstrate its excellent positioning and shape recovery abilities.Clinical Relevance-This indicates that FISTA-NET strategy can significantly improve the quality of CLT reconstruction, which is further beneficial to the assessment of disease activity and treatment effect based on CLT.

X-ray Cherenkov-luminescence tomography reconstruction with a three-component deep learning algorithm: Swin transformer, convolutional neural network, and locality module

SIGNIFICANCE

X-ray Cherenkov-luminescence tomography (XCLT) produces fast emission data from megavoltage (MV) x-ray scanning, in which the excitation location of molecules within tissue is reconstructed. However standard filtered backprojection (FBP) algorithms for XCLT sinogram reconstruction can suffer from insufficient data due to dose limitations, so there are limits in the reconstruction quality with some artifacts. We report a deep learning algorithm for XCLT with high image quality and improved quantitative accuracy.

AIM

To directly reconstruct the distribution of emission quantum yield for x-ray Cherenkov-luminescence tomography, we proposed a three-component deep learning algorithm that includes a Swin transformer, convolution neural network, and locality module model.

APPROACH

A data-to-image model x-ray Cherenkov-luminescence tomography is developed based on a Swin transformer, which is used to extract pixel-level prior information from the sinogram domain. Meanwhile, a convolutional neural network structure is deployed to transform the extracted pixel information from the sinogram domain to the image domain. Finally, a locality module is designed between the encoder and decoder connection structures for delivering features. Its performance was validated with simulation, physical phantom, and in vivo experiments.

RESULTS

This approach can better deal with the limits to data than conventional FBP methods. The method was validated with numerical and physical phantom experiments, with results showing that it improved the reconstruction performance mean square error ( > 94.1 % ), peak signal-to-noise ratio ( > 41.7 % ), and Pearson correlation ( > 19 % ) compared with the FBP algorithm. The Swin-CNN also achieved a 32.1% improvement in PSNR over the deep learning method AUTOMAP.

CONCLUSIONS

This study shows that the three-component deep learning algorithm provides an effective reconstruction method for x-ray Cherenkov-luminescence tomography.

Excitation-based fully connected network for precise NIR-II fluorescence molecular tomography

Fluorescence molecular tomography (FMT) is a novel imaging modality to obtain fluorescence biomarkers' three-dimensional (3D) distribution. However, the simplified mathematical model and complicated inverse problem limit it to achieving precise results. In this study, the second near-infrared (NIR-II) fluorescence imaging was adopted to mitigate tissue scattering and reduce noise interference. An excitation-based fully connected network was proposed to model the inverse process of NIR-II photon propagation and directly obtain the 3D distribution of the light source. An excitation block was embedded in the network allowing it to autonomously pay more attention to neurons related to the light source. The barycenter error was added to the loss function to improve the localization accuracy of the light source. Both numerical simulation and in vivo experiments showed the superiority of the novel NIR-II FMT reconstruction strategy over the baseline methods. This strategy was expected to facilitate the application of machine learning in biomedical research.

A novel weighted auxiliary set matching pursuit method for glioma in Cerenkov luminescence tomography reconstruction

Cerenkov luminescence tomography (CLT) is a promising three-dimensional imaging technology that has been actively investigated in preclinical studies. However, because of the ill-posedness in the inverse problem of CLT reconstruction, the reconstruction performance is still not satisfactory for broad biomedical applications. In this study, a novel weighted auxiliary set matching pursuit (WASMP) method was explored to enhance the accuracy of CLT reconstruction. The numerical simulations and in vivo imaging studies using tumor-bearing mice models were conducted to evaluate the performance of the WASMP method. The results of the above experiments proved that the WASMP method achieved superior reconstruction performance than other approaches in terms of positional accuracy and shape recovery. It further demonstrates that the atom selection strategy proposed in this study has a positive effect on improving the accuracy of atoms. The proposed WASMP improves the accuracy for CLT reconstruction for biomedical applications.

Elastic net-based non-negative iterative three-operator splitting strategy for Cerenkov luminescence tomography

Cerenkov luminescence tomography (CLT) provides a powerful optical molecular imaging technique for non-invasive detection and visualization of radiopharmaceuticals in living objects. However, the severe photon scattering effect causes ill-posedness of the inverse problem, and the location accuracy and shape recovery of CLT reconstruction results are unsatisfactory for clinical application. Here, to improve the reconstruction spatial location accuracy and shape recovery ability, a non-negative iterative three operator splitting (NNITOS) strategy based on elastic net (EN) regularization was proposed. NNITOS formalizes the CLT reconstruction as a non-convex optimization problem and splits it into three operators, the least square, L_1/2-norm regularization, and adaptive grouping manifold learning, then iteratively solved them. After stepwise iterations, the result of NNITOS converged progressively. Meanwhile, to speed up the convergence and ensure the sparsity of the solution, shrinking the region of interest was utilized in this strategy. To verify the effectiveness of the method, numerical simulations and in vivo experiments were performed. The result of these experiments demonstrated that, compared to several methods, NNITOS can achieve superior performance in terms of location accuracy, shape recovery capability, and robustness. We hope this work can accelerate the clinical application of CLT in the future.

Cherenkov Luminescence in Tumor Diagnosis and Treatment: A Review

Malignant tumors rank as a leading cause of death worldwide. Accurate diagnosis and advanced treatment options are crucial to win battle against tumors. In recent years, Cherenkov luminescence (CL) has shown its technical advantages and clinical transformation potential in many important fields, particularly in tumor diagnosis and treatment, such as tumor detection in vivo, surgical navigation, radiotherapy, photodynamic therapy, and the evaluation of therapeutic effect. In this review, we summarize the advances in CL for tumor diagnosis and treatment. We first describe the physical principles of CL and discuss the imaging techniques used in tumor diagnosis, including CL imaging, CL endoscope, and CL tomography. Then we present a broad overview of the current status of surgical resection, radiotherapy, photodynamic therapy, and tumor microenvironment monitoring using CL. Finally, we shed light on the challenges and possible solutions for tumor diagnosis and therapy using CL.

Intraoperative fluorescence molecular imaging accelerates the coming of precision surgery in China

Purpose

China has the largest cancer population globally. Surgery is the main choice for most solid cancer patients. Intraoperative fluorescence molecular imaging (FMI) has shown its great potential in assisting surgeons in achieving precise resection. We summarized the typical applications of intraoperative FMI and several new trends to promote the development of precision surgery.

Methods

The academic database and NIH clinical trial platform were systematically evaluated. We focused on the clinical application of intraoperative FMI in China. Special emphasis was placed on a series of typical studies with new technologies or high-level evidence. The emerging strategy of combining FMI with other modalities was also discussed.

Results

The clinical applications of clinically approved indocyanine green (ICG), methylene blue (MB), or fluorescein are on the rise in different surgical departments. Intraoperative FMI has achieved precise lesion detection, sentinel lymph node mapping, and lymphangiography for many cancers. Nerve imaging is also exploring to reduce iatrogenic injuries. Through different administration routes, these fluorescent imaging agents provided encouraging results in surgical navigation. Meanwhile, designing new cancer-specific fluorescent tracers is expected to be a promising trend to further improve the surgical outcome.

Conclusions

Intraoperative FMI is in a rapid development in China. In-depth understanding of cancer-related molecular mechanisms is necessary to achieve precision surgery. Molecular-targeted fluorescent agents and multi-modal imaging techniques might play crucial roles in the era of precision surgery.

Attention mechanism-based locally connected network for accurate and stable reconstruction in Cerenkov luminescence tomography

Cerenkov luminescence tomography (CLT) is a novel and highly sensitive imaging technique, which could obtain the three-dimensional distribution of radioactive probes to achieve accurate tumor detection. However, the simplified radiative transfer equation and ill-conditioned inverse problem cause a reconstruction error. In this study, a novel attention mechanism based locally connected (AMLC) network was proposed to reduce barycenter error and improve morphological restorability. The proposed AMLC network consisted of two main parts: a fully connected sub-network for providing a coarse reconstruction result, and a locally connected sub-network based on an attention matrix for refinement. Both numerical simulations and in vivo experiments were conducted to show the superiority of the AMLC network in accuracy and stability over existing methods (MFCNN, KNN-LC network). This method improved CLT reconstruction performance and promoted the application of machine learning in optical imaging research.

Harnessing the Power of Hybrid Light Propagation Model for Three-Dimensional Optical Imaging in Cancer Detection

Optical imaging is an emerging technology capable of qualitatively and quantitatively observing life processes at the cellular or molecular level and plays a significant role in cancer detection. In particular, to overcome the disadvantages of traditional optical imaging that only two-dimensionally and qualitatively detect biomedical information, the corresponding three-dimensional (3D) imaging technology is intensively explored to provide 3D quantitative information, such as localization and distribution and tumor cell volume. To retrieve these information, light propagation models that reflect the interaction between light and biological tissues are an important prerequisite and basis for 3D optical imaging. This review concentrates on the recent advances in hybrid light propagation models, with particular emphasis on their powerful use for 3D optical imaging in cancer detection. Finally, we prospect the wider application of the hybrid light propagation model and future potential of 3D optical imaging in cancer detection.

Alternating Direction Method of Multipliers Network for Bioluminescence Tomography Reconstruction

Bioluminescence tomography (BLT) is an effective noninvasive molecular imaging modality for three dimensional visualization of in vivo tumor research in small animals. The approaches of deep learning have shown great potential in the field of optical molecular imaging in recent years. However, the common problem with these existing end-to-end networks is the black box technology, whose solving process is not theoretically proven. In this work, we proposed a novel Alternating Direction Method of Multipliers Network (ADMM-Net) to solve the poor interpretation problem of internal process. The ADMM-Net combines the framework of deep learning on the basis of traditional ADMM algorithm to dynamically learn various parameters of the algorithm in the form of network. To evaluate the performance of our proposed network, we implemented numerical simulation experiments. The results show that the ADMM-Net can accurately reconstruct the location of the source, and the morphological similarity with the real source is also higher.

Prior Compensation Algorithm for Cerenkov Luminescence Tomography From Single-View Measurements

Cerenkov luminescence tomography (CLT) has attracted much attention because of the wide clinically-used probes and three-dimensional (3D) quantification ability. However, due to the serious morbidity of 3D optical imaging, the reconstructed images of CLT are not appreciable, especially when single-view measurements are used. Single-view CLT improves the efficiency of data acquisition. It is much consistent with the actual imaging environment of using commercial imaging system, but bringing the problem that the reconstructed results will be closer to the animal surface on the side where the single-view image is collected. To avoid this problem to the greatest extent possible, we proposed a prior compensation algorithm for CLT reconstruction based on depth calibration strategy. This method takes full account of the fact that the attenuation of light in the tissue will depend heavily on the depth of the light source as well as the distance between the light source and the detection plane. Based on this consideration, a depth calibration matrix was designed to calibrate the attenuation between the surface light flux and the density of the internal light source. The feature of the algorithm was that the depth calibration matrix directly acts on the system matrix of CLT reconstruction, rather than modifying the regularization penalty items. The validity and effectiveness of the proposed algorithm were evaluated with a numerical simulation and a mouse-based experiment, whose results illustrated that it located the radiation sources accurately by using single-view measurements.

Development of tomographic reconstruction for three-dimensional optical imaging: From the inversion of light propagation to artificial intelligence

Optical molecular tomography (OMT) is an imaging modality which uses an optical signal, especially near-infrared light, to reconstruct the three-dimensional information of the light source in biological tissue. With the advantages of being low-cost, noninvasive and having high sensitivity, OMT has been applied in preclinical and clinical research. However, due to its serious ill-posedness and ill-condition, the solution of OMT requires heavy data analysis and the reconstruction quality is limited. Recently, the artificial intelligence (commonly known as AI)-based methods have been proposed to provide a different tool to solve the OMT problem. In this paper, we review the progress on OMT algorithms, from conventional methods to AI-based methods, and we also give a prospective towards future developments in this domain.

Rising role of artificial intelligence in image reconstruction for biomedical imaging

In this editorial, we review recent progress on the applications of artificial intelligence (AI) in image reconstruction for biomedical imaging. Because it abandons prior information of traditional artificial design and adopts a completely data-driven mode to obtain deeper prior information via learning, AI technology plays an increasingly important role in biomedical image reconstruction. The combination of AI technology and the biomedical image reconstruction method has become a hotspot in the field. Favoring AI, the performance of biomedical image reconstruction has been improved in terms of accuracy, resolution, imaging speed, etc. We specifically focus on how to use AI technology to improve the performance of biomedical image reconstruction, and propose possible future directions in this field.