Fully Dense UNet for 2-D Sparse Photoacoustic Tomography Artifact Removal

Extractor-attention-predictor network for quantitative photoacoustic tomography

Quantitative photoacoustic tomography (qPAT) holds great potential in estimating chromophore concentrations, whereas the involved optical inverse problem, aiming to recover absorption coefficient distributions from photoacoustic images, remains challenging. To address this problem, we propose an extractor-attention-predictor network architecture (EAPNet), which employs a contracting-expanding structure to capture contextual information alongside a multilayer perceptron to enhance nonlinear modeling capability. A spatial attention module is introduced to facilitate the utilization of important information. We also use a balanced loss function to prevent network parameter updates from being biased towards specific regions. Our method obtains satisfactory quantitative metrics in simulated and real-world validations. Moreover, it demonstrates superior robustness to target properties and yields reliable results for targets with small size, deep location, or relatively low absorption intensity, indicating its broader applicability. The EAPNet, compared to the conventional UNet, exhibits improved efficiency, which significantly enhances performance while maintaining similar network size and computational complexity.

Fully and Weakly Supervised Deep Learning for Meniscal Injury Classification, and Location Based on MRI

Meniscal injury is a common cause of knee joint pain and a precursor to knee osteoarthritis (KOA). The purpose of this study is to develop an automatic pipeline for meniscal injury classification and localization using fully and weakly supervised networks based on MRI images. In this retrospective study, data were from the osteoarthritis initiative (OAI). The MR images were reconstructed using a sagittal intermediate-weighted fat-suppressed turbo spin-echo sequence. (1) We used 130 knees from the OAI to develop the LGSA-UNet model which fuses the features of adjacent slices and adjusts the blocks in Siam to enable the central slice to obtain rich contextual information. (2) One thousand seven hundred and fifty-six knees from the OAI were included to establish segmentation and classification models. The segmentation model achieved a DICE coefficient ranging from 0.84 to 0.93. The AUC values ranged from 0.85 to 0.95 in the binary models. The accuracy for the three types of menisci (normal, tear, and maceration) ranged from 0.60 to 0.88. Furthermore, 206 knees from the orthopedic hospital were used as an external validation data set to evaluate the performance of the model. The segmentation and classification models still performed well on the external validation set. To compare the diagnostic performances between the deep learning (DL) models and radiologists, the external validation sets were sent to two radiologists. The binary classification model outperformed the diagnostic performance of the junior radiologist (0.82-0.87 versus 0.74-0.88). This study highlights the potential of DL in knee meniscus segmentation and injury classification which can help improve diagnostic efficiency.

Removing Artifacts in Transcranial Photoacoustic Imaging With Polarized Self-Attention Dense-UNet

OBJECTIVE

Photoacoustic imaging (PAI) is a promising transcranial imaging technique. However, the distortion of photoacoustic signals induced by the skull significantly influences its imaging quality. We aimed to use deep learning for removing artifacts in PAI.

METHODS

In this study, we propose a polarized self-attention dense U-Net, termed PSAD-UNet, to correct the distortion and accurately recover imaged objects beneath bone plates. To evaluate the performance of the proposed method, a series of experiments was performed using a custom-built PAI system.

RESULTS

The experimental results showed that the proposed PSAD-UNet method could effectively implement transcranial PAI through a one- or two-layer bone plate. Compared with the conventional delay-and-sum and classical U-Net methods, PSAD-UNet can diminish the influence of bone plates and provide high-quality PAI results in terms of structural similarity and peak signal-to-noise ratio. The 3-D experimental results further confirm the feasibility of PSAD-UNet in 3-D transcranial imaging.

CONCLUSION

PSAD-UNet paves the way for implementing transcranial PAI with high imaging accuracy, which reveals broad application prospects in preclinical and clinical fields.

Image segmentation of mouse eye in vivo with optical coherence tomography based on Bayesian classification

OBJECTIVES

Optical coherence tomography (OCT) is a new imaging technology that uses an optical analog of ultrasound imaging for biological tissues. Image segmentation plays an important role in dealing with quantitative analysis of medical images.

METHODS

We have proposed a novel framework to deal with the low intensity problem, based on the labeled patches and Bayesian classification (LPBC) model. The proposed method includes training and testing phases. During the training phase, firstly, we manually select the sub-images of background and Region of Interest (ROI) from the training image, and then extract features by patches. Finally, we train the Bayesian model with the features. The segmentation threshold of each patch is computed by the learned Bayesian model.

RESULTS

In addition, we have collected a new dataset of mouse eyes in vivo with OCT, named MEVOCT, which can be found at URL https://17861318579.github.io/LPBC. MEVOCT consists of 20 high-resolution images. The resolution of every image is 2048 × 2048 pixels.

CONCLUSIONS

The experimental results demonstrate the effectiveness of the LPBC method on the new MEVOCT dataset. The ROI segmentation is of great importance for the distortion correction.

Gene-Based Predictive Modelling for Enhanced Detection of Systemic Lupus Erythematosus Using CNN-Based DL Algorithm

Systemic Lupus Erythematosus (SLE) is a multifaceted autoimmune disease that presents with a diverse array of clinical signs and unpredictable disease progression. Conventional diagnostic methods frequently fall short in terms of sensitivity and specificity, which can result in delayed diagnosis and less-than-optimal management. In this study, we introduce a novel approach for improving the identification of SLE through the use of gene-based predictive modelling and Stacked deep learning classifiers. The study proposes a new method for diagnosing SLE using Stacked Deep Learning Classifiers (SDLC) trained on Gene Expression Omnibus (GEO) database data. By combining transcriptomic data from GEO with clinical features and laboratory results, the SDLC model achieves a remarkable accuracy value of 0.996, outperforming traditional methods. Individual models within the SDLC, such as SBi-LSTM and ACNN, achieved accuracies of 92% and 95%, respectively. The SDLC's ensemble learning approach allows for identifying complex patterns in multi-modal data, enhancing accuracy in diagnosing SLE. This study emphasises the potential of deep learning methods, in conjunction with open repositories like GEO, to advance the diagnosis and management of SLE. Overall, this research shows strong performance and potential for improving precision medicine in managing SLE.

I2U-Net: A dual-path U-Net with rich information interaction for medical image segmentation

Although the U-shape networks have achieved remarkable performances in many medical image segmentation tasks, they rarely model the sequential relationship of hierarchical layers. This weakness makes it difficult for the current layer to effectively utilize the historical information of the previous layer, leading to unsatisfactory segmentation results for lesions with blurred boundaries and irregular shapes. To solve this problem, we propose a novel dual-path U-Net, dubbed I2U-Net. The newly proposed network encourages historical information re-usage and re-exploration through rich information interaction among the dual paths, allowing deep layers to learn more comprehensive features that contain both low-level detail description and high-level semantic abstraction. Specifically, we introduce a multi-functional information interaction module (MFII), which can model cross-path, cross-layer, and cross-path-and-layer information interactions via a unified design, making the proposed I2U-Net behave similarly to an unfolded RNN and enjoying its advantage of modeling time sequence information. Besides, to further selectively and sensitively integrate the information extracted by the encoder of the dual paths, we propose a holistic information fusion and augmentation module (HIFA), which can efficiently bridge the encoder and the decoder. Extensive experiments on four challenging tasks, including skin lesion, polyp, brain tumor, and abdominal multi-organ segmentation, consistently show that the proposed I2U-Net has superior performance and generalization ability over other state-of-the-art methods. The code is available at https://github.com/duweidai/I2U-Net.

U-Net enhanced real-time LED-based photoacoustic imaging

Photoacoustic (PA) imaging is hybrid imaging modality with good optical contrast and spatial resolution. Portable, cost-effective, smaller footprint light emitting diodes (LEDs) are rapidly becoming important PA optical sources. However, the key challenge faced by the LED-based systems is the low light fluence that is generally compensated by high frame averaging, consequently reducing acquisition frame-rate. In this study, we present a simple deep learning U-Net framework that enhances the signal-to-noise ratio (SNR) and contrast of PA image obtained by averaging low number of frames. The SNR increased by approximately four-fold for both in-class in vitro phantoms (4.39 ± 2.55) and out-of-class in vivo models (4.27 ± 0.87). We also demonstrate the noise invariancy of the network and discuss the downsides (blurry outcome and failure to reduce the salt & pepper noise). Overall, the developed U-Net framework can provide a real-time image enhancement platform for clinically translatable low-cost and low-energy light source-based PA imaging systems.

Compensating unknown speed of sound in learned fast 3D limited-view photoacoustic tomography

Real-time applications in three-dimensional photoacoustic tomography from planar sensors rely on fast reconstruction algorithms that assume the speed of sound (SoS) in the tissue is homogeneous. Moreover, the reconstruction quality depends on the correct choice for the constant SoS. In this study, we discuss the possibility of ameliorating the problem of unknown or heterogeneous SoS distributions by using learned reconstruction methods. This can be done by modelling the uncertainties in the training data. In addition, a correction term can be included in the learned reconstruction method. We investigate the influence of both and while a learned correction component can improve reconstruction quality further, we show that a careful choice of uncertainties in the training data is the primary factor to overcome unknown SoS. We support our findings with simulated and in vivo measurements in 3D.

FDB-Net: Fusion double branch network combining CNN and transformer for medical image segmentation

BACKGROUND

The rapid development of deep learning techniques has greatly improved the performance of medical image segmentation, and medical image segmentation networks based on convolutional neural networks and Transformer have been widely used in this field. However, due to the limitation of the restricted receptive field of convolutional operation and the lack of local fine information extraction ability of the self-attention mechanism in Transformer, the current neural networks with pure convolutional or Transformer structure as the backbone still perform poorly in medical image segmentation.

METHODS

In this paper, we propose FDB-Net (Fusion Double Branch Network, FDB-Net), a double branch medical image segmentation network combining CNN and Transformer, by using a CNN containing gnConv blocks and a Transformer containing Varied-Size Window Attention (VWA) blocks as the feature extraction backbone network, the dual-path encoder ensures that the network has a global receptive field as well as access to the target local detail features. We also propose a new feature fusion module (Deep Feature Fusion, DFF), which helps the image to simultaneously fuse features from two different structural encoders during the encoding process, ensuring the effective fusion of global and local information of the image.

CONCLUSION

Our model achieves advanced results in all three typical tasks of medical image segmentation, which fully validates the effectiveness of FDB-Net.

WITHDRAWN: Coronary artery segmentation based on ACMA-Net and unscented Kalman filter algorithm

This article has been withdrawn at the request of the author(s) and/or editor. The Publisher apologizes for any inconvenience this may cause. The full Elsevier Policy on Article Withdrawal can be found at https://www.elsevier.com/about/policies/article-withdrawal.

Fully dense generative adversarial network for removing artifacts caused by microwave dielectric effect in thermoacoustic imaging

Microwave-induced thermoacoustic (TA) imaging (MTAI) combines pulsed microwave excitation and ultrasound detection to provide high contrast and spatial resolution images through dielectric contrast, which holds great promise for clinical applications. However, artifacts caused by microwave dielectric effect will seriously affect the accuracy of MTAI images that will hinder the clinical translation of MTAI. In this work, we propose a deep learning-based method fully dense generative adversarial network (FD-GAN) for removing artifacts caused by microwave dielectric effect in MTAI. FD-GAN adds the fully dense block to the generative adversarial network (GAN) based on the mutual confrontation between generator and discriminator, which enables it to learn both local and global features related to the removal of artifacts and generate high-quality images. The practical feasibility was tested in simulated, experimental data. The results demonstrate that FD-GAN can effectively remove the artifacts caused by the microwave dielectric effect, and shows superiority in denoising, background suppression, and improvement of image distortion. Our approach is expected to significantly improve the accuracy and quality of MTAI images, thereby enhancing the diagnostic accuracy of this innovative imaging technique.

Enhancing Diagnostic Images to Improve the Performance of the Segment Anything Model in Medical Image Segmentation

Medical imaging serves as a crucial tool in current cancer diagnosis. However, the quality of medical images is often compromised to minimize the potential risks associated with patient image acquisition. Computer-aided diagnosis systems have made significant advancements in recent years. These systems utilize computer algorithms to identify abnormal features in medical images, assisting radiologists in improving diagnostic accuracy and achieving consistency in image and disease interpretation. Importantly, the quality of medical images, as the target data, determines the achievable level of performance by artificial intelligence algorithms. However, the pixel value range of medical images differs from that of the digital images typically processed via artificial intelligence algorithms, and blindly incorporating such data for training can result in suboptimal algorithm performance. In this study, we propose a medical image-enhancement scheme that integrates generic digital image processing and medical image processing modules. This scheme aims to enhance medical image data by endowing them with high-contrast and smooth characteristics. We conducted experimental testing to demonstrate the effectiveness of this scheme in improving the performance of a medical image segmentation algorithm.

Light-M: An efficient lightweight medical image segmentation framework for resource-constrained IoMT

The Internet of Medical Things (IoMT) is being incorporated into current healthcare systems. This technology intends to connect patients, IoMT devices, and hospitals over mobile networks, allowing for more secure, quick, and convenient health monitoring and intelligent healthcare services. However, existing intelligent healthcare applications typically rely on large-scale AI models, and standard IoMT devices have significant resource constraints. To alleviate this paradox, in this paper, we propose a Knowledge Distillation (KD)-based IoMT end-edge-cloud orchestrated architecture for medical image segmentation tasks, called Light-M, aiming to deploy a lightweight medical model in resource-constrained IoMT devices. Specifically, Light-M trains a large teacher model in the cloud server and employs computation in local nodes through imitation of the performance of the teacher model using knowledge distillation. Light-M contains two KD strategies: (1) active exploration and passive transfer (AEPT) and (2) self-attention-based inter-class feature variation (AIFV) distillation for the medical image segmentation task. The AEPT encourages the student model to learn undiscovered knowledge/features of the teacher model without additional feature layers, aiming to explore new features and outperform the teacher. To improve the distinguishability of the student for different classes, the student learns the self-attention-based feature variation (AIFV) between classes. Since the proposed AEPT and AIFV only appear in the training process, our framework does not involve any additional computation burden for a student model during the segmentation task deployment. Extensive experiments on cardiac images and public real-scene datasets demonstrate that our approach improves student model learning representations and outperforms state-of-the-art methods by combining two knowledge distillation strategies. Moreover, when deployed on the IoT device, the distilled student model takes only 29.6 ms for one sample at the inference step.

Photoacoustic imaging of squirrel monkey cortical responses induced by peripheral mechanical stimulation

Non-human primates (NHPs) are crucial models for studies of neuronal activity. Emerging photoacoustic imaging modalities offer excellent tools for studying NHP brains with high sensitivity and high spatial resolution. In this research, a photoacoustic microscopy (PAM) device was used to provide a label-free quantitative characterization of cerebral hemodynamic changes due to peripheral mechanical stimulation. A 5 × 5 mm area within the somatosensory cortex region of an adult squirrel monkey was imaged. A deep, fully connected neural network was characterized and applied to the PAM images of the cortex to enhance the vessel structures after mechanical stimulation on the forelimb digits. The quality of the PAM images was improved significantly with a neural network while preserving the hemodynamic responses. The functional responses to the mechanical stimulation were characterized based on the improved PAM images. This study demonstrates capability of PAM combined with machine learning for functional imaging of the NHP brain.

Dual output feature fusion networks for femoral segmentation and quantitative analysis of the knee joint

BACKGROUND

Magnetic resonance imaging (MRI) is the preferred imaging modality for diagnosing knee disease. Segmentation of the knee MRI images is essential for subsequent quantification of clinical parameters and treatment planning for knee prosthesis replacement. However, the segmentation remains difficult due to individual differences in anatomy, the difficulty of obtaining accurate edges at lower resolutions, and the presence of speckle noise and artifacts in the images. In addition, radiologists must manually measure the knee's parameters which is a laborious and time-consuming process.

PURPOSE

Automatic quantification of femoral morphological parameters can be of fundamental help in the design of prosthetic implants for the repair of the knee and the femur. Knowledge of knee femoral parameters can provide a basis for femoral repair of the knee, the design of fixation materials for femoral prostheses, and the replacement of prostheses.

METHODS

This paper proposes a new deep network architecture to comprehensively address these challenges. A dual output model structure is proposed, with a high and low layer fusion extraction feature module designed to extract rich features through the cross-fusion mechanism. A multi-scale edge information extraction spatial feature module is also developed to address the boundary-blurring problem.

RESULTS

Based on the precise automated segmentation results, 10 key clinical parameters were automatically measured for a knee femoral prosthesis replacement program. The correlation coefficients of the quantitative results of these parameters compared to manual results all achieved at least 0.92. The proposed method was extensively evaluated with MRIs of 78 patients' knees, and it consistently outperformed other methods used for segmentation.

CONCLUSIONS

The automated quantization process produced comparable measurements to those manually obtained by radiologists. This paper demonstrates the viability of automatic knee MRI image segmentation and quantitative analysis with the proposed method. This provides data to support the accuracy of assessing the progression and biomechanical changes of osteoarthritis of the knee using an automated process, thus saving valuable time for the radiologists and surgeons.

Machine and deep learning models for accurate detection of ischemia and scar with myocardial blood flow positron emission tomography imaging

BACKGROUND

Quantification of myocardial blood flow (MBF) is used for the noninvasive diagnosis of patients with coronary artery disease (CAD). This study compared traditional statistics, machine learning, and deep learning techniques in their ability to diagnose disease using only the rest and stress MBF values.

METHODS

This study included 3245 rest and stress rubidium-82 positron emission tomography (PET) studies and matching diagnostic labels from perfusion reports. Standard logistic regression, lasso logistic regression, support vector machine, random forest, multilayer perceptron, and dense U-Net were compared for per-patient detection and per-vessel localization of scars and ischemia.

RESULTS

Receiver-operator characteristic area under the curve (AUC) of machine learning models was significantly higher than those of traditional statistics models for per-patient detection of disease (0.92-0.95 vs. 0.87) but not for per-vessel localization of ischemia or scar. Random forest showed the highest AUC = 0.95 among the different models compared. On the final hold-out set for generalizability, random forest showed an AUC of 0.92 for detection and 0.89 for localization of perfusion abnormalities.

CONCLUSIONS

For per-vessel localization, simple models trained on segmental data performed similarly to a convolutional neural network trained on polar-map data, highlighting the need to justify the use of complex predictive algorithms through comparison with simpler methods.

Joint learning of sparse and limited-view guided waves signals for feature reconstruction and imaging

Sparse and limited-view ultrasonic guided wave imaging has become a research hotspot in the field. Studies have shown that traditional under-sampling ultrasonic imaging methods either require a significant amount of time to recover the full data or produce poor quality imaging results. To address these issues, this paper proposes an end-to-end ultrasonic guided wave joint learning imaging method for sparse and limited-view transducer arrays, which integrates sparse feature reconstruction and deep learning imaging methods. Numerical and experimental studies demonstrate that this approach significantly improves the quality of imaging results. The quality of imaging results for sparse and limited-view transducer arrays is evaluated and quantified using average correlation coefficients on the testing set. The feasibility and effectiveness of the proposed method have been verified.

BAG-Net: a boundary detection and multiple attention-guided network for liver ultrasound image automatic segmentation in ultrasound guided surgery

Objective.Automated segmentation of targets in ultrasound (US) images during US-guided liver surgery holds the potential to assist physicians in fast locating critical areas such as blood vessels and lesions. However, this remains a challenging task primarily due to the image quality issues associated with US, including blurred edges and low contrast. In addition, studies specifically targeting liver segmentation are relatively scarce possibly since studying deep abdominal organs under US is difficult. In this paper, we proposed a network named BAG-Net to address these challenges and achieve accurate segmentation of liver targets with varying morphologies, including lesions and blood vessels.Approach.The BAG-Net was designed with a boundary detection module together with a position module to locate the target, and multiple attention-guided modules combined with the depth supervision strategy to enhance detailed segmentation of the target area.Main Results.Our method was compared to other approaches and demonstrated superior performance on two liver US datasets. Specifically, the method achieved 93.9% precision, 91.2% recall, 92.4% Dice coefficient, and 86.2% IoU to segment the liver tumor. Additionally, we evaluated the capability of our network to segment tumors on the breast US dataset (BUSI), where it also achieved excellent results.Significance.Our proposed method was validated to effectively segment liver targets with diverse morphologies, providing suspicious areas for clinicians to identify lesions or other characteristics. In the clinic, the method is anticipated to improve surgical efficiency during US-guided surgery.

High-resolution iterative reconstruction at extremely low sampling rate for Fourier single-pixel imaging via diffusion model

The trade-off between imaging efficiency and imaging quality has always been encountered by Fourier single-pixel imaging (FSPI). To achieve high-resolution imaging, the increase in the number of measurements is necessitated, resulting in a reduction of imaging efficiency. Here, a novel high-quality reconstruction method for FSPI imaging via diffusion model was proposed. A score-based diffusion model is designed to learn prior information of the data distribution. The real-sampled low-frequency Fourier spectrum of the target is employed as a consistency term to iteratively constrain the model in conjunction with the learned prior information, achieving high-resolution reconstruction at extremely low sampling rates. The performance of the proposed method is evaluated by simulations and experiments. The results show that the proposed method has achieved superior quality compared with the traditional FSPI method and the U-Net method. Especially at the extremely low sampling rate (e.g., 1%), an approximately 241% improvement in edge intensity-based score was achieved by the proposed method for the coin experiment, compared with the traditional FSPI method. The method has the potential to achieve high-resolution imaging without compromising imaging speed, which will further expanding the application scope of FSPI in practical scenarios.

Conditional generative learning for medical image imputation

Image imputation refers to the task of generating a type of medical image given images of another type. This task becomes challenging when the difference between the available images, and the image to be imputed is large. In this manuscript, one such application is considered. It is derived from the dynamic contrast enhanced computed tomography (CECT) imaging of the kidneys: given an incomplete sequence of three CECT images, we are required to impute the missing image. This task is posed as one of probabilistic inference and a generative algorithm to generate samples of the imputed image, conditioned on the available images, is developed, trained, and tested. The output of this algorithm is the "best guess" of the imputed image, and a pixel-wise image of variance in the imputation. It is demonstrated that this best guess is more accurate than those generated by other, deterministic deep-learning based algorithms, including ones which utilize additional information and more complex loss terms. It is also shown that the pixel-wise variance image, which quantifies the confidence in the reconstruction, can be used to determine whether the result of the imputation meets a specified accuracy threshold and is therefore appropriate for a downstream task.

A dense and U-shaped transformer with dual-domain multi-loss function for sparse-view CT reconstruction

OBJECTIVE

CT image reconstruction from sparse-view projections is an important imaging configuration for low-dose CT, as it can reduce radiation dose. However, the CT images reconstructed from sparse-view projections by traditional analytic algorithms suffer from severe sparse artifacts. Therefore, it is of great value to develop advanced methods to suppress these artifacts. In this work, we aim to use a deep learning (DL)-based method to suppress sparse artifacts.

METHODS

Inspired by the good performance of DenseNet and Transformer architecture in computer vision tasks, we propose a Dense U-shaped Transformer (D-U-Transformer) to suppress sparse artifacts. This architecture exploits the advantages of densely connected convolutions in capturing local context and Transformer in modelling long-range dependencies, and applies channel attention to fusion features. Moreover, we design a dual-domain multi-loss function with learned weights for the optimization of the model to further improve image quality.

RESULTS

Experimental results of our proposed D-U-Transformer yield performance improvements on the well-known Mayo Clinic LDCT dataset over several representative DL-based models in terms of artifact suppression and image feature preservation. Extensive internal ablation experiments demonstrate the effectiveness of the components in the proposed model for sparse-view computed tomography (SVCT) reconstruction.

SIGNIFICANCE

The proposed method can effectively suppress sparse artifacts and achieve high-precision SVCT reconstruction, thus promoting clinical CT scanning towards low-dose radiation and high-quality imaging. The findings of this work can be applied to denoising and artifact removal tasks in CT and other medical images.

Unveiling precision: a data-driven approach to enhance photoacoustic imaging with sparse data

This study presents the Fourier Decay Perception Generative Adversarial Network (FDP-GAN), an innovative approach dedicated to alleviating limitations in photoacoustic imaging stemming from restricted sensor availability and biological tissue heterogeneity. By integrating diverse photoacoustic data, FDP-GAN notably enhances image fidelity and reduces artifacts, particularly in scenarios of low sampling. Its demonstrated effectiveness highlights its potential for substantial contributions to clinical applications, marking a significant stride in addressing pertinent challenges within the realm of photoacoustic acquisition techniques.

Accelerated model-based iterative reconstruction strategy for sparse-view photoacoustic tomography aided by multi-channel autoencoder priors

Photoacoustic tomography (PAT) commonly works in sparse view due to data acquisition limitations. However, reconstruction suffers from serious deterioration (e.g., severe artifacts) using traditional algorithms under sparse view. Here, a novel accelerated model-based iterative reconstruction strategy for sparse-view PAT aided by multi-channel autoencoder priors was proposed. A multi-channel denoising autoencoder network was designed to learn prior information, which provides constraints for model-based iterative reconstruction. This integration accelerates the iteration process, leading to optimal reconstruction outcomes. The performance of the proposed method was evaluated using blood vessel simulation data and experimental data. The results show that the proposed method can achieve superior sparse-view reconstruction with a significant acceleration of iteration. Notably, the proposed method exhibits excellent performance under extremely sparse condition (e.g., 32 projections) compared with the U-Net method, with an improvement of 48% in PSNR and 12% in SSIM for in vivo experimental data.

Rib segmentation in chest x-ray images based on unsupervised domain adaptation

Rib segmentation in 2D chest x-ray images is a crucial and challenging task. On one hand, chest x-ray images serve as the most prevalent form of medical imaging due to their convenience, affordability, and minimal radiation exposure. However, on the other hand, these images present intricate challenges including overlapping anatomical structures, substantial noise and artifacts, inherent anatomical complexity. Currently, most methods employ deep convolutional networks for rib segmentation, necessitating an extensive quantity of accurately labeled data for effective training. Nonetheless, achieving precise pixel-level labeling in chest x-ray images presents a notable difficulty. Additionally, many methods neglect the challenge of predicting fractured results and subsequent post-processing difficulties. In contrast, CT images benefit from being able to directly label as the 3D structure and patterns of organs or tissues. In this paper, we redesign rib segmentation task for chest x-ray images and propose a concise and efficient cross-modal method based on unsupervised domain adaptation with centerline loss function to prevent result discontinuity and address rigorous post-processing. We utilize digital reconstruction radiography images and the labels generated from 3D CT images to guide rib segmentation on unlabeled 2D chest x-ray images. Remarkably, our model achieved a higher dice score on the test samples and the results are highly interpretable, without requiring any annotated rib markings on chest x-ray images. Our code and demo will be released in 'https://github.com/jialin-zhao/RibsegBasedonUDA'.

SCA-Former: transformer-like network based on stream-cross attention for medical image segmentation

Objective. Deep convolutional neural networks (CNNs) have been widely applied in medical image analysis and achieved satisfactory performances. While most CNN-based methods exhibit strong feature representation capabilities, they face challenges in encoding long-range interaction information due to the limited receptive fields. Recently, the Transformer has been proposed to alleviate this issue, but its cost is greatly enlarging the model size, which may inhibit its promotion.Approach. To take strong long-range interaction modeling ability and small model size into account simultaneously, we propose a Transformer-like block-based U-shaped network for medical image segmentation, dubbed as SCA-Former. Furthermore, we propose a novel stream-cross attention (SCA) module to enforce the network to focus on finding a balance between local and global representations by extracting multi-scale and interactive features along spatial and channel dimensions. And SCA can effectively extract channel, multi-scale spatial, and long-range information for a more comprehensive feature representation.Main results. Experimental results demonstrate that SCA-Former outperforms the current state-of-the-art (SOTA) methods on three public datasets, including GLAS, ISIC 2017 and LUNG.Significance. This work exhibits a promising method to enhance the feature representation of convolutional neural networks and improve segmentation performance.

U structured network with three encoding paths for breast tumor segmentation

Breast ultrasound segmentation remains challenging because of the blurred boundaries, irregular shapes, and the presence of shadowing and speckle noise. The majority of approaches stack convolutional layers to extract advanced semantic information, which makes it difficult to handle multiscale issues. To address those issues, we propose a three-path U-structure network (TPUNet) that consists of a three-path encoder and an attention-based feature fusion block (AFF Block). Specifically, instead of simply stacking convolutional layers, we design a three-path encoder to capture multiscale features through three independent encoding paths. Additionally, we design an attention-based feature fusion block to weight and fuse feature maps in spatial and channel dimensions. The AFF Block encourages different paths to compete with each other in order to synthesize more salient feature maps. We also investigate a hybrid loss function for reducing false negative regions and refining the boundary segmentation, as well as the deep supervision to guide different paths to capture the effective features under the corresponding receptive field sizes. According to experimental findings, our proposed TPUNet achieves more excellent results in terms of quantitative analysis and visual quality than other rival approaches.

Adaptive machine learning method for photoacoustic computed tomography based on sparse array sensor data

BACKGROUND AND OBJECTIVE

Photoacoustic computed tomography (PACT) is a non-invasive biomedical imaging technology that has developed rapidly in recent decades, especially has shown potential for small animal studies and early diagnosis of human diseases. To obtain high-quality images, the photoacoustic imaging system needs a high-element-density detector array. However, in practical applications, due to the cost limitation, manufacturing technology, and the system requirement in miniaturization and robustness, it is challenging to achieve sufficient elements and high-quality reconstructed images, which may even suffer from artifacts. Different from the latest machine learning methods based on removing distortions and artifacts to recover high-quality images, this paper proposes an adaptive machine learning method to firstly predict and complement the photoacoustic sensor channel data from sparse array sampling and then reconstruct images through conventional reconstruction algorithms.

METHODS

We develop an adaptive machine learning method to predict and complement the photoacoustic sensor channel data. The model consists of XGBoost and a neural network named SS-net. To handle data sets of different sizes and improve the generalization, a tunable parameter is used to control the weights of XGBoost and SS-net outputs.

RESULTS

The proposed method achieved superior performance as demonstrated by simulation, phantom experiments, and in vivo experiment results. Compared with linear interpolation, XGBoost, CAE, and U-net, the simulation results show that the SSIM value is increased by 12.83%, 6.78%, 21.46%, and 12.33%. Moreover, the median R2 is increased by 34.4%, 8.1%, 28.6%, and 84.1% with the in vivo data.

CONCLUSIONS

This model provides a framework to predict the missed photoacoustic sensor data on a sparse ring-shaped array for PACT imaging and has achieved considerable improvements in reconstructing the objects. Compared with linear interpolation and other deep learning methods qualitatively and quantitatively, our proposed methods can effectively suppress artifacts and improve image quality. The advantage of our methods is that there is no need for preparing a large number of images as the training dataset, and the data for training is directly from the sensors. It has the potential to be applied to a wide range of photoacoustic imaging detector arrays for low-cost and user-friendly clinical applications.

Deep learning in mesoscale brain image analysis: A review

Mesoscale microscopy images of the brain contain a wealth of information which can help us understand the working mechanisms of the brain. However, it is a challenging task to process and analyze these data because of the large size of the images, their high noise levels, the complex morphology of the brain from the cellular to the regional and anatomical levels, the inhomogeneous distribution of fluorescent labels in the cells and tissues, and imaging artifacts. Due to their impressive ability to extract relevant information from images, deep learning algorithms are widely applied to microscopy images of the brain to address these challenges and they perform superiorly in a wide range of microscopy image processing and analysis tasks. This article reviews the applications of deep learning algorithms in brain mesoscale microscopy image processing and analysis, including image synthesis, image segmentation, object detection, and neuron reconstruction and analysis. We also discuss the difficulties of each task and possible directions for further research.

4D spectral-spatial computational photoacoustic dermoscopy

Photoacoustic dermoscopy (PAD) is an emerging non-invasive imaging technology aids in the diagnosis of dermatological conditions by obtaining optical absorption information of skin tissues. Despite advances in PAD, it remains unclear how to obtain quantitative accuracy of the reconstructed PAD images according to the optical and acoustic properties of multilayered skin, the wavelength and distribution of excitation light, and the detection performance of ultrasound transducers. In this work, a computing method of four-dimensional (4D) spectral-spatial imaging for PAD is developed to enable quantitative analysis and optimization of structural and functional imaging of skin. This method takes the optical and acoustic properties of heterogeneous skin tissues into account, which can be used to correct the optical field of excitation light, detectable ultrasonic field, and provide accurate single-spectrum analysis or multi-spectral imaging solutions of PAD for multilayered skin tissues. A series of experiments were performed, and simulation datasets obtained from the computational model were used to train neural networks to further improve the imaging quality of the PAD system. All the results demonstrated the method could contribute to the development and optimization of clinical PADs by datasets with multiple variable parameters, and provide clinical predictability of photoacoustic (PA) data for human skin.

Learning to deep learning: statistics and a paradigm test in selecting a UNet architecture to enhance MRI

OBJECTIVE

This study aims to assess the statistical significance of training parameters in 240 dense UNets (DUNets) used for enhancing low Signal-to-Noise Ratio (SNR) and undersampled MRI in various acquisition protocols. The objective is to determine the validity of differences between different DUNet configurations and their impact on image quality metrics.

MATERIALS AND METHODS

To achieve this, we trained all DUNets using the same learning rate and number of epochs, with variations in 5 acquisition protocols, 24 loss function weightings, and 2 ground truths. We calculated evaluation metrics for two metric regions of interest (ROI). We employed both Analysis of Variance (ANOVA) and Mixed Effects Model (MEM) to assess the statistical significance of the independent parameters, aiming to compare their efficacy in revealing differences and interactions among fixed parameters.

RESULTS

ANOVA analysis showed that, except for the acquisition protocol, fixed variables were statistically insignificant. In contrast, MEM analysis revealed that all fixed parameters and their interactions held statistical significance. This emphasizes the need for advanced statistical analysis in comparative studies, where MEM can uncover finer distinctions often overlooked by ANOVA.

DISCUSSION

These findings highlight the importance of utilizing appropriate statistical analysis when comparing different deep learning models. Additionally, the surprising effectiveness of the UNet architecture in enhancing various acquisition protocols underscores the potential for developing improved methods for characterizing and training deep learning models. This study serves as a stepping stone toward enhancing the transparency and comparability of deep learning techniques for medical imaging applications.

TBUnet: A Pure Convolutional U-Net Capable of Multifaceted Feature Extraction for Medical Image Segmentation

Many current medical image segmentation methods utilize convolutional neural networks (CNNs), with some extended U-Net-based networks relying on deep feature representations to achieve satisfactory results. However, due to the limited receptive fields of convolutional architectures, they are unable to explicitly model the varying range dependencies present in medical images. Recently, advancements in large kernel convolution have allowed for the extraction of a wider range of low frequency information, making this task more achievable. In this paper, we propose TBUnet for solving the problem of difficult to accurately segment lesions with heterogeneous structures and fuzzy borders, such as melanoma, colon polyps and breast cancer. The TBUnet is a pure convolutional network with three branches for extracting high frequency information, low frequency information, and boundary information, respectively. It is capable of extracting features in various areas. To fuse the feature maps from the three branches, TBUnet presents the FL (fusion layer) module, which is based on threshold and logical operation. We design the FE (feature enhancement) module on the skip-connection to emphasize the fine-grained features. In addition, our method varies the number of input channels in different branches at each stage of the network, so that the relationship between low and high frequency features can be learned. TBUnet yields 91.08 DSC on ISIC-2018 for melanoma segmentation, and achieves better performance than state-of-the-art medical image segmentation methods. Furthermore, experimental results with 82.48 DSC and 89.04 DSC obtained on the BUSI dataset and the Kvasir-SEG dataset show that TBUnet outperforms the advanced segmentation methods. Experiments demonstrate that TBUnet has excellent segmentation performance and generalisation capability.

Multikernel positional embedding convolutional neural network for photoacoustic reconstruction with sparse data

Photoacoustic imaging (PAI) is an emerging noninvasive imaging modality that merges the high contrast of optical imaging with the high resolution of ultrasonic imaging. Low-quality photoacoustic reconstruction with sparse data due to sparse spatial sampling and limited view detection is a major obstacle to the popularization of PAI for medical applications. Deep learning has been considered as the best solution to this problem in the past decade. In this paper, we propose what we believe to be a novel architecture, named DPM-UNet, which consists of the U-Net backbone with additional position embedding block and two multi-kernel-size convolution blocks, a dilated dense block and dilated multi-kernel-size convolution block. Our method was experimentally validated with both simulated data and in vivo data, achieving a SSIM of 0.9824 and a PSNR of 33.2744 dB. Furthermore, the reconstructed images of our proposed method were compared with those obtained by other advanced methods. The results have shown that our proposed DPM-UNet has a great advantage in PAI over other methods with respect to the imaging effect and memory consumption.

Deep learning on photoacoustic tomography to remove image distortion due to inaccurate measurement of the scanning radius

Photoacoustic tomography (PAT) is a non-invasive, non-ionizing hybrid imaging modality that holds great potential for various biomedical applications and the incorporation with deep learning (DL) methods has experienced notable advancements in recent times. In a typical 2D PAT setup, a single-element ultrasound detector (USD) is used to collect the PA signals by making a 360° full scan of the imaging region. The traditional backprojection (BP) algorithm has been widely used to reconstruct the PAT images from the acquired signals. Accurate determination of the scanning radius (SR) is required for proper image reconstruction. Even a slight deviation from its nominal value can lead to image distortion compromising the quality of the reconstruction. To address this challenge, two approaches have been developed and examined herein. The first framework includes a modified version of dense U-Net (DUNet) architecture. The second procedure involves a DL-based convolutional neural network (CNN) for image classification followed by a DUNet. The first protocol was trained with heterogeneous simulated images generated from three different phantoms to learn the relationship between the reconstructed and the corresponding ground truth (GT) images. In the case of the second scheme, the first stage was trained with the same heterogeneous dataset to classify the image type and the second stage was trained individually with the appropriate images. The performance of these architectures has been tested on both simulated and experimental images. The first method can sustain SR deviation up to approximately 6% for simulated images and 5% for experimental images and can accurately reproduce the GTs. The proposed DL-approach extends the limits further (approximately 7% and 8% for simulated and experimental images, respectively). Our results suggest that classification-based DL method does not need a precise assessment of SR for accurate PAT image formation.

Noise-insensitive defocused signal and resolution enhancement for optical-resolution photoacoustic microscopy via deep learning

Optical-resolution photoacoustic microscopy suffers from narrow depth of field and a significant deterioration in defocused signal intensity and spatial resolution. Here, a method based on deep learning was proposed to enhance the defocused resolution and signal-to-noise ratio. A virtual optical-resolution photoacoustic microscopy based on k-wave was used to obtain the datasets of deep learning with different noise levels. A fully dense U-Net was trained with randomly distributed sources to improve the quality of photoacoustic images. The results show that the PSNR of defocused signal was enhanced by more than 1.2 times. An over 2.6-fold enhancement in lateral resolution and an over 3.4-fold enhancement in axial resolution of defocused regions were achieved. The large volumetric and high-resolution imaging of blood vessels further verified that the proposed method can effectively overcome the deterioration of the signal and the spatial resolution due to the narrow depth of field of optical-resolution photoacoustic microscopy.

Sparse-view reconstruction for photoacoustic tomography combining diffusion model with model-based iteration

As a non-invasive hybrid biomedical imaging technology, photoacoustic tomography combines high contrast of optical imaging and high penetration of acoustic imaging. However, the conventional standard reconstruction under sparse view could result in low-quality image in photoacoustic tomography. Here, a novel model-based sparse reconstruction method for photoacoustic tomography via diffusion model was proposed. A score-based diffusion model is designed for learning the prior information of the data distribution. The learned prior information is utilized as a constraint for the data consistency term of an optimization problem based on the least-square method in the model-based iterative reconstruction, aiming to achieve the optimal solution. Blood vessels simulation data and the animal in vivo experimental data were used to evaluate the performance of the proposed method. The results demonstrate that the proposed method achieves higher-quality sparse reconstruction compared with conventional reconstruction methods and U-Net. In particular, under the extreme sparse projection (e.g., 32 projections), the proposed method achieves an improvement of ∼ 260 % in structural similarity and ∼ 30 % in peak signal-to-noise ratio for in vivo data, compared with the conventional delay-and-sum method. This method has the potential to reduce the acquisition time and cost of photoacoustic tomography, which will further expand the application range.

Deep Learning Enhanced Volumetric Photoacoustic Imaging of Vasculature in Human

The development of high-performance imaging processing algorithms is a core area of photoacoustic tomography. While various deep learning based image processing techniques have been developed in the area, their applications in 3D imaging are still limited due to challenges in computational cost and memory allocation. To address those limitations, this work implements a 3D fully-dense (3DFD) U-net to linear array based photoacoustic tomography and utilizes volumetric simulation and mixed precision training to increase efficiency and training size. Through numerical simulation, phantom imaging, and in vivo experiments, this work demonstrates that the trained network restores the true object size, reduces the noise level and artifacts, improves the contrast at deep regions, and reveals vessels subject to limited view distortion. With these enhancements, 3DFD U-net successfully produces clear 3D vascular images of the palm, arms, breasts, and feet of human subjects. These enhanced vascular images offer improved capabilities for biometric identification, foot ulcer evaluation, and breast cancer imaging. These results indicate that the new algorithm will have a significant impact on preclinical and clinical photoacoustic tomography.

DE-UFormer: U-shaped dual encoder architectures for brain tumor segmentation

Objective. In brain tumor segmentation tasks, the convolutional neural network (CNN) or transformer is usually acted as the encoder since the encoder is necessary to be used. On one hand, the convolution operation of CNN has advantages of extracting local information although its performance of obtaining global expressions is bad. On the other hand, the attention mechanism of the transformer is good at establishing remote dependencies while it is lacking in the ability to extract high-precision local information. Either high precision local information or global contextual information is crucial in brain tumor segmentation tasks. The aim of this paper is to propose a brain tumor segmentation model that can simultaneously extract and fuse high-precision local and global contextual information.Approach. We propose a network model DE-Uformer with dual encoders to obtain local features and global representations using both CNN encoder and Transformer encoder. On the basis of this, we further propose the nested encoder-aware feature fusion (NEaFF) module for effective deep fusion of the information under each dimension. It may establishe remote dependencies of features under a single encoder via the spatial attention Transformer. Meanwhile ,it also investigates how features extracted from two encoders are related with the cross-encoder attention transformer.Main results. The proposed algorithm segmentation have been performed on BraTS2020 dataset and private meningioma dataset. Results show that it is significantly better than current state-of-the-art brain tumor segmentation methods.Significance. The method proposed in this paper greatly improves the accuracy of brain tumor segmentation. This advancement helps healthcare professionals perform a more comprehensive analysis and assessment of brain tumors, thereby improving diagnostic accuracy and reliability. This fully automated brain model segmentation model with high accuracy is of great significance for critical decisions made by physicians in selecting treatment strategies and preoperative planning.

SECP-Net: SE-Connection Pyramid Network for Segmentation of Organs at Risk with Nasopharyngeal Carcinoma

Nasopharyngeal carcinoma (NPC) is a kind of malignant tumor. The accurate and automatic segmentation of computed tomography (CT) images of organs at risk (OAR) is clinically significant. In recent years, deep learning models represented by U-Net have been widely applied in medical image segmentation tasks, which can help to reduce doctors' workload. In the OAR segmentation of NPC, the sizes of the OAR are variable, and some of their volumes are small. Traditional deep neural networks underperform in segmentation due to the insufficient use of global and multi-size information. Therefore, a new SE-Connection Pyramid Network (SECP-Net) is proposed. For extracting global and multi-size information, the SECP-Net designs an SE-connection module and a pyramid structure for improving the segmentation performance, especially that of small organs. SECP-Net also uses an auto-context cascaded structure to further refine the segmentation results. Comparative experiments are conducted between SECP-Net and other recent methods on a private dataset with CT images of the head and neck and a public liver dataset. Five-fold cross-validation is used to evaluate the performance based on two metrics; i.e., Dice and Jaccard similarity. The experimental results show that SECP-Net can achieve SOTA performance in these two challenging tasks.

CoTrFuse: a novel framework by fusing CNN and transformer for medical image segmentation

Medical image segmentation is a crucial and intricate process in medical image processing and analysis. With the advancements in artificial intelligence, deep learning techniques have been widely used in recent years for medical image segmentation. One such technique is the U-Net framework based on the U-shaped convolutional neural networks (CNN) and its variants. However, these methods have limitations in simultaneously capturing both the global and the remote semantic information due to the restricted receptive domain caused by the convolution operation's intrinsic features. Transformers are attention-based models with excellent global modeling capabilities, but their ability to acquire local information is limited. To address this, we propose a network that combines the strengths of both CNN and Transformer, called CoTrFuse. The proposed CoTrFuse network uses EfficientNet and Swin Transformer as dual encoders. The Swin Transformer and CNN Fusion module are combined to fuse the features of both branches before the skip connection structure. We evaluated the proposed network on two datasets: the ISIC-2017 challenge dataset and the COVID-QU-Ex dataset. Our experimental results demonstrate that the proposed CoTrFuse outperforms several state-of-the-art segmentation methods, indicating its superiority in medical image segmentation. The codes are available athttps://github.com/BinYCn/CoTrFuse.

DAWTran: dynamic adaptive windowing transformer network for pneumothorax segmentation with implicit feature alignment

Objective. This study aims to address the significant challenges posed by pneumothorax segmentation in computed tomography images due to the resemblance between pneumothorax regions and gas-containing structures such as the trachea and bronchus.Approach. We introduce a novel dynamic adaptive windowing transformer (DAWTran) network incorporating implicit feature alignment for precise pneumothorax segmentation. The DAWTran network consists of an encoder module, which employs a DAWTran, and a decoder module. We have proposed a unique dynamic adaptive windowing strategy that enables multi-head self-attention to effectively capture multi-scale information. The decoder module incorporates an implicit feature alignment function to minimize information deviation. Moreover, we utilize a hybrid loss function to address the imbalance between positive and negative samples.Main results. Our experimental results demonstrate that the DAWTran network significantly improves the segmentation performance. Specifically, it achieves a higher dice similarity coefficient (DSC) of 91.35% (a larger DSC value implies better performance), showing an increase of 2.21% compared to the TransUNet method. Meanwhile, it significantly reduces the Hausdorff distance (HD) to 8.06 mm (a smaller HD value implies better performance), reflecting a reduction of 29.92% in comparison to the TransUNet method. Incorporating the dynamic adaptive windowing (DAW) mechanism has proven to enhance DAWTran's performance, leading to a 4.53% increase in DSC and a 15.85% reduction in HD as compared to SwinUnet. The application of the implicit feature alignment (IFA) further improves the segmentation accuracy, increasing the DSC by an additional 0.11% and reducing the HD by another 10.01% compared to the model only employing DAW.Significance. These results highlight the potential of the DAWTran network for accurate pneumothorax segmentation in clinical applications, suggesting that it could be an invaluable tool in improving the precision and effectiveness of diagnosis and treatment in related healthcare scenarios. The improved segmentation performance with the inclusion of DAW and IFA validates the effectiveness of our proposed model and its components.

Optimization of U-shaped pure transformer medical image segmentation network

In recent years, neural networks have made pioneering achievements in the field of medical imaging. In particular, deep neural networks based on U-shaped structures are widely used in different medical image segmentation tasks. In order to improve the early diagnosis and clinical decision-making system of lung diseases, it has become a key step to use the neural network for lung segmentation to assist in positioning and observing the shape. There is still the problem of low precision. For the sake of achieving better segmentation accuracy, an optimized pure Transformer U-shaped segmentation is proposed in this article. The optimization segmentation network adopts the method of adding skip connections and performing special splicing processing, which reduces the information loss in the encoding process and increases the information in the decoding process, so as to achieve the purpose of improving the segmentation accuracy. The final experiment shows that our improved network achieves 97.86% accuracy in segmentation of the "Chest Xray Masks and Labels" dataset, which is better than the full convolutional network or the combination of Transformer and convolution.

Enhanced resolution and sensitivity acoustic-resolution photoacoustic microscopy with semi/unsupervised GANs

Acoustic-resolution photoacoustic microscopy (AR-PAM) enables visualization of biological tissues at depths of several millimeters with superior optical absorption contrast. However, the lateral resolution and sensitivity of AR-PAM are generally lower than those of optical-resolution PAM (OR-PAM) owing to the intrinsic physical acoustic focusing mechanism. Here, we demonstrate a computational strategy with two generative adversarial networks (GANs) to perform semi/unsupervised reconstruction with high resolution and sensitivity in AR-PAM by maintaining its imaging capability at enhanced depths. The b-scan PAM images were prepared as paired (for semi-supervised conditional GAN) and unpaired (for unsupervised CycleGAN) groups for label-free reconstructed AR-PAM b-scan image generation and training. The semi/unsupervised GANs successfully improved resolution and sensitivity in a phantom and in vivo mouse ear test with ground truth. We also confirmed that GANs could enhance resolution and sensitivity of deep tissues without the ground truth.

Mitigating Under-Sampling Artifacts in 3D Photoacoustic Imaging Using Res-UNet Based on Digital Breast Phantom

In recent years, photoacoustic (PA) imaging has rapidly grown as a non-invasive screening technique for breast cancer detection using three-dimensional (3D) hemispherical arrays due to their large field of view. However, the development of breast imaging systems is hindered by a lack of patients and ground truth samples, as well as under-sampling problems caused by high costs. Most research related to solving these problems in the PA field were based on 2D transducer arrays or simple regular shape phantoms for 3D transducer arrays or images from other modalities. Therefore, we demonstrate an effective method for removing under-sampling artifacts based on deep neural network (DNN) to reconstruct high-quality PA images using numerical digital breast simulations. We constructed 3D digital breast phantoms based on human anatomical structures and physical properties, which were then subjected to 3D Monte-Carlo and K-wave acoustic simulations to mimic acoustic propagation for hemispherical transducer arrays. Finally, we applied a 3D delay-and-sum reconstruction algorithm and a Res-UNet network to achieve higher resolution on sparsely-sampled data. Our results indicate that when using a 757 nm laser with uniform intensity distribution illuminated on a numerical digital breast, the imaging depth can reach 3 cm with 0.25 mm spatial resolution. In addition, the proposed DNN can significantly enhance image quality by up to 78.4%, as measured by MS-SSIM, and reduce background artifacts by up to 19.0%, as measured by PSNR, even at an under-sampling ratio of 10%. The post-processing time for these improvements is only 0.6 s. This paper suggests a new 3D real time DNN method addressing the sparse sampling problem based on numerical digital breast simulations, this approach can also be applied to clinical data and accelerate the development of 3D photoacoustic hemispherical transducer arrays for early breast cancer diagnosis.

Automated Segmentation of Levator Ani Muscle from 3D Endovaginal Ultrasound Images

Levator ani muscle (LAM) avulsion is a common complication of vaginal childbirth and is linked to several pelvic floor disorders. Diagnosing and treating these conditions require imaging of the pelvic floor and examination of the obtained images, which is a time-consuming process subjected to operator variability. In our study, we proposed using deep learning (DL) to automate the segmentation of the LAM from 3D endovaginal ultrasound images (EVUS) to improve diagnostic accuracy and efficiency. Over one thousand images extracted from the 3D EVUS data of healthy subjects and patients with pelvic floor disorders were utilized for the automated LAM segmentation. A U-Net model was implemented, with Intersection over Union (IoU) and Dice metrics being used for model performance evaluation. The model achieved a mean Dice score of 0.86, demonstrating a better performance than existing works. The mean IoU was 0.76, indicative of a high degree of overlap between the automated and manual segmentation of the LAM. Three other models including Attention UNet, FD-UNet and Dense-UNet were also applied on the same images which showed comparable results. Our study demonstrated the feasibility and accuracy of using DL segmentation with U-Net architecture to automate LAM segmentation to reduce the time and resources required for manual segmentation of 3D EVUS images. The proposed method could become an important component in AI-based diagnostic tools, particularly in low socioeconomic regions where access to healthcare resources is limited. By improving the management of pelvic floor disorders, our approach may contribute to better patient outcomes in these underserved areas.

Performance evaluation of mesoscopic photoacoustic imaging

Photoacoustic mesoscopy visualises vascular architecture at high-resolution up to ~3 mm depth. Despite promise in preclinical and clinical imaging studies, with applications in oncology and dermatology, the accuracy and precision of photoacoustic mesoscopy is not well established. Here, we evaluate a commercial photoacoustic mesoscopy system for imaging vascular structures. Typical artefact types are first highlighted and limitations due to non-isotropic illumination and detection are evaluated with respect to rotation, angularity, and depth of the target. Then, using tailored phantoms and mouse models, we investigate system precision, showing coefficients of variation (COV) between repeated scans [short term (1 h): COV= 1.2%; long term (25 days): COV= 9.6%], from target repositioning (without: COV=1.2%, with: COV=4.1%), or from varying in vivo user experience (experienced: COV=15.9%, unexperienced: COV=20.2%). Our findings show robustness of the technique, but also underscore general challenges of limited-view photoacoustic systems in accurately imaging vessel-like structures, thereby guiding users when interpreting biologically-relevant information.

Non-uniform image reconstruction for fast photoacoustic microscopy of histology imaging

Photoacoustic microscopic imaging utilizes the characteristic optical absorption properties of pigmented materials in tissues to enable label-free observation of fine morphological and structural features. Since DNA/RNA can strongly absorb ultraviolet light, ultraviolet photoacoustic microscopy can highlight the cell nucleus without complicated sample preparations such as staining, which is comparable to the standard pathological images. Further improvements in the imaging acquisition speed are critical to advancing the clinical translation of photoacoustic histology imaging technology. However, improving the imaging speed with additional hardware is hampered by considerable costs and complex design. In this work, considering heavy redundancy in the biological photoacoustic images that overconsume the computing power, we propose an image reconstruction framework called non-uniform image reconstruction (NFSR), which exploits an object detection network to reconstruct low-sampled photoacoustic histology images into high-resolution images. The sampling speed of photoacoustic histology imaging is significantly improved, saving 90% of the time cost. Furthermore, NFSR focuses on the reconstruction of the region of interest while maintaining high PSNR and SSIM evaluation indicators of more than 99% but reducing the overall computation by 60%.

Invariant representations in deep learning for optoacoustic imaging

Image reconstruction in optoacoustic tomography (OAT) is a trending learning task highly dependent on measured physical magnitudes present at sensing time. A large number of different settings and also the presence of uncertainties or partial knowledge of parameters can lead to reconstruction algorithms that are specifically tailored and designed to a particular configuration, which could not be the one that will ultimately be faced in a final practical situation. Being able to learn reconstruction algorithms that are robust to different environments (e.g., the different OAT image reconstruction settings) or invariant to such environments is highly valuable because it allows us to focus on what truly matters for the application at hand and discard what are considered spurious features. In this work, we explore the use of deep learning algorithms based on learning invariant and robust representations for the OAT inverse problem. In particular, we consider the application of the ANDMask scheme due to its easy adaptation to the OAT problem. Numerical experiments are conducted showing that when out-of-distribution generalization (against variations in parameters such as the location of the sensors) is imposed, there is no degradation of the performance and, in some cases, it is even possible to achieve improvements with respect to standard deep learning approaches where invariance robustness is not explicitly considered.

ACU2E-Net: A novel predict-refine attention network for segmentation of soft-tissue structures in ultrasound images

Segmentation of anatomical structures in ultrasound images is a challenging task due to existence of artifacts inherit to the modality such as speckle noise, attenuation, shadowing, uneven textures and blurred boundaries. This paper presents a novel attention-based predict-refine network, called ACU2E-Net, for segmentation of soft-tissue structures in ultrasound images. The network consists of two modules: a predict module, which is built upon our newly proposed attentive coordinate convolution; and a novel multi-head residual refinement module, which consists of three parallel residual refinement modules. The attentive coordinate convolution is designed to improve the segmentation accuracy by perceiving the shape and positional information of the target anatomy. The proposed multi-head residual refinement module reduces both segmentation biases and variances by integrating residual refinement and ensemble strategies. Moreover, it avoids multi-pass training and inference commonly seen in ensemble methods. To show the effectiveness of our method, we collect a comprehensive dataset of thyroid ultrasound scans from 12 different imaging centers, and evaluate our proposed network against state-of-the-art segmentation methods. Comparisons against state-of-the-art models demonstrate the competitive performance of our newly designed network on both the transverse and sagittal thyroid images. Ablation studies show that proposed modules improve the segmentation Dice score of the baseline model from 79.62% to 80.97% and 82.92% while reducing the variance from 6.12% to 4.67% and 3.21% in transverse and sagittal views, respectively.