Equilibrium Optimization Algorithm with Deep Learning Enabled Prostate Cancer Detection on MRI Images

The enlargement of the prostate gland in the reproductive system of males is considered a form of prostate cancer (PrC). The survival rate is considerably improved with earlier diagnosis of cancer; thus, timely intervention should be administered. In this study, a new automatic approach combining several deep learning (DL) techniques was introduced to detect PrC from MRI and ultrasound (US) images. Furthermore, the presented method describes why a certain decision was made given the input MRI or US images. Many pretrained custom-developed layers were added to the pretrained model and employed in the dataset. The study presents an Equilibrium Optimization Algorithm with Deep Learning-based Prostate Cancer Detection and Classification (EOADL-PCDC) technique on MRIs. The main goal of the EOADL-PCDC method lies in the detection and classification of PrC. To achieve this, the EOADL-PCDC technique applies image preprocessing to improve the image quality. In addition, the EOADL-PCDC technique follows the CapsNet (capsule network) model for the feature extraction model. The EOA is based on hyperparameter tuning used to increase the efficiency of CapsNet. The EOADL-PCDC algorithm makes use of the stacked bidirectional long short-term memory (SBiLSTM) model for prostate cancer classification. A comprehensive set of simulations of the EOADL-PCDC algorithm was tested on the benchmark MRI dataset. The experimental outcome revealed the superior performance of the EOADL-PCDC approach over existing methods in terms of different metrics.

Automated Dual-Mode Cell Monitoring To Simultaneously Explore Calcium Dynamics and Contraction-Relaxation Kinetics within Drug-Treated Stem Cell-Derived Cardiomyocytes

This manuscript proposes a new dual-mode cell imaging system for studying the relationships between calcium dynamics and the contractility process of cardiomyocytes derived from human-induced pluripotent stem cells. Practically, this dual-mode cell imaging system provides simultaneously both live cell calcium imaging and quantitative phase imaging based on digital holographic microscopy. Specifically, thanks to the development of a robust automated image analysis, simultaneous measurements of both intracellular calcium, a key player of excitation-contraction coupling, and the quantitative phase image-derived dry mass redistribution, reflecting the effective contractility, namely, the contraction and relaxation processes, were achieved. Practically, the relationships between calcium dynamics and the contraction-relaxation kinetics were investigated in particular through the application of two drugs─namely, isoprenaline and E-4031─known to act precisely on calcium dynamics. Specifically, this new dual-mode cell imaging system enabled us to establish that calcium regulation can be divided into two phases, an early phase influencing the occurrence of the relaxation process followed by a late phase, which although not having a significant influence on the relaxation process affects significantly the beat frequency. In combination with cutting-edge technologies allowing the generation of human stem cell-derived cardiomyocytes, this dual-mode cell monitoring approach therefore represents a very promising technique, particularly in the fields of drug discovery and personalized medicine, to identify compounds likely to act more selectively on specific steps that compose the cardiomyocyte contractility.

Automated analysis of human cardiomyocytes dynamics with holographic image-based tracking for cardiotoxicity screening

This paper proposes a new non-invasive, low-cost, and fully automated platform to quantitatively analyze dynamics of human-induced pluripotent stem cell-derived cardiomyocytes (hiPS-CMs) at the single-cell level by holographic image-based tracking for cardiotoxicity screening. A dense Farneback optical flow method and holographic imaging informatics were combined to characterize the contractile motion of a single CM, which obviates the need for costly equipment to monitor a CM's mechanical beat activity. The reliability of the proposed platform was tested by single-cell motion characterization, synchronization analysis, motion speed measurement of fixed CMs versus live CMs, and noise sensitivity. The applicability of the motion characterization method was tested to determine the pharmacological effects of two cardiovascular drugs, isoprenaline (166 nM) and E-4031 (500 μM). The experiments were done using single CMs and multiple cells, and the results were compared to control conditions. Cardiomyocytes responded to isoprenaline by increasing the action potential (AP) speed and shortening the resting period, thus increasing the beat frequency. In the presence of E-4031, the AP speed was decreased, and the resting period was prolonged, thus decreasing the beat frequency. The findings offer insights into single hiPS-CMs' contractile motion and a deep understanding of their kinetics at the single-cell level for cardiotoxicity screening.

Automated phase unwrapping in digital holography with deep learning

Digital holography can provide quantitative phase images related to the morphology and content of biological samples. After the numerical image reconstruction, the phase values are limited between -π and π; thus, discontinuity may occur due to the modulo 2π operation. We propose a new deep learning model that can automatically reconstruct unwrapped focused-phase images by combining digital holography and a Pix2Pix generative adversarial network (GAN) for image-to-image translation. Compared with numerical phase unwrapping methods, the proposed GAN model overcomes the difficulty of accurate phase unwrapping due to abrupt phase changes and can perform phase unwrapping at a twice faster rate. We show that the proposed model can generalize well to different types of cell images and has high performance compared to recent U-net models. The proposed method can be useful in observing the morphology and movement of biological cells in real-time applications.

Deep Learning-based Phenotypic Assessment of Red Cell Storage Lesions for Safe Transfusions

This study presents a novel approach to automatically perform instant phenotypic assessment of red blood cell (RBC) storage lesion in phase images obtained by digital holographic microscopy. The proposed model combines a generative adversarial network (GAN) with marker-controlled watershed segmentation scheme. The GAN model performed RBC segmentations and classifications to develop ageing markers, and the watershed segmentation was used to completely separate overlapping RBCs. Our approach achieved good segmentation and classification accuracy with a Dices coefficient of 0.94 at a high throughput rate of about 152 cells per second. These results were compared with other deep neural network architectures. Moreover, our image-based deep learning models recognized the morphological changes that occur in RBCs during storage. Our deep learning-based classification results were in good agreement with previous findings on the changes in RBC markers (dominant shapes) affected by storage duration. We believe that our image-based deep learning models can be useful for automated assessment of RBC quality, storage lesions for safe transfusions, and diagnosis of RBC-related diseases.

High-throughput label-free cell detection and counting from diffraction patterns with deep fully convolutional neural networks

SIGNIFICANCE

Digital holographic microscopy (DHM) is a promising technique for the study of semitransparent biological specimen such as red blood cells (RBCs). It is important and meaningful to detect and count biological cells at the single cell level in biomedical images for biomarker discovery and disease diagnostics. However, the biological cell analysis based on phase information of images is inefficient due to the complexity of numerical phase reconstruction algorithm applied to raw hologram images. New cell study methods based on diffraction pattern directly are desirable.

AIM

Deep fully convolutional networks (FCNs) were developed on raw hologram images directly for high-throughput label-free cell detection and counting to assist the biological cell analysis in the future.

APPROACH

The raw diffraction patterns of RBCs were recorded by use of DHM. Ground-truth mask images were labeled based on phase images reconstructed from RBC holograms using numerical reconstruction algorithm. A deep FCN, which is UNet, was trained on the diffraction pattern images to achieve the label-free cell detection and counting.

RESULTS

The implemented deep FCNs provide a promising way to high-throughput and label-free counting of RBCs with a counting accuracy of 99% at a throughput rate of greater than 288 cells per second and 200  μm  ×  200  μm field of view at the single cell level. Compared to convolutional neural networks, the FCNs can get much better results in terms of accuracy and throughput rate.

CONCLUSIONS

High-throughput label-free cell detection and counting were successfully achieved from diffraction patterns with deep FCNs. It is a promising approach for biological specimen analysis based on raw hologram directly.

Noise-free quantitative phase imaging in Gabor holography with conditional generative adversarial network

This paper shows that deep learning can eliminate the superimposed twin-image noise in phase images of Gabor holographic setup. This is achieved by the conditional generative adversarial model (C-GAN), trained by input-output pairs of noisy phase images obtained from synthetic Gabor holography and the corresponding quantitative noise-free contrast-phase image obtained by the off-axis digital holography. To train the model, Gabor holograms are generated from digital off-axis holograms with spatial shifting of the real image and twin image in the frequency domain and then adding them with the DC term in the spatial domain. Finally, the digital propagation of the Gabor hologram with Fresnel approximation generates a super-imposed phase image for the C-GAN model input. Two models were trained: a human red blood cell model and an elliptical cancer cell model. Following the training, several quantitative analyses were conducted on the bio-chemical properties and similarity between actual noise-free phase images and the model output. Surprisingly, it is discovered that our model can recover other elliptical cell lines that were not observed during the training. Additionally, some misalignments can also be compensated with the trained model. Particularly, if the reconstruction distance is somewhat incorrect, this model can still retrieve in-focus images.

Automated single cardiomyocyte characterization by nucleus extraction from dynamic holographic images using a fully convolutional neural network

Human-induced pluripotent stem cell-derived cardiomyocytes (hiPS-CMs) beating can be efficiently characterized by time-lapse quantitative phase imaging (QPIs) obtained by digital holographic microscopy. Particularly, the CM's nucleus section can precisely reflect the associated rhythmic beating pattern of the CM suitable for subsequent beating pattern characterization. In this paper, we describe an automated method to characterize single CMs by nucleus extraction from QPIs and subsequent beating pattern reconstruction and quantification. However, accurate CM's nucleus extraction from the QPIs is a challenging task due to the variations in shape, size, orientation, and lack of special geometry. To this end, we propose a novel fully convolutional neural network (FCN)-based network architecture for accurate CM's nucleus extraction using pixel classification technique and subsequent beating pattern characterization. Our experimental results show that the beating profile of multiple extracted single CMs is less noisy and more informative compared to the whole image slide. Applying this method allows CM characterization at the single-cell level. Consequently, several single CMs are extracted from the whole slide QPIs and multiple parameters regarding their beating profile of each isolated CM are efficiently measured.

P1274 Early surgery versus watchful waiting in patients with moderate aortic stenosis and left ventricular systolic dysfunction

Background

Aortic stenosis (AS) induces significant pressure overload to the left ventricle (LV) and its burden may increase if there is concomitant LV systolic dysfunction. Severe AS with LV systolic dysfunction is a class I indication for aortic valve replacement (AVR) irrespective of symptoms, however, this recommendation is not well established in those with moderate AS and LV systolic dysfunction. In this study, we sought to investigate the clinical impact of surgical AVR among patients with moderate AS and LV systolic dysfunction.

Methods

From 2001 to 2017, we retrospectively but consecutively identified patients with moderate AS and LV systolic dysfunction from a single tertiary hospital. Moderate AS was defined as aortic valve area between 1.0 and 1.5cm2 and LV systolic dysfunction as LV ejection fraction less than 50%. The primary outcome was all-cause death and we additionally analyzed cardiac death as a secondary endpoint. The outcomes were compared between those who underwent early surgical AVR at the stage of moderate AS versus those who were followed without AVR at the outpatient clinic.

Results

Among a total of 257 patients with moderate AS and concomitant LV systolic dysfunction (70.0 ± 11.3 years, 63.4% of male), 34 patients received early AVR. Patients in the AVR group was younger than the observation group (64.2 ± 8.1 vs. 70.9 ± 11.5, respectively), and had a lower prevalence of hypertension and chronic kidney disease. During a mean of 3-year follow up, 112 patients (47.5%) died and the overall death rate was 15.367 per 100 person-year (PY). The AVR group showed a significantly lower rate of all-cause death than the observation group (5.241PY vs. 18.160PY, p-value = 0.002). After multivariable Cox proportional hazard regression adjusting for age, sex, comorbidities and laboratory data, early AVR at the stage of moderate AS significantly reduced the risk of all-cause death (hazard ratio [HR] 0.340, 95% confidence interval [CI] 0.117 - 0.985, p-value = 0.047). However, there was no risk reduction of cardiac death (HR 0.578 95% CI 0.150 - 2.231, p-value = 0.426).

Conclusions

In patients with moderate AS and LV systolic dysfunction, AVR reduces the risk of all-cause death. A prospective design study is warranted to confirm our findings in the near future.

Abstract P1274 Figure. Kaplan-Meier curves for deaths

P785 Left ventricular geometry and myocardial contractility modulate impact of statins on prognosis in patients with acute heart failure

Funding Acknowledgements

N/A

Background/Introduction: The benefit of statins in patients with heart failure (HF) remains controversial and the mechanism of action is largely speculative. We investigated whether survival benefit with statins differs according to left ventricular (LV) geometry and myocardial contractility in acute HF patients.

Methods

We enrolled 1792 acute HF patients receiving statins and 2296 patients not receiving statins admitted from 2009 to 2016. The LV and right ventricular (RV) global longitudinal strain (GLS) was assessed as a measure of myocardial contractility. Patients were classified into 2 groups based on ischemic etiology of HF and further divided into 4 subgroups according to the median values of LV-GLS or RV-GLS. The primary outcome was 5-year all-cause mortality. The study protocol was approved by the ethics committee at each institute and complied with the Declaration of Helsinki. The need for written informed consent was waived.

Results

During the 5-year follow-up, 1740 (40.4%) patients died and they had more unfavorable baseline characteristics. Statin therapy was significantly associated with improved survival in overall patients and in both groups with and without ischemic etiology (all p <0.001). Patients with concentric remodeling/hypertrophy and eccentric hypertrophy demonstrated survival benefit with statin therapy (P = 0.033, 0.004, and 0.008, respectively), while those with normal geometry did not (p = 0.123). In the non-ischemic HF group, survival benefit with statin therapy was confined to patients with low LV-GLS (p = 0.045) or those with low RV-GLS p = 0.003). On the contrary, in ischemic HF group, survival benefit with statin therapy was observed in all patients regardless of the values of LV-GLS or RV-GLS. Significant interactions were present between statin use and diabetes mellitus and IHD (p for interaction = 0.027 and 0.003, respectively) regarding mortality.

Conclusions

LV geometry and myocardial contractility may modulate the effects of statins in patients with acute HF. These echocardiographic measures can provide prognostic information to guide tailored statin treatment in this population. Our findings may also help to develop more well-designed prospective studies, in terms of a more homogenous study population, to confirm survival benefit with statin therapy.

Abstract P785 Figure. Multivariate Cox survival curves

Secure storage and retrieval schemes for multiple encrypted digital holograms with orthogonal phase encoding multiplexing

Recent developments in 3D computational optical imaging such as digital holographic microscopy has ushered in a new era for biological research. Therefore, efficient and secure storage and retrieval of digital holograms is a challenging task for future cloud computing services. In this study, we propose a novel scheme to securely store and retrieve multiple encrypted digital holograms by using phase encoding multiplexing. In the proposed schemes, an encrypted hologram can only be accessed using a binary phase mask, which is the key to retrieve the image. In addition, it is possible to independently store, retrieve, and manage the encrypted digital holograms without affecting other groups of the encrypted holograms multiplexed using different sets of binary phase masks, due to the orthogonality properties of the Hadamard matrices with high autocorrelation and low cross-correlation. The desired encrypted holograms may also be searched for, removed, and added independently of other groups of the encrypted holograms. More and more 3D images or digital holograms can be securely and efficiently stored, retrieved, and managed.

No-search focus prediction at the single cell level in digital holographic imaging with deep convolutional neural network

Digital propagation of an off-axis hologram can provide the quantitative phase-contrast image if the exact distance between the sensor plane (such as CCD) and the reconstruction plane is correctly provided. In this paper, we present a deep-learning convolutional neural network with a regression layer as the top layer to estimate the best reconstruction distance. The experimental results obtained using microsphere beads and red blood cells show that the proposed method can accurately predict the propagation distance from a filtered hologram. The result is compared with the conventional automatic focus-evaluation function. Additionally, our approach can be utilized at the single-cell level, which is useful for cell-to-cell depth measurement and cell adherent studies.

Automated quantitative analysis of multiple cardiomyocytes at the single-cell level with three-dimensional holographic imaging informatics

Cardiomyocytes derived from human pluripotent stem cells are a promising tool for disease modeling, drug compound testing, and cardiac toxicity screening. Bio-image segmentation is a prerequisite step in cardiomyocyte image analysis by digital holography (DH) in microscopic configuration and has provided satisfactory results. In this study, we quantified multiple cardiac cells from segmented 3-dimensional DH images at the single-cell level and measured multiple parameters describing the beating profile of each individual cell. The beating profile is extracted by monitoring dry-mass distribution during the mechanical contraction-relaxation activity caused by cardiac action potential. We present a robust two-step segmentation method for cardiomyocyte low-contrast image segmentation based on region and edge information. The segmented single-cell contains mostly the nucleus of the cell since it is the best part of the cardiac cell, which can be perfectly segmented. Clustering accuracy was assessed by a silhouette index evaluation for k-means clustering and the Dice similarity coefficient (DSC) of the final segmented image. 3D representation of single of cardiomyocytes. The cell contains mostly the nucleus section and a small area of cytoplasm.

Quantification of stored red blood cell fluctuations by time-lapse holographic cell imaging

We propose methods to quantitatively calculate the fluctuation rate of red blood cells with nanometric axial and millisecond temporal sensitivity at the single-cell level by using time-lapse holographic cell imaging. For this quantitative analysis, cell membrane fluctuations (CMFs) were measured for RBCs stored at different storage times. Measurements were taken over the whole membrane for both the ring and dimple sections separately. The measurements show that healthy RBCs that maintain their discocyte shape become stiffer with storage time. The correlation analysis demonstrates a significant negative correlation between CMFs and the sphericity coefficient, which characterizes the morphological type of erythrocyte. In addition, we show the correlation results between CMFs and other morphological properties such as projected surface area, surface area, mean corpuscular volume, and mean corpuscular hemoglobin.

Automated red blood cells extraction from holographic images using fully convolutional neural networks

In this paper, we present two models for automatically extracting red blood cells (RBCs) from RBCs holographic images based on a deep learning fully convolutional neural network (FCN) algorithm. The first model, called FCN-1, only uses the FCN algorithm to carry out RBCs prediction, whereas the second model, called FCN-2, combines the FCN approach with the marker-controlled watershed transform segmentation scheme to achieve RBCs extraction. Both models achieve good segmentation accuracy. In addition, the second model has much better performance in terms of cell separation than traditional segmentation methods. In the proposed methods, the RBCs phase images are first numerically reconstructed from RBCs holograms recorded with off-axis digital holographic microscopy. Then, some RBCs phase images are manually segmented and used as training data to fine-tune the FCN. Finally, each pixel in new input RBCs phase images is predicted into either foreground or background using the trained FCN models. The RBCs prediction result from the first model is the final segmentation result, whereas the result from the second model is used as the internal markers of the marker-controlled transform algorithm for further segmentation. Experimental results show that the given schemes can automatically extract RBCs from RBCs phase images and much better RBCs separation results are obtained when the FCN technique is combined with the marker-controlled watershed segmentation algorithm.

Automated three-dimensional morphology-based clustering of human erythrocytes with regular shapes: stomatocytes, discocytes, and echinocytes

We present unsupervised clustering methods for automatic grouping of human red blood cells (RBCs) extracted from RBC quantitative phase images obtained by digital holographic microscopy into three RBC clusters with regular shapes, including biconcave, stomatocyte, and sphero-echinocyte. We select some good features related to the RBC profile and morphology, such as RBC average thickness, sphericity coefficient, and mean corpuscular volume, and clustering methods, including density-based spatial clustering applications with noise, k-medoids, and k-means, are applied to the set of morphological features. The clustering results of RBCs using a set of three-dimensional features are compared against a set of two-dimensional features. Our experimental results indicate that by utilizing the introduced set of features, two groups of biconcave RBCs and old RBCs (suffering from the sphero-echinocyte process) can be perfectly clustered. In addition, by increasing the number of clusters, the three RBC types can be effectively clustered in an automated unsupervised manner with high accuracy. The performance evaluation of the clustering techniques reveals that they can assist hematologists in further diagnosis.

Three-dimensional image authentication scheme using sparse phase information in double random phase encoded integral imaging

In recent years, many studies have focused on authentication of two-dimensional (2D) images using double random phase encryption techniques. However, there has been little research on three-dimensional (3D) imaging systems, such as integral imaging, for 3D image authentication. We propose a 3D image authentication scheme based on a double random phase integral imaging method. All of the 2D elemental images captured through integral imaging are encrypted with a double random phase encoding algorithm and only partial phase information is reserved. All the amplitude and other miscellaneous phase information in the encrypted elemental images is discarded. Nevertheless, we demonstrate that 3D images from integral imaging can be authenticated at different depths using a nonlinear correlation method. The proposed 3D image authentication algorithm can provide enhanced information security because the decrypted 2D elemental images from the sparse phase cannot be easily observed by the naked eye. Additionally, using sparse phase images without any amplitude information can greatly reduce data storage costs and aid in image compression and data transmission.

Lossless and lossy compression of quantitative phase images of red blood cells obtained by digital holographic imaging

In this paper, we evaluate lossless and lossy compression techniques to compress quantitative phase images of red blood cells (RBCs) obtained by an off-axis digital holographic microscopy (DHM). The RBC phase images are numerically reconstructed from their digital holograms and are stored in 16-bit unsigned integer format. In the case of lossless compression, predictive coding of JPEG lossless (JPEG-LS), JPEG2000, and JP3D are evaluated, and compression ratio (CR) and complexity (compression time) are compared against each other. It turns out that JP2k can outperform other methods by having the best CR. In the lossy case, JP2k and JP3D with different CRs are examined. Because some data is lost in a lossy way, the degradation level is measured by comparing different morphological and biochemical parameters of RBC before and after compression. Morphological parameters are volume, surface area, RBC diameter, sphericity index, and the biochemical cell parameter is mean corpuscular hemoglobin (MCH). Experimental results show that JP2k outperforms JP3D not only in terms of mean square error (MSE) when CR increases, but also in compression time in the lossy compression way. In addition, our compression results with both algorithms demonstrate that with high CR values the three-dimensional profile of RBC can be preserved and morphological and biochemical parameters can still be within the range of reported values.

Human red blood cell recognition enhancement with three-dimensional morphological features obtained by digital holographic imaging

The classification of erythrocytes plays an important role in the field of hematological diagnosis, specifically blood disorders. Since the biconcave shape of red blood cell (RBC) is altered during the different stages of hematological disorders, we believe that the three-dimensional (3-D) morphological features of erythrocyte provide better classification results than conventional two-dimensional (2-D) features. Therefore, we introduce a set of 3-D features related to the morphological and chemical properties of RBC profile and try to evaluate the discrimination power of these features against 2-D features with a neural network classifier. The 3-D features include erythrocyte surface area, volume, average cell thickness, sphericity index, sphericity coefficient and functionality factor, MCH and MCHSD, and two newly introduced features extracted from the ring section of RBC at the single-cell level. In contrast, the 2-D features are RBC projected surface area, perimeter, radius, elongation, and projected surface area to perimeter ratio. All features are obtained from images visualized by off-axis digital holographic microscopy with a numerical reconstruction algorithm, and four categories of biconcave (doughnut shape), flat-disc, stomatocyte, and echinospherocyte RBCs are interested. Our experimental results demonstrate that the 3-D features can be more useful in RBC classification than the 2-D features. Finally, we choose the best feature set of the 2-D and 3-D features by sequential forward feature selection technique, which yields better discrimination results. We believe that the final feature set evaluated with a neural network classification strategy can improve the RBC classification accuracy.

Cell morphology-based classification of red blood cells using holographic imaging informatics

We present methods that automatically select a linear or nonlinear classifier for red blood cell (RBC) classification by analyzing the equality of the covariance matrices in Gabor-filtered holographic images. First, the phase images of the RBCs are numerically reconstructed from their holograms, which are recorded using off-axis digital holographic microscopy (DHM). Second, each RBC is segmented using a marker-controlled watershed transform algorithm and the inner part of the RBC is identified and analyzed. Third, the Gabor wavelet transform is applied to the segmented cells to extract a series of features, which then undergo a multivariate statistical test to evaluate the equality of the covariance matrices of the different shapes of the RBCs using selected features. When these covariance matrices are not equal, a nonlinear classification scheme based on quadratic functions is applied; otherwise, a linear classification is applied. We used the stomatocyte, discocyte, and echinocyte RBC for classifier training and testing. Simulation results demonstrated that 10 of the 14 RBC features are useful in RBC classification. Experimental results also revealed that the covariance matrices of the three main RBC groups are not equal and that a nonlinear classification method has a much lower misclassification rate. The proposed automated RBC classification method has the potential for use in drug testing and the diagnosis of RBC-related diseases.

Efficient asymmetric image authentication schemes based on photon counting-double random phase encoding and RSA algorithms

Recently, double random phase encoding (DRPE) has been integrated with the photon counting (PC) imaging technique for the purpose of secure image authentication. In this scheme, the same key should be securely distributed and shared between the sender and receiver, but this is one of the most vexing problems of symmetric cryptosystems. In this study, we propose an efficient asymmetric image authentication scheme by combining the PC-DRPE and RSA algorithms, which solves key management and distribution problems. The retrieved image from the proposed authentication method contains photon-limited encrypted data obtained by means of PC-DRPE. Therefore, the original image can be protected while the retrieved image can be efficiently verified using a statistical nonlinear correlation approach. Experimental results demonstrate the feasibility of our proposed asymmetric image authentication method.

Automated tracking of temporal displacements of a red blood cell obtained by time-lapse digital holographic microscopy

Red blood cell (RBC) phase images that are numerically reconstructed by digital holographic microscopy (DHM) can describe the cell structure and dynamics information beneficial for a quantitative analysis of RBCs. However, RBCs investigated with time-lapse DHM undergo temporal displacements when their membranes are loosely attached to the substrate during sedimentation on a glass surface or due to the microscope drift. Therefore, we need to develop a tracking algorithm to localize the same RBC among RBC image sequences and dynamically monitor its biophysical cell parameters; this information is helpful for studies on RBC-related diseases and drug tests. Here, we propose a method, which is a combination of the mean-shift algorithm and Kalman filter, to track a single RBC and demonstrate that the optical path length of the single RBC can be continually extracted from the tracked RBC. The Kalman filter is utilized to predict the target RBC position in the next frame. Then, the mean-shift algorithm starts execution from the predicted location, and a robust kernel, which is adaptive to changes in the RBC scale, shape, and direction, is designed to improve the accuracy of the tracking. Finally, the tracked RBC is segmented and parameters such as the RBC location are extracted to update the Kalman filter and the kernel function for mean-shift tracking; the characteristics of the target RBC are dynamically observed. Experimental results show the feasibility of the proposed algorithm.

Quantitative investigation of red blood cell three-dimensional geometric and chemical changes in the storage lesion using digital holographic microscopy

Quantitative phase information obtained by digital holographic microscopy (DHM) can provide new insight into the functions and morphology of single red blood cells (RBCs). Since the functionality of a RBC is related to its three-dimensional (3-D) shape, quantitative 3-D geometric changes induced by storage time can help hematologists realize its optimal functionality period. We quantitatively investigate RBC 3-D geometric changes in the storage lesion using DHM. Our experimental results show that the substantial geometric transformation of the biconcave-shaped RBCs to the spherocyte occurs due to RBC storage lesion. This transformation leads to progressive loss of cell surface area, surface-to-volume ratio, and functionality of RBCs. Furthermore, our quantitative analysis shows that there are significant correlations between chemical and morphological properties of RBCs.

Automated multi-parameter measurement of cardiomyocytes dynamics with digital holographic microscopy

Compounds tested during drug development may have adverse effects on the heart; therefore all new chemical entities have to undergo extensive preclinical assessment for cardiac liability. Conventional intensity-based imaging techniques are not robust enough to provide detailed information for cell structure and the captured images result in low-contrast, especially to cell with semi-transparent or transparent feature, which would affect the cell analysis. In this paper we show, for the first time, that digital holographic microscopy (DHM) integrated with information processing algorithms automatically provide dynamic quantitative phase profiles of beating cardiomyocytes. We experimentally demonstrate that relevant parameters of cardiomyocytes can be obtained by our automated algorithm based on DHM phase signal analysis and used to characterize the physiological state of resting cardiomyocytes. Our study opens the possibility of automated quantitative analysis of cardiomyocyte dynamics suitable for further drug safety testing and compounds selection as a new paradigm in drug toxicity screens.

Three-dimensional counting of morphologically normal human red blood cells via digital holographic microscopy

Counting morphologically normal cells in human red blood cells (RBCs) is extremely beneficial in the health care field. We propose a three-dimensional (3-D) classification method of automatically determining the morphologically normal RBCs in the phase image of multiple human RBCs that are obtained by off-axis digital holographic microscopy (DHM). The RBC holograms are first recorded by DHM, and then the phase images of multiple RBCs are reconstructed by a computational numerical algorithm. To design the classifier, the three typical RBC shapes, which are stomatocyte, discocyte, and echinocyte, are used for training and testing. Nonmain or abnormal RBC shapes different from the three normal shapes are defined as the fourth category. Ten features, including projected surface area, average phase value, mean corpuscular hemoglobin, perimeter, mean corpuscular hemoglobin surface density, circularity, mean phase of center part, sphericity coefficient, elongation, and pallor, are extracted from each RBC after segmenting the reconstructed phase images by using a watershed transform algorithm. Moreover, four additional properties, such as projected surface area, perimeter, average phase value, and elongation, are measured from the inner part of each cell, which can give significant information beyond the previous 10 features for the separation of the RBC groups; these are verified in the experiment by the statistical method of Hotelling's T-quare test. We also apply the principal component analysis algorithm to reduce the dimension number of variables and establish the Gaussian mixture densities using the projected data with the first eight principal components. Consequently, the Gaussian mixtures are used to design the discriminant functions based on Bayesian decision theory. To improve the performance of the Bayes classifier and the accuracy of estimation of its error rate, the leaving-one-out technique is applied. Experimental results show that the proposed method can yield good results for calculating the percentage of each typical normal RBC shape in a reconstructed phase image of multiple RBCs that will be favorable to the analysis of RBC-related diseases. In addition, we show that the discrimination performance for the counting of normal shapes of RBCs can be improved by using 3-D features of an RBC.

Simultaneous reconstruction of multiple depth images without off-focus points in integral imaging using a graphics processing unit

The reconstruction of multiple depth images with a ray back-propagation algorithm in three-dimensional (3D) computational integral imaging is computationally burdensome. Further, a reconstructed depth image consists of a focus and an off-focus area. Focus areas are 3D points on the surface of an object that are located at the reconstructed depth, while off-focus areas include 3D points in free-space that do not belong to any object surface in 3D space. Generally, without being removed, the presence of an off-focus area would adversely affect the high-level analysis of a 3D object, including its classification, recognition, and tracking. Here, we use a graphics processing unit (GPU) that supports parallel processing with multiple processors to simultaneously reconstruct multiple depth images using a lookup table containing the shifted values along the x and y directions for each elemental image in a given depth range. Moreover, each 3D point on a depth image can be measured by analyzing its statistical variance with its corresponding samples, which are captured by the two-dimensional (2D) elemental images. These statistical variances can be used to classify depth image pixels as either focus or off-focus points. At this stage, the measurement of focus and off-focus points in multiple depth images is also implemented in parallel on a GPU. Our proposed method is conducted based on the assumption that there is no occlusion of the 3D object during the capture stage of the integral imaging process. Experimental results have demonstrated that this method is capable of removing off-focus points in the reconstructed depth image. The results also showed that using a GPU to remove the off-focus points could greatly improve the overall computational speed compared with using a CPU.

Avalanche and bit independence characteristics of double random phase encoding in the Fourier and Fresnel domains

In this work, we evaluate the avalanche effect and bit independence properties of the double random phase encoding (DRPE) algorithm in the Fourier and Fresnel domains. Experimental results show that DRPE has excellent bit independence characteristics in both the Fourier and Fresnel domains. However, DRPE achieves better avalanche effect results in the Fresnel domain than in the Fourier domain. DRPE gives especially poor avalanche effect results in the Fourier domain when only one bit is changed in the plaintext or in the encryption key. Despite this, DRPE shows satisfactory avalanche effect results in the Fresnel domain when any other number of bits changes in the plaintext or in the encryption key. To the best of our knowledge, this is the first report on the avalanche effect and bit independence behaviors of optical encryption approaches for bit units.

Automated quantitative analysis of 3D morphology and mean corpuscular hemoglobin in human red blood cells stored in different periods

Quantitative phase (QP) images of red blood cells (RBCs), which are obtained by off-axis digital holographic microscopy, can provide quantitative information about three-dimensional (3D) morphology of human RBCs and the characteristic properties such as mean corpuscular hemoglobin (MCH) and MCH surface density (MCHSD). In this paper, we investigate modifications of the 3D morphology and MCH in RBCs induced by the period of storage time for the purpose of classification of RBCs with different periods of storage by using off-axis digital holographic microscopy. The classification of RBCs based on the duration of storage is highly relevant because a long storage of blood before transfusion may alter the functionality of RBCs and, therefore, cause complications in patients. To analyze any changes in the 3D morphology and MCH of RBCs due to storage, we use data sets from RBC samples stored for 8, 13, 16, 23, 27, 30, 34, 37, 40, 47, and 57 days, respectively. The data sets consist of more than 3,300 blood cells in eleven classes, with more than 300 blood cells per class. The classes indicate the storage period of RBCs and are listed in chronological order. Using the RBCs donated by healthy persons, the off-axis digital holographic microscopy reconstructs several quantitative phase images of RBC samples stored for eleven different periods. We employ marker-controlled watershed transform to remove the background in the RBC quantitative phase images obtained by the off-axis digital holographic microscopy. More than 300 single RBCs are extracted from the segmented quantitative phase images for each class. Such a large number of RBC samples enable us to obtain statistical distributions of the characteristic properties of RBCs after a specific period of storage. Experimental results show that the 3D morphology of the RBCs, in contrast to MCH, is essentially related to the aging of the RBCs.

Extraction of target specimens from bioholographic images using interactive graph cuts

It is necessary to extract target specimens from bioholographic images for high-level analysis such as object identification, recognition, and tracking with the advent of application of digital holographic microscopy to transparent or semi-transparent biological specimens. We present an interactive graph cuts approach to segment the needed target specimens in the reconstructed bioholographic images. This method combines both regional and boundary information and is robust to extract targets with weak boundaries. Moreover, this technique can achieve globally optimal results while minimizing an energy function. We provide a convenient user interface, which can easily differentiate the foreground/background for various types of holographic images, as well as a dynamically modified coefficient, which specifies the importance of the regional and boundary information. The extracted results from our scheme have been compared with those from an advanced level-set-based segmentation method using an unbiased comparison algorithm. Experimental results show that this interactive graph cut technique can not only extract different kinds of target specimens in bioholographic images, but also yield good results when there are multiple similar objects in the holographic image or when the object boundaries are very weak.

Automated segmentation of multiple red blood cells with digital holographic microscopy

We present a method to automatically segment red blood cells (RBCs) visualized by digital holographic microscopy (DHM), which is based on the marker-controlled watershed algorithm. Quantitative phase images of RBCs can be obtained by using off-axis DHM along to provide some important information about each RBC, including size, shape, volume, hemoglobin content, etc. The most important process of segmentation based on marker-controlled watershed is to perform an accurate localization of internal and external markers. Here, we first obtain the binary image via Otsu algorithm. Then, we apply morphological operations to the binary image to get the internal markers. We then apply the distance transform algorithm combined with the watershed algorithm to generate external markers based on internal markers. Finally, combining the internal and external markers, we modify the original gradient image and apply the watershed algorithm. By appropriately identifying the internal and external markers, the problems of oversegmentation and undersegmentation are avoided. Furthermore, the internal and external parts of the RBCs phase image can also be segmented by using the marker-controlled watershed combined with our method, which can identify the internal and external markers appropriately. Our experimental results show that the proposed method achieves good performance in terms of segmenting RBCs and could thus be helpful when combined with an automated classification of RBCs.

Automatic calculation of tree diameter from stereoscopic image pairs using digital image processing

Automatic operations play an important role in societies by saving time and improving efficiency. In this paper, we apply the digital image processing method to the field of lumbering to automatically calculate tree diameters in order to reduce culler work and enable a third party to verify tree diameters. To calculate the cross-sectional diameter of a tree, the image was first segmented by the marker-controlled watershed transform algorithm based on the hue saturation intensity (HSI) color model. Then, the tree diameter was obtained by measuring the area of every isolated region in the segmented image. Finally, the true diameter was calculated by multiplying the diameter computed in the image and the scale, which was derived from the baseline and disparity of correspondence points from stereoscopic image pairs captured by rectified configuration cameras.

Photon-counting three-dimensional integral imaging with compression of elemental images

In this paper, we present lossless compression of elemental images in photon-counting integral imaging. In order to verify the performance of the compression method applied to low light level three-dimensional (3D) integral imaging, we compute the correlation coefficient and peak to mean square error (PSNR) as metrics for 3D scene reconstruction integrity. We show quantitatively via experiments that a considerable compression of the elemental images in photon-counting integral imaging may be achievable without significant loss in the performance in terms of correlation and PSNR metrics. To the best of our knowledge, this is the first report on applying lossless compression algorithms in photon-counting 3D computational integral imaging.

Automated statistical quantification of three-dimensional morphology and mean corpuscular hemoglobin of multiple red blood cells

In this paper, we present an automated approach to quantify information about three-dimensional (3D) morphology, hemoglobin content and density of mature red blood cells (RBCs) using off-axis digital holographic microscopy (DHM) and statistical algorithms. The digital hologram of RBCs is recorded by a CCD camera using an off-axis interferometry setup and quantitative phase images of RBCs are obtained by a numerical reconstruction algorithm. In order to remove unnecessary parts and obtain clear targets in the reconstructed phase image with many RBCs, the marker-controlled watershed segmentation algorithm is applied to the phase image. Each RBC in the segmented phase image is three-dimensionally investigated. Characteristic properties such as projected cell surface, average phase, sphericity coefficient, mean corpuscular hemoglobin (MCH) and MCH surface density of each RBC is quantitatively measured. We experimentally demonstrate that joint statistical distributions of the characteristic parameters of RBCs can be obtained by our algorithm and efficiently used as a feature pattern to discriminate between RBC populations that differ in shape and hemoglobin content. Our study opens the possibility of automated RBC quantitative analysis suitable for the rapid classification of a large number of RBCs from an individual blood specimen, which is a fundamental step to develop a diagnostic approach based on DHM.

Pharmacogenetics of the mycophenolic acid targets inosine monophosphate dehydrogenases IMPDH1 and IMPDH2: gene sequence variation and functional genomics

BACKGROUND AND PURPOSE

Inosine monophosphate dehydrogenases, encoded by IMPDH1 and IMPDH2, are targets for the important immunosuppressive drug, mycophenolic acid (MPA). Variation in MPA response may result, in part, from genetic variation in IMPDH1 and IMPDH2.

EXPERIMENTAL APPROACH

We resequenced IMPDH1 and IMPDH2 using DNA from 288 individuals from three ethnic groups and performed functional genomic studies of the sequence variants observed.

KEY RESULTS

We identified 73 single nucleotide polymorphisms (SNPs) in IMPDH1, 59 novel, and 25 SNPs, 24 novel, in IMPDH2. One novel IMPDH1 allozyme (Leu275) had 10.2% of the wild-type activity as a result of accelerated protein degradation. Decreased activity of the previously reported IMPDH2 Phe263 allozyme was primarily due to decreased protein quantity, also with accelerated degradation. These observations with regard to the functional implications of variant allozymes were supported by the IMPDH1 and IMPDH2 X-ray crystal structures. A novel IMPDH2 intron 1 SNP, G > C IVS1(93), was associated with decreased mRNA quantity, possibly because of altered transcription.

CONCLUSIONS AND IMPLICATIONS

These results provide insight into the nature and extent of sequence variation in the IMPDH1 and IMPDH2 genes. They also describe the influence of gene sequence variation that alters the encoded amino acids on IMPDH function and provide a foundation for future translational studies designed to correlate sequence variation in these genes with outcomes in patients treated with MPA.

Automated three-dimensional microbial sensing and recognition using digital holography and statistical sampling

We overview an approach to providing automated three-dimensional (3D) sensing and recognition of biological micro/nanoorganisms integrating Gabor digital holographic microscopy and statistical sampling methods. For 3D data acquisition of biological specimens, a coherent beam propagates through the specimen and its transversely and longitudinally magnified diffraction pattern observed by the microscope objective is optically recorded with an image sensor array interfaced with a computer. 3D visualization of the biological specimen from the magnified diffraction pattern is accomplished by using the computational Fresnel propagation algorithm. For 3D recognition of the biological specimen, a watershed image segmentation algorithm is applied to automatically remove the unnecessary background parts in the reconstructed holographic image. Statistical estimation and inference algorithms are developed to the automatically segmented holographic image. Overviews of preliminary experimental results illustrate how the holographic image reconstructed from the Gabor digital hologram of biological specimen contains important information for microbial recognition.

Three dimensional imaging and recognition using truncated photon counting model and parametric maximum likelihood estimator

In this paper, a statistical approach is presented for three-dimensional (3D) visualization and recognition of objects having very small number of photons based on a parametric estimator. A truncated Poisson probability density function is assumed for modeling the distribution of small number of photons count observation. For 3D visualization and recognition of photon-limited objects, an integral imaging system is employed. We utilize virtual geometrical ray propagation for 3D reconstruction of objects. A maximum likelihood estimator (MLE) and statistical inference algorithms are applied to small number of photons counted elemental images captured with integral imaging. We have demonstrated that the MLE using a truncated Poisson model for estimating the average number of photon for each voxel of a photon starved 3D object has a small estimation error compared with the MLE using a Poisson model. Also, we present experiments to investigate the effect of 3D sensing parallax on the recognition performance under a fixed mean number of photons.

Three-dimensional speckle-noise reduction by using coherent integral imaging

We present a 3D imaging method to reduce speckle noise that exists in coherent imaging systems. This approach is based on integral imaging (II). The elemental images set having speckle-noise patterns of a 3D object is obtained by II technique under coherent illumination. The computational geometrical ray-propagation algorithm is applied to the elemental images in order to reconstruct the original 3D object. A uniform probability-density function is assumed for modeling the phase distribution of the speckle patterns. The statistical point estimator is used for 3D speckle removal. Speckle index is calculated to compare the computational reconstruction using the proposed method with that of conventional coherent image degraded by speckle patterns for 3D object reconstruction and by object recognition. Experimental results are presented. The speckle index, mean square error, and signal-to-noise ratio are used as performance metrics and are shown to have been significantly improved by the proposed method to reduce speckle noise in the 3D object reconstruction. 3D reconstruction experiments of objects with reduced speckle noise are presented. To the best of our knowledge, this is the first report on 3D speckle removal using II and statistical estimation algorithms.

Three-dimensional recognition of photon-starved events using computational integral imaging and statistical sampling

We present a statistical approach to recognize three-dimensional (3D) objects with a small number of photons captured by using integral imaging (II). For 3D recognition of the events, the photon-limited elemental image set of a 3D object is obtained using the II technique. A computational geometrical ray propagation algorithm and the parametric maximum likelihood estimator are applied to the photon-limited elemental image set to reconstruct the irradiance of the original 3D scene voxels. The sampling distributions for the statistical parameters of the reconstructed image are determined. Finally, hypothesis testing for the equality of the statistical parameters between reference and input data sets is performed for statistical classification of populations on the basis of sampling distribution information. It is shown that large data sets of photon-limited 3D images can be converted into sampling distributions with their own statistical parameters, resulting in a substantial data dimensionality reduction for processing.

3-D visualization and identification of biological microorganisms using partially temporal incoherent light in-line computational holographic imaging

We present a new method for three-dimensional (3-D) visualization and identification of biological microorganisms using partially temporal incoherent light in-line (PTILI) computational holographic imaging and multivariate statistical methods. For 3-D data acquisition of biological microorganisms, the band-pass filtered white light is used to illuminate a biological sample. The transversely and longitudinally diffracted pattern of the biological sample is magnified by microscope objective (MO) and is optically recorded with an image sensor array interfaced with a computer. Three-dimensional reconstruction of the biological sample from the diffraction pattern is accomplished by using computational Fresnel propagation method. Principal components analysis and nonparametric inference algorithms are applied to the 3-D complex amplitude biological sample for identification purposes. Experiments indicate that the proposed system can be useful for identifying biological microorganisms. To the best of our knowledge, this is the first report on using PTILI computational holographic microscopy for identification of biological microorganisms.

Three-dimensional visualization of objects in scattering medium by use of computational integral imaging

In this paper, we propose a method to three-dimensionally visualize objects in a scattering medium using integral imaging. Our approach is based on a particular use of the interference phenomenon between the ballistic photons getting through the scattering medium and the scattered photons being scattered by the medium. For three-dimensional (3D) sensing of the scattered objects, the synthetic aperture integral imaging system under coherent illumination records the scattered elemental images of the objects. Then, the computational geometrical ray propagation algorithm is applied to the scattered elemental images in order to eliminate the interference patterns between scattered and object beams. The original 3D information of the scattered objects is recovered by multiple imaging channels, each with a unique perspective of the object. We present both simulation and experimental results with virtual and real objects to demonstrate the proposed concepts.

Three-dimensional identification of stem cells by computational holographic imaging

We present an optical imaging system and mathematical algorithms for three-dimensional sensing and identification of stem cells. Data acquisition of stem cells is based on holographic microscopy in the Fresnel domain by illuminating the cells with a laser. In this technique, the holograms of stem cells are optically recorded with an image sensor array interfaced with a computer and three-dimensional images of the stem cells are reconstructed from the Gabor-filtered digital holograms. The Gabor wavelet transformation for feature extraction of the digital hologram is performed to improve the process of identification. The inverse Fresnel transformation of the Gabor-filtered digital hologram is performed to reconstruct the multi-scale three-dimensional images of the stem cells at different depths along the longitudinal direction. For recognition and classification of stem cells, a statistical approach using an empirical cumulative density function is introduced. The experiments indicate that the proposed system can be potentially useful for recognizing and classifying stem cells. To the best of our knowledge, this is the first report on using three-dimensional holographic microscopy for automated identification of stem cells.

Three-dimensional identification of biological microorganism using integral imaging

In this paper, we address the identification of biological microorganisms using microscopic integral imaging (II). II senses multi-view directional information of 3D objects illuminated by incoherent light. A micro-lenslet array generates a set of elemental images by projecting a 3D scene onto a detector array. In computational reconstruction of II, 3D volumetric scenes are numerically reconstructed by means of a geometrical ray projection method. The identification of the biological samples is performed using the 3D volume of the reconstructed object. In one approach, the multivariate statistical distribution of the reference sample is measured in 3D space and compared with an unknown input sample by means of statistical discriminant functions. The multivariate empirical cumulative density of the 3D volume image is determined for classification. On the other approach, the graph matching technique is applied to 3D volumetric images with Gabor feature extraction. The reference morphology is identified in unknown input samples using 3D grids. Experimental results are presented for the identification of sphacelaria alga and tribonema aequale alga. We present experimental results for both 3D and 2D imaging. To the best of our knowledge, this is the first report on 3D identification of microorganisms using II.