Perspectives using probe-based confocal laser endomicroscopy of the respiratory tract

Clustering-Guided Twin Contrastive Learning for Endomicroscopy Image Classification

Learning better representations is essential in medical image analysis for computer-aided diagnosis. However, learning discriminative semantic features is a major challenge due to the lack of large-scale well-annotated datasets. Thus, how can we learn a well-structured categorizable embedding space in limited-scale and unlabeled datasets? In this paper, we proposed a novel clustering-guided twin-contrastive learning framework (CTCL) that learns the discriminative representations of probe-based confocal laser endomicroscopy (pCLE) images for gastrointestinal (GI) tumor classification. Compared with traditional contrastive learning, in which only two randomly augmented views of the same instance are considered, the proposed CTCL aligns more semantically related and class-consistent samples by clustering, which improved intra-class tightness and inter-class variability to produce more informative representations. Furthermore, based on the inherent properties of CLE (geometric invariance and intrinsic noise), we proposed to regard CLE images with any angle rotation and CLE images with different noises as the same instance, respectively, for increased variability and diversity of samples. By optimizing CTCL in an end-to-end expectation-maximization framework, comprehensive experimental results demonstrated that CTCL-based visual representations achieved competitive performance on each downstream task as well as more robustness and transferability compared with existing state-of-the-art SSL and supervised methods. Notably, CTCL achieved 75.60%/78.45% and 64.12%/77.37% top-1 accuracy on the linear evaluation protocol and few-shot classification downstream tasks, respectively, which outperformed the previous best results by 1.27%/1.63% and 0.5%/3%, respectively. The proposed method holds great potential to assist pathologists in achieving an automated, fast, and high-precision diagnosis of GI tumors and accurately determining different stages of tumor development based on CLE images.

The cryobiopsy in interstitial lung diseases guided by probe-based confocal laser endomicroscopy is feasible

BACKGROUND

Transbronchial lung cryobiopsy (TBLB) is routinely used to diagnose the interstitial lung disease (ILD). These results are consistent with those of surgical lung biopsy. Fluoroscopy is also used to confirm the final position of the cryoprobe; however, it can increase radiation exposure for both patients and medical care personnel. Probe-based confocal laser endomicroscopy (pCLE) is a novel optical imaging technique that allows real-time imaging at the cellular level in vivo. pCLE technology can also be used to identify malignancy, acute rejection in lung transplantation, amiodarone lung, and pulmonary alveolar proteinosis and visualize elastin fibres in the alveolar compartment.

OBJECTIVES

The aim of this study is to investigate the ability of pCLE to distinguish fibrotic pulmonary issues from normal lung disease and the safety and feasibility of CLE-guided bronchoscopy and transbronchial lung cryobiopsy (TBLC) in patients with interstitial lung disease (ILD).

METHODS

pCLE images from 17 ILD patients were obtained during TBLB. These images were then compared with histology results to assess the correspondence rate.

RESULTS

pCLE imaging of the alveolar structures was performed. Key characteristics were visible, which could potentially influence the diagnostic rate (fibrotic areas) and the complication rate (blood vessel and pleura).

CONCLUSION

pCLE may reduce complications and increase the diagnostic yield. It is a potential guidance tool for cryobiopsy in the patients with ILD without fluoroscopy.

Context-aware dynamic filtering network for confocal laser endomicroscopy image denoising

Objective.As an emerging diagnosis technology for gastrointestinal diseases, confocal laser endomicroscopy (CLE) is limited by the physical structure of the fiber bundle, leading to the inevitable production of various forms of noise during the imaging process. However, existing denoising methods based on hand-crafted features inefficiently deal with realistic noise in CLE images. To alleviate this challenge, we proposed context-aware kernel estimation and multi-scale dynamic fusion modules to remove realistic noise in CLE images, including multiplicative and additive white noise.Approach.Specifically, a realistic noise statistics model with random noise specific to CLE data is constructed and further used to develop a self-supervised denoised model without the participation of clean images. Secondly, context-aware kernel estimation, which improves the representation of features by similar learnable region weights, addresses the problem of the non-uniform distribution of noises in CLE images and proposes a lightweight denoised model (CLENet). Thirdly, we have developed a multi-scale dynamic fusion module that decouples and recalibrates features, providing a precise and contextually enriched representation of features. Finally, we integrated two developed modules into a U-shaped backbone to build an efficient denoising network named U-CLENet.Main Results.Both proposed methods achieve comparable or better performance with low computational complexity on two gastrointestinal disease CLE image datasets using the same training benchmark.Significance.The proposed approaches improve the visual quality of unclear CLE images for various stages of tumor development, helping to reduce the rate of misdiagnosis in clinical decision-making and achieve computer graphics-assisted diagnosis.

Boosting few-shot confocal endomicroscopy image recognition with feature-level MixSiam

As an emerging early diagnostic technology for gastrointestinal diseases, confocal laser endomicroscopy lacks large-scale perfect annotated data, leading to a major challenge in learning discriminative semantic features. So, how should we learn representations without labels or a few labels? In this paper, we proposed a feature-level MixSiam method based on the traditional Siamese network that learns the discriminative features of probe-based confocal laser endomicroscopy (pCLE) images for gastrointestinal (GI) tumor classification. The proposed method is divided into two stages: self-supervised learning (SSL) and few-shot learning (FS). First, in the self-supervised learning stage, the novel feature-level-based feature mixing approach introduced more task-relevant information via regularization, facilitating the traditional Siamese structure can adapt to the large intra-class variance of the pCLE dataset. Then, in the few-shot learning stage, we adopted the pre-trained model obtained through self-supervised learning as the base learner in the few-shot learning pipeline, enabling the feature extractor to learn richer and more transferable visual representations for rapid generalization to other pCLE classification tasks when labeled data are limited. On two disjoint pCLE gastrointestinal image datasets, the proposed method is evaluated. With the linear evaluation protocol, feature-level MixSiam outperforms the baseline by 6% (Top-1) and the supervised model by 2% (Top1), which demonstrates the effectiveness of the proposed feature-level-based feature mixing method. Furthermore, the proposed method outperforms the previous baseline method for the few-shot classification task, which can help improve the classification of pCLE images lacking large-scale annotated data for different stages of tumor development.

Deep Graph-Based Multimodal Feature Embedding for Endomicroscopy Image Retrieval

Representation learning is a critical task for medical image analysis in computer-aided diagnosis. However, it is challenging to learn discriminative features due to the limited size of the data set and the lack of labels. In this article, we propose a deep graph-based multimodal feature embedding (DGMFE) framework for medical image retrieval with application to breast tissue classification by learning discriminative features of probe-based confocal laser endomicroscopy (pCLE). We first build a multimodality graph model based on the visual similarity between pCLE data and reference histology images. The latent similar pCLE-histology pairs are extracted by walking with the cyclic path on the graph while the dissimilar pairs are extracted based on the geodesic distance. Given the similar and dissimilar pairs, the latent feature space is discovered by reconstructing the similarity between pCLE and histology images via deep Siamese neural networks. The proposed method is evaluated on a clinical database with 700 pCLE mosaics. The accuracy of image retrieval demonstrates that DGMFE can outperform previous works on feature learning. Especially, the top-1 accuracy in an eight-class retrieval task is 0.739, thus demonstrating a 10% improvement compared to the state-of-the-art method.

From Mouse to Man and Back: Closing the Correlation Gap between Imaging and Histopathology for Lung Diseases

Lung diseases such as fibrosis, asthma, cystic fibrosis, infection and cancer are life-threatening conditions that slowly deteriorate quality of life and for which our diagnostic power is high, but our knowledge on etiology and/or effective treatment options still contains important gaps. In the context of day-to-day practice, clinical and preclinical studies, clinicians and basic researchers team up and continuously strive to increase insights into lung disease progression, diagnostic and treatment options. To unravel disease processes and to test novel therapeutic approaches, investigators typically rely on end-stage procedures such as serum analysis, cyto-/chemokine profiles and selective tissue histology from animal models. These techniques are useful but provide only a snapshot of disease processes that are essentially dynamic in time and space. Technology allowing evaluation of live animals repeatedly is indispensable to gain a better insight into the dynamics of lung disease progression and treatment effects. Computed tomography (CT) is a clinical diagnostic imaging technique that can have enormous benefits in a research context too. Yet, the implementation of imaging techniques in laboratories lags behind. In this review we want to showcase the integrated approaches and novel developments in imaging, lung functional testing and pathological techniques that are used to assess, diagnose, quantify and treat lung disease and that may be employed in research on patients and animals. Imaging approaches result in often novel anatomical and functional biomarkers, resulting in many advantages, such as better insight in disease progression and a reduction in the numbers of animals necessary. We here showcase integrated assessment of lung disease with imaging and histopathological technologies, applied to the example of lung fibrosis. Better integration of clinical and preclinical imaging technologies with pathology will ultimately result in improved clinical translation of (therapy) study results.

Image computing for fibre-bundle endomicroscopy: A review

Endomicroscopy is an emerging imaging modality, that facilitates the acquisition of in vivo, in situ optical biopsies, assisting diagnostic and potentially therapeutic interventions. While there is a diverse and constantly expanding range of commercial and experimental optical biopsy platforms available, fibre-bundle endomicroscopy is currently the most widely used platform and is approved for clinical use in a range of clinical indications. Miniaturised, flexible fibre-bundles, guided through the working channel of endoscopes, needles and catheters, enable high-resolution imaging across a variety of organ systems. Yet, the nature of image acquisition though a fibre-bundle gives rise to several inherent characteristics and limitations necessitating novel and effective image pre- and post-processing algorithms, ranging from image formation, enhancement and mosaicing to pathology detection and quantification. This paper introduces the underlying technology and most prevalent clinical applications of fibre-bundle endomicroscopy, and provides a comprehensive, up-to-date, review of relevant image reconstruction, analysis and understanding/inference methodologies. Furthermore, current limitations as well as future challenges and opportunities in fibre-bundle endomicroscopy computing are identified and discussed.

Advances in Optical Coherence Tomography and Confocal Laser Endomicroscopy in Pulmonary Diseases

Diagnosing and monitoring pulmonary diseases is highly dependent on imaging, physiological function tests and tissue sampling. Optical coherence tomography (OCT) and confocal laser endomicroscopy (CLE) are novel imaging techniques with near-microscopic resolution that can be easily and safely combined with conventional bronchoscopy. Disease-related pulmonary anatomical compartments can be visualized, real time, using these techniques. In obstructive lung diseases, airway wall layers and related structural remodelling can be identified and quantified. In malignant lung disease, normal and malignant areas of the central airways, lung parenchyma, lymph nodes and pleura can be discriminated. A growing number of interstitial lung diseases (ILDs) have been visualized using OCT or CLE. Several ILD-associated structural changes can be imaged: fibrosis, cellular infiltration, bronchi(ol)ectasis, cysts and microscopic honeycombing. Although not yet implemented in clinical practice, OCT and CLE have the potential to improve detection and monitoring pulmonary diseases and can contribute in unravelling the pathophysiology of disease and mechanism of action of novel treatments. Indeed, assessment of the airway wall layers with OCT might be helpful when evaluating treatments targeting airway remodelling. By visualizing individual malignant cells, CLE has the potential as a real-time lung cancer detection tool. In the future, both techniques could be combined with laser-enhanced fluorescent-labelled tracer detection. This review discusses the value of OCT and CLE in pulmonary medicine by summarizing the current evidence and elaborating on future perspectives.

Bronchoscopy for the diagnosis of peripheral lung lesions

Peripheral pulmonary lesions (PPLs) are generally considered as lesions in the peripheral one-third of the lung although a precise definition and radiographic anatomical landmarks separating central and peripheral lesion does not yet exist. The radiographic detection of such lesions has increased significantly with the adoption of lung cancer screening programs. These lesions are not directly visible by regular flexible bronchoscopes as they are usually distal to the lobar and segmental bronchi. Traditionally, depending on location and clinical stage at presentation, these lesions were typically sampled by computerized tomography (CT) guided needle or surgical biopsy although some centers also used ultrasound and fluoroscopy guided percutaneous needle biopsy. Due to lack of direct visualization, the yield for bronchoscopic guided sampling especially of the small <2 cm pulmonary nodules was very low. Therefore, sampling has been preferentially performed by percutaneous CT guidance, which had high yield of above 90% but it comes at the cost of higher risk complications like pneumothorax with reported rate of 15% to 28%. Directly proceeding to surgical resection is also considered in appropriate candidates with high suspicion of malignancy without any evidence of distant metastasis but the proportion of such cases of lung cancer is low. The manuscript discussed the various bronchoscopic diagnostic modalities for peripheral pulmonary lesions. It is important to note that most of the studies in this field are relatively small, not randomized, suffer from selection bias, have considerable heterogeneity in sampling methodology/instruments and usually have been performed in high volume institutions by dedicated highly experienced proceduralists. The prevalence of malignancy in most of the reported cohorts has also been high which may result in higher diagnostic yields. All these factors need to be kept in mind before generalizing the results to individual centers and practices.

Fibred confocal fluorescence microscopy in the diagnosis of interstitial lung diseases

BACKGROUND

Accurate diagnosis is critical to both therapeutic decisions and prognostication in interstitial lung diseases (ILD). However, surgical lung biopsies carry high complication rates. Fibred confocal fluorescence microscopy (FCFM) offers an alternative as it can visualize lung tissue in vivo at the cellular level with minimal adverse events. We wanted to investigate the diagnostic utility, and safety of using FCFM for patients with ILD.

METHODS

In patients with suspected ILD, FCFM images were obtained from multiple bronchopulmonary segments using a miniprobe inserted through the working channel of a flexible bronchoscope. The procedure was performed under moderate sedation in an outpatient setting. Morphometric measurements and fibre pattern analyses were co-related with computed tomography (CT) findings and patients' final diagnoses based on multi-disciplinary consensus.

RESULTS

One hundred and eighty four segments were imaged in 27 patients (18 males) with a median age of 67 years (range, 24-79 years). They were grouped into chronic fibrosing interstitial pneumonia (16 patients) and other ILDs. Six distinct FCFM patterns were observed: normal, increased fibres, densely packed fibres, hypercellular, thickened fibres and others/non-specific. The pattern resembling densely packed fibres was seen in at least one segment in 68.8% patients with chronic fibrosing interstitial pneumonia, but only 36.4% in other ILD (P=0.097). An association between inflammatory patterns on CT and a hypercellular pattern on FCFM was also found (P<0.001).

CONCLUSIONS

Our study shows the potential of FCFM in classifying ILD, but its role in further diagnosis remains limited.

Progress of research and clinical application of probe-based confocal laser endomicroscopy

Probe based confocal laser endomicroscopy (pCLE) is a new endoscopic diagnostic method which integrates micro-imaging technology into endoscopy and can identify the morphological structure of particular tissue surface, so that the ongoing procedure of endoscopy can make the histopathological diagnosis. pCLE has the "optical biopsy" function, avoids shortcomings in physical biopsies such as being time-consuming, cumbersome, and prone to sampling error, and thus improves the diagnostic accuracy and efficiency. The field of research of pCLE as a new endoscopic imaging technology has been expanded from the digestive cavity structure to the respiratory system and urinary system. This article reviews the progress of research and clinical application of pCLE.

Self-illuminating quantum dots for non-invasive bioluminescence imaging of mammalian gametes

Background

The fertility performance of animals is still a mystery and the full comprehension of mammalian gametes maturation and early embryonic development remains to be elucidated. The recent development in nanotechnology offers a new opportunity for real-time study of reproductive cells in their physiological environments. As a first step toward that goal, we evaluated the effectiveness of a fluorescent and luminescent nanoparticle for in vitro and ex vivo imaging of porcine gametes.

Methods

Freshly harvested boar sperm were labeled with red-shifted (655 nm) quantum dot nanoparticles conjugated (QD+) or not (QD−) with plasminogen antibody and evaluated. Subsets of labeled spermatozoa were loaded into straws and placed within the lumen of gilt reproductive tracts for ex vivo intra-uterine imaging. Porcine cumulus-oocyte complexes (COCs) were matured in the presence of QD− or QD+. Ovarian follicles were microinjected with QD− or QD+ and placed in culture for up to 4 days. After labeling, all samples were supplemented with coelenterazine, the luciferase substrate, and immediately submitted to bioluminescence analysis, followed by fluorescence and hyperspectral imaging. Data were analyzed with ANOVA and P < 0.05 indicated significant differences.

Results

All labeled-samples revealed bioluminescence emission that was confirmed by fluorescence and hyperspectral imaging of the QD localization within the cells and tissues. Over 76% of spermatozoa and both immature and mature COCs were successfully labeled with QD− or QD+. The QD− fluorescence appeared homogenously distributed in the oocytes, while found in the entire sperm length with a higher accumulation within the mid-piece. Labeled-follicles exhibited a progressive migration of QD nanoparticles within the follicle wall during culture. In contrast, QD+ fluorescence signals appeared condensed and stronger in the follicle cells, sperm head, and sub-plasma membrane area of mature oocytes. Weaker QD+ signals were detected in the cumulus cells. Fluorescence and hyperspectral microscope imaging showed comparable intracellular QD localization. Ex-vivo intra-uterine bioluminescence imaging of labeled spermatozoa revealed stronger signals captured over the oviducts, with uterine body allowing the lowest signal detection.

Conclusion

Findings indicate that conjugated and non-conjugated fluorescent nanoparticles can be used for effective labeling of mammalian gametes for in vitro monitoring and potential in vivo targeted-imaging.

Electronic supplementary material

The online version of this article (doi:10.1186/s12951-015-0097-1) contains supplementary material, which is available to authorized users.