Segmentation, Splitting, and Classification of Overlapping Bacteria in Microscope Images for Automatic Bacterial Vaginosis Diagnosis

Meta-learning with implicit gradients in a few-shot setting for medical image segmentation

Widely used traditional supervised deep learning methods require a large number of training samples but often fail to generalize on unseen datasets. Therefore, a more general application of any trained model is quite limited for medical imaging for clinical practice. Using separately trained models for each unique lesion category or a unique patient population will require sufficiently large curated datasets, which is not practical to use in a real-world clinical set-up. Few-shot learning approaches can not only minimize the need for an enormous number of reliable ground truth labels that are labour-intensive and expensive, but can also be used to model on a dataset coming from a new population. To this end, we propose to exploit an optimization-based implicit model agnostic meta-learning (iMAML) algorithm under few-shot settings for medical image segmentation. Our approach can leverage the learned weights from diverse but small training samples to perform analysis on unseen datasets with high accuracy. We show that, unlike classical few-shot learning approaches, our method improves generalization capability. To our knowledge, this is the first work that exploits iMAML for medical image segmentation and explores the strength of the model on scenarios such as meta-training on unique and mixed instances of lesion datasets. Our quantitative results on publicly available skin and polyp datasets show that the proposed method outperforms the naive supervised baseline model and two recent few-shot segmentation approaches by large margins. In addition, our iMAML approach shows an improvement of 2%-4% in dice score compared to its counterpart MAML for most experiments.

A Data-Efficient Framework for the Identification of Vaginitis Based on Deep Learning

Vaginitis is a gynecological disease affecting the health of millions of women all over the world. The traditional diagnosis of vaginitis is based on manual microscopy, which is time-consuming and tedious. The deep learning method offers a fast and reliable solution for an automatic early diagnosis of vaginitis. However, deep neural networks require massive well-annotated data. Manual annotation of microscopic images is highly cost extensive because it not only is a time-consuming process but also needs highly trained people (doctors, pathologists, or technicians). Most existing active learning approaches are not applicable in microscopic images due to the nature of complex backgrounds and numerous formed elements. To address the problem of high cost of labeling microscopic images, we present a data-efficient framework for the identification of vaginitis based on transfer learning and active learning strategies. The proposed informative sample selection strategy selected the minimal training subset, and then the pretrained convolutional neural network (CNN) was fine-tuned on the selected subset. The experiment results show that the proposed pipeline can save 37.5% annotation cost while maintaining competitive performance. The proposed promising novel framework can significantly save the annotation cost and has the potential of extending widely to other microscopic imaging applications, such as blood microscopic image analysis.

AGMB-Transformer: Anatomy-Guided Multi-Branch Transformer Network for Automated Evaluation of Root Canal Therapy

Accurate evaluation of the treatment result on X-ray images is a significant and challenging step in root canal therapy since the incorrect interpretation of the therapy results will hamper timely follow-up which is crucial to the patients' treatment outcome. Nowadays, the evaluation is performed in a manual manner, which is time-consuming, subjective, and error-prone. In this paper, we aim to automate this process by leveraging the advances in computer vision and artificial intelligence, to provide an objective and accurate method for root canal therapy result assessment. A novel anatomy-guided multi-branch Transformer (AGMB-Transformer) network is proposed, which first extracts a set of anatomy features and then uses them to guide a multi-branch Transformer network for evaluation. Specifically, we design a polynomial curve fitting segmentation strategy with the help of landmark detection to extract the anatomy features. Moreover, a branch fusion module and a multi-branch structure including our progressive Transformer and Group Multi-Head Self-Attention (GMHSA) are designed to focus on both global and local features for an accurate diagnosis. To facilitate the research, we have collected a large-scale root canal therapy evaluation dataset with 245 root canal therapy X-ray images, and the experiment results show that our AGMB-Transformer can improve the diagnosis accuracy from 57.96% to 90.20% compared with the baseline network. The proposed AGMB-Transformer can achieve a highly accurate evaluation of root canal therapy. To our best knowledge, our work is the first to perform automatic root canal therapy evaluation and has important clinical value to reduce the workload of endodontists.

Graph-Theoretic Post-Processing of Segmentation With Application to Dense Biofilms

Recent deep learning methods have provided successful initial segmentation results for generalized cell segmentation in microscopy. However, for dense arrangements of small cells with limited ground truth for training, the deep learning methods produce both over-segmentation and under-segmentation errors. Post-processing attempts to balance the trade-off between the global goal of cell counting for instance segmentation, and local fidelity to the morphology of identified cells. The need for post-processing is especially evident for segmenting 3D bacterial cells in densely-packed communities called biofilms. A graph-based recursive clustering approach, m-LCuts, is proposed to automatically detect collinearly structured clusters and applied to post-process unsolved cells in 3D bacterial biofilm segmentation. Construction of outlier-removed graphs to extract the collinearity feature in the data adds additional novelty to m-LCuts. The superiority of m-LCuts is observed by the evaluation in cell counting with over 90% of cells correctly identified, while a lower bound of 0.8 in terms of average single-cell segmentation accuracy is maintained. This proposed method does not need manual specification of the number of cells to be segmented. Furthermore, the broad adaptation for working on various applications, with the presence of data collinearity, also makes m-LCuts stand out from the other approaches.

In silico trio-biomarkers for bacterial vaginosis revealed by species dominance network analysis

BV (bacterial vaginosis) influences 20%–40% of women but its etiology is still poorly understood. An open question about the BV is which of the hundreds of bacteria found in the human vaginal microbiome (HVM) are the major force driving the vaginal microbiota dysbiosis. Here, we recast the question of microbial causality of BV by asking if there are any prevalent ‘signatures’ (network motifs) in the vaginal microbiome networks associated with it? We apply a new framework [species dominance network analysis by Ma & Ellison (2019): Ecological Monographs) to detect critical structures in HVM networks associated with BV risks and etiology. We reanalyzed the 16 s-rRNA gene sequencing datasets of a mixed-cohort of 25 BV patients and healthy women. In these datasets, we detected 15 trio-motifs that occurred exclusively in BV patients. We failed to find any of these 15 trio-motifs in three additional cohorts of 1535 healthy women. Most member-species of the 15 trio motifs are BV-associated anaerobic bacteria (BVAB), Ravel’s community-state type indicators, or the most dominant species; virtually all species interactions in these trios are high-salience skeletons, suggesting that those trios are strongly connected ‘cults’ associated with the occurrence of BV. The presence of the trio motifs unique to BV may act as indicators for its personalized diagnosis and could help elucidate a more mechanistic interpretation of its risks and etiology. We caution that scarcity of large longitudinal datasets of HVM also limited further verifications of our findings, and these findings require further clinical tests to launch their applications.

Deep Neural Networks Offer Morphologic Classification and Diagnosis of Bacterial Vaginosis

Bacterial vaginosis (BV) is caused by the excessive and imbalanced growth of bacteria in vagina, affecting 30 to 50% of women. Gram staining followed by Nugent scoring based on bacterial morphotypes under the microscope is considered the gold standard for BV diagnosis; this method is often labor-intensive and time-consuming, and results vary from person to person. We developed and optimized a convolutional neural network (CNN) model and evaluated its ability to automatically identify and classify three categories of Nugent scores from microscope images. The CNN model was first established with a panel of microscopic images with Nugent scores determined by experts. The model was trained by minimizing the cross-entropy loss function and optimized by using a momentum optimizer. The separate test sets of images collected from three hospitals were evaluated by the CNN model. The CNN model consisted of 25 convolutional layers, 2 pooling layers, and a fully connected layer. The model obtained 82.4% sensitivity and 96.6% specificity with the 5,815 validation images when altered vaginal flora and BV were considered the positive samples, which was better than the rates achieved by top-level technologists and obstetricians in China. The capability of our model for generalization was so strong that it exhibited 75.1% accuracy in three categories of Nugent scores on the independent test set of 1,082 images, which was 6.6% higher than the average of three technologists, who are hold bachelor's degrees in medicine and are qualified to make diagnostic decisions. When three technologists ran one specimen in triplicate, the precision of three categories of Nugent scores was 54.0%. One hundred three samples diagnosed by two technologists on different days showed a repeatability of 90.3%. The CNN model outperformed human health care practitioners in terms of accuracy and stability for three categories of Nugent score diagnosis. The deep learning model may offer translational applications in automating diagnosis of bacterial vaginosis with proper supporting hardware.

A Label-Free Optical Detection of Pathogens in Isopropanol as a First Step towards Real-Time Infection Prevention

The detection of pathogens is a major public health issue. Every year, thousands of people die because of nosocomial infections. It is therefore important to be able to detect possible outbreaks as early as possible, especially in the hospital environment. Various pathogen detection techniques have already been demonstrated. However, most of them require expensive and specific equipment, and/or complex protocols, which, most of the time, involve biochemical reaction and labelling steps. In this paper, a new method that combines microscopic imaging and machine learning is described. The main benefits of this approach are to be low-cost, label-free and easy to integrate in any suitable medical device, such as hand hygiene dispensers. The suitability of this pathogen detection method is validated using four bacteria, both in PBS (Phosphate Buffered Saline) and in isopropanol. In particular, we demonstrated an efficient pathogenic detection that is sensible to changes in the composition of a mixture of pathogens, even in alcohol-based solutions.

SSNOMBACTER: A collection of scattering-type scanning near-field optical microscopy and atomic force microscopy images of bacterial cells

BACKGROUND

In recent years, a variety of imaging techniques operating at nanoscale resolution have been reported. These techniques have the potential to enrich our understanding of bacterial species relevant to human health, such as antibiotic-resistant pathogens. However, owing to the novelty of these techniques, their use is still confined to addressing very particular applications, and their availability is limited owing to associated costs and required expertise. Among these, scattering-type scanning near field optical microscopy (s-SNOM) has been demonstrated as a powerful tool for exploring important optical properties at nanoscale resolution, depending only on the size of a sharp tip. Despite its huge potential to resolve aspects that cannot be tackled otherwise, the penetration of s-SNOM into the life sciences is still proceeding at a slow pace for the aforementioned reasons.

RESULTS

In this work we introduce SSNOMBACTER, a set of s-SNOM images collected on 15 bacterial species. These come accompanied by registered Atomic Force Microscopy images, which are useful for placing nanoscale optical information in a relevant topographic context.

CONCLUSIONS

The proposed dataset aims to augment the popularity of s-SNOM and for accelerating its penetration in life sciences. Furthermore, we consider this dataset to be useful for the development and benchmarking of image analysis tools dedicated to s-SNOM imaging, which are scarce, despite the high need. In this latter context we discuss a series of image processing and analysis applications where SSNOMBACTER could be of help.

Automatic Bacillus anthracis bacteria detection and segmentation in microscopic images using UNet+

Anthrax is one of the important diseases in humans and animals, caused by the gram-positive bacteria spores called Bacillus anthracis. The disease is still one of the health problems of developing countries. Due to fatigue and decreased visual acuity, microscopic diagnosis of diseases by humans may not be of good quality. In this paper, for the first time, a system for automatic and rapid diagnosis of anthrax disease simultaneously with detection and segmentation of B. anthracis bacteria in microscopic images has been proposed based on artificial intelligence and deep learning techniques. Two important architectures of deep neural networks including UNet and UNet++ have been used for detection and segmentation of the most important component of the image i.e. bacteria. Automated detection and segmentation of B. anthracis bacteria offers the same level of accuracy as the human diagnostic specialist and in some cases outperforms it. Experimental results show that these deep architectures especially UNet++ can be used effectively and efficiently to automate B. anthracis bacteria segmentation of microscopic images obtained under different conditions. UNet++ produces outstanding results despite the many challenges in this field, such as high image dimension, image artifacts, object crowding, and overlapping. We conducted our experiments on a dataset prepared privately and achieved an accuracy of 97% and the dice score of 0.96 on the patch test images. It also tested on whole raw images and a recall of 98% and accuracy of 97% is achieved, which shows excellent performance in the bacteria segmentation task. The low cost and high speed of diagnosis and no need for a specialist are other benefits of the proposed system.

Bacterial Vaginosis: Current Diagnostic Avenues and Future Opportunities

A healthy female genital tract harbors a microbiome dominated by lactic acid and hydrogen peroxide producing bacteria, which provide protection against infections by maintaining a low pH. Changes in the bacterial compositions of the vaginal microbiome can lead to bacterial vaginosis (BV), which is often associated with vaginal inflammation. Bacterial vaginosis increases the risk of acquiring sexually transmitted infections (STIs) like human immunodeficiency virus (HIV) and affects women's reproductive health negatively. In pregnant women, BV can lead to chorioamnionitis and adverse pregnancy outcomes, including preterm premature rupture of the membranes and preterm birth. In order to manage BV effectively, good diagnostic procedures are required. Traditionally clinical and microscopic methods have been used to diagnose BV; however, these methods require skilled staff and time and suffer from reduced sensitivity and specificity. New diagnostics, including highly sensitive and specific point-of-care (POC) tests, treatment modalities and vaccines can be developed based on the identification of biomarkers from the growing pool of vaginal microbiome and vaginal metabolome data. In this review the current and future diagnostic avenues will be discussed.

Image analysis and artificial intelligence in infectious disease diagnostics

BACKGROUND

Microbiologists are valued for their time-honed skills in image analysis, including identification of pathogens and inflammatory context in Gram stains, ova and parasite preparations, blood smears and histopathologic slides. They also must classify colony growth on a variety of agar plates for triage and assessment. Recent advances in image analysis, in particular application of artificial intelligence (AI), have the potential to automate these processes and support more timely and accurate diagnoses.

OBJECTIVES

To review current AI-based image analysis as applied to clinical microbiology; and to discuss future trends in the field.

SOURCES

Material sourced for this review included peer-reviewed literature annotated in the PubMed or Google Scholar databases and preprint articles from bioRxiv. Articles describing use of AI for analysis of images used in infectious disease diagnostics were reviewed.

CONTENT

We describe application of machine learning towards analysis of different types of microbiologic image data. Specifically, we outline progress in smear and plate interpretation as well as the potential for AI diagnostic applications in the clinical microbiology laboratory.

IMPLICATIONS

Combined with automation, we predict that AI algorithms will be used in the future to prescreen and preclassify image data, thereby increasing productivity and enabling more accurate diagnoses through collaboration between the AI and the microbiologist. Once developed, image-based AI analysis is inexpensive and amenable to local and remote diagnostic use.

Segmentation of Overlapping Cytoplasm in Cervical Smear Images via Adaptive Shape Priors Extracted From Contour Fragments

We present a novel approach for segmenting overlapping cytoplasm of cells in cervical smear images by leveraging the adaptive shape priors extracted from cytoplasm's contour fragments and shape statistics. The main challenge of this task is that many occluded boundaries in cytoplasm clumps are extremely difficult to be identified and, sometimes, even visually indistinguishable. Given a clump where multiple cytoplasms overlap, our method starts by cutting its contour into a set of contour fragments. We then locate the corresponding contour fragments of each cytoplasm by a grouping process. For each cytoplasm, according to the grouped fragments and a set of known shape references, we construct its shape and, then, connect the fragments to form a closed contour as the segmentation result, which is explicitly constrained by the constructed shape. We further integrate the intensity and curvature information, which is complementary to the shape priors extracted from contour fragments, into our framework to improve the segmentation accuracy. We propose to iteratively conduct fragments grouping, shape constructing, and fragments connecting for progressively refining the shape priors and improving the segmentation results. We extensively evaluate the effectiveness of our method on two typical cervical smear datasets. The experimental results demonstrate that our approach is highly effective and consistently outperforms the state-of-the-art approaches. The proposed method is general enough to be applied to other similar microscopic image segmentation tasks, where heavily overlapped objects exist.

Morphology-based classification of mycobacteria-infected macrophages with convolutional neural network: reveal EsxA-induced morphologic changes indistinguishable by naked eyes

EsxA is an essential virulence factor for Mycobacterium tuberculosis (Mtb) pathogenesis as well as an important biomarker for Mtb detection. In this study, we use light microscopy and deep learning-based image analysis to classify the morphologic changes of macrophages infected by Mycobacterium marinum (Mm), a surrogate model for Mtb. Macrophages were infected either with the mCherry-expressing Mm wild type strain (Mm(WT)), or a mutant strain with deletion of the esxA-esxB operon (Mm(ΔEsxA:B)). The mCherry serves as an infection marker to train the convolution neural network (CNN) and to validate the classification results. Data show that CNN can distinguish the Mm(WT)-infected cells from uninfected cells with an accuracy of 92.4% at 2 hours postinfection (hpi). However, the accuracy at 12 and 24 hpi is decreased to ∼75% and ∼83%, respectively, suggesting dynamic morphologic changes through different stages of infection. The accuracy of discriminating Mm(ΔEsxA:B)-infected cells from uninfected cells is lower than 80% at all time, which is consistent to attenuated virulence of Mm(ΔEsxA:B). Interestingly, CNN distinguishes Mm(WT)-infected cells from Mm(ΔEsxA:B)-infected cells with ∼90% accuracy, implicating EsxA induces unique morphologic changes in macrophages. Deconvolutional analysis successfully reconstructed the morphologic features used by CNN for classification, which are indistinguishable to naked eyes and distinct from intracellular mycobacteria. This study presents a deep learning-aided imaging analytical tool that can accurately detect virulent mycobacteria-infected macrophages by cellular morphologic changes. The observed morphologic changes induced by EsxA warrant further studies to fill the gap from molecular actions of bacterial virulence factors to cellular morphology.

An effective and accurate identification system of Mycobacterium tuberculosis using convolution neural networks

Tuberculosis (TB) remains the leading cause of morbidity and mortality from infectious disease in developing countries. The sputum smear microscopy remains the primary diagnostic laboratory test. However, microscopic examination is always time-consuming and tedious. Therefore, an effective computer-aided image identification system is needed to provide timely assistance in diagnosis. The current identification system usually suffers from complex color variations of the images, resulting in plentiful of false object detection. To overcome the dilemma, we propose a two-stage Mycobacterium tuberculosis identification system, consisting of candidate detection and classification using convolution neural networks (CNNs). The refined Faster region-based CNN was used to distinguish candidates of M. tuberculosis and the actual ones were classified by utilizing CNN-based classifier. We first compared three different CNNs, including ensemble CNN, single-member CNN, and deep CNN. The experimental results showed that both ensemble and deep CNNs were on par with similar identification performance when analyzing more than 19,000 images. A much better recall value was achieved by using our proposed system in comparison with conventional pixel-based support vector machine method for M. tuberculosis bacilli detection.

A Deep Convolutional Neural Network-Based Framework for Automatic Fetal Facial Standard Plane Recognition

Ultrasound imaging has become a prevalent examination method in prenatal diagnosis. Accurate acquisition of fetal facial standard plane (FFSP) is the most important precondition for subsequent diagnosis and measurement. In the past few years, considerable effort has been devoted to FFSP recognition using various hand-crafted features, but the recognition performance is still unsatisfactory due to the high intraclass variation of FFSPs and the high degree of visual similarity between FFSPs and other non-FFSPs. To improve the recognition performance, we propose a method to automatically recognize FFSP via a deep convolutional neural network (DCNN) architecture. The proposed DCNN consists of 16 convolutional layers with small 3 × 3 size kernels and three fully connected layers. A global average pooling is adopted in the last pooling layer to significantly reduce network parameters, which alleviates the overfitting problems and improves the performance under limited training data. Both the transfer learning strategy and a data augmentation technique tailored for FFSP are implemented to further boost the recognition performance. Extensive experiments demonstrate the advantage of our proposed method over traditional approaches and the effectiveness of DCNN to recognize FFSP for clinical diagnosis.