Net2Vis - A Visual Grammar for Automatically Generating Publication-Tailored CNN Architecture Visualizations

Dense neural network outperforms other machine learning models for scaling-up lichen cover maps in Eastern Canada

Lichen mapping is vital for caribou management plans and sustainable land conservation. Previous studies have used random forest, dense neural network, and convolutional neural network models for mapping lichen coverage. However, to date, it is not clear how these models rank in this task. In this study, these machine learning models were evaluated on their ability to predict lichen percent coverage in Sentinel-2 imagery in Québec and Labrador, Canada. The models were trained on 10-m resolution lichen coverage (%) maps created from 20 drone surveys collected in July 2019 and 2022. The dense neural network achieved a higher accuracy than the other two, with a reported mean absolute error of 5.2% and an R2 of 0.76. By comparison, the random forest model returned a mean absolute error of 5.5% (R2: 0.74) and the convolutional neural network had a mean absolute error of 5.3% (R2: 0.74). A regional lichen map was created using the trained dense neural network and a Sentinel-2 imagery mosaic. There was greater uncertainty on land covers that the model was not exposed to in training, such as mines and deep lakes. While the dense neural network requires more computational effort to train than a random forest model, the 5.9% performance gain in the test pixel comparison renders it the most suitable for lichen mapping. This study represents progress toward determining the appropriate methodology for generating accurate lichen maps from satellite imagery for caribou conservation and sustainable land management.

Quantification of pulmonary involvement in COVID-19 pneumonia: an upgrade of the LungQuant software for lung CT segmentation

Computed tomography (CT) scans are used to evaluate the severity of lung involvement in patients affected by COVID-19 pneumonia. Here, we present an improved version of the LungQuant automatic segmentation software (LungQuant v2), which implements a cascade of three deep neural networks (DNNs) to segment the lungs and the lung lesions associated with COVID-19 pneumonia. The first network (BB-net) defines a bounding box enclosing the lungs, the second one (U-net 1 ) outputs the mask of the lungs, and the final one (U-net 2 ) generates the mask of the COVID-19 lesions. With respect to the previous version (LungQuant v1), three main improvements are introduced: the BB-net, a new term in the loss function in the U-net for lesion segmentation and a post-processing procedure to separate the right and left lungs. The three DNNs were optimized, trained and tested on publicly available CT scans. We evaluated the system segmentation capability on an independent test set consisting of ten fully annotated CT scans, the COVID-19-CT-Seg benchmark dataset. The test performances are reported by means of the volumetric dice similarity coefficient (vDSC) and the surface dice similarity coefficient (sDSC) between the reference and the segmented objects. LungQuant v2 achieves a vDSC (sDSC) equal to 0.96 ± 0.01 (0.97 ± 0.01) and 0.69 ± 0.08 (0.83 ± 0.07) for the lung and lesion segmentations, respectively. The output of the segmentation software was then used to assess the percentage of infected lungs, obtaining a Mean Absolute Error (MAE) equal to 2%.

Prediction of the as Low as Diagnostically Acceptable CT Dose for Identification of the Inferior Alveolar Canal Using 3D Convolutional Neural Networks with Multi-Balancing Strategies

Ionizing radiation is necessary for diagnostic imaging and deciding the right radiation dose is extremely critical to obtain a decent quality image. However, increasing the dosage to improve the image quality has risks due to the potential harm from ionizing radiation. Thus, finding the optimal as low as diagnostically acceptable (ALADA) dosage is an open research problem that has yet to be tackled using artificial intelligence (AI) methods. This paper proposes a new multi-balancing 3D convolutional neural network methodology to build 3D multidetector computed tomography (MDCT) datasets and develop a 3D classifier model that can work properly with 3D CT scan images and balance itself over the heavy unbalanced multi-classes. The proposed models were exhaustively investigated through eighteen empirical experiments and three re-runs for clinical expert examination. As a result, it was possible to confirm that the proposed models improved the performance by an accuracy of 5% to 10% when compared to the baseline method. Furthermore, the resulting models were found to be consistent, and thus possibly applicable to different MDCT examinations and reconstruction techniques. The outcome of this paper can help radiologists to predict the suitability of CT dosages across different CT hardware devices and reconstruction algorithms. Moreover, the developed model is suitable for clinical application where the right dose needs to be predicted from numerous MDCT examinations using a certain MDCT device and reconstruction technique.

Quantitative FIB/SEM tomogram analysis of closed and open porosity of spheroidized graphite anode materials for LiBs applications

The electrochemical behaviour of rounded graphite particles as anode material in a lithium-ion battery strongly depends on the particle properties. The spheroidization process directly affects these properties, including the open porosity that determines the extent of direct contact between liquid electrolyte and carbon surface. Therefore, the quantification of the proportion between open and closed pores is of great interest. Here, we quantify the open and closed porosity of spheroidized porous graphite particles from FIB-SEM tomograms. Quantification is achieved based on two developments: (1) a new sample preparation strategy and (2) a newly developed image evaluation scheme based on neural networks. The sample preparation strategy involves embedding of many graphite powder particles in indium enabling the investigation of several graphite particles in one FIB/SEM tomogram with high stability and with high contrast between the conductive embedding material and porous graphite. A quantitative evaluation of closed and open porosity is achieved by machine learning in form of convolutional neural networks. The convolutional neural network is used to detect the bulk graphite and by further morphological operations, closed and open pores are identified. An error is determined by comparing automatically created quantifications with manual reference values. Our porosity data for two differently spheroidized graphite samples agree qualitatively well with corresponding results from nitrogen physisorption measurements. This approach may allow quantitative data evaluation from porous powders and support understanding of the correlation to the electrochemical behaviour in the lithium-ion battery.

Visual Analysis of Neural Architecture Spaces for Summarizing Design Principles

Recent advances in artificial intelligence largely benefit from better neural network architectures. These architectures are a product of a costly process of trial-and-error. To ease this process, we develop ArchExplorer, a visual analysis method for understanding a neural architecture space and summarizing design principles. The key idea behind our method is to make the architecture space explainable by exploiting structural distances between architectures. We formulate the pairwise distance calculation as solving an all-pairs shortest path problem. To improve efficiency, we decompose this problem into a set of single-source shortest path problems. The time complexity is reduced from O(kn²N) to O(knN). Architectures are hierarchically clustered according to the distances between them. A circle-packing-based architecture visualization has been developed to convey both the global relationships between clusters and local neighborhoods of the architectures in each cluster. Two case studies and a post-analysis are presented to demonstrate the effectiveness of ArchExplorer in summarizing design principles and selecting better-performing architectures.

Interpretable Classification of Tauopathies with a Convolutional Neural Network Pipeline Using Transfer Learning and Validation against Post-Mortem Clinical Cases of Alzheimer's Disease and Progressive Supranuclear Palsy

Neurodegenerative diseases, tauopathies, constitute a serious global health problem. The etiology of these diseases is unclear and an increase in their incidence has been projected in the next 30 years. Therefore, the study of the molecular mechanisms that might stop these neurodegenerative processes is very relevant. Classification of neurodegenerative diseases using Machine and Deep Learning algorithms has been widely studied for medical imaging such as Magnetic Resonance Imaging. However, post-mortem immunofluorescence imaging studies of the brains of patients have not yet been used for this purpose. These studies may represent a valuable tool for monitoring aberrant chemical changes or pathological post-translational modifications of the Tau polypeptide. We propose a Convolutional Neural Network pipeline for the classification of Tau pathology of Alzheimer's disease and Progressive Supranuclear Palsy by analyzing post-mortem immunofluorescence images with different Tau biomarkers performed with models generated with the architecture ResNet-IFT using Transfer Learning. These models' outputs were interpreted with interpretability algorithms such as Guided Grad-CAM and Occlusion Analysis. To determine the best classifier, four different architectures were tested. We demonstrated that our design was able to classify diseases with an accuracy of 98.41% on average whilst providing an interpretation concerning the proper classification involving different structural patterns in the immunoreactivity of the Tau protein in NFTs present in the brains of patients with Progressive Supranuclear Palsy and Alzheimer's disease.

Biometric recognition through gait analysis

The use of people recognition techniques has become critical in some areas. For instance, social or assistive robots carry out collaborative tasks in the robotics field. A robot must know who to work with to deal with such tasks. Using biometric patterns may replace identification cards or codes on access control to critical infrastructures. The usage of Red Green Blue Depth (RGBD) cameras is ubiquitous to solve people recognition. However, this sensor has some constraints, such as they demand high computational capabilities, require the users to face the sensor, or do not regard users' privacy. Furthermore, in the COVID-19 pandemic, masks hide a significant portion of the face. In this work, we present BRITTANY, a biometric recognition tool through gait analysis using Laser Imaging Detection and Ranging (LIDAR) data and a Convolutional Neural Network (CNN). A Proof of Concept (PoC) has been carried out in an indoor environment with five users to evaluate BRITTANY. A new CNN architecture is presented, allowing the classification of aggregated occupancy maps that represent the people's gait. This new architecture has been compared with LeNet-5 and AlexNet through the same datasets. The final system reports an accuracy of 88%.

Residual convolutional autoencoder combined with a non-negative matrix factorization to estimate fetal heart rate

The fetal heart rate (fHR) plays an important role in the determination of the good health of the fetus. Beside the traditional Doppler ultrasound technique, non-invasive fetal electrocardiography (fECG) has become an interesting alternative. However, extracting clean fECG from abdominal ECG (aECG) recordings is a challenging task due to the presence of the maternal ECG component and various noise sources. In this context, we propose a deep residual convolutional autoencoder network trained on synthetic aECG simulations followed by a transfer learning phase on real aECG recordings to extract the cleanest fECG. Afterwards, we propose to use a non-negative matrix factorization based approach on the obtained fECG to estimate the fHR. Our method is evaluated on three publicly available databases demonstrating that it can provide significant performance improvement against comparative methodologies. Clinical relevance- The presented method has the advantage of estimating the fetal heart rate from a single-channel abdominal electrocardiogram without prior knowledge on the noise sources nor the maternal R-peak locations.

Deep learning-based framework for motion-compensated image fusion in catheterization procedures

OBJECTIVE

Augmenting X-ray (XR) fluoroscopy with 3D anatomic overlays is an essential technique to improve the guidance of the catheterization procedures. Unfortunately, cardiac and respiratory motion compromises the augmented fluoroscopy. Motion compensation methods can be applied to update the overlay of a static model with regard to respiratory and cardiac motion. We investigate the feasibility of motion detection between two fluoroscopic frames by applying a convolutional neural network (CNN). Its integration in the existing open-source software framework 3D-XGuide is demonstrated, such extending its functionality to automatic motion detection and compensation.

METHODS

The CNN is trained on reference data generated from tracking of the rapid pacing catheter tip by applying template matching with normalized cross-correlation (CC). The developed CNN motion compensation model is packaged in a standalone web service, allowing for independent use via a REST API. For testing and demonstration purposes, we have extended the functionality of 3D-XGuide navigation framework by an additional motion compensation module, which uses the displacement predictions of the standalone CNN model service for motion compensation of the static 3D model overlay. We provide the source code on GitHub under BSD license.

RESULTS

The performance of the CNN motion compensation model was evaluated on a total of 1690 fluoroscopic image pairs from ten clinical datasets. The CNN model-based motion compensation method clearly overperformed the tracking of the rapid pacing catheter tip with CC with prediction frame rates suitable for live application in the clinical setting.

CONCLUSION

A novel CNN model-based method for automatic motion compensation during fusion of 3D anatomic models with XR fluoroscopy is introduced and its integration with a real software application demonstrated. Automatic motion extraction from 2D XR images using a CNN model appears as a substantial improvement for reliable augmentation during catheter interventions.

Body fat compartment determination by encoder-decoder convolutional neural network: application to amyotrophic lateral sclerosis

The objective of this study was to automate the discrimination and quantification of human abdominal body fat compartments into subcutaneous adipose tissue (SAT) and visceral adipose tissue (VAT) from T1-weighted MRI using encoder-decoder convolutional neural networks (CNN) and to apply the algorithm to a diseased patient sample, i.e., patients with amyotrophic lateral sclerosis (ALS). One-hundred-and-fifty-five participants (74 patients with ALS and 81 healthy controls) were split in training (50%), validation (6%), and test (44%) data. SAT and VAT volumes were determined by a novel automated CNN-based algorithm of U-Net like architecture in comparison with an established protocol with semi-automatic assessment as the reference. The dice coefficients between the CNN predicted masks and the reference segmentation were 0.87 ± 0.04 for SAT and 0.64 ± 0.17 for VAT in the control group and 0.87 ± 0.08 for SAT and 0.68 ± 0.15 for VAT in the ALS group. The significantly increased VAT/SAT ratio in the ALS group in comparison to controls confirmed the previous results. In summary, the CNN approach using CNN of U-Net architecture for automated segmentation of abdominal adipose tissue substantially facilitates data processing and offers the opportunity to automatically discriminate abdominal SAT and VAT compartments. Within the research field of neurodegenerative disorders with body composition alterations like ALS, the unbiased analysis of body fat components might pave the way for these parameters as a potential biological marker or a secondary read-out for clinical trials.

A Joint Model of Random Forest and Artificial Neural Network for the Diagnosis of Endometriosis

Endometriosis (EM), an estrogen-dependent inflammatory disease with unknown etiology, affects thousands of childbearing-age couples, and its early diagnosis is still very difficult. With the rapid development of sequencing technology in recent years, the accumulation of many sequencing data makes it possible to screen important diagnostic biomarkers from some EM-related genes. In this study, we utilized public datasets in the Gene Expression Omnibus (GEO) and Array-Express database and identified seven important differentially expressed genes (DEGs) (COMT, NAA16, CCDC22, EIF3E, AHI1, DMXL2, and CISD3) through the random forest classifier. Among these DEGs, AHI1, DMXL2, and CISD3 have never been reported to be associated with the pathogenesis of EMs. Our study indicated that these three genes might participate in the pathogenesis of EMs through oxidative stress, epithelial–mesenchymal transition (EMT) with the activation of the Notch signaling pathway, and mitochondrial homeostasis, respectively. Then, we put these seven DEGs into an artificial neural network to construct a novel diagnostic model for EMs and verified its diagnostic efficacy in two public datasets. Furthermore, these seven DEGs were included in 15 hub genes identified from the constructed protein–protein interaction (PPI) network, which confirmed the reliability of the diagnostic model. We hope the diagnostic model can provide novel sights into the understanding of the pathogenesis of EMs and contribute to the clinical diagnosis and treatment of EMs.