Convolutional neural networks: an overview and application in radiology

Predicting aggressive histopathological features in esophageal cancer with positron emission tomography using a deep convolutional neural network

Background

The presence of lymphovascular invasion (LVI) and perineural invasion (PNI) are of great prognostic importance in esophageal squamous cell carcinoma. Currently, positron emission tomography (PET) scans are the only means of functional assessment prior to treatment. We aimed to predict the presence of LVI and PNI in esophageal squamous cell carcinoma using PET imaging data by training a three-dimensional convolution neural network (3D-CNN).

Methods

Seven hundred and ninety-eight PET scans of patients with esophageal squamous cell carcinoma and 309 PET scans of patients with stage I lung cancer were collected. In the first part of this study, we built a 3D-CNN based on a residual network, ResNet, for a task to classify the scans into esophageal cancer or lung cancer. In the second stage, we collected the PET scans of 278 patients undergoing esophagectomy for a task to classify and predict the presence of LVI/PNI.

Results

In the first part, the model performance attained an area under the receiver operating characteristic curve (AUC) of 0.860. In the second part, we randomly split 80%, 10%, and 10% of our dataset into training set, validation set and testing set, respectively, for a task to classify the scans into the presence of LVI/PNI and evaluated the model performance on the testing set. Our 3D-CNN model attained an AUC of 0.668 in the testing set, which shows a better discriminative ability than random guessing.

Conclusions

A 3D-CNN can be trained, using PET imaging datasets, to predict LNV/PNI in esophageal cancer with acceptable accuracy.

NowDeepN: An Ensemble of Deep Learning Models for Weather Nowcasting Based on Radar Products’ Values Prediction

One of the hottest topics in today’s meteorological research is Weather nowcasting, which is the weather forecast for a short time period such as one to six hours. Radar is an important data source used by operational meteorologists for issuing nowcasting warnings. With the main goal of helping meteorologists in analysing radar data for issuing nowcasting warnings, we propose NowDeepN, a supervised learning based regression model which uses an ensemble of deep artificial neural networks for predicting the values for radar products at a certain time moment. The values predicted by NowDeepN may be used by meteorologists in estimating the future development of potential severe phenomena and would replace the time consuming process of extrapolating the radar echoes. NowDeepN is intended to be a proof of concept for the effectiveness of learning from radar data relevant patterns that would be useful for predicting future values for radar products based on their historical values. For assessing the performance of NowDeepN, a set of experiments on real radar data provided by the Romanian National Meteorological Administration is conducted. The impact of a data cleaning step introduced for correcting the erroneous radar products’ values is investigated both from the computational and meteorological perspectives. The experimental results also indicate the relevance of the features considered in the supervised learning task, highlighting that the radar products’ values at a certain geographical location at a time moment may be predicted from the products’ values from a neighboring area of that location at previous time moments. An overall Normalized Root Mean Squared Error less than 4% was obtained for NowDeepN on the cleaned radar data. Compared to similar related work from the nowcasting literature, NowDeepN outperforms several approaches and this emphasizes the performance of our proposal.

Artificial intelligence in radiology: relevance of collaborative work between radiologists and engineers for building a multidisciplinary team

The use of artificial intelligence (AI) algorithms in the field of radiology is becoming more common. Several studies have demonstrated the potential utility of machine learning (ML) and deep learning (DL) techniques as aids for radiologists to solve specific radiological challenges. The decision-making process, the establishment of specific clinical or radiological targets, the profile of the different professionals involved in the development of AI solutions, and the relation with partnerships and stakeholders are only some of the main issues that have to be faced and solved prior to starting the development of radiological AI solutions. Among all the players in this multidisciplinary team, the communication between radiologists and data scientists is essential for a successful collaborative work. There are specific skills that are inherent to radiological and medical training that are critical for identifying anatomical or clinical targets as well as for segmenting or labelling lesions. These skills would then have to be transferred, explained, and taught to the data science experts to facilitate their comprehension and integration into ML or DL algorithms. On the other hand, there is a wide range of complex software packages, deep neural-network architectures, and data transfer processes for which radiologists need the expertise of software engineers and data scientists in order to select the optimal manner to analyse and post-process this amount of data. This paper offers a summary of the top five challenges faced by radiologists and data scientists including tips and tricks to build a successful AI team.

Hybrid Deep Learning Model Based Indoor Positioning Using Wi-Fi RSSI Heat Maps for Autonomous Applications

Positioning using Wi-Fi received signal strength indication (RSSI) signals is an effective method for identifying the user positions in an indoor scenario. Wi-Fi RSSI signals in an autonomous system can be easily used for vehicle tracking in underground parking. In Wi-Fi RSSI signal based positioning, the positioning system estimates the signal strength of the access points (APs) to the receiver and identifies the user’s indoor positions. The existing Wi-Fi RSSI based positioning systems use raw RSSI signals obtained from APs and estimate the user positions. These raw RSSI signals can easily fluctuate and be interfered with by the indoor channel conditions. This signal interference in the indoor channel condition reduces localization performance of these existing Wi-Fi RSSI signal based positioning systems. To enhance their performance and reduce the positioning error, we propose a hybrid deep learning model (HDLM) based indoor positioning system. The proposed HDLM based positioning system uses RSSI heat maps instead of raw RSSI signals from APs. This results in better localization performance for Wi-Fi RSSI signal based positioning systems. When compared to the existing Wi-Fi RSSI based positioning technologies such as fingerprint, trilateration, and Wi-Fi fusion approaches, the proposed approach achieves reasonably better positioning results for indoor localization. The experiment results show that a combination of convolutional neural network and long short-term memory network (CNN-LSTM) used in the proposed HDLM outperforms other deep learning models and gives a smaller localization error than conventional Wi-Fi RSSI signal based localization approaches. From the experiment result analysis, the proposed system can be easily implemented for autonomous applications.

A New Multichannel Parallel Network Framework for the Special Structure of Multilead ECG

Electrocardiogram (ECG) contains the rhythmic features of continuous heartbeat and morphological features of ECG waveforms and varies among different diseases. Based on ECG signal features, we propose a combination of multiple neural networks, the multichannel parallel neural network (MLCNN-BiLSTM), to explore feature information contained in ECG. The MLCNN channel is used in extracting the morphological features of ECG waveforms. Compared with traditional convolutional neural network (CNN), the MLCNN can accurately extract strong relevant information on multilead ECG while ignoring irrelevant information. It is suitable for the special structures of multilead ECG. The Bidirectional Long Short-Term Memory (BiLSTM) channel is used in extracting the rhythmic features of ECG continuous heartbeat. Finally, by initializing the core threshold parameters and using the backpropagation algorithm to update automatically, the weighted fusion of the temporal-spatial features extracted from multiple channels in parallel is used in exploring the sensitivity of different cardiovascular diseases to morphological and rhythmic features. Experimental results show that the accuracy rate of multiple cardiovascular diseases is 87.81%, sensitivity is 86.00%, and specificity is 87.76%. We proposed the MLCNN-BiLSTM neural network that can be used as the first-round screening tool for clinical diagnosis of ECG.

The emerging role of deep learning in cytology

Deep learning (DL) is a component or subset of artificial intelligence. DL has contributed significant change in feature extraction and image classification. Various algorithmic models are used in DL such as a convolutional neural network (CNN), recurrent neural network, restricted Boltzmann machine, deep belief network and autoencoders. Of these, CNN is the most commonly used algorithm in the field of pathology for feature extraction and building neural network models. DL may be useful for tumour diagnosis, classification of the tumour and grading of the tumour in cytology. In this brief review, the basic concept of the DL and CNN are described. The application, prospects and challenges of the DL in the cytology are also discussed.

Automated Detection of Sleep Stages Using Deep Learning Techniques: A Systematic Review of the Last Decade (2010–2020)

Sleep is vital for one’s general well-being, but it is often neglected, which has led to an increase in sleep disorders worldwide. Indicators of sleep disorders, such as sleep interruptions, extreme daytime drowsiness, or snoring, can be detected with sleep analysis. However, sleep analysis relies on visuals conducted by experts, and is susceptible to inter- and intra-observer variabilities. One way to overcome these limitations is to support experts with a programmed diagnostic tool (PDT) based on artificial intelligence for timely detection of sleep disturbances. Artificial intelligence technology, such as deep learning (DL), ensures that data are fully utilized with low to no information loss during training. This paper provides a comprehensive review of 36 studies, published between March 2013 and August 2020, which employed DL models to analyze overnight polysomnogram (PSG) recordings for the classification of sleep stages. Our analysis shows that more than half of the studies employed convolutional neural networks (CNNs) on electroencephalography (EEG) recordings for sleep stage classification and achieved high performance. Our study also underscores that CNN models, particularly one-dimensional CNN models, are advantageous in yielding higher accuracies for classification. More importantly, we noticed that EEG alone is not sufficient to achieve robust classification results. Future automated detection systems should consider other PSG recordings, such as electroencephalogram (EEG), electrooculogram (EOG), and electromyogram (EMG) signals, along with input from human experts, to achieve the required sleep stage classification robustness. Hence, for DL methods to be fully realized as a practical PDT for sleep stage scoring in clinical applications, inclusion of other PSG recordings, besides EEG recordings, is necessary. In this respect, our report includes methods published in the last decade, underscoring the use of DL models with other PSG recordings, for scoring of sleep stages.

Automatic recognition of murmurs of ventricular septal defect using convolutional recurrent neural networks with temporal attentive pooling

Recognizing specific heart sound patterns is important for the diagnosis of structural heart diseases. However, the correct recognition of heart murmur depends largely on clinical experience. Accurately identifying abnormal heart sound patterns is challenging for young and inexperienced clinicians. This study is aimed at the development of a novel algorithm that can automatically recognize systolic murmurs in patients with ventricular septal defects (VSDs). Heart sounds from 51 subjects with VSDs and 25 subjects without a significant heart malformation were obtained in this study. Subsequently, the soundtracks were divided into different training and testing sets to establish the recognition system and evaluate the performance. The automatic murmur recognition system was based on a novel temporal attentive pooling-convolutional recurrent neural network (TAP-CRNN) model. On analyzing the performance using the test data that comprised 178 VSD heart sounds and 60 normal heart sounds, a sensitivity rate of 96.0% was obtained along with a specificity of 96.7%. When analyzing the heart sounds recorded in the second aortic and tricuspid areas, both the sensitivity and specificity were 100%. We demonstrated that the proposed TAP-CRNN system can accurately recognize the systolic murmurs of VSD patients, showing promising potential for the development of software for classifying the heart murmurs of several other structural heart diseases.

Deep learning and generative methods in cheminformatics and chemical biology: navigating small molecule space intelligently

The number of 'small' molecules that may be of interest to chemical biologists - chemical space - is enormous, but the fraction that have ever been made is tiny. Most strategies are discriminative, i.e. have involved 'forward' problems (have molecule, establish properties). However, we normally wish to solve the much harder generative or inverse problem (describe desired properties, find molecule). 'Deep' (machine) learning based on large-scale neural networks underpins technologies such as computer vision, natural language processing, driverless cars, and world-leading performance in games such as Go; it can also be applied to the solution of inverse problems in chemical biology. In particular, recent developments in deep learning admit the in silico generation of candidate molecular structures and the prediction of their properties, thereby allowing one to navigate (bio)chemical space intelligently. These methods are revolutionary but require an understanding of both (bio)chemistry and computer science to be exploited to best advantage. We give a high-level (non-mathematical) background to the deep learning revolution, and set out the crucial issue for chemical biology and informatics as a two-way mapping from the discrete nature of individual molecules to the continuous but high-dimensional latent representation that may best reflect chemical space. A variety of architectures can do this; we focus on a particular type known as variational autoencoders. We then provide some examples of recent successes of these kinds of approach, and a look towards the future.

Toward Smart Lockdown: A Novel Approach for COVID-19 Hotspots Prediction Using a Deep Hybrid Neural Network

COVID-19 caused the largest economic recession in the history by placing more than one third of world’s population in lockdown. The prolonged restrictions on economic and business activities caused huge economic turmoil that significantly affected the financial markets. To ease the growing pressure on the economy, scientists proposed intermittent lockdowns commonly known as “smart lockdowns”. Under smart lockdown, areas that contain infected clusters of population, namely hotspots, are placed on lockdown, while economic activities are allowed to operate in un-infected areas. In this study, we proposed a novel deep learning prediction framework for the accurate prediction of hotpots. We exploit the benefits of two deep learning models, i.e., Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) and propose a hybrid framework that has the ability to extract multi time-scale features from convolutional layers of CNN. The multi time-scale features are then concatenated and provide as input to 2-layers LSTM model. The LSTM model identifies short, medium and long-term dependencies by learning the representation of time-series data. We perform a series of experiments and compare the proposed framework with other state-of-the-art statistical and machine learning based prediction models. From the experimental results, we demonstrate that the proposed framework beats other existing methods with a clear margin.

Deep Learning CT Image Reconstruction in Clinical Practice

BACKGROUND

 Computed tomography (CT) is a central modality in modern radiology contributing to diagnostic medicine in almost every medical subspecialty, but particularly in emergency services. To solve the inverse problem of reconstructing anatomical slice images from the raw output the scanner measures, several methods have been developed, with filtered back projection (FBP) and iterative reconstruction (IR) subsequently providing criterion standards. Currently there are new approaches to reconstruction in the field of artificial intelligence utilizing the upcoming possibilities of machine learning (ML), or more specifically, deep learning (DL).

METHOD

 This review covers the principles of present CT image reconstruction as well as the basic concepts of DL and its implementation in reconstruction. Subsequently commercially available algorithms and current limitations are being discussed.

RESULTS AND CONCLUSION

 DL is an ML method that utilizes a trained artificial neural network to solve specific problems. Currently two vendors are providing DL image reconstruction algorithms for the clinical routine. For these algorithms, a decrease in image noise and an increase in overall image quality that could potentially facilitate the diagnostic confidence in lesion conspicuity or may translate to dose reduction for given clinical tasks have been shown. One study showed equal diagnostic accuracy in the detection of coronary artery stenosis for DL reconstructed images compared to IR at higher image quality levels. Consequently, a lot more research is necessary and should aim at diagnostic superiority in the clinical context covering a broadness of pathologies to demonstrate the reliability of such DL approaches.

KEY POINTS

  · Following iterative reconstruction, there is a new approach to CT image reconstruction in the clinical routine using deep learning (DL) as a method of artificial intelligence.. · DL image reconstruction algorithms decrease image noise, improve image quality, and have potential to reduce radiation dose.. · Diagnostic superiority in the clinical context should be demonstrated in future trials..

CITATION FORMAT

· Arndt C, Güttler F, Heinrich A et al. Deep Learning CT Image Reconstruction in Clinical Practice. Fortschr Röntgenstr 2021; 193: 252 - 261.

RGB Image Prioritization Using Convolutional Neural Network on a Microprocessor for Nanosatellites

Nanosatellites are being widely used in various missions, including remote sensing applications. However, the difficulty lies in mission operation due to downlink speed limitation in nanosatellites. Considering the global cloud fraction of 67%, retrieving clear images through the limited downlink capacity becomes a larger issue. In order to solve this problem, we propose an image prioritization method based on cloud coverage using CNN. The CNN is designed to be lightweight and to be able to prioritize RGB images for nanosatellite application. As previous CNNs are too heavy for onboard processing, new strategies are introduced to lighten the network. The input size is reduced, and patch decomposition is implemented for reduced memory usage. Replication padding is applied on the first block to suppress border ambiguity in the patches. The depth of the network is reduced for small input size adaptation, and the number of kernels is reduced to decrease the total number of parameters. Lastly, a multi-stream architecture is implemented to suppress the network from optimizing on color features. As a result, the number of parameters was reduced down to 0.4%, and the inference time was reduced down to 4.3% of the original network while maintaining approximately 70% precision. We expect that the proposed method will enhance the downlink capability of clear images in nanosatellites by 112%.

Automatic diagnosis for cysts and tumors of both jaws on panoramic radiographs using a deep convolution neural network

OBJECTIVES

The purpose of this study was to automatically diagnose odontogenic cysts and tumors of both jaws on panoramic radiographs using deep learning. We proposed a novel framework of deep convolution neural network (CNN) with data augmentation for detection and classification of the multiple diseases.

METHODS

We developed a deep CNN modified from YOLOv3 for detecting and classifying odontogenic cysts and tumors of both jaws. Our data set of 1282 panoramic radiographs comprised 350 dentigerous cysts (DCs), 302 periapical cysts (PCs), 300 odontogenic keratocysts (OKCs), 230 ameloblastomas (ABs), and 100 normal jaws with no disease. In addition, the number of radiographs was augmented 12-fold by flip, rotation, and intensity changes. We evaluated the classification performance of the developed CNN by calculating sensitivity, specificity, accuracy, and area under the curve (AUC) for diseases of both jaws.

RESULTS

The overall classification performance for the diseases improved from 78.2% sensitivity, 93.9% specificity,91.3% accuracy, and 0.86 AUC using the CNN with unaugmented data set to 88.9% sensitivity, 97.2% specificity, 95.6% accuracy, and 0.94 AUC using the CNN with augmented data set. CNN using augmented data set had the following sensitivities, specificities, accuracies, and AUCs: 91.4%, 99.2%, 97.8%, and 0.96 for DCs, 82.8%, 99.2%, 96.2%, and 0.92 for PCs, 98.4%,92.3%,94.0%, and 0.97 for OKCs, 71.7%, 100%, 94.3%, and 0.86 for ABs, and 100.0%, 95.1%, 96.0%, and 0.97 for normal jaws, respectively.

CONCLUSION

The CNN method we developed for automatically diagnosing odontogenic cysts and tumors of both jaws on panoramic radiographs using data augmentation showed high sensitivity, specificity, accuracy, and AUC despite the limited number of panoramic images involved.

Deep convolutional neural network recovers pure absorbance spectra from highly scatter-distorted spectra of cells

Infrared spectroscopy of cells and tissues is prone to Mie scattering distortions, which grossly obscure the relevant chemical signals. The state-of-the-art Mie extinction extended multiplicative signal correction (ME-EMSC) algorithm is a powerful tool for the recovery of pure absorbance spectra from highly scatter-distorted spectra. However, the algorithm is computationally expensive and the correction of large infrared imaging datasets requires weeks of computations. In this paper, we present a deep convolutional descattering autoencoder (DSAE) which was trained on a set of ME-EMSC corrected infrared spectra and which can massively reduce the computation time for scatter correction. Since the raw spectra showed large variability in chemical features, different reference spectra matching the chemical signals of the spectra were used to initialize the ME-EMSC algorithm, which is beneficial for the quality of the correction and the speed of the algorithm. One DSAE was trained on the spectra, which were corrected with different reference spectra and validated on independent test data. The DSAE outperformed the ME-EMSC correction in terms of speed, robustness, and noise levels. We confirm that the same chemical information is contained in the DSAE corrected spectra as in the spectra corrected with ME-EMSC.

One-dimensional convolutional neural network (1D-CNN) image reconstruction for electrical impedance tomography

In recent years, due to the strong autonomous learning ability of neural network algorithms, they have been applied for electrical impedance tomography (EIT). Although their imaging accuracy is greatly improved compared with traditional algorithms, generalization for both simulation and experimental data is required to be improved. According to the characteristics of voltage data collected in EIT, a one-dimensional convolutional neural network (1D-CNN) is proposed to solve the inverse problem of image reconstruction. Abundant samples are generated with numerical simulation to improve the edge-preservation of reconstructed images. The TensorFlow-graphics processing unit environment and Adam optimizer are used to train and optimize the network, respectively. The reconstruction results of the new network are compared with the Deep Neural Network (DNN) and 2D-CNN to prove the effectiveness and edge-preservation. The anti-noise and generalization capabilities of the new network are also validated. Furthermore, experiments with the EIT system are carried out to verify the practicability of the new network. The average image correlation coefficient of the new network increases 0.0320 and 0.0616 compared with the DNN and 2D-CNN, respectively, which demonstrates that the proposed method could give better reconstruction results, especially for the distribution of complex geometries.

Recognition of Drivers’ Activity Based on 1D Convolutional Neural Network

Background and objective: Driving a car is a complex activity which involves movements of the whole body. Many studies on drivers’ behavior are conducted to improve road traffic safety. Such studies involve the registration and processing of multiple signals, such as electroencephalography (EEG), electrooculography (EOG) and the images of the driver’s face. In our research, we attempt to develop a classifier of scenarios related to learning to drive based on the data obtained in real road traffic conditions via smart glasses. In our approach, we try to minimize the number of signals which can be used to recognize the activities performed while driving a car. Material and methods: We attempt to evaluate the drivers’ activities using both electrooculography (EOG) and a deep learning approach. To acquire data we used JINS MEME smart glasses furnished with 3-point EOG electrodes, 3-axial accelerometer and 3-axial gyroscope. Sensor data were acquired on 20 drivers (ten experienced and ten learner drivers) on the same 28.7 km route under real road conditions in southern Poland. The drivers performed several tasks while wearing the smart glasses and the tasks were linked to the signal during the drive. For the recognition of four activities (parking, driving through a roundabout, city traffic and driving through an intersection), we used one-dimensional convolutional neural network (1D CNN). Results: The maximum accuracy was 95.6% on validation set and 99.8% on training set. The results prove that the model based on 1D CNN can classify the actions performed by drivers accurately. Conclusions: We have proved the feasibility of recognizing drivers’ activity based solely on EOG data, regardless of the driving experience and style. Our findings may be useful in the objective assessment of driving skills and thus, improving driving safety.

Deep convolution neural networks based artifact suppression in under-sampled radial acquisitions of myocardial T 1 mapping images

We developed a deep convolutional neural network (CNN) based method to remove streaking artefact from accelerated radial acquisitions of myocardial T ₁-mapping images. A deep CNN based on a modified U-Net architecture was developed and trained to remove the streaking artefacts from under-sampled T ₁ mapping images. A total of 2090 T ₁-weighted images for 33 patients (55 ± 15 years, 19 males) and five healthy subjects (30 ± 14 years, 2 males) were used for training and testing the network. The images were acquired using radial slice interleaved T ₁ mapping sequence (STONE) and retrospectively under-sampled to achieve acceleration rate of 4 (corresponding to 48 spokes). The dataset was split into training and testing subsets with 23 subjects (60%) and 15 subjects (40%), respectively. For generating voxel-wise T ₁ maps, a two-parameter fitting model was used. Network performance was evaluated using normalized mean square error (NMSE), structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR) metrics. The proposed network allowed fast (<0.3 s/image) removal of the artefact from all T ₁-weighted testing images and the corresponding T ₁ maps with PSNR = 64.3 ± 1.02, NMSE = 0.2 ± 0.09 and SSIM = 0.9 ± 0.3 × 10^-4. There was no statistically significant difference between the measured T ₁ maps for both per-subject (reference: 1085 ± 37 ms, CNN: 1088 ± 37 ms, p = 0.4) and per-segment (reference: 1084 ± 48 ms, CNN: 1083 ± 58 ms, p = 0.9) analyses. In summary, deep CNN allows fast and reliable removal of streaking artefact from under-sampled radial T ₁ mapping images. Our results show that the highly non-linear operations of deep CNN processing of T ₁ mapping images do not impact accurate reconstruction of myocardial T ₁ maps.

Artificial Intelligence and Big Data

Medical images play an important role in ophthalmology and radiology. Medical image analysis has greatly benefited from the application of "deep learning" techniques in clinical and experimental radiology. Clinical applications and their relevance for radiological imaging in ophthalmology are presented.

The State of Radiology AI: Considerations for Purchase Decisions and Current Market Offerings

Purpose

To provide an overview of important factors to consider when purchasing radiology artificial intelligence (AI) software and current software offerings by type, subspecialty, and modality.

Materials and Methods

Important factors for consideration when purchasing AI software, including key decision makers, data ownership and privacy, cost structures, performance indicators, and potential return on investment are described. For the market overview, a list of radiology AI companies was aggregated from the Radiological Society of North America and the Society for Imaging Informatics in Medicine conferences (November 2016-June 2019), then narrowed to companies using deep learning for imaging analysis and diagnosis. Software created for image enhancement, reporting, or workflow management was excluded. Software was categorized by task (repetitive, quantitative, explorative, and diagnostic), modality, and subspecialty.

Results

A total of 119 software offerings from 55 companies were identified. There were 46 algorithms that currently have Food and Drug Administration and/or Conformité Européenne approval (as of November 2019). Of the 119 offerings, distribution of software targets was 34 of 70 (49%), 21 of 70 (30%), 14 of 70 (20%), and one of 70 (1%) for diagnostic, quantitative, repetitive, and explorative tasks, respectively. A plurality of companies are focused on nodule detection at chest CT and two-dimensional mammography. There is very little activity in certain subspecialties, including pediatrics and nuclear medicine. A comprehensive table is available on the website hitilab.org/pages/ai-companies.

Conclusion

The radiology AI marketplace is rapidly maturing, with an increase in product offerings. Radiologists and practice administrators should educate themselves on current product offerings and important factors to consider before purchase and implementation.© RSNA, 2020See also the invited commentary by Sala and Ursprung in this issue.

Landmark annotation and mandibular lateral deviation analysis of posteroanterior cephalograms using a convolutional neural network

Background/purpose

Facial asymmetry is relatively common in the general population. Here, we propose a fully automated annotation system that supports analysis of mandibular deviation and detection of facial asymmetry in posteroanterior (PA) cephalograms by means of a deep learning-based convolutional neural network (CNN) algorithm.

Materials and methods

In this retrospective study, 400 PA cephalograms were collected from the medical records of patients aged 4 years 2 months-80 years 3 months (mean age, 17 years 10 months; 255 female patients and 145 male patients). A deep CNN with two optimizers and a random forest algorithm were trained using 320 PA cephalograms; in these images, four PA landmarks were independently identified and manually annotated by two orthodontists.

Results

The CNN algorithms had a high coefficient of determination (R ² ), compared with the random forest algorithm (CNN-stochastic gradient descent, R ²  = 0.715; CNN-Adam, R ²  = 0.700; random forest, R ²  = 0.486). Analysis of the best and worst performances of the algorithms for each landmark demonstrated that the right latero-orbital landmark was most difficult to detect accurately by using the CNN. Based on the annotated landmarks, reference lines were defined using an algorithm coded in Python. The CNN and random forest algorithms exhibited similar accuracy for the distance between the menton and vertical reference line.

Conclusion

Our findings imply that the proposed deep CNN algorithm for detection of facial asymmetry may enable prompt assessment and reduce the effort involved in orthodontic diagnosis.

Diagnostic accuracy of deep-learning with anomaly detection for a small amount of imbalanced data: discriminating malignant parotid tumors in MRI

We hypothesized that, in discrimination between benign and malignant parotid gland tumors, high diagnostic accuracy could be obtained with a small amount of imbalanced data when anomaly detection (AD) was combined with deep leaning (DL) model and the L₂-constrained softmax loss. The purpose of this study was to evaluate whether the proposed method was more accurate than other commonly used DL or AD methods. Magnetic resonance (MR) images of 245 parotid tumors (22.5% malignant) were retrospectively collected. We evaluated the diagnostic accuracy of the proposed method (VGG16-based DL and AD) and that of classification models using conventional DL and AD methods. A radiologist also evaluated the MR images. ROC and precision-recall (PR) analyses were performed, and the area under the curve (AUC) was calculated. In terms of diagnostic performance, the VGG16-based model with the L₂-constrained softmax loss and AD (local outlier factor) outperformed conventional DL and AD methods and a radiologist (ROC-AUC = 0.86 and PR-ROC = 0.77). The proposed method could discriminate between benign and malignant parotid tumors in MR images even when only a small amount of data with imbalanced distribution is available.

Quantitative neurotoxicology: Potential role of artificial intelligence/deep learning approach

Neurotoxicity studies are important in the preclinical stages of drug development process, because exposure to certain compounds that may enter the brain across a permeable blood brain barrier damages neurons and other supporting cells such as astrocytes. This could, in turn, lead to various neurological disorders such as Parkinson's or Huntington's disease as well as various dementias. Toxicity assessment is often done by pathologists after these exposures by qualitatively or semiquantitatively grading the severity of neurotoxicity in histopathology slides. Quantification of the extent of neurotoxicity supports qualitative histopathological analysis and provides a better understanding of the global extent of brain damage. Stereological techniques such as the utilization of an optical fractionator provide an unbiased quantification of the neuronal damage; however, the process is time-consuming. Advent of whole slide imaging (WSI) introduced digital image analysis which made quantification of neurotoxicity automated, faster and with reduced bias, making statistical comparisons possible. Although automated to a certain level, simple digital image analysis requires manual efforts of experts which is time-consuming and limits analysis of large datasets. Digital image analysis coupled with a deep learning artificial intelligence model provides a good alternative solution to time-consuming stereological and simple digital analysis. Deep learning models could be trained to identify damaged or dead neurons in an automated fashion. This review has focused on and discusses studies demonstrating the role of deep learning in segmentation of brain regions, toxicity detection and quantification of degenerated neurons as well as the estimation of area/volume of degeneration.

Automatic detection of acute ischemic stroke using non-contrast computed tomography and two-stage deep learning model

BACKGROUND AND OBJECTIVE

Currently, it is challenging to detect acute ischemic stroke (AIS)-related changes on computed tomography (CT) images. Therefore, we aimed to develop and evaluate an automatic AIS detection system involving a two-stage deep learning model.

METHODS

We included 238 cases from two different institutions. AIS-related findings were annotated on each of the 238 sets of head CT images by referring to head magnetic resonance imaging (MRI) images in which an MRI examination was performed within 24 h following the CT scan. These 238 annotated cases were divided into a training set including 189 cases and test set including 49 cases. Subsequently, a two-stage deep learning detection model was constructed from the training set using the You Only Look Once v3 model and Visual Geometry Group 16 classification model. Then, the two-stage model performed the AIS detection process in the test set. To assess the detection model's results, a board-certified radiologist also evaluated the test set head CT images with and without the aid of the detection model. The sensitivity of AIS detection and number of false positives were calculated for the evaluation of the test set detection results. The sensitivity of the radiologist with and without the software detection results was compared using the McNemar test. A p-value of less than 0.05 was considered statistically significant.

RESULTS

For the two-stage model and radiologist without and with the use of the software results, the sensitivity was 37.3%, 33.3%, and 41.3%, respectively, and the number of false positives per one case was 1.265, 0.327, and 0.388, respectively. On using the two-stage detection model's results, the board-certified radiologist's detection sensitivity significantly improved (p-value = 0.0313).

CONCLUSIONS

Our detection system involving the two-stage deep learning model significantly improved the radiologist's sensitivity in AIS detection.

Deep learning-level melanoma detection by interpretable machine learning and imaging biomarker cues

SIGNIFICANCE

Melanoma is a deadly cancer that physicians struggle to diagnose early because they lack the knowledge to differentiate benign from malignant lesions. Deep machine learning approaches to image analysis offer promise but lack the transparency to be widely adopted as stand-alone diagnostics.

AIM

We aimed to create a transparent machine learning technology (i.e., not deep learning) to discriminate melanomas from nevi in dermoscopy images and an interface for sensory cue integration.

APPROACH

Imaging biomarker cues (IBCs) fed ensemble machine learning classifier (Eclass) training while raw images fed deep learning classifier training. We compared the areas under the diagnostic receiver operator curves.

RESULTS

Our interpretable machine learning algorithm outperformed the leading deep-learning approach 75% of the time. The user interface displayed only the diagnostic imaging biomarkers as IBCs.

CONCLUSIONS

From a translational perspective, Eclass is better than convolutional machine learning diagnosis in that physicians can embrace it faster than black box outputs. Imaging biomarkers cues may be used during sensory cue integration in clinical screening. Our method may be applied to other image-based diagnostic analyses, including pathology and radiology.

Spectral features based convolutional neural network for accurate and prompt identification of schizophrenic patients

Schizophrenia is a fatal mental disorder, which affects millions of people globally by the disturbance in their thinking, feeling and behaviour. In the age of the internet of things assisted with cloud computing and machine learning techniques, the computer-aided diagnosis of schizophrenia is essentially required to provide its patients with an opportunity to own a better quality of life. In this context, the present paper proposes a spectral features based convolutional neural network (CNN) model for accurate identification of schizophrenic patients using spectral analysis of multichannel EEG signals in real-time. This model processes acquired EEG signals with filtering, segmentation and conversion into frequency domain. Then, given frequency domain segments are divided into six distinct spectral bands like delta, theta-1, theta-2, alpha, beta and gamma. The spectral features including mean spectral amplitude, spectral power and Hjorth descriptors (Activity, Mobility and Complexity) are extracted from each band. These features are independently fed to the proposed spectral features-based CNN and long short-term memory network (LSTM) models for classification. This work also makes use of raw time-domain and frequency-domain EEG segments for classification using temporal CNN and spectral CNN models of same architectures respectively. The overall analysis of simulation results of all models exhibits that the proposed spectral features based CNN model is an efficient technique for accurate and prompt identification of schizophrenic patients among healthy individuals with average classification accuracies of 94.08% and 98.56% for two different datasets with optimally small classification time.

Automated detection of mild and multi-class diabetic eye diseases using deep learning

Diabetic eye disease is a collection of ocular problems that affect patients with diabetes. Thus, timely screening enhances the chances of timely treatment and prevents permanent vision impairment. Retinal fundus images are a useful resource to diagnose retinal complications for ophthalmologists. However, manual detection can be laborious and time-consuming. Therefore, developing an automated diagnose system reduces the time and workload for ophthalmologists. Recently, the image classification using Deep Learning (DL) in between healthy or diseased retinal fundus image classification already achieved a state of the art performance. While the classification of mild and multi-class diseases remains an open challenge, therefore, this research aimed to build an automated classification system considering two scenarios: (i) mild multi-class diabetic eye disease (DED), and (ii) multi-class DED. Our model tested on various datasets, annotated by an opthalmologist. The experiment conducted employing the top two pretrained convolutional neural network (CNN) models on ImageNet. Furthermore, various performance improvement techniques were employed, i.e., fine-tune, optimization, and contrast enhancement. Maximum accuracy of 88.3% obtained on the VGG16 model for multi-class classification and 85.95% for mild multi-class classification.

A Convolutional Neural Network based system for Colorectal cancer segmentation on MRI images

The aim of the study is to present a new Convolutional Neural Network (CNN) based system for the automatic segmentation of the colorectal cancer. The algorithm implemented consists of several steps: a pre-processing to normalize and highlights the tumoral area, the classification based on CNNs, and a post-processing aimed at reducing false positive elements. The classification is performed using three CNNs: each of them classifies the same regions of interest acquired from three different MR sequences. The final segmentation mask is obtained by a majority voting. Performances were evaluated using a semi-automatic segmentation revised by an experienced radiologist as reference standard. The system obtained Dice Similarity Coefficient (DSC) of 0.60, Precision (Pr) of 0.76 and Recall (Re) of 0.55 on the testing set. After applying the leave-one-out validation, we obtained a median DSC=0.58, Pr=0.74, Re=0.54. The promising results obtained by this system, if validated on a larger dataset, could strongly improve personalized medicine.

Automatic classification between COVID-19 pneumonia, non-COVID-19 pneumonia, and the healthy on chest X-ray image: combination of data augmentation methods

This study aimed to develop and validate computer-aided diagnosis (CXDx) system for classification between COVID-19 pneumonia, non-COVID-19 pneumonia, and the healthy on chest X-ray (CXR) images. From two public datasets, 1248 CXR images were obtained, which included 215, 533, and 500 CXR images of COVID-19 pneumonia patients, non-COVID-19 pneumonia patients, and the healthy samples, respectively. The proposed CADx system utilized VGG16 as a pre-trained model and combination of conventional method and mixup as data augmentation methods. Other types of pre-trained models were compared with the VGG16-based model. Single type or no data augmentation methods were also evaluated. Splitting of training/validation/test sets was used when building and evaluating the CADx system. Three-category accuracy was evaluated for test set with 125 CXR images. The three-category accuracy of the CAD system was 83.6% between COVID-19 pneumonia, non-COVID-19 pneumonia, and the healthy. Sensitivity for COVID-19 pneumonia was more than 90%. The combination of conventional method and mixup was more useful than single type or no data augmentation method. In conclusion, this study was able to create an accurate CADx system for the 3-category classification. Source code of our CADx system is available as open source for COVID-19 research.

Optical performance monitoring using digital coherent receivers and convolutional neural networks

We experimentally demonstrate accurate modulation format identification, optical signal to noise ratio (OSNR) estimation, and bit error ratio (BER) estimation of optical signals for wavelength division multiplexed optical communication systems using convolutional neural networks (CNN). We assess the benefits and challenges of extracting information at two distinct points within the demodulation process: immediately after timing recovery and immediately prior to symbol unmapping. For the former, we use 3D Stokes-space based signal representations. For the latter, we use conventional I-Q constellation images created using demodulated symbols. We demonstrate these methods on simulated and experimental dual-polarized waveforms for 32-GBaud QPSK, 8QAM, 16QAM, and 32QAM. Our results show that CNN extracts distinct and learnable features at both the early stage of demodulation where the information can be used to optimize subsequent stages and near the end of demodulation where the constellation images are readily available. Modulation format identification is demonstrated with >99.8% accuracy, OSNR estimation with <0.5 dB average discrepancy and BER estimation with percentage error of <25%.

Automatic Assessment of ASPECTS Using Diffusion-Weighted Imaging in Acute Ischemic Stroke Using Recurrent Residual Convolutional Neural Network

The early detection and rapid quantification of acute ischemic lesions play pivotal roles in stroke management. We developed a deep learning algorithm for the automatic binary classification of the Alberta Stroke Program Early Computed Tomographic Score (ASPECTS) using diffusion-weighted imaging (DWI) in acute stroke patients. Three hundred and ninety DWI datasets with acute anterior circulation stroke were included. A classifier algorithm utilizing a recurrent residual convolutional neural network (RRCNN) was developed for classification between low (1–6) and high (7–10) DWI-ASPECTS groups. The model performance was compared with a pre-trained VGG16, Inception V3, and a 3D convolutional neural network (3DCNN). The proposed RRCNN model demonstrated higher performance than the pre-trained models and 3DCNN with an accuracy of 87.3%, AUC of 0.941, and F1-score of 0.888 for classification between the low and high DWI-ASPECTS groups. These results suggest that the deep learning algorithm developed in this study can provide a rapid assessment of DWI-ASPECTS and may serve as an ancillary tool that can assist physicians in making urgent clinical decisions.

Enhanced registration of ultrasound volumes by segmentation of resection cavity in neurosurgical procedures

PURPOSE

Neurosurgeons can have a better understanding of surgical procedures by comparing ultrasound images obtained at different phases of the tumor resection. However, establishing a direct mapping between subsequent acquisitions is challenging due to the anatomical changes happening during surgery. We propose here a method to improve the registration of ultrasound volumes, by excluding the resection cavity from the registration process.

METHODS

The first step of our approach includes the automatic segmentation of the resection cavities in ultrasound volumes, acquired during and after resection. We used a convolution neural network inspired by the 3D U-Net. Then, subsequent ultrasound volumes are registered by excluding the contribution of resection cavity.

RESULTS

Regarding the segmentation of the resection cavity, the proposed method achieved a mean DICE index of 0.84 on 27 volumes. Concerning the registration of the subsequent ultrasound acquisitions, we reduced the mTRE of the volumes acquired before and during resection from 3.49 to 1.22 mm. For the set of volumes acquired before and after removal, the mTRE improved from 3.55 to 1.21 mm.

CONCLUSIONS

We proposed an innovative registration algorithm to compensate the brain shift affecting ultrasound volumes obtained at subsequent phases of neurosurgical procedures. To the best of our knowledge, our method is the first to exclude automatically segmented resection cavities in the registration of ultrasound volumes in neurosurgery.

Issues associated with deploying CNN transfer learning to detect COVID-19 from chest X-rays

Covid-19 first occurred in Wuhan, China in December 2019. Subsequently, the virus spread throughout the world and as of June 2020 the total number of confirmed cases are above 4.7 million with over 315,000 deaths. Machine learning algorithms built on radiography images can be used as a decision support mechanism to aid radiologists to speed up the diagnostic process. The aim of this work is to conduct a critical analysis to investigate the applicability of convolutional neural networks (CNNs) for the purpose of COVID-19 detection in chest X-ray images and highlight the issues of using CNN directly on the whole image. To accomplish this task, we use 12-off-the-shelf CNN architectures in transfer learning mode on 3 publicly available chest X-ray databases together with proposing a shallow CNN architecture in which we train it from scratch. Chest X-ray images are fed into CNN models without any preprocessing to replicate researches used chest X-rays in this manner. Then a qualitative investigation performed to inspect the decisions made by CNNs using a technique known as class activation maps (CAM). Using CAMs, one can map the activations contributed to the decision of CNNs back to the original image to visualize the most discriminating region(s) on the input image. We conclude that CNN decisions should not be taken into consideration, despite their high classification accuracy, until clinicians can visually inspect and approve the region(s) of the input image used by CNNs that lead to its prediction.

Revealing the Computational Meaning of Neocortical Interarea Signals

To understand the function of the neocortex, which is a hierarchical distributed network, it is useful giving meaning to the signals transmitted between these areas from the computational viewpoint. The overall anatomical structure or organs related to this network, including the neocortex, thalamus, and basal ganglia, has been roughly revealed, and much physiological knowledge, though often fragmentary, is being accumulated. The computational theories involving the neocortex have also been developed considerably. By introducing the assumption “The signals transmitted by interarea axonal projections of pyramidal cells in the neocortex carry different meanings for each cell type, common to all areas,” derived from its nature as a distributed network in the neocortex, allows us to specify the computational meanings of interarea signals. In this paper, first, the types of signals exchanged between neocortical areas are investigated, taking into account biological constraints, and employing theories such as predictive coding, reinforcement learning, representation emulation theory, and BDI logic as theoretical starting points, two types of feedforward signals (observation and deviation) and three types of feedback signals (prediction, plan, and intention) are identified. Next, based on the anatomical knowledge of the neocortex and thalamus, the pathways connecting the areas are organized and summarized as three corticocortical pathways and two thalamocortical pathways. Using this summation as preparation, this paper proposes a hypothesis that gives meaning to each type of signals transmitted in the different pathways in the neocortex, from the viewpoint of their functions. This hypothesis reckons that the feedforward corticocortical pathway transmits observation signals, the feedback corticocortical pathway transmits prediction signals, and the corticothalamic pathway mediated by core relay cells transmits deviation signals. The thalamocortical pathway, which is mediated by matrix relay cells, would be responsible for transmitting the signals that activate a part of prediction signals as intentions, due to the reason that the nature of the other available feedback pathways are not sufficient for conveying plans and intentions as signals. The corticocortical pathway, which is projected from various IT cells to the first layer, would be responsible for transmitting signals that activate a part of prediction signals as plans.

Prognostic Value of Transfer Learning Based Features in Resectable Pancreatic Ductal Adenocarcinoma

Background: Pancreatic Ductal Adenocarcinoma (PDAC) is one of the most aggressive cancers with an extremely poor prognosis. Radiomics has shown prognostic ability in multiple types of cancer including PDAC. However, the prognostic value of traditional radiomics pipelines, which are based on hand-crafted radiomic features alone is limited. Methods: Convolutional neural networks (CNNs) have been shown to outperform radiomics models in computer vision tasks. However, training a CNN from scratch requires a large sample size which is not feasible in most medical imaging studies. As an alternative solution, CNN-based transfer learning models have shown the potential for achieving reasonable performance using small datasets. In this work, we developed and validated a CNN-based transfer learning model for prognostication of overall survival in PDAC patients using two independent resectable PDAC cohorts. Results: The proposed transfer learning-based prognostication model for overall survival achieved the area under the receiver operating characteristic curve of 0.81 on the test cohort, which was significantly higher than that of the traditional radiomics model (0.54). To further assess the prognostic value of the models, the predicted probabilities of death generated from the two models were used as risk scores in a univariate Cox Proportional Hazard model and while the risk score from the traditional radiomics model was not associated with overall survival, the proposed transfer learning-based risk score had significant prognostic value with hazard ratio of 1.86 (95% Confidence Interval: 1.15-3.53, p-value: 0.04). Conclusions: This result suggests that transfer learning-based models may significantly improve prognostic performance in typical small sample size medical imaging studies.

COVID-CAPS: A capsule network-based framework for identification of COVID-19 cases from X-ray images

Novel Coronavirus disease (COVID-19) has abruptly and undoubtedly changed the world as we know it at the end of the 2nd decade of the 21st century. COVID-19 is extremely contagious and quickly spreading globally making its early diagnosis of paramount importance. Early diagnosis of COVID-19 enables health care professionals and government authorities to break the chain of transition and flatten the epidemic curve. The common type of COVID-19 diagnosis test, however, requires specific equipment and has relatively low sensitivity. Computed tomography (CT) scans and X-ray images, on the other hand, reveal specific manifestations associated with this disease. Overlap with other lung infections makes human-centered diagnosis of COVID-19 challenging. Consequently, there has been an urgent surge of interest to develop Deep Neural Network (DNN)-based diagnosis solutions, mainly based on Convolutional Neural Networks (CNNs), to facilitate identification of positive COVID-19 cases. CNNs, however, are prone to lose spatial information between image instances and require large datasets. The paper presents an alternative modeling framework based on Capsule Networks, referred to as the COVID-CAPS, being capable of handling small datasets, which is of significant importance due to sudden and rapid emergence of COVID-19. Our results based on a dataset of X-ray images show that COVID-CAPS has advantage over previous CNN-based models. COVID-CAPS achieved an Accuracy of 95.7%, Sensitivity of 90%, Specificity of 95.8%, and Area Under the Curve (AUC) of 0.97, while having far less number of trainable parameters in comparison to its counterparts. To potentially and further improve diagnosis capabilities of the COVID-CAPS, pre-training and transfer learning are utilized based on a new dataset constructed from an external dataset of X-ray images. This is in contrary to existing works on COVID-19 detection where pre-training is performed based on natural images. Pre-training with a dataset of similar nature further improved accuracy to 98.3% and specificity to 98.6%.

Performance improvement of mediastinal lymph node severity detection using GAN and Inception network

BACKGROUND AND OBJECTIVE

In lung cancer, the determination of mediastinal lymph node (MLN) status as benign or malignant influence treatment planning and survival rate. Invasive pathological tests for the classification of MLNs into benign and malignant have various shortcomings like painfulness, the risk associated with anesthesia, and depends to a large extent on skillset and preferences of the surgeon performing the test. Hence, computer-aided system for MLNs severity detection has been explored widely by the researchers. Very recently, in our earlier concluded work on non-invasive method for MLNs differential diagnosis in computed tomography (CT) images, combination of different data augmentation approaches and state-of-art fully convolutional network (FCN) were implemented to enhance the performance of malignancy detection. However, the performance of FCN network were highly depended on the selection of appropriate data augmentation approach and control of their hyperparameters. Moreover, a standard practice to get hierarchical features in convolutional neural network (CNN) models requires deeper stacking of layers. This leads to an increase in number of trainable parameters which prone to overfitting of the network.

METHODS

In view of the above mention limitations, in this paper, authors have proposed an approach that includes: 1) Generative Adversarial Network (GAN) for data augmentation, and 2) Inception network for malignancy detection. Unlike conventional data augmentation strategy, GAN based augmentation approach generates data that correlates to original data distribution. In the case of Inception based model, it uses multiple size kernels with factorized convolution for hierarchical feature extraction. This helps to a significant reduction in trainable parameters and the problem of overfitting.

RESULTS

In this paper, experiments with different GAN approaches, as well as with different Inception architectures, are conducted to evaluate and justify the selection of appropriate GAN and Inception architecture, respectively for MLNs severity detection. The proposed approach achieves superior results with an average accuracy, sensitivity, specificity, and area under curve of 94.95%, 93.65%, 96.67%, and 95%, respectively.

CONCLUSION

The obtained results validate the usefulness of GANs for data augmentation in the differential diagnosis of benign and malignant MLNs. The proposed Inception network based classifier for malignancy detection shows promising results compared to all investigated methods presented in various literature.

Computer-aided detection of COVID-19 from X-ray images using multi-CNN and Bayesnet classifier

Corona virus disease-2019 (COVID-19) is a pandemic caused by novel coronavirus. COVID-19 is spreading rapidly throughout the world. The gold standard for diagnosing COVID-19 is reverse transcription-polymerase chain reaction (RT-PCR) test. However, the facility for RT-PCR test is limited, which causes early diagnosis of the disease difficult. Easily available modalities like X-ray can be used to detect specific symptoms associated with COVID-19. Pre-trained convolutional neural networks are widely used for computer-aided detection of diseases from smaller datasets. This paper investigates the effectiveness of multi-CNN, a combination of several pre-trained CNNs, for the automated detection of COVID-19 from X-ray images. The method uses a combination of features extracted from multi-CNN with correlation based feature selection (CFS) technique and Bayesnet classifier for the prediction of COVID-19. The method was tested using two public datasets and achieved promising results on both the datasets. In the first dataset consisting of 453 COVID-19 images and 497 non-COVID images, the method achieved an AUC of 0.963 and an accuracy of 91.16%. In the second dataset consisting of 71 COVID-19 images and 7 non-COVID images, the method achieved an AUC of 0.911 and an accuracy of 97.44%. The experiments performed in this study proved the effectiveness of pre-trained multi-CNN over single CNN in the detection of COVID-19.

Novel Feature Selection and Voting Classifier Algorithms for COVID-19 Classification in CT Images

Diagnosis is a critical preventive step in Coronavirus research which has similar manifestations with other types of pneumonia. CT scans and X-rays play an important role in that direction. However, processing chest CT images and using them to accurately diagnose COVID-19 is a computationally expensive task. Machine Learning techniques have the potential to overcome this challenge. This article proposes two optimization algorithms for feature selection and classification of COVID-19. The proposed framework has three cascaded phases. Firstly, the features are extracted from the CT scans using a Convolutional Neural Network (CNN) named AlexNet. Secondly, a proposed features selection algorithm, Guided Whale Optimization Algorithm (Guided WOA) based on Stochastic Fractal Search (SFS), is then applied followed by balancing the selected features. Finally, a proposed voting classifier, Guided WOA based on Particle Swarm Optimization (PSO), aggregates different classifiers' predictions to choose the most voted class. This increases the chance that individual classifiers, e.g. Support Vector Machine (SVM), Neural Networks (NN), k-Nearest Neighbor (KNN), and Decision Trees (DT), to show significant discrepancies. Two datasets are used to test the proposed model: CT images containing clinical findings of positive COVID-19 and CT images negative COVID-19. The proposed feature selection algorithm (SFS-Guided WOA) is compared with other optimization algorithms widely used in recent literature to validate its efficiency. The proposed voting classifier (PSO-Guided-WOA) achieved AUC (area under the curve) of 0.995 that is superior to other voting classifiers in terms of performance metrics. Wilcoxon rank-sum, ANOVA, and T-test statistical tests are applied to statistically assess the quality of the proposed algorithms as well.

Convolutional neural networks versus radiologists in characterization of small hypoattenuating hepatic nodules on CT: a critical diagnostic challenge in staging of colorectal carcinoma

Our objective was to compare the diagnostic performance and diagnostic confidence of convolutional neural networks (CNN) to radiologists in characterizing small hypoattenuating hepatic nodules (SHHN) in colorectal carcinoma (CRC) on CT scans. Retrospective review of CRC CT scans over 6-years yielded 199 patients (550 SHHN) defined as < 1 cm in diameter. The reference standard was established through 1-year stability/MRI for benign or nodule evolution for malignant nodules. Five CNNs underwent supervised training on 150 patients (412 SHHN). The remaining 49 patients (138 SHHN) were used as testing-set to compare performance of 3 radiologists to CNN, measured through ROC AUC analysis of confidence rating assigned to each nodule by the radiologists. Multivariable modeling was used to compensate for radiologist bias from visible findings other than SHHN. In characterizing SHHN as benign or malignant, the radiologists’ mean AUC ROC (0.96) was significantly higher than CNN (0.84, p = 0.0004) but equivalent to CNN adjusted through multivariable modeling for presence of synchronous ≥ 1 cm liver metastases (0.95, p = 0.9). The diagnostic confidence of radiologists and CNN were analyzed. There were significantly lower number of nodules rated with low confidence by CNN (19.6%) and CNN with liver metastatic status (18.1%) than two (38.4%, 44.2%, p < 0.0001) but not a third radiologist (11.1%, p = 0.09). We conclude that in CRC, CNN in combination with liver metastatic status equaled expert radiologists in characterizing SHHN but with better diagnostic confidence.

Intelligent Imaging in Nuclear Medicine: the Principles of Artificial Intelligence, Machine Learning and Deep Learning

The emergence of artificial intelligence (AI) in nuclear medicine has occurred over the last 50 years but more recent developments in machine learning (ML) and deep learning (DL) have driven new capabilities of AI in nuclear medicine. In nuclear medicine, the artificial neural network (ANN) is the backbone of ML and DL. The inputs may be radiomic features that have been extracted from the image files or, if using a convolutional neural network (CNN), may be the images themselves. AI in nuclear medicine re-engineers and re-imagines clinical and research capabilities. An understanding of the principles of AI, ML and DL contextualised to nuclear medicine allows richer engagement in clinical and research applications, and capacity for problem solving where required. Simple applications of ML include quality assurance, risk assessment, business analytics and rudimentary classifications. More complex applications of DL for detection, localisation, classification, segmentation, quantitation and radiomic feature extraction using CNNs can be applied to general nuclear medicine, SPECT, PET, CT and MRI. There are also applications of ANNs and ML that allow small datasets (and larger ones) to be analysed in parallel to conventional statistical analysis. AI has assimilated into the clinical and research practice of nuclear medicine with little disruption. The emergence of ML and DL applications, however, has produced a seismic significant shift in the clinical and research landscape that demands at least rudimentary understanding of the principles of AI, ANNs and CNNs among nuclear medicine professionals.

Detection of hip osteoarthritis by using plain pelvic radiographs with deep learning methods

OBJECTIVE

The incidence of osteoarthritis is gradually increasing in public due to aging and increase in obesity. Various imaging methods are used in the diagnosis of hip osteoarthritis, and plain pelvic radiography is the first preferred imaging method in the diagnosis of hip osteoarthritis. In this study, we aimed to develop a computer-aided diagnosis method that will help physicians for the diagnosis of hip osteoarthritis by interpreting plain pelvic radiographs.

MATERIALS AND METHODS

In this retrospective study, convolutional neural networks were used and transfer learning was applied with the pre-trained VGG-16 network. Our dataset consisted of 221 normal hip radiographs and 213 hip radiographs with osteoarthritis. In this study, the training of the network was performed using a total of 426 hip osteoarthritis images and a total of 442 normal pelvic images obtained by flipping the raw data set.

RESULTS

Training results were evaluated with performance metrics such as accuracy, sensitivity, specificity, and precision calculated by using the confusion matrix. We achieved accuracy, sensitivity, specificity and precision results at 90.2%, 97.6%, 83.0%, and 84.7% respectively.

CONCLUSION

We achieved promising results with this computer-aided diagnosis method that we tried to develop using convolutional neural networks based on transfer learning. This method can help clinicians for the diagnosis of hip osteoarthritis while interpreting plain pelvic radiographs, also provides assistance for a second objective interpretation. It may also reduce the need for advanced imaging methods in the diagnosis of hip osteoarthritis.

Understanding artificial intelligence based radiology studies: What is overfitting?

Artificial intelligence (AI) is a broad umbrella term used to encompass a wide variety of subfields dedicated to creating algorithms to perform tasks that mimic human intelligence. As AI development grows closer to clinical integration, radiologists will need to become familiar with the principles of artificial intelligence to properly evaluate and use this powerful tool. This series aims to explain certain basic concepts of artificial intelligence, and their applications in medical imaging starting with a concept of overfitting.

Development of Deep Learning Models for Predicting the Effects of Exposure to Engineered Nanomaterials on Daphnia magna

This study presents the results of applying deep learning methodologies within the ecotoxicology field, with the objective of training predictive models that can support hazard assessment and eventually the design of safer engineered nanomaterials (ENMs). A workflow applying two different deep learning architectures on microscopic images of Daphnia magna is proposed that can automatically detect possible malformations, such as effects on the length of the tail, and the overall size, and uncommon lipid concentrations and lipid deposit shapes, which are due to direct or parental exposure to ENMs. Next, classification models assign specific objects (heart, abdomen/claw) to classes that depend on lipid densities and compare the results with controls. The models are statistically validated in terms of their prediction accuracy on external D. magna images and illustrate that deep learning technologies can be useful in the nanoinformatics field, because they can automate time-consuming manual procedures, accelerate the investigation of adverse effects of ENMs, and facilitate the process of designing safer nanostructures. It may even be possible in the future to predict impacts on subsequent generations from images of parental exposure, reducing the time and cost involved in long-term reproductive toxicity assays over multiple generations.

Multi-Channel Transfer Learning of Chest X-ray Images for Screening of COVID-19

The 2019 novel coronavirus (COVID-19) has spread rapidly all over the world. The standard test for screening COVID-19 patients is the polymerase chain reaction test. As this method is time consuming, as an alternative, chest X-rays may be considered for quick screening. However, specialization is required to read COVID-19 chest X-ray images as they vary in features. To address this, we present a multi-channel pre-trained ResNet architecture to facilitate the diagnosis of COVID-19 chest X-ray. Three ResNet-based models were retrained to classify X-rays in a one-against-all basis from (a) normal or diseased, (b) pneumonia or non-pneumonia, and (c) COVID-19 or non-COVID19 individuals. Finally, these three models were ensembled and fine-tuned using X-rays from 1579 normal, 4245 pneumonia, and 184 COVID-19 individuals to classify normal, pneumonia, and COVID-19 cases in a one-against-one framework. Our results show that the ensemble model is more accurate than the single model as it extracts more relevant semantic features for each class. The method provides a precision of 94% and a recall of 100%. It could potentially help clinicians in screening patients for COVID-19, thus facilitating immediate triaging and treatment for better outcomes.