Deep and Hybrid Learning Technique for Early Detection of Tuberculosis Based on X-ray Images Using Feature Fusion

Design of Interoperable Electronic Health Record (EHR) Application for Early Detection of Lung Diseases Using a Decision Support System by Expanding Deep Learning Techniques

Background

Electronic health records (EHRs) are live, digital patient records that provide a thorough overview of a person's complete health data. Electronic health records (EHRs) provide better healthcare decisions and evidence-based patient treatment and track patients' clinical development. The EHR offers a new range of opportunities for analyzing and contrasting exam findings and other data, creating a proper information management mechanism to boost effectiveness, quick resolutions, and identifications.

Aim

The aim of this studywas to implement an interoperable EHR system to improve the quality of care through the decision support system for the identification of lung cancer in its early stages.

Objective

The main objective of the proposed system was to develop an Android application for maintaining an EHR system and decision support system using deep learning for the early detection of diseases. The second objective was to study the early stages of lung disease to predict/detect it using a decision support system.

Methods

To extract the EHR data of patients, an android application was developed. The android application helped in accumulating the data of each patient. The accumulated data were used to create a decision support system for the early prediction of lung cancer. To train, test, and validate the prediction of lung cancer, a few samples from the ready dataset and a few data from patients were collected. The valid data collection from patients included an age range of 40 to 70, and both male and female patients. In the process of experimentation, a total of 316 images were considered. The testing was done by considering the data set into 80:20 partitions. For the evaluation purpose, a manual classification was done for 3 different diseases, such as large cell carcinoma, adenocarcinoma, and squamous cell carcinoma diseases in lung cancer detection.

Results

The first model was tested for interoperability constraints of EHR with data collection and updations. When it comes to the disease detection system, lung cancer was predicted for large cell carcinoma, adenocarcinoma, and squamous cell carcinoma type by considering 80:20 training and testing ratios. Among the considered 336 images, the prediction of large cell carcinoma was less compared to adenocarcinoma and squamous cell carcinoma. The analysis also showed that large cell carcinoma occurred majorly in males due to smoking and was found as breast cancer in females.

Conclusion

As the challenges are increasing daily in healthcare industries, a secure, interoperable EHR could help patients and doctors access patient data efficiently and effectively using an Android application. Therefore, a decision support system using a deep learning model was attempted and successfully used for disease detection. Early disease detection for lung cancer was evaluated, and the model achieved an accuracy of 93%. In future work, the integration of EHR data can be performed to detect various diseases early.

Predicting of diabetic retinopathy development stages of fundus images using deep learning based on combined features

The number of diabetic retinopathy (DR) patients is increasing every year, and this causes a public health problem. Therefore, regular diagnosis of diabetes patients is necessary to avoid the progression of DR stages to advanced stages that lead to blindness. Manual diagnosis requires effort and expertise and is prone to errors and differing expert diagnoses. Therefore, artificial intelligence techniques help doctors make a proper diagnosis and resolve different opinions. This study developed three approaches, each with two systems, for early diagnosis of DR disease progression. All colour fundus images have been subjected to image enhancement and increasing contrast ROI through filters. All features extracted by the DenseNet-121 and AlexNet (Dense-121 and Alex) were fed to the Principal Component Analysis (PCA) method to select important features and reduce their dimensions. The first approach is to DR image analysis for early prediction of DR disease progression by Artificial Neural Network (ANN) with selected, low-dimensional features of Dense-121 and Alex models. The second approach is to DR image analysis for early prediction of DR disease progression is by integrating important and low-dimensional features of Dense-121 and Alex models before and after PCA. The third approach is to DR image analysis for early prediction of DR disease progression by ANN with the radiomic features. The radiomic features are a combination of the features of the CNN models (Dense-121 and Alex) separately with the handcrafted features extracted by Discrete Wavelet Transform (DWT), Local Binary Pattern (LBP), Fuzzy colour histogram (FCH), and Gray Level Co-occurrence Matrix (GLCM) methods. With the radiomic features of the Alex model and the handcrafted features, ANN reached a sensitivity of 97.92%, an AUC of 99.56%, an accuracy of 99.1%, a specificity of 99.4% and a precision of 99.06%.

Blood Slide Image Analysis to Classify WBC Types for Prediction Haematology Based on a Hybrid Model of CNN and Handcrafted Features

White blood cells (WBCs) are one of the main components of blood produced by the bone marrow. WBCs are part of the immune system that protects the body from infectious diseases and an increase or decrease in the amount of any type that causes a particular disease. Thus, recognizing the WBC types is essential for diagnosing the patient's health and identifying the disease. Analyzing blood samples to determine the amount and WBC types requires experienced doctors. Artificial intelligence techniques were applied to analyze blood samples and classify their types to help doctors distinguish between types of infectious diseases due to increased or decreased WBC amounts. This study developed strategies for analyzing blood slide images to classify WBC types. The first strategy is to classify WBC types by the SVM-CNN technique. The second strategy for classifying WBC types is by SVM based on hybrid CNN features, which are called VGG19-ResNet101-SVM, ResNet101-MobileNet-SVM, and VGG19-ResNet101-MobileNet-SVM techniques. The third strategy for classifying WBC types by FFNN is based on a hybrid model of CNN and handcrafted features. With MobileNet and handcrafted features, FFNN achieved an AUC of 99.43%, accuracy of 99.80%, precision of 99.75%, specificity of 99.75%, and sensitivity of 99.68%.

Hybrid Models for Endoscopy Image Analysis for Early Detection of Gastrointestinal Diseases Based on Fused Features

The gastrointestinal system contains the upper and lower gastrointestinal tracts. The main tasks of the gastrointestinal system are to break down food and convert it into essential elements that the body can benefit from and expel waste in the form of feces. If any organ is affected, it does not work well, which affects the body. Many gastrointestinal diseases, such as infections, ulcers, and benign and malignant tumors, threaten human life. Endoscopy techniques are the gold standard for detecting infected parts within the organs of the gastrointestinal tract. Endoscopy techniques produce videos that are converted into thousands of frames that show the disease's characteristics in only some frames. Therefore, this represents a challenge for doctors because it is a tedious task that requires time, effort, and experience. Computer-assisted automated diagnostic techniques help achieve effective diagnosis to help doctors identify the disease and give the patient the appropriate treatment. In this study, many efficient methodologies for analyzing endoscopy images for diagnosing gastrointestinal diseases were developed for the Kvasir dataset. The Kvasir dataset was classified by three pre-trained models: GoogLeNet, MobileNet, and DenseNet121. The images were optimized, and the gradient vector flow (GVF) algorithm was applied to segment the regions of interest (ROIs), isolating them from healthy regions and saving the endoscopy images as Kvasir-ROI. The Kvasir-ROI dataset was classified by the three pre-trained GoogLeNet, MobileNet, and DenseNet121 models. Hybrid methodologies (CNN-FFNN and CNN-XGBoost) were developed based on the GVF algorithm and achieved promising results for diagnosing disease based on endoscopy images of gastroenterology. The last methodology is based on fused CNN models and their classification by FFNN and XGBoost networks. The hybrid methodology based on the fused CNN features, called GoogLeNet-MobileNet-DenseNet121-XGBoost, achieved an AUC of 97.54%, accuracy of 97.25%, sensitivity of 96.86%, precision of 97.25%, and specificity of 99.48%.

Automatic Analysis of MRI Images for Early Prediction of Alzheimer's Disease Stages Based on Hybrid Features of CNN and Handcrafted Features

Alzheimer's disease (AD) is considered one of the challenges facing health care in the modern century; until now, there has been no effective treatment to cure it, but there are drugs to slow its progression. Therefore, early detection of Alzheimer's is vital to take needful measures before it develops into brain damage which cannot be treated. Magnetic resonance imaging (MRI) techniques have contributed to the diagnosis and prediction of its progression. MRI images require highly experienced doctors and radiologists, and the analysis of MRI images takes time to analyze each slice. Thus, deep learning techniques play a vital role in analyzing a huge amount of MRI images with high accuracy to detect Alzheimer's and predict its progression. Because of the similarities in the characteristics of the early stages of Alzheimer's, this study aimed to extract the features in several methods and integrate the features extracted from more than one method into the same features matrix. This study contributed to the development of three methodologies, each with two systems, with all systems aimed at achieving satisfactory accuracy for the detection of AD and predicting the stages of its progression. The first methodology is by Feed Forward Neural Network (FFNN) with the features of GoogLeNet and DenseNet-121 models separately. The second methodology is by FFNN network with combined features between GoogLeNet and Dense-121 models before and after high-dimensionality reduction of features using the Principal Component Analysis (PCA) algorithm. The third methodology is by FFNN network with combined features between GoogLeNet and Dense-121 models separately and features extracted by Discrete Wavelet Transform (DWT), Local Binary Pattern (LBP) and Gray Level Co-occurrence Matrix (GLCM) methods called handcrafted features. All systems yielded super results in detecting AD and predicting the stages of its progression. With the combined features of the DenseNet-121 and handcrafted, the FFNN achieved an accuracy of 99.7%, sensitivity of 99.64%, AUC of 99.56%, precision of 99.63%, and a specificity of 99.67%.

A novel CT image de-noising and fusion based deep learning network to screen for disease (COVID-19)

A COVID-19, caused by SARS-CoV-2, has been declared a global pandemic by WHO. It first appeared in China at the end of 2019 and quickly spread throughout the world. During the third layer, it became more critical. COVID-19 spread is extremely difficult to control, and a huge number of suspected cases must be screened for a cure as soon as possible. COVID-19 laboratory testing takes time and can result in significant false negatives. To combat COVID-19, reliable, accurate and fast methods are urgently needed. The commonly used Reverse Transcription Polymerase Chain Reaction has a low sensitivity of approximately 60% to 70%, and sometimes even produces negative results. Computer Tomography (CT) has been observed to be a subtle approach to detecting COVID-19, and it may be the best screening method. The scanned image's quality, which is impacted by motion-induced Poisson or Impulse noise, is vital. In order to improve the quality of the acquired image for post segmentation, a novel Impulse and Poisson noise reduction method employing boundary division max/min intensities elimination along with an adaptive window size mechanism is proposed. In the second phase, a number of CNN techniques are explored for detecting COVID-19 from CT images and an Assessment Fusion Based model is proposed to predict the result. The AFM combines the results for cutting-edge CNN architectures and generates a final prediction based on choices. The empirical results demonstrate that our proposed method performs extensively and is extremely useful in actual diagnostic situations.

Hybrid Techniques for Diagnosing Endoscopy Images for Early Detection of Gastrointestinal Disease Based on Fusion Features

Gastrointestinal (GI) diseases, particularly tumours, are considered one of the most widespread and dangerous diseases and thus need timely health care for early detection to reduce deaths. Endoscopy technology is an effective technique for diagnosing GI diseases, thus producing a video containing thousands of frames. However, it is difficult to analyse all the images by a gastroenterologist, and it takes a long time to keep track of all the frames. Thus, artificial intelligence systems provide solutions to this challenge by analysing thousands of images with high speed and effective accuracy. Hence, systems with different methodologies are developed in this work. The first methodology for diagnosing endoscopy images of GI diseases is by using VGG-16 + SVM and DenseNet-121 + SVM. The second methodology for diagnosing endoscopy images of gastrointestinal diseases by artificial neural network (ANN) is based on fused features between VGG-16 and DenseNet-121 before and after high-dimensionality reduction by the principal component analysis (PCA). The third methodology is by ANN and is based on the fused features between VGG-16 and handcrafted features and features fused between DenseNet-121 and the handcrafted features. Herein, handcrafted features combine the features of gray level cooccurrence matrix (GLCM), discrete wavelet transform (DWT), fuzzy colour histogram (FCH), and local binary pattern (LBP) methods. All systems achieved promising results for diagnosing endoscopy images of the gastroenterology data set. The ANN network reached an accuracy, sensitivity, precision, specificity, and an AUC of 98.9%, 98.70%, 98.94%, 99.69%, and 99.51%, respectively, based on fused features of the VGG-16 and the handcrafted.

AI Techniques of Dermoscopy Image Analysis for the Early Detection of Skin Lesions Based on Combined CNN Features

Melanoma is one of the deadliest types of skin cancer that leads to death if not diagnosed early. Many skin lesions are similar in the early stages, which causes an inaccurate diagnosis. Accurate diagnosis of the types of skin lesions helps dermatologists save patients’ lives. In this paper, we propose hybrid systems based on the advantages of fused CNN models. CNN models receive dermoscopy images of the ISIC 2019 dataset after segmenting the area of lesions and isolating them from healthy skin through the Geometric Active Contour (GAC) algorithm. Artificial neural network (ANN) and Random Forest (Rf) receive fused CNN features and classify them with high accuracy. The first methodology involved analyzing the area of skin lesions and diagnosing their type early using the hybrid models CNN-ANN and CNN-RF. CNN models (AlexNet, GoogLeNet and VGG16) receive lesions area only and produce high depth feature maps. Thus, the deep feature maps were reduced by the PCA and then classified by ANN and RF networks. The second methodology involved analyzing the area of skin lesions and diagnosing their type early using the hybrid CNN-ANN and CNN-RF models based on the features of the fused CNN models. It is worth noting that the features of the CNN models were serially integrated after reducing their high dimensions by Principal Component Analysis (PCA). Hybrid models based on fused CNN features achieved promising results for diagnosing dermatoscopic images of the ISIC 2019 data set and distinguishing skin cancer from other skin lesions. The AlexNet-GoogLeNet-VGG16-ANN hybrid model achieved an AUC of 94.41%, sensitivity of 88.90%, accuracy of 96.10%, precision of 88.69%, and specificity of 99.44%.

Histopathological Analysis for Detecting Lung and Colon Cancer Malignancies Using Hybrid Systems with Fused Features

Lung and colon cancer are among humanity's most common and deadly cancers. In 2020, there were 4.19 million people diagnosed with lung and colon cancer, and more than 2.7 million died worldwide. Some people develop lung and colon cancer simultaneously due to smoking which causes lung cancer, leading to an abnormal diet, which also causes colon cancer. There are many techniques for diagnosing lung and colon cancer, most notably the biopsy technique and its analysis in laboratories. Due to the scarcity of health centers and medical staff, especially in developing countries. Moreover, manual diagnosis takes a long time and is subject to differing opinions of doctors. Thus, artificial intelligence techniques solve these challenges. In this study, three strategies were developed, each with two systems for early diagnosis of histological images of the LC25000 dataset. Histological images have been improved, and the contrast of affected areas has been increased. The GoogLeNet and VGG-19 models of all systems produced high dimensional features, so redundant and unnecessary features were removed to reduce high dimensionality and retain essential features by the PCA method. The first strategy for diagnosing the histological images of the LC25000 dataset by ANN uses crucial features of GoogLeNet and VGG-19 models separately. The second strategy uses ANN with the combined features of GoogLeNet and VGG-19. One system reduced dimensions and combined, while the other combined high features and then reduced high dimensions. The third strategy uses ANN with fusion features of CNN models (GoogLeNet and VGG-19) and handcrafted features. With the fusion features of VGG-19 and handcrafted features, the ANN reached a sensitivity of 99.85%, a precision of 100%, an accuracy of 99.64%, a specificity of 100%, and an AUC of 99.86%.

Multi-Models of Analyzing Dermoscopy Images for Early Detection of Multi-Class Skin Lesions Based on Fused Features

Melanoma is a cancer that threatens life and leads to death. Effective detection of skin lesion types by images is a challenging task. Dermoscopy is an effective technique for detecting skin lesions. Early diagnosis of skin cancer is essential for proper treatment. Skin lesions are similar in their early stages, so manual diagnosis is difficult. Thus, artificial intelligence techniques can analyze images of skin lesions and discover hidden features not seen by the naked eye. This study developed hybrid techniques based on hybrid features to effectively analyse dermoscopic images to classify two datasets, HAM10000 and PH2, of skin lesions. The images have been optimized for all techniques, and the problem of imbalance between the two datasets has been resolved. The HAM10000 and PH2 datasets were classified by pre-trained MobileNet and ResNet101 models. For effective detection of the early stages skin lesions, hybrid techniques SVM-MobileNet, SVM-ResNet101 and SVM-MobileNet-ResNet101 were applied, which showed better performance than pre-trained CNN models due to the effectiveness of the handcrafted features that extract the features of color, texture and shape. Then, handcrafted features were combined with the features of the MobileNet and ResNet101 models to form a high accuracy feature. Finally, features of MobileNet-handcrafted and ResNet101-handcrafted were sent to ANN for classification with high accuracy. For the HAM10000 dataset, the ANN with MobileNet and handcrafted features achieved an AUC of 97.53%, accuracy of 98.4%, sensitivity of 94.46%, precision of 93.44% and specificity of 99.43%. Using the same technique, the PH2 data set achieved 100% for all metrics.

Hybrid Techniques for the Diagnosis of Acute Lymphoblastic Leukemia Based on Fusion of CNN Features

Acute lymphoblastic leukemia (ALL) is one of the deadliest forms of leukemia due to the bone marrow producing many white blood cells (WBC). ALL is one of the most common types of cancer in children and adults. Doctors determine the treatment of leukemia according to its stages and its spread in the body. Doctors rely on analyzing blood samples under a microscope. Pathologists face challenges, such as the similarity between infected and normal WBC in the early stages. Manual diagnosis is prone to errors, differences of opinion, and the lack of experienced pathologists compared to the number of patients. Thus, computer-assisted systems play an essential role in assisting pathologists in the early detection of ALL. In this study, systems with high efficiency and high accuracy were developed to analyze the images of C-NMC 2019 and ALL-IDB2 datasets. In all proposed systems, blood micrographs were improved and then fed to the active contour method to extract WBC-only regions for further analysis by three CNN models (DenseNet121, ResNet50, and MobileNet). The first strategy for analyzing ALL images of the two datasets is the hybrid technique of CNN-RF and CNN-XGBoost. DenseNet121, ResNet50, and MobileNet models extract deep feature maps. CNN models produce high features with redundant and non-significant features. So, CNN deep feature maps were fed to the Principal Component Analysis (PCA) method to select highly representative features and sent to RF and XGBoost classifiers for classification due to the high similarity between infected and normal WBC in early stages. Thus, the strategy for analyzing ALL images using serially fused features of CNN models. The deep feature maps of DenseNet121-ResNet50, ResNet50-MobileNet, DenseNet121-MobileNet, and DenseNet121-ResNet50-MobileNet were merged and then classified by RF classifiers and XGBoost. The RF classifier with fused features for DenseNet121-ResNet50-MobileNet reached an AUC of 99.1%, accuracy of 98.8%, sensitivity of 98.45%, precision of 98.7%, and specificity of 98.85% for the C-NMC 2019 dataset. With the ALL-IDB2 dataset, hybrid systems achieved 100% results for AUC, accuracy, sensitivity, precision, and specificity.

Multi-Techniques for Analyzing X-ray Images for Early Detection and Differentiation of Pneumonia and Tuberculosis Based on Hybrid Features

An infectious disease called tuberculosis (TB) exhibits pneumonia-like symptoms and traits. One of the most important methods for identifying and diagnosing pneumonia and tuberculosis is X-ray imaging. However, early discrimination is difficult for radiologists and doctors because of the similarities between pneumonia and tuberculosis. As a result, patients do not receive the proper care, which in turn does not prevent the disease from spreading. The goal of this study is to extract hybrid features using a variety of techniques in order to achieve promising results in differentiating between pneumonia and tuberculosis. In this study, several approaches for early identification and distinguishing tuberculosis from pneumonia were suggested. The first proposed system for differentiating between pneumonia and tuberculosis uses hybrid techniques, VGG16 + support vector machine (SVM) and ResNet18 + SVM. The second proposed system for distinguishing between pneumonia and tuberculosis uses an artificial neural network (ANN) based on integrating features of VGG16 and ResNet18, before and after reducing the high dimensions using the principal component analysis (PCA) method. The third proposed system for distinguishing between pneumonia and tuberculosis uses ANN based on integrating features of VGG16 and ResNet18 separately with handcrafted features extracted by local binary pattern (LBP), discrete wavelet transform (DWT) and gray level co-occurrence matrix (GLCM) algorithms. All the proposed systems have achieved superior results in the early differentiation between pneumonia and tuberculosis. An ANN based on the features of VGG16 with LBP, DWT and GLCM (LDG) reached an accuracy of 99.6%, sensitivity of 99.17%, specificity of 99.42%, precision of 99.63%, and an AUC of 99.58%.

Early Diagnosis of Oral Squamous Cell Carcinoma Based on Histopathological Images Using Deep and Hybrid Learning Approaches

Oral squamous cell carcinoma (OSCC) is one of the most common head and neck cancer types, which is ranked the seventh most common cancer. As OSCC is a histological tumor, histopathological images are the gold diagnosis standard. However, such diagnosis takes a long time and high-efficiency human experience due to tumor heterogeneity. Thus, artificial intelligence techniques help doctors and experts to make an accurate diagnosis. This study aimed to achieve satisfactory results for the early diagnosis of OSCC by applying hybrid techniques based on fused features. The first proposed method is based on a hybrid method of CNN models (AlexNet and ResNet-18) and the support vector machine (SVM) algorithm. This method achieved superior results in diagnosing the OSCC data set. The second proposed method is based on the hybrid features extracted by CNN models (AlexNet and ResNet-18) combined with the color, texture, and shape features extracted using the fuzzy color histogram (FCH), discrete wavelet transform (DWT), local binary pattern (LBP), and gray-level co-occurrence matrix (GLCM) algorithms. Because of the high dimensionality of the data set features, the principal component analysis (PCA) algorithm was applied to reduce the dimensionality and send it to the artificial neural network (ANN) algorithm to diagnose it with promising accuracy. All the proposed systems achieved superior results in histological image diagnosis of OSCC, the ANN network based on the hybrid features using AlexNet, DWT, LBP, FCH, and GLCM achieved an accuracy of 99.1%, specificity of 99.61%, sensitivity of 99.5%, precision of 99.71%, and AUC of 99.52%.