Human skin type classification using image processing and deep learning approaches

Cosmetics consumers need to be aware of their skin type before purchasing products. Identifying skin types can be challenging, especially when they vary from oily to dry in different areas, with skin specialist providing more accurate results. In recent years, artificial intelligence and machine learning have been utilized across various fields, including medicine, to assist in identifying and predicting situations. This study developed a skin type classification model using a Convolutional Neural Networks (CNN) deep learning algorithms. The dataset consisted of normal, oily, and dry skin images, with 112 images for normal skin, 120 images for oily skin, and 97 images for dry skin. Image quality was enhanced using the Contrast Limited Adaptive Histogram Equalization (CLAHE) technique, with data augmentation by rotation applied to increase dataset variety, resulting in a total of 1,316 images. CNN architectures including MobileNet-V2, EfficientNet-V2, InceptionV2, and ResNet-V1 were optimized and evaluated. Findings showed that the EfficientNet-V2 architecture performed the best, achieving an accuracy of 91.55% with average loss of 22.74%. To further improve the model, hyperparameter tuning was conducted, resulting in an accuracy of 94.57% and a loss of 13.77%. The Model performance was validated using 10-fold cross-validation and tested on unseen data, achieving an accuracy of 89.70% with a loss of 21.68%.

Augmenting healthy brain magnetic resonance images using generative adversarial networks

Machine learning applications in the medical sector face a lack of medical data due to privacy issues. For instance, brain tumor image-based classification suffers from the lack of brain images. The lack of such images produces some classification problems, i.e., class imbalance issues which can cause a bias toward one class over the others. This study aims to solve the imbalance problem of the "no tumor" class in the publicly available brain magnetic resonance imaging (MRI) dataset. Generative adversarial network (GAN)-based augmentation techniques were used to solve the imbalance classification problem. Specifically, deep convolutional GAN (DCGAN) and single GAN (SinGAN). Moreover, the traditional-based augmentation techniques were implemented using the rotation method. Thus, several VGG16 classification experiments were conducted, including (i) the original dataset, (ii) the DCGAN-based dataset, (iii) the SinGAN-based dataset, (iv) a combination of the DCGAN and SinGAN dataset, and (v) the rotation-based dataset. However, the results show that the original dataset achieved the highest accuracy, 73%. Additionally, SinGAN outperformed DCGAN by a significant margin of 4%. In contrast, experimenting with the non-augmented original dataset resulted in the highest classification loss value, which explains the effect of the imbalance issue. These results provide a general view of the effect of different image augmentation techniques on enlarging the healthy brain dataset.

Identification of traffic signs for advanced driving assistance systems in smart cities using deep learning

The ability of Advanced Driving Assistance Systems (ADAS) is to identify and understand all objects around the vehicle under varying driving conditions and environmental factors is critical. Today's vehicles are equipped with advanced driving assistance systems that make driving safer and more comfortable. A camera mounted on the car helps the system recognise and detect traffic signs and alerts the driver about various road conditions, like if construction work is ahead or if speed limits have changed. The goal is to identify the traffic sign and process the image in a minimal processing time. A custom convolutional neural network model is used to classify the traffic signs with higher accuracy than the existing models. Image augmentation techniques are used to expand the dataset artificially, and that allows one to learn how the image looks from different perspectives, such as when viewed from different angles or when it looks blurry due to poor weather conditions. The algorithms used to detect traffic signs are YOLO v3 and YOLO v4-tiny. The proposed solution for detecting a specific set of traffic signs performed well, with an accuracy rate of 95.85%.

SCovNet: A skip connection-based feature union deep learning technique with statistical approach analysis for the detection of COVID-19

Background and Objective

The global population has been heavily impacted by the COVID-19 pandemic of coronavirus. Infections are spreading quickly around the world, and new spikes (Delta, Delta Plus, and Omicron) are still being made. The real-time reverse transcription-polymerase chain reaction (RT-PCR) is the method most often used to find viral RNA in a nasopharyngeal swab. However, these diagnostic approaches require human involvement and consume more time per prediction. Moreover, the existing conventional test mainly suffers from false negatives, so there is a chance for the virus to spread quickly. Therefore, a rapid and early diagnosis of COVID-19 patients is needed to overcome these problems.

Methods

Existing approaches based on deep learning for COVID detection are suffering from unbalanced datasets, poor performance, and gradient vanishing problems. A customized skip connection-based network with a feature union approach has been developed in this work to overcome some of the issues mentioned above. Gradient information from chest X-ray (CXR) images to subsequent layers is bypassed through skip connections. In the script's title, "SCovNet" refers to a skip-connection-based feature union network for detecting COVID-19 in a short notation. The performance of the proposed model was tested with two publicly available CXR image databases, including balanced and unbalanced datasets.

Results

A modified skip connection-based CNN model was suggested for a small unbalanced dataset (Kaggle) and achieved remarkable performance. In addition, the proposed model was also tested with a large GitHub database of CXR images and obtained an overall best accuracy of 98.67% with an impressive low false-negative rate of 0.0074.

Conclusions

The results of the experiments show that the proposed method works better than current methods at finding early signs of COVID-19. As an additional point of interest, we must mention the innovative hierarchical classification strategy provided for this work, which considered both balanced and unbalanced datasets to get the best COVID-19 identification rate.

A combined deformable model and medical transformer algorithm for medical image segmentation

Deep learning-based segmentation models usually require substantial data, and the model usually suffers from poor generalization due to the lack of training data and inefficient network structure. We proposed to combine the deformable model and medical transformer neural network on the image segmentation task to alleviate the aforementioned problems. The proposed method first employs a statistical shape model to generate simulated contours of the target object, and then the thin plate spline is applied to create a realistic texture. Finally, a medical transformer network was constructed to segment three types of medical images, including prostate MR image, heart US image, and tongue color images. The segmentation accuracy of the three tasks achieved 89.97%, 91.90%, and 94.25%, respectively. The experimental results show that the proposed method improves medical image segmentation performance.

Automated urinary sediment detection for Fabry disease using deep-learning algorithms

Fabry disease is a congenital lysosomal storage disease, and most of these cases develop organ damage in middle age. There are some promising therapeutic options for this disorder, which can stabilize the progression of the disease. However, a long delay in diagnosis prevents early intervention, resulting in treatment failure. Because Fabry disease is a rare disease, it is not well recognized and disease specific screening tests are rarely performed. Hence, a novel approach to for detecting patients with a widely practiced clinical test is crucial for the early detection of the disease. Recently, decision support systems based on artificial intelligence (AI) have been developed in many clinical fields. However, the construction of these models requires datasets from a large number of samples; this aspect is one of the main obstacles in AI-based approaches for rare diseases. In this study, with a novel image amplification method to construct the dataset for AI-model training, we built the deep neural-network model to detect Fabry cases from their urine samples. Sensitivity, specificity, and the AUC of the models on validation dataset were 0.902 (95% CI, 0.900–0.903), 0.977 (0.950–0.980), and 0.968 (0.964–0.972), respectively. This model could also extract disease-specific findings that are interpretable with human recognition. These results indicate that we can apply novel AI models for rare diseases based on this image amplification method we developed. We expect this approach could contribute to the diagnosis of Fabry disease.

Synopsis

This is the first reported AI-based decision support system to detect undiagnosed Fabry cases, and our new image amplification method will contribute to the AI models for other rare disorders.

Texture-less surface reconstruction using shape-based image augmentation

The development of intelligent Robot-Assisted Minimally Invasive Surgery demands geometric reconstruction from endoscopic images. However, images of human tissue surfaces are commonly texture-less. Obtaining the dense depth map of a texture-less scene is very difficult because traditional feature-based 3D reconstruction methods cannot detect enough features to build dense correspondences for depth computation. Given this problem, this study proposes a novel reconstruction method based on our shape-based image augmentation method. The main contribution of this method is the provision of a novel means to resolve the texture-less problem mainly on the input data level. In our method, we first calculate two shape gradient maps using Shape-From-Shading (SFS) method and we build Fast Point Feature Histogram (FPFH) 3D descriptor map according to the shape. Second, a series of augmented images can be computed by combining shape gradient maps, FPFH map, and the original image with different weights. Finally, we detect features on the new augmented images. Based on feature calculated sparse depth information and SFS calculated dense shape information, we further integrate a rectified dense depth map. Experiments show that our method can reconstruct texture-less surfaces with good accuracy.

A dataset of fortunella margarita images for object detection of deep learning based methods

Crops require appropriate planting techniques at different growth stages. Judgments on crop maturity affect the yield of crops. The planting and management of crops rely heavily on experienced farmers, which can reduce planting costs and increase yields. With the advancement of smart agriculture [1], images of crops can be used to accurately determine the growth stage of crops and estimate crop yields [2]. This can be combined with drones or smartphones to predict the growth stage and yield of Fortunella margarita for farmers in the future. This article presents an F. margarita image dataset. We classified F. margarita into three growth stages: mature, immature, and growing. In this dataset, an image may contain plants in several growth stages. The images were divided into seven categories according to growth stage. The dataset contains a total of 1031 original images. The total number of images was increased to 6611 through data augmentation. In addition, the dataset includes 6611 annotations with 7 categories of manually marked positions of F. margarita. Field images were captured in Jiaoxi, Yilan County, Taiwan, using smartphones. The dataset can serve as a resource for researchers who use different algorithms of machine learning or deep learning for object detection, image segmentation, and multiclass classification.

A comprehensive survey of recent trends in deep learning for digital images augmentation

Deep learning proved its efficiency in many fields of computer science such as computer vision, image classifications, object detection, image segmentation, and more. Deep learning models primarily depend on the availability of huge datasets. Without the existence of many images in datasets, different deep learning models will not be able to learn and produce accurate models. Unfortunately, several fields don't have access to large amounts of evidence, such as medical image processing. For example. The world is suffering from the lack of COVID-19 virus datasets, and there is no benchmark dataset from the beginning of 2020. This pandemic was the main motivation of this survey to deliver and discuss the current image data augmentation techniques which can be used to increase the number of images. In this paper, a survey of data augmentation for digital images in deep learning will be presented. The study begins and with the introduction section, which reflects the importance of data augmentation in general. The classical image data augmentation taxonomy and photometric transformation will be presented in the second section. The third section will illustrate the deep learning image data augmentation. Finally, the fourth section will survey the state of the art of using image data augmentation techniques in the different deep learning research and application.

Hierarchical CNN-based occlusal surface morphology analysis for classifying posterior tooth type using augmented images from 3D dental surface models

OBJECTIVE

3D Digitization of dental model is growing in popularity for dental application. Classification of tooth type from single 3D point cloud model without assist of relative position among teeth is still a challenging task.

METHODS

In this paper, 8-class posterior tooth type classification (first premolar, second premolar, first molar, second molar in maxilla and mandible respectively) was investigated by convolutional neural network (CNN)-based occlusal surface morphology analysis. 3D occlusal surface was transformed to depth image for basic CNN-based classification. Considering the logical hierarchy of tooth categories, a hierarchical classification structure was proposed to decompose 8-class classification task into two-stage cascaded classification subtasks. Image augmentations including traditional geometrical transformation and deep convolutional generative adversarial networks (DCGANs) were applied for each subnetworks and cascaded network.

RESULTS

Results indicate that combing traditional and DCGAN-based augmented images to train CNN models can improve classification performance. In the paper, we achieve overall accuracy 91.35%, macro precision 91.49%, macro-recall 91.29%, and macro-F1 0.9139 for the 8-class posterior tooth type classification, which outperform other deep learning models. Meanwhile, Grad-cam results demonstrate that CNN model trained by our augmented images will focus on smaller important region for better generality. And anatomic landmarks of cusp, fossa, and groove work as important regions for cascaded classification model.

CONCLUSION

The reported work has proved that using basic CNN to construct two-stage hierarchical structure can achieve the best classification performance of posterior tooth type in 3D model without assistance of relative position information. The proposed method has advantages of easy training, great ability to learn discriminative features from small image region.

HistoClean: Open-source software for histological image pre-processing and augmentation to improve development of robust convolutional neural networks

The growth of digital pathology over the past decade has opened new research pathways and insights in cancer prediction and prognosis. In particular, there has been a surge in deep learning and computer vision techniques to analyse digital images. Common practice in this area is to use image pre-processing and augmentation to prevent bias and overfitting, creating a more robust deep learning model. This generally requires consultation of documentation for multiple coding libraries, as well as trial and error to ensure that the techniques used on the images are appropriate. Herein we introduce HistoClean; a user-friendly, graphical user interface that brings together multiple image processing modules into one easy to use toolkit. HistoClean is an application that aims to help bridge the knowledge gap between pathologists, biomedical scientists and computer scientists by providing transparent image augmentation and pre-processing techniques which can be applied without prior coding knowledge. In this study, we utilise HistoClean to pre-process images for a simple convolutional neural network used to detect stromal maturity, improving the accuracy of the model at a tile, region of interest, and patient level. This study demonstrates how HistoClean can be used to improve a standard deep learning workflow via classical image augmentation and pre-processing techniques, even with a relatively simple convolutional neural network architecture. HistoClean is free and open-source and can be downloaded from the Github repository here: https://github.com/HistoCleanQUB/HistoClean.

Exploring partial intrinsic and extrinsic symmetry in 3D medical imaging

We present a novel methodology to detect imperfect bilateral symmetry in CT of human anatomy. In this paper, the structurally symmetric nature of the pelvic bone is explored and is used to provide interventional image augmentation for treatment of unilateral fractures in patients with traumatic injuries. The mathematical basis of our solution is based on the incorporation of attributes and characteristics that satisfy the properties of intrinsic and extrinsic symmetry and are robust to outliers. In the first step, feature points that satisfy intrinsic symmetry are automatically detected in the Möbius space defined on the CT data. These features are then pruned via a two-stage RANSAC to attain correspondences that satisfy also the extrinsic symmetry. Then, a disparity function based on Tukey's biweight robust estimator is introduced and minimized to identify a symmetry plane parametrization that yields maximum contralateral similarity. Finally, a novel regularization term is introduced to enhance similarity between bone density histograms across the partial symmetry plane, relying on the important biological observation that, even if injured, the dislocated bone segments remain within the body. Our extensive evaluations on various cases of common fracture types demonstrate the validity of the novel concepts and the accuracy of the proposed method.

Data augmentation based malware detection using convolutional neural networks

Due to advancements in malware competencies, cyber-attacks have been broadly observed in the digital world. Cyber-attacks can hit an organization hard by causing several damages such as data breach, financial loss, and reputation loss. Some of the most prominent examples of ransomware attacks in history are WannaCry and Petya, which impacted companies' finances throughout the globe. Both WannaCry and Petya caused operational processes inoperable by targeting critical infrastructure. It is quite impossible for anti-virus applications using traditional signature-based methods to detect this type of malware because they have different characteristics on each contaminated computer. The most important feature of this type of malware is that they change their contents using their mutation engines to create another hash representation of the executable file as they propagate from one computer to another. To overcome this method that attackers use to camouflage malware, we have created three-channel image files of malicious software. Attackers make different variants of the same software because they modify the contents of the malware. In the solution to this problem, we created variants of the images by applying data augmentation methods. This article aims to provide an image augmentation enhanced deep convolutional neural network (CNN) models for detecting malware families in a metamorphic malware environment. The main contributions of the article consist of three components, including image generation from malware samples, image augmentation, and the last one is classifying the malware families by using a CNN model. In the first component, the collected malware samples are converted into binary file to 3-channel images using the windowing technique. The second component of the system create the augmented version of the images, and the last part builds a classification model. This study uses five different deep CNN model for malware family detection. The results obtained by the classifier demonstrate accuracy up to 98%, which is quite satisfactory.

Analysis and best parameters selection for person recognition based on gait model using CNN algorithm and image augmentation

Person Recognition based on Gait Model (PRGM) and motion features is are indeed a challenging and novel task due to their usages and to the critical issues of human pose variation, human body occlusion, camera view variation, etc. In this project, a deep convolution neural network (CNN) was modified and adapted for person recognition with Image Augmentation (IA) technique depending on gait features. Adaptation aims to get best values for CNN parameters to get best CNN model. In Addition to the CNN parameters Adaptation, the design of CNN model itself was adapted to get best model structure; Adaptation in the design was affected the type, the number of layers in CNN and normalization between them. After choosing best parameters and best design, Image augmentation was used to increase the size of train dataset with many copies of the image to boost the number of different images that will be used to train Deep learning algorithms. The tests were achieved using known dataset (Market dataset). The dataset contains sequential pictures of people in different gait status. The image in CNN model as matrix is extracted to many images or matrices by the convolution, so dataset size may be bigger by hundred times to make the problem a big data issue. In this project, results show that adaptation has improved the accuracy of person recognition using gait model comparing to model without adaptation. In addition, dataset contains images of person carrying things. IA technique improved the model to be robust to some variations such as image dimensions (quality and resolution), rotations and carried things by persons. Results for 200 persons recognition, validation accuracy was about 82% without IA and 96.23 with IA. For 800 persons recognition, validation accuracy was 93.62% without IA.