Detection and Segmentation of Manufacturing Defects with Convolutional Neural Networks and Transfer Learning

Deep convolutional neural network for weld defect classification in radiographic images

The quality of welds is critical to the safety of structures in construction, so early detection of irregularities is crucial. Advances in machine vision inspection technologies, such as deep learning models, have improved the detection of weld defects. This paper presents a new CNN model based on ResNet50 to classify four types of weld defects in radiographic images: crack, pore, non-penetration, and no defect. Stratified cross-validation, data augmentation, and regularization were used to improve generalization and avoid over-fitting. The model was tested on three datasets, RIAWELC, GDXray, and a private dataset of low image quality, obtaining an accuracy of 98.75 %, 90.255 %, and 75.83 %, respectively. The model proposed in this paper achieves high accuracies on different datasets and constitutes a valuable tool to improve the efficiency and effectiveness of quality control processes in the welding industry. Moreover, experimental tests show that the proposed approach performs well on even low-resolution images.

Semantic segmentation of defects based on DCNN and its application on fatigue lifetime prediction for SLM Ti-6Al-4V alloy

Inevitable defects have an important impact on the fatigue lifetime of additive manufacturing materials. Therefore, it is critical to thoroughly characterize the characteristics of the defects, which requires effective semantic segmentation of the imaged defects. In this paper, a defect dataset for SLM Ti-6Al-4V alloy was obtained by synchrotron radiation computed tomography. Then a semantic segmentation method was developed based on the DeepLabV3 + network to automatically extract defects. Cropping and undersampling were introduced in the dataset pre-processing. A weighted scheme based on the ratio between the number of defect and matrix pixels was applied in the classification layer, and morphological operations were employed in image post-processing to improve the accuracy of identifying small-target defect. Finally, the above method was applied to segment the X-ray computed tomography data for two batches of SLM Ti-6Al-4V materials, and the defect segmentation results were used to predict the fatigue lifetime. The semantic segmentation method performs well with a pixel recognition accuracy of 98.2% for the test dataset, and the error in the predicted fatigue lifetime lies within the scatter band of ±2.2. This article is part of the theme issue 'Physics-informed machine learning and its structural integrity applications (Part 2)'.

An Improved Mask R-CNN Micro-Crack Detection Model for the Surface of Metal Structural Parts

Micro-crack detection is an essential task in critical equipment health monitoring. Accurate and timely detection of micro-cracks can ensure the healthy and stable service of equipment. Aiming at improving the low accuracy of the conventional target detection model during the task of detecting micro-cracks on the surface of metal structural parts, this paper built a micro-cracks dataset and explored a detection performance optimization method based on Mask R-CNN. Firstly, we improved the original FPN structure, adding a bottom-up feature fusion path to enhance the information utilization rate of the underlying feature layer. Secondly, we added the methods of deformable convolution kernel and attention mechanism to ResNet, which can improve the efficiency of feature extraction. Lastly, we modified the original loss function to optimize the network training effect and model convergence rate. The ablation comparison experiments shows that all the improvement schemes proposed in this paper have improved the performance of the original Mask R-CNN. The integration of all the improvement schemes can produce the most significant performance improvement effects in recognition, classification, and positioning simultaneously, thus proving the rationality and feasibility of the improved scheme in this paper.

Analysis of the Possibility of Using Selected Tools and Algorithms in the Classification and Recognition of Type of Microstructure

The aim of this research was to develop a solution based on existing methods and tools that would allow the automatic classification of selected images of cast iron microstructures. As part of the work, solutions based on artificial intelligence were tested and modified. Their task is to assign a specific class in the analyzed microstructure images. In the analyzed set, the examined samples appear in various zoom levels, photo sizes and colors. As is known, the components of the microstructure are different. In the examined photo, there does not have to be only one type of precipitate in each photo that indicates the correct microstructure of the same type of alloy, different shapes may appear in different amounts. This article also addresses the issue of data preparation. In order to isolate one type of structure element, the possibilities of using methods such as HOG (histogram of oriented gradients) and thresholding (the image was transformed into black objects on a white background) were checked. In order to avoid the slow preparation of training data, our solution was proposed to facilitate the labeling of data for training. The HOG algorithm combined with SVM and random forest were used for the classification process. In order to compare the effectiveness of the operation, the Faster R-CNN and Mask R-CNN algorithms were also used. The results obtained from the classifiers were compared to the microstructure assessment performed by experts.

A Systematic Review on Deep Learning with CNNs Applied to Surface Defect Detection

Surface defect detection with machine learning has become an important tool in industries and a large field of study for researchers or workers in recent years. It is necessary to have a simplified source of information that helps us to better focus on one type of surface. In this systematic review, we present a classification for surface defect detection based on convolutional neural networks (CNNs) focused on surface types. Findings: Out of 253 records identified, 59 primary studies were eligible. Following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, we analyzed the structures of each study and the concepts related to defects and their types on surfaces. The presented review is mainly focused on finding a classification for the types of surfaces most used in industry (metal, building, ceramic, wood, and special). We delve into the specifics of each surface category, offering illustrative examples of their applications within both industrial and laboratory settings. Furthermore, we propose a new taxonomy of machine learning based on the obtained results and collected information. We summarized the studies and extracted the main characteristics such as type of surface, problem types, timeline, type of network, techniques, and datasets. Among the most relevant results of our analysis, we found that the metallic surface is the most used, as it is the one found in 62.71% of the studies, and the most prevalent problem type is classification, accounting for 49.15% of the total. Furthermore, we observe that transfer learning was employed in 83.05% of the studies, while data augmentation was utilized in 59.32%. Our findings also provide insights into the cameras most frequently employed, along with the strategies adopted to address illumination challenges present in certain articles and the approach to creating datasets for real-world applications. The main results presented in this review allow for a quick and efficient search of information for researchers and professionals interested in improving the results of their defect detection projects. Finally, we analyzed the trends that could open new fields of study for future research in the area of surface defect detection.

An Actinic Keratosis Auxiliary Diagnosis Method Based on an Enhanced MobileNet Model

Actinic keratosis (AK) is a common precancerous skin lesion with significant harm, and it is often confused with non-actinic keratoses (NAK). At present, the diagnosis of AK mainly depends on clinical experience and histopathology. Due to the high difficulty of diagnosis and easy confusion with other diseases, this article aims to develop a convolutional neural network that can efficiently, accurately, and automatically diagnose AK. This article improves the MobileNet model and uses the AK and NAK images in the HAM10000 dataset for training and testing after data preprocessing, and we performed external independent testing using a separate dataset to validate our preprocessing approach and to demonstrate the performance and generalization capability of our model. It further compares common deep learning models in the field of skin diseases (including the original MobileNet, ResNet, GoogleNet, EfficientNet, and Xception). The results show that the improved MobileNet has achieved 0.9265 in accuracy and 0.97 in Area Under the ROC Curve (AUC), which is the best among the comparison models. At the same time, it has the shortest training time, and the total time of five-fold cross-validation on local devices only takes 821.7 s. Local experiments show that the method proposed in this article has high accuracy and stability in diagnosing AK. Our method will help doctors diagnose AK more efficiently and accurately, allowing patients to receive timely diagnosis and treatment.

Research of U-Net-Based CNN Architectures for Metal Surface Defect Detection

The quality, wear and safety of metal structures can be controlled effectively, provided that surface defects, which occur on metal structures, are detected at the right time. Over the past 10 years, researchers have proposed a number of neural network architectures that have shown high efficiency in various areas, including image classification, segmentation and recognition. However, choosing the best architecture for this particular task is often problematic. In order to compare various techniques for detecting defects such as “scratch abrasion”, we created and investigated U-Net-like architectures with encoders such as ResNet, SEResNet, SEResNeXt, DenseNet, InceptionV3, Inception-ResNetV2, MobileNet and EfficientNet. The relationship between training validation metrics and final segmentation test metrics was investigated. The correlation between the loss function, the , , , and validation metrics and test metrics was calculated. Recognition accuracy was analyzed as affected by the optimizer during neural network training. In the context of this problem, neural networks trained using the stochastic gradient descent optimizer with Nesterov momentum were found to have the best generalizing properties. To select the best model during its training on the basis of the validation metrics, the main test metrics of recognition quality (Dice similarity coefficient) were analyzed depending on the validation metrics. The ResNet and DenseNet models were found to achieve the best generalizing properties for our task. The highest recognition accuracy was attained using the U-Net model with a ResNet152 backbone. The results obtained on the test dataset were and .

Imbalance Modelling for Defect Detection in Ceramic Substrate by Using Convolutional Neural Network

The complexity of defect detection in a ceramic substrate causes interclass and intraclass imbalance problems. Identifying flaws in ceramic substrates has traditionally relied on aberrant material occurrences and characteristic quantities. However, defect substrates in ceramic are typically small and have a wide variety of defect distributions, thereby making defect detection more challenging and difficult. Thus, we propose a method for defect detection based on unsupervised learning and deep learning. First, the proposed method conducts K-means clustering for grouping instances according to their inherent complex characteristics. Second, the distribution of rarely occurring instances is balanced by using augmentation filters. Finally, a convolutional neural network is trained by using the balanced dataset. The effectiveness of the proposed method was validated by comparing the results with those of other methods. Experimental results show that the proposed method outperforms other methods.

Industrial Laser Welding Defect Detection and Image Defect Recognition Based on Deep Learning Model Developed

Aiming at the problem of the poor robustness of existing methods to deal with diverse industrial weld image data, we collected a series of asymmetric laser weld images in the largest laser equipment workshop in Asia, and studied these data based on an industrial image processing algorithm and deep learning algorithm. The median filter was used to remove the noises in weld images. The image enhancement technique was adopted to increase the image contrast in different areas. The deep convolutional neural network (CNN) was employed for feature extraction; the activation function and the adaptive pooling approach were improved. Transfer Learning (TL) was introduced for defect detection and image classification on the dataset. Finally, a deep learning-based model was constructed for weld defect detection and image recognition. Specific instance datasets verified the model’s performance. The results demonstrate that this model can accurately identify weld defects and eliminate the complexity of manually extracting features, reaching a recognition accuracy of 98.75%. Hence, the reliability and automation of detection and recognition are improved significantly. The research results can provide a theoretical and practical reference for the defect detection of sheet metal laser welding and the development of the industrial laser manufacturing industry.

Recognition Rate Advancement and Data Error Improvement of Pathology Cutting with H-DenseUNet for Hepatocellular Carcinoma Image

Due to the fact that previous studies have rarely investigated the recognition rate discrepancy and pathology data error when applied to different databases, the purpose of this study is to investigate the improvement of recognition rate via deep learning-based liver lesion segmentation with the incorporation of hospital data. The recognition model used in this study is H-DenseUNet, which is applied to the segmentation of the liver and lesions, and a mixture of 2D/3D Hybrid-DenseUNet is used to reduce the recognition time and system memory requirements. Differences in recognition results were determined by comparing the training files of the standard LiTS competition data set with the training set after mixing in an additional 30 patients. The average error value of 9.6% was obtained by comparing the data discrepancy between the actual pathology data and the pathology data after the analysis of the identified images imported from Kaohsiung Chang Gung Memorial Hospital. The average error rate of the recognition output after mixing the LiTS database with hospital data for training was 1%. In the recognition part, the Dice coefficient was 0.52 after training 50 epochs using the standard LiTS database, while the Dice coefficient was increased to 0.61 after adding 30 hospital data to the training. After importing 3D Slice and ITK-Snap software, a 3D image of the lesion and liver segmentation can be developed. It is hoped that this method could be used to stimulate more research in addition to the general public standard database in the future, as well as to study the applicability of hospital data and improve the generality of the database.

StyleGANs and Transfer Learning for Generating Synthetic Images in Industrial Applications

Deep learning applications on computer vision involve the use of large-volume and representative data to obtain state-of-the-art results due to the massive number of parameters to optimise in deep models. However, data are limited with asymmetric distributions in industrial applications due to rare cases, legal restrictions, and high image-acquisition costs. Data augmentation based on deep learning generative adversarial networks, such as StyleGAN, has arisen as a way to create training data with symmetric distributions that may improve the generalisation capability of built models. StyleGAN generates highly realistic images in a variety of domains as a data augmentation strategy but requires a large amount of data to build image generators. Thus, transfer learning in conjunction with generative models are used to build models with small datasets. However, there are no reports on the impact of pre-trained generative models, using transfer learning. In this paper, we evaluate a StyleGAN generative model with transfer learning on different application domains—training with paintings, portraits, Pokémon, bedrooms, and cats—to generate target images with different levels of content variability: bean seeds (low variability), faces of subjects between 5 and 19 years old (medium variability), and charcoal (high variability). We used the first version of StyleGAN due to the large number of publicly available pre-trained models. The Fréchet Inception Distance was used for evaluating the quality of synthetic images. We found that StyleGAN with transfer learning produced good quality images, being an alternative for generating realistic synthetic images in the evaluated domains.

Quality Control of PET Bottles Caps with Dedicated Image Calibration and Deep Neural Networks

Product quality control is currently the leading trend in industrial production. It is heading towards the exact analysis of each product before reaching the end customer. Every stage of production control is of particular importance in the food and pharmaceutical industries, where, apart from visual issues, additional safety regulations are demanded. Many production processes can be controlled completely contactless through the use of machine vision cameras and advanced image processing techniques. The most dynamically growing sector of image analysis methods are solutions based on deep neural networks. Their major advantages are fast performance, robustness, and the fact that they can be exploited even in complicated classification problems. However, the use of machine learning methods on high-performance production lines may be limited by inference time or, in the case of multiformated production lines, training time. The article presents a novel data preprocessing (or calibration) method. It uses prior knowledge about the optical system, which enables the use of the lightweight Convolutional Neural Network (CNN) model for product quality control of polyethylene terephthalate (PET) bottle caps. The combination of preprocessing with the lightweight CNN model resulted in at least a five-fold reduction in prediction and training time compared to the lighter standard models tested on ImageNet, without loss of accuracy.

A Standardized PMML Format for Representing Convolutional Neural Networks with Application to Defect Detection

Convolutional neural networks are becoming a popular tool for image processing in the engineering and manufacturing sectors. However, managing the storage and distribution of trained models is still a difficult task, partially due to the lack of standardized methods for deep neural network representation. Additionally, the interoperability between different machine learning frameworks remains poor. This paper seeks to address this issue by proposing a standardized format for convolutional neural networks, based on the Predictive Model Markup Language (PMML). A new standardized schema is proposed to represent a range of convolutional neural networks, including classification, regression and semantic segmentation systems. To demonstrate the practical application of this standard, a semantic segmentation model, which is trained to detect casting defects in Xray images, is represented in the proposed PMML format. A high-performance scoring engine is developed to evaluate images and videos against the PMML model. The utility of the proposed format and the scoring engine is evaluated by benchmarking the performance of the defect detection models on a range of different computational platforms.