A High-Quality Sample Generation Method for Improving Steel Surface Defect Inspection

Defect inspection is a critical task in ensuring the surface quality of steel plates. Deep neural networks have the potential to achieve excellent inspection accuracy if defect samples are sufficient. Nevertheless, it is very different to collect enough samples using cameras alone. To a certain extent, generative models can alleviate this problem but poor sample quality can greatly affect the final inspection performance. A sample generation method, which employs a generative adversarial network (GAN), is proposed to generate high-quality defect samples for training accurate inspection models. To improve generation quality, we propose a production-and-elimination, two-stage sample generation process by simulating the formation of defects on the surface of steel plates. The production stage learns to generate defects on defect-free background samples, and the elimination stage learns to erase defects on defective samples. By minimizing the differences between the samples at both stages, the proposed model can make generated background samples close to real ones while guiding the generated defect samples to be more realistic. Experimental results show that the proposed method has the ability to generate high-quality samples that can help train powerful inspection models and thereby improve inspection performance.

Structural Design and Simulation of a Multi-Channel and Dual Working Condition Wafer Defect Inspection Prototype

Detecting and classifying defects on unpatterned wafers is a key part of wafer front-end inspection. Defect inspection schemes vary depending on the type and location of the defects. In this paper, the structure of the prototype is designed to meet the requirements of wafer surface and edge defect inspection. This prototype has four inspection channels: scattering, reflection, phase, and contour, with two working conditions: surface and edge inspection. The key structure of the prototype was simulated using Ansys. The simulation results show that the maximum deformation of the optical detection subsystem is 19.5 μm and the fundamental frequency of the prototype is 96.9 Hz; thus, these results meet the requirements of optical performance stability and structural design. The experimental results show that the prototype meets the requirements of the inspection sensitivity better than 200 nm equivalent PSL spherical defects.

Two-Stream Network One-Class Classification Model for Defect Inspections

Defect inspection is important to ensure consistent quality and efficiency in industrial manufacturing. Recently, machine vision systems integrating artificial intelligence (AI)-based inspection algorithms have exhibited promising performance in various applications, but practically, they often suffer from data imbalance. This paper proposes a defect inspection method using a one-class classification (OCC) model to deal with imbalanced datasets. A two-stream network architecture consisting of global and local feature extractor networks is presented, which can alleviate the representation collapse problem of OCC. By combining an object-oriented invariant feature vector with a training-data-oriented local feature vector, the proposed two-stream network model prevents the decision boundary from collapsing to the training dataset and obtains an appropriate decision boundary. The performance of the proposed model is demonstrated in the practical application of automotive-airbag bracket-welding defect inspection. The effects of the classification layer and two-stream network architecture on the overall inspection accuracy were clarified by using image samples collected in a controlled laboratory environment and from a production site. The results are compared with those of a previous classification model, demonstrating that the proposed model can improve the accuracy, precision, and F1 score by up to 8.19%, 10.74%, and 4.02%, respectively.

Defect Detection in Printed Circuit Boards Using Semi-Supervised Learning

Defect inspection is essential in the semiconductor industry to fabricate printed circuit boards (PCBs) with minimum defect rates. However, conventional inspection systems are labor-intensive and time-consuming. In this study, a semi-supervised learning (SSL)-based model called PCB_SS was developed. It was trained using labeled and unlabeled images under two different augmentations. Training and test PCB images were acquired using automatic final vision inspection systems. The PCB_SS model outperformed a completely supervised model trained using only labeled images (PCB_FS). The performance of the PCB_SS model was more robust than that of the PCB_FS model when the number of labeled data is limited or comprises incorrectly labeled data. In an error-resilience test, the proposed PCB_SS model maintained stable accuracy (error increment of less than 0.5%, compared with 4% for PCB_FS) for noisy training data (with as much as 9.0% of the data labeled incorrectly). The proposed model also showed superior performance when comparing machine-learning and deep-learning classifiers. The unlabeled data utilized in the PCB_SS model helped with the generalization of the deep-learning model and improved its performance for PCB defect detection. Thus, the proposed method alleviates the burden of the manual labeling process and provides a rapid and accurate automatic classifier for PCB inspections.

Analysis of Training Deep Learning Models for PCB Defect Detection

Recently, many companies have introduced automated defect detection methods for defect-free PCB manufacturing. In particular, deep learning-based image understanding methods are very widely used. In this study, we present an analysis of training deep learning models to perform PCB defect detection stably. To this end, we first summarize the characteristics of industrial images, such as PCB images. Then, the factors that can cause changes (contamination and quality degradation) to the image data in the industrial field are analyzed. Subsequently, we organize defect detection methods that can be applied according to the situation and purpose of PCB defect detection. In addition, we review the characteristics of each method in detail. Our experimental results demonstrated the impact of various degradation factors, such as defect detection methods, data quality, and image contamination. Based on our overview of PCB defect detection and experiment results, we present knowledge and guidelines for correct PCB defect detection.

Vision-Based Defect Inspection and Condition Assessment for Sewer Pipes: A Comprehensive Survey

Due to the advantages of economics, safety, and efficiency, vision-based analysis techniques have recently gained conspicuous advancements, enabling them to be extensively applied for autonomous constructions. Although numerous studies regarding the defect inspection and condition assessment in underground sewer pipelines have presently emerged, we still lack a thorough and comprehensive survey of the latest developments. This survey presents a systematical taxonomy of diverse sewer inspection algorithms, which are sorted into three categories that include defect classification, defect detection, and defect segmentation. After reviewing the related sewer defect inspection studies for the past 22 years, the main research trends are organized and discussed in detail according to the proposed technical taxonomy. In addition, different datasets and the evaluation metrics used in the cited literature are described and explained. Furthermore, the performances of the state-of-the-art methods are reported from the aspects of processing accuracy and speed.

Drain Structural Defect Detection and Mapping Using AI-Enabled Reconfigurable Robot Raptor and IoRT Framework

Human visual inspection of drains is laborious, time-consuming, and prone to accidents. This work presents an AI-enabled robot-assisted remote drain inspection and mapping framework using our in-house developed reconfigurable robot Raptor. The four-layer IoRT serves as a bridge between the users and the robots, through which seamless information sharing takes place. The Faster RCNN ResNet50, Faster RCNN ResNet101, and Faster RCNN Inception-ResNet-v2 deep learning frameworks were trained using a transfer learning scheme with six typical concrete defect classes and deployed in an IoRT framework remote defect detection task. The efficiency of the trained CNN algorithm and drain inspection robot Raptor was evaluated through various real-time drain inspection field trials using the SLAM technique. The experimental results indicate that robot’s maneuverability was stable, and its mapping and localization were also accurate in different drain types. Finally, for effective drain maintenance, the SLAM-based defect map was generated by fusing defect detection results in the lidar-SLAM map.

Product Inspection Methodology via Deep Learning: An Overview

In this study, we present a framework for product quality inspection based on deep learning techniques. First, we categorize several deep learning models that can be applied to product inspection systems. In addition, we explain the steps for building a deep-learning-based inspection system in detail. Second, we address connection schemes that efficiently link deep learning models to product inspection systems. Finally, we propose an effective method that can maintain and enhance a product inspection system according to improvement goals of the existing product inspection systems. The proposed system is observed to possess good system maintenance and stability owing to the proposed methods. All the proposed methods are integrated into a unified framework and we provide detailed explanations of each proposed method. In order to verify the effectiveness of the proposed system, we compare and analyze the performance of the methods in various test scenarios. We expect that our study will provide useful guidelines to readers who desire to implement deep-learning-based systems for product inspection.

A New Stitching Method for Dark-Field Surface Defects Inspection Based on Simplified Target-Tracking and Path Correction

A camera-based dark-field imaging system can effectively detect defects of microns on large optics by scanning and stitching sub-apertures with a small field of view. However, conventional stitching methods encounter problems of mismatches and location deviations, since few defects exist on the tested fine surface. In this paper, a highly efficient stitching method is proposed, based on a simplified target-tracking and adaptive scanning path correction. By increasing the number of sub-apertures and switching to camera perspective, the defects can be regarded as moving targets. A target-tracking procedure is firstly performed to obtain the marked targets. Then, the scanning path is corrected by minimizing the sum of deviations. The final stitching results are updated by re-using the target-tracking method. An experiment was carried out on an inspection of our specially designed testing sample. Subsequently, 118 defects were identified out of 120 truly existing defects, without stitching mismatches. The experiment results show that this method can help to reduce mismatches and location deviations of defects, and it was also effective in increasing the detectability for weak defects.

Sensing Sub-10 nm Wide Perturbations in Background Nanopatterns Using Optical Pseudoelectrodynamics Microscopy (OPEM)

Using light as a probe to investigate perturbations with deep subwavelength dimensions in large-scale wafers is challenging because of the diffraction limit and the weak Rayleigh scattering. In this Letter, we report on a nondestructive noninterference far-field imaging method, which is built upon electrodynamic principles (mechanical work and force) of the light-matter interaction, rather than the intrinsic properties of light. We demonstrate sensing of nanoscale perturbations with sub-10 nm features in semiconductor nanopatterns. This framework is implemented using a visible-light bright-field microscope with a broadband source and a through-focus scanning apparatus. This work creates a new paradigm for exploring light-matter interactions at the nanoscale using microscopy that can potentially be extended to many other problems, for example, bioimaging, material analysis, and nanometrology.

In-TFT-array-process micro defect inspection using nonlinear principal component analysis

Defect inspection plays a critical role in thin film transistor liquid crystal display (TFT-LCD) manufacture, and has received much attention in the field of automatic optical inspection (AOI). Previously, most focus was put on the problems of macro-scale Mura-defect detection in cell process, but it has recently been found that the defects which substantially influence the yield rate of LCD panels are actually those in the TFT array process, which is the first process in TFT-LCD manufacturing. Defect inspection in TFT array process is therefore considered a difficult task. This paper presents a novel inspection scheme based on kernel principal component analysis (KPCA) algorithm, which is a nonlinear version of the well-known PCA algorithm. The inspection scheme can not only detect the defects from the images captured from the surface of LCD panels, but also recognize the types of the detected defects automatically. Results, based on real images provided by a LCD manufacturer in Taiwan, indicate that the KPCA-based defect inspection scheme is able to achieve a defect detection rate of over 99% and a high defect classification rate of over 96% when the imbalanced support vector machine (ISVM) with 2-norm soft margin is employed as the classifier. More importantly, the inspection time is less than 1 s per input image.