1
|
Cumbajin E, Rodrigues N, Costa P, Miragaia R, Frazão L, Costa N, Fernández-Caballero A, Carneiro J, Buruberri LH, Pereira A. A Systematic Review on Deep Learning with CNNs Applied to Surface Defect Detection. J Imaging 2023; 9:193. [PMID: 37888300 PMCID: PMC10607335 DOI: 10.3390/jimaging9100193] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2023] [Revised: 08/29/2023] [Accepted: 09/18/2023] [Indexed: 10/28/2023] Open
Abstract
Surface defect detection with machine learning has become an important tool in industries and a large field of study for researchers or workers in recent years. It is necessary to have a simplified source of information that helps us to better focus on one type of surface. In this systematic review, we present a classification for surface defect detection based on convolutional neural networks (CNNs) focused on surface types. Findings: Out of 253 records identified, 59 primary studies were eligible. Following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, we analyzed the structures of each study and the concepts related to defects and their types on surfaces. The presented review is mainly focused on finding a classification for the types of surfaces most used in industry (metal, building, ceramic, wood, and special). We delve into the specifics of each surface category, offering illustrative examples of their applications within both industrial and laboratory settings. Furthermore, we propose a new taxonomy of machine learning based on the obtained results and collected information. We summarized the studies and extracted the main characteristics such as type of surface, problem types, timeline, type of network, techniques, and datasets. Among the most relevant results of our analysis, we found that the metallic surface is the most used, as it is the one found in 62.71% of the studies, and the most prevalent problem type is classification, accounting for 49.15% of the total. Furthermore, we observe that transfer learning was employed in 83.05% of the studies, while data augmentation was utilized in 59.32%. Our findings also provide insights into the cameras most frequently employed, along with the strategies adopted to address illumination challenges present in certain articles and the approach to creating datasets for real-world applications. The main results presented in this review allow for a quick and efficient search of information for researchers and professionals interested in improving the results of their defect detection projects. Finally, we analyzed the trends that could open new fields of study for future research in the area of surface defect detection.
Collapse
Affiliation(s)
- Esteban Cumbajin
- Computer Science and Communications Research Centre, School of Technology and Management, Polytechnic of Leiria, 2411-901 Leiria, Portugal; (E.C.); (N.R.); (P.C.); (R.M.); (L.F.); (N.C.)
| | - Nuno Rodrigues
- Computer Science and Communications Research Centre, School of Technology and Management, Polytechnic of Leiria, 2411-901 Leiria, Portugal; (E.C.); (N.R.); (P.C.); (R.M.); (L.F.); (N.C.)
| | - Paulo Costa
- Computer Science and Communications Research Centre, School of Technology and Management, Polytechnic of Leiria, 2411-901 Leiria, Portugal; (E.C.); (N.R.); (P.C.); (R.M.); (L.F.); (N.C.)
| | - Rolando Miragaia
- Computer Science and Communications Research Centre, School of Technology and Management, Polytechnic of Leiria, 2411-901 Leiria, Portugal; (E.C.); (N.R.); (P.C.); (R.M.); (L.F.); (N.C.)
| | - Luís Frazão
- Computer Science and Communications Research Centre, School of Technology and Management, Polytechnic of Leiria, 2411-901 Leiria, Portugal; (E.C.); (N.R.); (P.C.); (R.M.); (L.F.); (N.C.)
| | - Nuno Costa
- Computer Science and Communications Research Centre, School of Technology and Management, Polytechnic of Leiria, 2411-901 Leiria, Portugal; (E.C.); (N.R.); (P.C.); (R.M.); (L.F.); (N.C.)
| | - Antonio Fernández-Caballero
- Instituto de Investigación en Informática de Albacete, 02071 Albacete, Spain
- Departamento de Sistemas Informáticos, Universidad de Castilla-La Mancha, 02071 Albacete, Spain
| | - Jorge Carneiro
- Grestel-Produtos Cerâmicos S.A, Zona Industrial de Vagos-Lote 78, 3840-385 Vagos, Portugal; (J.C.); (L.H.B.)
| | - Leire H. Buruberri
- Grestel-Produtos Cerâmicos S.A, Zona Industrial de Vagos-Lote 78, 3840-385 Vagos, Portugal; (J.C.); (L.H.B.)
| | - António Pereira
- Computer Science and Communications Research Centre, School of Technology and Management, Polytechnic of Leiria, 2411-901 Leiria, Portugal; (E.C.); (N.R.); (P.C.); (R.M.); (L.F.); (N.C.)
- INOV INESC Inovação, Institute of New Technologies, Leiria Office, 2411-901 Leiria, Portugal
| |
Collapse
|
2
|
Jiang Y, Cai M, Zhang D. Lightweight Network DCR-YOLO for Surface Defect Detection on Printed Circuit Boards. SENSORS (BASEL, SWITZERLAND) 2023; 23:7310. [PMID: 37687766 PMCID: PMC10490101 DOI: 10.3390/s23177310] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/10/2023] [Revised: 08/09/2023] [Accepted: 08/13/2023] [Indexed: 09/10/2023]
Abstract
To resolve the problems associated with the small target presented by printed circuit board surface defects and the low detection accuracy of these defects, the printed circuit board surface-defect detection network DCR-YOLO is designed to meet the premise of real-time detection speed and effectively improve the detection accuracy. Firstly, the backbone feature extraction network DCR-backbone, which consists of two CR residual blocks and one common residual block, is used for small-target defect extraction on printed circuit boards. Secondly, the SDDT-FPN feature fusion module is responsible for the fusion of high-level features to low-level features while enhancing feature fusion for the feature fusion layer, where the small-target prediction head YOLO Head-P3 is located, to further enhance the low-level feature representation. The PCR module enhances the feature fusion mechanism between the backbone feature extraction network and the SDDT-FPN feature fusion module at different scales of feature layers. The C5ECA module is responsible for adaptive adjustment of feature weights and adaptive attention to the requirements of small-target defect information, further enhancing the adaptive feature extraction capability of the feature fusion module. Finally, three YOLO-Heads are responsible for predicting small-target defects for different scales. Experiments show that the DCR-YOLO network model detection map reaches 98.58%; the model size is 7.73 MB, which meets the lightweight requirement; and the detection speed reaches 103.15 fps, which meets the application requirements for real-time detection of small-target defects.
Collapse
Affiliation(s)
- Yuanyuan Jiang
- School of Electrical and Information Engineering, Anhui University of Science and Technology, Huainan 232000, China; (Y.J.); (D.Z.)
- Institute of Environment-Friendly Materials and Occupational Health, Anhui University of Science and Technology, Wuhu 241003, China
| | - Mengnan Cai
- School of Electrical and Information Engineering, Anhui University of Science and Technology, Huainan 232000, China; (Y.J.); (D.Z.)
| | - Dong Zhang
- School of Electrical and Information Engineering, Anhui University of Science and Technology, Huainan 232000, China; (Y.J.); (D.Z.)
| |
Collapse
|