1
|
Li B, Qin H, Xiong W, Li Y, Feng S, Hu W, Maybank S. Ranking-Based Color Constancy With Limited Training Samples. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2023; 45:12304-12320. [PMID: 37216258 DOI: 10.1109/tpami.2023.3278832] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/24/2023]
Abstract
Computational color constancy is an important component of Image Signal Processors (ISP) for white balancing in many imaging devices. Recently, deep convolutional neural networks (CNN) have been introduced for color constancy. They achieve prominent performance improvements comparing with those statistics or shallow learning-based methods. However, the need for a large number of training samples, a high computational cost and a huge model size make CNN-based methods unsuitable for deployment on low-resource ISPs for real-time applications. In order to overcome these limitations and to achieve comparable performance to CNN-based methods, an efficient method is defined for selecting the optimal simple statistics-based method (SM) for each image. To this end, we propose a novel ranking-based color constancy method (RCC) that formulates the selection of the optimal SM method as a label ranking problem. RCC designs a specific ranking loss function, and uses a low rank constraint to control the model complexity and a grouped sparse constraint for feature selection. Finally, we apply the RCC model to predict the order of the candidate SM methods for a test image, and then estimate its illumination using the predicted optimal SM method (or fusing the results estimated by the top k SM methods). Comprehensive experiment results show that the proposed RCC outperforms nearly all the shallow learning-based methods and achieves comparable performance to (sometimes even better performance than) deep CNN-based methods with only 1/2000 of the model size and training time. RCC also shows good robustness to limited training samples and good generalization crossing cameras. Furthermore, to remove the dependence on the ground truth illumination, we extend RCC to obtain a novel ranking-based method without ground truth illumination (RCC_NO) that learns the ranking model using simple partial binary preference annotations provided by untrained annotators rather than experts. RCC_NO also achieves better performance than the SM methods and most shallow learning-based methods with low costs of sample collection and illumination measurement.
Collapse
|
3
|
Wang Z, Yin D, Ji S. Real-time factory smoke detection based on two-stage relation-guided algorithm. Sci Rep 2022; 12:1786. [PMID: 35110591 PMCID: PMC8810845 DOI: 10.1038/s41598-022-05523-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2021] [Accepted: 01/13/2022] [Indexed: 11/09/2022] Open
Abstract
Recently, air quality analysis based on image sensing devices has attracted much attention. Since most smoke images in real scenes have challenging variances, which is difficult for existing object detection methods. To keep real-time factory smoke under efficient and universal social supervision, this paper proposes a mobile-platform-running efficient smoke detection algorithm based on image analysis techniques. We introduce the two-stage smoke detection (TSSD) algorithm based on the lightweight detection framework, in which the prior knowledge and contextual information are modeled into the relation-guided module to reduce the smoke search space, which can therefore significantly improve the performance of the single-stage method. Experimental results show that the proposed TSSD algorithm can robustly improve the detection accuracy of the single-stage method and the model has good compatibility for image resolution inputs. Compared with various state-of-the-art detection methods, the accuracy \documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$AP_{mean}$$\end{document}APmean of our proposed TSSD model reaches 59.24\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$\%$$\end{document}%, even surpassing the current detection model Faster RCNN. In addition, the detection speed of our proposed model can reach 50 ms (20 FPS), meeting the real-time requirements. This knowledge-based system has the advantages of high stability, high accuracy, fast detection speed. It can be widely used in some scenes with smoke detection requirements, such as on the mobile terminal carrier, providing great potential for practical environmental applications.
Collapse
Affiliation(s)
- Zhenyu Wang
- School of Control and Computer Engineering, North China Electric Power University, Beijing, 102206, China.
| | - Duokun Yin
- School of Control and Computer Engineering, North China Electric Power University, Beijing, 102206, China
| | - Senrong Ji
- School of Control and Computer Engineering, North China Electric Power University, Beijing, 102206, China
| |
Collapse
|
5
|
Zhu X, Li X, Ong K, Zhang W, Li W, Li L, Young D, Su Y, Shang B, Peng L, Xiong W, Liu Y, Liao W, Xu J, Wang F, Liao Q, Li S, Liao M, Li Y, Rao L, Lin J, Shi J, You Z, Zhong W, Liang X, Han H, Zhang Y, Tang N, Hu A, Gao H, Cheng Z, Liang L, Yu W, Ding Y. Hybrid AI-assistive diagnostic model permits rapid TBS classification of cervical liquid-based thin-layer cell smears. Nat Commun 2021; 12:3541. [PMID: 34112790 PMCID: PMC8192526 DOI: 10.1038/s41467-021-23913-3] [Citation(s) in RCA: 21] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/06/2021] [Accepted: 05/24/2021] [Indexed: 02/05/2023] Open
Abstract
Technical advancements significantly improve earlier diagnosis of cervical cancer, but accurate diagnosis is still difficult due to various factors. We develop an artificial intelligence assistive diagnostic solution, AIATBS, to improve cervical liquid-based thin-layer cell smear diagnosis according to clinical TBS criteria. We train AIATBS with >81,000 retrospective samples. It integrates YOLOv3 for target detection, Xception and Patch-based models to boost target classification, and U-net for nucleus segmentation. We integrate XGBoost and a logical decision tree with these models to optimize the parameters given by the learning process, and we develop a complete cervical liquid-based cytology smear TBS diagnostic system which also includes a quality control solution. We validate the optimized system with >34,000 multicenter prospective samples and achieve better sensitivity compared to senior cytologists, yet retain high specificity while achieving a speed of <180s/slide. Our system is adaptive to sample preparation using different standards, staining protocols and scanners.
Collapse
Affiliation(s)
- Xiaohui Zhu
- Department of Pathology, Nanfang Hospital and Basic Medical College, Southern Medical University, Guangzhou, Guangdong Province, PR China
- Guangdong Province Key Laboratory of Molecular Tumor Pathology, Guangzhou, Guangdong Province, PR China
| | - Xiaoming Li
- Department of Pathology, Shenzhen Bao'an People's Hospital (group), Shenzhen, Guangdong Province, PR China
| | - Kokhaur Ong
- Institute of Molecular and Cell Biology, A*STAR, Singapore, Singapore
- Bioinformatics Institute, A*STAR, Singapore, Singapore
| | - Wenli Zhang
- Department of Pathology, Nanfang Hospital and Basic Medical College, Southern Medical University, Guangzhou, Guangdong Province, PR China
- Guangdong Province Key Laboratory of Molecular Tumor Pathology, Guangzhou, Guangdong Province, PR China
| | - Wencai Li
- The First Affiliated Hospital of Zhengzhou University, Zhengzhou, Henan Province, PR China
| | - Longjie Li
- Institute of Molecular and Cell Biology, A*STAR, Singapore, Singapore
| | - David Young
- Institute of Molecular and Cell Biology, A*STAR, Singapore, Singapore
| | - Yongjian Su
- Guangzhou F.Q.PATHOTECH Co., Ltd, Guangzhou, Guangdong Province, PR China
| | - Bin Shang
- Guangzhou F.Q.PATHOTECH Co., Ltd, Guangzhou, Guangdong Province, PR China
| | - Linggan Peng
- Guangzhou F.Q.PATHOTECH Co., Ltd, Guangzhou, Guangdong Province, PR China
| | - Wei Xiong
- Guangzhou Kaipu Biotechnology Co., Ltd, Guangzhou, Guangdong Province, PR China
| | - Yunke Liu
- Laboratory Department, Guangzhou Tianhe District Maternal and Child Health Care Hospital, Guangzhou, Guangdong Province, PR China
| | - Wenting Liao
- State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Sun Yat-sen University Cancer Center, Guangzhou, Guangdong Province, PR China
| | - Jingjing Xu
- The First Affiliated Hospital of Zhengzhou University, Zhengzhou, Henan Province, PR China
| | - Feifei Wang
- Department of Pathology, Nanfang Hospital and Basic Medical College, Southern Medical University, Guangzhou, Guangdong Province, PR China
- Guangdong Province Key Laboratory of Molecular Tumor Pathology, Guangzhou, Guangdong Province, PR China
| | - Qing Liao
- Department of Pathology, Nanfang Hospital and Basic Medical College, Southern Medical University, Guangzhou, Guangdong Province, PR China
- Guangdong Province Key Laboratory of Molecular Tumor Pathology, Guangzhou, Guangdong Province, PR China
| | - Shengnan Li
- Guangzhou F.Q.PATHOTECH Co., Ltd, Guangzhou, Guangdong Province, PR China
| | - Minmin Liao
- Department of Pathology, Nanfang Hospital and Basic Medical College, Southern Medical University, Guangzhou, Guangdong Province, PR China
- Guangdong Province Key Laboratory of Molecular Tumor Pathology, Guangzhou, Guangdong Province, PR China
| | - Yu Li
- Department of Pathology, Nanfang Hospital and Basic Medical College, Southern Medical University, Guangzhou, Guangdong Province, PR China
- Guangdong Province Key Laboratory of Molecular Tumor Pathology, Guangzhou, Guangdong Province, PR China
| | - Linshang Rao
- Guangzhou F.Q.PATHOTECH Co., Ltd, Guangzhou, Guangdong Province, PR China
| | - Jinquan Lin
- Guangzhou F.Q.PATHOTECH Co., Ltd, Guangzhou, Guangdong Province, PR China
| | - Jianyuan Shi
- Guangzhou F.Q.PATHOTECH Co., Ltd, Guangzhou, Guangdong Province, PR China
| | - Zejun You
- Guangzhou F.Q.PATHOTECH Co., Ltd, Guangzhou, Guangdong Province, PR China
| | - Wenlong Zhong
- Guangzhou Huayin medical inspection center Co., Ltd, Guangzhou, Guangdong Province, PR China
| | - Xinrong Liang
- Guangzhou Huayin medical inspection center Co., Ltd, Guangzhou, Guangdong Province, PR China
| | - Hao Han
- Institute of Molecular and Cell Biology, A*STAR, Singapore, Singapore
| | - Yan Zhang
- Department of Pathology, Nanfang Hospital and Basic Medical College, Southern Medical University, Guangzhou, Guangdong Province, PR China
- Department of Pathology, Shenzhen Longhua District Maternity & Child Healthcare Hospital, Shenzhen, PR China
| | - Na Tang
- Department of Pathology, Shenzhen First People's Hospital, Shenzhen, Guangdong Province, PR China
| | - Aixia Hu
- Department of Pathology, Henan Provincial People's Hospital, Zhengzhou, Henan Province, PR China
| | - Hongyi Gao
- Department of Pathology, Guangdong Provincial Women's and Children's Dispensary, Shenzhen, Guangdong Province, PR China
| | - Zhiqiang Cheng
- Department of Pathology, Shenzhen First People's Hospital, Shenzhen, Guangdong Province, PR China.
| | - Li Liang
- Department of Pathology, Nanfang Hospital and Basic Medical College, Southern Medical University, Guangzhou, Guangdong Province, PR China.
- Guangdong Province Key Laboratory of Molecular Tumor Pathology, Guangzhou, Guangdong Province, PR China.
| | - Weimiao Yu
- Institute of Molecular and Cell Biology, A*STAR, Singapore, Singapore.
- Bioinformatics Institute, A*STAR, Singapore, Singapore.
| | - Yanqing Ding
- Department of Pathology, Nanfang Hospital and Basic Medical College, Southern Medical University, Guangzhou, Guangdong Province, PR China.
- Guangdong Province Key Laboratory of Molecular Tumor Pathology, Guangzhou, Guangdong Province, PR China.
| |
Collapse
|
6
|
Durve M, Bonaccorso F, Montessori A, Lauricella M, Tiribocchi A, Succi S. Tracking droplets in soft granular flows with deep learning techniques. EUROPEAN PHYSICAL JOURNAL PLUS 2021; 136:864. [PMID: 34458055 PMCID: PMC8380117 DOI: 10.1140/epjp/s13360-021-01849-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/06/2021] [Accepted: 08/07/2021] [Indexed: 05/09/2023]
Abstract
The state-of-the-art deep learning-based object recognition YOLO algorithm and object tracking DeepSORT algorithm are combined to analyze digital images from fluid dynamic simulations of multi-core emulsions and soft flowing crystals and to track moving droplets within these complex flows. The YOLO network was trained to recognize the droplets with synthetically prepared data, thereby bypassing the labor-intensive data acquisition process. In both applications, the trained YOLO + DeepSORT procedure performs with high accuracy on the real data from the fluid simulations, with low error levels in the inferred trajectories of the droplets and independently computed ground truth. Moreover, using commonly used desktop GPUs, the developed application is capable of analyzing data at speeds that exceed the typical image acquisition rates of digital cameras (30 fps), opening the interesting prospect of realizing a low-cost and practical tool to study systems with many moving objects, mostly but not exclusively, biological ones. Besides its practical applications, the procedure presented here marks the first step towards the automatic extraction of effective equations of motion of many-body soft flowing systems.
Collapse
Affiliation(s)
- Mihir Durve
- Center for Life Nano- & Neuro-Science, Fondazione Istituto Italiano di Tecnologia (IIT), 00161 Rome, Italy
| | - Fabio Bonaccorso
- Center for Life Nano- & Neuro-Science, Fondazione Istituto Italiano di Tecnologia (IIT), 00161 Rome, Italy
- Istituto per le Applicazioni del Calcolo CNR, via dei Taurini 19, Rome, Italy
- Department of Physics and INFN, University of Rome Tor Vergata, Via della Ricerca Scientifica 1, 00133 Rome, Italy
| | - Andrea Montessori
- Istituto per le Applicazioni del Calcolo CNR, via dei Taurini 19, Rome, Italy
| | - Marco Lauricella
- Istituto per le Applicazioni del Calcolo CNR, via dei Taurini 19, Rome, Italy
| | - Adriano Tiribocchi
- Istituto per le Applicazioni del Calcolo CNR, via dei Taurini 19, Rome, Italy
| | - Sauro Succi
- Center for Life Nano- & Neuro-Science, Fondazione Istituto Italiano di Tecnologia (IIT), 00161 Rome, Italy
- Istituto per le Applicazioni del Calcolo CNR, via dei Taurini 19, Rome, Italy
- Institute for Applied Computational Science, John A. Paulson School of Engineering and Applied Sciences, Harvard University, Cambridge, USA
| |
Collapse
|