1
|
Lee SH, Paik SH, Kang SY, Phillips Z, Kim JB, Kim BJ, Kim BM. Convolutional neural networks can detect orthostatic hypotension in Parkinson's disease using resting-state functional near-infrared spectroscopy data. JOURNAL OF BIOPHOTONICS 2024:e202400138. [PMID: 38952169 DOI: 10.1002/jbio.202400138] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/02/2024] [Revised: 05/10/2024] [Accepted: 06/10/2024] [Indexed: 07/03/2024]
Abstract
Neurological disorders such as Parkinson's disease (PD) often adversely affect the vascular system, leading to alterations in blood flow patterns. Functional near-infrared spectroscopy (fNIRS) is used to monitor hemodynamic changes via signal measurement. This study investigated the potential of using resting-state fNIRS data through a convolutional neural network (CNN) to evaluate PD with orthostatic hypotension. The CNN demonstrated significant efficacy in analyzing fNIRS data, and it outperformed the other machine learning methods. The results indicate that judicious input data selection can enhance accuracy by over 85%, while including the correlation matrix as an input further improves the accuracy to more than 90%. This study underscores the promising role of CNN-based fNIRS data analysis in the diagnosis and management of the PD. This approach enhances diagnostic accuracy, particularly in resting-state conditions, and can reduce the discomfort and risks associated with current diagnostic methods, such as the head-up tilt test.
Collapse
Affiliation(s)
- Seung Hyun Lee
- Global Health Technology Research Center, Korea University, Seoul, Republic of Korea
| | | | - Shin-Young Kang
- Department of Biomedical Engineering, Korea University, Seoul, Republic of Korea
| | - Zephaniah Phillips
- Global Health Technology Research Center, Korea University, Seoul, Republic of Korea
| | - Jung Bin Kim
- Department of Neurology, Korea University Anam Hospital, Korea University College of Medicine, Seoul, Republic of Korea
| | - Byung-Jo Kim
- Department of Neurology, Korea University Anam Hospital, Korea University College of Medicine, Seoul, Republic of Korea
| | - Beop-Min Kim
- Department of Biomedical Engineering, Korea University, Seoul, Republic of Korea
| |
Collapse
|
2
|
Wang Q, Li Y, Su H, Zhong N, Xu Q, Li X. Deep neural network to differentiate internet gaming disorder from healthy controls during stop-signal task: a multichannel near-infrared spectroscopy study. BIOMED ENG-BIOMED TE 2023; 68:457-468. [PMID: 37099486 DOI: 10.1515/bmt-2023-0030] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/18/2023] [Accepted: 03/27/2023] [Indexed: 04/27/2023]
Abstract
Internet Gaming Disorder (IGD), as one of worldwide mental health issues, leads to negative effects on physical and mental health and has attracted public attention. Most studies on IGD are based on screening scales and subjective judgments of doctors, without objective quantitative assessment. However, public understanding of internet gaming disorder lacks objectivity. Therefore, the researches on internet gaming disorder still have many limitations. In this paper, a stop-signal task (SST) was designed to assess inhibitory control in patients with IGD based on prefrontal functional near-infrared spectroscopy (fNIRS). According to the scale, the subjects were divided into health and gaming disorder. A total of 40 subjects (24 internet gaming disorders; 16 healthy controls) signals were used for deep learning-based classification. The seven algorithms used for classification and comparison were deep learning algorithms (DL) and machine learning algorithms (ML), with four and three algorithms in each category, respectively. After applying hold-out method, the performance of the model was verified by accuracy. DL models outperformed traditional ML algorithms. Furthermore, the classification accuracy of the two-dimensional convolution neural network (2D-CNN) was 87.5% among all models. This was the highest accuracy out of all models that were tested. The 2D-CNN was able to outperform the other models due to its ability to learn complex patterns in data. This makes it well-suited for image classification tasks. The findings suggested that a 2D-CNN model is an effective approach for predicting internet gaming disorder. The results show that this is a reliable method with high accuracy to identify patients with IGD and demonstrate that the use of fNIRS to facilitate the development of IGD diagnosis has great potential.
Collapse
Affiliation(s)
- Qiwen Wang
- College of Medical Instruments, Shanghai University of Medicine & Health Sciences, Shanghai, China
- School of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai, China
| | - Yongkang Li
- College of Medical Instruments, Shanghai University of Medicine & Health Sciences, Shanghai, China
- School of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai, China
| | - Hang Su
- Shanghai Mental health Center, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Na Zhong
- Shanghai Mental health Center, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Qi Xu
- College of Medical Instruments, Shanghai University of Medicine & Health Sciences, Shanghai, China
- School of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai, China
| | - Xiaoou Li
- College of Medical Instruments, Shanghai University of Medicine & Health Sciences, Shanghai, China
- School of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai, China
| |
Collapse
|
3
|
Zhang Y, Liu D, Li T, Zhang P, Li Z, Gao F. CGAN-rIRN: a data-augmented deep learning approach to accurate classification of mental tasks for a fNIRS-based brain-computer interface. BIOMEDICAL OPTICS EXPRESS 2023; 14:2934-2954. [PMID: 37342712 PMCID: PMC10278643 DOI: 10.1364/boe.489179] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/03/2023] [Revised: 05/11/2023] [Accepted: 05/15/2023] [Indexed: 06/23/2023]
Abstract
Functional near-infrared spectroscopy (fNIRS) is increasingly used to investigate different mental tasks for brain-computer interface (BCI) control due to its excellent environmental and motion robustness. Feature extraction and classification strategy for fNIRS signal are essential to enhance the classification accuracy of voluntarily controlled BCI systems. The limitation of traditional machine learning classifiers (MLCs) lies in manual feature engineering, which is considered as one of the drawbacks that reduce accuracy. Since the fNIRS signal is a typical multivariate time series with multi-dimensionality and complexity, it makes the deep learning classifier (DLC) ideal for classifying neural activation patterns. However, the inherent bottleneck of DLCs is the requirement of substantial-scale, high-quality labeled training data and expensive computational resources to train deep networks. The existing DLCs for classifying mental tasks do not fully consider the temporal and spatial properties of fNIRS signals. Therefore, a specifically-designed DLC is desired to classify multi-tasks with high accuracy in fNIRS-BCI. To this end, we herein propose a novel data-augmented DLC to accurately classify mental tasks, which employs a convolution-based conditional generative adversarial network (CGAN) for data augmentation and a revised Inception-ResNet (rIRN) based DLC. The CGAN is utilized to generate class-specific synthetic fNIRS signals to augment the training dataset. The network architecture of rIRN is elaborately designed in accordance with the characteristics of the fNIRS signal, with serial multiple spatial and temporal feature extraction modules (FEMs), where each FEM performs deep and multi-scale feature extraction and fusion. The results of the paradigm experiments show that the proposed CGAN-rIRN approach improves the single-trial accuracy for mental arithmetic and mental singing tasks in both the data augmentation and classifier, as compared to the traditional MLCs and the commonly used DLCs. The proposed fully data-driven hybrid deep learning approach paves a promising way to improve the classification performance of volitional control fNIRS-BCI.
Collapse
Affiliation(s)
- Yao Zhang
- College of Precision Instrument and Optoelectronics Engineering, Tianjin University, Tianjin 300070, China
| | - Dongyuan Liu
- College of Precision Instrument and Optoelectronics Engineering, Tianjin University, Tianjin 300070, China
| | - Tieni Li
- College of Precision Instrument and Optoelectronics Engineering, Tianjin University, Tianjin 300070, China
| | - Pengrui Zhang
- College of Precision Instrument and Optoelectronics Engineering, Tianjin University, Tianjin 300070, China
| | - Zhiyong Li
- College of Precision Instrument and Optoelectronics Engineering, Tianjin University, Tianjin 300070, China
| | - Feng Gao
- College of Precision Instrument and Optoelectronics Engineering, Tianjin University, Tianjin 300070, China
- Tianjin Key Laboratory of Biomedical Detecting Techniques and Instruments, Tianjin 300070, China
| |
Collapse
|
4
|
Eastmond C, Subedi A, De S, Intes X. Deep learning in fNIRS: a review. NEUROPHOTONICS 2022; 9:041411. [PMID: 35874933 PMCID: PMC9301871 DOI: 10.1117/1.nph.9.4.041411] [Citation(s) in RCA: 26] [Impact Index Per Article: 13.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/26/2022] [Accepted: 06/22/2022] [Indexed: 05/28/2023]
Abstract
Significance: Optical neuroimaging has become a well-established clinical and research tool to monitor cortical activations in the human brain. It is notable that outcomes of functional near-infrared spectroscopy (fNIRS) studies depend heavily on the data processing pipeline and classification model employed. Recently, deep learning (DL) methodologies have demonstrated fast and accurate performances in data processing and classification tasks across many biomedical fields. Aim: We aim to review the emerging DL applications in fNIRS studies. Approach: We first introduce some of the commonly used DL techniques. Then, the review summarizes current DL work in some of the most active areas of this field, including brain-computer interface, neuro-impairment diagnosis, and neuroscience discovery. Results: Of the 63 papers considered in this review, 32 report a comparative study of DL techniques to traditional machine learning techniques where 26 have been shown outperforming the latter in terms of the classification accuracy. In addition, eight studies also utilize DL to reduce the amount of preprocessing typically done with fNIRS data or increase the amount of data via data augmentation. Conclusions: The application of DL techniques to fNIRS studies has shown to mitigate many of the hurdles present in fNIRS studies such as lengthy data preprocessing or small sample sizes while achieving comparable or improved classification accuracy.
Collapse
Affiliation(s)
- Condell Eastmond
- Center for Modeling, Simulation and Imaging for Medicine, Rensselaer Polytechnic, Department of Biomedical Engineering, Troy, New York, United States
| | - Aseem Subedi
- Center for Modeling, Simulation and Imaging for Medicine, Rensselaer Polytechnic, Department of Biomedical Engineering, Troy, New York, United States
| | - Suvranu De
- Center for Modeling, Simulation and Imaging for Medicine, Rensselaer Polytechnic, Department of Biomedical Engineering, Troy, New York, United States
| | - Xavier Intes
- Center for Modeling, Simulation and Imaging for Medicine, Rensselaer Polytechnic, Department of Biomedical Engineering, Troy, New York, United States
| |
Collapse
|
5
|
Ren H, Zhou S, Zhang L, Zhao F, Qiao L. Identifying Individuals by fNIRS-Based Brain Functional Network Fingerprints. Front Neurosci 2022; 16:813293. [PMID: 35221902 PMCID: PMC8873366 DOI: 10.3389/fnins.2022.813293] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/11/2021] [Accepted: 01/03/2022] [Indexed: 11/13/2022] Open
Abstract
Individual identification based on brain functional network (BFN) has attracted a lot of research interest in recent years, since it provides a novel biometric for identity authentication, as well as a feasible way of exploring the brain at an individual level. Previous studies have shown that an individual can be identified by its BFN fingerprint estimated from functional magnetic resonance imaging, electroencephalogram, or magnetoencephalography data. Functional near-infrared spectroscopy (fNIRS) is an emerging imaging technique that, by measuring the changes in blood oxygen concentration, can respond to cerebral activities; in this paper, we investigate whether fNIRS-based BFN could be used as a “fingerprint” to identify individuals. In particular, Pearson's correlation is first used to calculate BFN based on the preprocessed fNIRS signals, and then the nearest neighbor scheme is used to match the estimated BFNs between different individuals. Through the experiments on an open-access fNIRS dataset, we have two main findings: (1) under the cases of cross-task (i.e., resting, right-handed, left-handed finger tapping, and foot tapping), the BFN fingerprints generally work well for the individual identification, and, more interestingly, (2) the accuracy under cross-task is well above the accuracy under cross-view (i.e., oxyhemoglobin and de-oxyhemoglobin). These findings indicate that fNIRS-based BFN fingerprint is a potential biometric for identifying individual.
Collapse
Affiliation(s)
- Haonan Ren
- School of Mathematics Science, Liaocheng University, Liaocheng, China
| | - Shufeng Zhou
- School of Mathematics Science, Liaocheng University, Liaocheng, China
| | - Limei Zhang
- School of Mathematics Science, Liaocheng University, Liaocheng, China
| | - Feng Zhao
- School of Computer Science and Technology, Shandong Technology and Business University, Yantai, China
| | - Lishan Qiao
- School of Mathematics Science, Liaocheng University, Liaocheng, China
- *Correspondence: Lishan Qiao
| |
Collapse
|
6
|
Wang J, Grant T, Velipasalar S, Geng B, Hirshfield L. Taking a Deeper Look at the Brain: Predicting Visual Perceptual and Working Memory Load from High-Density fNIRS Data. IEEE J Biomed Health Inform 2021; 26:2308-2319. [PMID: 34882566 DOI: 10.1109/jbhi.2021.3133871] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Predicting workload using physiological sensors has taken on a diffuse set of methods in recent years. However, the majority of these methods train models on small datasets, with small numbers of channel locations on the brain, limiting a models ability to transfer across participants, tasks, or experimental sessions. In this paper, we introduce a new method of modeling a large, cross-participant and cross-session set of high density functional near infrared spectroscopy (fNIRS) data by using an approach grounded in cognitive load theory and employing a Bi-Directional Gated Recurrent Unit (BiGRU) incorporating attention mechanism and self-supervised label augmentation (SLA). We show that our proposed CNN-BiGRU-SLA model can learn and classify different levels of working memory load (WML) and visual processing load (VPL) across participants. Importantly, we leverage a multi-label classification scheme, where our models are trained to predict simultaneously occurring levels of WML and VPL. We evaluate our model using leave-one-participant-out (LOOCV) as well as 10-fold cross validation. Using LOOCV, for binary classification (off/on), we reached an F1-score of 0.9179 for WML and 0.8907 for VPL across 22 participants (each participant did 2 sessions). For multi-level (off, low, high) classification, we reached an F1-score of 0.7972 for WML and 0.7968 for VPL. Using 10-fold cross validation, for multi-level classification, we reached an F1-score of 0.7742 for WML and 0.7741 for VPL.
Collapse
|
7
|
The Role of Visual Features in Text-Based CAPTCHAs: An fNIRS Study for Usable Security. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2021; 2021:8842420. [PMID: 34054941 PMCID: PMC8112922 DOI: 10.1155/2021/8842420] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/01/2020] [Revised: 04/08/2021] [Accepted: 04/18/2021] [Indexed: 11/17/2022]
Abstract
To mitigate dictionary attacks or similar undesirable automated attacks to information systems, developers mostly prefer using CAPTCHA challenges as Human Interactive Proofs (HIPs) to distinguish between human users and scripts. Appropriate use of CAPTCHA requires a setup that balances between robustness and usability during the design of a challenge. The previous research reveals that most usability studies have used accuracy and response time as measurement criteria for quantitative analysis. The present study aims at applying optical neuroimaging techniques for the analysis of CAPTCHA design. The functional Near-Infrared Spectroscopy technique was used to explore the hemodynamic responses in the prefrontal cortex elicited by CAPTCHA stimulus of varying types. The findings suggest that regions in the left and right dorsolateral and right dorsomedial prefrontal cortex respond to the degrees of line occlusion, rotation, and wave distortions present in a CAPTCHA. The systematic addition of the visual effects introduced nonlinear effects on the behavioral and prefrontal oxygenation measures, indicative of the emergence of Gestalt effects that might have influenced the perception of the overall CAPTCHA figure.
Collapse
|
8
|
Wang R, Hao Y, Yu Q, Chen M, Humar I, Fortino G. Depression Analysis and Recognition based on Functional Near-infrared Spectroscopy. IEEE J Biomed Health Inform 2021; 25:4289-4299. [PMID: 33929968 DOI: 10.1109/jbhi.2021.3076762] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Depression is the result of a complex interaction of social, psychological and physiological elements. It is now considered to be a major threat to people's physical health, and even as a threat to their lives. Research into the brain disorders of patients suffering from depression can help doctors to understand the pathogenesis of depression and facilitate its diagnosis and treatment. Functional near-infrared spectroscopy (fNIRS) is a non-invasive approach to the detection of brain functions and activities based on changes to the hemoglobin's oxygenation. In this paper, a comprehensive fNIRS-based depression-processing architecture, including the layers of source, feature and model, is first established to guide the deep modeling for fNIRS. In view of the complexity of depression, we propose a methodology in the time and frequency domains for feature extraction and deep neural networks for depression recognition and combining with current research. It is found that compared to non-depressed people, patients with depression have a weaker encephalic area connectivity and lower level of activation in the prefrontal lobe during brain activity. Finally, based on raw data, manual features and channel correlations, to recognize depression, the AlexNet model shows the best performance, especially in terms of the correlation features and presents an accuracy rate of 0.90 and a precision rate of 0.91, which is higher than ResNet18 and machine-learning algorithms on other data. Therefore, the correlation of brain regions can effectively recognize depression (from cases of non-depression), making it significant for the recognition of brain functions in the clinical diagnosis and treatment of depression.
Collapse
|
9
|
Qing K, Huang R, Hong KS. Decoding Three Different Preference Levels of Consumers Using Convolutional Neural Network: A Functional Near-Infrared Spectroscopy Study. Front Hum Neurosci 2021; 14:597864. [PMID: 33488372 PMCID: PMC7815930 DOI: 10.3389/fnhum.2020.597864] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/22/2020] [Accepted: 12/02/2020] [Indexed: 11/17/2022] Open
Abstract
This study decodes consumers' preference levels using a convolutional neural network (CNN) in neuromarketing. The classification accuracy in neuromarketing is a critical factor in evaluating the intentions of the consumers. Functional near-infrared spectroscopy (fNIRS) is utilized as a neuroimaging modality to measure the cerebral hemodynamic responses. In this study, a specific decoding structure, called CNN-based fNIRS-data analysis, was designed to achieve a high classification accuracy. Compared to other methods, the automated characteristics, constant training of the dataset, and learning efficiency of the proposed method are the main advantages. The experimental procedure required eight healthy participants (four female and four male) to view commercial advertisement videos of different durations (15, 30, and 60 s). The cerebral hemodynamic responses of the participants were measured. To compare the preference classification performances, CNN was utilized to extract the most common features, including the mean, peak, variance, kurtosis, and skewness. Considering three video durations, the average classification accuracies of 15, 30, and 60 s videos were 84.3, 87.9, and 86.4%, respectively. Among them, the classification accuracy of 87.9% for 30 s videos was the highest. The average classification accuracies of three preferences in females and males were 86.2 and 86.3%, respectively, showing no difference in each group. By comparing the classification performances in three different combinations (like vs. so-so, like vs. dislike, and so-so vs. dislike) between two groups, male participants were observed to have targeted preferences for commercial advertising, and the classification performance 88.4% between "like" vs. "dislike" out of three categories was the highest. Finally, pairwise classification performance are shown as follows: For female, 86.1% (like vs. so-so), 87.4% (like vs. dislike), 85.2% (so-so vs. dislike), and for male 85.7, 88.4, 85.1%, respectively.
Collapse
Affiliation(s)
- Kunqiang Qing
- School of Mechanical Engineering, Pusan National University, Busan, South Korea
| | - Ruisen Huang
- School of Mechanical Engineering, Pusan National University, Busan, South Korea
| | - Keum-Shik Hong
- School of Mechanical Engineering, Pusan National University, Busan, South Korea
- Department of Cogno-Mechatronics Engineering, Pusan National University, Busan, South Korea
| |
Collapse
|
10
|
Asgher U, Khalil K, Khan MJ, Ahmad R, Butt SI, Ayaz Y, Naseer N, Nazir S. Enhanced Accuracy for Multiclass Mental Workload Detection Using Long Short-Term Memory for Brain-Computer Interface. Front Neurosci 2020; 14:584. [PMID: 32655353 PMCID: PMC7324788 DOI: 10.3389/fnins.2020.00584] [Citation(s) in RCA: 27] [Impact Index Per Article: 6.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2019] [Accepted: 05/12/2020] [Indexed: 11/30/2022] Open
Abstract
Cognitive workload is one of the widely invoked human factors in the areas of human-machine interaction (HMI) and neuroergonomics. The precise assessment of cognitive and mental workload (MWL) is vital and requires accurate neuroimaging to monitor and evaluate the cognitive states of the brain. In this study, we have decoded four classes of MWL using long short-term memory (LSTM) with 89.31% average accuracy for brain-computer interface (BCI). The brain activity signals are acquired using functional near-infrared spectroscopy (fNIRS) from the prefrontal cortex (PFC) region of the brain. We performed a supervised MWL experimentation with four varying MWL levels on 15 participants (both male and female) and 10 trials of each MWL per participant. Real-time four-level MWL states are assessed using fNIRS system, and initial classification is performed using three strong machine learning (ML) techniques, support vector machine (SVM), k-nearest neighbor (k-NN), and artificial neural network (ANN) with obtained average accuracies of 54.33, 54.31, and 69.36%, respectively. In this study, novel deep learning (DL) frameworks are proposed, which utilizes convolutional neural network (CNN) and LSTM with 87.45 and 89.31% average accuracies, respectively, to solve high-dimensional four-level cognitive states classification problem. Statistical analysis, t-test, and one-way F-test (ANOVA) are also performed on accuracies obtained through ML and DL algorithms. Results show that the proposed DL (LSTM and CNN) algorithms significantly improve classification performance as compared with ML (SVM, ANN, and k-NN) algorithms.
Collapse
Affiliation(s)
- Umer Asgher
- School of Mechanical and Manufacturing Engineering (SMME), National University of Sciences and Technology (NUST), Islamabad, Pakistan
| | - Khurram Khalil
- School of Mechanical and Manufacturing Engineering (SMME), National University of Sciences and Technology (NUST), Islamabad, Pakistan
| | - Muhammad Jawad Khan
- School of Mechanical and Manufacturing Engineering (SMME), National University of Sciences and Technology (NUST), Islamabad, Pakistan
| | - Riaz Ahmad
- School of Mechanical and Manufacturing Engineering (SMME), National University of Sciences and Technology (NUST), Islamabad, Pakistan
- Directorate of Quality Assurance and International Collaboration, National University of Sciences and Technology (NUST), Islamabad, Pakistan
| | - Shahid Ikramullah Butt
- School of Mechanical and Manufacturing Engineering (SMME), National University of Sciences and Technology (NUST), Islamabad, Pakistan
| | - Yasar Ayaz
- School of Mechanical and Manufacturing Engineering (SMME), National University of Sciences and Technology (NUST), Islamabad, Pakistan
- National Center of Artificial Intelligence (NCAI) – NUST, Islamabad, Pakistan
| | - Noman Naseer
- Department of Mechatronics Engineering, Air University, Islamabad, Pakistan
| | - Salman Nazir
- Training and Assessment Research Group, Department of Maritime Operations, University of South-Eastern Norway, Kongsberg, Norway
| |
Collapse
|
11
|
Pradhan P, Guo S, Ryabchykov O, Popp J, Bocklitz TW. Deep learning a boon for biophotonics? JOURNAL OF BIOPHOTONICS 2020; 13:e201960186. [PMID: 32167235 DOI: 10.1002/jbio.201960186] [Citation(s) in RCA: 24] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/15/2019] [Revised: 02/22/2020] [Accepted: 03/10/2020] [Indexed: 06/10/2023]
Abstract
This review covers original articles using deep learning in the biophotonic field published in the last years. In these years deep learning, which is a subset of machine learning mostly based on artificial neural network geometries, was applied to a number of biophotonic tasks and has achieved state-of-the-art performances. Therefore, deep learning in the biophotonic field is rapidly growing and it will be utilized in the next years to obtain real-time biophotonic decision-making systems and to analyze biophotonic data in general. In this contribution, we discuss the possibilities of deep learning in the biophotonic field including image classification, segmentation, registration, pseudostaining and resolution enhancement. Additionally, we discuss the potential use of deep learning for spectroscopic data including spectral data preprocessing and spectral classification. We conclude this review by addressing the potential applications and challenges of using deep learning for biophotonic data.
Collapse
Affiliation(s)
- Pranita Pradhan
- Institute of Physical Chemistry and Abbe Center of Photonics, Friedrich-Schiller-University, Jena, Germany
- Leibniz Institute of Photonic Technology (Leibniz-IPHT), Member of Leibniz Research Alliance 'Health Technologies', Jena, Germany
| | - Shuxia Guo
- Institute of Physical Chemistry and Abbe Center of Photonics, Friedrich-Schiller-University, Jena, Germany
- Leibniz Institute of Photonic Technology (Leibniz-IPHT), Member of Leibniz Research Alliance 'Health Technologies', Jena, Germany
| | - Oleg Ryabchykov
- Institute of Physical Chemistry and Abbe Center of Photonics, Friedrich-Schiller-University, Jena, Germany
- Leibniz Institute of Photonic Technology (Leibniz-IPHT), Member of Leibniz Research Alliance 'Health Technologies', Jena, Germany
| | - Juergen Popp
- Institute of Physical Chemistry and Abbe Center of Photonics, Friedrich-Schiller-University, Jena, Germany
- Leibniz Institute of Photonic Technology (Leibniz-IPHT), Member of Leibniz Research Alliance 'Health Technologies', Jena, Germany
| | - Thomas W Bocklitz
- Institute of Physical Chemistry and Abbe Center of Photonics, Friedrich-Schiller-University, Jena, Germany
- Leibniz Institute of Photonic Technology (Leibniz-IPHT), Member of Leibniz Research Alliance 'Health Technologies', Jena, Germany
| |
Collapse
|
12
|
Breast Cancer Identification via Thermography Image Segmentation with a Gradient Vector Flow and a Convolutional Neural Network. JOURNAL OF HEALTHCARE ENGINEERING 2019; 2019:9807619. [PMID: 31915519 PMCID: PMC6935451 DOI: 10.1155/2019/9807619] [Citation(s) in RCA: 27] [Impact Index Per Article: 5.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/02/2019] [Revised: 06/28/2019] [Accepted: 08/13/2019] [Indexed: 11/17/2022]
Abstract
Breast cancer is the most common cancer among women worldwide with about half a million cases reported each year. Mammary thermography can offer early diagnosis at low cost if adequate thermographic images of the breasts are taken. The identification of breast cancer in an automated way can accelerate many tasks and applications of pathology. This can help complement diagnosis. The aim of this work is to develop a system that automatically captures thermographic images of breast and classifies them as normal and abnormal (without cancer and with cancer). This paper focuses on a segmentation method based on a combination of the curvature function k and the gradient vector flow, and for classification, we proposed a convolutional neural network (CNN) using the segmented breast. The aim of this paper is to compare CNN results with other classification techniques. Thus, every breast is characterized by its shape, colour, and texture, as well as left or right breast. These data were used for training as well as to compare the performance of CNN with three classification techniques: tree random forest (TRF), multilayer perceptron (MLP), and Bayes network (BN). CNN presents better results than TRF, MLP, and BN.
Collapse
|
13
|
Liu X, Kim CS, Hong KS. An fNIRS-based investigation of visual merchandising displays for fashion stores. PLoS One 2018; 13:e0208843. [PMID: 30533055 PMCID: PMC6289445 DOI: 10.1371/journal.pone.0208843] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2018] [Accepted: 11/25/2018] [Indexed: 02/06/2023] Open
Abstract
This paper investigates a brain-based approach for visual merchandising display (VMD) in fashion stores. In marketing, VMD has become a research topic of interest. However, VMD research using brain activation information is rare. We examine the hemodynamic responses (HRs) in the prefrontal cortex (PFC) using functional near-infrared spectroscopy (fNIRS) while positive/negative displays of four stores (menswear, womenswear, underwear, and sportswear) are shown to 20 subjects. As features for classifying the HRs, the mean, variance, peak, skewness, kurtosis, t-value, and slope of the signals for a 20-sec time window for the activated channels are analyzed. Linear discriminant analysis is used for classifying two-class (positive and negative displays) and four-class (four fashion stores) models. PFC brain activation maps based on t-values depicting the data from the 16 channels are provided. In the two-class classification, the underwear store had the highest average classification result of 67.04%, whereas the menswear store had the lowest value of 64.15%. Men's classification accuracy for the underwear stores with positive and negative displays was 71.38%, whereas the highest classification accuracy obtained by women for womenswear stores was 73%. The average accuracy over the 20 subjects for positive displays was 50.68%, while that of negative displays was 51.07%. Therefore, these findings suggest that human brain activation is involved in the evaluation of the fashion store displays. It is concluded that fNIRS can be used as a brain-based tool in the evaluation of fashion stores in a daily life environment.
Collapse
Affiliation(s)
- Xiaolong Liu
- School of Mechanical Engineering, Pusan National University, Geumjeong-gu, Busan, Republic of Korea
- School of Life Science and Technology, University of Electronic Science and Technology of China, West Hi-Tech Zone, Chengdu, Sichuan, P. R. China
| | - Chang-Seok Kim
- Department of Cogno-Mechatronics Engineering, Pusan National University, Geumjeong-gu, Busan, Republic of Korea
| | - Keum-Shik Hong
- School of Mechanical Engineering, Pusan National University, Geumjeong-gu, Busan, Republic of Korea
- Department of Cogno-Mechatronics Engineering, Pusan National University, Geumjeong-gu, Busan, Republic of Korea
- * E-mail:
| |
Collapse
|