1
|
Rahman MA, Brown DJ, Mahmud M, Harris M, Shopland N, Heym N, Sumich A, Turabee ZB, Standen B, Downes D, Xing Y, Thomas C, Haddick S, Premkumar P, Nastase S, Burton A, Lewis J. Enhancing biofeedback-driven self-guided virtual reality exposure therapy through arousal detection from multimodal data using machine learning. Brain Inform 2023; 10:14. [PMID: 37341863 DOI: 10.1186/s40708-023-00193-9] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2022] [Accepted: 05/15/2023] [Indexed: 06/22/2023] Open
Abstract
Virtual reality exposure therapy (VRET) is a novel intervention technique that allows individuals to experience anxiety-evoking stimuli in a safe environment, recognise specific triggers and gradually increase their exposure to perceived threats. Public-speaking anxiety (PSA) is a prevalent form of social anxiety, characterised by stressful arousal and anxiety generated when presenting to an audience. In self-guided VRET, participants can gradually increase their tolerance to exposure and reduce anxiety-induced arousal and PSA over time. However, creating such a VR environment and determining physiological indices of anxiety-induced arousal or distress is an open challenge. Environment modelling, character creation and animation, psychological state determination and the use of machine learning (ML) models for anxiety or stress detection are equally important, and multi-disciplinary expertise is required. In this work, we have explored a series of ML models with publicly available data sets (using electroencephalogram and heart rate variability) to predict arousal states. If we can detect anxiety-induced arousal, we can trigger calming activities to allow individuals to cope with and overcome distress. Here, we discuss the means of effective selection of ML models and parameters in arousal detection. We propose a pipeline to overcome the model selection problem with different parameter settings in the context of virtual reality exposure therapy. This pipeline can be extended to other domains of interest where arousal detection is crucial. Finally, we have implemented a biofeedback framework for VRET where we successfully provided feedback as a form of heart rate and brain laterality index from our acquired multimodal data for psychological intervention to overcome anxiety.
Collapse
Affiliation(s)
- Muhammad Arifur Rahman
- Department of Computer Science, Nottingham Trent University, Clifton Lane, Nottingham, NG11 8NS, UK
| | - David J Brown
- Department of Computer Science, Nottingham Trent University, Clifton Lane, Nottingham, NG11 8NS, UK
| | - Mufti Mahmud
- Department of Computer Science, Nottingham Trent University, Clifton Lane, Nottingham, NG11 8NS, UK.
- Medical Technologies Innovation Facility, Nottingham Trent University, Clifton Lane, Nottingham, NG11 8NS, UK.
- Computing and Informatics Research Centre, Nottingham Trent University, Clifton Lane, Nottingham, NG11 8NS, UK.
| | - Matthew Harris
- Department of Computer Science, Nottingham Trent University, Clifton Lane, Nottingham, NG11 8NS, UK
| | - Nicholas Shopland
- Department of Computer Science, Nottingham Trent University, Clifton Lane, Nottingham, NG11 8NS, UK
| | - Nadja Heym
- School of Social Sciences, Nottingham Trent University, Shakespeare St, Nottingham, NG1 4FQ, UK
| | - Alexander Sumich
- School of Social Sciences, Nottingham Trent University, Shakespeare St, Nottingham, NG1 4FQ, UK
| | - Zakia Batool Turabee
- School of Social Sciences, Nottingham Trent University, Shakespeare St, Nottingham, NG1 4FQ, UK
| | - Bradley Standen
- Department of Computer Science, Nottingham Trent University, Clifton Lane, Nottingham, NG11 8NS, UK
| | - David Downes
- Nottingham School of Art & Design, Nottingham Trent University, Shakespeare St, Nottingham, NG1 4FQ, UK
| | - Yangang Xing
- School of ADBE, Nottingham Trent University, Shakespeare St, Nottingham, NG1 4FQ, UK
| | - Carolyn Thomas
- Nottingham School of Art & Design, Nottingham Trent University, Shakespeare St, Nottingham, NG1 4FQ, UK
| | - Sean Haddick
- Department of Computer Science, Nottingham Trent University, Clifton Lane, Nottingham, NG11 8NS, UK
| | - Preethi Premkumar
- Division of Psychology, London South Bank University, London, SE1 0AA, UK
| | | | - Andrew Burton
- Department of Computer Science, Nottingham Trent University, Clifton Lane, Nottingham, NG11 8NS, UK
| | - James Lewis
- Department of Computer Science, Nottingham Trent University, Clifton Lane, Nottingham, NG11 8NS, UK
| |
Collapse
|
3
|
Fabietti M, Mahmud M, Lotfi A, Kaiser MS, Averna A, Guggenmos DJ, Nudo RJ, Chiappalone M, Chen J. SANTIA: a Matlab-based open-source toolbox for artifact detection and removal from extracellular neuronal signals. Brain Inform 2021; 8:14. [PMID: 34283328 PMCID: PMC8292498 DOI: 10.1186/s40708-021-00135-3] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/02/2021] [Accepted: 06/29/2021] [Indexed: 12/11/2022] Open
Abstract
Neuronal signals generally represent activation of the neuronal networks and give insights into brain functionalities. They are considered as fingerprints of actions and their processing across different structures of the brain. These recordings generate a large volume of data that are susceptible to noise and artifacts. Therefore, the review of these data to ensure high quality by automatically detecting and removing the artifacts is imperative. Toward this aim, this work proposes a custom-developed automatic artifact removal toolbox named, SANTIA (SigMate Advanced: a Novel Tool for Identification of Artifacts in Neuronal Signals). Developed in Matlab, SANTIA is an open-source toolbox that applies neural network-based machine learning techniques to label and train models to detect artifacts from the invasive neuronal signals known as local field potentials.
Collapse
Affiliation(s)
- Marcos Fabietti
- Department of Computer Science, Nottingham Trent University, Clifton Lane, Nottingham, NG11 8NS, UK
| | - Mufti Mahmud
- Department of Computer Science, Nottingham Trent University, Clifton Lane, Nottingham, NG11 8NS, UK.
- Medical Technologies Innovation Facility, Nottingham Trent University, Clifton Lane, Nottingham, NG11 8NS, UK.
- Computing and Informatics Research Centre, Nottingham Trent University, Clifton Lane, Nottingham, NG11 8NS, UK.
| | - Ahmad Lotfi
- Department of Computer Science, Nottingham Trent University, Clifton Lane, Nottingham, NG11 8NS, UK
| | - M Shamim Kaiser
- Institute of Information Technology, Jahangirnagar University, Savar, Dhaka, 1342, Bangladesh
| | - Alberto Averna
- Department of Health Sciences, University of Milan, Via di Rudinì, 8, 20142, Milan, Italy
| | - David J Guggenmos
- Department of Physical Medicine and Rehabilitation, University of Kansas Medical Center, 3901 Rainbow Blvd, Kansas City, 66160, USA
| | - Randolph J Nudo
- Department of Physical Medicine and Rehabilitation, University of Kansas Medical Center, 3901 Rainbow Blvd, Kansas City, 66160, USA
| | - Michela Chiappalone
- Department of informatics, Bioengineering, Robotics and System Engineering-DIBRIS, University of Genova, Via All'Opera Pia, 13, 16145, Genoa, Italy
| | - Jianhui Chen
- Faculty of Information Technology, International WIC Institute, Beijing University of Technology, Beijing, 100124, China
- Beijing International Collaboration Base on Brain Informatics and Wisdom Services, Beijing, 100124, China
| |
Collapse
|
4
|
Mahmud M, Kaiser MS, McGinnity TM, Hussain A. Deep Learning in Mining Biological Data. Cognit Comput 2021; 13:1-33. [PMID: 33425045 PMCID: PMC7783296 DOI: 10.1007/s12559-020-09773-x] [Citation(s) in RCA: 99] [Impact Index Per Article: 33.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/01/2020] [Accepted: 09/28/2020] [Indexed: 02/06/2023]
Abstract
Recent technological advancements in data acquisition tools allowed life scientists to acquire multimodal data from different biological application domains. Categorized in three broad types (i.e. images, signals, and sequences), these data are huge in amount and complex in nature. Mining such enormous amount of data for pattern recognition is a big challenge and requires sophisticated data-intensive machine learning techniques. Artificial neural network-based learning systems are well known for their pattern recognition capabilities, and lately their deep architectures-known as deep learning (DL)-have been successfully applied to solve many complex pattern recognition problems. To investigate how DL-especially its different architectures-has contributed and been utilized in the mining of biological data pertaining to those three types, a meta-analysis has been performed and the resulting resources have been critically analysed. Focusing on the use of DL to analyse patterns in data from diverse biological domains, this work investigates different DL architectures' applications to these data. This is followed by an exploration of available open access data sources pertaining to the three data types along with popular open-source DL tools applicable to these data. Also, comparative investigations of these tools from qualitative, quantitative, and benchmarking perspectives are provided. Finally, some open research challenges in using DL to mine biological data are outlined and a number of possible future perspectives are put forward.
Collapse
Affiliation(s)
- Mufti Mahmud
- Department of Computer Science, Nottingham Trent University, Clifton, NG11 8NS Nottingham, UK
- Medical Technology Innovation Facility, Nottingham Trent University, NG11 8NS Clifton, Nottingham, UK
| | - M. Shamim Kaiser
- Institute of Information Technology, Jahangirnagar University, Savar 1342 Dhaka, Bangladesh
| | - T. Martin McGinnity
- Department of Computer Science, Nottingham Trent University, Clifton, NG11 8NS Nottingham, UK
- Intelligent Systems Research Centre, Ulster University, Northern Ireland BT48 7JL Derry, UK
| | - Amir Hussain
- School of Computing , Edinburgh, EH11 4BN Edinburgh, UK
| |
Collapse
|