Human-Avatar Symbiosis for the Treatment of Auditory Verbal Hallucinations in Schizophrenia through Virtual/Augmented Reality and Brain-Computer Interfaces

A Systematic Review of using Virtual and Augmented Reality for the Diagnosis and Treatment of Psychotic Disorders

Objective

Immersive virtual reality (VR) and augmented reality (AR) have the potential to improve the treatment and diagnosis of individuals experiencing psychosis. Although commonly used in creative industries, emerging evidence reveals that VR is a valuable tool to potentially improve clinical outcomes, including medication adherence, motivation, and rehabilitation. However, the efficacy and future directions of this novel intervention require further study. The aim of this review is to search for evidence of efficacy in enhancing existing psychosis treatment and diagnosis with AR/VR.

Methods

2069 studies involving AR/VR as a diagnostic and treatment option were reviewed via PRISMA guidelines in five databases: PubMed, PsychInfo, Embase, and CINAHL.

Results

Of the initial 2069 articles, 23 original articles were eligible for inclusion. One study applied VR to the diagnosis of schizophrenia. Most studies demonstrated that the addition of VR therapies and rehabilitation methods to treatment-as-usual (medication, psychotherapy, social skills training) was more effective than traditional methods alone in treating psychosis disorders. Studies also support the feasibility, safety, and acceptability of VR to patients. No articles using AR as a diagnostic or treatment option were found.

Conclusions

VR is efficacious in diagnosing and treating individuals experiencing psychosis and is a valuable augmentation of evidence-based treatments.

Supplementary Information

The online version contains supplementary material available at 10.1007/s40501-023-00287-5.

Behavioral intention of mental health practitioners toward the adoption of virtual humans in affect recognition training

This paper explores the key factors influencing mental health professionals' behavioral intention to adopt virtual humans as a means of affect recognition training. Therapies targeting social cognition deficits are in high demand given that these deficits are related to a loss of functioning and quality of life in several neuropsychiatric conditions such as schizophrenia, autism spectrum disorders, affective disorders, and acquired brain injury. Therefore, developing new therapies would greatly improve the quality of life of this large cohort of patients. A questionnaire based on the second revision of the Unified Theory of Acceptance and Use of Technology (UTAUT2) questionnaire was used for this study. One hundred and twenty-four mental health professionals responded to the questionnaire after viewing a video presentation of the system. The results confirmed that mental health professionals showed a positive intention to use virtual reality tools to train affect recognition, as they allow manipulation of social interaction with patients. Further studies should be conducted with therapists from other countries to reach more conclusions.

Altered effective connectivity in the emotional network induced by immersive virtual reality rehabilitation for post-stroke depression

Post-stroke depression (PSD) is a serious complication of stroke that significantly restricts rehabilitation. The use of immersive virtual reality for stroke survivors is promising. Herein, we investigated the effects of a novel immersive virtual reality training system on PSD and explored induced effective connectivity alterations in emotional networks using multivariate Granger causality analysis (GCA). Forty-four patients with PSD were equally allocated into an immersive-virtual reality group and a control group. In addition to their usual rehabilitation treatments, the participants in the immersive-virtual reality group participated in an immersive-virtual reality rehabilitation program, while the patients in the control group received 2D virtual reality rehabilitation training. The Hamilton Depression Rating Scale, modified Barthel Index (MBI), and resting-state functional magnetic resonance imaging (rsfMRI) data were collected before and after a 4-week intervention. rsfMRI data were analyzed using multivariate GCA. We found that the immersive virtual reality training was more effective in improving depression in patients with PSD but had no statistically significant improvement in MBI scores compared to the control group. The GCA showed that the following causal connectivities were strengthened after immersive virtual reality training: from the amygdala, insula, middle temporal gyrus, and caudate nucleus to the dorsolateral prefrontal cortex; from the insula to the medial prefrontal cortex; and from the thalamus to the posterior superior temporal sulcus. These causal connectivities were weakened after treatment in the control group. Our results indicated the neurotherapeutic use of immersive virtual reality rehabilitation as an effective non-pharmacological intervention for PSD; the alteration of causal connectivity in emotional networks might constitute the neural mechanisms underlying immersive-virtual reality rehabilitation in PSD.

A scoping review on monitoring mental health using smart wearable devices

With the continuous development of the times, social competition is becoming increasingly fierce, people are facing enormous pressure and mental health problems have become common. Long-term and persistent mental health problems can lead to severe mental disorders and even death in individuals. The real-time and accurate prediction of individual mental health has become an effective method to prevent the occurrence of mental health disorders. In recent years, smart wearable devices have been widely used for monitoring mental health and have played an important role. This paper provides a comprehensive review of the application fields, application mechanisms, common signals, common techniques and results of smart wearable devices for the detection of mental health problems, aiming to achieve more efficient and accurate prediction for individual mental health, and to achieve early identification, early prevention and early intervention to provide a reference for improving the level of individual mental health.

Virtual Reality as a Moderator of Psychedelic-Assisted Psychotherapy

Psychotherapy with the use of psychedelic substances, including psilocybin, lysergic acid diethylamide (LSD), ketamine, and 3,4-methylenedioxymethamphetamine (MDMA), has demonstrated promise in treatment of post-traumatic stress disorder (PTSD), anxiety, addiction, and treatment-resistant depression. Psychedelic-assisted psychotherapy (PP) represents a unique psychopharmacological model that leverages the profound effects of the psychedelic experience. That experience is characterized by strong dependency on two key factors: participant mindset and the therapeutic environment. As such, therapeutic models that utilize psychedelics reflect the need for careful design that promotes an open, flexible, trusting mindset and a supportive setting. To meet this need, the PP model is increasingly supplemented by auxiliary methods, including meditation, relaxation, visualization or spiritual practices. We suggest virtual reality (VR) as a full-spectrum tool able to capitalize on and catalyze the innately therapeutic aspects of the psychedelic experience, such as detachment from familiar reality, alteration of self-experience, augmentation of sensory perception and induction of mystical-type experiences. This is facilitated by VR’s evidenced capacity to: aid relaxation and reduce anxiety; buffer from external stimuli; promote a mindful presence; train the mind to achieve altered states of consciousness (ASC); evoke mystical states; enhance therapeutic alliance and encourage self-efficacy. While these unique VR features appear promising, VR’s potential role in PP remains speculative due to lack of empirical evidence on the combined use of VR and PP. Given the increased commercial interest in this synergy there is an urgent need to evaluate this approach. We suggest specific VR models and their role within PP protocols to inspire future direction in scientific research, and provide a list of potential disadvantages, side effects and limitations that need to be carefully considered. These include sensory overstimulation, cyber-sickness, triggering memories of past traumatic events as well as distracting from the inner experience or strongly influencing its contents. A balanced, evidence-based approach may provide continuity across all phases of treatment, support transition into and out of an ASC, deepen acute ASC experiences including mystical states and enrich the psychotherapeutic process of integration. We conclude that the potential application of VR in modulating psychedelic-assisted psychotherapy demands further exploration and an evidence-based approach to both design and implementation.

Therapeutic use of virtual reality relaxation in schizophrenia: A pilot study

Virtual reality can be used for psychotherapeutic purposes. Our main objective is to assess the feasibility and the tolerance of a relaxation therapy in virtual reality from patients suffering from schizophrenia, by studying the risks of symptoms of depersonalization or cybersickness. It is a prospective, open, five-week clinical study. Evaluations were carried out before and after each session. They were 13 patients aged 18 to 65. Analyzes showed a good tolerance and acceptance regarding virtual reality and anxiolytic effects have been observed between baseline values and those obtained after the first session and after the fifth session (p < 0.05).

Brain-Computer Interface: Advancement and Challenges

Brain-Computer Interface (BCI) is an advanced and multidisciplinary active research domain based on neuroscience, signal processing, biomedical sensors, hardware, etc. Since the last decades, several groundbreaking research has been conducted in this domain. Still, no comprehensive review that covers the BCI domain completely has been conducted yet. Hence, a comprehensive overview of the BCI domain is presented in this study. This study covers several applications of BCI and upholds the significance of this domain. Then, each element of BCI systems, including techniques, datasets, feature extraction methods, evaluation measurement matrices, existing BCI algorithms, and classifiers, are explained concisely. In addition, a brief overview of the technologies or hardware, mostly sensors used in BCI, is appended. Finally, the paper investigates several unsolved challenges of the BCI and explains them with possible solutions.

How Interpersonal Distance Between Avatar and Human Influences Facial Affect Recognition in Immersive Virtual Reality

Purpose: The purpose of this study was to determine the optimal interpersonal distance (IPD) between humans and affective avatars in facial affect recognition in immersive virtual reality (IVR). The ideal IPD is the one in which the humans show the highest number of hits and the shortest reaction times in recognizing the emotions displayed by avatars. The results should help design future therapies to remedy facial affect recognition deficits.

Methods: A group of 39 healthy volunteers participated in an experiment in which participants were shown 65 dynamic faces in IVR and had to identify six basic emotions plus neutral expression presented by the avatars. We decided to limit the experiment to five different distances: D1 (35 cm), D2 (55 cm), D3 (75 cm), D4 (95 cm), and D5 (115 cm), all belonging to the intimate and personal interpersonal spaces. Of the total of 65 faces, 13 faces were presented for each of the included distances. The views were shown at different angles: 50% in frontal view, 25% from the right profile, and 25% from the left profile. The order of appearance of the faces presented to each participant was randomized.

Results: The overall success rate in facial emotion identification was 90.33%, being D3 the IPD with the best overall emotional recognition hits, although statistically significant differences could not be found between the IPDs. Consistent with results obtained in previous studies, identification rates for negative emotions were higher with increasing IPD, whereas the recognition task improved for positive emotions when IPD was closer. In addition, the study revealed irregular behavior in the facial detection of the emotion surprise.

Conclusions: IVR allows us to reliably assess facial emotion recognition using dynamic avatars as all the IPDs tested showed to be effective. However, no statistically significant differences in facial emotion recognition were found among the different IPDs.

Digital Technologies for Schizophrenia Management: A Descriptive Review

While the implementation of digital technology in psychiatry appears promising, there is an urgent need to address the implications of the absence of ethical design in the early development of such technologies. Some authors have noted the gap between technology development and ethical analysis and have called for an upstream examination of the ethical issues raised by digital technologies. In this paper, we address this suggestion, particularly in relation to digital healthcare technologies for patients with schizophrenia spectrum disorders. The introduction of digital technologies in psychiatry offers a broad spectrum of diagnostic and treatment options tailored to the health needs and goals of patients' care. These technologies include wearable devices, smartphone applications for high-immersive virtual realities, smart homes, telepsychiatry and messaging systems for patients in rural areas. The availability of these technologies could increase access to mental health services and improve the diagnostics of mental disorders. In this descriptive review, we systematize ethical concerns about digital technologies for mental health with a particular focus on individuals suffering from schizophrenia. There are many unsolved dilemmas and conflicts of interest in the implementation of these technologies, such as (1) the lack of evidence on efficacy and impact on self-perception; (2) the lack of clear standards for the safety of their daily implementation; (3) unclear roles of technology and a shift in the responsibilities of all parties; (4) no guarantee of data confidentiality; and (5) the lack of a user-centered design that meets the particular needs of patients with schizophrenia. mHealth can improve care in psychiatry and make mental healthcare services more efficient and personalized while destigmatizing mental health disorders. To ensure that these technologies will benefit people with mental health disorders, we need to heighten sensitivity to ethical issues among mental healthcare specialists, health policy makers, software developers, patients themselves and their proxies. Additionally, we need to develop frameworks for furthering sustainable development in the digital technologies industry and for the responsible usage of such technologies for patients with schizophrenia in the clinical setting. We suggest that digital technology in psychiatry, particularly for schizophrenia and other serious mental health disorders, should be integrated into treatment with professional supervision rather than as a self-treatment tool.

Converging Robotic Technologies in Targeted Neural Rehabilitation: A Review of Emerging Solutions and Challenges

Recent advances in the field of neural rehabilitation, facilitated through technological innovation and improved neurophysiological knowledge of impaired motor control, have opened up new research directions. Such advances increase the relevance of existing interventions, as well as allow novel methodologies and technological synergies. New approaches attempt to partially overcome long-term disability caused by spinal cord injury, using either invasive bridging technologies or noninvasive human-machine interfaces. Muscular dystrophies benefit from electromyography and novel sensors that shed light on underlying neuromotor mechanisms in people with Duchenne. Novel wearable robotics devices are being tailored to specific patient populations, such as traumatic brain injury, stroke, and amputated individuals. In addition, developments in robot-assisted rehabilitation may enhance motor learning and generate movement repetitions by decoding the brain activity of patients during therapy. This is further facilitated by artificial intelligence algorithms coupled with faster electronics. The practical impact of integrating such technologies with neural rehabilitation treatment can be substantial. They can potentially empower nontechnically trained individuals-namely, family members and professional carers-to alter the programming of neural rehabilitation robotic setups, to actively get involved and intervene promptly at the point of care. This narrative review considers existing and emerging neural rehabilitation technologies through the perspective of replacing or restoring functions, enhancing, or improving natural neural output, as well as promoting or recruiting dormant neuroplasticity. Upon conclusion, we discuss the future directions for neural rehabilitation research, diagnosis, and treatment based on the discussed technologies and their major roadblocks. This future may eventually become possible through technological evolution and convergence of mutually beneficial technologies to create hybrid solutions.

Validation of dynamic virtual faces for facial affect recognition

The ability to recognise facial emotions is essential for successful social interaction. The most common stimuli used when evaluating this ability are photographs. Although these stimuli have proved to be valid, they do not offer the level of realism that virtual humans have achieved. The objective of the present paper is the validation of a new set of dynamic virtual faces (DVFs) that mimic the six basic emotions plus the neutral expression. The faces are prepared to be observed with low and high dynamism, and from front and side views. For this purpose, 204 healthy participants, stratified by gender, age and education level, were recruited for assessing their facial affect recognition with the set of DVFs. The accuracy in responses was compared with the already validated Penn Emotion Recognition Test (ER-40). The results showed that DVFs were as valid as standardised natural faces for accurately recreating human-like facial expressions. The overall accuracy in the identification of emotions was higher for the DVFs (88.25%) than for the ER-40 faces (82.60%). The percentage of hits of each DVF emotion was high, especially for neutral expression and happiness emotion. No statistically significant differences were discovered regarding gender. Nor were significant differences found between younger adults and adults over 60 years. Moreover, there is an increase of hits for avatar faces showing a greater dynamism, as well as front views of the DVFs compared to their profile presentations. DVFs are as valid as standardised natural faces for accurately recreating human-like facial expressions of emotions.

Combining brain-computer interface and virtual reality for rehabilitation in neurological diseases: A narrative review

BACKGROUND

The traditional rehabilitation for neurological diseases lacks the active participation of patients, its process is monotonous and tedious, and the effects need to be improved. Therefore, a new type of rehabilitation technology with more active participation combining brain-computer interface (BCI) with virtual reality (VR) has developed rapidly in recent years and has been used in rehabilitation in neurological diseases.

OBJECTIVES

This narrative review analyzed and characterized the development and application of the new training system (BCI-VR) in rehabilitation of neurological diseases from the perspective of the BCI paradigm, to provide a pathway for future research in this field.

METHODS

The review involved a search of the Web of Science-Science Citation Index/Social Sciences Citation Index and the China National Knowledge Infrastructure databases; 39 papers were selected. Advantages and challenges of BCI-VR - based neurological rehabilitation were analyzed in detail.

RESULTS

Most BCI-VR studies included could be classified by 3 major BCI paradigms: motor imagery, P300, and steady-state visual-evoked potential. Integrating VR scenes into BCI systems could effectively promote the recovery process from nervous system injuries as compared with traditional methods.

CONCLUSION

As compared with rehabilitation based on traditional BCI, rehabilitation based on BCI-VR can provide better feedback information for patients and promote the recovery of brain function. By solving the challenges and continual development, the BCI-VR system can be broadly applied to the clinical treatment of various neurological diseases.

Understanding auditory verbal hallucinations in healthy individuals and individuals with psychiatric disorders

Auditory verbal hallucinations(AVHs) are psychiatric manifestations that are common in patients with psychiatric disorders and can occur in healthy individuals. This review summarizes the existing literature on the phenomenological features of auditory verbal hallucinations, imaging findings, and interventions, focusing on patients with schizophrenia who experience auditory verbal hallucinations, in addition to patients with borderline personality disorder, bipolar disorder, major depressive disorder, and posttraumatic stress disorder, as well as healthy individuals. The phenomenological features of AVHs vary in different psychiatric disorders, and the symptoms are associated with changes in specific brain structures and disturbances in brain function, blood flow, and metabolism. Interventions for auditory verbal hallucinations include antipsychotic drugs, neurostimulation, and cognitive behavioral therapy.

Digital Technology for Internet Access by Patients With Early-Stage Schizophrenia in Spain: Multicenter Research Study

Background

Digital technology and social networks are part of everyday life in the current internet age, especially among young people. To date, few studies have been published worldwide on the pattern of use of digital technology devices and applications in patients with early-stage schizophrenia and even fewer comparing them with healthy participants (not using data from general population surveys) from the same demographic areas. In Spain, no such study has been carried out.

Objective

The aim of this study was to analyze how patients with early-stage schizophrenia use internet and social networks compared with healthy participants matched by age and gender and also to examine which devices are utilized to access internet resources.

Methods

A cross-sectional, multicentric study was carried out through a semistructured interview asking about the use of digital technology devices and internet. The sample comprised 90 patients and 90 healthy participants. The semistructured interview was conducted on 30 outpatients and 30 healthy subjects in each of the 3 different cities (Madrid, Alicante, and Cuenca). Student t test was used for continuous variables and chi-square test for categorical variables. In the case of ordinal variables, nonparametric Mann-Whitney U and Kruskal-Wallis H tests for independent samples were performed to compare groups.

Results

The results indicated that a large proportion of patients with early-stage schizophrenia have access to different digital devices and use them frequently. In addition, both groups coincide in the order of preference and the purpose for which they use the devices. However, a lower frequency of use of most digital technology devices was detected in patients compared with healthy participants. In the case of some devices, this was due to the impossibility of access and not a lack of interest.

Conclusions

To our knowledge, this is the first study to analyze patterns of internet access and use of digital technology devices and applications in Spanish patients with early-stage schizophrenia compared with healthy participants from the same demographic areas. The results on significant access and use of digital technology and internet shown in this cross-sectional study will allow enhanced and more efficient treatment strategies to be planned, utilizing digital technology devices, for patients with early-stage schizophrenia.

Potential Applications of Digital Technology in Assessment, Treatment, and Self-help for Hallucinations

The field of digital mental health is rapidly expanding with digital tools being used in assessment, intervention, and supporting self-help. The application of digital mental health to hallucinations is, however, at a very early stage. This report from a working group of the International Consortium on Hallucinations Research considers particular synergies between the phenomenon of hallucinations and digital tools that are being developed. Highlighted uses include monitoring and managing intermittently occurring hallucinations in daily life; therapeutic applications of audio and video media including virtual and augmented reality; targeting verbal aspects of hallucinations; and using avatars to represent hallucinatory voices. Although there is a well-established Internet-based peer support network, digital resources for hallucinations have yet to be implemented in routine practice. Implementation may benefit from identifying how to market resources to the broad range of populations who experience hallucinations and identifying sustainable funding models. It is envisaged that digital tools will contribute to improved self-management and service provision for people experiencing hallucinations.