1
|
Liu W, Zhang Y, Zhang B, Xiong Q, Zhao H, Li S, Liu J, Bian Y. Self-Guided DMT: Exploring a Novel Paradigm of Dance Movement Therapy in Mixed Reality for Children with ASD. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 2024; 30:2119-2128. [PMID: 38457325 DOI: 10.1109/tvcg.2024.3372063] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/10/2024]
Abstract
Children diagnosed with Autism Spectrum Disorder (ASD) often exhibit motor disorders. Dance Movement Therapy (DMT) has shown great potential for improving the motor control ability of children with ASD. However, traditional DMT methods often lack vividness and are difficult to implement effectively. To address this issue, we propose a Mixed Reality DMT approach, utilizing interactive virtual agents. This approach offers immersive training content and multi-sensory feedback. To improve the training performance of children with ASD, we introduce a novel training paradigm featuring a self-guided mode. This paradigm enables the rapid creation of a virtual twin agent of the child with ASD using a single photo to embody oneself, which can then guide oneself during training. We conducted an experiment with the participation of 24 children diagnosed with ASD (or ASD propensity), recording their training performance under various experimental conditions. Through expert rating, behavior coding of training sessions, and statistical analysis, our findings revealed that the use of the twin agent for self-guidance resulted in noticeable improvements in the training performance of children with ASD. These improvements were particularly evident in terms of enhancing movement quality and refining overall target-related responses. Our study holds clinical potential in the field of medical treatment and rehabilitation for children with ASD.
Collapse
|
2
|
Koehler JC, Dong MS, Bierlich AM, Fischer S, Späth J, Plank IS, Koutsouleris N, Falter-Wagner CM. Machine learning classification of autism spectrum disorder based on reciprocity in naturalistic social interactions. Transl Psychiatry 2024; 14:76. [PMID: 38310111 PMCID: PMC10838326 DOI: 10.1038/s41398-024-02802-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/13/2023] [Revised: 01/19/2024] [Accepted: 01/23/2024] [Indexed: 02/05/2024] Open
Abstract
Autism spectrum disorder is characterized by impaired social communication and interaction. As a neurodevelopmental disorder typically diagnosed during childhood, diagnosis in adulthood is preceded by a resource-heavy clinical assessment period. The ongoing developments in digital phenotyping give rise to novel opportunities within the screening and diagnostic process. Our aim was to quantify multiple non-verbal social interaction characteristics in autism and build diagnostic classification models independent of clinical ratings. We analyzed videos of naturalistic social interactions in a sample including 28 autistic and 60 non-autistic adults paired in dyads and engaging in two conversational tasks. We used existing open-source computer vision algorithms for objective annotation to extract information based on the synchrony of movement and facial expression. These were subsequently used as features in a support vector machine learning model to predict whether an individual was part of an autistic or non-autistic interaction dyad. The two prediction models based on reciprocal adaptation in facial movements, as well as individual amounts of head and body motion and facial expressiveness showed the highest precision (balanced accuracies: 79.5% and 68.8%, respectively), followed by models based on reciprocal coordination of head (balanced accuracy: 62.1%) and body (balanced accuracy: 56.7%) motion, as well as intrapersonal coordination processes (balanced accuracy: 44.2%). Combinations of these models did not increase overall predictive performance. Our work highlights the distinctive nature of non-verbal behavior in autism and its utility for digital phenotyping-based classification. Future research needs to both explore the performance of different prediction algorithms to reveal underlying mechanisms and interactions, as well as investigate the prospective generalizability and robustness of these algorithms in routine clinical care.
Collapse
Affiliation(s)
| | - Mark Sen Dong
- Department of Psychiatry and Psychotherapy, Medical Faculty, LMU, Munich, Germany
| | - Afton M Bierlich
- Department of Psychiatry and Psychotherapy, Medical Faculty, LMU, Munich, Germany
| | - Stefanie Fischer
- Department of Psychiatry and Psychotherapy, Medical Faculty, LMU, Munich, Germany
- Goethe University Frankfurt, University Hospital, Department of Psychiatry, Psychosomatic Medicine and Psychotherapy, Frankfurt am Main, Germany
| | - Johanna Späth
- Department of Psychiatry and Psychotherapy, Medical Faculty, LMU, Munich, Germany
| | - Irene Sophia Plank
- Department of Psychiatry and Psychotherapy, Medical Faculty, LMU, Munich, Germany
| | - Nikolaos Koutsouleris
- Department of Psychiatry and Psychotherapy, Medical Faculty, LMU, Munich, Germany
- Max Planck Institute of Psychiatry, Munich, Germany
- Institute of Psychiatry, Psychology and Neuroscience, King's College, London, UK
| | | |
Collapse
|
3
|
Bloch C, Tepest R, Koeroglu S, Feikes K, Jording M, Vogeley K, Falter-Wagner CM. Interacting with autistic virtual characters: intrapersonal synchrony of nonverbal behavior affects participants' perception. Eur Arch Psychiatry Clin Neurosci 2024:10.1007/s00406-023-01750-3. [PMID: 38270620 DOI: 10.1007/s00406-023-01750-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/29/2023] [Accepted: 12/18/2023] [Indexed: 01/26/2024]
Abstract
Temporal coordination of communicative behavior is not only located between but also within interaction partners (e.g., gaze and gestures). This intrapersonal synchrony (IaPS) is assumed to constitute interpersonal alignment. Studies show systematic variations in IaPS in individuals with autism, which may affect the degree of interpersonal temporal coordination. In the current study, we reversed the approach and mapped the measured nonverbal behavior of interactants with and without ASD from a previous study onto virtual characters to study the effects of the differential IaPS on observers (N = 68), both with and without ASD (crossed design). During a communication task with both characters, who indicated targets with gaze and delayed pointing gestures, we measured response times, gaze behavior, and post hoc impression formation. Results show that character behavior indicative of ASD resulted in overall enlarged decoding times in observers and this effect was even pronounced in observers with ASD. A classification of observer's gaze types indicated differentiated decoding strategies. Whereas non-autistic observers presented with a rather consistent eyes-focused strategy associated with efficient and fast responses, observers with ASD presented with highly variable decoding strategies. In contrast to communication efficiency, impression formation was not influenced by IaPS. The results underline the importance of timing differences in both production and perception processes during multimodal nonverbal communication in interactants with and without ASD. In essence, the current findings locate the manifestation of reduced reciprocity in autism not merely in the person, but in the interactional dynamics of dyads.
Collapse
Affiliation(s)
- Carola Bloch
- Department of Psychiatry and Psychotherapy, Medical Faculty, LMU Clinic, Ludwig-Maximilians-University, 80336, Munich, Germany.
- Department of Psychiatry and Psychotherapy, Faculty of Medicine and University Hospital Cologne, University of Cologne, 50937, Cologne, Germany.
| | - Ralf Tepest
- Department of Psychiatry and Psychotherapy, Faculty of Medicine and University Hospital Cologne, University of Cologne, 50937, Cologne, Germany
| | - Sevim Koeroglu
- Department of Psychiatry and Psychotherapy, Faculty of Medicine and University Hospital Cologne, University of Cologne, 50937, Cologne, Germany
| | - Kyra Feikes
- Department of Psychiatry and Psychotherapy, Faculty of Medicine and University Hospital Cologne, University of Cologne, 50937, Cologne, Germany
| | - Mathis Jording
- Cognitive Neuroscience, Institute of Neuroscience and Medicine (INM-3), Forschungszentrum Juelich, 52425, Juelich, Germany
| | - Kai Vogeley
- Department of Psychiatry and Psychotherapy, Faculty of Medicine and University Hospital Cologne, University of Cologne, 50937, Cologne, Germany
- Cognitive Neuroscience, Institute of Neuroscience and Medicine (INM-3), Forschungszentrum Juelich, 52425, Juelich, Germany
| | - Christine M Falter-Wagner
- Department of Psychiatry and Psychotherapy, Medical Faculty, LMU Clinic, Ludwig-Maximilians-University, 80336, Munich, Germany
| |
Collapse
|
4
|
Zhou K, Cai R, Ma Y, Tan Q, Wang X, Li J, Shum HPH, Li FWB, Jin S, Liang X. A Video-Based Augmented Reality System for Human-in-the-Loop Muscle Strength Assessment of Juvenile Dermatomyositis. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 2023; PP:2456-2466. [PMID: 37027743 DOI: 10.1109/tvcg.2023.3247092] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/19/2023]
Abstract
As the most common idiopathic inflammatory myopathy in children, juvenile dermatomyositis (JDM) is characterized by skin rashes and muscle weakness. The childhood myositis assessment scale (CMAS) is commonly used to measure the degree of muscle involvement for diagnosis or rehabilitation monitoring. On the one hand, human diagnosis is not scalable and may be subject to personal bias. On the other hand, automatic action quality assessment (AQA) algorithms cannot guarantee 100% accuracy, making them not suitable for biomedical applications. As a solution, we propose a video-based augmented reality system for human-in-the-loop muscle strength assessment of children with JDM. We first propose an AQA algorithm for muscle strength assessment of JDM using contrastive regression trained by a JDM dataset. Our core insight is to visualize the AQA results as a virtual character facilitated by a 3D animation dataset, so that users can compare the real-world patient and the virtual character to understand and verify the AQA results. To allow effective comparisons, we propose a video-based augmented reality system. Given a feed, we adapt computer vision algorithms for scene understanding, evaluate the optimal way of augmenting the virtual character into the scene, and highlight important parts for effective human verification. The experimental results confirm the effectiveness of our AQA algorithm, and the results of the user study demonstrate that humans can more accurately and quickly assess the muscle strength of children using our system.
Collapse
|
5
|
Wiebe A, Kannen K, Selaskowski B, Mehren A, Thöne AK, Pramme L, Blumenthal N, Li M, Asché L, Jonas S, Bey K, Schulze M, Steffens M, Pensel MC, Guth M, Rohlfsen F, Ekhlas M, Lügering H, Fileccia H, Pakos J, Lux S, Philipsen A, Braun N. Virtual reality in the diagnostic and therapy for mental disorders: A systematic review. Clin Psychol Rev 2022; 98:102213. [PMID: 36356351 DOI: 10.1016/j.cpr.2022.102213] [Citation(s) in RCA: 21] [Impact Index Per Article: 10.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/19/2021] [Revised: 08/21/2022] [Accepted: 10/11/2022] [Indexed: 01/27/2023]
Abstract
BACKGROUND Virtual reality (VR) technologies are playing an increasingly important role in the diagnostics and treatment of mental disorders. OBJECTIVE To systematically review the current evidence regarding the use of VR in the diagnostics and treatment of mental disorders. DATA SOURCE Systematic literature searches via PubMed (last literature update: 9th of May 2022) were conducted for the following areas of psychopathology: Specific phobias, panic disorder and agoraphobia, social anxiety disorder, generalized anxiety disorder, posttraumatic stress disorder (PTSD), obsessive-compulsive disorder, eating disorders, dementia disorders, attention-deficit/hyperactivity disorder, depression, autism spectrum disorder, schizophrenia spectrum disorders, and addiction disorders. ELIGIBILITY CRITERIA To be eligible, studies had to be published in English, to be peer-reviewed, to report original research data, to be VR-related, and to deal with one of the above-mentioned areas of psychopathology. STUDY EVALUATION For each study included, various study characteristics (including interventions and conditions, comparators, major outcomes and study designs) were retrieved and a risk of bias score was calculated based on predefined study quality criteria. RESULTS Across all areas of psychopathology, k = 9315 studies were inspected, of which k = 721 studies met the eligibility criteria. From these studies, 43.97% were considered assessment-related, 55.48% therapy-related, and 0.55% were mixed. The highest research activity was found for VR exposure therapy in anxiety disorders, PTSD and addiction disorders, where the most convincing evidence was found, as well as for cognitive trainings in dementia and social skill trainings in autism spectrum disorder. CONCLUSION While VR exposure therapy will likely find its way successively into regular patient care, there are also many other promising approaches, but most are not yet mature enough for clinical application. REVIEW REGISTRATION PROSPERO register CRD42020188436. FUNDING The review was funded by budgets from the University of Bonn. No third party funding was involved.
Collapse
Affiliation(s)
- Annika Wiebe
- Department of Psychiatry and Psychotherapy, University Hospital Bonn, Bonn, Germany
| | - Kyra Kannen
- Department of Psychiatry and Psychotherapy, University Hospital Bonn, Bonn, Germany
| | - Benjamin Selaskowski
- Department of Psychiatry and Psychotherapy, University Hospital Bonn, Bonn, Germany
| | - Aylin Mehren
- Department of Psychiatry and Psychotherapy, University Hospital Bonn, Bonn, Germany
| | - Ann-Kathrin Thöne
- School of Child and Adolescent Cognitive Behavior Therapy (AKiP), Faculty of Medicine and University Hospital Cologne, University of Cologne, Cologne, Germany
| | - Lisa Pramme
- Department of Psychiatry and Psychotherapy, University Hospital Bonn, Bonn, Germany
| | - Nike Blumenthal
- Department of Psychiatry and Psychotherapy, University Hospital Bonn, Bonn, Germany
| | - Mengtong Li
- Department of Psychiatry and Psychotherapy, University Hospital Bonn, Bonn, Germany
| | - Laura Asché
- Department of Psychiatry and Psychotherapy, University Hospital Bonn, Bonn, Germany
| | - Stephan Jonas
- Institute for Digital Medicine, University Hospital Bonn, Bonn, Germany
| | - Katharina Bey
- Department of Psychiatry and Psychotherapy, University Hospital Bonn, Bonn, Germany
| | - Marcel Schulze
- Department of Psychiatry and Psychotherapy, University Hospital Bonn, Bonn, Germany
| | - Maria Steffens
- Department of Psychiatry and Psychotherapy, University Hospital Bonn, Bonn, Germany
| | - Max Christian Pensel
- Department of Psychiatry and Psychotherapy, University Hospital Bonn, Bonn, Germany
| | - Matthias Guth
- Department of Psychiatry and Psychotherapy, University Hospital Bonn, Bonn, Germany
| | - Felicia Rohlfsen
- Department of Psychiatry and Psychotherapy, University Hospital Bonn, Bonn, Germany
| | - Mogda Ekhlas
- Department of Psychiatry and Psychotherapy, University Hospital Bonn, Bonn, Germany
| | - Helena Lügering
- Department of Psychiatry and Psychotherapy, University Hospital Bonn, Bonn, Germany
| | - Helena Fileccia
- Department of Psychiatry and Psychotherapy, University Hospital Bonn, Bonn, Germany
| | - Julian Pakos
- Department of Psychiatry and Psychotherapy, University Hospital Bonn, Bonn, Germany
| | - Silke Lux
- Department of Psychiatry and Psychotherapy, University Hospital Bonn, Bonn, Germany
| | - Alexandra Philipsen
- Department of Psychiatry and Psychotherapy, University Hospital Bonn, Bonn, Germany
| | - Niclas Braun
- Department of Psychiatry and Psychotherapy, University Hospital Bonn, Bonn, Germany.
| |
Collapse
|