1
|
Palaparthi A, Alluri RK, Titze IR. Deep Learning for Neuromuscular Control of Vocal Source for Voice Production. APPLIED SCIENCES (BASEL, SWITZERLAND) 2024; 14:769. [PMID: 39071945 PMCID: PMC11281313 DOI: 10.3390/app14020769] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Indexed: 07/30/2024]
Abstract
A computational neuromuscular control system that generates lung pressure and three intrinsic laryngeal muscle activations (cricothyroid, thyroarytenoid, and lateral cricoarytenoid) to control the vocal source was developed. In the current study, LeTalker, a biophysical computational model of the vocal system was used as the physical plant. In the LeTalker, a three-mass vocal fold model was used to simulate self-sustained vocal fold oscillation. A constant/ǝ/vowel was used for the vocal tract shape. The trachea was modeled after MRI measurements. The neuromuscular control system generates control parameters to achieve four acoustic targets (fundamental frequency, sound pressure level, normalized spectral centroid, and signal-to-noise ratio) and four somatosensory targets (vocal fold length, and longitudinal fiber stress in the three vocal fold layers). The deep-learning-based control system comprises one acoustic feedforward controller and two feedback (acoustic and somatosensory) controllers. Fifty thousand steady speech signals were generated using the LeTalker for training the control system. The results demonstrated that the control system was able to generate the lung pressure and the three muscle activations such that the four acoustic and four somatosensory targets were reached with high accuracy. After training, the motor command corrections from the feedback controllers were minimal compared to the feedforward controller except for thyroarytenoid muscle activation.
Collapse
Affiliation(s)
- Anil Palaparthi
- Utah Center for Vocology, University of Utah, Salt Lake City, UT 84112, USA
| | - Rishi K. Alluri
- School of Biological Sciences, University of Utah, Salt Lake City, UT 84112, USA
| | - Ingo R. Titze
- Utah Center for Vocology, University of Utah, Salt Lake City, UT 84112, USA
| |
Collapse
|
2
|
Mancone S, Diotaiuti P, Valente G, Corrado S, Bellizzi F, Vilarino GT, Andrade A. The Use of Voice Assistant for Psychological Assessment Elicits Empathy and Engagement While Maintaining Good Psychometric Properties. Behav Sci (Basel) 2023; 13:550. [PMID: 37503997 PMCID: PMC10376154 DOI: 10.3390/bs13070550] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2023] [Revised: 06/20/2023] [Accepted: 06/28/2023] [Indexed: 07/29/2023] Open
Abstract
This study aimed to use the Alexa vocal assistant as an administerer of psychometric tests, assessing the efficiency and validity of this measurement. A total of 300 participants were administered the Interpersonal Reactivity Index (IRI). After a week, the administration was repeated, but the participants were randomly divided into groups of 100 participants each. In the first, the test was administered by means of a paper version; in the second, the questionnaire was read to the participants in person, and the operator contemporaneously recorded the answers declared by the participants; in the third group, the questionnaire was directly administered by the Alexa voice device, after specific reprogramming. The third group was also administered, as a post-session survey, the Engagement and Perceptions of the Bot Scale (EPVS), a short version of the Communication Styles Inventory (CSI), the Marlowe-Crowne Social Desirability Scale (MCSDS), and an additional six items to measure degrees of concentration, ease, and perceived pressure at the beginning and at the end of the administration. The results confirmed that the IRI did keep measurement invariance within the three conditions. The administration through vocal assistant showed an empathic activation effect significantly superior to the conditions of pencil-paper and operator-in-presence. The results indicated an engagement and positive evaluation of the interactive experience, with reported perceptions of closeness, warmth, competence, and human-likeness associated with higher values of empathetic activation and lower values of personal discomfort.
Collapse
Affiliation(s)
- Stefania Mancone
- Department of Human Sciences, Society and Health, University of Cassino and Southern Lazio, 03043 Cassino, Italy
| | - Pierluigi Diotaiuti
- Department of Human Sciences, Society and Health, University of Cassino and Southern Lazio, 03043 Cassino, Italy
| | - Giuseppe Valente
- Department of Human Sciences, Society and Health, University of Cassino and Southern Lazio, 03043 Cassino, Italy
| | - Stefano Corrado
- Department of Human Sciences, Society and Health, University of Cassino and Southern Lazio, 03043 Cassino, Italy
| | - Fernando Bellizzi
- Department of Human Sciences, Society and Health, University of Cassino and Southern Lazio, 03043 Cassino, Italy
| | - Guilherme Torres Vilarino
- Health and Sports Science Center, Department of Physical Education, Santa Catarina State University, Florianópolis 88035-901, Brazil
| | - Alexandro Andrade
- Health and Sports Science Center, Department of Physical Education, Santa Catarina State University, Florianópolis 88035-901, Brazil
| |
Collapse
|
3
|
Kannampallil T, Ajilore OA, Lv N, Smyth JM, Wittels NE, Ronneberg CR, Kumar V, Xiao L, Dosala S, Barve A, Zhang A, Tan KC, Cao KK, Patel CR, Gerber BS, Johnson JA, Kringle EA, Ma J. Effects of a virtual voice-based coach delivering problem-solving treatment on emotional distress and brain function: a pilot RCT in depression and anxiety. Transl Psychiatry 2023; 13:166. [PMID: 37173334 PMCID: PMC10175049 DOI: 10.1038/s41398-023-02462-x] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/06/2022] [Revised: 04/14/2023] [Accepted: 04/26/2023] [Indexed: 05/15/2023] Open
Abstract
Consumer-based voice assistants have the ability to deliver evidence-based treatment, but their therapeutic potential is largely unknown. In a pilot trial of a virtual voice-based coach, Lumen, delivering problem-solving treatment, adults with mild-to-moderate depression and/or anxiety were randomized to the Lumen intervention (n = 42) or waitlist control (n = 21). The main outcomes included changes in neural measures of emotional reactivity and cognitive control, and Hospital Anxiety and Depression Scale [HADS] symptom scores over 16 weeks. Participants were 37.8 years (SD = 12.4), 68% women, 25% Black, 24% Latino, and 11% Asian. Activation of the right dlPFC (neural region of interest in cognitive control) decreased in the intervention group but increased in the control group, with an effect size meeting the prespecified threshold for a meaningful effect (Cohen's d = 0.3). Between-group differences in the change in activation of the left dlPFC and bilateral amygdala were observed, but were of smaller magnitude (d = 0.2). Change in right dlPFC activation was also meaningfully associated (r ≥ 0.4) with changes in self-reported problem-solving ability and avoidance in the intervention. Lumen intervention also led to decreased HADS depression, anxiety, and overall psychological distress scores, with medium effect sizes (Cohen's d = 0.49, 0.51, and 0.55, respectively), compared with the waitlist control group. This pilot trial showed promising effects of a novel digital mental health intervention on cognitive control using neuroimaging and depression and anxiety symptoms, providing foundational evidence for a future confirmatory study.
Collapse
Affiliation(s)
- Thomas Kannampallil
- Department of Anesthesiology, Washington University School of Medicine, St Louis, MO, USA
- Institute for Informatics, Washington University School of Medicine, St Louis, MO, USA
| | - Olusola A Ajilore
- Department of Psychiatry, University of Illinois at Chicago, Chicago, IL, USA
| | - Nan Lv
- Department of Medicine, University of Illinois at Chicago, Chicago, IL, USA
| | - Joshua M Smyth
- Department of Biobehavioral Health, The Pennsylvania State University, University Park, PA, USA
| | - Nancy E Wittels
- Department of Medicine, University of Illinois at Chicago, Chicago, IL, USA
| | - Corina R Ronneberg
- Department of Medicine, University of Illinois at Chicago, Chicago, IL, USA
| | - Vikas Kumar
- Department of Medicine, University of Illinois at Chicago, Chicago, IL, USA
| | - Lan Xiao
- Department of Epidemiology and Population Health, Stanford University, Stanford, USA
| | - Susanth Dosala
- Department of Medicine, University of Illinois at Chicago, Chicago, IL, USA
| | - Amruta Barve
- Department of Medicine, University of Illinois at Chicago, Chicago, IL, USA
| | - Aifeng Zhang
- Department of Psychiatry, University of Illinois at Chicago, Chicago, IL, USA
| | - Kevin C Tan
- Department of Medicine, University of Illinois at Chicago, Chicago, IL, USA
| | - Kevin K Cao
- Department of Medicine, University of Illinois at Chicago, Chicago, IL, USA
| | - Charmi R Patel
- Department of Medicine, University of Illinois at Chicago, Chicago, IL, USA
| | - Ben S Gerber
- Department of Population & Quantitative Health Sciences, University of Massachusetts Medical School, Worcester, MA, USA
| | - Jillian A Johnson
- Department of Biobehavioral Health, The Pennsylvania State University, University Park, PA, USA
| | - Emily A Kringle
- Department of Medicine, University of Illinois at Chicago, Chicago, IL, USA
| | - Jun Ma
- Department of Medicine, University of Illinois at Chicago, Chicago, IL, USA.
| |
Collapse
|
4
|
Kannampallil T, Ronneberg CR, Wittels NE, Kumar V, Lv N, Smyth JM, Gerber BS, Kringle EA, Johnson JA, Yu P, Steinman LE, Ajilore OA, Ma J. Design and Formative Evaluation of a Virtual Voice-Based Coach for Problem-solving Treatment: Observational Study. JMIR Form Res 2022; 6:e38092. [PMID: 35969431 PMCID: PMC9419044 DOI: 10.2196/38092] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2022] [Revised: 07/01/2022] [Accepted: 07/04/2022] [Indexed: 11/13/2022] Open
Abstract
Background
Artificial intelligence has provided new opportunities for human interactions with technology for the practice of medicine. Among the recent artificial intelligence innovations, personal voice assistants have been broadly adopted. This highlights their potential for health care–related applications such as behavioral counseling to promote healthy lifestyle habits and emotional well-being. However, the use of voice-based applications for behavioral therapy has not been previously evaluated.
Objective
This study aimed to conduct a formative user evaluation of Lumen, a virtual voice-based coach developed as an Alexa skill that delivers evidence-based, problem-solving treatment for patients with mild to moderate depression and/or anxiety.
Methods
A total of 26 participants completed 2 therapy sessions—an introductory (session 1) and a problem-solving (session 2)—with Lumen. Following each session with Lumen, participants completed user experience, task-related workload, and work alliance surveys. They also participated in semistructured interviews addressing the benefits, challenges and barriers to Lumen use, and design recommendations. We evaluated the differences in user experience, task load, and work alliance between sessions using 2-tailed paired t tests. Interview transcripts were coded using an inductive thematic analysis to characterize the participants’ perspectives regarding Lumen use.
Results
Participants found Lumen to provide high pragmatic usability and favorable user experience, with marginal task load during interactions for both Lumen sessions. However, participants experienced a higher temporal workload during the problem-solving session, suggesting a feeling of being rushed during their communicative interactions. On the basis of the qualitative analysis, the following themes were identified: Lumen’s on-demand accessibility and the delivery of a complex problem-solving treatment task with a simplistic structure for achieving therapy goals; themes related to Lumen improvements included streamlining and improved personalization of conversations, slower pacing of conversations, and providing additional context during therapy sessions.
Conclusions
On the basis of an in-depth formative evaluation, we found that Lumen supported the ability to conduct cognitively plausible interactions for the delivery of behavioral therapy. Several design suggestions identified from the study including reducing temporal and cognitive load during conversational interactions, developing more natural conversations, and expanding privacy and security features were incorporated in the revised version of Lumen. Although further research is needed, the promising findings from this study highlight the potential for using Lumen to deliver personalized and accessible mental health care, filling a gap in traditional mental health services.
Collapse
Affiliation(s)
| | | | - Nancy E Wittels
- University of Illinois at Chicago, Chicago, IL, United States
| | - Vikas Kumar
- University of Illinois at Chicago, Chicago, IL, United States
| | - Nan Lv
- University of Illinois at Chicago, Chicago, IL, United States
| | - Joshua M Smyth
- The Pennsylvania State University, University Park, PA, United States
| | - Ben S Gerber
- University of Massachusetts Medical School, Worcester, MA, United States
| | - Emily A Kringle
- University of Illinois at Chicago, Chicago, IL, United States
| | - Jillian A Johnson
- The Pennsylvania State University, University Park, PA, United States
| | - Philip Yu
- University of Illinois at Chicago, Chicago, IL, United States
| | | | - Olu A Ajilore
- University of Illinois at Chicago, Chicago, IL, United States
| | - Jun Ma
- University of Illinois at Chicago, Chicago, IL, United States
| |
Collapse
|
5
|
Affiliation(s)
| | - Xiao Lin
- Nanjing University of Aeronautics and Astronautics, Nanjing, China
| | | | | | - Jian Mou
- School of Business, Pusan National University, 2, Busandaehak-ro 63beon-gil, Geumjeong-gu, Busan, 46241 South Korea
| |
Collapse
|
6
|
Kannampallil T, Ma J. Digital Translucence: Adapting Telemedicine Delivery Post-COVID-19. Telemed J E Health 2020; 26:1120-1122. [PMID: 32427529 DOI: 10.1089/tmj.2020.0158] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/04/2023] Open
Abstract
In nearly 1 month, with a rapidly expanding corona virus disease 2019 (COVID-19), telemedicine has been transformed into an essential service for delivering routine clinical care. This transformation occurred as a crisis management response-driven by the need to provide care for patients with physical distancing measures in place. However, the current rapid adoption of telemedicine presents a transitional state between one that existed before the pandemic and one that could potentially be better aligned with the delivery of a personalized model of care. Using the conceptual framework of digital translucence-situating virtual encounters with more nuanced information regarding patients-we describe the role of integrated remote monitoring and virtual care tools aligned with the patient's electronic health record for adapting telemedicine delivery post-COVID-19.
Collapse
Affiliation(s)
- Thomas Kannampallil
- Institute for Informatics, Washington University School of Medicine, St. Louis, Missouri, USA.,Department of Anesthesiology, Washington University School of Medicine, St. Louis, Missouri, USA
| | - Jun Ma
- Department of Medicine, College of Medicine, University of Illinois at Chicago, Chicago, Illinois, USA
| |
Collapse
|