1
|
Landers C, Blasimme A, Vayena E. Sync fast and solve things-best practices for responsible digital health. NPJ Digit Med 2024; 7:113. [PMID: 38704413 PMCID: PMC11069566 DOI: 10.1038/s41746-024-01105-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2023] [Accepted: 04/15/2024] [Indexed: 05/06/2024] Open
Abstract
Digital health innovation is expected to transform healthcare, but it also generates ethical and societal concerns, such as privacy risks, and biases that can compound existing health inequalities. While such concerns are widely recognized, existing regulatory principles, oversight methods and ethical frameworks seem out of sync with digital health innovation. New governance and innovation best practices are thus needed to bring such principles to bear with the reality of business, innovation, and regulation.To grant practical insight into best practices for responsible digital health innovation, we conducted a qualitative study based on an interactive engagement methodology. We engaged key stakeholders (n = 46) operating at the translational frontier of digital health. This approach allowed us to identify three clusters of governance and innovation best practices in digital health innovation: i) inclusive co-creation, ii) responsive regulation, and iii) value-driven innovation. Our study shows that realizing responsible digital health requires diverse stakeholders' commitment to adapt innovation and regulation practices, embracing co-creation as the default modus operandi for digital health development. We describe these collaborative practices and show how they can ensure that innovation is neither slowed by overregulation, nor leads to unethical outcomes.
Collapse
Affiliation(s)
| | | | - Effy Vayena
- Health Ethics and Policy Lab, ETH Zurich, Switzerland.
| |
Collapse
|
2
|
Krisher L, Boeldt DL, Sigmon CAN, Rimel SE, Newman LS. Pragmatic Approach to the Assessment and Use of Digital Mental Health Interventions for Health Workers. Am J Public Health 2024; 114:171-179. [PMID: 38354345 PMCID: PMC10916727 DOI: 10.2105/ajph.2023.307505] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/14/2023] [Indexed: 02/16/2024]
Abstract
Symptoms of poor mental health among working people, especially health workers, are on the rise in the United States, contributing to a burgeoning market of thousands of mental health technology products, few of which have undergone rigorous evaluation. Most research on these products focuses on deploying digital mental health interventions as ancillary support in clinical practice and community settings. Little is known of the effectiveness of these tools when employers offer them. We describe the landscape of digital mental health interventions, providing an overview of mental health conditions that are addressed with technology-based solutions in the workplace and the products and services available. We argue for employers to apply a methodical approach to evaluating and selecting technologies for their organizations, and we review relevant frameworks for evaluation. Considering the rapidly evolving landscape of digital mental health interventions, we offer evidence-informed recommendations to organizations and decision-makers seeking to support workplace mental health and well-being, and we advocate the development of products that help organizations assess how they can mitigate workplace conditions that may contribute to poorer mental health. (Am J Public Health. 2024;114(S2):S171-S179. https://doi.org/10.2105/AJPH.2023.307505).
Collapse
Affiliation(s)
- Lyndsay Krisher
- Lyndsay Krisher is with the Center for Health, Work & Environment and the Department of Environmental & Occupational Health, Colorado School of Public Health, University of Colorado Anschutz Medical Campus, Aurora. Debra L. Boeldt, Chloe A. Nicksic Sigmon, and Sarah E. Rimel are with the National Mental Health Innovation Center, University of Colorado Anschutz Medical Campus. Lee S. Newman is with the Center for Health, Work & Environment and the Department of Environmental & Occupational Health, Colorado School of Public Health, Aurora
| | - Debra L Boeldt
- Lyndsay Krisher is with the Center for Health, Work & Environment and the Department of Environmental & Occupational Health, Colorado School of Public Health, University of Colorado Anschutz Medical Campus, Aurora. Debra L. Boeldt, Chloe A. Nicksic Sigmon, and Sarah E. Rimel are with the National Mental Health Innovation Center, University of Colorado Anschutz Medical Campus. Lee S. Newman is with the Center for Health, Work & Environment and the Department of Environmental & Occupational Health, Colorado School of Public Health, Aurora
| | - Chloe A Nicksic Sigmon
- Lyndsay Krisher is with the Center for Health, Work & Environment and the Department of Environmental & Occupational Health, Colorado School of Public Health, University of Colorado Anschutz Medical Campus, Aurora. Debra L. Boeldt, Chloe A. Nicksic Sigmon, and Sarah E. Rimel are with the National Mental Health Innovation Center, University of Colorado Anschutz Medical Campus. Lee S. Newman is with the Center for Health, Work & Environment and the Department of Environmental & Occupational Health, Colorado School of Public Health, Aurora
| | - Sarah E Rimel
- Lyndsay Krisher is with the Center for Health, Work & Environment and the Department of Environmental & Occupational Health, Colorado School of Public Health, University of Colorado Anschutz Medical Campus, Aurora. Debra L. Boeldt, Chloe A. Nicksic Sigmon, and Sarah E. Rimel are with the National Mental Health Innovation Center, University of Colorado Anschutz Medical Campus. Lee S. Newman is with the Center for Health, Work & Environment and the Department of Environmental & Occupational Health, Colorado School of Public Health, Aurora
| | - Lee S Newman
- Lyndsay Krisher is with the Center for Health, Work & Environment and the Department of Environmental & Occupational Health, Colorado School of Public Health, University of Colorado Anschutz Medical Campus, Aurora. Debra L. Boeldt, Chloe A. Nicksic Sigmon, and Sarah E. Rimel are with the National Mental Health Innovation Center, University of Colorado Anschutz Medical Campus. Lee S. Newman is with the Center for Health, Work & Environment and the Department of Environmental & Occupational Health, Colorado School of Public Health, Aurora
| |
Collapse
|
3
|
Olaya F, Brin M, Caraballo PB, Halpern M, Jia H, Ramírez SO, Padilla JJ, Stonbraker S, Schnall R. A randomized controlled trial of the dissemination of an mHealth intervention for improving health outcomes: the WiseApp for Spanish-speakers living with HIV study protocol. BMC Public Health 2024; 24:201. [PMID: 38233908 PMCID: PMC10792787 DOI: 10.1186/s12889-023-17538-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2023] [Accepted: 12/19/2023] [Indexed: 01/19/2024] Open
Abstract
BACKGROUND While there is no cure for HIV, adherence to antiretroviral therapy can extend the lifespan and improve the quality of life of people with HIV. Despite the global reduction of HIV infection rates in recent years, New York City and La Romana, Dominican Republic, continue to report high infection rates among Latino populations. Many people with HIV remain virally unsuppressed in these geographic hotspots, suggesting a need for additional interventions to overcome medication adherence barriers. Tailored and culturally appropriate mobile health (mHealth) technology can be an engaging way to improve adherence. The primary objective of this trial is to test the effectiveness of an mHealth tool to improve HIV medication adherence among Spanish-speaking people living in New York City and the Dominican Republic. METHODS The WiseApp study is a two-arm randomized controlled trial among 248 people with HIV across the New York and Dominican Republic sites over the course of 12 months. Participants are randomly assigned to either receive a CleverCap pill bottle that is linked to the WiseApp (intervention) or standard of care (control). All participants complete surveys at baseline, 3-month, 6-month, and 12-month follow-up visits and the study team obtains HIV-1 viral load and CD4 count results through blood draw at each study timepoint. DISCUSSION The use of mHealth technologies to improve medication adherence among people with HIV has been implemented in recent years. Although some studies have found improvement in adherence to antiretroviral therapy in the short term, there is limited information about how these interventions improve adherence among Spanish-speaking populations. Disproportionate rates of HIV infection among Latinos in New York City suggest an existing inequitable approach in reaching and treating this population. Due to a lack of mHealth studies with Latino populations, and apps tailored to Spanish-speakers, the WiseApp study will not only demonstrate the effectiveness of this particular mHealth app but will also contribute to the mHealth research community as a whole. TRIAL REGISTRATION This trial was registered with Clinicaltrials.gov (NCT05398185) on 5/31/2022.
Collapse
Affiliation(s)
- Felix Olaya
- School of Nursing, Columbia University, 560 W 168th St, New York, NY, 10032, USA
| | - Maeve Brin
- School of Nursing, Columbia University, 560 W 168th St, New York, NY, 10032, USA
| | | | - Mina Halpern
- Clínica de Familia La Romana, La Romana, Dominican Republic
| | - Haomiao Jia
- School of Nursing, Columbia University, 560 W 168th St, New York, NY, 10032, USA
| | - Sergio Ozoria Ramírez
- NYU Steinhardt School of Culture, Education, and Human Development, New York, NY, 10003, USA
| | | | - Samantha Stonbraker
- University of Colorado College of Nursing, Anschutz Medical Campus, Aurora, CO, 80045, USA
| | - Rebecca Schnall
- School of Nursing, Columbia University, 560 W 168th St, New York, NY, 10032, USA.
| |
Collapse
|
4
|
Dinesh DN, Rao MN, Sinha C. Language adaptations of mental health interventions: User interaction comparisons with an AI-enabled conversational agent (Wysa) in English and Spanish. Digit Health 2024; 10:20552076241255616. [PMID: 38798884 PMCID: PMC11119503 DOI: 10.1177/20552076241255616] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2023] [Accepted: 04/29/2024] [Indexed: 05/29/2024] Open
Abstract
Background In recent times, digital mental health interventions (DMHIs) have been proven to be efficacious; however, most are available only for English speakers, leaving limited options for non-English languages like Spanish. Research shows that mental health services in one's dominant language show better outcomes. Conversational agents (CAs) offer promise in supporting mental health in non-English populations. This study compared a culturally adapted version of an artificial intelligence (AI)-led mental health app, called Wysa, in Spanish and English. Objectives To compare user engagement patterns on Wysa-Spanish and Wysa-English and to understand expressions of distress and preferred language in both versions of Wysa. Methods We adopted a cross-sectional retrospective exploratory design with mixed methods, analyzing users from 10 Spanish-speaking countries between 1 February and 1 August 2022. A quantitative sample A (n = 2767) was used for descriptive statistics, including user engagement metrics with a Wilcoxon test. A subset qualitative sample B (n = 338) was examined for word count differences based on valence, and a content analysis was conducted to examine idioms of distress. Results Compared to Wysa-English, Wysa-Spanish had more sessions (P < .001, d = 0.18) and a greater volume of disclosure of distress. In Wysa-Spanish, the average length of a conversation was significantly longer than in Wysa-English (P < .001, d = 0.44). Users preferred interventions with free text responses ("Thought recording") in Spanish (P < .01, d = 0.41), and Spanish messages were significantly longer (P < .01, d = 0.24). Wysa-Spanish saw more frequent expressions of negative emotions and feelings of self-harm and suicide. Conclusion Given the high engagement within the Spanish version of Wysa, the findings demonstrate the need for culturally adapted DMHIs among non-English populations, emphasizing the importance of considering linguistic and cultural differences in the development of DMHIs to improve accessibility for diverse populations.
Collapse
Affiliation(s)
| | - M Namrata Rao
- Healthcare and Clinical Development, Wysa Inc., Boston, Massachusetts, USA
| | - Chaitali Sinha
- Healthcare and Clinical Development, Wysa Inc., Boston, Massachusetts, USA
| |
Collapse
|
5
|
Jibb LA, Sivaratnam S, Hashemi E, Chu CH, Nathan PC, Chartrand J, Alberts NM, Masama T, Pease HG, Torres LB, Cortes HG, Zworth M, Kuczynski S, Fortier MA. Parent and clinician perceptions and recommendations on a pediatric cancer pain management app: A qualitative co-design study. PLOS DIGITAL HEALTH 2023; 2:e0000169. [PMID: 38019890 PMCID: PMC10686487 DOI: 10.1371/journal.pdig.0000169] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/22/2022] [Accepted: 10/14/2023] [Indexed: 12/01/2023]
Abstract
Pain is one of the most prevalent and burdensome pediatric cancer symptoms for young children and their families. A significant proportion of pain episodes are experienced in environments where management options are limited, including at home. Digital innovations such as apps may have positive impacts on pain outcomes for young children in these environments. Our overall aim is to co-design such an app and the objective of this study was to explore the perceptions of children's parents about app utility, needed system features, and challenges. We recruited parents of young children with cancer and multidisciplinary pediatric oncology clinicians from two pediatric cancer care centers to participate in audio-recorded, semi-structured, co-design interviews. We conducted interviews structured around technology acceptance and family caregiving theories until data saturation was reached. Audio-recordings were then transcribed, coded, and analyzed using thematic analysis. Forty-two participants took part in the process. Participants endorsed the concept of an app as a useful, safe, and convenient way to engage caregivers in managing their young child's pain. Overall, the app was valued as a means to provide real-time, multimodal informational and procedural pain support to parents, while also reducing the emotional burden of pain care. Recommendations for intervention design included accessibility-focused features, comprehensive symptom tracking, and embedded scientific- and clinically-sound symptom assessments and management advice. Predicted challenges to app use included the workload burden it may place on parents and clinicians. The insights gathered will inform the design principles of our future childhood cancer pain digital research.
Collapse
Affiliation(s)
- Lindsay A. Jibb
- Lawrence S. Bloomberg Faculty of Nursing, University of Toronto, Toronto, Canada
- Child Health Evaluative Sciences, Hospital for Sick Children, Toronto, Canada
| | - Surabhi Sivaratnam
- Child Health Evaluative Sciences, Hospital for Sick Children, Toronto, Canada
- Temerty Faculty of Medicine, University of Toronto, Toronto, Canada
| | - Elham Hashemi
- Child Health Evaluative Sciences, Hospital for Sick Children, Toronto, Canada
| | - Charlene H. Chu
- Lawrence S. Bloomberg Faculty of Nursing, University of Toronto, Toronto, Canada
- KITE-Toronto Rehabilitation Institute, University Health Network, Toronto, Canada
| | - Paul C. Nathan
- Child Health Evaluative Sciences, Hospital for Sick Children, Toronto, Canada
- Temerty Faculty of Medicine, University of Toronto, Toronto, Canada
- Institute of Health Policy Management and Evaluation, University of Toronto, Toronto, Canada
- Division of Hematology/Oncology, Hospital for Sick Children, Toronto, Canada
| | - Julie Chartrand
- Faculty of Health Sciences, University of Ottawa, Ottawa, Canada
- Research Institute, Children’s Hospital of Eastern Ontario, Ottawa, Canada
| | | | - Tatenda Masama
- Division of Hematology/Oncology, Hospital for Sick Children, Toronto, Canada
| | - Hannah G. Pease
- Sue and Bill Gross School of Nursing, University of California Irvine, Irvine, California, United States of America
| | - Lessley B. Torres
- Sue and Bill Gross School of Nursing, University of California Irvine, Irvine, California, United States of America
- Department of Pediatric Psychology, Children’s Hospital of Orange County, Orange, California, United States of America
- UCI Center on Stress and Health, University of California Irvine, Irvine, California, United States of America
| | - Haydee G. Cortes
- Sue and Bill Gross School of Nursing, University of California Irvine, Irvine, California, United States of America
- Department of Pediatric Psychology, Children’s Hospital of Orange County, Orange, California, United States of America
- UCI Center on Stress and Health, University of California Irvine, Irvine, California, United States of America
| | - Mallory Zworth
- Division of Hematology/Oncology, Hospital for Sick Children, Toronto, Canada
| | - Susan Kuczynski
- Ontario Parents Advocating for Children with Cancer, Toronto, Canada
| | - Michelle A. Fortier
- Sue and Bill Gross School of Nursing, University of California Irvine, Irvine, California, United States of America
- Department of Pediatric Psychology, Children’s Hospital of Orange County, Orange, California, United States of America
- UCI Center on Stress and Health, University of California Irvine, Irvine, California, United States of America
| |
Collapse
|
6
|
Lesher A, McDuffie L, Smith T, Foster A, Ruggiero K, Barroso J, Gavrilova Y. Optimizing an Outpatient mHealth Intervention for Children with Burns: A Convergent Mixed-Methods Study. J Burn Care Res 2023; 44:1092-1099. [PMID: 36779787 PMCID: PMC10483473 DOI: 10.1093/jbcr/irad020] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/27/2022] [Indexed: 02/14/2023]
Abstract
Burn injury is one of the most common traumatic injuries in childhood. Fortunately, 90% of pediatric burns may be treated in the outpatient setting after appropriate burn triage. Patients with burns face significant geographic disparities in accessing expert burn care due to regionalized care. To aid patients and their families during acute outpatient burn recovery, we developed a smartphone app, Telemedicine Optimized Burn Intervention (TOBI). With this app, we aimed to increase access to care by allowing secure, streamlined communication between patients and burn providers, including messaging and wound image transfer. The purpose of this study was to systematically evaluate user feedback to optimize the patient and provider experience. TOBI was evaluated using a convergent mixed-methods approach consisting of qualitative semi-structured interviews and quantitative measurements of app usability via the mHealth App Usability Questionnaire. Participants included 15 caregivers of pediatric patients with burns who used TOBI during treatment and ten burn providers. Users found TOBI to be a highly usable application in terms of usefulness, ease of use, satisfaction, and functionality. Qualitative data provided insight into user experience, satisfaction and preferences, difficulty navigating, usability and acceptability, and potential improvements. Although most users were highly satisfied, improvements were needed to optimize the burn app. We systematically made these improvements before we released TOBI for routine patient use. This study uncovered helpful recommendations for app improvements that can be generalized to other mobile health apps to increase their appeal and adoption.
Collapse
Affiliation(s)
- Aaron Lesher
- Department of Surgery, Medical University of South Carolina, Charleston, SC, USA
| | - Lucas McDuffie
- Department of Surgery, Medical University of South Carolina, Charleston, SC, USA
| | - Tiffany Smith
- Department of Surgery, Medical University of South Carolina, Charleston, SC, USA
| | - Abigail Foster
- Department of Public Health, College of Charleston, Charleston, SC, USA
| | - Kenneth Ruggiero
- College of Nursing, Medical University of South Carolina, Charleston, SC, USA
| | - Julie Barroso
- School of Nursing, Vanderbilt University, Nashville, TN, USA
| | - Yulia Gavrilova
- Department of Surgery, Medical University of South Carolina, Charleston, SC, USA
- Department of Psychiatry and Behavioral Sciences, Medical University of South Carolina, Charleston, SC, USA
| |
Collapse
|
7
|
Lopez B, Gottlieb BR, Naples JG. Longer Times to Delivery of Otolaryngology Care for Patients With Limited English Proficiency. Otolaryngol Head Neck Surg 2023; 169:651-659. [PMID: 37194741 DOI: 10.1002/ohn.363] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/05/2023] [Revised: 03/28/2023] [Accepted: 04/06/2023] [Indexed: 05/18/2023]
Abstract
OBJECTIVE Limited English proficiency (LEP) is known to contribute to poorer health outcomes and delays in management. However, to our knowledge, no other studies have explored the impact of LEP on delays to care within otolaryngology. This study aims to investigate the relationship between LEP and the time to delivery of otolaryngology care. METHODS We retrospectively reviewed 1125 electronic referrals to an otolaryngologist from primary care providers at 2 health centers in the greater Boston area, between January 2015 and December 2019. Multivariable logistic regression analyses were conducted to determine if patient LEP status (preferred language non-English and language interpreter use) has an impact on total time to appointment (TTTA). RESULTS Patients with non-English preferred languages were 2.6 times more likely to experience extended TTTA (odds ratio [OR] = 2.61, 95% confidence interval [CI] = 1.99-3.42, p < .001) relative to English-speaking patients. Patients who required interpreter use were 2.4 times more likely to experience extended TTTA (OR = 2.42, 95% CI = 1.84-3.18, p < .001) relative to patients who did not require an interpreter. There was no difference in age, sex, insurance type, education level, or marital status. TTTA did not vary by diagnosis category (p = .09). DISCUSSION LEP is an important factor that influences the time to appointment in our cohort. Notably, the impact of LEP on appointment wait times was independent of diagnosis. IMPLICATIONS FOR PRACTICE Clinicians should recognize LEP as a factor that can impact the overall delivery of care in otolaryngology. Specifically, mechanisms to streamline care for LEP patients should be considered.
Collapse
Affiliation(s)
- Betzamel Lopez
- Harvard Medical School, Boston, Massachusetts, USA
- Department of Otolaryngology-Head and Neck Surgery, University of Miami School of Medicine, Miami, Florida, USA
| | - Barbara R Gottlieb
- Harvard Medical School, Boston, Massachusetts, USA
- Department of Medicine, Brigham and Women's Hospital, Boston, Massachusetts, USA
| | - James G Naples
- Harvard Medical School, Boston, Massachusetts, USA
- Department of Otolaryngology-Head and Neck Surgery, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, Massachusetts, USA
| |
Collapse
|
8
|
Lyles CR, Nguyen OK, Khoong EC, Aguilera A, Sarkar U. Multilevel Determinants of Digital Health Equity: A Literature Synthesis to Advance the Field. Annu Rev Public Health 2023; 44:383-405. [PMID: 36525960 PMCID: PMC10329412 DOI: 10.1146/annurev-publhealth-071521-023913] [Citation(s) in RCA: 20] [Impact Index Per Article: 20.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/23/2022]
Abstract
Current digital health approaches have not engaged diverse end users or reduced health or health care inequities, despite their promise to deliver more tailored and personalized support to individuals at the right time and the right place. To achieve digital health equity, we must refocus our attention on the current state of digital health uptake and use across the policy, system, community, individual, and intervention levels. We focus here on (a) outlining a multilevel framework underlying digital health equity; (b) summarizingfive types of interventions/programs (with example studies) that hold promise for advancing digital health equity; and (c) recommending future steps for improving policy, practice, and research in this space.
Collapse
Affiliation(s)
- Courtney R Lyles
- Department of Medicine, Division of General Internal Medicine at Zuckerberg San Francisco General Hospital, University of California-San Francisco, San Francisco, California, USA;
- Department of Epidemiology and Biostatistics, University of California-San Francisco, San Francisco, California, USA
- Center for Vulnerable Populations, University of California-San Francisco, San Francisco, California, USA
- School of Public Health, University of California-Berkeley, Berkeley, California, USA
| | - Oanh Kieu Nguyen
- Center for Vulnerable Populations, University of California-San Francisco, San Francisco, California, USA
- Department of Medicine, Division of Hospital Medicine at Zuckerberg San Francisco General Hospital, University of California-San Francisco, San Francisco, California, USA
- Chan Zuckerberg Biohub, San Francisco, California, USA
| | - Elaine C Khoong
- Department of Medicine, Division of General Internal Medicine at Zuckerberg San Francisco General Hospital, University of California-San Francisco, San Francisco, California, USA;
- Center for Vulnerable Populations, University of California-San Francisco, San Francisco, California, USA
| | - Adrian Aguilera
- Center for Vulnerable Populations, University of California-San Francisco, San Francisco, California, USA
- School of Social Welfare, University of California-Berkeley, Berkeley, California, USA
- Department of Psychiatry, University of California-San Francisco, San Francisco, California, USA
| | - Urmimala Sarkar
- Department of Medicine, Division of General Internal Medicine at Zuckerberg San Francisco General Hospital, University of California-San Francisco, San Francisco, California, USA;
- Center for Vulnerable Populations, University of California-San Francisco, San Francisco, California, USA
| |
Collapse
|
9
|
Shroff A, Roulston C, Fassler J, Dierschke NA, Todd JSP, Ríos-Herrera Á, Plastino KA, Schleider JL. A Digital Single-Session Intervention Platform for Youth Mental Health: Cultural Adaptation, Evaluation, and Dissemination. JMIR Ment Health 2023; 10:e43062. [PMID: 36787180 PMCID: PMC9975917 DOI: 10.2196/43062] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/28/2022] [Revised: 01/09/2023] [Accepted: 01/09/2023] [Indexed: 02/15/2023] Open
Abstract
BACKGROUND Despite the proliferation of evidence-based digital mental health programs for young people, their low uptake and inconsistent implementation preclude them from benefiting youths at scale. Identifying effective implementation strategies for evidence-based supports is especially critical in regions where treatment access is lowest owing to mental health provider shortages. OBJECTIVE The goal of this academic-community partnership, funded by the City of San Antonio Metropolitan Health District, was to culturally adapt, disseminate, and gauge the acceptability and utility of an evidence-based digital mental health platform-Project Youth Empowerment and Support (YES)-among English- and Spanish-speaking youths living in south Texas. METHODS Project YES is an open-access, anonymous platform containing 3 evidence-based, self-guided interventions for youth mental health. Project YES was culturally adapted via focus groups and co-design sessions with San Antonio youths with lived experience of depression and anxiety; translated into Spanish; and disseminated throughout San Antonio, Texas, via community and school partnerships. RESULTS During the project period (April 2021 to December 2021), 1801 San Antonio youths began and 894 (49.64%) of them completed a 30-minute, single-session intervention within Project YES (aged 11-17 years; n=718, 39.87% male; n=961, 53.36% female; and n=3, 0.17% intersex; n=1477, 82.01% Hispanic; n=77, 4.28% non-Hispanic White; n=113, 6.27% Black; n=28, 1.55% Asian; and n=93, 5.16% other). This completion rate (49.64%) surpassed those previously observed for Project YES (eg, 34% when disseminated via social media). San Antonio youths rated Project YES as highly acceptable across all metrics, both in English and Spanish. In addition, the youths who completed Project YES-ENGLISH reported significant improvements in hopelessness (Cohen d=0.33; P<001), self-hate (Cohen d=0.27; P<001), and perceived agency (Cohen d=0.25; P<001) from before to after the intervention, and the youths who completed Project YES-SPANISH reported significant improvements in self-hate (Cohen d=0.37; P=.049) from before to after the intervention. CONCLUSIONS The results indicate that Project YES-an open-access, free, and anonymous web-based single-session intervention platform-is an acceptable, accessible, and applicable mental health support for English- and Spanish-speaking San Antonio youths.
Collapse
Affiliation(s)
- Akash Shroff
- Department of Psychology, Stony Brook University, Stony Brook, NY, United States
| | - Chantelle Roulston
- Department of Psychology, Stony Brook University, Stony Brook, NY, United States
| | - Julia Fassler
- Department of Psychology, Stony Brook University, Stony Brook, NY, United States
| | - Nicole A Dierschke
- University of Texas Teen Health, University of Texas Health Science Center at San Antonio, San Antonio, TX, United States
| | - Jennifer San Pedro Todd
- University of Texas Teen Health, University of Texas Health Science Center at San Antonio, San Antonio, TX, United States
| | - Ámbar Ríos-Herrera
- University of Texas Teen Health, University of Texas Health Science Center at San Antonio, San Antonio, TX, United States
| | - Kristen A Plastino
- University of Texas Teen Health, University of Texas Health Science Center at San Antonio, San Antonio, TX, United States
| | | |
Collapse
|
10
|
Abstract
IMPORTANCE As more patients and clinicians are turning to mental health smartphone apps to expand access to services, little is known about the current state of the app marketplaces and what these apps are actually offering in terms of features, privacy, price, and services. OBJECTIVE To assess the current state of mental health apps, explore the association between app privacy scores and popularity as measured by star ratings and downloads, and to understand opportunities and challenges facing the commercial app landscape. DESIGN, SETTING, AND PARTICIPANTS This cross-sectional study had trained raters using the public-facing M-Health Index and Navigation Database (MIND) to assess and review 578 mental health apps. The sample of apps used in this analysis were pulled from MIND and include apps across various conditions including schizophrenia, eating disorders, sleep, and more. Analysis of these apps was conducted in June 2022. EXPOSURES There were 578 mental health apps rated across 105 dimensions derived from the American Psychiatric Association's app evaluation framework. MAIN OUTCOMES AND MEASURES App raters assessed each app across 6 categories: (1) app origin and accessibility, (2) privacy and security, (3) clinical foundation, (4) features and engagement, (5) inputs and outputs, and (6) interoperability. Privacy scores were determined by 5 MIND criteria, including (1) having a privacy policy, (2) reporting security measures in place, (3) declaring data use and purpose, (4) allowing for the deletion of data, and (5) allowing users to opt out of data collection. Correlations between privacy scores and popularity metrics (star ratings and number of downloads) were measured. RESULTS This study included 578 mental health apps that were identified, assessed, and analyzed across 105 MIND dimensions. Psychoeducation, goal setting, and mindfulness were among the top app features. Of the 578 apps analyzed, 443 (77%) had a privacy policy. This analysis of apps with a privacy policy revealed that there was no statistically significant correlation between privacy scores and Apple App Store (r = 0.058, P = .29) or Google Play Store star ratings (r = 0.041; P = .48). The number of app downloads on the Google Play Store, however, was weakly correlated with privacy scores (χ25 = 22.1; P < .001). CONCLUSIONS AND RELEVANCE In this cross-sectional study of mental health apps, findings indicate that the current app marketplaces primarily offered basic features such as psychoeducation, goal tracking, and mindfulness but fewer innovative features such as biofeedback or specialized therapies. Privacy challenges remained common, and app popularity metrics provided little help in identifying apps with more privacy.
Collapse
Affiliation(s)
- Erica Camacho
- Department of Psychiatry, Beth Israel Deaconess Medical Center, Boston, Massachusetts
| | - Asher Cohen
- Department of Psychiatry, Beth Israel Deaconess Medical Center, Boston, Massachusetts
| | - John Torous
- Department of Psychiatry, Beth Israel Deaconess Medical Center, Boston, Massachusetts
| |
Collapse
|
11
|
Cantisano LM, Gonzalez-Soltero R, Blanco-Fernández A, Belando-Pedreño N. ePSICONUT: An e-Health Programme to Improve Emotional Health and Lifestyle in University Students. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2022; 19:ijerph19159253. [PMID: 35954601 PMCID: PMC9368425 DOI: 10.3390/ijerph19159253] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/16/2022] [Revised: 07/17/2022] [Accepted: 07/26/2022] [Indexed: 12/10/2022]
Abstract
The use of information and communication technologies in the health field is known as eHealth. Nowadays, the application of technological and digital tools for maintaining/improving physical and mental health is experiencing an exponential boom. These tools have been perceived as a powerful support for face-to-face therapies and lifestyle changes. Nevertheless, there is not enough scientific research that analyses the impact and consequences of eHealth interventions. More studies are needed to validate its application. Therefore, the aim of this study was to evaluate the impact of eHealth tools in a programme called ePSICONUT. This programme was created to promote healthy lifestyle habits in university students. The sample consisted of 16 university students from the Dominican Republic aged 18−24 years (x¯ = 20.69; s = 1.74). ePSICONUT was developed in 12 weeks and its impact was analyzed by comparing the initial and the final psychological and lifestyle tests results, which were completed online by the participants. Results reported that the professionally supervised use of eHealth tools was associated with better psychological well-being, lees anxiety and depression, and better lifestyle habits (such as diet quality), even in stressful and changing situations such as the COVID-19 pandemic circumstances. However, more studies are needed to validate and promote the use of eHealth-based intervention programmes.
Collapse
Affiliation(s)
- Luisa Marilia Cantisano
- School of Psychology, Pontificia Universidad Católica Madre y Maestra (PUCMM), Santiago De Los Caballeros 51000, Dominican Republic;
- Department of Medicine, Faculty of Biomedical and Health Sciences, Universidad Europea de Madrid, 28670 Villaviciosa de Odón, Spain;
| | - Rocio Gonzalez-Soltero
- Department of Medicine, Faculty of Biomedical and Health Sciences, Universidad Europea de Madrid, 28670 Villaviciosa de Odón, Spain;
- Correspondence: (R.G.-S.); (N.B.-P.)
| | - Ascensión Blanco-Fernández
- Department of Medicine, Faculty of Biomedical and Health Sciences, Universidad Europea de Madrid, 28670 Villaviciosa de Odón, Spain;
| | - Noelia Belando-Pedreño
- Faculty of Sports Sciences, Department of Physical Activity and Sport, Universidad Europea de Madrid, 28670 Villaviciosa de Odón, Spain
- Correspondence: (R.G.-S.); (N.B.-P.)
| |
Collapse
|
12
|
Minen MT, George A, Camacho E, Yao L, Sahu A, Campbell M, Soviero M, Hossain Q, Verma D, Torous J. Assessment of Smartphone Apps for Common Neurologic Conditions (Headache, Insomnia, and Pain): Cross-sectional Study. JMIR Mhealth Uhealth 2022; 10:e36761. [PMID: 35727625 PMCID: PMC9257611 DOI: 10.2196/36761] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/24/2022] [Revised: 05/17/2022] [Accepted: 05/25/2022] [Indexed: 11/30/2022] Open
Abstract
BACKGROUND There are thousands of apps for individuals struggling with headache, insomnia, and pain, but it is difficult to establish which of these apps are best suited for patients' specific needs. If clinicians were to have access to a platform that would allow them to make an informed decision on the efficacy and feasibility of smartphone apps for patient care, they would feel confident in prescribing specific apps. OBJECTIVE We sought to evaluate the quality of apps for some of the top common, disabling neurologic conditions (headache, insomnia, and pain) based on principles derived from the American Psychiatric Association's (APA) app evaluation model. METHODS We used the Mobile Health Index and Navigation database and expanded upon the database's current supported conditions by adding 177 new app entries. Each app was rated for consistency with the APA's app evaluation model, which includes 105 objective questions based on the following 5 major classes of consideration: (1) accessibility, (2) privacy and security, (3) clinical foundation, (4) engagement style, and (5) interoperability. These characteristics were evaluated to gain a broader understanding of the significant features of each app category in comparison against a control group. RESULTS Approximately 90% (187/201) of all apps evaluated were free to download, but only 50% (63/201) of headache- and pain-related apps were truly free. Most (87/106, 81%) sleep apps were not truly free to use. The apps had similar limitations with limited privacy, accessibility, and crisis management resources. For example, only 17% (35/201) of the apps were available in Spanish. The apps offered mostly self-help tools with little tailoring; symptom tracking was the most common feature in headache- (32/48, 67%) and pain-related apps (21/47, 45%), whereas mindfulness was the most common feature in sleep-related apps (73/106, 69%). CONCLUSIONS Although there are many apps for headache, pain, and insomnia, all 3 types of apps have room for improvement around accessibility and privacy. Pain and headache apps share many common features, whereas insomnia apps offer mostly mindfulness-based resources. Given the many available apps to pick from, clinicians and patients should seek apps that offer the highest-quality features, such as complete privacy, remedial features, and the ability to download the app at no cost. These results suggest that there are many opportunities for the improvement of apps centered on headache, insomnia, and pain.
Collapse
Affiliation(s)
- Mia T Minen
- Department of Neurology, New York University Langone Health, New York, NY, United States
| | - Alexis George
- Department of Neurology, New York University Langone Health, New York, NY, United States
| | - Erica Camacho
- Department of Psychiatry, Beth Israel Deaconess Medical Center, New York, NY, United States
| | - Leslie Yao
- Barnard College, New York, NY, United States
| | - Ananya Sahu
- Barnard College, New York, NY, United States
| | | | - Mia Soviero
- Barnard College, New York, NY, United States
| | - Quazi Hossain
- The City College of New York, New York, NY, United States
| | - Deepti Verma
- The City College of New York, New York, NY, United States
| | - John Torous
- Department of Psychiatry, Beth Israel Deaconess Medical Center, New York, NY, United States
| |
Collapse
|
13
|
Morton E, Nicholas J, Yang L, Lapadat L, Barnes SJ, Provencher MD, Depp C, Chan M, Kulur R, Michalak EE. Evaluating the quality, safety, and functionality of commonly used smartphone apps for bipolar disorder mood and sleep self-management. Int J Bipolar Disord 2022; 10:10. [PMID: 35368207 PMCID: PMC8977125 DOI: 10.1186/s40345-022-00256-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/17/2021] [Accepted: 02/01/2022] [Indexed: 12/03/2022] Open
Abstract
Background Individuals with bipolar disorder (BD) are increasingly turning to smartphone applications (apps) for health information and self-management support. While reviews have raised concerns regarding the effectiveness and safety of publicly available apps for BD, apps surveyed may not reflect what individuals with BD are using. The present study had two aims: first, to characterize the use of health apps to support mood and sleep amongst people with BD, and second, to evaluate the quality, safety and functionality of the most commonly used self-management apps. Methods A web-based survey was conducted to explore which apps people with BD reported using to support self-management of mood and sleep. The characteristics of the most commonly nominated apps were described using a standardized framework, including their privacy policy, clinical foundations, and functionality. Results Respondents (n = 919) were 77.9% female with a mean age of 36.9 years. 41.6% of participants (n = 382) reported using a self-management app to support mood or sleep. 110 unique apps were nominated in relation to mood, and 104 unique apps nominated in relation to sleep; however, most apps were only mentioned once. The nine most frequently nominated apps related to mood and sleep were subject to further evaluation. All reviewed apps offered a privacy policy, however user control over data was limited and the complexity of privacy policies was high. Only one app was developed for BD populations. Half of reviewed apps had published peer-reviewed evidence to support their claims of efficacy, but little research was specific to BD. Conclusion Findings illustrate the potential of smartphone apps to increase the reach of psychosocial interventions amongst people with BD. Apps were largely created by commercial developers and designed for the general population, highlighting a gap in the development and dissemination of evidence-informed apps for BD. There may be risks in using generic health apps for BD self-management; clinicians should enquire about patients’ app use to foster conversations about their particular benefits and limitations. Supplementary Information The online version contains supplementary material available at 10.1186/s40345-022-00256-6.
Collapse
|
14
|
Leong QY, Sridhar S, Blasiak A, Tadeo X, Yeo G, Remus A, Ho D. Characteristics of Mobile Health Platforms for Depression and Anxiety: Content Analysis Through a Systematic Review of the Literature and Systematic Search of Two App Stores. J Med Internet Res 2022; 24:e27388. [PMID: 35119370 PMCID: PMC8857696 DOI: 10.2196/27388] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2021] [Revised: 06/05/2021] [Accepted: 11/08/2021] [Indexed: 12/20/2022] Open
Abstract
Background Mobile health (mHealth) platforms show promise in the management of mental health conditions such as anxiety and depression. This has resulted in an abundance of mHealth platforms available for research or commercial use. Objective The objective of this review is to characterize the current state of mHealth platforms designed for anxiety or depression that are available for research, commercial use, or both. Methods A systematic review was conducted using a two-pronged approach: searching relevant literature with prespecified search terms to identify platforms in published research and simultaneously searching 2 major app stores—Google Play Store and Apple App Store—to identify commercially available platforms. Key characteristics of the mHealth platforms were synthesized, such as platform name, targeted condition, targeted group, purpose, technology type, intervention type, commercial availability, and regulatory information. Results The literature and app store searches yielded 169 and 179 mHealth platforms, respectively. Most platforms developed for research purposes were designed for depression (116/169, 68.6%), whereas the app store search reported a higher number of platforms developed for anxiety (Android: 58/179, 32.4%; iOS: 27/179, 15.1%). The most common purpose of platforms in both searches was treatment (literature search: 122/169, 72.2%; app store search: 129/179, 72.1%). With regard to the types of intervention, cognitive behavioral therapy and referral to care or counseling emerged as the most popular options offered by the platforms identified in the literature and app store searches, respectively. Most platforms from both searches did not have a specific target age group. In addition, most platforms found in app stores lacked clinical and real-world evidence, and a small number of platforms found in the published research were available commercially. Conclusions A considerable number of mHealth platforms designed for anxiety or depression are available for research, commercial use, or both. The characteristics of these mHealth platforms greatly vary. Future efforts should focus on assessing the quality—utility, safety, and effectiveness—of the existing platforms and providing developers, from both commercial and research sectors, a reporting guideline for their platform description and a regulatory framework to facilitate the development, validation, and deployment of effective mHealth platforms.
Collapse
Affiliation(s)
- Qiao Ying Leong
- N.1 Institute for Health, National University of Singapore, Singapore, Singapore.,The Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore
| | - Shreya Sridhar
- N.1 Institute for Health, National University of Singapore, Singapore, Singapore
| | - Agata Blasiak
- N.1 Institute for Health, National University of Singapore, Singapore, Singapore.,The Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore.,Department of Biomedical Engineering, NUS Engineering, National University of Singapore, Singapore, Singapore.,Department of Pharmacology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore
| | - Xavier Tadeo
- N.1 Institute for Health, National University of Singapore, Singapore, Singapore.,The Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore
| | - GeckHong Yeo
- N.1 Institute for Health, National University of Singapore, Singapore, Singapore.,The Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore
| | - Alexandria Remus
- N.1 Institute for Health, National University of Singapore, Singapore, Singapore.,The Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore.,Department of Biomedical Engineering, NUS Engineering, National University of Singapore, Singapore, Singapore
| | - Dean Ho
- N.1 Institute for Health, National University of Singapore, Singapore, Singapore.,The Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore.,Department of Biomedical Engineering, NUS Engineering, National University of Singapore, Singapore, Singapore.,Department of Pharmacology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore.,Health District @ Queenstown, Singapore, Singapore
| |
Collapse
|
15
|
Lagan S, D'Mello R, Vaidyam A, Bilden R, Torous J. Assessing mental health apps marketplaces with objective metrics from 29,190 data points from 278 apps. Acta Psychiatr Scand 2021; 144:201-210. [PMID: 33835483 DOI: 10.1111/acps.13306] [Citation(s) in RCA: 24] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/30/2021] [Accepted: 04/02/2021] [Indexed: 01/31/2023]
Abstract
OBJECTIVE Utilizing a standard framework that may help clinicians and patients to identify relevant mental health apps, we sought to gain a comprehensive picture of the space by searching for, downloading, and reviewing 278 mental health apps from both the iOS and Android stores. METHODS 278 mental health apps from the Apple iOS store and Google Play store were downloaded and reviewed in a standardized manner by trained app raters using a validated framework. Apps were evaluated with this framework comprising 105 questions and covering app origin and accessibility, privacy and security, inputs and outputs, clinical foundation, features and engagement style, and interoperability. RESULTS Our results confirm that app stars and downloads-even for the most popular apps by these metrics-did not correlate with more clinically relevant metrics related to privacy/security, effectiveness, and engagement. Most mental health apps offer similar functionality, with 16.5% offering both mood tracking and journaling and 7% offering psychoeducation, deep breathing, mindfulness, journaling, and mood tracking. Only 36.4% of apps were updated with a 100-day window, and 7.5% of apps had not been updated in four years. CONCLUSION Current app marketplace metrics commonly used to evaluate apps do not offer an accurate representation of individual apps or a comprehensive overview of the entire space. The majority of apps overlap in terms of features offered, with many domains and other features not well represented. Selecting an appropriate app continues to require personal matching given no clear trends or guidance offered by quantitative metrics alone.
Collapse
Affiliation(s)
- Sarah Lagan
- Department of Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA
| | - Ryan D'Mello
- Department of Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA
| | - Aditya Vaidyam
- Department of Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA
| | - Rebecca Bilden
- Department of Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA
| | - John Torous
- Department of Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA
| |
Collapse
|
16
|
Connolly SL, Kuhn E, Possemato K, Torous J. Digital Clinics and Mobile Technology Implementation for Mental Health Care. Curr Psychiatry Rep 2021; 23:38. [PMID: 33961135 PMCID: PMC8103883 DOI: 10.1007/s11920-021-01254-8] [Citation(s) in RCA: 25] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 04/26/2021] [Indexed: 12/25/2022]
Abstract
PURPOSE OF REVIEW Interest in digital mental health, especially smartphone apps, has expanded in light of limited access to mental health services and the need for remote care during COVID-19. Digital clinics, in which apps are blended into routine care, offer a potential solution to common implementation challenges including low user engagement and lack of clinical integration of apps. RECENT FINDINGS While the number of mental health apps available in commercial marketplaces continues to rise, there are few examples of successful implementation of these apps into care settings. We review one example of a digital clinic created within an academic medical center and another within the Department of Veterans Affairs. We then discuss how implementation science can inform new efforts to effectively integrate mental health technologies across diverse use cases. Integrating mental health apps into care settings is feasible but requires careful attention to multiple domains that will influence implementation success, including characteristics of the innovation (e.g., utility and complexity of the app), the recipients of the technology (e.g., patients and clinicians), and context (e.g., healthcare system buy-in, reimbursement, and regulatory policies). Examples of effective facilitation strategies that can be utilized to improve implementation efforts include co-production of technology involving all end users, specialized trainings for staff and patients, creation of new team members to aid in app usage (e.g., digital navigators), and re-design of clinical workflows.
Collapse
Affiliation(s)
- Samantha L Connolly
- Center for Healthcare Organization and Implementation Research, VA Boston Healthcare System, Boston, MA, USA
- Department of Psychiatry, Harvard Medical School, Boston, MA, USA
| | - Eric Kuhn
- National Center for PTSD, Dissemination and Training Division, VA Palo Alto Health Care System, Palo Alto, CA, USA
- Department of Psychiatry and Behavioral Sciences, Stanford University School of Medicine, Stanford, CA, USA
| | - Kyle Possemato
- VA Center for Integrated Healthcare, Syracuse Veterans Affairs Medical Center, Syracuse, NY, USA
| | - John Torous
- Division of Digital Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, 330 Brookline Ave, Boston, MA, 02446, USA.
| |
Collapse
|