1
|
Stallard P, Whittle K, Moore E, Medina-Lara A, Morrish N, Rhodes S, Taylor G, Cliffe B. Acceptability, use and safety of the BlueIce self-harm prevention app: qualitative findings from the Beating Adolescent Self-Harm (BASH) randomised controlled trial. BMJ MENTAL HEALTH 2024; 27:e300961. [PMID: 38925663 PMCID: PMC11202734 DOI: 10.1136/bmjment-2023-300961] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/07/2023] [Accepted: 05/22/2024] [Indexed: 06/28/2024]
Abstract
BACKGROUND Little is known about the social validity of self-harm prevention apps for young adolescents with severe mental health problems who repeatedly self-harm. OBJECTIVE We assessed the acceptability, use and safety of BlueIce, a self-harm prevention app for young adolescents who self-harm. METHODS Mixed methods study involving a content analysis of postuse interviews. Participants were a clinical group of 60 UK adolescents aged 12-17 with repeated self-harm, randomised to receive BlueIce. FINDINGS BlueIce was used by 57/60 (95%) respondents with 47/57 (82%) using BlueIce when thinking about self-harm. 17/47 (36%) who were thinking about self-harm used it on more than six occasions with 36/47 (77%) reporting that BlueIce prevented at least one episode of self-harm. 33/47 (70%) reported occasions when they used the app but still went on to self-harm. Reasons why the app was not used or not helpful included feeling too distressed, a negative mindset, prior decision to self-harm or forgetting. BlueIce was rated 4.09 (SD=0.75) out of 5 stars, with high mean ratings out of 10 for ease of use (8.70, SD=1.37) and good for acceptability (7.68, SD=2.05) and helpfulness (6.77, SD=1.72). No respondent identified BlueIce as triggering any episode of self-harm. CONCLUSION These findings are consistent with previous evaluations and highlight the acceptability, use and safety of BlueIce. Self-reports indicate that BlueIce prevented some episodes of self-harm. CLINICAL IMPLICATIONS Our results highlight the acceptability of the BlueIce self-harm app for young adolescents who repeatedly self-harm.
Collapse
Affiliation(s)
- Paul Stallard
- Department for Health, University of Bath, Bath, UK
- Oxford Health NHS Foundation Trust, Oxford, UK
| | | | - Emma Moore
- Oxford Health NHS Foundation Trust, Oxford, UK
| | | | - Nia Morrish
- Department of Public Health and Sport Sciences, University of Exeter, Exeter, UK
| | | | | | | |
Collapse
|
2
|
Nogueira-Leite D, Marques-Cruz M, Cruz-Correia R. Individuals' attitudes toward digital mental health apps and implications for adoption in Portugal: web-based survey. BMC Med Inform Decis Mak 2024; 24:99. [PMID: 38637866 PMCID: PMC11025147 DOI: 10.1186/s12911-024-02488-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2023] [Accepted: 03/22/2024] [Indexed: 04/20/2024] Open
Abstract
BACKGROUND The literature is consensual regarding the academic community exhibiting higher levels of mental disorder prevalence than the general population. The potential of digital mental health apps for improving access to resources to cope with these issues is ample. However, studies have yet to be performed in Portugal on individuals' attitudes and perceptions toward digital mental health applications or their preferences and decision drivers on obtaining mental health care, self-assessment, or treatment. OBJECTIVE This study aims to understand the determinants of digital mental health applications use in the Portuguese academic community of Porto, along with potential adoption barriers and enablers. METHODS A cross-sectional, web-based survey was delivered via dynamic email to the University of Porto's academic community. Data collection occurred between September 20 and October 20, 2022. We used structural equation modeling to build three models, replicating a peer-reviewed and published study and producing a newly full mediation model shaped by the collected data. We tested the relationships between use of digital mental health apps and perceived stress, perceived need to seek help for mental health, perceived stigma, past use of mental health services, privacy concerns, and social influence. RESULTS Of the 539 participants, 169 (31.4%) reported having used digital mental health apps. Perceived stress and a latent variable, comprising perceptions of mental health problems and coping strategies, were positively associated with mental health app use, while privacy concerns regarding one's information being accessible to others were negatively associated. Perceived stigma, need to seek help, and close relationships did not have a statistically significant direct effect. CONCLUSIONS These findings can inform product and policy development of new, better-targeted digital mental health app interventions, with implications for researchers and academia, industry, and policymakers. Our study concludes that, to maximize adherence to these apps, they should have low to no financial charges, demonstrate evidence of their helpfulness and focus on the timely delivery of care. We also conclude that to foster digital mental health app use, there is a need to improve mental health literacy, namely regarding self-awareness of one's conditions, acceptable stress levels, and overall behavior towards mental health. TRIAL REGISTRATION RR2-10.2196/41040.
Collapse
Affiliation(s)
- Diogo Nogueira-Leite
- Health Data Science Ph.D. Program, Faculty of Medicine of the University of Porto, Porto, Portugal.
- Department of Community Medicine, Health Information and Decision-Making, Faculty of Medicine of the University of Porto, Rua Dr. Plácido da Costa, Porto, 4200-450, Portugal.
- Nova School of Business and Economics Health Economics & Management Knowledge Center, New University of Lisbon, Lisbon, Portugal.
| | - Manuel Marques-Cruz
- Health Data Science Ph.D. Program, Faculty of Medicine of the University of Porto, Porto, Portugal
- Department of Community Medicine, Health Information and Decision-Making, Faculty of Medicine of the University of Porto, Rua Dr. Plácido da Costa, Porto, 4200-450, Portugal
- Nova School of Business and Economics Health Economics & Management Knowledge Center, New University of Lisbon, Lisbon, Portugal
| | - Ricardo Cruz-Correia
- Department of Community Medicine, Health Information and Decision-Making, Faculty of Medicine of the University of Porto, Rua Dr. Plácido da Costa, Porto, 4200-450, Portugal
- Center for Health Technology and Services Research, Porto, Portugal
| |
Collapse
|
3
|
Hyzy M, Bond R, Mulvenna M, Bai L, Frey AL, Carracedo JM, Daly R, Leigh S. Don't judge a book or health app by its cover: User ratings and downloads are not linked to quality. PLoS One 2024; 19:e0298977. [PMID: 38437233 PMCID: PMC10911617 DOI: 10.1371/journal.pone.0298977] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/16/2023] [Accepted: 02/01/2024] [Indexed: 03/06/2024] Open
Abstract
OBJECTIVE To analyse the relationship between health app quality with user ratings and the number of downloads of corresponding health apps. MATERIALS AND METHODS Utilising a dataset of 881 Android-based health apps, assessed via the 300-point objective Organisation for the Review of Care and Health Applications (ORCHA) assessment tool, we explored whether subjective user-level indicators of quality (user ratings and downloads) correlate with objective quality scores in the domains of user experience, data privacy and professional/clinical assurance. For this purpose, we applied spearman correlation and multiple linear regression models. RESULTS For user experience, professional/clinical assurance and data privacy scores, all models had very low adjusted R squared values (< .02). Suggesting that there is no meaningful link between subjective user ratings or the number of health app downloads and objective quality measures. Spearman correlations suggested that prior downloads only had a very weak positive correlation with user experience scores (Spearman = .084, p = .012) and data privacy scores (Spearman = .088, p = .009). There was a very weak negative correlation between downloads and professional/clinical assurance score (Spearman = -.081, p = .016). Additionally, user ratings demonstrated a very weak correlation with no statistically significant correlations observed between user ratings and the scores (all p > 0.05). For ORCHA scores multiple linear regression had adjusted R-squared = -.002. CONCLUSION This study highlights that widely available proxies which users may perceive to signify the quality of health apps, namely user ratings and downloads, are inaccurate predictors for estimating quality. This indicates the need for wider use of quality assurance methodologies which can accurately determine the quality, safety, and compliance of health apps. Findings suggest more should be done to enable users to recognise high-quality health apps, including digital health literacy training and the provision of nationally endorsed "libraries".
Collapse
Affiliation(s)
- Maciej Hyzy
- School of Computing, Ulster University, Belfast, United Kingdom
- ORCHA, Sci-Tech Daresbury, Violet V2, Daresbury, United Kingdom
| | - Raymond Bond
- School of Computing, Ulster University, Belfast, United Kingdom
| | | | - Lu Bai
- School of Electronics, Electrical Engineering and Computer Science, Queen’s University Belfast, Belfast, United Kingdom
| | - Anna-Lena Frey
- ORCHA, Sci-Tech Daresbury, Violet V2, Daresbury, United Kingdom
| | | | - Robert Daly
- ORCHA, Sci-Tech Daresbury, Violet V2, Daresbury, United Kingdom
| | - Simon Leigh
- ORCHA, Sci-Tech Daresbury, Violet V2, Daresbury, United Kingdom
- Warwick Medical School, University of Warwick, Coventry, United Kingdom
| |
Collapse
|
4
|
Vamos CA, Puccio JA, Griner SB, Logan RG, Piepenbrink R, Richardson Cayama M, Lovett SM, Mahony H, Daley EM. Health literacy needs and preferences for a technology-based intervention to improve college students' sexual and reproductive health. JOURNAL OF AMERICAN COLLEGE HEALTH : J OF ACH 2024; 72:477-486. [PMID: 35298353 DOI: 10.1080/07448481.2022.2040517] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/12/2021] [Revised: 01/13/2022] [Accepted: 02/06/2022] [Indexed: 06/14/2023]
Abstract
Objective: To explore health literacy needs and preferences for a technology-based intervention (app) to improve sexual and reproductive health (SRH) among college students. Participants: In Spring 2019, in-depth interviews were conducted with 20 participants (10 male, 10 female) from a large, public university. Methods: Interview guide was developed based on Integrated Model of Health Literacy domains and Diffusion of Innovation constructs. Data were analyzed in MaxQDA using applied thematic analysis. Results: Dominant themes included accessing health information and services, evaluating options to make decisions, intervention utility and characteristics, and the emergent theme of credibility. Specific topics included accessing STI testing, contraceptive decision making, information on human papillomavirus (HPV) and the HPV vaccine, patient-provider communication, app design and function elements, and modifying the app to meet the SRH needs of diverse college students. Conclusions: Findings identified areas where an app could address college students' SRH literacy, ultimately improving SRH outcomes among this population.
Collapse
Affiliation(s)
- Cheryl A Vamos
- University of South Florida, College of Public Health, Tampa, FL, USA
| | - Joseph A Puccio
- University of South Florida, Morsani College of Medicine, Tampa, FL, USA
| | - Stacey B Griner
- University of North Texas Health Science Center at Fort Worth, School of Public Health, Fort Worth, TX, USA
| | | | | | | | - Sharonda M Lovett
- University of South Florida, College of Public Health, Tampa, FL, USA
| | - Helen Mahony
- University of South Florida, College of Public Health, Tampa, FL, USA
| | - Ellen M Daley
- University of South Florida, College of Public Health, Tampa, FL, USA
| |
Collapse
|
5
|
Hyzy M, Bond R, Mulvenna M, Bai L, Dix A, Daly R, Frey AL, Leigh S. Quality of Digital Health Interventions Across Different Health Care Domains: Secondary Data Analysis Study. JMIR Mhealth Uhealth 2023; 11:e47043. [PMID: 37995121 PMCID: PMC10704310 DOI: 10.2196/47043] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/06/2023] [Revised: 07/06/2023] [Accepted: 10/18/2023] [Indexed: 11/24/2023] Open
Abstract
BACKGROUND There are more than 350,000 digital health interventions (DHIs) in the app stores. To ensure that they are effective and safe to use, they should be assessed for compliance with best practice standards. OBJECTIVE The objective of this paper was to examine and compare the compliance of DHIs with best practice standards and adherence to user experience (UX), professional and clinical assurance (PCA), and data privacy (DP). METHODS We collected assessment data from 1574 DHIs using the Organisation for the Review of Care and Health Apps Baseline Review (OBR) assessment tool. As part of the assessment, each DHI received a score out of 100 for each of the abovementioned areas (ie, UX, PCA, and DP). These 3 OBR scores are combined to make up the overall ORCHA score (a proxy for quality). Inferential statistics, probability distributions, Kruskal-Wallis, Wilcoxon rank sum test, Cliff delta, and Dunn tests were used to conduct the data analysis. RESULTS We found that 57.3% (902/1574) of the DHIs had an Organisation for the Review of Care and Health Apps (ORCHA) score below the threshold of 65. The overall median OBR score (ORCHA score) for all DHIs was 61.5 (IQR 51.0-73.0) out of 100. A total of 46.2% (12/26) of DHI's health care domains had a median equal to or above the ORCHA threshold score of 65. For the 3 assessment areas (UX, DP, and PCA), DHIs scored the highest for the UX assessment 75.2 (IQR 70.0-79.6), followed by DP 65.1 (IQR 55.0-73.4) and PCA 49.6 (IQR 31.9-76.1). UX scores had the least variance (SD 13.9), while PCA scores had the most (SD 24.8). Respiratory and urology DHIs were consistently highly ranked in the National Institute for Health and Care Excellence Evidence Standards Framework tiers B and C based on their ORCHA score. CONCLUSIONS There is a high level of variability in the ORCHA scores of DHIs across different health care domains. This suggests that there is an urgent need to improve compliance with best practices in some health care areas. Possible explanations for the observed differences might include varied market maturity and commercial interests within the different health care domains. More investment to support the development of higher-quality DHIs in areas such as ophthalmology, allergy, women's health, sexual health, and dental care may be needed.
Collapse
Affiliation(s)
- Maciej Hyzy
- School of Computing, Ulster University, Belfast, United Kingdom
| | - Raymond Bond
- School of Computing, Ulster University, Belfast, United Kingdom
| | | | - Lu Bai
- School of Electronics, Electrical Engineering and Computer Science, Queen's University Belfast, Belfast, United Kingdom
| | - Alan Dix
- Swansea University, Swansea, United Kingdom
| | - Robert Daly
- Organisation for the Review of Care and Health Apps (ORCHA), Daresbury, United Kingdom
| | - Anna-Lena Frey
- Organisation for the Review of Care and Health Apps (ORCHA), Daresbury, United Kingdom
| | - Simon Leigh
- Organisation for the Review of Care and Health Apps (ORCHA), Daresbury, United Kingdom
| |
Collapse
|
6
|
Frey AL, Baines R, Hunt S, Kent R, Andrews T, Leigh S. Association Between the Characteristics of mHealth Apps and User Input During Development and Testing: Secondary Analysis of App Assessment Data. JMIR Mhealth Uhealth 2023; 11:e46937. [PMID: 37991822 PMCID: PMC10701645 DOI: 10.2196/46937] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2023] [Revised: 06/15/2023] [Accepted: 07/11/2023] [Indexed: 11/23/2023] Open
Abstract
BACKGROUND User involvement is increasingly acknowledged as a central part of health care innovation. However, meaningful user involvement during the development and testing of mobile health apps is often not fully realized. OBJECTIVE This study aims to examine in which areas user input is most prevalent and whether there is an association between user inclusion and compliance with best practices for mobile health apps. METHODS A secondary analysis was conducted on an assessment data set of 1595 health apps. The data set contained information on whether the apps had been developed or tested with user input and whether they followed best practices across several domains. Background information was also available regarding the apps' country of origin, targeted condition areas, subjective user ratings, download numbers, and risk (as per the National Institute for Health and Care Excellence Evidence Standards Framework [ESF]). Descriptive statistics, Mann-Whitney U tests, and Pearson chi-square analyses were applied to the data. RESULTS User involvement was reported by 8.71% (139/1595) of apps for only the development phase, by 33.67% (537/1595) of apps for only the testing phase, by 21.88% (349/1595) of apps for both phases, and by 35.74% (570/1595) of apps for neither phase. The highest percentage of health apps with reported user input during development was observed in Denmark (19/24, 79%); in the condition areas of diabetes (38/79, 48%), cardiology (15/32, 47%), pain management (20/43, 47%), and oncology (25/54, 46%); and for high app risk (ESF tier 3a; 105/263, 39.9%). The highest percentage of health apps with reported user input during testing was observed in Belgium (10/11, 91%), Sweden (29/34, 85%), and France (13/16, 81%); in the condition areas of neurodiversity (42/52, 81%), respiratory health (58/76, 76%), cardiology (23/32, 72%), and diabetes (56/79, 71%); and for high app risk (ESF tier 3a; 176/263, 66.9%). Notably, apps that reported seeking user input during testing demonstrated significantly more downloads than those that did not (P=.008), and user inclusion was associated with better compliance with best practices in clinical assurance, data privacy, risk management, and user experience. CONCLUSIONS The countries and condition areas in which the highest percentage of health apps with user involvement were observed tended to be those with higher digital maturity in health care and more funding availability, respectively. This suggests that there may be a trade-off between developers' willingness or ability to involve users and the need to meet challenges arising from infrastructure limitations and financial constraints. Moreover, the finding of a positive association between user inclusion and compliance with best practices indicates that, where no other guidance is available, users may benefit from prioritizing health apps developed with user input as the latter may be a proxy for broader app quality.
Collapse
Affiliation(s)
- Anna-Lena Frey
- Organisation for the Review of Care and Health Apps, Daresbury, United Kingdom
| | - Rebecca Baines
- Organisation for the Review of Care and Health Apps, Daresbury, United Kingdom
- Centre for Health Technology, University of Plymouth, Plymouth, United Kingdom
| | - Sophie Hunt
- Organisation for the Review of Care and Health Apps, Daresbury, United Kingdom
| | - Rachael Kent
- Department of Digital Humanities, King's College London, London, United Kingdom
| | - Tim Andrews
- Organisation for the Review of Care and Health Apps, Daresbury, United Kingdom
- Warwick Medical School, University of Warwick, Coventry, United Kingdom
| | - Simon Leigh
- Organisation for the Review of Care and Health Apps, Daresbury, United Kingdom
- Warwick Medical School, University of Warwick, Coventry, United Kingdom
| |
Collapse
|
7
|
Melcher J, Lavoie J, Hays R, D'Mello R, Rauseo-Ricupero N, Camacho E, Rodriguez-Villa E, Wisniewski H, Lagan S, Vaidyam A, Torous J. Digital phenotyping of student mental health during COVID-19: an observational study of 100 college students. JOURNAL OF AMERICAN COLLEGE HEALTH : J OF ACH 2023; 71:736-748. [PMID: 33769927 DOI: 10.1080/07448481.2021.1905650] [Citation(s) in RCA: 12] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/05/2023]
Abstract
Objective: This study assessed the feasibility of capturing smartphone based digital phenotyping data in college students during the COVID-19 pandemic with the goal of understanding how digital biomarkers of behavior correlate with mental health. Participants: Participants were 100 students enrolled in 4-year universities. Methods: Each participant attended a virtual visit to complete a series of gold-standard mental health assessments, and then used a mobile app for 28 days to complete mood assessments and allow for passive collection of GPS, accelerometer, phone call, and screen time data. Students completed another virtual visit at the end of the study to collect a second round of mental health assessments. Results: In-app daily mood assessments were strongly correlated with their corresponding gold standard clinical assessment. Sleep variance among students was correlated to depression scores (ρ = .28) and stress scores (ρ = .27). Conclusions: Digital Phenotyping among college students is feasible on both an individual and a sample level. Studies with larger sample sizes are necessary to understand population trends, but there are practical applications of the data today.
Collapse
Affiliation(s)
- Jennifer Melcher
- Division of Digital Psychiatry, Department of Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, Massachusetts, USA
| | - Joel Lavoie
- Division of Digital Psychiatry, Department of Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, Massachusetts, USA
| | - Ryan Hays
- Division of Digital Psychiatry, Department of Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, Massachusetts, USA
| | - Ryan D'Mello
- Division of Digital Psychiatry, Department of Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, Massachusetts, USA
| | - Natali Rauseo-Ricupero
- Division of Digital Psychiatry, Department of Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, Massachusetts, USA
| | - Erica Camacho
- Division of Digital Psychiatry, Department of Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, Massachusetts, USA
| | - Elena Rodriguez-Villa
- Division of Digital Psychiatry, Department of Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, Massachusetts, USA
| | - Hannah Wisniewski
- Division of Digital Psychiatry, Department of Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, Massachusetts, USA
| | - Sarah Lagan
- Division of Digital Psychiatry, Department of Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, Massachusetts, USA
| | - Aditya Vaidyam
- Division of Digital Psychiatry, Department of Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, Massachusetts, USA
| | - John Torous
- Division of Digital Psychiatry, Department of Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, Massachusetts, USA
| |
Collapse
|
8
|
Currey D, Hays R, Torous J. Digital Phenotyping Models of Symptom Improvement in College Mental Health: Generalizability Across Two Cohorts. JOURNAL OF TECHNOLOGY IN BEHAVIORAL SCIENCE 2023:1-14. [PMID: 37362062 PMCID: PMC9978275 DOI: 10.1007/s41347-023-00310-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 04/13/2022] [Revised: 01/17/2023] [Accepted: 02/18/2023] [Indexed: 06/28/2023]
Abstract
Smartphones can be used to gain insight into mental health conditions through the collection of survey and sensor data. However, the external validity of this digital phenotyping data is still being explored, and there is a need to assess if predictive models derived from this data are generalizable. The first dataset (V1) of 632 college students was collected between December 2020 and May 2021. The second dataset (V2) was collected using the same app between November and December 2021 and included 66 students. Students in V1 could enroll in V2. The main difference between the V1 and V2 studies was that we focused on protocol methods in V2 to ensure digital phenotyping data had a lower degree of missing data than in the V1 dataset. We compared survey response counts and sensor data coverage across the two datasets. Additionally, we explored whether models trained to predict symptom survey improvement could generalize across datasets. Design changes in V2, such as a run-in period and data quality checks, resulted in significantly higher engagement and sensor data coverage. The best-performing model was able to predict a 50% change in mood with 28 days of data, and models were able to generalize across datasets. The similarities between the features in V1 and V2 suggest that our features are valid across time. In addition, models must be able to generalize to new populations to be used in practice, so our experiments provide an encouraging result toward the potential of personalized digital mental health care.
Collapse
Affiliation(s)
- Danielle Currey
- Beth Israel Deaconess Medical Center, Harvard Medical School, 330 Brookline Ave, MA 02215 Boston, USA
- Case Western Reserve University School of Medicine, Cleveland, OH USA
| | - Ryan Hays
- Beth Israel Deaconess Medical Center, Harvard Medical School, 330 Brookline Ave, MA 02215 Boston, USA
| | - John Torous
- Beth Israel Deaconess Medical Center, Harvard Medical School, 330 Brookline Ave, MA 02215 Boston, USA
| |
Collapse
|
9
|
Balaskas A, Schueller SM, Cox AL, Rashleigh C, Doherty G. Examining young adults daily perspectives on usage of anxiety apps: A user study. PLOS DIGITAL HEALTH 2023; 2:e0000185. [PMID: 36812622 PMCID: PMC9931254 DOI: 10.1371/journal.pdig.0000185] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/17/2022] [Accepted: 12/14/2022] [Indexed: 01/27/2023]
Abstract
The growing number of mental health smartphone applications has led to increased interest in how these tools might support users in different models of care. However, research on the use of these interventions in real-world settings has been scarce. It is important to understand how apps are used in a deployment setting, especially among populations where such tools might add value to current models of care. The objective of this study is to explore the daily use of commercially-available mobile apps for anxiety that integrate CBT, with a focus on understanding reasons for and barriers for app use and engagement. This study recruited 17 young adults (age M = 24.17 years) while on a waiting list to receive therapy in a Student Counselling Service. Participants were asked to select up to two of a list of three selected apps (Wysa, Woebot, and Sanvello) and instructed to use the apps for two weeks. Apps were selected because they used techniques from cognitive behavioral therapy, and offer diverse functionality for anxiety management. Qualitative and quantitative data were gathered through daily questionnaires to capture participants' experiences with the mobile apps. In addition, eleven semi-structured interviews were conducted at the end of the study. We used descriptive statistics to analyze participants' interaction with different app features and used a general inductive approach to analyze the collected qualitative data. The results highlight that users form opinions about the apps during the first days of app use. A number of barriers to sustained use are identified including cost-related issues, inadequate content to support long-term use, and a lack of customization options for different app functions. The app features used differ among participants with self-monitoring and treatment elements being the most used features.
Collapse
Affiliation(s)
- Andreas Balaskas
- School of Computer Science and Statistics, Trinity College Dublin, Dublin, Ireland
- * E-mail:
| | - Stephen M. Schueller
- Department of Psychological Science, University of California, Irvine, Irvine, California, United States of America
| | - Anna L. Cox
- UCLIC, University College London, London, United Kingdom
| | - Chuck Rashleigh
- Student Counselling Services, Trinity College Dublin, Dublin, Ireland
| | - Gavin Doherty
- School of Computer Science and Statistics, Trinity College Dublin, Dublin, Ireland
| |
Collapse
|
10
|
Alon N, Torous J. Current challenges for evaluating mobile health applications. J Am Med Inform Assoc 2022; 30:617-624. [PMID: 36484621 PMCID: PMC9933055 DOI: 10.1093/jamia/ocac244] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2022] [Revised: 09/22/2022] [Accepted: 12/05/2022] [Indexed: 12/14/2022] Open
Abstract
As mobile health applications continue to proliferate without clear regulation, the need for app evaluation frameworks to offer guidance to patients and clinicians also expands. However, this expanding number of app evaluation frameworks itself can be a source of confusion and often contradictory recommendations. In pursuit of better frameworks that offer innovation for app evaluation, we present 4 challenges that app evaluation frameworks must overcome as well as examples from our own experience toward overcoming them. The recommendations are applicable to all health apps from any field of medicine, although we use examples from mental health as they are illustrative.
Collapse
Affiliation(s)
- Noy Alon
- Division of Digital Psychiatry, Beth Israel Deaconess Medical Center, Boston, Massachusetts, USA
| | - John Torous
- Corresponding Author: John Torous, MD, MBI, Division of Digital Psychiatry, Beth Israel Deaconess Medical Center, 330 Brookline Ave, Boston, MA 02215, USA;
| |
Collapse
|
11
|
Currey D, Torous J. Digital Phenotyping Data to Predict Symptom Improvement and App Personalization: Protocol for a Prospective Study. JMIR Res Protoc 2022; 11:e37954. [DOI: 10.2196/37954] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2022] [Revised: 07/18/2022] [Accepted: 10/27/2022] [Indexed: 11/30/2022] Open
Abstract
Background
Smartphone apps that capture surveys and sensors are increasingly being leveraged to collect data on clinical conditions. In mental health, this data could be used to personalize psychiatric support offered by apps so that they are more effective and engaging. Yet today, few mental health apps offer this type of support, often because of challenges associated with accurately predicting users’ actual future mental health.
Objective
In this protocol, we present a study design to explore engagement with mental health apps in college students, using the Technology Acceptance Model as a theoretical framework, and assess the accuracy of predicting mental health changes using digital phenotyping data.
Methods
There are two main goals of this study. First, we present a logistic regression model fit on data from a prior study on college students and prospectively test this model on a new student cohort to assess its accuracy. Second, we will provide users with data-driven activity suggestions every 4 days to determine whether this type of personalization will increase engagement or attitudes toward the app compared to those receiving no personalized recommendations.
Results
The study was completed in the spring of 2022, and the manuscript is currently in review at JMIR Publications.
Conclusions
This is one of the first digital phenotyping algorithms to be prospectively validated. Overall, our results will inform the potential of digital phenotyping data to serve as tailoring data in adaptive interventions and to increase rates of engagement.
International Registered Report Identifier (IRRID)
PRR1-10.2196/37954
Collapse
|
12
|
Singh MK, Malmon A, Horne L, Felten O. Addressing burgeoning unmet needs in college mental health. JOURNAL OF AMERICAN COLLEGE HEALTH : J OF ACH 2022:1-4. [PMID: 36170437 DOI: 10.1080/07448481.2022.2115302] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/12/2021] [Revised: 06/30/2022] [Accepted: 08/15/2022] [Indexed: 06/16/2023]
Abstract
America is experiencing burgeoning mental health needs of their college students. Measuring the impact of mental health challenges for these students and the natural ways they adapt to them might enable smart triage of limited mental health resources. This may, in part, be achieved through a combination of technology-assisted personalized measurement-based care, treatment matching, and peer-support. Helping students self-monitor and organize their personal peer networks can destigmatize and increase accessibility to timely mental health care, especially for students of marginalized identities, who might otherwise be hesitant to receive care or be misdiagnosed. A collaborative effort among students, educators, clinicians, and health technology innovators may provide more tractable solutions for student unmet needs than any single entity or resource alone. Novel resources, tailored through a healthy equity lens that is individualized and culturally-sensitive, may meaningfully meet a student's needs, preferences, and acceptability, and translate to daily use and informed decision-making.
Collapse
|
13
|
Assessing engagement features in an observational study of mental health apps in college students. Psychiatry Res 2022; 310:114470. [PMID: 35227991 DOI: 10.1016/j.psychres.2022.114470] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/19/2021] [Revised: 02/12/2022] [Accepted: 02/19/2022] [Indexed: 11/23/2022]
|
14
|
de Chantal PL, Chagnon A, Cardinal M, Faieta J, Guertin A. Evidence of User-Expert Gaps in Health App Ratings and Implications for Practice. Front Digit Health 2022; 4:765993. [PMID: 35252957 PMCID: PMC8891373 DOI: 10.3389/fdgth.2022.765993] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2021] [Accepted: 01/25/2022] [Indexed: 11/17/2022] Open
Abstract
Searching the commercial Google Play Store and App Store is one of the most common strategies for discovering mobile applications for digital health, both among consumers and healthcare professionals. However, several studies have suggested a possible mismatch between this strategy and the objective of finding apps in physical and mental health that are both clinically relevant and reliable from a privacy standpoint. This study provides direct evidence of a gap between the five-star user rating system and expert ratings from a curated library of over 1,200 apps that cover both physical and mental health. An objective metric is derived to assess the strength of the user-expert gap for each app, which in turn allows identifying missed opportunities—low user ratings and high expert ratings—and overrated apps—high user ratings and low expert ratings. Implications for practice and care delivery are discussed.
Collapse
Affiliation(s)
- Pier-Luc de Chantal
- Department of Psychology, Université du Québec à Montréal, Montréal, QC, Canada
- Therappx, Granby, QC, Canada
- *Correspondence: Pier-Luc de Chantal
| | - Alexandre Chagnon
- Therappx, Granby, QC, Canada
- Department of Pharmacy, Université Laval, Québec City, QC, Canada
| | | | - Julie Faieta
- Therappx, Granby, QC, Canada
- Department of Rehabilitation Science and Technology, University of Pittsburgh, Pittsburg, PA, United States
| | | |
Collapse
|
15
|
Spadaro B, Martin-Key NA, Funnell E, Bahn S. mHealth Solutions for Perinatal Mental Health: Scoping Review and Appraisal Following the mHealth Index and Navigation Database Framework. JMIR Mhealth Uhealth 2022; 10:e30724. [PMID: 35037894 PMCID: PMC8804959 DOI: 10.2196/30724] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2021] [Revised: 09/06/2021] [Accepted: 11/19/2021] [Indexed: 11/18/2022] Open
Abstract
Background The ever-increasing pressure on health care systems has resulted in the underrecognition of perinatal mental disorders. Digital mental health tools such as apps could provide an option for accessible perinatal mental health screening and assessment. However, there is a lack of information regarding the availability and features of perinatal app options. Objective This study aims to evaluate the current state of diagnostic and screening apps for perinatal mental health available on the Google Play Store (Android) and Apple App Store (iOS) and to review their features following the mHealth Index and Navigation Database framework. Methods Following a scoping review approach, the Apple App Store and Google Play Store were systematically searched to identify perinatal mental health assessment apps. A total of 14 apps that met the inclusion criteria were downloaded and reviewed in a standardized manner using the mHealth Index and Navigation Database framework. The framework comprised 107 questions, allowing for a comprehensive assessment of app origin, functionality, engagement features, security, and clinical use. Results Most apps were developed by for-profit companies (n=10), followed by private individuals (n=2) and trusted health care companies (n=2). Out of the 14 apps, 3 were available only on Android devices, 4 were available only on iOS devices, and 7 were available on both platforms. Approximately one-third of the apps (n=5) had been updated within the last 180 days. A total of 12 apps offered the Edinburgh Postnatal Depression Scale in its original version or in rephrased versions. Engagement, input, and output features included reminder notifications, connections to therapists, and free writing features. A total of 6 apps offered psychoeducational information and references. Privacy policies were available for 11 of the 14 apps, with a median Flesch-Kincaid reading grade level of 12.3. One app claimed to be compliant with the Health Insurance Portability and Accountability Act standards and 2 apps claimed to be compliant with General Data Protection Regulation. Of the apps that could be accessed in full (n=10), all appeared to fulfill the claims stated in their description. Only 1 app referenced a relevant peer-reviewed study. All the apps provided a warning for use, highlighting that the mental health assessment result should not be interpreted as a diagnosis or as a substitute for medical care. Only 3 apps allowed users to export or email their mental health test results. Conclusions These results indicate that there are opportunities to improve perinatal mental health assessment apps. To this end, we recommend focusing on the development and validation of more comprehensive assessment tools, ensuring data protection and safety features are adequate for the intended app use, and improving data sharing features between users and health care professionals for timely support.
Collapse
Affiliation(s)
- Benedetta Spadaro
- Cambridge Centre for Neuropsychiatric Research, Department of Chemical Engineering and Biotechnology, University of Cambridge, Cambridge, United Kingdom
| | - Nayra A Martin-Key
- Cambridge Centre for Neuropsychiatric Research, Department of Chemical Engineering and Biotechnology, University of Cambridge, Cambridge, United Kingdom
| | - Erin Funnell
- Cambridge Centre for Neuropsychiatric Research, Department of Chemical Engineering and Biotechnology, University of Cambridge, Cambridge, United Kingdom
| | - Sabine Bahn
- Cambridge Centre for Neuropsychiatric Research, Department of Chemical Engineering and Biotechnology, University of Cambridge, Cambridge, United Kingdom
- Psyomics Ltd, Cambridge, United Kingdom
| |
Collapse
|
16
|
Dederichs M, Nitsch FJ, Apolinário-Hagen J. Piloting an Innovative Concept of e-Mental Health and mHealth Workshops With Medical Students Using a Participatory Co-design Approach and App Prototyping: Case Study. JMIR MEDICAL EDUCATION 2022; 8:e32017. [PMID: 35006085 PMCID: PMC8787659 DOI: 10.2196/32017] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/13/2021] [Revised: 10/22/2021] [Accepted: 11/06/2021] [Indexed: 05/16/2023]
Abstract
BACKGROUND Medical students show low levels of e-mental health literacy. Moreover, there is a high prevalence of common mental illnesses among medical students. Mobile health (mHealth) apps can be used to maintain and promote medical students' well-being. To date, the potential of mHealth apps for promoting mental health among medical students is largely untapped because they seem to lack familiarity with mHealth. In addition, little is known about medical students' preferences regarding mHealth apps for mental health promotion. There is a need for guidance on how to promote competence-based learning on mHealth apps in medical education. OBJECTIVE The aim of this case study is to pilot an innovative concept for an educative workshop following a participatory co-design approach and to explore medical students' preferences and ideas for mHealth apps through the design of a hypothetical prototype. METHODS We conducted a face-to-face co-design workshop within an elective subject with 26 participants enrolled at a medical school in Germany on 5 consecutive days in early March 2020. The aim of the workshop was to apply the knowledge acquired from the lessons on e-mental health and mHealth app development. Activities during the workshop included group work, plenary discussions, storyboarding, developing personas (prototypical users), and designing prototypes of mHealth apps. The workshop was documented in written and digitalized form with the students' permission. RESULTS The participants' feedback suggests that the co-design workshop was well-received. The medical students presented a variety of ideas for the design of mHealth apps. Among the common themes that all groups highlighted in their prototypes were personalization, data security, and the importance of scientific evaluation. CONCLUSIONS Overall, this case study indicates the feasibility and acceptance of a participatory design workshop for medical students. The students made suggestions for improvements at future workshops (eg, use of free prototype software, shift to e-learning, and more time for group work). Our results can be (and have already been) used as a starting point for future co-design workshops to promote competence-based collaborative learning on digital health topics in medical education.
Collapse
Affiliation(s)
- Melina Dederichs
- Institute of Occupational, Social and Environmental Medicine, Centre for Health and Society, Faculty of Medicine, Heinrich Heine University Düsseldorf, Düsseldorf, Germany
| | - Felix Jan Nitsch
- Comparative Psychology, Institute for Experimental Psychology, Heinrich-Heine-University Düsseldorf, Düsseldorf, Germany
| | - Jennifer Apolinário-Hagen
- Institute of Occupational, Social and Environmental Medicine, Centre for Health and Society, Faculty of Medicine, Heinrich Heine University Düsseldorf, Düsseldorf, Germany
| |
Collapse
|
17
|
Broad A, Biswakarma R, Harper JC. A survey of women’s experiences of using period tracker applications: Attitudes, ovulation prediction and how the accuracy of the app in predicting period start dates affects their feelings and behaviours. WOMEN'S HEALTH 2022; 18:17455057221095246. [PMID: 35465788 PMCID: PMC9047811 DOI: 10.1177/17455057221095246] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
Introduction: Using an online survey, the aim of this study was to ask women about their
real-life experiences of using period tracker apps, their attitudes towards
using their app, the information the app provided regarding ovulation and
how the accuracy of the app in predicting period start dates affects their
feelings and behaviours if their period comes earlier or later than
predicted. Methods: This mixed-methods observational study was conducted by an online survey of
50 multiple-choice and open-ended questions. The survey was generated with
Qualtrics XM® and promoted via social media. It was open to any
person who had used a period tracker. Results: From 375 total responses, 330 complete responses were obtained, giving a
completion rate of 88.0%. Respondents were aged between 14 and 54, with a
mean age of 26.0 (±7.81). When asked what was the best thing about using the
app, 29.7% (98/330) of respondents selected ‘To know when I’m ovulating’.
Respondents were asked if their period ever started earlier than the app
predicted; 54.9% (189/330) said it had and 72.1% (238/330) said it had
started later than predicted. When asked how they felt if their period
arrived earlier or later than expected, thematic analysis of periods
starting earlier revealed four themes: feeling unaffected, being
frustrated/unprepared, feeling anxious/stressed and feeling
confused/intrigued. Thematic analysis when their period arrived later
revealed six themes: anxious/concerned about pregnancy, disappointed about
pregnancy, seeking advice/informing healthcare professionals, thoughts about
menopause, feeling unaffected and being better prepared. Conclusion: Period trackers need to be clearer on their intended use and reliability,
especially for period due date and ovulation. Qualitative analysis shows the
impact of inaccurate predictions on aspects of the users’ health. This study
calls for period tracker app companies to update their apps to provide
transparency to their users about their intended use and capabilities.
Collapse
Affiliation(s)
- Anna Broad
- EGA Institute for Women’s Health, University College London, London, UK
| | - Rina Biswakarma
- Institute for Education, University College London, London, UK
| | - Joyce C Harper
- EGA Institute for Women’s Health, University College London, London, UK
| |
Collapse
|
18
|
Abstract
Mobile health applications (apps) have transformed the possibilities for health promotion and disease self-management; however, their promise is not fully realized owing to their reliance on commercial ecosystems for development and distribution. This review provides an overview of the types of mobile health apps and describes key stakeholders in terms of how apps are used, developed, and regulated. I outline key challenges facing consumers, public health professionals, and policy makers in evaluating the quality of health apps and summarize what is known about the impact of apps on health outcomes and health equity. I suggest that factors within the wider mobile ecosystem largely define the impact of health apps and, most notably, practices around the collection and commercialization of user data. Finally, I suggest that upstream public health strategies, grounded in an understanding of corporate influences on health, are necessary to promote healthy digital environments in which mobile health app innovation can flourish. Expected final online publication date for the Annual Review of Public Health, Volume 43 is April 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.
Collapse
Affiliation(s)
- Quinn Grundy
- Lawrence S. Bloomberg Faculty of Nursing, University of Toronto, Toronto, Ontario, Canada;
| |
Collapse
|
19
|
Lagan S, D'Mello R, Vaidyam A, Bilden R, Torous J. Assessing mental health apps marketplaces with objective metrics from 29,190 data points from 278 apps. Acta Psychiatr Scand 2021; 144:201-210. [PMID: 33835483 DOI: 10.1111/acps.13306] [Citation(s) in RCA: 24] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/30/2021] [Accepted: 04/02/2021] [Indexed: 01/31/2023]
Abstract
OBJECTIVE Utilizing a standard framework that may help clinicians and patients to identify relevant mental health apps, we sought to gain a comprehensive picture of the space by searching for, downloading, and reviewing 278 mental health apps from both the iOS and Android stores. METHODS 278 mental health apps from the Apple iOS store and Google Play store were downloaded and reviewed in a standardized manner by trained app raters using a validated framework. Apps were evaluated with this framework comprising 105 questions and covering app origin and accessibility, privacy and security, inputs and outputs, clinical foundation, features and engagement style, and interoperability. RESULTS Our results confirm that app stars and downloads-even for the most popular apps by these metrics-did not correlate with more clinically relevant metrics related to privacy/security, effectiveness, and engagement. Most mental health apps offer similar functionality, with 16.5% offering both mood tracking and journaling and 7% offering psychoeducation, deep breathing, mindfulness, journaling, and mood tracking. Only 36.4% of apps were updated with a 100-day window, and 7.5% of apps had not been updated in four years. CONCLUSION Current app marketplace metrics commonly used to evaluate apps do not offer an accurate representation of individual apps or a comprehensive overview of the entire space. The majority of apps overlap in terms of features offered, with many domains and other features not well represented. Selecting an appropriate app continues to require personal matching given no clear trends or guidance offered by quantitative metrics alone.
Collapse
Affiliation(s)
- Sarah Lagan
- Department of Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA
| | - Ryan D'Mello
- Department of Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA
| | - Aditya Vaidyam
- Department of Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA
| | - Rebecca Bilden
- Department of Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA
| | - John Torous
- Department of Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA
| |
Collapse
|
20
|
O'Dwyer N, Rickwood D, Buckmaster D, Watsford C. Therapeutic interventions in Australian primary care, youth mental health settings for young people with borderline personality disorder or borderline traits. Borderline Personal Disord Emot Dysregul 2020; 7:23. [PMID: 33042549 PMCID: PMC7542340 DOI: 10.1186/s40479-020-00138-2] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/07/2020] [Accepted: 09/08/2020] [Indexed: 02/06/2023] Open
Abstract
BACKGROUND This study aimed to investigate what therapeutic interventions were being applied by clinicians working with young people with a diagnosis of Borderline Personality Disorder or borderline traits in Australian primary mental health care settings. Given the current lack of evidence-based guidelines for treatment with this client population, investigating what is being implemented is needed. The study also aimed to determine whether the interventions clinicians are using are effective in reducing distress and increasing functioning for these clients. METHODS Participant data came from the national minimum data set for headspace youth mental health centers across Australia. Young people's data were included in the study if the young person was diagnosed with Borderline Personality Disorder or borderline traits during their first episode of care (N = 701). Clinician data that indicated the type of intervention used at each client session and outcome measures routinely captured were analyzed to determine interventions used and outcomes achieved. RESULTS Results demonstrated that CBT was the most frequently used modality of intervention followed by supportive counselling and IPT, but that most clients received a variety of intervention types. There were no or only weak relationships between changes in outcomes and the amount of any type of intervention that was provided. No significant relationship was found with the amount of CBT a client received and changes in symptoms or functioning, despite being the most commonly employed modality. CONCLUSIONS The study highlights the need for evidence-based treatment guidelines for early intervention in young people with borderline personality disorder traits.
Collapse
Affiliation(s)
- Nikki O'Dwyer
- Faculty of Health, University of Canberra, Canberra, ACT 2601 Australia
| | - Debra Rickwood
- Faculty of Health, University of Canberra, Canberra, ACT 2601 Australia
| | - Dean Buckmaster
- Faculty of Health, University of Canberra, Canberra, ACT 2601 Australia
| | - Clare Watsford
- Faculty of Health, University of Canberra, Canberra, ACT 2601 Australia
| |
Collapse
|