Cocreative Development of the QoL-ME: A Visual and Personalized Quality of Life Assessment App for People With Severe Mental Health Problems

Human-Centered Design Approaches in Digital Mental Health Interventions: Exploratory Mapping Review

BACKGROUND

Digital mental health interventions have a great potential to alleviate mental illness and increase access to care. However, these technologies face significant challenges, especially in terms of user engagement and adoption. It has been suggested that this issue stems from a lack of user perspective in the development process; accordingly, several human-centered design approaches have been developed over the years to consider this important aspect. Yet, few human-centered design approaches to digital solutions exist in the field of mental health, and rarely are end users involved in their development.

OBJECTIVE

The main objective of this literature review is to understand how human-centered design is considered in e-mental health intervention research.

METHODS

An exploratory mapping review was conducted of mental health journals with the explicit scope of covering e-mental health technology. The human-centered design approaches reported and the core elements of design activity (ie, object, context, design process, and actors involved) were examined among the eligible studies.

RESULTS

A total of 30 studies met the inclusion criteria, of which 22 mentioned using human-centered design approaches or specific design methods in the development of an e-mental health solution. Reported approaches were classified as participatory design (11/27, 41%), codesign (6/27, 22%), user-centered design (5/27, 19%), or a specific design method (5/27, 19%). Just over half (15/27, 56%) of the approaches mentioned were supported by references. End users were involved in each study to some extent but not necessarily in designing. About 27% (8/30) of all the included studies explicitly mentioned the presence of designers on their team.

CONCLUSIONS

Our results show that some attempts have indeed been made to integrate human-centered design approaches into digital mental health technology development. However, these attempts rely very little on designers and design research. Researchers from other domains and technology developers would be wise to learn the underpinnings of human-centered design methods before selecting one over another. Inviting designers for assistance when implementing a particular approach would also be beneficial. To further motivate interest in and use of human-centered design principles in the world of e-mental health, we make nine suggestions for better reporting of human-centered design approaches in future research.

Data Visualization for Chronic Neurological and Mental Health Condition Self-management: Systematic Review of User Perspectives

BACKGROUND

Remote measurement technologies (RMT) such as mobile health devices and apps are increasingly used by those living with chronic neurological and mental health conditions. RMT enables real-world data collection and regular feedback, providing users with insights about their own conditions. Data visualizations are an integral part of RMT, although little is known about visualization design preferences from the perspectives of those living with chronic conditions.

OBJECTIVE

The aim of this review was to explore the experiences and preferences of individuals with chronic neurological and mental health conditions on data visualizations derived from RMT to manage health.

METHODS

In this systematic review, we searched peer-reviewed literature and conference proceedings (PubMed, IEEE Xplore, EMBASE, Web of Science, Association for Computing Machinery Computer-Human Interface proceedings, and the Cochrane Library) for original papers published between January 2007 and September 2021 that reported perspectives on data visualization of people living with chronic neurological and mental health conditions. Two reviewers independently screened each abstract and full-text article, with disagreements resolved through discussion. Studies were critically appraised, and extracted data underwent thematic synthesis.

RESULTS

We identified 35 eligible publications from 31 studies representing 12 conditions. Coded data coalesced into 3 themes: desire for data visualization, impact of visualizations on condition management, and visualization design considerations. Data visualizations were viewed as an integral part of users' experiences with RMT, impacting satisfaction and engagement. However, user preferences were diverse and often conflicting both between and within conditions.

CONCLUSIONS

When used effectively, data visualizations are valuable, engaging components of RMT. They can provide structure and insight, allowing individuals to manage their own health more effectively. However, visualizations are not "one-size-fits-all," and it is important to engage with potential users during visualization design to understand when, how, and with whom the visualizations will be used to manage health.

The Extent of User Involvement in the Design of Self-tracking Technology for Bipolar Disorder: Literature Review

BACKGROUND

The number of self-monitoring apps for bipolar disorder (BD) is increasing. The involvement of users in human-computer interaction (HCI) research has a long history and is becoming a core concern for designers working in this space. The application of models of involvement, such as user-centered design, is becoming standardized to optimize the reach, adoption, and sustained use of this type of technology.

OBJECTIVE

This paper aims to examine the current ways in which users are involved in the design and evaluation of self-monitoring apps for BD by investigating 3 specific questions: are users involved in the design and evaluation of technology? If so, how does this happen? And what are the best practice ingredients regarding the design of mental health technology?

METHODS

We reviewed the available literature on self-tracking technology for BD and make an overall assessment of the level of user involvement in design. The findings were reviewed by an expert panel, including an individual with lived experience of BD, to form best practice ingredients for the design of mental health technology. This combines the existing practices of patient and public involvement and HCI to evolve from the generic guidelines of user-centered design and to those that are tailored toward mental health technology.

RESULTS

For the first question, it was found that out of the 11 novel smartphone apps included in this review, 4 (36%) self-monitoring apps were classified as having no mention of user involvement in design, 1 (9%) self-monitoring app was classified as having low user involvement, 4 (36%) self-monitoring apps were classified as having medium user involvement, and 2 (18%) self-monitoring apps were classified as having high user involvement. For the second question, it was found that despite the presence of extant approaches for the involvement of the user in the process of design and evaluation, there is large variability in whether the user is involved, how they are involved, and to what extent there is a reported emphasis on the voice of the user, which is the ultimate aim of such design approaches. For the third question, it is recommended that users are involved in all stages of design with the ultimate goal of empowering and creating empathy for the user.

CONCLUSIONS

Users should be involved early in the design process, and this should not just be limited to the design itself, but also to associated research ensuring end-to-end involvement. Communities in health care-based design and HCI design need to work together to increase awareness of the different methods available and to encourage the use and mixing of the methods as well as establish better mechanisms to reach the target user group. Future research using systematic literature search methods should explore this further.

Considerations of diversity, equity, and inclusion in mental health apps: A scoping review of evaluation frameworks

Mental health applications or apps are a promising approach to reduce the pervasive unmet mental health need observed among marginalized groups. However, these groups face unique challenges to engage in and benefit from these interventions, and apps that consider issues of diversity, equity, and inclusion (DEI) may be better positioned to reach them. App evaluation frameworks emerge as crucial tools for researchers, clinicians, and users to select evidence-based apps. Nevertheless, it is unclear the extent to which existing assessment tools capture DEI factors. The present scoping review identified 68 studies that employed one or more assessment tools to evaluate a mental health app, leading to 44 unique app evaluation frameworks. Results showed that most frameworks were developed after 2015, and only 58% of them considered at least one DEI criterion. Frameworks that performed exceptionally well were those that combined multiple standardized and validated measures. This lack of consideration for DEI variables may limit the ability of app-based interventions to serve marginalized communities, or even worse, create new disparities. Therefore, we provide recommendations to improve current app evaluation frameworks' cultural robustness and clinical utility, maximizing their effectiveness when working with individuals from marginalized communities.

Ensuring patient and public involvement in the transition to AI-assisted mental health care: A systematic scoping review and agenda for design justice

Background

Machine‐learning algorithms and big data analytics, popularly known as ‘artificial intelligence’ (AI), are being developed and taken up globally. Patient and public involvement (PPI) in the transition to AI‐assisted health care is essential for design justice based on diverse patient needs.

Objective

To inform the future development of PPI in AI‐assisted health care by exploring public engagement in the conceptualization, design, development, testing, implementation, use and evaluation of AI technologies for mental health.

Methods

Systematic scoping review drawing on design justice principles, and (i) structured searches of Web of Science (all databases) and Ovid (MEDLINE, PsycINFO, Global Health and Embase); (ii) handsearching (reference and citation tracking); (iii) grey literature; and (iv) inductive thematic analysis, tested at a workshop with health researchers.

Results

The review identified 144 articles that met inclusion criteria. Three main themes reflect the challenges and opportunities associated with PPI in AI‐assisted mental health care: (a) applications of AI technologies in mental health care; (b) ethics of public engagement in AI‐assisted care; and (c) public engagement in the planning, development, implementation, evaluation and diffusion of AI technologies.

Conclusion

The new data‐rich health landscape creates multiple ethical issues and opportunities for the development of PPI in relation to AI technologies. Further research is needed to understand effective modes of public engagement in the context of AI technologies, to examine pressing ethical and safety issues and to develop new methods of PPI at every stage, from concept design to the final review of technology in practice. Principles of design justice can guide this agenda.

Key Variables for Effective eHealth Designs for Individuals With and Without Mental Health Disorders: 2^12-4 Fractional Factorial Experiment

BACKGROUND

eHealth applications not only offer the potential to increase service convenience and responsiveness but also expand the ability to tailor services to improve relevance, engagement, and use. To achieve these goals, it is critical that the designs are intuitive. Limited research exists on designs that work for those with a severe mental illness (SMI), many of whom have difficulty traveling for treatments, reject or infrequently seek treatment, and tend to discontinue treatments for significant periods.

OBJECTIVE

This study aims to evaluate the influence of 12 design variables (eg, navigational depth, reading level, and use of navigational lists) on the usability of eHealth application websites for those with and without SMI.

METHODS

A 2^12-4 fractional factorial experiment was used to specify the designs of 256 eHealth websites. This approach systematically varied the 12 design variables. The final destination contents of all websites were identical, and only the designs of the navigational pages varied. The 12 design elements were manipulated systematically to allow the assessment of combinations of design elements rather than only one element at a time. Of the 256 websites, participants (n=222) sought the same information on 8 randomly selected websites. Mixed effect regressions, which accounted for the dependency of the 8 observations within participants, were used to test for main effects and interactions on the ability and time to find information. Classification and regression tree analyses were used to identify effects among the 12 variables on participants' abilities to locate information, for the sample overall and each of the 3 diagnostic groups of participants (schizophrenia spectrum disorder [SSD], other mental illnesses, and no mental illness).

RESULTS

The best and worst designs were identified for each of these 4 groups. The depth of a website's navigation, that is, the number of screens users needed to navigate to find the desired content, had the greatest influence on usability (ability to find information) and efficiency (time to find information). The worst performing designs for those with SSD had a 9% success rate, and the best had a 51% success rate: the navigational designs made a 42% difference in usability. For the group with other mental illnesses, the design made a 50% difference, and for those with no mental illness, a 55% difference was observed. The designs with the highest usability had several key design similarities, as did those with the poorest usability.

CONCLUSIONS

It is possible to identify evidence-based strategies for designing eHealth applications that result in significantly better performance. These improvements in design benefit all users. For those with SSD or other SMIs, there are designs that are highly effective. Both the best and worst designs have key similarities but vary in some characteristics.

Effect of Age, Breed, and Sex on the Health-Related Quality of Life of Owner Assessed Healthy Dogs

Using an app, this exploratory study generated information on HRQL in a large cohort of dogs deemed healthy according to the owner. It forms the basis for further studies investigating the natural history of HRQL of dogs to inform the interpretation of interventional studies, but highlights the risks of relying on owner impression of health status. A previously published health-related quality of life (HRQL) instrument (VetMetrica™) that generates scores in four domains of quality of life in dogs - Energetic and Enthusiastic (E/E), Happy and Content (H/C), Active and Comfortable (A/C), and Calm and Relaxed (C/R), generated information on HRQL in 4,217 dogs (3 months−21 years). Dogs were categorized by age; young, 3–47 months, middle-aged, 48–95 months, and old, 96 months and older. Owners considered 2,959 dogs (3–95 months) to be “in perfect health” and these were used to explore the relationship between age, sex, breed and HRQL in apparently healthy dogs. Mean score was significantly greater (better) in young compared to middle-aged dogs in E/E, H/C and A/C and declined with advancing age. In H/C there was a small but significant difference in mean score between female and male dogs (mean greater in females), with a similar rate of decline in each gender with advancing age. In E/E there were very small but statistically significant differences in mean scores between certain breeds. In A/C there was a statistically significant interaction between breed and age and the rate of decline with advancing age differed with breed. Overall, age, breed, and sex predicted very little of the variation seen in HRQL scores. Data from a subset of 152 dogs, for whom clinical information was available, were used to examine the agreement between clinical evidence and owner opinion. According to the clinical records, 89 dogs were healthy and 63 had evidence of chronic disease. There was an approximately 40% disagreement between owner opinion on health status and clinical evidence of chronic disease (35% disagreement in all dogs and 43% in old dogs). HRQL scores were generally higher in dogs for whom there was no evidence of disease in the clinical record.

Psychometric Properties of the QoL-ME: A Visual and Personalized Quality of Life Assessment App for People With Severe Mental Health Problems

Background: Quality of Life (QoL) assessment in people with severe mental health problems may benefit from improved personalization and accessibility. Therefore, an innovative, digital, visual, and personalized QoL assessment app for people with severe mental health problems was recently developed: the QoL-ME. The aim of this study was to evaluate the psychometric quality of the QoL-ME by assessing its reliability, validity, and responsiveness. Methods: To examine the reliability of the QoL-ME, the internal consistency of its subscales was assessed using Cronbach's Alpha. Correlations between the QoL-ME and the MANSA were computed to appraise the construct validity of the QoL-ME. Internal responsiveness was evaluated using the standardized response mean and external responsiveness was investigated using hierarchical regression. Results: Cronbach's Alpha's of the subscales of the QoL-ME ranged between 0.5 and 0.84. In accordance with expectations, the language-based core version of the QoL-ME correlated strongly (r = between 0.55 and 0.76) with the MANSA, whilst the picture-based additional modules of the QoL-ME correlated moderately (r = 0.3) with the MANSA. The standardized response mean was 0.23 and the regression model revealed a coefficient β of -0.01. Conclusions: The QoL-ME has adequate psychometric properties. In comparison with similar pictorial instruments, both the QoL-ME's reliability and validity can be considered as sufficient. The results indicate that the responsiveness of the QoL-ME is insufficient. Additional research is needed to evaluate and potentially modify the instrument to improve its responsiveness.

Visual and Personalized Quality of Life Assessment App for People With Severe Mental Health Problems: Qualitative Evaluation

BACKGROUND

QoL-ME is a digital visual personalized quality of life assessment app for people with severe mental health problems. Research reveals that e-mental health apps frequently suffer from low engagement and fall short of expectations regarding their impact on patients' daily lives. Studies often indicate that e-mental health apps ought to respect the needs and preferences of end users to achieve optimal user engagement.

OBJECTIVE

The aim of this study was to explore the experiences of users regarding the usability and functionality of QoL-ME and whether the app is actionable and beneficial for patients.

METHODS

End users (n=8) of QoL-ME contributed to semistructured interviews. An interview guide was used to direct the interviews. All interviews were audiorecorded and transcribed verbatim. Transcriptions were analyzed and coded thematically.

RESULTS

Analysis revealed 3 main themes: (1) benefit, (2) actionability, and (3) characteristics of the QoL-ME. The first theme reveals that the QoL-ME app was beneficial for the majority of respondents, primarily by prompting them to reflect on their quality of life. The current version is not yet actionable; the actionability of the QoL-ME app may be improved by enabling users to view their scores over time and by supplying practical advice for quality of life improvements. Overall, participants had positive experiences with the usability, design, and content of the app.

CONCLUSIONS

The QoL-ME app can be beneficial to users as it provides them with insight into their quality of life and elicits reflection. Incorporating more functionalities that facilitate self-management, such as advice and strategies for improving areas that are lacking, will likely make the app actionable. Patients positively regarded the usability, design, and contents of the QoL-ME app.