Usability Evaluations of Mobile Mental Health Technologies: Systematic Review

Design of Digital Mental Health Platforms for Family Member Cocompletion: Scoping Review

BACKGROUND

The COVID-19 pandemic placed an additional mental health burden on individuals and families, resulting in widespread service access problems. Digital mental health interventions suggest promise for improved accessibility. Recent reviews have shown emerging evidence for individual use and early evidence for multiusers. However, attrition rates remain high for digital mental health interventions, and additional complexities exist when engaging multiple family members together.

OBJECTIVE

As such, this scoping review aims to detail the reported evidence for digital mental health interventions designed for family use with a focus on the build and design characteristics that promote accessibility and engagement and enable cocompletion by families.

METHODS

A systematic literature search of MEDLINE, Embase, PsycINFO, Web of Science, and CINAHL databases was conducted for articles published in the English language from January 2002 to March 2024. Eligible records included empirical studies of digital platforms containing some elements designed for cocompletion by related people as well as some components intended to be completed without therapist engagement. Platforms were included in cases in which clinical evidence had been documented.

RESULTS

Of the 9527 papers reviewed, 85 (0.89%) met the eligibility criteria. A total of 24 unique platforms designed for co-use by related parties were identified. Relationships between participants included couples, parent-child dyads, family caregiver-care recipient dyads, and families. Common platform features included the delivery of content via structured interventions with no to minimal tailoring or personalization offered. Some interventions provided live contact with therapists. User engagement indicators and findings varied and included user experience, satisfaction, completion rates, and feasibility. Our findings are more remarkable for what was absent in the literature than what was present. Contrary to expectations, few studies reported any design and build characteristics that enabled coparticipation. No studies reported on platform features for enabling cocompletion or considerations for ensuring individual privacy and safety. None examined platform build or design characteristics as moderators of intervention effect, and none offered a formative evaluation of the platform itself.

CONCLUSIONS

In this early era of digital mental health platform design, this novel review demonstrates a striking absence of information about design elements associated with the successful engagement of multiple related users in any aspect of a therapeutic process. There remains a large gap in the literature detailing and evaluating platform design, highlighting a significant opportunity for future cross-disciplinary research. This review details the incentive for undertaking such research; suggests design considerations when building digital mental health platforms for use by families; and offers recommendations for future development, including platform co-design and formative evaluation.

Outcomes of Best-Practice Guided Digital Mental Health Interventions for Youth and Young Adults with Emerging Symptoms: Part II. A Systematic Review of User Experience Outcomes

Although many young people demonstrate resilience and strength, research and clinical evidence highlight an upward trend in mental health concerns among those aged 12 to 25 years. Youth-specific digital mental health interventions (DMHIs) aim to address this trend by providing timely access to mental health support for young people (12-25 years). However, there is a considerable gap in understanding young people user experiences with digital interventions. This review, co-designed with Australia's leading mental health organization Beyond Blue, utilizes a systematic methodology to synthesize evidence on user experience in youth-oriented digital mental health interventions that are fully or partially guided. Five relevant online databases were searched for articles published from 2018 to 2023, yielding 22,482 articles for screening and 22 studies were included in the present analysis. User experience outcomes relating to satisfaction and engagement were assessed for each included intervention, with experience indicators relating to usefulness, usability, value, credibility, and desirability being examined. Elements associated with positive/negative outcomes were extracted. Elements shown to positively influence user experience included peer engagement, modern app-based delivery, asynchronous support, and personalized content. In contrast, users disliked static content, homework/log-keeping, the requirement for multiple devices, and social media integration. Asynchronous interventions showed high satisfaction but faced engagement issues, with combined asynchronous/synchronous interventions reporting better completion rates. DMHIs offer a promising platform for youth mental health support and has the potential to dramatically increase the reach of interventions through the adoption of technological and user experience best practices. While young people respond positively to many aspects of intervention modernization, such as interactive, app-based design, other concepts, such as social media integration, they need to be adopted by the field more cautiously to ensure trust and engagement.Trial Registration CRD42023405812.

Developing A Mobile App With a Human-Centered Design Lens to Improve Access to Mental Health Care (Mentallys Project): Protocol for an Initial Co-Design Process

BACKGROUND

Co-design is one of the human-centered design approaches that allows end users to significantly and positively impact the design of mental health technologies. It is a promising approach to foster user acceptance and engagement in digital mental health solutions. Surprisingly, there is a lack of understanding of what co-design is in this field. In this paper, co-design is approached as a cocreation process involving persons with a lived experience of mental health problems, health professionals, and design experts who lead and facilitate the overall creative process.

OBJECTIVE

This paper describes an initial co-design research protocol for the development of a mobile app that aims to improve access to mental health care. It highlights the characteristics of a co-design approach in e-mental health rooted in human-centered design and led by design experts alongside health experts. The paper focuses on the first steps (phase 1) of the co-design process of the ongoing Mentallys project.

METHODS

This Mentallys project will be located in Montréal (Quebec, Canada). The method approach will be based on the "method stories," depicting the "making of" this project and reflecting adjustments needed to the protocol throughout the project in specific situations. Phase 1 of the process will focus on the desirability of the app. Targeted participants will include people with a lived experience of mental health problems, peer support workers and clinicians, and 3 facilitators (all design experts or researchers). Web-based sessions will be organized because of the COVID-19 pandemic, using Miro (RealtimeBoard Inc) and Zoom (Zoom Video Communications, Inc). Data collection will be based on the comments, thoughts, and new ideas of participants around the imaginary prototypes. Thematic analysis will be carried out after each session to inform a new version of the prototype.

RESULTS

We conducted 2 stages in phase 1 of the process. During stage 1, we explored ideas through group co-design workshops (divergent thinking). Six co-design workshops were held: 2 with only clinicians (n=7), 2 with peer support workers (n=5) and people with a lived experience of mental health problems (n=2), and 2 with all of them (n=14). A total of 6 facilitators participated in conducting activities in subgroups. During stage 2, ideas were refined through 10 dyad co-design sessions (convergent thinking). Stage 2 involved 3 participants (n=3) and 1 facilitator. Thematic analysis was performed after stage 1, while analytic questioning is being performed for stage 2. Both stages allowed several iterations of the prototypes.

CONCLUSIONS

The design of the co-design process, the leadership of the design expertise throughout the process, and the different forms of co-design activities are key elements in this project. We highly recommend that health researchers partner with professional designers or design researchers who are familiar with co-design.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/47220.

Human-to-Computer Interactivity Features Incorporated Into Behavioral Health mHealth Apps: Systematic Search

BACKGROUND

While there are thousands of behavioral health apps available to consumers, users often quickly discontinue their use, which limits their therapeutic value. By varying the types and number of ways that users can interact with behavioral health mobile health apps, developers may be able to support greater therapeutic engagement and increase app stickiness.

OBJECTIVE

The main objective of this analysis was to systematically characterize the types of user interactions that are available in behavioral health apps and then examine if greater interactivity was associated with greater user satisfaction, as measured by app metrics.

METHODS

Using a modified PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analysis) methodology, we searched several different app clearinghouse websites and identified 76 behavioral health apps that included some type of interactivity. We then filtered the results to ensure we were examining behavioral health apps and further refined our search to include apps that identified one or more of the following terms: peer or therapist forum, discussion, feedback, professional, licensed, buddy, friend, artificial intelligence, chatbot, counselor, therapist, provider, mentor, bot, coach, message, comment, chat room, community, games, care team, connect, share, and support in the app descriptions. In the final group of 34 apps, we examined the presence of 6 types of human-machine interactivities: human-to-human with peers, human-to-human with providers, human-to-artificial intelligence, human-to-algorithms, human-to-data, and novel interactive smartphone modalities. We also downloaded information on app user ratings and visibility, as well as reviewed other key app features.

RESULTS

We found that on average, the 34 apps reviewed included 2.53 (SD 1.05; range 1-5) features of interactivity. The most common types of interactivities were human-to-data (n=34, 100%), followed by human-to-algorithm (n=15, 44.2%). The least common type of interactivity was human-artificial intelligence (n=7, 20.5%). There were no significant associations between the total number of app interactivity features and user ratings or app visibility. We found that a full range of therapeutic interactivity features were not used in behavioral health apps.

CONCLUSIONS

Ideally, app developers would do well to include more interactivity features in behavioral health apps in order to fully use the capabilities of smartphone technologies and increase app stickiness. Theoretically, increased user engagement would occur by using multiple types of user interactivity, thereby maximizing the benefits that a person would receive when using a mobile health app.

A systematic review and meta-analysis of digital interventions targeted at individuals with borderline personality disorder (BPD), emotionally unstable personality disorder (EUPD), and related symptoms

OBJECTIVES

The present review investigates the impact of digital interventions for individuals with features of borderline personality disorder (BPD)/emotional unstable personality disorder (EUPD) as digital interventions show promise as therapeutic tools in underserved groups. BPD/EUPD features are identified as clinically relevant, yet previous reviews on the use of digital interventions fail to include subthreshold symptomatology.

METHODS

Five online databases were searched for terminology in three categories: BPD/EUPD and related symptoms, mental-health interventions, and digital technology. Additionally, four relevant journals and two trial registers were searched for additional papers meeting the inclusion criteria.

RESULTS

Twelve articles met all inclusion criteria. Meta-analyses revealed statistically significant differences in symptom measures between intervention and control groups at postintervention and decreases in BPD/EUPD symptomatology and well-being from pre- to postintervention. Service users' engagement, satisfaction, and acceptability of interventions were high. Results support the previous literature on the value of using digital interventions in populations with BPD/EUPD.

CONCLUSION

Overall, it was identified that digital interventions show promise for successful implementation with this population.

System Integrated Digital Empowering and teleRehabilitation to promote patient Activation and well-Being in chronic disabilities: A usability and acceptability study

Introduction

Telerehabilitation systems represent a promising way for the management of chronic disability, delivering technology-enabled rehabilitation outside the hospital setting. However, usability and acceptability assessment with users represents a critical starting point when using digital healthcare solutions. This study aims at evaluating the user experience with a Telerehabilitation system (SIDERA^∧B) from the end-user side.

Methods

SIDERA^∧B consists of an asynchronous delivery of rehabilitation activities through multimedia digital contents and tele-monitoring of vital parameters with technological devices for individualized, home-based management of chronic conditions. Usability (with the System Usability Scale, SUS) and acceptability (using the Technology Acceptance Model, TAM - and The Service User Technology Acceptance Questionnaire, SUTAQ) data were analyzed from the dataset of the SIDERA^∧B project (N = 112 patients with Chronic Heart Failure, Parkinson's Disease and Chronic Obstructive Pulmonary Disease). The possible influence of five external factors (i.e., technological expertise, education, sex, age, and level of disability) on TAM domains was tested using Spearman's Correlation analysis.

Results

Results showed a satisfactory level of technological usability (SUS Median = 77.5) and good scores in usability and learnability SUS subdomains (mean scores > 2.5). Regarding technological acceptability, participants showed high scores (Median > 4) in "Behavioral Intention", "Perceived Usefulness", and "Perceived Ease of Use" TAM domains. Finally, results from the SUTAQ scale highlighted that the SIDERA^∧B system obtained optimal scores in all domains, especially in "Increased accessibility," "Care personnel concerns," and "Satisfaction." Age (rho = -0.291, p = 0.002) and disability level (WHODAS Total score: rho = -0.218, p = 0.021) were the two external factors inversely associated with the Perceived Ease of Use.

Discussion

The age of digital transformation requires everyone to understand, accept and master the changes affecting modern-day healthcare. The usability and acceptability of the SIDERA^∧B system were high across all end-users, despite the medium-low level of the technological expertise of the sample. These findings support the efficiency and the suitability of these digital solutions in the modern digital age transition of rehabilitation from inside to outside the clinic.

Usability of a mobile application for health professionals in home care services: a user-centered approach

The use of mobile devices by healthcare professionals has led to rapid growth in the development of mobile healthcare applications designed to improve healthcare services. This study was conducted to assess the acceptability and usability of a mobile application for health professionals in relation to their work in hospitalization at home. A mixed methods approach was used. Acceptance, included the satisfaction of the professionals, attitudes toward using the application, and intention or willingness to continue using the application. Usability tests were performed in laboratory analyzing five controlled clinical tasks, and the interaction of the participants with the mobile application was based on the six basic facial expressions published by the American Psychological Association. Perceived satisfaction was assessed using the computer system usability questionnaire. Thirty-two participants completed the task scenarios and questionnaire. More than 90 per cent of participants were able to complete the tasks with only some difficult with vital signs. Satisfaction had a score of 6.18/7 (SD: 0.76), and recommendation of the mobile application had a score of 6.21/7 (SD: 0.81). This study showed a significant usability and acceptability of this mobile application, in terms of effectiveness, efficiency, and satisfaction.

Moderated digital social therapy for young people with emerging mental health problems: A user-centered mixed-method design and usability study

Introduction

Over 25% of Dutch young people are psychologically unhealthy. Individual and societal consequences that follow from having mental health complaints at this age are substantial. Young people need care which is often unavailable. ENgage YOung people earlY (ENYOY) is a moderated digital social therapy-platform that aims to help youngsters with emerging mental health complaints. Comprehensive research is being conducted into the effects and to optimize and implement the ENYOY-platform throughout the Netherlands. The aim of this study is to explore the usability and user experience of the ENYOY-platform.

Methods

A user-centered mixed-method design was chosen. 26 young people aged 16-25 with emerging mental health complaints participated. Semi-structured interviews were conducted to explore usability, user-friendliness, impact, accessibility, inclusivity, and connection (Phase 1). Phase 2 assessed usability problems using the concurrent and retrospective Think Aloud-method. User experience and perceived helpfulness were assessed using a 10-point rating scale and semi-structured interviews (Phase 3). The Health Information Technology Usability Evaluation Scale (Health-ITUES; Phase 1) and System Usability Scale (SUS; Phase 2 and 3) were administered. Qualitative data was analyzed using thematic analysis. Task completion rate and time were tracked and usability problems were categorized using the Nielsen's rating scale (Phase 2).

Results

Adequate to high usability was found (Phase 1 Health-ITUES 4.0(0.34); Phase 2 SUS 69,5(13,70); Phase 3 SUS 71,6(5,63)). Findings from Phase 1 (N = 10) indicated that users viewed ENYOY as a user-friendly, safe, accessible, and inclusive initiative which helped them reduce their mental health complaints and improve quality of life. Phase 2 (N = 10) uncovered 18 usability problems of which 5 of major severity (e.g. troubles accessing the platform). Findings from Phase 3 (N = 6) suggested that users perceived the coaching calls the most helpful [9(0.71)] followed by the therapy content [6.25(1.41)]. Users liked the social networking aspect but rated it least helpful [6(2.1)] due to inactivity.

Conclusion

The ENYOY-platform has been found to have adequate to high usability and positive user experiences were reported. All findings will be transferred to the developmental team to improve the platform. Other evaluation methods and paring these with quantitative outcomes could provide additional insight in future research.

Usability and acceptability of internet-based interventions for family carers of people living with dementia: systematic review

Objectives: Internet-based interventions can help carers of people living with dementia to cope with care-related challenges and can help improve their wellbeing. This systematic review aimed at investigating the criteria of usability and acceptability of self-guided internet-based interventions for family carers of people living with dementia. Method: Searches were conducted on PubMed, Virtual Health Library Regional Portal (Americas), MEDLINE, PsycINFO, Scopus, and Cochrane. Studies published up to December 2019, in English, Portuguese, or Spanish, were eligible. We followed the definition/criteria from ISO ISO-9241-11 for usability (efficiency, effectiveness, and satisfaction) and acceptability (barriers for using and utility). Methodological quality was evaluated using specific tools according to each study design. Results: Ten studies were included, all of which had high methodological quality. Carers of people living with dementia indicated that internet-based interventions were mostly effective, efficient, and satisfactory. They considered these to be informative, relevant, and functional, highlighting the utility and intention of using the resource in the future. The high heterogeneity in the terms and methods used to evaluate usability and acceptability hindered cross-study comparisons, however internet-based interventions were considered useful and acceptable by most carers. Conclusion: Future research should consider expanding the criteria of usability and acceptability to better reflect the needs of this population.

Environmental conditions, mobile digital culture, mobile usability, knowledge of app in COVID-19 risk mitigation: A structural equation model analysis

INTRODUCTION

The mobile digital culture (MDC) supports individual lives, communities, and real-time organizational surveillance during COVID-19 emergencies. Hence, the study examined the advancement in smart health devices evidence in smartphone apps technologies in surveillance, control, and tracking potential virus areas among high-risk populations.

OBJECTIVE

The study explored how environmental condition and MDC mediates between knowledge of App and mobile usability in the prevention of COVID-19 infection in high-risk areas.

METHODS

Using the concept of UTAUT, the study conceptualized that mobile usability, MDC, knowledge of App and environmental condition, are essential for COVID-19 mitigation. A cross-sectional method was adopted through an online survey to assess data from n = 459 mobile users. The association of the study models was appraised through structural equation models (Amos v.24.0).

RESULT

We found mobile usability, knowledge of App, and MDC were statistically significant to COVID-19 mitigation. Environment condition as mediator had no effect in the study models. However, moderating effect of MDC shows a negative influence on the association between COVID-19 mitigation and knowledge of apps.

CONCLUSION

Future policies should consider the development of mHealth technology to improve end-user experience. Also, future policies should entail data privacy to reduce the infringement of data collected. This approach will lead to a confidential, high acceptance of usability of mHealth apps infectious disease prevention.

Perceived software usability and usability-related stress in German craft enterprises

BACKGROUND

Digitalisation is reaching German craft enterprises which must increasingly deal with complex software. The usability of the software is crucial for the effective use in everyday work and insufficient usability is associated with increased stress and strain. It thus seems necessary to identify possible usability-related stressors in craft-enterprise software.

OBJECTIVE

Here we examine how users evaluate the usability of craft-enterprise software and whether usability is associated with perceived work-related stress.

METHODS

To assess the perceived usability of German craft-enterprise software, an online survey was conducted with 161 participants from craft enterprises using a validated usability questionnaire based on ISO 9241-110. In addition, 26 guideline-based interviews were conducted to identify the experience of managers and employees with craft-enterprise software, possible usability-related stress and strain.

RESULTS

Both studies show that craft enterprises use software but only to a limited extend. While back-office administrative processes are routinely supported by software, mobile software support to manage customer service is used less frequently. Increasing complexity of craft-enterprise software is posing growing demands on the users. Software usability was rated to be crucial in the online survey, but the available software is only rated to be acceptable. This was also reflected in the interviews. Participants described usability problems and their relevance as stressors during software use. In consequence, the users experience strain such as feeling overwhelmed and frustrated.

CONCLUSION

It can be concluded that improving the usability of craft-enterprise software should reduce work-related stress and support craft enterprises to master the digitalisation process.

Introduction of Technology to Support Young People’s Care and Mental Health—A Rapid Evidence Review

Background

Technology and its use within mental health services has advanced dramatically over recent years. Opportunities for mental health services to utilise technology to introduce novel, effective, and more efficient means of delivering assessment, and treatment are increasing.

Objective

The current rapid-evidence paper reviews evidence regarding the introduction of novel technology to support young people’s mental health and psychological well-being.

Methods

A rapid evidence review was conducted. PSYCHINFO and CINAHL were searched for research articles between 2016 and 2021 that were specific to young people, mental health, and technology developments within this domain. N = 27 studies which explored the introduction, feasibility, and value of technology for mental health purposes were included in a narrative synthesis. Quality or risk of bias analyses were not completed.

Results

Overall, technological advancements in young people’s care were considered positive and engaging for young people. Factors including resources, efficiency of care, engagement, therapeutic effectiveness, ethical considerations, therapeutic alliance, and flexibility were considered within this review. Nevertheless, potential barriers include clinician concerns, socioeconomic factors, and motivation.

Conclusion

Effective and sustained use of technology within young people’s mental health services will depend on the technology’s usability, efficiency, and ability to engage young people. This paper expands on existing research by reviewing a broader range of technology proposed to support young people’s mental health and well-being. This will assist in the application of novel technological advancements by indicating effectiveness, preferences, potential barriers, and recommendations for the feasibility and efficacy of introducing technology into young people’s services.

A Biofeedback-Based Mobile App With Serious Games for Young Adults With Anxiety in the United Arab Emirates: Development and Usability Study

Background

Following the outbreak of COVID-19, several studies have reported that young adults encountered a rise in anxiety symptoms, which could negatively affect their quality of life. Promising evidence suggests that mobile apps with biofeedback, serious games, breathing exercises, and positive messaging, among other features, are useful for anxiety self-management and treatment.

Objective

This study aimed to develop and evaluate the usability of a biofeedback-based app with serious games for young adults with anxiety in the United Arab Emirates (UAE).

Methods

This study consists of two phases: Phase I describes the design and development of the app, while Phase II presents the results of a usability evaluation by experts. To elicit the app’s requirements during Phase I, we conducted (1) a survey to investigate preferences of young adults in the UAE for mobile games for stress relief; (2) an analysis of serious games for anxiety; and (3) interviews with mental health professionals and young adults in the UAE. In Phase II, five experts tested the usability of the developed app using a set of Nielsen’s usability heuristics.

Results

A fully functional biofeedback-based app with serious games was co-designed with mental health professionals. The app included 4 games (ie, a biofeedback game, card game, arcade game, and memory game), 2 relaxation techniques (ie, a breathing exercise and yoga videos), and 2 additional features (ie, positive messaging and a mood tracking calendar). The results of Phase II showed that the developed app is efficient, simple, and easy to use. Overall, the app design scored an average of 4 out of 5.

Conclusions

The elicitation techniques used in Phase I resulted in the development of an easy-to-use app for the self-management of anxiety. Further research is required to determine the app’s usability and effectiveness in the target population.

A Web-Based Stress Management Intervention for University Students in Indonesia (Rileks): Feasibility Study Using a Pretest-Posttest Design

Background

University students are susceptible to excessive stress. A web-based stress management intervention holds promise to improve stress but is still at a novel stage in Indonesia.

Objective

The aim of this paper was to report the feasibility of the intervention we developed—Rileks—among university students in Indonesia in terms of acceptability and usability, and to propose recommendations for future improvements.

Methods

A single-group pretest and posttest design was used. Participants with scores of 15 or higher on the stress subscale of the 42-item Depression Anxiety Stress Scales were given access to the intervention (N=68). The main outcome measures were the 8-item Client Satisfaction Questionnaire (CSQ-8) score, the System Usability Scale (SUS) score, and intervention uptake. Participants’ experience in each session was evaluated using closed- and open-ended questions for future improvements. Descriptive statistics were used to examine primary outcome and qualitative session evaluations. Participants’ responses to each topic of the open questions were summarized.

Results

The intervention was evaluated as being satisfactory (CSQ-8 mean score 21.89, SD 8.72; range 8-32). However, the intervention’s usability was still below expectation (SUS mean score 62.8, SD 14.74; range 0-100). The core modules were completed by 10 out of 68 participants (15%), and the study dropout rate was 63% (43/68) at postassessment. In general, the module content was rated positively, with some notes for improvement covering content and technical aspects.

Conclusions

This study indicates that Rileks is potentially feasible for Indonesian university students. In order to be optimally applied in such a context and before scaling up web-based interventions in Indonesia, in general, further development and refinement are needed.

International Registered Report Identifier (IRRID)

RR2-10.2196/11493

A Reference Architecture for Data-Driven and Adaptive Internet-Delivered Psychological Treatment Systems: Software Architecture Development and Validation Study

BACKGROUND

Internet-delivered psychological treatment (IDPT) systems are software applications that offer psychological treatments via the internet. Such IDPT systems have become one of the most commonly practiced and widely researched forms of psychotherapy. Evidence shows that psychological treatments delivered by IDPT systems can be an effective way of treating mental health morbidities. However, current IDPT systems have high dropout rates and low user adherence. The primary reason is that the current IDPT systems are not flexible, adaptable, and personalized as they follow a fixed tunnel-based treatment architecture. A fixed tunnel-based architecture follows predefined, sequential treatment content for every patient, irrespective of their context, preferences, and needs. Moreover, current IDPT systems have poor interoperability, making it difficult to reuse and share treatment materials. There is a lack of development and documentation standards, conceptual frameworks, and established (clinical) guidelines for such IDPT systems. As a result, several ad hoc forms of IDPT models exist. Consequently, developers and researchers have tended to reinvent new versions of IDPT systems, making them more complex and less interoperable.

OBJECTIVE

This study aimed to design, develop, and evaluate a reference architecture (RA) for adaptive systems that can facilitate the design and development of adaptive, interoperable, and reusable IDPT systems.

METHODS

This study was conducted in collaboration with a large interdisciplinary project entitled INTROMAT (Introducing Mental Health through Adaptive Technology), which brings together information and communications technology researchers, information and communications technology industries, health researchers, patients, clinicians, and patients' next of kin to reach its vision. First, we investigated previous studies and state-of-the-art works based on the project's problem domain and goals. On the basis of the findings from these investigations, we identified 2 primary gaps in current IDPT systems: lack of adaptiveness and limited interoperability. Second, we used model-driven engineering and Domain-Driven Design techniques to design, develop, and validate the RA for building adaptive, interoperable, and reusable IDPT systems to address these gaps. Third, based on the proposed RA, we implemented a prototype as the open-source software. Finally, we evaluated the RA and open-source implementation using empirical (case study) and nonempirical approaches (software architecture analysis method, expert evaluation, and software quality attributes).

RESULTS

This paper outlines an RA that supports flexible user modeling and the adaptive delivery of treatments. To evaluate the proposed RA, we developed an open-source software based on the proposed RA. The open-source framework aims to improve development productivity, facilitate interoperability, increase reusability, and expedite communication with domain experts.

CONCLUSIONS

Our results showed that the proposed RA is flexible and capable of adapting interventions based on patients' needs, preferences, and context. Furthermore, developers and researchers can extend the proposed RA to various health care interventions.

Usability Testing of a Patient-Centered Mobile Health App for Supporting and Guiding the Pediatric Emergency Department Patient Journey: Mixed Methods Study

BACKGROUND

Patient experience in emergency departments (EDs) remains often suboptimal and can be a source of stress, particularly in pediatric settings. In an attempt to support patients and their families before, during, and after their visit to a pediatric ED, a mobile health (mHealth) app was developed by a multidisciplinary team based on patient-centered care principles.

OBJECTIVE

This study aims to evaluate the usability (effectiveness, efficiency, and satisfaction) of a new mHealth app, InfoKids, by potential end users through usability testing.

METHODS

The app was assessed through an in-laboratory, video-recorded evaluation in which participants had to execute 9 goal-oriented tasks, ranging from account creation to the reception of a diagnostic sheet at the end of the emergency care episode. Effectiveness was measured based on the task completion rate, efficiency on time on task, and user satisfaction according to answers to the System Usability Scale questionnaire. Think-aloud usability sessions were also transcribed and analyzed. Usability problems were rated for their severity and categorized according to ergonomic criteria.

RESULTS

A total of 17 parents participated in the study. The overall completion rate was 97.4% (149/153). Overall, they reported good effectiveness, with the task successfully completed in 88.2% (135/153) of cases (95% CI 83%-93%). Each task, with the exception of the first, created difficulties for some participants but did not prevent their completion by most participants. Users reported an overall good to excellent perceived usability of the app. However, ergonomic evaluation identified 14 usability problems occurring 81 time. Among these, 50% (7/14) were serious as their severity was rated as either major or catastrophic and indicated areas of improvements for the app. Following the suggested usability improvements by participants, mitigation measures were listed to further improve the app and avoid barriers to its adoption.

CONCLUSIONS

Usability of the InfoKids app was evaluated as good to excellent by users. Areas of improvement were identified, and mitigation measures were proposed to inform its development toward a universal app for all ED patients visiting a digitalized institution. Its contribution could also be useful in paving the way for further research on mobile apps aimed at supporting and accompanying patients in their care episodes, as research in this area is scarce.

Participatory Design of a Mobile App to Safeguard Mental Resilience in the Context of Drug Use in Young Adults: Multi-Method Study

Background

Existing mental health apps are largely not aimed at generally healthy young people who may be experimenting with addictive substances and mind-altering experiences.

Objective

The aim of this study is to examine the interest and expectations of young people regarding a proposed smartphone app designed to help protect and promote mental health and resilience in the face of risks associated with substance use.

Methods

The study was based on agile system development and had 3 empirical substudies. Our feasibility study (study 1) included an anonymous questionnaire that examined the potential interest of young people in this type of app. It was answered by 339 Israelis aged 18-30 years. The second part of the feasibility study was a pilot study with 1.2% (4/339) of the people who answered the questionnaire and expressed interest in participating in a focus group. They tested and refined the elements planned for the focus groups. Study 2 was a participatory design study involving 7 focus groups of 5 to 7 participants each (young people aged 18-35 years, n=38). Persona development, open discussion, and a Technology Acceptance Model questionnaire were used to elicit user expectations and requirements for the app and to understand the perceived usefulness and usability of the proposed features. Study 3 comprised in-depth interviews with experts in the field of youth mental health and drug use to enlist their professional opinion regarding the value of such an app and recommendations about the features it should include.

Results

The mock-up for the proposed app had five key features: personalized assessment of risk for a drug-associated mental crisis, support for self-monitoring, useful information (eg, warning signs and first-aid guidelines), resilience-building exercises, and a support center. Participants rated highly the usefulness of all 5 main features and 96% (24/25) of the specific features we proposed within those main categories. The participants also suggested additional features as well as a new user persona we had not considered: the parents or family members of the young person. The focus groups rated highly the perceived usability of the app. Most of the experts saw value in all the main features and suggested specific knowledge sources for the app’s content. Finally, participants of both the feasibility study and the participatory design study expressed moderate to high interest in using the app for self-help and high interest in using the app to help friends.

Conclusions

The findings provide preliminary encouraging support for the 5 main features suggested by the research team and reinforce recommendations for mobile health apps found in the literature. The findings emphasize the insight that this kind of app should be designed primarily for use by individuals seeking to help others.

Human-Computer Interaction in Digital Mental Health

Human-computer interaction (HCI) has contributed to the design and development of some efficient, user-friendly, cost-effective, and adaptable digital mental health solutions. But HCI has not been well-combined into technological developments resulting in quality and safety concerns. Digital platforms and artificial intelligence (AI) have a good potential to improve prediction, identification, coordination, and treatment by mental health care and suicide prevention services. AI is driving web-based and smartphone apps; mostly it is used for self-help and guided cognitive behavioral therapy (CBT) for anxiety and depression. Interactive AI may help real-time screening and treatment in outdated, strained or lacking mental healthcare systems. The barriers for using AI in mental healthcare include accessibility, efficacy, reliability, usability, safety, security, ethics, suitable education and training, and socio-cultural adaptability. Apps, real-time machine learning algorithms, immersive technologies, and digital phenotyping are notable prospects. Generally, there is a need for faster and better human factors in combination with machine interaction and automation, higher levels of effectiveness evaluation and the application of blended, hybrid or stepped care in an adjunct approach. HCI modeling may assist in the design and development of usable applications, and to effectively recognize, acknowledge, and address the inequities of mental health care and suicide prevention and assist in the digital therapeutic alliance.

COVID-19 smart surveillance: Examination of Knowledge of Apps and mobile thermometer detectors (MTDs) in a high-risk society

Background

Technological innovations gained momentum and supported COVID-19 intelligence surveillance among high-risk populations globally. We examined technology surveillance using mobile thermometer detectors (MTDs), knowledge of App, and self-efficacy as a means of sensing body temperature as a measure of COVID-19 risk mitigation. In a cross-sectional survey, we explored COVID-19 risk mitigation, mobile temperature detectable by network syndromic surveillance mobility, detachable from clinicians, and laboratory diagnoses to elucidate the magnitude of community monitoring.

Materials and Methods

In a cross-sectional survey, we create in-depth comprehension of risk mitigation, mobile temperature Thermometer detector, and other variables for surveillance and monitoring among 850 university students and healthcare workers. An applied structural equation model was adopted for analysis with Amos v.24. We established that mobile usability knowledge of APP could effectively aid in COVID-19 intelligence risk mitigation. Moreover, both self-efficacy and mobile temperature positively strengthened data visualization for public health decision-making.

Results

The algorithms utilize a validated point-of-center test to ascertain the HealthCode scanning system for a positive or negative COVID-19 notification. The MTD is an alternative personal self-testing procedure used to verify temperature rates based on previous SARS-CoV-2 and future mobility digital health. Personal self-care of MTD mobility and knowledge of mHealth apps can specifically manage COVID-19 mitigation in high or low terrestrial areas. We found mobile usability, mobile self-efficacy, and app knowledge were statistically significant to COVID-19 mitigation. Additionally, interaction strengthened the positive relationship between self-efficacy and COVID-19. Data aggregation is entrusted with government database agencies, using natural language processing and machine learning mechanisms to validate and analyze.

Conclusion

The study shows that temperature thermometer detectors, mobile usability, and knowledge of App enhanced COVID-19 risk mitigation in a high or low-risk environment. The standardizing dataset is necessary to ensure privacy and security preservation of data ethics.

Towards Understanding the Usability Attributes of AI-Enabled eHealth Mobile Applications

Mobile application (app) use is increasingly becoming an essential part of our daily lives. Due to their significant usefulness, people rely on them to perform multiple tasks seamlessly in almost all aspects of everyday life. Similarly, there has been immense progress in artificial intelligence (AI) technology, especially deep learning, computer vision, natural language processing, and robotics. These technologies are now actively being implemented in smartphone apps and healthcare, providing multiple healthcare services. However, several factors affect the usefulness of mobile healthcare apps, and usability is an important one. There are various healthcare apps developed for each specific task, and the success of these apps depends on their performance. This study presents a systematic review of the existing apps and discusses their usability attributes. It highlights the usability models, outlines, and guidelines proposed in previous research for designing apps with improved usability characteristics. Thirty-nine research articles were reviewed and examined to identify the usability attributes, framework, and app design conducted. The results showed that satisfaction, efficiency, and learnability are the most important usability attributes to consider when designing eHealth mobile apps. Surprisingly, other significant attributes for healthcare apps, such as privacy and security, were not among the most indicated attributes in the studies.

Adapting and Implementing Apps for Mental Healthcare

PURPOSE OF REVIEW

To describe examples of adapting apps for use in mental healthcare and to formulate recommendations for successful adaptation in mental healthcare settings.

RECENT FINDINGS

International examples are given to explore implementation procedures to address this multitude of challenges. There are only few published examples of adapting apps for use in mental healthcare. From these examples and from results of studies in implementation science in general clinical settings, it can be concluded that the process of adapting apps for mental healthcare needs to address clinician training and information needs, user needs which include cultural adaptation and go beyond mere translation, and organizational needs for blending app use into everyday clinical mental healthcare workflows.

Development of the Shift Smartphone App to Support the Emotional Well-Being of Junior Physicians: Design of a Prototype and Results of Usability and Acceptability Testing

Background

Junior physicians report higher levels of psychological distress than senior doctors and report several barriers to seeking professional mental health support, including concerns about confidentiality and career progression. Mobile health (mHealth) apps may be utilized to help overcome these barriers to assist the emotional well-being of this population and encourage help-seeking.

Objective

This study describes the development and pilot trial of the Shift mHealth app to provide an unobtrusive avenue for junior physicians to seek information about, and help for, well-being and mental health concerns, which is sensitive to workplace settings.

Methods

A 4-phase iterative development process was undertaken to create the content and features of Shift involving junior physicians using the principles of user-centered design. These 4 phases were—needs assessment, on the basis of interviews with 12 junior physicians; prototype development with user experience feedback from 2 junior physicians; evaluation, consisting of a pilot trial with 22 junior physicians to assess the usability and acceptability of the initial prototype; and redesign, including user experience workshops with 51 junior physicians.

Results

Qualitative results informed the content and design of Shift to ensure that the app was tailored to junior physicians’ needs. The Shift app prototype contained cognitive behavioral, mindfulness, value-based actions, and psychoeducational modules, as well as a tracking function that visualized patterns of daily variations in mood and health behaviors. Pilot-testing revealed possible issues with the organization of the app content, which were addressed through a thorough restructuring and redesign of Shift with the help of junior physicians across 3 user experience workshops.

Conclusions

This study demonstrates the importance of ongoing end user involvement in the creation of a specialized mHealth app for a unique working population experiencing profession-specific stressors and barriers to help-seeking. The development and pilot trial of this novel Shift mHealth app are the first steps in addressing the mental health and support-seeking needs of junior physicians, although further research is required to validate its effectiveness and appropriateness on a larger scale.

Portable technologies for digital phenotyping of bipolar disorder: A systematic review

BACKGROUND

Bias-prone psychiatric interviews remain the mainstay of bipolar disorder (BD) assessment. The development of digital phenotyping promises to improve BD management. We present a systematic review of the evidence about the use of portable digital devices for the identification of BD, BD types and BD mood states and for symptom assessment.

METHODS

We searched Web of Knowledge^SM, Scopus ®, IEEE Xplore, and ACM Digital Library databases (until 5/1/2021) for articles evaluating the use of portable/wearable digital devices, such as smartphone apps, wearable sensors, audio and/or visual recordings, and multimodal tools. The protocol is registered in PROSPERO (CRD42020200086).

RESULTS

We included 62 studies (2325 BD; 724 healthy controls, HC): 27 using smartphone apps, either for recording self-assessments (n = 10) or for passively gathering metadata (n = 7) or both (n = 10); 15 using wearable sensors for physiological parameters; 17 analysing audio and/or video recordings; 3 using multiple technologies. Two thirds of the included studies applied artificial intelligence (AI)-based approaches. They achieved fair to excellent classification performances.

LIMITATIONS

The included studies had small sample sizes and marked heterogeneity. Evidence of overfitting emerged, limiting generalizability. The absence of clear guidelines about reporting classification performances, with no shared standard metrics, makes results hardly interpretable and comparable.

CONCLUSIONS

New technologies offer a noteworthy opportunity to BD digital phenotyping with objectivity and high granularity. AI-based models could deliver important support in clinical decision-making. Further research and cooperation between different stakeholders are needed for addressing methodological, ethical and socio-economic considerations.

'Care co-ordinator in my pocket': a feasibility study of mobile assessment and therapy for psychosis (TechCare)

OBJECTIVES

The aim of the project was to examine the acceptability and feasibility of a mobile phone application-based intervention 'TechCare', for individuals with psychosis in the North West of England. The main objectives were to determine whether appropriate individuals could be identified and recruited to the study and whether the TechCare App would be an acceptable intervention for individuals with psychosis.

METHODS

This was a mixed methods feasibility study, consisting of a test-run and feasibility evaluation of the TechCare App intervention.

SETTING

Early Intervention Services (EIS) for psychosis, within an NHS Trust in the North West of England.

PARTICIPANTS

Sixteen participants (test-run n=4, feasibility study n=12) aged between 18 and 65 years recruited from the East, Central and North Lancashire EIS.

INTERVENTION

A 6-week intervention, with the TechCare App assessing participants' symptoms and responses in real-time and providing a personalised-guided self-help-based psychological intervention based on the principles of Cognitive Behaviorual Therapy (CBT).

RESULTS

A total of 83.33% (n=10) of participants completed the 6-week feasibility study, with 70% of completers achieving the set compliance threshold of ≥33% engagement with the TechCare App system. Analysis of the qualitative data suggested that participants held the view that the TechCare was both an acceptable and feasible means of delivering interventions in real-time.

CONCLUSION

Innovative digital clinical technologies, such as the TechCare App, have the potential to increase access to psychological interventions, reduce health inequality and promote self-management with a real-time intervention, through enabling access to mental health resources in a stigma-free, evidence-based and time-independent manner.

TRIAL REGISTRATION NUMBER

ClinicalTrials.gov Identifier: NCT02439619.

Usability Evaluation of an Offline Electronic Data Capture App in a Prospective Multicenter Dementia Registry (digiDEM Bayern): Mixed Method Study

BACKGROUND

Digital registries have been shown to provide an efficient way of gaining a better understanding of the clinical complexity and long-term progression of diseases. The paperless method of electronic data capture (EDC) during a patient interview saves both time and resources. In the prospective multicenter project "Digital Dementia Registry Bavaria (digiDEM Bayern)," interviews are also performed on site in rural areas with unreliable internet connectivity. It must be ensured that EDC can still be performed in such a context and that there is no need to fall back on paper-based questionnaires. In addition to a web-based data collection solution, the EDC system REDCap (Research Electronic Data Capture) offers the option to collect data offline via an app and to synchronize it afterward.

OBJECTIVE

The aim of this study was to evaluate the usability of the REDCap app as an offline EDC option for a lay user group and to examine the necessary technology acceptance of using mobile devices for data collection. The feasibility of the app-based offline data collection in the digiDEM Bayern dementia registry project was then evaluated before going live.

METHODS

An exploratory mixed method design was employed in the form of an on-site usability test with the "Thinking Aloud" method combined with an online questionnaire including the System Usability Scale (SUS). The acceptance of mobile devices for data collection was surveyed based on five categories of the technology acceptance model.

RESULTS

Using the "Thinking Aloud" method, usability issues were identified and solutions were accordingly derived. Evaluation of the REDCap app resulted in a SUS score of 74, which represents "good" usability. After evaluating the technology acceptance questionnaire, it can be concluded that the lay user group is open to mobile devices as interview tools.

CONCLUSIONS

The usability evaluation results show that a lay user group generally agree that data collecting partners in the digiDEM project can handle the REDCap app well. The usability evaluation provided statements about positive aspects and could also identify usability issues relating to the REDCap app. In addition, the current technology acceptance in the sample showed that heterogeneous groups of different ages with diverse experiences in handling mobile devices are also ready for the use of app-based EDC systems. Based on these results, it can be assumed that the offline use of an app-based EDC system on mobile devices is a viable solution for collecting data in a decentralized registry-based research project.

Methodological Quality of User-Centered Usability Evaluation of Ambient Assisted Living Solutions: A Systematic Literature Review

This study aimed to determine the methodological quality of user-centered usability evaluation of Ambient Assisted Living (AAL) solutions by (i) identifying the characteristics of the AAL studies reporting on user-centered usability evaluation, (ii) systematizing the methods, procedures and instruments being used, and (iii) verifying if there is evidence of a common understanding on methods, procedures, and instruments for user-centered usability evaluation. An electronic search was conducted on Web of Science, Scopus, and IEEE Xplore databases, combining relevant keywords. Then, titles and abstracts were screened against inclusion and exclusion criteria, and the full texts of the eligible studies were retrieved and screened for inclusion. A total of 44 studies were included. The results show a great heterogeneity of methods, procedures, and instruments to evaluate the usability of AAL solutions and, in general, the researchers fail to consider and report relevant methodological aspects. Guidelines and instruments to assess the quality of the studies might help improving the experimental design and reporting of studies on user-centered usability evaluation of AAL solutions.

The Box: Methods and Results of a Real World Experience of mHealth Implementation in Clinical Practice

Background

Mobile health (mHealth) is an emerging field of scientific interest worldwide. Potential benefits include increased patient engagement, improved clinical outcomes, and reduced health care costs. However, mHealth is often studied in projects or trials, and structural implantation in clinical practice is less common.

Objective

The purpose of this paper is to outline the design of the Box and its implementation and use in an outpatient clinic setting. The impact on logistical outcomes and patient and provider satisfaction is discussed.

Methods

In 2016, an mHealth care track including smartphone-compatible devices, named the Box, was implemented in the cardiology department of a tertiary medical center in the Netherlands. Patients with myocardial infarction, rhythm disorders, cardiac surgery, heart failure, and congenital heart disease received devices to measure daily weight, blood pressure, heart rate, temperature, and oxygen saturation. In addition, professional and patient user comments on the experience with the care track were obtained via structured interviews.

Results

From 2016 to April 2020, a total of 1140 patients were connected to the mHealth care track. On average, a Box cost €350 (US $375), not including extra staff costs. The median patient age was 60.8 (IQR 52.9-69.3) years, and 73.59% (839/1140) were male. A median of 260 (IQR 105-641) measurements was taken on a median of 189 (IQR 98-372) days. Patients praised the ease of use of the devices and felt more involved with their illness and care. Professionals reported more productive outpatient consultations as well as improved insight into health parameters such as blood pressure and weight. A feedback loop from the hospital to patient to focus on measurements was commented as an important improvement by both patients and professionals.

Conclusions

In this study, the design and implementation of an mHealth care track for outpatient follow-up of patients with various cardiovascular diseases is described. Data from these 4 years indicate that mHealth is feasible to incorporate in outpatient management and is generally well-accepted by patients and providers. Limitations include the need for manual measurement data checks and the risk of data overload. Moreover, the tertiary care setting in which the Box was introduced may limit the external validity of logistical and financial end points to other medical centers. More evidence is needed to show the effects of mHealth on clinical outcomes and on cost-effectiveness.

Comprehensively addressing postpartum maternal health: a content and image review of commercially available mobile health apps

BACKGROUND

The United States is currently facing a maternal morbidity and mortality crisis, with the highest rates of any resource-rich nation. In efforts to address this, new guidelines for postpartum care suggest that mobile health (mHealth) apps can help provide complementary clinical support for new mothers during the postpartum period. However, to date no study has evaluated the quality of existing mHealth tools targeted to this time period in terms of sufficiency of maternal health information, inclusivity of people of color, and app usability.

METHODS

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) standards were used to review the peripartum apps from the Apple and Google Play stores in either the Health/Fitness, Medical, or Education categories. Apps were evaluated for extent and quality of maternal health information and inclusivity of people of color using an a priori coding scheme. App usability was evaluated using the Mobile Application Rating Scale (MARS) score.

RESULTS

Of the 301 apps from the Apple and Google Play stores, 25 met criteria for final evaluation. Of the 30 maternal health topics coded for, the median number addressed by apps was 19.5 (65%). Peripartum behaviors were more frequently addressed than peripartum outpatient care topics and peripartum acute health risks. The coverage of maternal health information and inclusivity of people of color in app imagery both correlated positively with the MARS usability score of the app. Only 8 apps (32%) portrayed greater than 24% images of people of color- the percent of non-white Americans according to 2019 census estimates. There was no correlation between MARS usability score and number of app users, as estimated by number of ratings for the app available on the app store. In addition, apps with evidence-based maternal health information had greater MARS engagement, information, and aesthetics scores. However, presence of evidence-based information did not correlate with greater numbers of app users.

CONCLUSIONS

Current commercially available peripartum apps range widely in quality. Overall current app offerings generally do not provide adequate maternal health information and are not optimally accessible to the target users in terms of inclusivity of women of color or app usability. Apps delivering evidence-based information and more usable design are more likely to meet these standards but are not more likely to be downloaded by users.

Framework for the Design Engineering and Clinical Implementation and Evaluation of mHealth Apps for Sleep Disturbance: Systematic Review

Background

Mobile health (mHealth) apps offer a scalable option for treating sleep disturbances at a population level. However, there is a lack of clarity about the development and evaluation of evidence-based mHealth apps.

Objective

The aim of this systematic review was to provide evidence for the design engineering and clinical implementation and evaluation of mHealth apps for sleep disturbance.

Methods

A systematic search of studies published from the inception of databases through February 2020 was conducted using 5 databases (MEDLINE, Embase, Cochrane Library, PsycINFO, and CINAHL).

Results

A total of 6015 papers were identified using the search strategy. After screening, 15 papers were identified that examined the design engineering and clinical implementation and evaluation of 8 different mHealth apps for sleep disturbance. Most of these apps delivered cognitive behavioral therapy for insomnia (CBT-I, n=4) or modified CBT-I (n=2). Half of the apps (n=4) identified adopting user-centered design or multidisciplinary teams in their design approach. Only 3 papers described user and data privacy. End-user acceptability and engagement were the most frequently assessed implementation metrics. Only 1 app had available evidence assessing all 4 implementation metrics (ie, acceptability, engagement, usability, and adherence). Most apps were prototype versions (n=5), with few matured apps. A total of 6 apps had supporting papers that provided a quantitative evaluation of clinical outcomes, but only 1 app had a supporting, adequately powered randomized controlled trial.

Conclusions

This is the first systematic review to synthesize and examine evidence for the design engineering and clinical implementation and evaluation of mHealth apps for sleep disturbance. The minimal number of apps with published evidence for design engineering and clinical implementation and evaluation contrasts starkly with the number of commercial sleep apps available. Moreover, there appears to be no standardization and consistency in the use of best practice design approaches and implementation assessments, along with very few rigorous efficacy evaluations. To facilitate the development of successful and evidence-based apps for sleep disturbance, we developed a high-level framework to guide researchers and app developers in the end-to-end process of app development and evaluation.

Acceptability, Validity, and Engagement With a Mobile App for Frequent, Continuous Multiyear Assessment of Youth Health Behaviors (mNCANDA): Mixed Methods Study

BACKGROUND

Longitudinal studies of many health behaviors often rely on infrequent self-report assessments. The measurement of psychoactive substance use among youth is expected to improve with more frequent mobile assessments, which can reduce recall bias. Researchers have used mobile devices for longitudinal research, but studies that last years and assess youth continuously at a fine-grained, temporal level (eg, weekly) are rare. A tailored mobile app (mNCANDA [mobile National Consortium on Alcohol and Neurodevelopment in Adolescence]) and a brief assessment protocol were designed specifically to provide a feasible platform to elicit responses to health behavior assessments in longitudinal studies, including NCANDA (National Consortium on Alcohol and Neurodevelopment in Adolescence).

OBJECTIVE

This study aimed to determine whether an acceptable mobile app system could provide repeatable and valid assessment of youth's health behaviors in different developmental stages over extended follow-up.

METHODS

Participants were recruited (n=534; aged 17-28 years) from a larger longitudinal study of neurodevelopment. Participants used mNCANDA to register reports of their behaviors for up to 18 months. Response rates as a function of time measured using mNCANDA and participant age were modeled using generalized estimating equations to evaluate response rate stability and age effects. Substance use reports captured using mNCANDA were compared with responses from standardized interviews to assess concurrent validity. Reactivity was assessed by evaluating patterns of change in substance use after participants initiated weekly reports via mNCANDA. Quantitative feedback about the app was obtained from the participants. Qualitative interviews were conducted with a subset of participants who used the app for at least one month to obtain feedback on user experience, user-derived explanations of some quantitative results, and suggestions for system improvements.

RESULTS

The mNCANDA protocol adherence was high (mean response rate 82%, SD 27%) and stable over time across all age groups. The median time to complete each assessment was 51 s (mean response time 1.14, SD 1.03 min). Comparisons between mNCANDA and interview self-reports on recent (previous 30 days) alcohol and cannabis use days demonstrate close agreement (eg, within 1 day of reported use) for most observations. Models used to identify reactivity failed to detect changes in substance use patterns subsequent to enrolling in mNCANDA app assessments (P>.39). Most participants (64/76, 84%) across the age range reported finding the mNCANDA system acceptable. Participants provided recommendations for improving the system (eg, tailoring signaling times).

CONCLUSIONS

mNCANDA provides a feasible, multi-year, continuous, fine-grained (eg, weekly) assessment of health behaviors designed to minimize respondent burden and provides acceptable regimes for long-term self-reporting of health behaviors. Fine-grained characterization of variability in behaviors over relatively long periods (eg, up to 18 months) may, through the use of mNCANDA, improve our understanding of the relationship between substance use exposure and neurocognitive development.

Procedures of User-Centered Usability Assessment for Digital Solutions: Scoping Review of Reviews Reporting on Digital Solutions Relevant for Older Adults

BACKGROUND

The assessment of usability is a complex process that involves several steps and procedures. It is important to standardize the evaluation and reporting of usability procedures across studies to guide researchers, facilitate comparisons across studies, and promote high-quality usability studies. The first step to standardizing is to have an overview of how usability study procedures are reported across the literature.

OBJECTIVE

This scoping review of reviews aims to synthesize the procedures reported for the different steps of the process of conducting a user-centered usability assessment of digital solutions relevant for older adults and to identify potential gaps in the present reporting of procedures. The secondary aim is to identify any principles or frameworks guiding this assessment in view of a standardized approach.

METHODS

This is a scoping review of reviews. A 5-stage scoping review methodology was used to identify and describe relevant literature published between 2009 and 2020 as follows: identify the research question, identify relevant studies, select studies for review, chart data from selected literature, and summarize and report results. The research was conducted on 5 electronic databases: PubMed, ACM Digital Library, IEEE, Scopus, and Web of Science. Reviews that met the inclusion criteria (reporting on user-centered usability evaluation procedures for any digital solution that could be relevant for older adults and were published in English) were identified, and data were extracted for further analysis regarding study evaluators, study participants, methods and techniques, tasks, and test environment.

RESULTS

A total of 3958 articles were identified. After a detailed screening, 20 reviews matched the eligibility criteria. The characteristics of the study evaluators and participants and task procedures were only briefly and differently reported. The methods and techniques used for the assessment of usability are the topics that were most commonly and comprehensively reported in the reviews, whereas the test environment was seldom and poorly characterized.

CONCLUSIONS

A lack of a detailed description of several steps of the process of assessing usability and no evidence on good practices of performing it suggests that there is a need for a consensus framework on the assessment of user-centered usability evaluation. Such a consensus would inform researchers and allow standardization of procedures, which are likely to result in improved study quality and reporting, increased sensitivity of the usability assessment, and improved comparability across studies and digital solutions. Our findings also highlight the need to investigate whether different ways of assessing usability are more sensitive than others. These findings need to be considered in light of review limitations.

Wearable Devices: Current Status and Opportunities in Pain Assessment and Management

INTRODUCTION

We investigated the possibilities and opportunities for using wearable devices that measure physical activity and physiometric signals in conjunction with ecological momentary assessment (EMA) data to improve the assessment and treatment of pain.

METHODS

We considered studies with cross-sectional and longitudinal designs as well as interventional or observational studies correlating pain scores with measures derived from wearable devices. A search was also performed on studies that investigated physical activity and physiometric signals among patients with pain.

RESULTS

Few studies have assessed the possibility of incorporating wearable devices as objective tools for contextualizing pain and physical function in free-living environments. Of the studies that have been conducted, most focus solely on physical activity and functional outcomes as measured by a wearable accelerometer. Several studies report promising correlations between pain scores and signals derived from wearable devices, objectively measured physical activity, and physical function. In addition, there is a known association between physiologic signals that can be measured by wearable devices and pain, though studies using wearable devices to measure these signals and associate them with pain in free-living environments are limited.

CONCLUSION

There exists a great opportunity to study the complex interplay between physiometric signals, physical function, and pain in a real-time fashion in free-living environments. The literature supports the hypothesis that wearable devices can be used to develop reproducible biosignals that correlate with pain. The combination of wearable devices and EMA will likely lead to the development of clinically meaningful endpoints that will transform how we understand and treat pain patients.

Community health programs delivered through information and communications technology in high-income countries: a scoping review (Preprint)

Background

The COVID-19 pandemic has required widespread and rapid adoption of information and communications technology (ICT) platforms by health professionals. Transitioning health programs from face-to-face to remote delivery using ICT platforms has introduced new challenges.

Objective

The objective of this review is to scope for ICT-delivered health programs implemented within the community health setting in high-income countries and rapidly disseminate findings to health professionals.

Methods

The Joanna Briggs Institute’s scoping review methodology guided the review of the literature.

Results

The search retrieved 7110 unique citations. Each title and abstract was screened by at least two reviewers, resulting in 399 citations for full-text review. Of these 399 citations, 72 (18%) were included. An additional 27 citations were identified through reviewing the reference lists of the included studies, resulting in 99 citations. Citations examined 83 ICT-delivered programs from 19 high-income countries. Variations in program design, ICT platforms, research design, and outcomes were evident.

Conclusions

Included programs and research were heterogeneous, addressing prevalent chronic diseases. Evidence was retrieved for the effectiveness of nurse and allied health ICT-delivered programs. Findings indicated that outcomes for participants receiving ICT-delivered programs, when compared with participants receiving in-person programs, were either equivalent or better. Gaps included a paucity of co-designed programs, qualitative research around group programs, programs for patients and carers, and evaluation of cost-effectiveness. During COVID-19 and beyond, health professionals in the community health setting are encouraged to build on existing knowledge and address evidence gaps by developing and evaluating innovative ICT-delivered programs in collaboration with consumers and carers.