Understanding the quality, effectiveness and attributes of top-rated smartphone health apps

A shot in the dark: the impact of online visibility on the search for an effective sleep app

STUDY OBJECTIVES

Dictated by consumer ratings and concealed algorithms, high levels of online visibility are granted to certain sleep apps on mainstream modes of app selection. Yet, it remains unclear to what extent these highly visible apps are evidence-based. The objectives of this review were to identify and describe the apps with the greatest online visibility when searching for a sleep app and to assess the claimed and actual research associated with them.

METHODS

A keyword search was conducted in Google Play and Google search. Titles of the most visible apps were retrieved. App descriptions were examined to identify research claims made about app effectiveness on sleep and other health-related outcomes. A follow-up search on PubMed and Google Scholar was conducted to verify claims.

RESULTS

The keyword search identified 53 highly visible apps. Examination of app store descriptions found that no reference to research was made for the majority of apps (n = 45, 84.9%). Published research studies were available for just 3 apps, with most studies evaluating app impact on non sleep-related outcomes. There was some evidence to attesting to the effectiveness of 2 apps in improving sleep.

CONCLUSIONS

This review demonstrates how, when carrying out a typical search for a sleep app, information about the evidence base for the majority of highly visible apps is not available. Results highlight the need for the improvement of mainstream modes of app selection in terms of better consumer-app specificity and increased transparency regarding the access to information about the evidence base for apps.

CITATION

Power N, Boivin DB, Perreault M. A shot in the dark: the impact of online visibility on the search for an effective sleep app. J Clin Sleep Med. 2024;20(8):1379-1386.

Clinical Validation of Digital Healthcare Solutions: State of the Art, Challenges and Opportunities

Digital health technologies (DHTs) at the intersection of health, medical informatics, and business aim to enhance patient care through personalised digital approaches. Ensuring the efficacy and reliability of these innovations demands rigorous clinical validation. A PubMed literature review (January 2006 to July 2023) identified 1250 papers, highlighting growing academic interest. A focused narrative review (January 2018 to July 2023) delved into challenges, highlighting issues such as diverse regulatory landscapes, adoption issues in complex healthcare systems, and a plethora of evaluation frameworks lacking pragmatic guidance. Existing frameworks often omit crucial criteria, neglect empirical evidence, and clinical effectiveness is rarely included as a criterion for DHT quality. The paper underscores the urgency of addressing challenges in accreditation, adoption, business models, and integration to safeguard the quality, efficacy, and safety of DHTs. A pivotal illustration of collaborative efforts to address these challenges is exemplified by the Digital Health Validation Center, dedicated to generating clinical evidence of innovative healthcare technologies and facilitating seamless technology transfer. In conclusion, it is necessary to harmonise evaluation approaches and frameworks, improve regulatory clarity, and commit to collaboration to integrate rigorous clinical validation and empirical evidence throughout the DHT life cycle.

The Economics of Treatment for Depression

The global prevalence of depression has risen over the past three decades across all socioeconomic groups and geographic regions, with a particularly rapid increase in prevalence among adolescents (aged 12-17 years) in the United States. Depression imposes large health, economic, and societal costs, including reduced life span and quality of life, medical costs, and reduced educational attainment and workplace productivity. A wide range of treatment modalities for depression are available, but socioeconomic disparities in treatment access are driven by treatment costs, lack of culturally tailored options, stigma, and provider shortages, among other barriers. This review highlights the need for comparative research to better understand treatments' relative efficacy, cost-effectiveness, scalability, and potential heterogeneity in efficacy across socioeconomic groups and country and cultural contexts. To address the growing burden of depression, mental health policy could consider reducing restrictions on the supply of providers, implementing digital interventions, reducing stigma, and promoting healthy lifestyles.

Attributes, Quality, and Downloads of Dementia-Related Mobile Apps for Patients With Dementia and Their Caregivers: App Review and Evaluation Study

BACKGROUND

The adoption of mobile health (mHealth) apps among older adults (>65 years) is rapidly increasing. However, use of such apps has not been fully effective in supporting people with dementia and their caregivers in their daily lives. This is mainly attributed to the heterogeneous quality of mHealth apps, highlighting the need for improved app quality in the development of dementia-related mHealth apps.

OBJECTIVE

The aims of this study were (1) to assess the quality and content of mobile apps for dementia management and (2) to investigate the relationship between app quality and download numbers.

METHODS

We reviewed dementia-related mHealth apps available in the Google Play Store and Apple App Store in Taiwan. The identified mobile apps were stratified according to a random sampling approach and evaluated by five independent reviewers with sufficient training and proficiency in the field of mHealth and the related health care sector. App quality was scored according to the user version of the Mobile Application Rating Scale. A correlation analysis was then performed between the app quality score and number of app downloads.

RESULTS

Among the 17 apps that were evaluated, only one was specifically designed to provide dementia-related education. The mean score for the overall app quality was 3.35 (SD 0.56), with the engagement (mean 3.04, SD 0.82) and information (mean 3.14, SD 0.88) sections of the scale receiving the lowest ratings. Our analyses showed clear differences between the top three- and bottom three-rated apps, particularly in the entertainment and interest subsections of the engagement category where the ratings ranged from 1.4 to 5. The top three apps had a common feature in their interface, which included memory, attention, focus, calculation, and speed-training games, whereas the apps that received lower ratings were found to be deficient in providing adequate information. Although there was a correlation between the number of downloads (5000 or more) and app quality (t15=4.087, P<.001), this may not be a significant determinant of the app's perceived impact.

CONCLUSIONS

The quality of dementia-related mHealth apps is highly variable. In particular, our results show that the top three quality apps performed well in terms of engagement and information, and they all received more than 5000 downloads. The findings of this study are limited due to the small sample size and possibility of disregarding exceptional occurrences. Publicly available expert ratings of mobile apps could help people with dementia and their caregivers choose a quality mHealth app.

Design, development and evaluation of a mobile-based self-care application for patients with COVID-19 not requiring hospitalization; a study of Northwest of Iran

BACKGROUND

Given the effective role of a mobile applications in disease management, disease monitoring, and self-care in patients with COVID-19 disease, we aimed to design, development and evaluation of a self-care Mobile app for COVID-19 patients not requiring hospitalization.

METHODS

The design, development and evaluation the usability of the self-care and education mobile app for patients with COVID-19 disease were conducted in two main phases at 2021 in Northwest of IRAN; (1) Determine the features and capabilities and (2) Design, development and evaluation of self-care mobile App. JAVA programming languages and Android Operating System were used and selected to design and development of a mobile app. There were 25 participants who conducted evaluations of the mobile app's usability and impact using the mobile health app usability a Questionnaire of User Interface Satisfaction was administered to assess the usability of the developed application. The results were analyzed via Excel 2013.

RESULTS

The model of developing a mobile app as an Information System was the Waterfall model. The smartphone application based on a set of capabilities and features was designed and consists of two main parts: the login screen for user registration, and the main home menu. The user interface includes three main pages or activities; (a) Main Menu for quick access to all of the pages, (b) Symptom management and monitoring to monitor the signs and symptoms during the illness, and (c) Set Reminders and Alarms to notify patients. The users' mean score of the application usability was calculated as 7.91 out of 9 indicating a good level of satisfaction.

CONCLUSION

This app can be a guideline and a useful tools for managing and monitoring symptoms, reminding medications, and implementing self-care instructions in outpatients. The authors suggest evaluating the efficacy and functionality test of mobile-based applications for COVID-19 in clinical trial studies.

Trust and confidence in using telehealth in people with chronic kidney disease: A cross-sectional study

Consumer trust and confidence in telehealth is pivotal to successful service implementation and effective consultations. This cross-sectional study measured trust and confidence in telephone and video consultations and associated with experience in telehealth modalities among people with chronic kidney disease at a metropolitan hospital in Australia. Self-report data were collected using validated trust and confidence in telehealth scales and 5-point Likert responses. Non-parametric tests were used to compare trust and confidence in telephone and video consultations (Wilcoxon Matched Pairs) and associations with telehealth experience (Mann-Whitney). Of the 156 survey participants, 96.2% had used telephone consultations and 28.9% had used video. Overall trust and confidence in using telehealth were high. Confidence (range 1-5) in using telephone consultations (mean 3.75 ± 0.71) was significantly higher than video consultation (mean 3.64 ± 0.74), p = 0.039. Trust in telephone consultations (mean 3.93 ± 0.64) was significantly higher than in video consultations (mean 3.67 ± 0.66), p < 0.001. There was a significant association between experience with telephone consultations and reported levels of trust and confidence in telephone consultations. Experience with video was significantly related to trust in video consultations, but not confidence. Given the substantial difference in experience between telehealth modalities, trust and confidence may change as further exposure occurs.

Cultural Adaptation of the Actionable Health App Evaluation in Japan: Protocol for a Web-Based Modified Delphi Expert Consensus Study

BACKGROUND

With an increase in both the number of mental health disorders people are experiencing and the difficulty in accessing mental health care, the demand for accessible mental health care services has increased. The use of mobile devices has allowed people to receive care in their daily lives without restrictions on time or location. However, the majority of publicly available mobile health apps are not evidence-based, and the top-rated apps are not always safe or user-friendly and may not offer clinically beneficial results.

OBJECTIVE

This study aims to create a cultural adaptation of the American Psychiatric Association's comprehensive app evaluation framework in Japan using a web-based modified Delphi expert consensus.

METHODS

A web-based modified Delphi study includes developing the Japanese version of the comprehensive app evaluation framework and 3 Delphi rounds. In the first round, our working group sends a questionnaire to the panelists, who then complete it. In the second and third rounds, the working group sends a questionnaire and a summary of the panelists' answers based on each of the previous rounds. The panelists answer the questionnaires based on this summary. The summarization procedure is automated to help reduce the biases that can be generated when panelists' answers are summarized and when the panelists receive them. The working group sends only the result of the summarization with the next round's questionnaire. All interactions between the working group and the panelists will be conducted on Qualtrics (Qualtrics Japan LLC), a questionnaire platform. To culturally validate the comprehensive mental health app evaluation framework, participants from the following three categories will be recruited in Japan: (1) researchers, (2) practitioners, and (3) app developers.

RESULTS

This study received funding from a crowdfunding campaign in Japan (April 2023). The Delphi study began in January 2023 and will be completed in December 2023. We had already completed the translation of the 105 original app evaluation item questions by December 2022.

CONCLUSIONS

While the need for treatment using mental health apps is increasing, no framework that can be used to develop a centralized database for health apps is available or accessible, and no consensus has been reached among stakeholders in Japan about an appropriate framework. The results of the web-based modified Delphi method presented in this paper may provide direction for the development and use of mental health apps in the future among the relevant stakeholders. Furthermore, this study will enhance recognition of the framework among researchers, clinicians, mental health app developers, and users, in addition to devising new instruments to help users or practitioners efficiently choose the right app for their situations.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/44469.

Health Care Equity Through Intelligent Edge Computing and Augmented Reality/Virtual Reality: A Systematic Review

Intellectual capital is a scarce resource in the healthcare industry. Making the most of this resource is the first step toward achieving a completely intelligent healthcare system. However, most existing centralized and deep learning-based systems are unable to adapt to the growing volume of global health records and face application issues. To balance the scarcity of healthcare resources, the emerging trend of IoMT (Internet of Medical Things) and edge computing will be very practical and cost-effective. A full examination of the transformational role of intelligent edge computing in the IoMT era to attain health care equity is offered in this research. Intelligent edge computing-aided distribution and collaborative information management is a possible approach for a long-term digital healthcare system. Furthermore, IEC (Intelligent Edge Computing) encourages digital health data to be processed only at the edge, minimizing the amount of information exchanged with central servers/the internet. This significantly increases the privacy of digital health data. Another critical component of a sustainable healthcare system is affordability in digital healthcare. Affordability in digital healthcare is another key component of a sustainable healthcare system. Despite its importance, it has received little attention due to its complexity. In isolated and rural areas where expensive equipment is unavailable, IEC with AR / VR, also known as edge device shadow, can play a significant role in the inexpensive data collection process. Healthcare equity becomes a reality by combining intelligent edge device shadows and edge computing.

Rigorous and rapid evidence assessment in digital health with the evidence DEFINED framework

Dozens of frameworks have been proposed to assess evidence for digital health interventions (DHIs), but existing frameworks may not facilitate DHI evidence reviews that meet the needs of stakeholder organizations including payers, health systems, trade organizations, and others. These organizations may benefit from a DHI assessment framework that is both rigorous and rapid. Here we propose a framework to assess Evidence in Digital health for EFfectiveness of INterventions with Evaluative Depth (Evidence DEFINED). Designed for real-world use, the Evidence DEFINED Quick Start Guide may help streamline DHI assessment. A checklist is provided summarizing high-priority evidence considerations in digital health. Evidence-to-recommendation guidelines are proposed, specifying degrees of adoption that may be appropriate for a range of evidence quality levels. Evidence DEFINED differs from prior frameworks in its inclusion of unique elements designed for rigor and speed. Rigor is increased by addressing three gaps in prior frameworks. First, prior frameworks are not adapted adequately to address evidence considerations that are unique to digital health. Second, prior frameworks do not specify evidence quality criteria requiring increased vigilance for DHIs in the current regulatory context. Third, extant frameworks rarely leverage established, robust methodologies that were developed for non-digital interventions. Speed is achieved in the Evidence DEFINED Framework through screening optimization and deprioritization of steps that may have limited value. The primary goals of Evidence DEFINED are to a) facilitate standardized, rapid, rigorous DHI evidence assessment in organizations and b) guide digital health solutions providers who wish to generate evidence that drives DHI adoption.

Translating Research Evidence Into Marketplace Application: Cohort Study of Internet-Based Intervention Platforms for Perinatal Depression

BACKGROUND

Internet-based intervention platforms may improve access to mental health care for women with perinatal depression (PND). Though the majority of platforms in the market lack an evidence base, a small number of them are supported by research evidence.

OBJECTIVE

This study aims to assess the current status of internet-based PND intervention platforms supported by published evidence, understand the reasons behind the disappearance of any of these previously accessible platforms, examine adjustments made by those active platforms between research trials and market implementation, and evaluate their current quality.

METHODS

A cohort of internet-based PND intervention platforms was first identified by systematic searches in multiple academic databases from database inception until March 26, 2021. We searched on the World Wide Web and the iOS and Android app stores to assess which of these were available in the marketplace between April and May 2021. The basic characteristics of all platforms were collected. For inaccessible platforms, inquiries were made via email to the authors of publications to determine the reasons for their unavailability. We compared the intervention-related information of accessible platforms in the marketplace with that reported in original publications and conducted quality assessments using the App Evaluation Model of the American Psychiatric Association. Fisher exact tests were used to compare the functional characteristics in publications of available and unavailable platforms and to investigate potential associations between functional adjustments or quality indices and platform survival time.

RESULTS

Out of 35 platforms supported by research evidence, only 19 (54%) were still accessible in the marketplace. The main reason for platforms disappearing was the termination of research projects. No statistically significant differences were found in functional characteristics between available and unavailable platforms. A total of 18 (95%) platforms adapted their core functions from what was reported in related publications. The adjustments included changes to intervention methods (11/19, 58%), target population (10/19, 53%), human resources for intervention support (9/19, 47%), mood assessment and monitoring (8/19, 42%), communication modality (4/19, 21%), and platform type (2/19, 11%). Quality issues across platforms included low frequency of update, lack of crisis management mechanism, poor user interactivity, and weak evidence base or absence of citation of supporting evidence. Platforms that survived longer than 10 years had a higher tendency to use external resources from third parties compared to those that survived less than 10 years (P=.04). No significant differences were found for functional adjustments or other quality indices.

CONCLUSIONS

Internet-based platforms supported by evidence were not effectively translated into real-world practice. It is unclear if adjustments to accessible platforms made during actual operation may undermine the proven validity of the original research. Future research to explore the reasons behind the success of the implementation of evidence-based platforms in the marketplace is warranted.

Navigating the complexities of mobile medical app development from idea to launch, a guide for clinicians and biomedical researchers

With today’s pace of rapid technological advancement, many patient issues in modern medicine are increasingly solvable by mobile app solutions, which also have the potential to transform how clinical research is conducted. However, many critical challenges in the app development process impede bringing these translational technologies to patients, caused in large part by the lack of knowledge among clinicians and biomedical researchers of “what it takes” to design, develop, and maintain a successful medical app. Indeed, problems requiring mobile app solutions are often nuanced, requiring more than just clinical expertise, and issues such as the cost and effort required to develop and maintain a well-designed, sustainable, and scalable mobile app are frequently underestimated. To bridge this skill set gap, we established an academic unit of designers, software engineers, and scientists that leverage human-centered design methodologies and multi-disciplinary collaboration to develop clinically viable smartphone apps. In this report, we discuss major misconceptions clinicians and biomedical researchers often hold regarding medical app development, the steps we took to establish this unit to address these issues and the best practices and lessons learned from successfully ideating, developing, and launching medical apps. Overall, this report will serve as a blueprint for clinicians and biomedical researchers looking to better benefit their patients or colleagues via medical mobile apps.

Examining a Resilience Mental Health App in Adolescents: Acceptability and Feasibility Study

BACKGROUND

Resilience is defined as the ability to rely on internal characteristics and external strengths to adapt to adverse events. Although universal resilience-enhancing programs are effective for adolescents, there is a need for interventions that are more easily accessible and can be customized for individual teens. Phone apps are easy to use, can be tailored to individuals, and have demonstrated positive effects for mental health outcomes.

OBJECTIVE

This study aimed to examine the feasibility and acceptability of a resilience app for adolescents. This app aimed to enhance resilience through modules focused on depression prevention, stress management, and healthy lifestyle approaches containing videos, measures, and practice suggestions. Furthermore, the study aimed to evaluate the effect of short-term app use on changes in resilience.

METHODS

In study 1, individual interviews and focus groups were conducted with adolescents, parents, teachers, and clinicians to discuss possible incentives for using a mental health app, the benefits of app use, and concerns associated with app use. Feedback from study 1 led to ideas for the prototype. In study 2, individual interviews and focus groups were conducted with adolescents, parents, teachers, and clinicians to gather feedback about the resilience app prototype. Feedback from study 2 led to changes in the prototype, although not all suggestions could be implemented. In study 3, 40 adolescents used the app for 30 days to determine feasibility and acceptability. Additionally, resilience and secondary mental health outcomes were measured before and after app use. Dependent samples 2-tailed t tests were conducted to determine whether there were changes in resilience and secondary mental health outcomes among the adolescents before and after app use.

RESULTS

Multiple themes were identified through study 1 individual interviews and focus groups, including app content, features, engagement, benefits, concerns, and improvement. Specifically, the adolescents provided helpful suggestions for making the prototype more appealing and functional for teen users. Study 2 adolescents and adults reported that the prototype was feasible and acceptable through the Computer System Usability Questionnaire (mean 6.30, SD 1.03) and Mobile App Rating Scale (mean 4.08, SD 0.61). In study 2, there were no significant differences in resilience and mental health outcomes after using the app for 30 days. There was variation between the participants in the extent to which they used the app, which may have led to variation in the results. The users appeared to prefer the depression module and survey sections, which provided mental health feedback.

CONCLUSIONS

Qualitative and quantitative data provide evidence that youth are interested in a resilience mental health app and that the current prototype is feasible. Although there were no significant mental health changes in study 3 users, practical implications and future directions are discussed for mental health app research.

A Systematic Assessment of the Quality of Smartphone Applications for Gastroesophageal Reflux Disease

Background and Aims

Smartphone applications aimed at patients with gastroesophageal reflux disease (GERD) have been downloaded more than 100,000 times, yet no systematic assessment of their quality has been completed. This study aimed to objectively assess the quality of GERD smartphone applications for patient education and disease management.

Methods

The Apple App Store and Google Play Store were systematically searched for relevant applications. Two independent reviewers performed the application screening and eligibility assessment. Included applications were graded using the validated Mobile Application Rating Scale, which encompasses 4 domains (engagement, functionality, aesthetics, and information) as well as an overall application quality score. The associations between overall application quality, user ratings and download numbers were evaluated.

Results

Of the 4816 unique applications identified, 46 met inclusion criteria (patient education = 37, disease management = 9). Mean overall application quality score was 3.02 ± 0.40 out of 5 ("acceptable"), with 61% (28/46) rated as "poor" (score 2.0-2.9). Applications scored highest for aesthetics (3.24 ± 0.48) and functionality (3.88 ± 0.37) and lowest for information (2.58 ± 0.64) and engagement (2.39 ± 0.65). Disease management applications were of significantly higher quality than education-focused applications (3.59 ± 0.38 vs 2.88 ± 0.26, P < .001). There was no correlation between graded quality and either user ratings or the number of downloads.

Conclusion

While numerous smartphone applications exist to support patients with GERD, their quality is variable. Patient education applications are of particularly low quality. Our findings can help to inform the selection of applications by patients and guide clinicians' recommendations. This study also highlights the need for higher-quality, evidence-informed applications aimed at GERD patient education.

Introducing an implementation framework for augmenting care with digital technology for early psychosis patients: theory and motivation

BACKGROUND

Treatment programs for early-course psychosis are evidence-based interventions that provide specialty care to improve outcomes in patients. Digital technologies offer the potential to augment services and meet the growing demand for care.

AIMS

We introduce a framework to guide the assessment of site readiness for technology and their ability to successfully introduce, implement, and sustain digital technology use. While broader in use that early course psychosis, we focus on this use case to introduce the theory and clinical application.

METHODS

Adapting the replicating effective programs framework, we present an early psychosis focused model. Considering the unique opportunities and challenges of these programs, we present a five-stage evaluation framework. Informed by our clinical experience and recent literature, we present tools and examples to help programs plan and execute successful technology implementation.

RESULTS

The AACCS framework is comprised of five stages: (1) Access (e.g. identifying access to and comfort with technology), (2) Align (e.g. understanding aspects technology can augment), (3) Connect (e.g. customizing technology to stakeholder needs), (4) Care (e.g. implementing technology into treatment), and (5) Sustain (e.g. creating sustainable services). Site visits revealed patients have access to digital tools and are open to implementation into care, while staff prefers digital skills training.

CONCLUSIONS

This framework assists programs in identifying clinical targets to be augmented with technology, stages of implementation, and recommendations for sustaining meaningful technology use.

Accessibility and availability of smartphone apps for schizophrenia

App-based interventions have the potential to enhance access to and quality of care for patients with schizophrenia. However, less is known about the current state of schizophrenia apps in research and how those translate to publicly available apps. This study, therefore, aimed to review schizophrenia apps offered on marketplaces and research literature with a focus on accessibility and availability. A search of recent reviews, gray literature, PubMed, and Google Scholar was conducted in August 2022. A search of the U.S. Apple App Store and Google Play App Store was conducted in July 2022. All eligible studies and apps were systematically screened/reviewed. The academic research search produced 264 results; 60 eligible studies were identified. 51.7% of research apps were built on psychosis-specific platforms and 48.3% of research apps were built on non-specific platforms. 83.3% of research apps offered monitoring functionalities. Only nine apps, two designed on psychosis-specific platforms and seven on non-specific platforms were easily accessible. The search of app marketplaces uncovered 537 apps; only six eligible marketplace apps were identified. 83.3% of marketplace apps only offered psychoeducation. All marketplace apps lacked frequent updates with the average time since last update 1121 days. There are few clinically relevant apps accessible to patients on the commercial marketplaces. While research efforts are expanding, many research apps are unavailable today. Better translation of apps from research to the marketplace and a focus on sustainable interventions are important targets for the field.

Comparing Professional and Consumer Ratings of Mental Health Apps: Mixed Methods Study

Background

As the number of mental health apps has grown, increasing efforts have been focused on establishing quality tailored reviews. These reviews prioritize clinician and academic views rather than the views of those who use them, particularly those with lived experiences of mental health problems. Given that the COVID-19 pandemic has increased reliance on web-based and mobile mental health support, understanding the views of those with mental health conditions is of increasing importance.

Objective

This study aimed to understand the opinions of people with mental health problems on mental health apps and how they differ from established ratings by professionals.

Methods

A mixed methods study was conducted using a web-based survey administered between December 2020 and April 2021, assessing 11 mental health apps. We recruited individuals who had experienced mental health problems to download and use 3 apps for 3 days and complete a survey. The survey consisted of the One Mind PsyberGuide Consumer Review Questionnaire and 2 items from the Mobile App Rating Scale (star and recommendation ratings from 1 to 5). The consumer review questionnaire contained a series of open-ended questions, which were thematically analyzed and using a predefined protocol, converted into binary (positive or negative) ratings, and compared with app ratings by professionals and star ratings from app stores.

Results

We found low agreement between the participants’ and professionals’ ratings. More than half of the app ratings showed disagreement between participants and professionals (198/372, 53.2%). Compared with participants, professionals gave the apps higher star ratings (3.58 vs 4.56) and were more likely to recommend the apps to others (3.44 vs 4.39). Participants’ star ratings were weakly positively correlated with app store ratings (r=0.32, P=.01). Thematic analysis found 11 themes, including issues of user experience, ease of use and interactivity, privacy concerns, customization, and integration with daily life. Participants particularly valued certain aspects of mental health apps, which appear to be overlooked by professional reviewers. These included functions such as the ability to track and measure mental health and providing general mental health education. The cost of apps was among the most important factors for participants. Although this is already considered by professionals, this information is not always easily accessible.

Conclusions

As reviews on app stores and by professionals differ from those by people with lived experiences of mental health problems, these alone are not sufficient to provide people with mental health problems with the information they desire when choosing a mental health app. App rating measures must include the perspectives of mental health service users to ensure ratings represent their priorities. Additional work should be done to incorporate the features most important to mental health service users into mental health apps.

Assessment of Smartphone Apps for Common Neurologic Conditions (Headache, Insomnia, and Pain): Cross-sectional Study

BACKGROUND

There are thousands of apps for individuals struggling with headache, insomnia, and pain, but it is difficult to establish which of these apps are best suited for patients' specific needs. If clinicians were to have access to a platform that would allow them to make an informed decision on the efficacy and feasibility of smartphone apps for patient care, they would feel confident in prescribing specific apps.

OBJECTIVE

We sought to evaluate the quality of apps for some of the top common, disabling neurologic conditions (headache, insomnia, and pain) based on principles derived from the American Psychiatric Association's (APA) app evaluation model.

METHODS

We used the Mobile Health Index and Navigation database and expanded upon the database's current supported conditions by adding 177 new app entries. Each app was rated for consistency with the APA's app evaluation model, which includes 105 objective questions based on the following 5 major classes of consideration: (1) accessibility, (2) privacy and security, (3) clinical foundation, (4) engagement style, and (5) interoperability. These characteristics were evaluated to gain a broader understanding of the significant features of each app category in comparison against a control group.

RESULTS

Approximately 90% (187/201) of all apps evaluated were free to download, but only 50% (63/201) of headache- and pain-related apps were truly free. Most (87/106, 81%) sleep apps were not truly free to use. The apps had similar limitations with limited privacy, accessibility, and crisis management resources. For example, only 17% (35/201) of the apps were available in Spanish. The apps offered mostly self-help tools with little tailoring; symptom tracking was the most common feature in headache- (32/48, 67%) and pain-related apps (21/47, 45%), whereas mindfulness was the most common feature in sleep-related apps (73/106, 69%).

CONCLUSIONS

Although there are many apps for headache, pain, and insomnia, all 3 types of apps have room for improvement around accessibility and privacy. Pain and headache apps share many common features, whereas insomnia apps offer mostly mindfulness-based resources. Given the many available apps to pick from, clinicians and patients should seek apps that offer the highest-quality features, such as complete privacy, remedial features, and the ability to download the app at no cost. These results suggest that there are many opportunities for the improvement of apps centered on headache, insomnia, and pain.

Assessing the Clinical Robustness of Digital Health Startups: Cross-sectional Observational Analysis

BACKGROUND

The digital health sector has experienced rapid growth over the past decade. However, health care technology stakeholders lack a comprehensive understanding of clinical robustness and claims across the industry.

OBJECTIVE

This analysis aimed to examine the clinical robustness and public claims made by digital health companies.

METHODS

A cross-sectional observational analysis was conducted using company data from the Rock Health Digital Health Venture Funding Database, the US Food and Drug Administration, and the US National Library of Medicine. Companies were included if they sell products targeting the prevention, diagnosis, or treatment phases of the care continuum. Clinical robustness was defined using regulatory filings and clinical trials completed by each company. Public claims data included clinical, economic, and engagement claims regarding product outcomes made by each company on its website.

RESULTS

A total of 224 digital health companies with an average age of 7.7 years were included in our cohort. Average clinical robustness was 2.5 (1.8 clinical trials and 0.8 regulatory filings) with a median score of 1. Ninety-eight (44%) companies had a clinical robustness score of 0, while 45 (20%) companies had a clinical robustness score of 5 or more. The average number of public claims was 1.3 (0.5 clinical, 0.4 economic, and 0.4 engagement); the median number of claims was 1. No correlation was observed between clinical robustness and number of clinical claims (r2=0.02), clinical robustness and total funding (r2=0.08), or clinical robustness and company age (r2=0.18).

CONCLUSIONS

Many digital health companies have a low level of clinical robustness and do not make many claims as measured by regulatory filings, clinical trials, and public data shared online. Companies and customers may benefit from investing in greater clinical validation efforts.

Evaluating the quality, safety, and functionality of commonly used smartphone apps for bipolar disorder mood and sleep self-management

Background

Individuals with bipolar disorder (BD) are increasingly turning to smartphone applications (apps) for health information and self-management support. While reviews have raised concerns regarding the effectiveness and safety of publicly available apps for BD, apps surveyed may not reflect what individuals with BD are using. The present study had two aims: first, to characterize the use of health apps to support mood and sleep amongst people with BD, and second, to evaluate the quality, safety and functionality of the most commonly used self-management apps.

Methods

A web-based survey was conducted to explore which apps people with BD reported using to support self-management of mood and sleep. The characteristics of the most commonly nominated apps were described using a standardized framework, including their privacy policy, clinical foundations, and functionality.

Results

Respondents (n = 919) were 77.9% female with a mean age of 36.9 years. 41.6% of participants (n = 382) reported using a self-management app to support mood or sleep. 110 unique apps were nominated in relation to mood, and 104 unique apps nominated in relation to sleep; however, most apps were only mentioned once. The nine most frequently nominated apps related to mood and sleep were subject to further evaluation. All reviewed apps offered a privacy policy, however user control over data was limited and the complexity of privacy policies was high. Only one app was developed for BD populations. Half of reviewed apps had published peer-reviewed evidence to support their claims of efficacy, but little research was specific to BD.

Conclusion

Findings illustrate the potential of smartphone apps to increase the reach of psychosocial interventions amongst people with BD. Apps were largely created by commercial developers and designed for the general population, highlighting a gap in the development and dissemination of evidence-informed apps for BD. There may be risks in using generic health apps for BD self-management; clinicians should enquire about patients’ app use to foster conversations about their particular benefits and limitations.

Supplementary Information

The online version contains supplementary material available at 10.1186/s40345-022-00256-6.

Meeting Users Where They Are: User-centered Design of an Automated Text Messaging Tool to Support the Mental Health of Young Adults

Young adults have high rates of mental health conditions, but most do not want or cannot access formal treatment. We therefore recruited young adults with depression or anxiety symptoms to co-design a digital tool for self-managing their mental health concerns. Through study activities-consisting of an online discussion group and a series of design workshops-participants highlighted the importance of easy-to-use digital tools that allow them to exercise independence in their self-management. They described ways that an automated messaging tool might benefit them by: facilitating experimentation with diverse concepts and experiences; allowing variable depth of engagement based on preferences, availability, and mood; and collecting feedback to personalize the tool. While participants wanted to feel supported by an automated tool, they cautioned against incorporating an overtly human-like motivational tone. We discuss ways to apply these findings to improve the design and dissemination of digital mental health tools for young adults.

Review of Mobile Apps for Women With Anxiety in Pregnancy: Maternity Care Professionals' Guide to Locating and Assessing Anxiety Apps

Background

Mental health and pregnancy apps are widely available and have the potential to improve health outcomes and enhance women’s experience of pregnancy. Women frequently access digital information throughout their pregnancy. However, health care providers and women have little information to guide them toward potentially helpful or effective apps.

Objective

This review aimed to evaluate a methodology for systematically searching and reviewing commercially available apps that support pregnant women with symptoms of anxiety in order to assist maternity care professionals in identifying resources that they could recommend for these women.

Methods

A stepwise systematic approach was used to identify, select, describe, and assess the most popular and highly user-rated apps available in the United Kingdom from January to March 2021. This included developing a script-based search strategy and search process, writing evaluation criteria, and conducting a narrative description and evaluation of the selected apps.

Results

Useful search terms were identified, which included nonclinical, aspirational, and problem-based phrases. There were 39 apps selected for inclusion in the review. No apps specifically targeted women with anxiety in pregnancy. Of the 39 apps included in the review, 33 (85%) focused solely on mind-body techniques to promote relaxation, stress reduction, and psychological well-being. Only 8 of the 39 (21%) apps included in the review reported that health care professionals had contributed to app development and only 1/39 (3%) provided empirical evidence on the effectiveness and acceptability of the app. The top 12/39 (31%) apps were evaluated by 2 independent reviewers using the developed criteria and scores. There was a small negative correlation between the reviewers’ scores and app user rating scores, with higher user rating scores associated with lower reviewer scores.

Conclusions

App developers, publishers, and maternity care professionals should seek advice from women with lived experience of anxiety symptoms in pregnancy to locate, promote, and optimize the visibility of apps for pregnant women. There is a lack of resources that provide coping strategies based on current evidence for the treatment of anxiety in pregnancy. Maternity care providers are limited in their ability to locate and recommend acceptable and trustworthy apps because of the lack of information on the evidence base, development, and testing of apps. Maternity care professionals and women need access to libraries of trusted apps that have been evaluated against relevant and established criteria.

Process and Outcome Evaluations of Smartphone Apps for Bipolar Disorder: Scoping Review

Background

Mental health apps (MHAs) provide opportunities for accessible, immediate, and innovative approaches to better understand and support the treatment of mental health disorders, especially those with a high burden, such as bipolar disorder (BD). Many MHAs have been developed, but few have had their effectiveness evaluated.

Objective

This systematic scoping review explores current process and outcome measures of MHAs for BD with the aim to provide a comprehensive overview of current research. This will identify the best practice for evaluating MHAs for BD and inform future studies.

Methods

A systematic literature search of the health science databases PsycINFO, MEDLINE, Embase, EBSCO, Scopus, and Web of Science was undertaken up to January 2021 (with no start date) to narratively assess how studies had evaluated MHAs for BD.

Results

Of 4051 original search results, 12 articles were included. These 12 studies included 435 participants, and of these, 343 had BD type I or II. Moreover, 11 of the 12 studies provided the ages (mean 37 years) of the participants. One study did not report age data. The male to female ratio of the 343 participants was 137:206. The most widely employed validated outcome measure was the Young Mania Rating Scale, being used 8 times. The Hamilton Depression Rating Scale-17/Hamilton Depression Rating Scale was used thrice; the Altman Self-Rating Mania Scale, Quick Inventory of Depressive Symptomatology, and Functional Assessment Staging Test were used twice; and the Coping Inventory for Stressful Situations, EuroQoL 5-Dimension Health Questionnaire, Generalized Anxiety Disorder Scale-7, Inventory of Depressive Symptomatology, Mindfulness Attention Awareness Scale, Major Depression Index, Morisky-Green 8-item, Perceived Stress Scale, and World Health Organization Quality of Life-BREF were used once. Self-report measures were captured in 9 different studies, 6 of which used MONARCA. Mood and energy levels were the most commonly used self-report measures, being used 4 times each. Furthermore, 11 of the 12 studies discussed the various confounding factors and barriers to the use of MHAs for BD.

Conclusions

Reported low adherence rates, usability challenges, and privacy concerns act as barriers to the use of MHAs for BD. Moreover, as MHA evaluation is itself developing, guidance for clinicians in how to aid patient choices in mobile health needs to develop. These obstacles could be ameliorated by incorporating co-production and co-design using participatory patient approaches during the development and evaluation stages of MHAs for BD. Further, including qualitative aspects in trials that examine patient experience of both mental ill health and the MHA itself could result in a more patient-friendly fit-for-purpose MHA for BD.

Data Collection Mechanisms in Health and Wellness Apps: Review and Analysis

Background

There has been a steady rise in the availability of health wearables and built-in smartphone sensors that can be used to collect health data reliably and conveniently from end users. Given the feature overlaps and user tendency to use several apps, these are important factors impacting user experience. However, there is limited work on analyzing the data collection aspect of mobile health (mHealth) apps.

Objective

This study aims to analyze what data mHealth apps across different categories usually collect from end users and how these data are collected. This information is important to guide the development of a common data model from current widely adopted apps. This will also inform what built-in sensors and wearables, a comprehensive mHealth platform should support.

Methods

In our empirical investigation of mHealth apps, we identified app categories listed in a curated mHealth app library, which was then used to explore the Google Play Store for health and medical apps that were then filtered using our selection criteria. We downloaded these apps from a mirror site hosting Android apps and analyzed them using a script that we developed around the popular AndroGuard tool. We analyzed the use of Bluetooth peripherals and built-in sensors to understand how a given app collects health data.

Results

We retrieved 3251 apps meeting our criteria, and our analysis showed that 10.74% (349/3251) of these apps requested Bluetooth access. We found that 50.9% (259/509) of the Bluetooth service universally unique identifiers to be known in these apps, with the remainder being vendor specific. The most common health-related Bluetooth Low Energy services using known universally unique identifiers were Heart Rate, Glucose, and Body Composition. App permissions showed the most used device module or sensor to be the camera (669/3251, 20.57%), closely followed by location (598/3251, 18.39%), with the highest occurrence in the staying healthy app category.

Conclusions

We found that not many health apps used built-in sensors or peripherals for collecting health data. The small number of the apps using Bluetooth, with an even smaller number of apps using standard Bluetooth Low Energy services, indicates a wider use of proprietary algorithms and custom services, which restrict the device use. The use of standard profiles could open this ecosystem further and could provide end users more options for apps. The relatively small proportion of apps using built-in sensors along with a high reliance on manual data entry suggests the need for more research into using sensors for data collection in health and fitness apps, which may be more desirable and improve end user experience.

Education on Depression in Mental Health Apps: Systematic Assessment of Characteristics and Adherence to Evidence-Based Guidelines

BACKGROUND

Suboptimal understanding of depression and mental health disorders by the general population is an important contributor to the wide treatment gap in depression. Mental health literacy encompasses knowledge and beliefs about mental disorders and supports their recognition, management, and prevention. Besides knowledge improvement, psychoeducational interventions reduce symptoms of depression, enhance help-seeking behavior, and decrease stigma. Mental health apps often offer educational content, but the trustworthiness of the included information is unclear.

OBJECTIVE

The aim of this study is to systematically evaluate adherence to clinical guidelines on depression of the information offered by mental health apps available in major commercial app stores.

METHODS

A systematic assessment of the educational content regarding depression in the apps available in the Apple App Store and Google Play was conducted in July 2020. A systematic search for apps published or updated since January 2019 was performed using 42matters. Apps meeting the inclusion criteria were downloaded and assessed using two smartphones: an iPhone 7 (iOS version 14.0.1) and a Sony XPERIA XZs (Android version 8.0.0). The 156-question assessment checklist comprised general characteristics of apps, appraisal of 38 educational topics and their adherence to evidence-based clinical guidelines, as well as technical aspects and quality assurance. The results were tabulated and reported as a narrative review, using descriptive statistics.

RESULTS

The app search retrieved 2218 apps, of which 58 were included in the analysis (Android apps: n=29, 50%; iOS apps: n=29, 50%). Of the 58 included apps, 37 (64%) apps offered educational content within a more comprehensive depression or mental health management app. Moreover, 21% (12/58) of apps provided non-evidence-based information. Furthermore, 88% (51/58) of apps included up to 20 of the educational topics, the common ones being listing the symptoms of depression (52/58, 90%) and available treatments (48/58, 83%), particularly psychotherapy. Depression-associated stigma was mentioned by 38% (22/58) of the apps, whereas suicide risk was mentioned by 71% (41/58), generally as an item in a list of symptoms. Of the 58 included apps, 44 (76%) highlighted the importance of help seeking, 29 (50%) emphasized the importance of involving the user's support network. In addition, 52% (30/58) of apps referenced their content, and 17% (10/58) included advertisements.

CONCLUSIONS

Information in mental health and depression apps is often brief and incomplete, with 1 in 5 apps providing non-evidence-based information. Given the unmet needs and stigma associated with the disease, it is imperative that apps seize the opportunity to offer quality, evidence-based education or point the users to relevant resources. A multistakeholder consensus on a more stringent development and publication process for mental health apps is essential.

Evidence of User-Expert Gaps in Health App Ratings and Implications for Practice

Searching the commercial Google Play Store and App Store is one of the most common strategies for discovering mobile applications for digital health, both among consumers and healthcare professionals. However, several studies have suggested a possible mismatch between this strategy and the objective of finding apps in physical and mental health that are both clinically relevant and reliable from a privacy standpoint. This study provides direct evidence of a gap between the five-star user rating system and expert ratings from a curated library of over 1,200 apps that cover both physical and mental health. An objective metric is derived to assess the strength of the user-expert gap for each app, which in turn allows identifying missed opportunities—low user ratings and high expert ratings—and overrated apps—high user ratings and low expert ratings. Implications for practice and care delivery are discussed.

Characteristics of Mobile Health Platforms for Depression and Anxiety: Content Analysis Through a Systematic Review of the Literature and Systematic Search of Two App Stores

Background

Mobile health (mHealth) platforms show promise in the management of mental health conditions such as anxiety and depression. This has resulted in an abundance of mHealth platforms available for research or commercial use.

Objective

The objective of this review is to characterize the current state of mHealth platforms designed for anxiety or depression that are available for research, commercial use, or both.

Methods

A systematic review was conducted using a two-pronged approach: searching relevant literature with prespecified search terms to identify platforms in published research and simultaneously searching 2 major app stores—Google Play Store and Apple App Store—to identify commercially available platforms. Key characteristics of the mHealth platforms were synthesized, such as platform name, targeted condition, targeted group, purpose, technology type, intervention type, commercial availability, and regulatory information.

Results

The literature and app store searches yielded 169 and 179 mHealth platforms, respectively. Most platforms developed for research purposes were designed for depression (116/169, 68.6%), whereas the app store search reported a higher number of platforms developed for anxiety (Android: 58/179, 32.4%; iOS: 27/179, 15.1%). The most common purpose of platforms in both searches was treatment (literature search: 122/169, 72.2%; app store search: 129/179, 72.1%). With regard to the types of intervention, cognitive behavioral therapy and referral to care or counseling emerged as the most popular options offered by the platforms identified in the literature and app store searches, respectively. Most platforms from both searches did not have a specific target age group. In addition, most platforms found in app stores lacked clinical and real-world evidence, and a small number of platforms found in the published research were available commercially.

Conclusions

A considerable number of mHealth platforms designed for anxiety or depression are available for research, commercial use, or both. The characteristics of these mHealth platforms greatly vary. Future efforts should focus on assessing the quality—utility, safety, and effectiveness—of the existing platforms and providing developers, from both commercial and research sectors, a reporting guideline for their platform description and a regulatory framework to facilitate the development, validation, and deployment of effective mHealth platforms.

Towards Understanding the Usability Attributes of AI-Enabled eHealth Mobile Applications

Mobile application (app) use is increasingly becoming an essential part of our daily lives. Due to their significant usefulness, people rely on them to perform multiple tasks seamlessly in almost all aspects of everyday life. Similarly, there has been immense progress in artificial intelligence (AI) technology, especially deep learning, computer vision, natural language processing, and robotics. These technologies are now actively being implemented in smartphone apps and healthcare, providing multiple healthcare services. However, several factors affect the usefulness of mobile healthcare apps, and usability is an important one. There are various healthcare apps developed for each specific task, and the success of these apps depends on their performance. This study presents a systematic review of the existing apps and discusses their usability attributes. It highlights the usability models, outlines, and guidelines proposed in previous research for designing apps with improved usability characteristics. Thirty-nine research articles were reviewed and examined to identify the usability attributes, framework, and app design conducted. The results showed that satisfaction, efficiency, and learnability are the most important usability attributes to consider when designing eHealth mobile apps. Surprisingly, other significant attributes for healthcare apps, such as privacy and security, were not among the most indicated attributes in the studies.

Digital Technologies for Emotion-Regulation Assessment and Intervention: A Conceptual Review

The ability to regulate emotions in response to stress is central to healthy development. While early research in emotion regulation predominantly employed static, self-report measurement, the past decade has seen a shift in focus toward understanding the dynamic nature of regulation processes. This is reflected in recent refinements in the definition of emotion regulation, which emphasize the importance of the ability to flexibly adapt regulation efforts across contexts. The latest proliferation of digital technologies employed in mental health research offers the opportunity to capture the state- and context-sensitive nature of emotion regulation. In this conceptual review, we examine the use of digital technologies (ecological momentary assessment; wearable and smartphone technology, physical activity, acoustic data, visual data, and geo-location; smart home technology; virtual reality; social media) in the assessment of emotion regulation and describe their application to interventions. We also discuss challenges and ethical considerations, and outline areas for future research.

Applications mobiles pour soutenir la santé mentale des jeunes : opportunités et défis

Introduction L’adolescence est une période sensible pendant laquelle plusieurs changements ont le potentiel d’affecter le développement cognitif et psychosocial de l’individu. C’est aussi au cours de cette période que l’on note la plus forte augmentation des troubles de santé mentale. Plusieurs initiatives cherchent à prévenir l’émergence ou la détérioration de ces troubles auprès de jeunes en situation de vulnérabilité parce que vivant des difficultés émotionnelles ou du comportement, ou bien en raison d’expériences adverses auxquelles ils sont (ou ont été) confrontés. À ces efforts, le recours à la technologie est habituellement perçu comme naturel, voire souhaitable, auprès de ces « natifs du numérique ».

Objectif Cette recension critique propose d’examiner les avantages et inconvénients documentés dans la littérature et rencontrés dans le cadre du développement d’une application mobile (app) afin d’évaluer leur potentiel à soutenir la santé mentale et la résilience des jeunes en contextes d’adversité et les facteurs susceptibles d’amoindrir ces impacts.

Résultats/Constats Les apps, par leur format et mode d’utilisation, permettent de transmettre aux jeunes une information juste et fondée sur les meilleures connaissances tout en étant arrimée à leur réalité. Les apps représentent aussi une opportunité d’engager certains jeunes dans un processus de changement ou de les soutenir dans une éventuelle demande d’aide. Bien qu’elles soient disponibles en tout temps et qu’elles peuvent respecter leurs besoins d’autonomie et de confidentialité, de nombreux défis demeurent afin qu’elles puissent soutenir la santé mentale des jeunes, incluant le peu d’attention accordée à la validation scientifique, la protection des données personnelles et la capacité de rétention et d’engagement des jeunes.

Conclusion. Cette analyse critique des connaissances propose une réflexion qui pourrait s’avérer utile afin de soutenir le développement de futures apps rencontrant les meilleurs standards auprès d’une multitude d’acteurs susceptibles de les créer et de les utiliser.

Self-guided Cognitive Behavioral Therapy Apps for Depression: Systematic Assessment of Features, Functionality, and Congruence With Evidence

Background

Mental health disorders affect 1 in 10 people globally, of whom approximately 300 million are affected by depression. At least half of the people affected by depression remain untreated. Although cognitive behavioral therapy (CBT) is an effective treatment, access to mental health specialists, habitually challenging, has worsened because of the COVID-19 pandemic. Internet-based CBT is an effective and feasible strategy to increase access to treatment for people with depression. Mental health apps may further assist in facilitating self-management for people affected by depression; however, accessing the correct app may be cumbersome given the large number and wide variety of apps offered by public app marketplaces.

Objective

This study aims to systematically assess the features, functionality, data security, and congruence with evidence of self-guided CBT-based apps targeting users affected by depression that are available in major app stores.

Methods

We conducted a systematic assessment of self-guided CBT-based apps available in Google Play and the Apple App Store. Apps launched or updated since August 2018 were identified through a systematic search in the 42matters database using CBT-related terms. Apps meeting the inclusion criteria were downloaded and assessed using a Samsung Galaxy J7 Pro (Android 9) and iPhone 7 (iOS 13.3.1). Apps were appraised using a 182-question checklist developed by the research team, assessing their general characteristics, technical aspects and quality assurance, and CBT-related features, including 6 evidence-based CBT techniques (ie, psychoeducation, behavioral activation, cognitive restructuring, problem solving, relaxation, and exposure for comorbid anxiety) as informed by a CBT manual, CBT competence framework, and a literature review of internet-based CBT clinical trial protocols. The results were reported as a narrative review using descriptive statistics.

Results

The initial search yielded 3006 apps, of which 98 met the inclusion criteria and were systematically assessed. There were 20 well-being apps; 65 mental health apps, targeting two or more common mental health disorders, including depression; and 13 depression apps. A total of 28 apps offered at least four evidence-based CBT techniques, particularly depression apps. Cognitive restructuring was the most common technique, offered by 79% (77/98) of the apps. Only one-third of the apps offered suicide risk management resources, whereas 17% (17/98) of the apps offered COVID-19–related information. Although most apps included a privacy policy, only a third of the apps presented it before account creation. In total, 82% (74/90) of privacy policies stated sharing data with third-party service providers. Half of the app development teams included academic institutions or health care providers.

Conclusions

Only a few self-guided CBT-based apps offer comprehensive CBT programs or suicide risk management resources. Sharing of users’ data is widespread, highlighting shortcomings in health app market governance. To fulfill their potential, self-guided CBT-based apps should follow evidence-based clinical guidelines, be patient centered, and enhance users’ data security.

Psychiatric mental health nursing in the international year of the nurse and COVID-19: One hospital's perspective on resilience and innovation - Past, present and future

In the International Year of the Nurse, we are in the midst of a global pandemic. Historically, nurses have been at the forefront of crises influencing standards of care and public health policy. Managing psychiatric patients during the current COVID-19 pandemic has challenged the psychiatric nurses' role within the therapeutic milieu fostering innovative practices to meet patient needs. Our behavioral health center met the challenges with resilience, creativity and commitment. The next challenge is to learn from experience. Building on innovative technology opens the window to new models of care. Understanding resilience is critical to preparing for the next crisis.

Usability Testing of a Digital Assessment Routing Tool: Protocol for an Iterative Convergent Mixed Methods Study

BACKGROUND

Musculoskeletal conditions account for 16% of global disability, resulting in a negative effect on millions of patients and an increasing burden on health care utilization. Digital technologies that improve health care outcomes and efficiency are considered a priority; however, innovations are often inadequately developed and poorly adopted. Further, they are rarely tested with sufficient rigor in clinical trials-the gold standard for clinical proof of efficacy. We have developed a new musculoskeletal Digital Assessment Routing Tool (DART) that allows users to self-assess and be directed to the right care. DART requires usability testing in preparation for clinical trials.

OBJECTIVE

This study will use the iterative convergent mixed methods design to assess and mitigate all serious usability issues to optimize user experience and adoption. Using this methodology, we will provide justifiable confidence to progress to full-scale randomized controlled trials when DART is integrated into clinical management pathways. This study protocol will provide a blueprint for future usability studies of mobile health solutions.

METHODS

We will collect qualitative and quantitative data from 20-30 participants aged 18 years and older for 4 months. The exact number of participants recruited will be dependent on the number of iterative cycles required to reach the study end points. Building on previous internal testing and stakeholder involvement, quantitative data collection is defined by the constructs within the ISO 9241-210-2019 standard and the system usability scale, providing a usability score for DART. Guided by the participant responses to quantitative questioning, the researcher will focus the qualitative data collection on specific usability problems. These will then be graded to provide the rationale for further DART system improvements throughout the iterative cycles.

RESULTS

This study received approval from the Queen Mary University of London Ethics of Research Committee (QMREC2018/48/048) on June 4, 2020. At manuscript submission, study recruitment was on-going, with data collection to be completed and results published in 2021.

CONCLUSIONS

This study will provide evidence concerning mobile health DART system usability and acceptance determining system improvements required to support user adoption and minimize suboptimal system usability as a potential confounder within subsequent noninferiority clinical trials. Success should produce a safe effective system with excellent usability, facilitating quicker and easier patient access to appropriate care while reducing the burden on primary and secondary care musculoskeletal services. This deliberately rigorous approach to mobile health innovation could be used as a guide for other developers of similar apps.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/27205.

Scoping review: Development and assessment of evaluation frameworks of mobile health apps for recommendations to consumers

Objective

The study sought to review the different assessment items that have been used within existing health app evaluation frameworks aimed at individual, clinician, or organizational users, and to analyze the scoring and evaluation methods used in these frameworks.

Materials and Methods

We searched multiple bibliographic databases and conducted backward searches of reference lists, using search terms that were synonyms of “health apps,” “evaluation,” and “frameworks.” The review covered publications from 2011 to April 2020. Studies on health app evaluation frameworks and studies that elaborated on the scaling and scoring mechanisms applied in such frameworks were included.

Results

Ten common domains were identified across general health app evaluation frameworks. A list of 430 assessment criteria was compiled across 97 identified studies. The most frequently used scaling mechanism was a 5-point Likert scale. Most studies have adopted summary statistics to generate the total scoring of each app, and the most popular approach taken was the calculation of mean or average scores. Other frameworks did not use any scaling or scoring mechanism and adopted criteria-based, pictorial, or descriptive approaches, or “threshold” filter.

Discussion

There is wide variance in the approaches to evaluating health apps within published frameworks, and this variance leads to ongoing uncertainty in how to evaluate health apps.

Conclusions

A new evaluation framework is needed that can integrate the full range of evaluative criteria within one structure, and provide summative guidance on health app rating, to support individual app users, clinicians, and health organizations in choosing or recommending the best health app.

Assessing the quality of mobile applications in chronic disease management: a scoping review

While there has been a rapid growth of digital health apps to support chronic diseases, clear standards on how to best evaluate the quality of these evolving tools are absent. This scoping review aims to synthesize the emerging field of mobile health app quality assessment by reviewing criteria used by previous studies to assess the quality of mobile apps for chronic disease management. A literature review was conducted in September 2017 for published studies that use a set of quality criteria to directly evaluate two or more patient-facing apps supporting promote chronic disease management. This resulted in 8182 citations which were reviewed by research team members, resulting in 65 articles for inclusion. An inductive coding schema to synthesize the quality criteria utilized by included articles was developed, with 40 unique quality criteria identified. Of the 43 (66%) articles that reported resources used to support criteria selection, 19 (29%) used clinical guidelines, and 10 (15%) used behavior change theory. The most commonly used criteria included the presence of user engagement or behavior change functions (97%, n = 63) and technical features of the app such as customizability (20%, n = 13, while Usability was assessed by 24 studies (36.9%). This study highlights the significant variation in quality criteria employed for the assessment of mobile health apps. Future methods for app evaluation will benefit from approaches that leverage the best evidence regarding the clinical impact and behavior change mechanisms while more directly reflecting patient needs when evaluating the quality of apps.

Quantity and type of peer-reviewed evidence for popular free medical apps: Cross-sectional review

INTRODUCTION

- Mobile apps are being increasingly used as a tool to deliver clinical care. Evidence of efficacy for such apps varies, and appropriate levels of evidence may depend on the app's intended use. The UK's National Institute for Health and Care Excellence (NICE) recently developed an evidence standards framework, aiming to explicitly set out the required standards of evidence for different categories of digital health technologies. To determine current compliance with the evidence standards framework, the current study quantified the amount and type of peer-reviewed evidence associated with a cross-section of popular medical apps.

METHODS

- Apps were identified by selecting the top 100 free medical apps in the Apple App Store and all free apps in the NHS Apps Library. Each app was assigned to one of the four tiers (1, 2, 3a, 3b) in the NICE evidence standards framework. For each app, we conducted searches in Ovid-MEDLINE, Web of Science, Google Scholar, and via manufacturer websites to identify any published articles that assessed the app. This allowed us to determine our primary outcome, whether apps in tiers 3a/3b were more likely than apps in tier 1/2 to be associated with academic peer-reviewed evidence.

RESULTS

- We reviewed 125 apps in total (Apple App Store (n = 72), NHS Apps Library (n = 45), both (n = 8), of which 54 were categorized into the higher evidence standards framework tiers, 3a/3b. After screening, we extracted 105 relevant articles which were associated with 25 of the apps. Only 6 articles, pertaining to 3 apps, were reports of randomised controlled trials. Apps in tiers 3a/3b were more likely to be associated with articles than apps in lower tiers (χ² = 5.54, p = .01). The percentage of tier 3a/3b apps with associated articles was similar for both the NHS Apps Library (10/28) and Apple App store (7/24), (χ² = 0.042, p = .84).

DISCUSSION

- Apps that were in higher tiers 3a and 3b, indicating higher clinical risk, were more likely to have an associated article than those in lower categories. However, even in these tiers, supporting peer-reviewed evidence was missing in the majority of instances. In our sample, Apps from the NHS Apps Library were more no more likely to have supporting evidence than popular Apple App Store apps. This is of concern, given that NHS approval may influence uptake of app usage.

Digital health developments and drawbacks: a review and analysis of top-returned apps for bipolar disorder

BACKGROUND

Although a growing body of literature highlights the potential benefit of smartphone-based mobile apps to aid in self-management and treatment of bipolar disorder, it is unclear whether such evidence-based apps are readily available and accessible to a user of the app store.

RESULTS

Using our systematic framework for the evaluation of mental health apps, we analyzed the accessibility, privacy, clinical foundation, features, and interoperability of the top-returned 100 apps for bipolar disorder. Only 56% of the apps mentioned bipolar disorder specifically in their title, description, or content. Only one app's efficacy was supported in a peer-reviewed study, and 32 apps lacked privacy policies. The most common features provided were mood tracking, journaling, and psychoeducation.

CONCLUSIONS

Our analysis reveals substantial limitations in the current digital environment for individuals seeking an evidence-based, clinically usable app for bipolar disorder. Although there have been academic advances in development of digital interventions for bipolar disorder, this work has yet to be translated to the publicly available app marketplace. This unmet need of digital mood management underscores the need for a comprehensive evaluation system of mental health apps, which we have endeavored to provide through our framework and accompanying database (apps.digitalpsych.org).

Evaluation of Treatment Descriptions and Alignment With Clinical Guidance of Apps for Depression on App Stores: Systematic Search and Content Analysis

BACKGROUND

The use of apps for the treatment of depression shows great promise. However, there is uncertainty regarding the alignment of publicly available apps for depression with clinical guidance, their treatment fidelity and evidence base, and their overall safety.

OBJECTIVE

Built on previous analyses and reviews, this study aims to explore the treatment and safety issues of publicly available apps for depression.

METHODS

We conducted a content analysis of apps for depression in the 2 main UK app stores (Google Play and Apple App Store). App store listings were analyzed for intervention content, treatment fidelity, and fit with the National Institute for Health and Care Excellence (NICE) guidelines for the treatment of depression in adults.

RESULTS

A total of 353 apps for depression were included in the review. App descriptions reported the use of 20 treatment approaches and 37 treatment strategies. Many apps used transdiagnostic (155/353, 43.9%) and multitheoretical interventions to treat multiple disorders including depression. Although many interventions appeared to be evidence-informed, there were issues with treatment fidelity, research evidence, and fit with clinical guidelines. None of the apps fully aligned with the NICE guidelines for depression.

CONCLUSIONS

App developers have adopted many evidence-informed treatments in their interventions; however, more work is needed to improve clinical validity, treatment fidelity, and the safety of apps. We urge developers to consult relevant guidelines and standards, and to engage in reflective questioning on treatment and safety to address these issues and to improve treatment content and intervention design.

Review of Popularity and Quality Standards of Opioid-Related Smartphone Apps

Purpose of review

Opioid misuse, addiction, and related harm is a global crisis that affects public health and social and economic welfare. Many of the strategies being used to combat the opioid crisis could benefit from improved access and dissemination, such as that afforded by smartphone apps. The goal of this study was to characterize the purpose, audience, quality and popularity of opioid-related smartphone apps. Using web scraping, available information from 619 opioid-related apps (e.g., popularity metrics) was downloaded from Google Play, and 59 apps met criteria for review. The apps were additionally coded for quality by two raters using an 8-item screener for the American Psychiatric Association App Evaluation Model.

Findings

Sixty one percent of apps targeted patients, 29% providers, 8% the general community, and 2% healthcare trainees. Regarding app purpose, 49% addressed treatment, 27% prevention, and 24% overdose. Only one app met all criteria on the screener for quality, and there was no association between a total score calculated for the screener and measures of app popularity (e.g., star ratings; R²=0.10, p=0.19).

Summary

Opioid-related apps available for consumers addressed key stakeholders (patients, providers, community) and were consistent with strategies to address the opioid crisis (prevention, treatment, overdose). However, there was little evidence that available opioid-related apps meet basic quality standards, and no relationship was found between app quality and popularity. This review was conducted at the level of consumer decision-making (i.e., the app store), where only a handful of opioid-related apps met quality standards enough to warrant a more detailed evaluation of the app before recommendation for use. Because smartphone apps could be a critical tool to increase access to and utilization of opioid prevention, treatment, and recovery services, further development and testing is sorely needed.

The Internet of Things: Impact and Implications for Health Care Delivery

The Internet of Things (IoT) is a system of wireless, interrelated, and connected digital devices that can collect, send, and store data over a network without requiring human-to-human or human-to-computer interaction. The IoT promises many benefits to streamlining and enhancing health care delivery to proactively predict health issues and diagnose, treat, and monitor patients both in and out of the hospital. Worldwide, government leaders and decision makers are implementing policies to deliver health care services using technology and more so in response to the novel COVID-19 pandemic. It is now becoming increasingly important to understand how established and emerging IoT technologies can support health systems to deliver safe and effective care. The aim of this viewpoint paper is to provide an overview of the current IoT technology in health care, outline how IoT devices are improving health service delivery, and outline how IoT technology can affect and disrupt global health care in the next decade. The potential of IoT-based health care is expanded upon to theorize how IoT can improve the accessibility of preventative public health services and transition our current secondary and tertiary health care to be a more proactive, continuous, and coordinated system. Finally, this paper will deal with the potential issues that IoT-based health care generates, barriers to market adoption from health care professionals and patients alike, confidence and acceptability, privacy and security, interoperability, standardization and remuneration, data storage, and control and ownership. Corresponding enablers of IoT in current health care will rely on policy support, cybersecurity-focused guidelines, careful strategic planning, and transparent policies within health care organizations. IoT-based health care has great potential to improve the efficiency of the health system and improve population health.

Smartphone Apps for College Mental Health: A Concern for Privacy and Quality of Current Offerings

OBJECTIVE

The expanding need for mental health services on college campuses has increased interest in smartphone apps for mental health. Although apps can be effective tools, they can also present risks regarding privacy and lack of efficacy, and little is known about the nature or quality of mental health apps that colleges are recommending to students. This study sought to fill that knowledge gap.

METHODS

Sixty college counseling center Web sites were examined for suggested mobile apps offered as resources for students. The features of each app were assessed, including date of last update, privacy policy, and whether any research had been published in peer-reviewed journals.

RESULTS

Twenty-six college counseling centers suggested a total of 218 unique apps. Of these, 28% were no longer available for download. Of the 158 remaining apps, only 44% had been updated in the past 6 months, and 39% had no privacy policy. Of the 97 existing apps with privacy policies, 88% collected user's data and 49% shared users' data with third parties. Efficacy studies had been published in peer-reviewed journals for only 16% of existing apps.

CONCLUSIONS

College counseling centers are not suggesting safe or up-to-date mobile apps. There is an urgent need to help centers identify, curate, and recommend more appropriate apps. The authors suggest that centers use the American Psychiatric Association's app evaluation framework; develop an app review process, with input from clinicians and students; or employ a digital navigator to select apps and help connect students to the appropriate app.

Assessing the Food and Drug Administration's Risk-Based Framework for Software Precertification With Top Health Apps in the United States: Quality Improvement Study

Background

As the development of mobile health apps continues to accelerate, the need to implement a framework that can standardize the categorization of these apps to allow for efficient yet robust regulation is growing. However, regulators and researchers are faced with numerous challenges, as apps have a wide variety of features, constant updates, and fluid use cases for consumers. As past regulatory efforts have failed to match the rapid innovation of these apps, the US Food and Drug Administration (FDA) has proposed that the Software Precertification (Pre-Cert) Program and a new risk-based framework could be the solution.

Objective

This study aims to determine whether the risk-based framework proposed by the FDA’s Pre-Cert Program could standardize categorization of top health apps in the United States.

Methods

In this quality improvement study during summer 2019, the top 10 apps for 6 disease conditions (addiction, anxiety, depression, diabetes, high blood pressure, and schizophrenia) in Apple iTunes and Android Google Play Store in the United States were classified using the FDA’s risk-based framework. Data on the presence of well-defined app features, user engagement methods, popularity metrics, medical claims, and scientific backing were collected.

Results

The FDA’s risk-based framework classifies an app’s risk by the disease condition it targets and what information that app provides. Of the 120 apps tested, 95 apps were categorized as targeting a nonserious health condition, whereas only 7 were categorized as targeting a serious condition and 18 were categorized as targeting a critical condition. As the majority of apps targeted a nonserious condition, their risk categorization was largely determined by the information they provided. The apps that were assessed as not requiring FDA review were more likely to be associated with the integration of external devices than those assessed as requiring FDA review (15/58, 26% vs 5/62, 8%; P=.03) and health information collection (24/58, 41% vs 9/62, 15%; P=.008). Apps exempt from the review were less likely to offer health information (25/58, 43% vs 45/62, 72%; P<.001), to connect users with professional care (7/58, 12% vs 14/62, 23%; P=.04), and to include an intervention (8/58, 14% vs 35/62, 55%; P<.001).

Conclusions

The FDA’s risk-based framework has the potential to improve the efficiency of the regulatory review process for health apps. However, we were unable to identify a standard measure that differentiated apps requiring regulatory review from those that would not. Apps exempt from the review also carried concerns regarding privacy and data security. Before the framework is used to assess the need for a formal review of digital health tools, further research and regulatory guidance are needed to ensure that the Pre-Cert Program operates in the greatest interest of public health.

User Perspectives of Mood-Monitoring Apps Available to Young People: Qualitative Content Analysis

BACKGROUND

Mobile health apps are increasingly available and used in a clinical context to monitor young people's mood and mental health. Despite the benefits of accessibility and cost-effectiveness, consumer engagement remains a hurdle for uptake and continued use. Hundreds of mood-monitoring apps are publicly available to young people on app stores; however, few studies have examined consumer perspectives. App store reviews held on Google and Apple platforms provide a large, rich source of naturally generated, publicly available user reviews. Although commercial developers use these data to modify and improve their apps, to date, there has been very little in-depth evaluation of app store user reviews within scientific research, and our current understanding of what makes apps engaging and valuable to young people is limited.

OBJECTIVE

This study aims to gain a better understanding of what app users consider useful to encourage frequent and prolonged use of mood-monitoring apps appropriate for young people.

METHODS

A systematic approach was applied to the selection of apps and reviews. We identified mood-monitoring apps (n=53) by a combination of automated application programming interface (API) methods. We only included apps appropriate for young people based on app store age categories (apps available to those younger than 18 years). We subsequently downloaded all available user reviews via API data scraping methods and selected a representative subsample of reviews (n=1803) for manual qualitative content analysis.

RESULTS

The qualitative content analysis revealed 8 main themes: accessibility (34%), flexibility (21%), recording and representation of mood (18%), user requests (17%), reflecting on mood (16%), technical features (16%), design (13%), and health promotion (11%). A total of 6 minor themes were also identified: notification and reminders; recommendation; privacy, security, and transparency; developer; adverts; and social/community.

CONCLUSIONS

Users value mood-monitoring apps that can be personalized to their needs, have a simple and intuitive design, and allow accurate representation and review of complex and fluctuating moods. App store reviews are a valuable repository of user engagement feedback and provide a wealth of information about what users value in an app and what user needs are not being met. Users perceive mood-monitoring apps positively, but over 20% of reviews identified the need for improvement.

Technology Evaluation and Assessment Criteria for Health Apps (TEACH-Apps): Pilot Study

BACKGROUND

Despite the emergence of app evaluation tools, there remains no well-defined process receptive to diverse local needs, rigorous standards, and current content. The need for such a process to assist in the implementation of app evaluation across all medical fields is evident. Such a process has the potential to increase stakeholder engagement and catalyze interest and engagement with present-day app evaluation models.

OBJECTIVE

This study aimed to develop and pilot test the Technology Evaluation and Assessment Criteria for Health apps (TEACH-apps).

METHODS

Tailoring a well-known implementation framework, Replicating Effective Programs, we present a new process to approach the challenges faced in implementing app evaluation tools today. As a culmination of our experience implementing this process and feedback from stakeholders, we present the four-part process to aid the implementation of mobile health technology. This paper outlines the theory, evidence, and initial versions of the process.

RESULTS

The TEACH-apps process is designed to be broadly usable and widely applicable across all fields of health. The process comprises four parts: (1) preconditions (eg, gathering apps and considering local needs), (2) preimplementation (eg, customizing criteria and offering digital skills training), (3) implementation (eg, evaluating apps and creating educational handouts), and (4) maintenance and evolution (eg, repeating the process every 90 days and updating content). TEACH-apps has been tested internally at our hospital, and there is growing interest in partnering health care facilities to test the system at their sites.

CONCLUSIONS

This implementation framework introduces a process that equips stakeholders, clinicians, and users with the foundational tools to make informed decisions around app use and increase app evaluation engagement. The application of this process may lead to the selection of more culturally appropriate and clinically relevant tools in health care.

Actionable health app evaluation: translating expert frameworks into objective metrics

As use and availability of mobile health apps have increased, so too has the need for a thorough, accessible framework for app evaluation. The American Psychiatric Association's app evaluation model has emerged as a way to critically assess an app by considering accessibility, privacy and security, clinical foundation, engagement, and interoperability; however, there is no centralized database where users can view how various health apps perform when assessed via the APA model. In this perspective, we propose and outline our effort to translate the APA's model for the evaluation of health apps into a set of objective metrics that can be published online, making the framework actionable and accessible to a broad audience. The questions from the APA model were operationalized into 105 objective questions that are either binary or numeric. These questions serve as the foundation of an online database, where app evaluation consists of answering these 105 questions and can be crowdsourced. While the database has yet to be published and crowdsourced, initial internal testing demonstrated excellent interrater reliability. The database proposed here introduces a public and interactive approach to data collection that is guided by the APA model. The published product enables users to sort through the many mobile health apps and filter them according to individual preferences and priorities, making the ever-growing health app market more navigable.

An Integrated Blueprint for Digital Mental Health Services Amidst COVID-19

In-person traditional approaches to mental health care services are facing difficulties amidst the coronavirus disease (COVID-19) crisis. The recent implementation of social distancing has redirected attention to nontraditional mental health care delivery to overcome hindrances to essential services. Telehealth has been established for several decades but has only been able to play a small role in health service delivery. Mobile and teledigital health solutions for mental health are well poised to respond to the upsurge in COVID-19 cases. Screening and tracking with real-time automation and machine learning are useful for both assisting psychological first-aid resources and targeting interventions. However, rigorous evaluation of these new opportunities is needed in terms of quality of interventions, effectiveness, and confidentiality. Service delivery could be broadened to include trained, unlicensed professionals, who may help health care services in delivering evidence-based strategies. Digital mental health services emerged during the pandemic as complementary ways of assisting community members with stress and transitioning to new ways of living and working. As part of a hybrid model of care, technologies (mobile and online platforms) require consolidated and consistent guidelines as well as consensus, expert, and position statements on the screening and tracking (with real-time automation and machine learning) of mental health in general populations as well as considerations and initiatives for underserved and vulnerable subpopulations.

The Model of Gamification Principles for Digital Health Interventions: Evaluation of Validity and Potential Utility

BACKGROUND

Although gamification continues to be a popular approach to increase engagement, motivation, and adherence to behavioral interventions, empirical studies have rarely focused on this topic. There is a need to empirically evaluate gamification models to increase the understanding of how to integrate gamification into interventions.

OBJECTIVE

The model of gamification principles for digital health interventions proposes a set of five independent yet interrelated gamification principles. This study aimed to examine the validity and reliability of this model to inform its use in Web- and mobile-based apps.

METHODS

A total of 17 digital health interventions were selected from a curated website of mobile- and Web-based apps (PsyberGuide), which makes independent and unbiased ratings on various metrics. A total of 133 independent raters trained in gamification evaluation techniques were instructed to evaluate the apps and rate the degree to which gamification principles are present. Multiple ratings (n≥20) were collected for each of the five gamification principles within each app. Existing measures, including the PsyberGuide credibility score, mobile app rating scale (MARS), and the app store rating of each app were collected, and their relationship with the gamification principle scores was investigated.

RESULTS

Apps varied widely in the degree of gamification implemented (ie, the mean gamification rating ranged from 0.17≤m≤4.65 out of 5). Inter-rater reliability of gamification scores for each app was acceptable (κ≥0.5). There was no significant correlation between any of the five gamification principles and the PsyberGuide credibility score (P≥.49 in all cases). Three gamification principles (supporting player archetypes, feedback, and visibility) were significantly correlated with the MARS score, whereas three principles (meaningful purpose, meaningful choice, and supporting player archetypes) were significantly correlated with the app store rating. One gamification principle was statistically significant with both the MARS and the app store rating (supporting player archetypes).

CONCLUSIONS

Overall, the results support the validity and potential utility of the model of gamification principles for digital health interventions. As expected, there was some overlap between several gamification principles and existing app measures (eg, MARS). However, the results indicate that the gamification principles are not redundant with existing measures and highlight the potential utility of a 5-factor gamification model structure in digital behavioral health interventions. These gamification principles may be used to improve user experience and enhance engagement with digital health programs.