Influences on the Uptake of Health and Well-being Apps and Curated App Portals: Think-Aloud and Interview Study

Web-Based Tool Designed to Encourage Supplemental Nutrition Assistance Program Use in Urban College Students: Usability Testing Study

BACKGROUND

Food insecurity continues to be a risk for college students in the United States. It is associated with numerous problems, such as chronic health conditions, increased stress and anxiety, and a lower grade point average. After COVID-19, the Supplemental Nutrition Assistance Program (SNAP) benefits were extended to college-aged students; however, there were some barriers to participation, which persisted such as lack of perceived food insecurity risk, lack of knowledge regarding the SNAP application process, the complexity of determining eligibility, and stigma associated with needing social assistance. A technology-enhanced tool was developed to address these barriers to SNAP enrollment and encourage at-risk college students to apply for SNAP.

OBJECTIVE

The purpose of this study was to test the usability and acceptability of a web-based SNAP screening tool designed for college-aged students.

METHODS

College students aged 18-25 years were recruited to participate in 2 rounds of usability testing during fall 2022. Participants tested the prototype of a web-based SNAP screener tool using a standardized think-aloud method. The usability and acceptability of the tool were assessed using a semistructured interview and a 10-item validated System Usability Scale questionnaire. Audio recordings and field notes were systematically reviewed by extracting and sorting feedback as positive or negative comments. System Usability Scale questionnaire data were analyzed using the Wilcoxon signed rank test and sign test.

RESULTS

A total of 12 students (mean age 21.8, SD 2.8 years; n=6, 50% undergraduate; n=11, 92% female; n=7, 58% Hispanic or Black or African American; n=9, 78% low or very low food security) participated in both rounds of user testing. Round 1 testing highlighted overall positive experiences with the tool, with most participants (10/12) stating that the website fulfills its primary objective as a support tool to encourage college students to apply for SNAP. However, issues related to user interface design, navigation, and wording of some questions in the screening tool were noted. Key changes after round 1 reflected these concerns, including improved design of response buttons and tool logo and improved clarity of screening questions. The overall system usability showed slight, but not statistically significant, improvement between round 1 and round 2 (91.25 vs 92.50; P=.10, respectively).

CONCLUSIONS

Overall usability findings suggest that this web-based tool was highly usable and acceptable to urban college students and could be an effective and appealing approach as a support tool to introduce college students to the SNAP application process. The findings from this study will inform further development of the tool, which could eventually be disseminated publicly among various college campuses.

Feasibility and Acceptability of a Health App Platform Providing Individuals With a Budget to Purchase Preselected Apps to Work on Their Health and Well-Being: Quantitative Evaluation Study

BACKGROUND

The potential of health apps for health promotion and disease prevention is widely recognized. However, uptake is limited due to barriers individuals face in finding suitable and trustworthy apps, such as the overwhelming amount of available health apps. Therefore, the health app platform "FitKnip" was developed, enabling individuals to purchase preselected, trustworthy health apps with a budget of 100 euros (a currency exchange rate of EUR €1=US $1.0831 is applicable). The platform aimed to empower individuals to improve their health and vitality, ultimately supporting a more healthy society.

OBJECTIVE

The primary aim of this study was to evaluate the health app platform in terms of feasibility and acceptability. Potential effects on health empowerment and health outcomes were secondarily explored.

METHODS

This quantitative study was part of a mixed methods study with a prospective pre-post interventional design. We collected web-based user data, and self-reported web-based questionnaires were collected over 5 measurements over an 8-month period. Use statistics were tracked on the platform, including the number of purchased apps and euros spent per user registered within the health app platform. We measured the user-friendliness of the health app platform using the System Usability Scale (SUS) and satisfaction using the Client Satisfaction Questionnaire-8 (CSQ-8) and several 10-point Likert items. We asked participants to indicate, on a scale from 1 (not at all) to 10 (completely), how much the health app platform contributed to various areas related to health empowerment. We assessed health-related quality of life by the 12-item Short-Form Health Survey (SF-12) and one's perceived level of stress by the 10-item Perceived Stress Scale (PSS-10).

RESULTS

A total of 1650 participants were included, of whom 42% (685/1650) bought at least 1 app. The majority of those purchased one app (244/685, 35.6%). The health app platform was rated as user-friendly (SUS mean 66.5, SD 20.7; range 66.5-70.0), and the acceptability of the health app platform was moderate (CSQ-8 mean 20.0, SD 1.5; range 19.6-20.0). Results furthermore showed that participants were generally satisfied to highly satisfied with the ease of the payment system to purchase apps on the platform (median 8, IQR 7-10), the look and feel of the platform (median 7, IQR 6-8), as well as the provided budget of 100 euros (median 9, IQR 7-10). Participants were less satisfied with the amount (median 6, IQR 4-7) and diversity (median 6, IQR 4-7) of apps offered on the platform.

CONCLUSIONS

A health app platform is a promising initiative to enhance public health. Feasibility and acceptability are critical for success, as they ensure that such a platform is accessible, user-friendly, and meets end users' needs and preferences. This can help to increase uptake, engagement, and ultimately the platform's adoption and effectiveness.

A Health App Platform Providing a Budget to Purchase Preselected Apps as an Innovative Way to Support Public Health: Qualitative Study With End Users and Other Stakeholders

BACKGROUND

eHealth has the potential to improve health outcomes. However, this potential is largely untapped. Individuals face an overload of apps and have difficulties choosing suitable apps for themselves. In the FitKnip experiment, individuals were given access to a health app platform, where they could purchase reliable preselected health apps with a personal budget of €100 (US $107.35). By conducting a prospective study, we aimed to scientifically evaluate the FitKnip experiment as an innovative way to improve population health.

OBJECTIVE

The aim of the experiment was to scientifically evaluate the FitKnip experiment as an innovative way to improve population health. More specifically, we conducted an in-depth qualitative evaluation of the concept and acceptability of FitKnip, its perceived impact on health empowerment, as well as the roles of stakeholders for the future implementation of a health app platform through focus group interviews.

METHODS

This study followed a phenomenological research design and included 7 focus group interviews with end users and 1 with stakeholders, held between July and December 2020. End users were recruited through various institutions in the Netherlands, for example, insurance companies and local governments. All focus groups were semistructured using interview guides and were held via videoconferencing due to the COVID-19 pandemic measures. Each participant received access to a health app platform where they were enabled to purchase reliable, preselected health apps with a budget of €100 (US $107.35). The budget was valid for the entire research period. The health app platform offered 38 apps. A third party, a health care coalition, selected the apps to be included in FitKnip. The analyses were conducted according to the principles of the Framework Method.

RESULTS

A priori formulated themes were concept, acceptability, health empowerment, and outcomes, and the roles of stakeholders for the future implementation of a health app platform. Both end users (n=31) and stakeholders (n=5) were enthusiastic about the concept of a health app platform. End users indicated missing apps regarding physical health and lifestyle and needing more guidance toward suitable apps. End users saw health empowerment as a precondition to using a health app platform and achieving health outcomes depending on the purchased mobile apps. End users and stakeholders identified potential providers and financing parties of FitKnip. Stakeholders recommended the establishment of a reputable national or international quality guidelines or certification for health and wellbeing apps, that can demonstrate the quality and reliability of mobile health applications.

CONCLUSIONS

This study showed the need for a personalized and flexible platform. Next to this, a deeper understanding of the roles of stakeholders in such initiatives is needed especially on financing and reimbursement of health promotion and digital health services. A personalized, flexible health app platform is a promising initiative to support individuals in their health.

Exploring how informed mental health app selection may impact user engagement and satisfaction

The prevalence of mental health app use by people suffering from mental health disorders is rapidly growing. The integration of mental health apps shows promise in increasing the accessibility and quality of treatment. However, a lack of continued engagement is one of the significant challenges of such implementation. In response, the M-health Index and Navigation Database (MIND)- derived from the American Psychiatric Association's app evaluation framework- was created to support patient autonomy and enhance engagement. This study aimed to identify factors influencing engagement with mental health apps and explore how MIND may affect user engagement around selected apps. We conducted a longitudinal online survey over six weeks after participants were instructed to find mental health apps using MIND. The survey included demographic information, technology usage, access to healthcare, app selection information, System Usability Scale, the Digital Working Alliance Inventory, and the General Self-Efficacy Scale questions. Quantitative analysis was performed to analyze the data. A total of 321 surveys were completed (178 at the initial, 90 at the 2-week mark, and 53 at the 6-week mark). The most influential factors when choosing mental health apps included cost (76%), condition supported by the app (59%), and app features offered (51%), while privacy and clinical foundation to support app claims were among the least selected filters. The top ten apps selected by participants were analyzed for engagement. Rates of engagement among the top-ten apps decreased by 43% from the initial to week two and 22% from week two to week six on average. In the context of overall low engagement with mental health apps, implementation of mental health app databases like MIND can play an essential role in maintaining higher engagement and satisfaction. Together, this study offers early data on how educational approaches like MIND may help bolster mental health apps engagement.

Assessing the Dynamics of the Mental Health Apple and Android App Marketplaces

Barriers to mental health care, including stigma, costs, and mental health professional shortages, have been exacerbated by the COVID-19 pandemic. Smartphone apps have the potential to increase scalability and improve access to mental health information, support, and interventions. However, evaluating these apps and selecting ones for use in care remain challenging, especially as apps are often updating and changing. Recommending apps requires knowledge of how stable apps are as the experience of one user several months ago may or may not be the same. A sample of 347 apps of the 650 apps on the M-health Index and Navigation Database (MIND) https://mindapps.org were reviewed between September 1, 2021, and January 5, 2022. Apps were selected by time since their last review, with updates occurring on average approximately 4 months from the last review. Eleven trained app evaluators reviewed apps across 105 evaluation criteria in 9 categories. Results were compared to initial ratings, identifying the changes that occurred. The average app updates every 433 days, though 19% were updated in the last 3 months and some nearly weekly. Changes in privacy and features made up the highest percentage of changes, both at 38%. The most frequently observed privacy-related change was increased privacy policy reading level. Functionality parameters changed in 28% of apps. The most common functionality change was the removal of an accessibility feature. Clinical foundations changed in 18% of apps and 9% added supporting studies. Cost structure changed in 17% of apps, with 10% adding a fee for use of the app. Engagement features changed in 17% of the apps, with additions and removals of validated assessments or screeners most common. The dynamic nature of the app stores is reflected in app privacy, features, and functionality. These changes, reflected by the increased reading levels required to understand privacy policies, the decrease in accessibility features, and the additions of fees to access mobile apps, reflect the need to constantly review apps and understand how they are evolving. Patient and clinicians should use the most recent and updated possible when evaluating apps.

Current challenges for evaluating mobile health applications

As mobile health applications continue to proliferate without clear regulation, the need for app evaluation frameworks to offer guidance to patients and clinicians also expands. However, this expanding number of app evaluation frameworks itself can be a source of confusion and often contradictory recommendations. In pursuit of better frameworks that offer innovation for app evaluation, we present 4 challenges that app evaluation frameworks must overcome as well as examples from our own experience toward overcoming them. The recommendations are applicable to all health apps from any field of medicine, although we use examples from mental health as they are illustrative.

Impact of social influence on users' continuance intention toward sports and fitness applications

The purpose of this paper is to explore how social influence (SI), which is disaggregated into subjective norms (SN), social image (SIM), and social identity (SID), predicts perceived usefulness (PU), perceived pleasure (PP), and continuance intention (CI) toward sports and fitness applications. The underlying context is the socialization and gamification of exercise during the Covid-19 pandemic. Based on the theory of SI and the technology acceptance model, a theoretical framework was built where PU and PP mediate the influence of SI on CI, and proposed hypotheses were tested. The responses of 296 Keep users (a popular sports and fitness application in China) to a questionnaire survey were analyzed. SN and SIM were found to have significant positive effects on SID; SID has significant positive effects on PU and PP; both PU and PP have significant positive effects on the CI of users; SID and PU positively and significantly mediate the relationship between SN/SIM and CI; PU positively and significantly mediates the SID-CI relationship. However, the role of PP in mediating the influence of SI on CI is non-significant. This paper deepens the current understanding of the mechanisms that influence the relationship between SI and CI under the context of socialization and gamification services.

"I feel like it is asking if he is a stalker … but I also feel like it is asking if he cares": exploring young South African women and men's perceptions of the Sexual Relationship Power Scale

Background

Gender inequity and the subsequent health impacts disproportionately affect communities in the Global South. However, most gender equity measures, such as Pulerwitz’ (2000) Sexual Relationship Power Scale (SRPS), are developed and validated in the Global North and then applied in Global South settings without investigation of context applicability or validity. This study examines the SRPS’ validity evidence, comprehensiveness, and contemporary relevance for young South African women and men.

Methods

Between 2019 and 2021, 38 cognitive interviews (CIs) were conducted among previous participants of a South African youth cohort study ‘AYAZAZI’ (2015–2017) to explore youth’s perceptions of the SRPS. The SRPS measures women’s perceptions of their partner’s controlling behaviours, and men’s perceptions of their own controlling behaviours. Using CIs, participants responded to a 13-item adaptation of the SRPS for use among South African youth (strongly agree-strongly disagree), and then were asked to think-aloud their reasoning for responses, their understanding and perceived relevance of each item, and made overall suggestions for scale adaptations. An item appraisal coding process was applied, whereby Cognitive Coding assessed the types of cognitive problems youth had with understanding the items, and Question Feature Coding assessed which item features caused problems for participant understandings. Finally, youth recommendations for scale adaptations were summarized.

Results

Overall, 21 women and 17 men aged 21–30 participated in CIs in Durban and Soweto, South Africa. Cognitive Coding revealed 1. Comprehension issues, and 2. Judgements related to items’ applicability to lived experiences and identities (e.g., being unmarried). Question Feature Coding revealed items’ 1. Lack of clarity or vagueness in wording and 2. Logical problems in assumptions leading to multiple interpretations (e.g., item ‘my partner always need to know where I am’ interpreted as both controlling and caring behaviour). Multiple, overlapping issues revealed how many items failed to “fit” within the present-day living realities of South African youth. Youth recommended several item adaptations and additions, including strength-based items, to existing measures of gender equity and relationship power.

Conclusion

Given identified issues, several adaptations including revising items to be more inclusive, contemporary, context specific, relational, and strength-based are needed to validly measure gender equity and power dynamics within the relationships of South African youth.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12889-022-13686-9.

Perceptions of Factors Influencing Engagement With Health and Well-being Apps in the United Kingdom: Qualitative Interview Study

BACKGROUND

Digital health devices, such as health and well-being smartphone apps, could offer an accessible and cost-effective way to deliver health and well-being interventions. A key component of the effectiveness of health and well-being apps is user engagement. However, engagement with health and well-being apps is typically poor. Previous studies have identified a list of factors that could influence engagement; however, most of these studies were conducted on a particular population or for an app targeting a particular behavior. An understanding of the factors that influence engagement with a wide range of health and well-being apps can inform the design and the development of more engaging apps in general.

OBJECTIVE

The aim of this study is to explore user experiences of and reasons for engaging and not engaging with a wide range of health and well-being apps.

METHODS

A sample of adults in the United Kingdom (N=17) interested in using a health or well-being app participated in a semistructured interview to explore experiences of engaging and not engaging with these apps. Participants were recruited via social media platforms. Data were analyzed with the framework approach, informed by the Capability, Opportunity, Motivation-Behaviour (COM-B) model and the Theoretical Domains Framework, which are 2 widely used frameworks that incorporate a comprehensive set of behavioral influences.

RESULTS

Factors that influence the capability of participants included available user guidance, statistical and health information, reduced cognitive load, well-designed reminders, self-monitoring features, features that help establish a routine, features that offer a safety net, and stepping-stone app characteristics. Tailoring, peer support, and embedded professional support were identified as important factors that enhance user opportunities for engagement with health and well-being apps. Feedback, rewards, encouragement, goal setting, action planning, self-confidence, and commitment were judged to be the motivation factors that affect engagement with health and well-being apps.

CONCLUSIONS

Multiple factors were identified across all components of the COM-B model that may be valuable for the development of more engaging health and well-being apps. Engagement appears to be influenced primarily by features that provide user guidance, promote minimal cognitive load, support self-monitoring (capability), provide embedded social support (opportunity), and provide goal setting with action planning (motivation). This research provides recommendations for policy makers, industry, health care providers, and app developers for increasing effective engagement.

Understanding Uptake of Digital Health Products: Methodology Tutorial for a Discrete Choice Experiment Using the Bayesian Efficient Design

Understanding the preferences of potential users of digital health products is beneficial for digital health policy and planning. Stated preference methods could help elicit individuals' preferences in the absence of observational data. A discrete choice experiment (DCE) is a commonly used stated preference method-a quantitative methodology that argues that individuals make trade-offs when engaging in a decision by choosing an alternative of a product or a service that offers the greatest utility, or benefit. This methodology is widely used in health economics in situations in which revealed preferences are difficult to collect but is much less used in the field of digital health. This paper outlines the stages involved in developing a DCE. As a case study, it uses the application of a DCE to reveal preferences in targeting the uptake of smoking cessation apps. It describes the establishment of attributes, the construction of choice tasks of 2 or more alternatives, and the development of the experimental design. This tutorial offers a guide for researchers with no prior knowledge of this research technique.