Development of the mHealth App Trustworthiness checklist

Thematic analysis of tools for health innovators and organisation leaders to develop digital health solutions fit for climate change

OBJECTIVES

While ethicists have largely underscored the risks raised by digital health solutions that operate with or without artificial intelligence (AI), limited research has addressed the need to also mitigate their environmental footprint and equip health innovators as well as organisation leaders to meet responsibility requirements that go beyond clinical safety, efficacy and ethics. Drawing on the Responsible Innovation in Health framework, this qualitative study asks: (1) what are the practice-oriented tools available for innovators to develop environmentally sustainable digital solutions and (2) how are organisation leaders supposed to support them in this endeavour?

METHODS

Focusing on a subset of 34 tools identified through a comprehensive scoping review (health sciences, computer sciences, engineering and social sciences), our qualitative thematic analysis identifies and illustrates how two responsibility principles-environmental sustainability and organisational responsibility-are meant to be put in practice.

RESULTS

Guidance to make environmentally sustainable digital solutions is found in 11 tools whereas organisational responsibility is described in 33 tools. The former tools focus on reducing energy and materials consumption as well as pollution and waste production. The latter tools highlight executive roles for data risk management, data ethics and AI ethics. Only four tools translate environmental sustainability issues into tangible organisational responsibilities.

CONCLUSIONS

Recognising that key design and development decisions in the digital health industry are largely shaped by market considerations, this study indicates that significant work lies ahead for medical and organisation leaders to support the development of solutions fit for climate change.

'You have to put a lot of trust in me': autonomy, trust, and trustworthiness in the context of mobile apps for mental health

Trust and trustworthiness are essential for good healthcare, especially in mental healthcare. New technologies, such as mobile health apps, can affect trust relationships. In mental health, some apps need the trust of their users for therapeutic efficacy and explicitly ask for it, for example, through an avatar. Suppose an artificial character in an app delivers healthcare. In that case, the following questions arise: Whom does the user direct their trust to? Whether and when can an avatar be considered trustworthy? Our study aims to analyze different dimensions of trustworthiness in the context of mobile health app use. We integrate O'Neill's account of autonomy, trust, and trustworthiness into a model of trustworthiness as a relational concept with four relata: B is trustworthy with respect to A regarding the performance of Z because of C. Together with O'Neill's criteria of trustworthiness (honesty, competence, and reliability), this four-sided model is used to analyze different dimensions of trustworthiness in an exemplary case of mobile health app use. Our example focuses on an app that uses an avatar and is intended to treat sleep difficulties. The conceptual analysis shows that interpreting trust and trustworthiness in health app use is multi-layered and involves a net of interwoven universal obligations. At the same time, O'Neill's approach to autonomy, trust, and trustworthiness offers a normative account to structure and analyze these complex relations of trust and trustworthiness using mobile health apps.

Extending the Privacy Calculus to the mHealth Domain: Survey Study on the Intention to Use mHealth Apps in Germany

BACKGROUND

With the increasing digitalization of the health sector, more and more mobile health (mHealth) apps are coming to the market to continuously collect and process sensitive health data for the benefit of patients and providers. These technologies open up new opportunities to make the health care system more efficient and save costs but also pose potential threats such as loss of data or finances.

OBJECTIVE

This study aims to present an empirical review and adaptation of the extended privacy calculus model to the mHealth domain and to understand what factors influence the intended usage of mHealth technologies.

METHODS

A survey study was conducted to empirically validate our model, using a case vignette as cover story. Data were collected from 250 German participants and analyzed using a covariance-based structural equation model.

RESULTS

The model explains R2=79.3% of the variance in intention to use. The 3 main factors (social norms, attitude to privacy, and perceived control over personal data) influenced the intention to use mHealth apps, albeit partially indirectly. The intention to use mHealth apps is driven by the perceived benefits of the technology, trust in the provider, and social norms. Privacy concerns have no bearing on the intention to use. The attitude to privacy has a large inhibiting effect on perceived benefits, as well as on trust in the provider. Perceived control over personal data clearly dispels privacy concerns and supports the relationship of trust between the user and the provider.

CONCLUSIONS

Based on the privacy calculus, our domain-specific model explains the intention to use mHealth apps better than previous, more general models. The findings allow health care providers to improve their products and to increase usage by targeting specific user groups.

Assessing Patient Adherence to and Engagement With Digital Interventions for Depression in Clinical Trials: Systematic Literature Review

BACKGROUND

New approaches to the treatment of depression are necessary for patients who do not respond to current treatments or lack access to them because of barriers such as cost, stigma, and provider shortage. Digital interventions for depression are promising; however, low patient engagement could limit their effectiveness.

OBJECTIVE

This systematic literature review (SLR) assessed how participant adherence to and engagement with digital interventions for depression have been measured in the published literature, what levels of adherence and engagement have been reported, and whether higher adherence and increased engagement are linked to increased efficacy.

METHODS

We focused on a participant population of adults (aged ≥18 years) with depression or major depressive disorder as the primary diagnosis and included clinical trials, feasibility studies, and pilot studies of digital interventions for treating depression, such as digital therapeutics. We screened 756 unique records from Ovid MEDLINE, Embase, and Cochrane published between January 1, 2000, and April 15, 2022; extracted data from and appraised the 94 studies meeting the inclusion criteria; and performed a primarily descriptive analysis. Otsuka Pharmaceutical Development & Commercialization, Inc (Princeton, New Jersey, United States) funded this study.

RESULTS

This SLR encompassed results from 20,111 participants in studies using 47 unique web-based interventions (an additional 10 web-based interventions were not described by name), 15 mobile app interventions, 5 app-based interventions that are also accessible via the web, and 1 CD-ROM. Adherence was most often measured as the percentage of participants who completed all available modules. Less than half (44.2%) of the participants completed all the modules; however, the average dose received was 60.7% of the available modules. Although engagement with digital interventions was measured differently in different studies, it was most commonly measured as the number of modules completed, the mean of which was 6.4 (means ranged from 1.0 to 19.7) modules. The mean amount of time participants engaged with the interventions was 3.9 (means ranged from 0.7 to 8.4) hours. Most studies of web-based (34/45, 76%) and app-based (8/9, 89%) interventions found that the intervention group had substantially greater improvement for at least 1 outcome than the control group (eg, care as usual, waitlist, or active control). Of the 14 studies that investigated the relationship between engagement and efficacy, 9 (64%) found that increased engagement with digital interventions was significantly associated with improved participant outcomes. The limitations of this SLR include publication bias, which may overstate engagement and efficacy, and low participant diversity, which reduces the generalizability.

CONCLUSIONS

Patient adherence to and engagement with digital interventions for depression have been reported in the literature using various metrics. Arriving at more standardized ways of reporting adherence and engagement would enable more effective comparisons across different digital interventions, studies, and populations.

Using participatory action research to develop a new self-management program: Results from the design stage of Managing My MS My Way

BACKGROUND

Self-management programs have been used with success in several clinical populations, and there is a growing body of evidence to support their use among persons with multiple sclerosis (MS). This group aimed to develop a novel self-management program, Managing My MS My Way (M⁴W), which is based in social cognitive theory and contains evidence-based strategies that have been shown to be effective for persons with MS. Furthermore, persons with MS would serve as stakeholders throughout the development process to ensure that the program would be useful and encourage adoption. This paper outlines the initial development stages of M⁴W, including determining 1) stakeholders' interest in a self-management program, 2) the general focus of the program, 3) the delivery method of the program, 4) the content of the program, and 5) potential barriers and adaptations.

METHODS

A three-stage study consisting of an anonymous survey (n = 187) to determine interest, topic, and delivery format; semi-structured interviews (n = 6) to follow-up on the survey results; and semi-structured interviews (n = 10) to refine the content and identify barriers.

RESULTS

Over 80% of survey participants were somewhat or very interested in a self-management program. Fatigue was the topic with the greatest amount of interest (64.7%). An internet-based program (e.g., mobile health or mHealth) was the most preferred delivery method (37.4%), with the first group of stakeholders proposing a module-based system with an initial in-person orientation session. The second group of stakeholders were overall enthusiastic about the program, giving moderate to high confidence scores for each of the proposed interventional strategies. Suggestions included skipping sections that were not applicable to them, setting reminders, and seeing their progress (e.g., visualizing their fatigue scores as they move through the program). In addition, stakeholders recommended larger font sizes and speech-to-text entry.

CONCLUSIONS

Input from the stakeholders has been incorporated into the prototype of M⁴W. The next steps will be to test this prototype with another group of stakeholders to assess its initial usability and identify issues before developing the functional prototype.

Menstrual Tracking Mobile App Review by Consumers and Health Care Providers: Quality Evaluations Study

BACKGROUND

Women's menstrual cycle is an important component of their overall health. Physiological cycles and associated symptoms can be monitored continuously and used as indicators in various fields. Menstrual apps are accessible and can be used to promote overall female health. However, no study has evaluated these apps' functionality from both consumers' and health care providers' perspectives. As such, the evidence indicating whether the menstrual apps available on the market provide user satisfaction is insufficient.

OBJECTIVE

This study was performed to investigate the key content and quality of menstrual apps from the perspectives of health care providers and consumers. We also analyzed the correlations between health care provider and consumer evaluation scores. On the basis of this analysis, we offer technical and policy recommendations that could increase the usability and convenience of future app.

METHODS

We searched the Google Play Store and iOS App Store using the keywords "period" and "menstrual cycle" in English and Korean and identified relevant apps. An app that met the following inclusion criteria was selected as a research app: nonduplicate; with >10,000 reviews; last updated ≤180 days ago; relevant to this topic; written in Korean or English; available free of charge; and currently operational. App quality was evaluated by 6 consumers and 4 health care providers using Mobile Application Rating Scale (MARS) and user version of the Mobile Application Rating Scale (uMARS). We then analyzed the correlations among MARS scores, uMARS scores, star ratings, and the number of reviews.

RESULTS

Of the 34 apps, 31 (91%) apps could be used to predict the menstrual cycle, and 2 (6%) apps provided information pertinent to health screening. All apps that scored highly in the MARS evaluation offer a symptom logging function and provide the user with personalized notifications. The "Bom Calendar" app had the highest MARS (4.51) and uMARS (4.23) scores. The MARS (2.22) and uMARS (4.15) scores for the "Menstrual calendar-ovulation & pregnancy calendar" app were different. In addition, there was no relationship between MARS and uMARS scores (r=0.32; P=.06).

CONCLUSIONS

We compared consumer and health care provider ratings for menstrual apps. Continuous monitoring of app quality from consumer and health care provider perspectives is necessary to guide their development and update content.

Exploring the Needs and Expectations of Expectant and New Parents for an mHealth Application to Support the First 1000 Days of Life: Steps toward a Co-Design Approach

To improve maternal and child health, it is essential to adhere to health-promoting and preventive measures. However, reliable information as well as effective tools are not easy to identify in this field. Our cross-sectional study investigated the needs and expectations of expectant and new mothers and fathers as potential primary users of a hypothetical application supporting the first 1000 days of life. Between May and August 2022, we recruited expectant and new parents by administering an 83-item 5-point Likert scale questionnaire related to the content, functionalities, and technical features of the hypothetical app. We stratified responses using sociodemographic characteristics and then performed ward hierarchical clustering. The 94 women and 69 men involved in our study generally agreed with the proposed content, but expressed low interest in certain app functionalities or features, including those related to the interaction mechanism and interactivity. Women were generally more demanding than men. Our findings, resulting from the engagement of end-users, may be useful for designers and technology providers to implement mHealth solutions that, in addition to conveying reliable information, are tailored to the needs and preferences of end-users in the first 1000 days of life.

Digital Health Technologies for Long-term Self-management of Osteoporosis: Systematic Review and Meta-analysis

Background

Osteoporosis is the fourth most common chronic disease worldwide. The adoption of preventative measures and effective self-management interventions can help improve bone health. Mobile health (mHealth) technologies can play a key role in the care and self-management of patients with osteoporosis.

Objective

This study presents a systematic review and meta-analysis of the currently available mHealth apps targeting osteoporosis self-management, aiming to determine the current status, gaps, and challenges that future research could address, as well as propose appropriate recommendations.

Methods

A systematic review of all English articles was conducted, in addition to a survey of all apps available in iOS and Android app stores as of May 2021. A comprehensive literature search (2010 to May 2021) of PubMed, Scopus, EBSCO, Web of Science, and IEEE Xplore was conducted. Articles were included if they described apps dedicated to or useful for osteoporosis (targeting self-management, nutrition, physical activity, and risk assessment) delivered on smartphone devices for adults aged ≥18 years. Of the 32 articles, a random effects meta-analysis was performed on 13 (41%) studies of randomized controlled trials, whereas the 19 (59%) remaining studies were only included in the narrative synthesis as they did not provide enough data.

Results

In total, 3906 unique articles were identified. Of these 3906 articles, 32 (0.81%) articles met the inclusion criteria and were reviewed in depth. The 32 studies comprised 14,235 participants, of whom, on average, 69.5% (n=9893) were female, with a mean age of 49.8 (SD 17.8) years. The app search identified 23 relevant apps for osteoporosis self-management. The meta-analysis revealed that mHealth-supported interventions resulted in a significant reduction in pain (Hedges g −1.09, 95% CI −1.68 to −0.45) and disability (Hedges g −0.77, 95% CI −1.59 to 0.05). The posttreatment effect of the digital intervention was significant for physical function (Hedges g 2.54, 95% CI −4.08 to 4.08) but nonsignificant for well-being (Hedges g 0.17, 95% CI −1.84 to 2.17), physical activity (Hedges g 0.09, 95% CI −0.59 to 0.50), anxiety (Hedges g −0.29, 95% CI −6.11 to 5.53), fatigue (Hedges g −0.34, 95% CI −5.84 to 5.16), calcium (Hedges g −0.05, 95% CI −0.59 to 0.50), vitamin D intake (Hedges g 0.10, 95% CI −4.05 to 4.26), and trabecular score (Hedges g 0.06, 95% CI −1.00 to 1.12).

Conclusions

Osteoporosis apps have the potential to support and improve the management of the disease and its symptoms; they also appear to be valuable tools for patients and health professionals. However, most of the apps that are currently available lack clinically validated evidence of their efficacy and focus on a limited number of symptoms. A more holistic and personalized approach within a cocreation design ecosystem is needed.

Trial Registration

PROSPERO 2021 CRD42021269399; https://tinyurl.com/2sw454a9

The Application and Prospect of Mobile Health (mHealth) in Health Service for Older People Living Alone in Community: A Narrative Review

As a result of improvements in life expectancy and reductions in fertility rate, the increasing world population ageing brings huge challenges for both developed and developing countries. Such factors as fewer children, migration of children and widowhood further increase the number of older people living alone. Older adults prefer age in place, which means care in the home. As the main place older people live in, care in community absolutely needs more attention. Optimizing health services for the elderly living in community is of positive significance to health promotion and happiness enhancement. But the traditional health service for the elderly has drawbacks of poor timeliness and high labor cost. The rapid development of modern science and technology make it possible to apply mHealth in health service for the elderly. At present, mHealth is relatively mature in the communities of developed countries. This article presents the application of mHealth in many developed countries, as references for developing countries.

Digital Contact Tracing Apps for COVID-19: Development of a Citizen-Centered Evaluation Framework

Background

The silent transmission of COVID-19 has led to an exponential growth of fatal infections. With over 4 million deaths worldwide, the need to control and stem transmission has never been more critical. New COVID-19 vaccines offer hope. However, administration timelines, long-term protection, and effectiveness against potential variants are still unknown. In this context, contact tracing and digital contact tracing apps (CTAs) continue to offer a mechanism to help contain transmission, keep people safe, and help kickstart economies. However, CTAs must address a wide range of often conflicting concerns, which make their development/evolution complex. For example, the app must preserve citizens’ privacy while gleaning their close contacts and as much epidemiological information as possible.

Objective

In this study, we derived a compare-and-contrast evaluative framework for CTAs that integrates and expands upon existing works in this domain, with a particular focus on citizen adoption; we call this framework the Citizen-Focused Compare-and-Contrast Evaluation Framework (C³EF) for CTAs.

Methods

The framework was derived using an iterative approach. First, we reviewed the literature on CTAs and mobile health app evaluations, from which we derived a preliminary set of attributes and organizing pillars. These attributes and the probing questions that we formulated were iteratively validated, augmented, and refined by applying the provisional framework against a selection of CTAs. Each framework pillar was then subjected to internal cross-team scrutiny, where domain experts cross-checked sufficiency, relevancy, specificity, and nonredundancy of the attributes, and their organization in pillars. The consolidated framework was further validated on the selected CTAs to create a finalized version of C³EF for CTAs, which we offer in this paper.

Results

The final framework presents seven pillars exploring issues related to CTA design, adoption, and use: (General) Characteristics, Usability, Data Protection, Effectiveness, Transparency, Technical Performance, and Citizen Autonomy. The pillars encompass attributes, subattributes, and a set of illustrative questions (with associated example answers) to support app design, evaluation, and evolution. An online version of the framework has been made available to developers, health authorities, and others interested in assessing CTAs.

Conclusions

Our CTA framework provides a holistic compare-and-contrast tool that supports the work of decision-makers in the development and evolution of CTAs for citizens. This framework supports reflection on design decisions to better understand and optimize the design compromises in play when evolving current CTAs for increased public adoption. We intend this framework to serve as a foundation for other researchers to build on and extend as the technology matures and new CTAs become available.

Improving Pelvic Floor Muscle Training Adherence Among Pregnant Women: Validation Study

BACKGROUND

Mobile health apps, for example, the Tät, have been shown to be potentially effective in improving pelvic floor muscle training (PFMT) among women, but their effectiveness in pregnant women was limited. Adherence to daily PFMT will improve pelvic floor muscle strength leading to urinary incontinence (UI) improvement during the pregnancy.

OBJECTIVE

This study aims to document the validation process in developing the Kegel Exercise Pregnancy Training app, which was designed to improve the PFMT adherence among pregnant women.

METHODS

We utilized an intervention mapping approach incorporated within the mobile health development and evaluation framework. The framework involved the following steps: (1) conceptualization, (2) formative research, (3) pretesting, (4) pilot testing, (5) randomized controlled trial, and (6) qualitative research. The user-centered design-11 checklist was used to evaluate the user-centeredness properties of the app.

RESULTS

A cross-sectional study was conducted to better understand PFMT and UI among 440 pregnant women. The study reported a UI prevalence of 40.9% (180/440), with less than half having good PFMT practice despite their good knowledge. Five focus group discussions were conducted to understand the app design preferred by pregnant women. They agreed a more straightforward design should be used for better app usability. From these findings, a prototype was designed and developed accordingly, and the process conformed to the user-centered design-11 (UCD-11) checklist. A PFMT app was developed based on the mHealth development and evaluation framework model, emphasizing higher user involvement in the application design and development. The application was expected to improve its usability, acceptability, and ease of use.

CONCLUSIONS

The Kegel Exercise Pregnancy Training app was validated using a thorough design and development process to ensure its effectiveness in evaluating the usability of the final prototype in our future randomized control trial study.

Recommender systems for mental health apps: advantages and ethical challenges

Recommender systems assist users in receiving preferred or relevant services and information. Using such technology could be instrumental in addressing the lack of relevance digital mental health apps have to the user, a leading cause of low engagement. However, the use of recommender systems for digital mental health apps, particularly those driven by personal data and artificial intelligence, presents a range of ethical considerations. This paper focuses on considerations particular to the juncture of recommender systems and digital mental health technologies. While separate bodies of work have focused on these two areas, to our knowledge, the intersection presented in this paper has not yet been examined. This paper identifies and discusses a set of advantages and ethical concerns related to incorporating recommender systems into the digital mental health (DMH) ecosystem. Advantages of incorporating recommender systems into DMH apps are identified as (1) a reduction in choice overload, (2) improvement to the digital therapeutic alliance, and (3) increased access to personal data & self-management. Ethical challenges identified are (1) lack of explainability, (2) complexities pertaining to the privacy/personalization trade-off and recommendation quality, and (3) the control of app usage history data. These novel considerations will provide a greater understanding of how DMH apps can effectively and ethically implement recommender systems.

Evidence-based guiding principles to build public trust in personal data use in health systems

Objective

Public trust in health systems is pivotal for their effective and efficient functioning. In particular, public trust is essential for personal data use, as demonstrated in debates in many countries, for example, about whether data from COVID-19 contact tracing apps should be pooled or remain on individuals’ smartphones. Low levels of public trust pose a risk not only to health system legitimacy but can also harm population health.

Methods

Synthesising our previous qualitative and theoretical research in the English National Health Service which enabled us to conceptualise the nature of public trust in health systems, we present guiding principles designed to rebuild public trust, if lost, and to maintain high levels of public trust in personal data use within the health system, if not.

Results

To build public trust, health system actors need to not rush trust building; engage with the public; keep the public safe; offer autonomy to the public; plan for diverse trust relationships; recognise that trust is shaped by both emotion and rational thought; represent the public interest; and work towards realising a net benefit for the health system and the public.

Conclusions

Beyond policymakers and government officials, the guiding principles address a wide range of actors within health systems so that they can work collectively to build public trust. The guiding principles can be used to inform policymaking in health and health care and to analyse the performance of different governments to see if those governments that operate in greater conformity with the guiding principles perform better.

Patient and physician perspectives on the use and outcome measures of mHealth apps: Exploratory survey and focus group study

Objective

Factors that physicians and patients consider when making decisions about using or recommending health apps are not well understood. We explored these factors to better assess how to support such decision making.

Methods

We conducted an exploratory cross-sectional study in Ontario using qualitative focus groups and quantitative surveys. 133 physicians and 94 community dwelling adults completed online surveys and we held two focus groups of nine community dwelling participants who had cardiovascular risk factors and an interest in using mHealth apps. Quantitative survey data was analyzed descriptively. Focus groups were audio-recorded and transcribed verbatim prior to inductive thematic content analysis. We integrated the results from the surveys and focus groups to understand factors that influence physicians' and patients' selection and use of such apps.

Results

Physicians recommend apps to patients but the level of evidence they prefer to use to guide selection did not align with what they were currently using. Patients trusted recommendations and reviews from medical organizations and healthcare professionals when selecting apps and were motivated to continue using apps when they supported goal setting and tracking, data sharing, decision making, and empowerment.

Conclusions

The findings highlight the significance of evaluating mHealth apps based on metrics that patients and physicians value beyond usage and clinical outcome data. Patients engage with apps that support them in confidently managing their health. Increased training and awareness of apps and creating a more rigorous evidence base showing the value of apps to supporting health goals will support greater adoption and acceptance of mHealth apps.

Mobile Health Apps That Act as Surgical Preparatory Guides: App Store Search and Quality Evaluation

BACKGROUND

Mobile health (mHealth) apps are becoming increasingly common in surgical practices for training, education, and communication. Factors leading to increased delays, morbidity, and mortality in surgery include inadequate preoperative patient preparation due to a failure to identify patients and procedure details, and missing instruments and equipment required for the procedure. Many apps are available for supporting preoperative, intraoperative, and postoperative care. However, there is a lack of studies that assess the quality of apps that act as surgical preparatory guides.

OBJECTIVE

The aim of this study is to evaluate the quality of apps that act as surgical preparatory guides for operating room personnel through an in-house quality assessment tool.

METHODS

The quality assessment tool comprises 35 questions categorized into 5 sections: (1) engagement (customization, interactivity, target audience; 19 points), (2) functionality (performance, ease of use, navigation; 12 points), (3) aesthetics (layout, visual appeal; 6 points), (4) information (quality and quantity of information, visual information, credibility; 29 points), and (5) privacy and security (4 points). An app search was conducted in the Australian Apple and Google Play stores using the following keywords: "surgical apps", "surgical preferences", "surgeon preferences", "operating room", and "perioperative procedures". The overall total scores and scores for each section were reported as medians and IQRs, expressed as raw scores and percentages.

RESULTS

A total of 5 unique apps were evaluated on both iOS and Android platforms. The median overall score across all apps was 35/70 (50%; IQR 38.6%-64.3%). ScrubUp (48/70, 69%) and MySurgeon (42/70, 60%) had the highest overall scores, followed by PrefCard (35/70, 50%) and Scrubnote (28/70, 40%). The lowest scoring app was BrainPadd (26/70, 37%). The sections with the highest median scores, in decreasing order, were privacy and security (4/4, 100%; IQR 75%-100%), aesthetics (5/6, 83%; IQR 75%-91.7%), engagement (15/19, 79%; IQR 57.9%-86.8%), functionality (7/12, 58%; IQR 29.2%-75%), and information (5/29, 17%; IQR 15.5%-34.5%). Most apps scored well (4/4, 100%) on privacy and security, except for Scrubnote (2/4, 50%). ScrubUp received a perfect score for aesthetics (6/6, 100%). MySurgeon (17/19, 90%) had the highest engagement score, while ScrubUp and MySurgeon had the highest functionality scores (9/12, 75% each). All apps scored below 50% for the information section, with ScrubUp having the highest score of 13/29 (45%).

CONCLUSIONS

ScrubUp and MySurgeon had the highest quality scores and can be used as adjuncts to hospital protocols by operating room personnel for their surgical preparation. Developers are encouraged to develop appropriate apps for surgical preparation based on relevant guidelines and standards, as well as the quality evaluation criteria in our tool. Operating room personnel can also use this tool as a guide to select and assess their preferred apps in their practices.

Characteristics of Acute Childhood Illness Apps for Parents: Environmental Scan

Background

Providing parents with resources that aid in the identification and management of acute childhood illnesses helps those parents feel better equipped to assess their children’s health and significantly changes parental health-seeking behaviors. Some of these resources are limited by accessibility and scalability. Remote locations and staffing limitations create challenges for parents aiming to access their child’s health information. Mobile health apps offer a scalable, accessible solution for improving health literacy by enabling access to health information through mobile devices. 

Objective

The aim of our study is to create an inventory of acute childhood illness apps that are available to North American parents and caregivers, assess their quality, and identify the areas in which future apps can be improved.

Methods

We conducted an environmental scan to identify and summarize app information for parents and digital health researchers. The Google and Apple app marketplaces were used as search platforms. We built a list of search terms and searched the platforms for apps targeted at parents and related to acute pediatric illnesses in the United States and Canada. We assessed apps meeting the inclusion criteria using the Mobile App Rating Scale (MARS), a validated tool for assessing the quality of health apps. The MARS examines apps on 5 subscales: engagement, functionality, aesthetics, information quality, and subjective quality. Data were analyzed by MARS subscale averages and individual item scores.

Results

Overall, 650 unique apps were screened, and 53 (8.2%) were included. On a scale of 1-5, apps had an average engagement score of 2.82/5 (SD 0.86), functionality score of 3.98/5 (SD 0.72), aesthetics score of 3.09/5 (SD 0.87), information quality score of 2.73/5 (SD 1.32), and subjective quality score of 2.20/5 (SD 0.79). On the same scale of 1-5, app scores ranged from 2.2/5 to 4.5/5 (mean 3.2, SD 0.6). The top 3 MARS-scored apps were Baby and Child First Aid (4.5/5), Ada (4.5/5), and HANDi Paediatric (4.2/5). Taken together, the top 3 apps covered topics of emergency pediatric first aid, identification of (and appropriate response to) common childhood illnesses, a means of checking symptoms, and a means of responding to emergency situations. There was a lack of Canadian-based app content available to parents in both marketplaces; this space was filled with content originating primarily in the United Kingdom and the United States. In addition, published evidence of the impact of the included apps was poor: of 53 apps, only 5 (9%) had an evidence base showing that the app had been trialed for usability or efficacy.

Conclusions

There is a need for evidence-based acute childhood illness apps of Canadian origin. This environmental scan offers a comprehensive picture of the health app landscape by examining trends in acute childhood illness apps that are readily available to parents and by identifying gaps in app design.

Developing eHealth tools for diverse older adults: Lessons learned from the PREPARE for Your Care Program

BACKGROUND/OBJECTIVES

Electronic Health (eHealth) tools offer opportunities for people to access health information online; yet, most tools are not designed to meet the unique needs of diverse older adults, leading to health disparities. Our goal was to provide guidance for the development of eHealth tools for diverse older populations for use in geriatric care models.

DESIGN

Guidance for eHealth tools was compiled from user design resources and eHealth design literature. Pragmatic examples were provided from an evidenced-based eHealth tool called PREPAREforYourCare.org (PREPARE). We used quantitative feasibility data from PREPARE research studies and qualitative analysis of PREPARE focus groups, cognitive interviews, and feedback from randomized trials to further inform our recommendations.

RESULTS

Guidance and lessons learned include: (1) define clear objectives and a conceptual framework; (2) co-create with the target population; (3) optimize the design and layout for accessibility and ease of use, such as text at the 5th grade reading level, closed captioning, etc.; (4) use simple, standardized navigation design; (5) use actionable information to enhance behavior change, such as modeling of behaviors; (6) align accompanying written materials with the eHealth tool; and (7) create tracking mechanisms for ongoing user feedback. PREPARE is used as a case example to provide pragmatic illustrations for how the guidance may be operationalized.

CONCLUSION

eHealth tools can be tailored to the unique characteristics, preferences, and needs of diverse older populations. Following the "lessons learned" may help decrease health disparities among diverse older adults and ensure eHealth tools are readily accessible and culturally appropriate.

Refining clinical algorithms for a neonatal digital platform for low-income countries: a modified Delphi technique

OBJECTIVES

To determine whether a panel of neonatal experts could address evidence gaps in local and international neonatal guidelines by reaching a consensus on four clinical decision algorithms for a neonatal digital platform (NeoTree).

DESIGN

Two-round, modified Delphi technique.

SETTING AND PARTICIPANTS

Participants were neonatal experts from high-income and low-income countries (LICs).

METHODS

This was a consensus-generating study. In round 1, experts rated items for four clinical algorithms (neonatal sepsis, hypoxic ischaemic encephalopathy, respiratory distress of the newborn, hypothermia) and justified their responses. Items meeting consensus for inclusion (≥80% agreement) were incorporated into the algorithms. Items not meeting consensus were either excluded, included following revisions or included if they contained core elements of evidence-based guidelines. In round 2, experts rated items from round 1 that did not reach consensus.

RESULTS

Fourteen experts participated in round 1, 10 in round 2. Nine were from high-income countries, five from LICs. Experts included physicians and nurse practitioners with an average neonatal experience of 20 years, 12 in LICs. After two rounds, a consensus was reached on 43 of 84 items (52%). Per experts' recommendations, items in line with local and WHO guidelines yet not meeting consensus were still included to encourage consistency for front-line healthcare workers. As a result, the final algorithms included 53 items (62%).

CONCLUSION

Four algorithms in a neonatal digital platform were reviewed and refined by consensus expert opinion. Revisions to NeoTree will be made in response to these findings. Next steps include clinical validation of the algorithms.

Scoping review: Development and assessment of evaluation frameworks of mobile health apps for recommendations to consumers

Objective

The study sought to review the different assessment items that have been used within existing health app evaluation frameworks aimed at individual, clinician, or organizational users, and to analyze the scoring and evaluation methods used in these frameworks.

Materials and Methods

We searched multiple bibliographic databases and conducted backward searches of reference lists, using search terms that were synonyms of “health apps,” “evaluation,” and “frameworks.” The review covered publications from 2011 to April 2020. Studies on health app evaluation frameworks and studies that elaborated on the scaling and scoring mechanisms applied in such frameworks were included.

Results

Ten common domains were identified across general health app evaluation frameworks. A list of 430 assessment criteria was compiled across 97 identified studies. The most frequently used scaling mechanism was a 5-point Likert scale. Most studies have adopted summary statistics to generate the total scoring of each app, and the most popular approach taken was the calculation of mean or average scores. Other frameworks did not use any scaling or scoring mechanism and adopted criteria-based, pictorial, or descriptive approaches, or “threshold” filter.

Discussion

There is wide variance in the approaches to evaluating health apps within published frameworks, and this variance leads to ongoing uncertainty in how to evaluate health apps.

Conclusions

A new evaluation framework is needed that can integrate the full range of evaluative criteria within one structure, and provide summative guidance on health app rating, to support individual app users, clinicians, and health organizations in choosing or recommending the best health app.

Insights into mobile health application market via a content analysis of marketplace data with machine learning

BACKGROUND

Despite the benefits offered by an abundance of health applications promoted on app marketplaces (e.g., Google Play Store), the wide adoption of mobile health and e-health apps is yet to come.

OBJECTIVE

This study aims to investigate the current landscape of smartphone apps that focus on improving and sustaining health and wellbeing. Understanding the categories that popular apps focus on and the relevant features provided to users, which lead to higher user scores and downloads will offer insights to enable higher adoption in the general populace. This study on 1,000 mobile health applications aims to shed light on the reasons why particular apps are liked and adopted while many are not.

METHODS

User-generated data (i.e. review scores) and company-generated data (i.e. app descriptions) were collected from app marketplaces and manually coded and categorized by two researchers. For analysis, Artificial Neural Networks, Random Forest and Naïve Bayes Artificial Intelligence algorithms were used.

RESULTS

The analysis led to features that attracted more download behavior and higher user scores. The findings suggest that apps that mention a privacy policy or provide videos in description lead to higher user scores, whereas free apps with in-app purchase possibilities, social networking and sharing features and feedback mechanisms lead to higher number of downloads. Moreover, differences in user scores and the total number of downloads are detected in distinct subcategories of mobile health apps.

CONCLUSION

This study contributes to the current knowledge of m-health application use by reviewing mobile health applications using content analysis and machine learning algorithms. The content analysis adds significant value by providing classification, keywords and factors that influence download behavior and user scores in a m-health context.

Opportunities and Challenges of a Self-Management App to Support People With Spinal Cord Injury in the Prevention of Pressure Injuries: Qualitative Study

BACKGROUND

Mobile health applications can offer tailored self-management support to individuals living with chronic health conditions. However, there are several challenges to the adoption of these technologies in practice. Co-design is a promising approach to overcoming some of these challenges by enabling the development of solutions that meet the actual needs and preferences of the relevant stakeholder groups.

OBJECTIVE

Taking spinal cord injury as a case in point, the overall objectives of this study were to identify the perceived benefits of a co-designed self-management app that could promote its uptake and to explore the factors that may impede adoption.

METHODS

We adopted a qualitative research approach guided by the Technology Acceptance Model. Data were collected through semistructured interviews with individuals with spinal cord injury (n=15) and two focus groups with health care professionals specialized in spinal cord injury (n=7, n=5). Prior to the interviews and focus groups, study participants were given time to explore the app prototype. All interviews were transcribed verbatim and analyzed using inductive thematic analysis.

RESULTS

Findings of our analysis indicate that study participants perceived the app prototype as potentially useful for supporting individuals with spinal cord injury in preventing pressure injuries. In particular, we identified three concrete use cases highlighting the benefits of the app for different audiences: (1) a companion for newly injured individuals, (2) an emergency kit and motivational support, and 3) a guide for informal caregivers and family members. We also uncovered several challenges that might impede the adoption of the self-management app in practice, including (1) challenges in motivating individuals to use the app, (2) concerns about the misuse and abuse of the app, and (3) organizational and maintenance challenges.

CONCLUSIONS

This study adds to a growing body of research that investigates individuals' adoption and nonadoption behavior regarding mobile health solutions. Building on earlier work, we make recommendations on how to address the barriers to the adoption of mobile health solutions identified by this study. In particular, there is a need to foster trust in mobile health among prospective users, including both patients and health care professionals. Moreover, increasing personal relevance of mobile health solutions through personalization may be a promising approach to promote uptake. Last but not least, organizational support also plays an instrumental role in mobile health adoption. We conclude that even though co-design is promoted as a promising approach to develop self-management tools, co-design does not guarantee adoption. More research is needed to identify the most promising strategies to promote the adoption of evidence-based mobile health solutions in practice.

Experiences, Attitudes, and Needs of Users of a Pregnancy and Parenting App (Baby Buddy) During the COVID-19 Pandemic: Mixed Methods Study

BACKGROUND

The COVID-19 pandemic has impacted the lives of expectant parents and parents of young babies, with disruptions in health care provision and loss of social support.

OBJECTIVE

This study investigated the impact of the COVID-19 pandemic and its associated lockdown on this population through the lens of users of the UK National Health Service-approved pregnancy and parenting smartphone app, Baby Buddy. The study aims were threefold: to gain insights into the attitudes and experiences of expectant and recent parents (with babies under 24 weeks of age) during the COVID-19 pandemic; to investigate whether Baby Buddy is meeting users' needs during this time; and to identify ways to revise the content of Baby Buddy to better support its users now and in future.

METHODS

A mixed methods study design combining a web-based survey with semistructured telephone interviews among Baby Buddy users in the United Kingdom was applied. Data were collected from April 15 to mid-June 2020, corresponding to weeks 4-13 of the lockdown in the United Kingdom.

RESULTS

A total of 436 expectant (n=244, 56.0%) and recent (n=192, 44.0%) parents responded to the web-based survey, of which 79.1% (n=345) were aged 25-39 years and 17.2% (n=75) spoke English as their second language. Of the 436 respondents, 88.5% (386/436) reported increased levels of anxiety around pregnancy, birth, and being a new parent, and 58.0% (253/436) were concerned about their emotional and mental health. Of the 244 pregnant respondents, 43.4% (n=106) were concerned about their physical health. Telephone interviews with 13 pregnant women and 19 recent parents revealed similarly increased levels of anxiety due to reduced health care provision and loss of support from friends and family. Although a minority of respondents identified some positive outcomes of lockdown, such as family bonding, many telephone interviewees reported feeling isolated, disregarded, and overwhelmed. Recent parents were particularly anxious about the impact of the lockdown on their baby's development and socialization. Many interviewees were also concerned about their physical health as a consequence of both limited access to face-to-face medical appointments and their own poorer dietary and physical activity behaviors. Across both samples, 97.0% (423/436) of respondents reported that Baby Buddy was currently helping them, with many commenting that its role was even more important given the lack of face-to-face support from health care and parenting organizations. Greater speed in updating digital content to reflect changes due to the pandemic was suggested.

CONCLUSIONS

The COVID-19 pandemic has created heightened anxiety and stress among expectant parents and those with a young baby, and for many, lockdown has had an adverse impact on their physical and mental well-being. With reductions in health care and social support, expectant and new parents are increasingly relying on web-based resources. As a free, evidence-based app, Baby Buddy is well positioned to meet this need. The app could support its users even more by actively directing them to the wealth of existing content relevant to their concerns and by adding content to give users the knowledge and confidence to meet new challenges.

The Mobile Health App Trustworthiness Checklist: Usability Assessment

BACKGROUND

The mobile health (mHealth) app trustworthiness (mHAT) checklist was created to identify end users' opinions on the characteristics of trustworthy mHealth apps and to communicate this information to app developers. To ensure that the checklist is suited for all relevant stakeholders, it is necessary to validate its contents.

OBJECTIVE

The purpose of this study was to assess the feasibility of the mHAT checklist by modifying its contents according to ratings and suggestions from stakeholders familiar with the process of developing, managing, or curating mHealth apps.

METHODS

A 44-item online survey was administered to relevant stakeholders. The survey was largely comprised of the mHAT checklist items, which respondents rated on a 5-point Likert scale, ranging from completely disagree (1) to completely agree (5).

RESULTS

In total, seven professional backgrounds were represented in the survey: administrators (n=6), health professionals (n=7), information technology personnel (n=6), managers (n=2), marketing personnel (n=3), researchers (n=5), and user experience researchers (n=8). Aside from one checklist item-"the app can inform end users about errors in measurements"-the combined positive ratings (ie, completely agree and agree) of the checklist items overwhelmingly exceeded the combined negative ratings (ie, completely disagree and disagree). Meanwhile, two additional items were included in the checklist: (1) business or funding model of the app and (2) details on app uninstallation statistics.

CONCLUSIONS

Our results indicate that the mHAT checklist is a valuable resource for a broad range of stakeholders to develop trustworthy mHealth apps. Future studies should examine if the checklist works best for certain mHealth apps or in specific settings.