Systematic review of cost-effectiveness analysis of behavior change communication apps: Assessment of key methods

Assessing the Quality and Impact of eHealth Tools: Systematic Literature Review and Narrative Synthesis

BACKGROUND

Technological advancements have opened the path for many technology providers to easily develop and introduce eHealth tools to the public. The use of these tools is increasingly recognized as a critical quality driver in health care; however, choosing a quality tool from the myriad of tools available for a specific health need does not come without challenges.

OBJECTIVE

This review aimed to systematically investigate the literature to understand the different approaches and criteria used to assess the quality and impact of eHealth tools by considering sociotechnical factors (from technical, social, and organizational perspectives).

METHODS

A structured search was completed following the participants, intervention, comparators, and outcomes framework. We searched the PubMed, Cochrane, Web of Science, Scopus, and ProQuest databases for studies published between January 2012 and January 2022 in English, which yielded 675 results, of which 40 (5.9%) studies met the inclusion criteria. The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines and the Cochrane Handbook for Systematic Reviews of Interventions were followed to ensure a systematic process. Extracted data were analyzed using NVivo (QSR International), with a thematic analysis and narrative synthesis of emergent themes.

RESULTS

Similar measures from the different papers, frameworks, and initiatives were aggregated into 36 unique criteria grouped into 13 clusters. Using the sociotechnical approach, we classified the relevant criteria into technical, social, and organizational assessment criteria. Technical assessment criteria were grouped into 5 clusters: technical aspects, functionality, content, data management, and design. Social assessment criteria were grouped into 4 clusters: human centricity, health outcomes, visible popularity metrics, and social aspects. Organizational assessment criteria were grouped into 4 clusters: sustainability and scalability, health care organization, health care context, and developer.

CONCLUSIONS

This review builds on the growing body of research that investigates the criteria used to assess the quality and impact of eHealth tools and highlights the complexity and challenges facing these initiatives. It demonstrates that there is no single framework that is used uniformly to assess the quality and impact of eHealth tools. It also highlights the need for a more comprehensive approach that balances the social, organizational, and technical assessment criteria in a way that reflects the complexity and interdependence of the health care ecosystem and is aligned with the factors affecting users' adoption to ensure uptake and adherence in the long term.

The Influence of a Wearable-Based Reward Program on Health Care Costs: Retrospective, Propensity Score-Matched Cohort Study

BACKGROUND

Mobile health (mHealth) technology holds great promise as an easily accessible and effective solution to improve population health at scale. Despite the abundance of mHealth offerings, only a minority are grounded in evidence-based practice, whereas even fewer have line of sight into population-level health care spending, limiting the clinical utility of such tools.

OBJECTIVE

This study aimed to explore the influence of a health plan-sponsored, wearable-based, and reward-driven digital health intervention (DHI) on health care spending over 1 year. The DHI was delivered through a smartphone-based mHealth app available only to members of a large commercial health plan and leveraged a combination of behavioral economics, user-generated sensor data from the connected wearable device, and claims history to create personalized, evidence-based recommendations for each user.

METHODS

This study deployed a propensity score-matched, 2-group, and pre-post observational design. Adults (≥18 years of age) enrolled in a large, national commercial health plan and self-enlisted in the DHI for ≥7 months were allocated to the intervention group (n=56,816). Members who were eligible for the DHI but did not enlist were propensity score-matched to the comparison group (n=56,816). Average (and relative change from baseline) medical and pharmacy spending per user per month was computed for each member of the intervention and comparison groups during the pre- (ie, 12 months) and postenlistment (ie, 7-12 months) periods using claims data.

RESULTS

Baseline characteristics and medical spending were similar between groups (P=.89). On average, the total included sample population (N=113,632) consisted of young to middle-age (mean age 38.81 years), mostly White (n=55,562, 48.90%), male (n=46,731, 41.12%) and female (n=66,482, 58.51%) participants. Compared to a propensity score-matched cohort, DHI users demonstrated approximately US $10 per user per month lower average medical spending (P=.02) with a concomitant increase in preventive care activities and decrease in nonemergent emergency department admissions. These savings translated to approximately US $6.8 million in avoidable health care costs over the course of 1 year.

CONCLUSIONS

This employer-sponsored, digital health engagement program has a high likelihood for return on investment within 1 year owing to clinically meaningful changes in health-seeking behaviors and downstream medical cost savings. Future research should aim to elucidate health behavior-related mechanisms in support of these findings and continue to explore novel strategies to ensure equitable access of DHIs to underserved populations that stand to benefit the most.

Challenges and Limitations in Distributional Cost-Effectiveness Analysis: A Systematic Literature Review

BACKGROUND

Cost-effectiveness is a tool to maximize health benefits and to improve efficiency in healthcare. However, efficient outcomes are not always the most equitable ones. Distributional cost-effectiveness analysis (DCEA) offers a framework for incorporating equity concerns into cost-effectiveness analysis.

OBJECTIVE

This systematic review aims to outline the challenges and limitations in applying DCEA in healthcare settings.

METHODS

We searched Medline, Scopus, BASE, APA Psych, and JSTOR databases. We also included Google Scholar. We searched for English-language peer-reviewed academic publications, while books, editorials and commentary papers were excluded. Titles and abstract screening, full-text screening, reference list reviews, and data extraction were performed by the main researcher. Another researcher checked every paper for eligibility. Details, such as study population, disease area, intervention and comparators, costs and health effects, cost-effectiveness findings, equity analysis and effects, and modelling technique, were extracted. Thematic analysis was applied, focusing on challenges, obstacles, and gaps in DCEA.

RESULTS

In total, 615 references were identified, of which 18 studies met the inclusion criteria. Most of these studies were published after 2017. DCEA studies were mainly conducted in Europe and Africa and used quality health-adjusted measurements. In the included studies, absolute inequality indices were used more frequently than relative inequality indices. Every stage of the DCEA presented challenges and/or limitations.

CONCLUSION

This review provides an overview of the literature on the DCEA in healthcare as well as the challenges and limitations related to the different steps needed to conduct the analysis. In particular, we found problems with data availability, the relative unfamiliarity of this analysis among policymakers, and challenges in estimating differences among socioeconomic groups.

Recommendations for developing a lifecycle, multidimensional assessment framework for mobile medical apps

Digital health and mobile medical apps (MMAs) have shown great promise in transforming health care, but their adoption in clinical care has been unsatisfactory, and regulatory guidance and coverage decisions have been lacking or incomplete. A multidimensional assessment framework for regulatory, policymaking, health technology assessment, and coverage purposes based on the MMA lifecycle is needed. A targeted review of relevant policy documents from international sources was conducted to map current MMA assessment frameworks, to formulate 10 recommendations, subsequently shared amongst an expert panel of key stakeholders. Recommendations go beyond economic dimensions such as cost and economic evaluation and also include MMA development and update, classification and evidentiary requirements, performance and maintenance monitoring, usability testing, clinical evidence requirements, safety and security, equity considerations, organizational assessment, and additional outcome domains (patient empowerment and environmental impact). The COVID-19 pandemic greatly expanded the use of MMAs, but temporary policies governing their use and oversight need consolidation through well-developed frameworks to support decision-makers, producers and introduction into clinical care processes, especially in light of the strong international, cross-border character of MMAs, the new EU medical device and health technology assessment regulations, and the Next Generation EU funding earmarked for health digitalization.

Measuring Mobile Phone Application Usability for Anticoagulation from the Perspective of Patients, Caregivers, and Healthcare Professionals

Oral anticoagulants (OAC) are recommended for preventing stroke and systemic embolism in atrial fibrillation. Proper use is imperative for maximizing anticoagulation therapy's effectiveness and safety. In preparation for the implementation of a smartphone-based SmartMed app (application) aiming to promote patient self-management, medication adherence, and data collection for patients on anticoagulation therapy, its usability assessment can ensure the value of OAC app development and adoption. We evaluated the SmartMed app's usability using the System Usability Scale (SUS) and the app-specific domain of the Mobile App Rating Scale (MARS) for its perceived impact on taking OAC regularly. We recruited 25 OAC users and their home caregivers and 59 healthcare professionals, including pharmacists, nurses, and cardiac surgeons from one medical center and one regional hospital in Taiwan. All participants (n = 84) thought the SmartMed app was useful, with mean SUS and MARS scores of 81.49 (±14.42) and 4.65 (±0.49), respectively. Usability evaluation revealed that fewer experiences with smartphone apps and different healthcare professionals (pharmacists versus nurses or cardiac surgeons) were associated with lower SUS scores and perceived impact. Throughout the evaluation process, the SmartMed app's design was considered helpful from multiple stakeholders' perspectives. Further ongoing mobile technology supports are necessary to establish the SmartMed app's effectiveness.

Methods and Lessons From Costing a Large mHealth Intervention at Scale in India

The use of mobile devices to deliver public health interventions is rapidly increasing, particularly in low resource settings. Despite their proliferation, several mHealth interventions in developing countries fail to reach geographical scale, and long-term sustainability for most remains uncertain. There is a need to cost for such programs, to enable better planning and budgeting and tailor programs as required. Cost estimates can contribute to a more informed debate on resource allocation priorities and help make choices clearer for policymakers. This paper has two main objectives: (1) present a detailed protocol on determining the costs of a large national mHealth job aid and behavior change communication tool known as Integrated Child Development Services - Common Application Software (ICDS-CAS) in India, and (2) to present lessons for policymakers on how to ensure financial planning for scaling mHealth interventions. The study uses the Activity Based Costing—Ingredients (ABC-I) method. The major advantage of the ABC-I method is the clarity it brings to costs for each input and activity, across levels and geographies. It also accounts for indirect costs. There are five key lessons while costing for mHealth programs. First, that there are many activities and ingredients that must be budgeted for and discussed while planning and implementing mHealth programs. Second, the ABC-I method described in this paper provides great clarity on costs, yet its major limitation is the availability of data, which must be mitigated with the careful use of assumptions. Third, mHealth technology life cycles have financial implications which must be accounted for. Fourth, determining cost locations and all sources of funding including non-government sources is crucial. Fifth, since costing estimates are subject to a set of assumptions, a disaggregation of costs allows for scenario-building, which is useful while planning ahead and accounting for program changes. The evidence generated can be used for more informed debate on resource allocation priorities, given competing priorities in low- and middle-income countries.