MedAd-AppQ: A quality assessment tool for medication adherence apps on iOS and android platforms

Development and validation of the mobile adherence satisfaction scale (MASS) for medication adherence apps

OBJECTIVE

To develop and validate the Mobile Adherence Satisfaction Scale (MASS) for assessing user satisfaction with mobile health applications aimed to improve medication adherence.

METHODS

The study involved patients over 18 with asthma, hypertension, heart failure, or diabetes, who used the CareAide® app for six months. Scale development included a literature review, expert consultations, and patient interviews, initially identifying 129 items. These were refined to 27 using a two-round Delphi technique and grouped into six dimensions: user interface, perceived usability, system quality, service quality, feature satisfaction, and general satisfaction. A pilot study with 30 participants further refined the model, which was then validated with 135 participants using exploratory and confirmatory factor analyses in SPSS 29 and SmartPLS 4. Data were collected via self-administered questionnaires.

RESULTS

A total of 135 complete questionnaires were analysed. Respondents had an average age of 66.7 years (SD = 11.6) with 42.2 % male (n = 57) and 57.8 % female (n = 78). After removal of an item due to cross loading, exploratory factor analysis resulted six dimensions and 26 items with Kaiser-Meyer-Olkin measure of 0.837 and Bartlett's Test of Sphericity (χ2(n = 325) = 2085.673, P < 0.001). The confirmatory factor analysis confirmed high reliability and validity: Cronbach's alpha values > 0.70 for each dimension and an overall alpha of 0.89, with Composite Reliability and Average Variance Extracted both >0.70 and >0.50, respectively, for each dimension. Structural model indicated a significant positive impact of user interface (β = 0.226, P = 0.006) and feature satisfaction (β = 0.230, P = 0.002) on general satisfaction, explaining 23.1 % of the variance (R2 = 0.231).

CONCLUSION

The study developed and validated the MASS, a reliable tool for assessing user satisfaction with mHealth apps. User interface design and feature satisfaction are key for long-term engagement and consistent medication adherence.

A sociotechnical framework to assess patient-facing eHealth tools: results of a modified Delphi process

Among the thousands of eHealth tools available, the vast majority do not get past pilot phases because they cannot prove value, and only a few have been systematically assessed. Although multiple eHealth assessment frameworks have been developed, these efforts face multiple challenges. This study aimed to address some of these challenges by validating and refining an initial list of 55 assessment criteria based on previous frameworks through a two-round modified Delphi process with in-between rounds of interviews. The expert panel (n = 57) included participants from 18 countries and 9 concerned parties. A consensus was reached on 46 criteria that were classified into foundational and contextual criteria. The 36 foundational criteria focus on evaluating the eHealth tool itself and were grouped into nine clusters: technical aspects, clinical utility and safety, usability and human centricity, functionality, content, data management, endorsement, maintenance, and developer. The 10 contextual criteria focus on evaluating the factors that vary depending on the context the tool is being evaluated for and were grouped into seven clusters: data-protection compliance, safety regulatory compliance, interoperability and data integration, cultural requirements, affordability, cost-benefit, and implementability. The classification of criteria into foundational and contextual helps us assess not only the quality of an isolated tool, but also its potential fit in a specific setting. Criteria subscales may be particularly relevant when determining the strengths and weaknesses of the tool being evaluated. This granularity enables different concerned parties to make informed decisions about which tools to consider according to their specific needs and priorities.

Pain Management Mobile Applications: A Systematic Review of Commercial and Research Efforts

Shared decision making is crucial in the pain domain. The subjective nature of pain demands solutions that can facilitate pain assessment and management. The aim of the current study is to review the current trends in both the commercial and the research domains in order to reveal the key issues and guidelines that could further help in the effective development of pain-focused apps. We searched for scientific publications and commercial apps in 22 databases and the two major app stores. Out of 3612 articles and 336 apps, 69 met the requirements for inclusion following the PRISMA guidelines. An analysis of their features (technological approach, design methodology, evaluation strategy, and others) identified critical points that have to be taken into consideration in future efforts. For example, commercial and research efforts target different types of pain, while no participatory design is followed in the majority of the cases examined. Moreover, the evaluation of the final apps remains a challenge that hinders their success. The examined domain is expected to experience a substantial increase. More research is needed towards the development of non-intrusive wearables and sensors for pain detection and assessment, along with artificial intelligence techniques and open data.

Evaluation criteria for health apps supporting medication adherence in early-stage technology development - a scoping review

Introduction

Health apps offer an approach to improve the patients' management of their medication. Although the Digital Healthcare Act (DVG) has created a claim in the statutory health insurance (SHI), the large number of health apps available and their varying quality make it difficult for service providers and especially for medical laypersons to select an adequate high-quality medication app. Manufacturers need guidance for the development of high-quality apps right from the start. Various general evaluation concepts for health apps have been available to date. However, the requirements that should be met by healthcare depend largely on the field of application and the type of apps. This article aims to provide an overview of the international evidence on specific criteria for the evaluation of medication apps.

Methods

Within the framework of a scoping review, a systematic search was conducted in PubMed and EMBASE on January 29, 2020. The search was limited to publications from 2007 onwards as well as to English and German articles. Additionally, a semi-systematic research of reference lists of the previously included articles as well as a structured search of websites of relevant stakeholders were conducted. Inclusion criteria were the following: the publication deals with health apps that can be used on smartphones and focus on supporting medication intake; the publication does not refer to evaluation criteria for a single app exclusively. The included publications were examined in a qualitative content analysis searching for evaluation criteria and categorizing them according to the framework criteria of the DVG and the Digital Health Applications Ordinance (DiGAV).

Results

2,542 articles were identified in the systematic search (999 in PubMed, 1,543 in EMBASE, 560 duplicates). A total of 16 studies met the inclusion criteria. The semi-systematic research and the structured search identified one further study. A catalog of criteria was developed based on the included 17 studies. This catalog covers the general topics "patient orientation" (data protection and security, consumer protection, user friendliness) and "quality/core functions of medication apps" (reminder, self-monitoring, (drug) information, motivation to change behavior, drug/patient safety, robustness) as well as "interoperability/cooperation". Due to its specific importance for medication apps, the subcategory "motivation for behavioral change" stands out beneath the general topic "quality/core functions of medication apps". This category aims to evaluate the design of individual functions with regard to their potential to actually change the behavior of app users.

Discussion

The criteria for the evaluation of health apps mentioned in the DiGAV intersected with the criteria identified in the literature research. However, the area of positive health care effects was hardly covered by the included studies. In the development of the criteria catalog, it was not possible to weight the identified criteria. Therefore, the catalog should be understood as a supporting checklist for service providers, manufacturers, and/or users.

Conclusions

A large variety of possible evaluation criteria for medication apps could be shown. Future research should focus on the possibilities of weighting these diverse evaluation criteria, using not only clinical studies but also methods to identify preferences.

Assessing the Quality and Impact of eHealth Tools: Systematic Literature Review and Narrative Synthesis

BACKGROUND

Technological advancements have opened the path for many technology providers to easily develop and introduce eHealth tools to the public. The use of these tools is increasingly recognized as a critical quality driver in health care; however, choosing a quality tool from the myriad of tools available for a specific health need does not come without challenges.

OBJECTIVE

This review aimed to systematically investigate the literature to understand the different approaches and criteria used to assess the quality and impact of eHealth tools by considering sociotechnical factors (from technical, social, and organizational perspectives).

METHODS

A structured search was completed following the participants, intervention, comparators, and outcomes framework. We searched the PubMed, Cochrane, Web of Science, Scopus, and ProQuest databases for studies published between January 2012 and January 2022 in English, which yielded 675 results, of which 40 (5.9%) studies met the inclusion criteria. The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines and the Cochrane Handbook for Systematic Reviews of Interventions were followed to ensure a systematic process. Extracted data were analyzed using NVivo (QSR International), with a thematic analysis and narrative synthesis of emergent themes.

RESULTS

Similar measures from the different papers, frameworks, and initiatives were aggregated into 36 unique criteria grouped into 13 clusters. Using the sociotechnical approach, we classified the relevant criteria into technical, social, and organizational assessment criteria. Technical assessment criteria were grouped into 5 clusters: technical aspects, functionality, content, data management, and design. Social assessment criteria were grouped into 4 clusters: human centricity, health outcomes, visible popularity metrics, and social aspects. Organizational assessment criteria were grouped into 4 clusters: sustainability and scalability, health care organization, health care context, and developer.

CONCLUSIONS

This review builds on the growing body of research that investigates the criteria used to assess the quality and impact of eHealth tools and highlights the complexity and challenges facing these initiatives. It demonstrates that there is no single framework that is used uniformly to assess the quality and impact of eHealth tools. It also highlights the need for a more comprehensive approach that balances the social, organizational, and technical assessment criteria in a way that reflects the complexity and interdependence of the health care ecosystem and is aligned with the factors affecting users' adoption to ensure uptake and adherence in the long term.

User-Centric Design Methodology for mHealth Apps. The PainApp Paradigm for Chronic Pain

The paper presents a user-centric methodology in order to design successful mobile health (mHealth) applications. In addition to the theoretical background, such an example is presented with an application targeting chronic pain. The pain domain was decided due to its significance in many aspects: its complexity, dispersion in the population, the financial burden it causes, etc. The paper presents a step-by-step plan in order to build mobile health applications. Participatory design and interdisciplinarity are only some of the critical issues towards the desired result. In the given example (development of the PainApp), a participatory design was followed with a team of seventeen stakeholders that drove the design and development phases. Three physicians, one behavioral scientist, three IT and UX experts, and ten patients collaborated together to develop the final solution. The several features implemented in the PainApp solution are presented in details. The application is threefold: it supports the management, reporting, and treatment effectiveness monitoring. The paper is giving details on the methodological approach while presenting insights on the actual plan and the steps followed for having a patient-centric solution. Key success factors and barriers to mobile health applications that support the need for such an approach are also presented.

Mobile Health Apps That Act as Surgical Preparatory Guides: App Store Search and Quality Evaluation

BACKGROUND

Mobile health (mHealth) apps are becoming increasingly common in surgical practices for training, education, and communication. Factors leading to increased delays, morbidity, and mortality in surgery include inadequate preoperative patient preparation due to a failure to identify patients and procedure details, and missing instruments and equipment required for the procedure. Many apps are available for supporting preoperative, intraoperative, and postoperative care. However, there is a lack of studies that assess the quality of apps that act as surgical preparatory guides.

OBJECTIVE

The aim of this study is to evaluate the quality of apps that act as surgical preparatory guides for operating room personnel through an in-house quality assessment tool.

METHODS

The quality assessment tool comprises 35 questions categorized into 5 sections: (1) engagement (customization, interactivity, target audience; 19 points), (2) functionality (performance, ease of use, navigation; 12 points), (3) aesthetics (layout, visual appeal; 6 points), (4) information (quality and quantity of information, visual information, credibility; 29 points), and (5) privacy and security (4 points). An app search was conducted in the Australian Apple and Google Play stores using the following keywords: "surgical apps", "surgical preferences", "surgeon preferences", "operating room", and "perioperative procedures". The overall total scores and scores for each section were reported as medians and IQRs, expressed as raw scores and percentages.

RESULTS

A total of 5 unique apps were evaluated on both iOS and Android platforms. The median overall score across all apps was 35/70 (50%; IQR 38.6%-64.3%). ScrubUp (48/70, 69%) and MySurgeon (42/70, 60%) had the highest overall scores, followed by PrefCard (35/70, 50%) and Scrubnote (28/70, 40%). The lowest scoring app was BrainPadd (26/70, 37%). The sections with the highest median scores, in decreasing order, were privacy and security (4/4, 100%; IQR 75%-100%), aesthetics (5/6, 83%; IQR 75%-91.7%), engagement (15/19, 79%; IQR 57.9%-86.8%), functionality (7/12, 58%; IQR 29.2%-75%), and information (5/29, 17%; IQR 15.5%-34.5%). Most apps scored well (4/4, 100%) on privacy and security, except for Scrubnote (2/4, 50%). ScrubUp received a perfect score for aesthetics (6/6, 100%). MySurgeon (17/19, 90%) had the highest engagement score, while ScrubUp and MySurgeon had the highest functionality scores (9/12, 75% each). All apps scored below 50% for the information section, with ScrubUp having the highest score of 13/29 (45%).

CONCLUSIONS

ScrubUp and MySurgeon had the highest quality scores and can be used as adjuncts to hospital protocols by operating room personnel for their surgical preparation. Developers are encouraged to develop appropriate apps for surgical preparation based on relevant guidelines and standards, as well as the quality evaluation criteria in our tool. Operating room personnel can also use this tool as a guide to select and assess their preferred apps in their practices.

Identification and Evaluation of Methodologies to Assess the Quality of Mobile Health Apps in High-, Low-, and Middle-Income Countries: Rapid Review

BACKGROUND

In recent years, there has been rapid growth in the availability and use of mobile health (mHealth) apps around the world. A consensus regarding an accepted standard to assess the quality of such apps has yet to be reached. A factor that exacerbates the challenge of mHealth app quality assessment is variations in the interpretation of quality and its subdimensions. Consequently, it has become increasingly difficult for health care professionals worldwide to distinguish apps of high quality from those of lower quality. This exposes both patients and health care professionals to unnecessary risks. Despite progress, limited understanding of the contributions of researchers in low- and middle-income countries (LMICs) exists on this topic. Furthermore, the applicability of quality assessment methodologies in LMIC settings remains relatively unexplored.

OBJECTIVE

This rapid review aims to identify current methodologies in the literature to assess the quality of mHealth apps, understand what aspects of quality these methodologies address, determine what input has been made by authors from LMICs, and examine the applicability of such methodologies in LMICs.

METHODS

This review was registered with PROSPERO (International Prospective Register of Systematic Reviews). A search of PubMed, EMBASE, Web of Science, and Scopus was performed for papers related to mHealth app quality assessment methodologies, which were published in English between 2005 and 2020. By taking a rapid review approach, a thematic and descriptive analysis of the papers was performed.

RESULTS

Electronic database searches identified 841 papers. After the screening process, 52 papers remained for inclusion. Of the 52 papers, 5 (10%) proposed novel methodologies that could be used to evaluate mHealth apps of diverse medical areas of interest, 8 (15%) proposed methodologies that could be used to assess apps concerned with a specific medical focus, and 39 (75%) used methodologies developed by other published authors to evaluate the quality of various groups of mHealth apps. The authors in 6% (3/52) of papers were solely affiliated to institutes in LMICs. A further 15% (8/52) of papers had at least one coauthor affiliated to an institute in an LMIC.

CONCLUSIONS

Quality assessment of mHealth apps is complex in nature and at times subjective. Despite growing research on this topic, to date, an all-encompassing appropriate means for evaluating the quality of mHealth apps does not exist. There has been engagement with authors affiliated to institutes across LMICs; however, limited consideration of current generic methodologies for application in LMIC settings has been identified.

TRIAL REGISTRATION

PROSPERO CRD42020205149; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=205149.

The effect of a mobile application on treatment adherence and symptom management in patients using oral anticancer agents: A randomized controlled trial

PURPOSE

The use of mobile health technologies in the management of oral anticancer agents (OAA) can be beneficial in terms of treatment adherence and symptom management. This study was conducted to investigate the effect of a mobile application developed for patients using OAA on treatment adherence and symptom management.

METHOD

The study was conducted using a randomized controlled trial design, and it was carried out on 77 patients. Data were collected from the Oral Chemotherapy Adherence Scale (OCAS), and the Memorial Symptom Assessment Scale (MSAS). Data were collected at the beginning of the research, and face-to-face interviews were conducted after one, three, and six months. Patients in the intervention group were followed up for six months using the mobile application.

RESULTS

It was found that there was no difference between the intervention and control groups in the baseline OCAS mean scores (p > 0.05), and the mean score of the intervention group increased over the first, third- and sixth-month measurements (p < 0.05). It was found that there was no difference between the intervention and control groups in the MSAS mean scores (p > .05), and there was a decrease in the mean MSAS score of the intervention group between the third- and sixth-month follow-up (p < .05).

CONCLUSION

The present study results showed that the mobile application is effective in managing symptoms and increasing treatment adherence. A well-designed mobile health application that increases treatment adherence, decreases symptom severity, and supports patients' self-management could be beneficial for patients using OAA. CLINICALTRIALS.

GOV IDENTIFIER

NCT04626830.

Scoping review: Development and assessment of evaluation frameworks of mobile health apps for recommendations to consumers

Objective

The study sought to review the different assessment items that have been used within existing health app evaluation frameworks aimed at individual, clinician, or organizational users, and to analyze the scoring and evaluation methods used in these frameworks.

Materials and Methods

We searched multiple bibliographic databases and conducted backward searches of reference lists, using search terms that were synonyms of “health apps,” “evaluation,” and “frameworks.” The review covered publications from 2011 to April 2020. Studies on health app evaluation frameworks and studies that elaborated on the scaling and scoring mechanisms applied in such frameworks were included.

Results

Ten common domains were identified across general health app evaluation frameworks. A list of 430 assessment criteria was compiled across 97 identified studies. The most frequently used scaling mechanism was a 5-point Likert scale. Most studies have adopted summary statistics to generate the total scoring of each app, and the most popular approach taken was the calculation of mean or average scores. Other frameworks did not use any scaling or scoring mechanism and adopted criteria-based, pictorial, or descriptive approaches, or “threshold” filter.

Discussion

There is wide variance in the approaches to evaluating health apps within published frameworks, and this variance leads to ongoing uncertainty in how to evaluate health apps.

Conclusions

A new evaluation framework is needed that can integrate the full range of evaluative criteria within one structure, and provide summative guidance on health app rating, to support individual app users, clinicians, and health organizations in choosing or recommending the best health app.

Converting and expanding a mobile support intervention: Focus group and field-testing findings from individuals in active tuberculosis treatment

BACKGROUND

Non-adherence to tuberculosis (TB) treatment jeopardizes the individual's health and contributes to disease transmission and drug resistance. New patient-centered strategies are needed to improve TB treatment outcomes.

PURPOSE

To convert and expand a texting-based intervention into a mobile optimized application (app), evaluate the feasibility of an added self-administered paper-based drug metabolite test, and identify needs and preferences to inform their iterative design.

METHODS

Qualitative methods using focus groups and field testing with patients in active TB treatment were used to gather initial input on the converted intervention design, content and issues using at home test strips to report medication adherence. Seven participants were recruited from an outpatient clinic within a regional public reference hospital specialized in respiratory diseases in Argentina. Thematic analyses were conducted on the transcripts and session notes.

RESULTS

Participants considered interactive communication, access to answers to frequently asked questions, and tracking of progress in treatment as important. Participants reported having many questions and uncertainties at initiation of treatment and emphasized a need for reliable information, assurance and support from both providers and peers. Other suggestions included streamlining the graphical user interface for easier and shorter data entry times and usability.

CONCLUSIONS

Overall feedback from the participants regarding the intervention was positive, reporting that it was useful and relevant, and they were eager to contribute their ideas for improvement and additional functionality. Valuable feedback to improve functionality and meet the needs of end-users were obtained to inform the generation of new design ideas for refinement and testing in a pilot study.

Quality Principles of App Description Texts and Their Significance in Deciding to Use Health Apps as Assessed by Medical Students: Survey Study

Background

Currently, there are no binding requirements for manufacturers prescribing which information must be included in the app descriptions of health apps.

Objective

The aim of this study was to investigate how medical students perceive a selection of quality principles, intended for usage decisions in the app context, and establish whether the information presented in a sample of app descriptions is perceived as sufficient for facilitating an informed usage decision.

Methods

A total of 123 students (mean age 24.2 years, SD 3.4) participating in a 6-week teaching module covering cardiology and pulmonology at the University of Göttingen (original enrollment 152 students, response rate 80.9%) were included. Students were asked to read 3 store description texts of cardiological or pneumological apps and initially assess whether the descriptions sufficed for a usage decision. Subsequently, they were queried on their perception of the relevance of 9 predefined quality principles, formulated for usage decisions. An appraisal of whether the app description texts contained sufficient information to satisfy these quality principles followed. By means of 20 guiding questions, participants were then asked to identify relevant information (or a lack thereof) within the descriptions. A reassessment of whether the description texts sufficed for making a usage decision ensued. A total of 343 complete datasets were obtained.

Results

A majority of the quality principles were described as “very important” and “important” for making a usage decision. When accessed via the predefined principles, students felt unable to identify sufficient information within the app descriptions in 68.81% (2124/3087) of cases. Notably, information regarding undesired effects (91.8%, 315/343), ethical soundness (90.1%, 309/343), measures taken to avert risks (89.2%, 306/343), conflicts of interest (88.3%, 303/343), and the location of data storage (87.8%, 301/343) was lacking. Following participants’ engagement with the quality principles, statistically significant changes in their assessment of whether the app descriptions sufficed for a usage decision can be seen—McNemar-Bowker test (3)=45.803919, P<.001, Cohen g=.295. In 34.1% (117/343) cases, the assessment was revised. About 3 quarters of changed assessments were seen more critically (76.9%, 90/117). Although, initially, 70% (240/343) had been considered “sufficient,” this rate was reduced to 54.2% (186/343) in the second assessment.

Conclusions

In a considerable number of app descriptions, participants were unable to locate the information necessary for making an informed usage decision. Participants’ sensitization to the quality principles led to changes in their assessment of app descriptions as a tool for usage decisions. Better transparency in app descriptions released by manufacturers and the exposure of users to quality principles could collectively form the basis for well-founded usage decisions.

Design Considerations in the Development of App-Based Oral Anticancer Medication Management Systems: a Qualitative Evaluation of Pharmacists' and Patients' Perspectives

Smartphone apps can potentially help in enhancing oral anticancer medication (OAM) adherence. Patient adoption and efficacy of such apps depends on inclusion of user-centred and evidence-based features. The objective of this study was to identify important design considerations from the perspectives of patients taking OAMs, caregivers and oncology pharmacists. The study employed a qualitative study design. Data were collected using in-depth interviews with patients (n = 15), caregivers (n = 3) and pharmacists (n = 16). Interviews were audio-recorded, transcribed verbatim and inductive thematic analysis approach was used in data analysis. Monitoring medication-related problems, medication information, replacement of or integration with current systems and accessibility of app content on devices other than smartphones were the key themes identified in the analysis. Flexible input methods for monitored data, glanceability of monitored reports/information, near real-time adherence enhancing and symptom management interventions and customisable reminder options were design considerations identified under the monitoring medication-related problems theme. Participants suggested the provision of focused and easily understandable medication information with a potential for personalisation. Integration of app-based adherence systems with patients' electronic medical records with added mechanisms for alerts in the dispensing system was also suggested as a key design requirement to improve quality of patient care and facilitate adoption by clinicians. Finally, smartphones were the most favoured platform with optional accessibility of app content on other devices. In conclusion, important design considerations were identified through a user-centred design approach. The findings will help developers and clinicians in the design of new app-based systems and evaluation of existing ones.