Sticky apps, not sticky hands: A systematic review and content synthesis of hand hygiene mobile apps

Reporting quality of published reviews of commercial and publicly available mobile health apps (mHealth app reviews): a scoping review protocol

INTRODUCTION

Reviews of commercial and publicly available smartphone (mobile) health applications (mHealth app reviews) are being undertaken and published. However, there is variation in the conduct and reporting of mHealth app reviews, with no existing reporting guidelines. Building on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, we aim to develop the Consensus for APP Review Reporting Items (CAPPRRI) guidance, to support the conduct and reporting of mHealth app reviews. This scoping review of published mHealth app reviews will explore their alignment, deviation, and modification to the PRISMA 2020 items for systematic reviews and identify a list of possible items to include in CAPPRRI.

METHOD AND ANALYSIS

We are following the Joanna Briggs Institute approach and Arksey and O'Malley's five-step process. Patient and public contributors, mHealth app review, digital health research and evidence synthesis experts, healthcare professionals and a specialist librarian gave feedback on the methods. We will search SCOPUS, CINAHL Plus, AMED, EMBASE, Medline, APA PsycINFO and the ACM Digital Library for articles reporting mHealth app reviews and use a two-step screening process to identify eligible articles. Information on whether the authors have reported, or how they have modified the PRISMA 2020 items in their reporting, will be extracted. Data extraction will also include the article characteristics, protocol and registration information, review question frameworks used, information about the search and screening process, how apps have been evaluated and evidence of stakeholder engagement. This will be analysed using a content synthesis approach and presented using descriptive statistics and summaries. This protocol is registered on OSF (https://osf.io/5ahjx).

ETHICS AND DISSEMINATION

Ethical approval is not required. The findings will be disseminated through peer-reviewed journal publications (shared on our project website and on the EQUATOR Network website where the CAPPRRI guidance has been registered as under development), conference presentations and blog and social media posts in lay language.

Virtual reality and augmented reality smartphone applications for upskilling care home workers in hand hygiene: a realist multi-site feasibility, usability, acceptability, and efficacy study

OBJECTIVES

To assess the feasibility and implementation, usability, acceptability and efficacy of virtual reality (VR), and augmented reality (AR) smartphone applications for upskilling care home workers in hand hygiene and to explore underlying learning mechanisms.

MATERIALS AND METHODS

Care homes in Northwest England were recruited. We took a mixed-methods and pre-test and post-test approach by analyzing uptake and completion rates of AR, immersive VR or non-immersive VR training, validated and bespoke questionnaires, observations, videos, and interviews. Quantitative data were analyzed descriptively. Qualitative data were analyzed using a combined inductive and deductive approach.

RESULTS

Forty-eight care staff completed AR training (n = 19), immersive VR training (n = 21), or non-immersive VR training (n = 8). The immersive VR and AR training had good usability with System Usability Scale scores of 84.40 and 77.89 (of 100), respectively. They had high acceptability, with 95% of staff supporting further use. The non-immersive VR training had borderline poor usability, scoring 67.19 and only 63% would support further use. There was minimal improved knowledge, with an average of 6% increase to the knowledge questionnaire. Average hand hygiene technique scores increased from 4.77 (of 11) to 7.23 after the training. Repeated practice, task realism, feedback and reminding, and interactivity were important learning mechanisms triggered by AR/VR. Feasibility and implementation considerations included managerial support, physical space, providing support, screen size, lagging Internet, and fitting the headset.

CONCLUSIONS

AR and immersive VR apps are feasible, usable, and acceptable for delivering training. Future work should explore whether they are more effective than previous training and ensure equity in training opportunities.

A systematic review of menopause apps with an emphasis on osteoporosis

BACKGROUND

Menopause can significantly hasten bone loss. Mobile phones provide an efficient way to manage, track and understand menopause using apps. A previous review of menopause apps found numerous apps designed to help women manage menopause. However, it did not use validated measures to assess the quality of the apps and did not focus on content related to osteoporosis.

METHODS

This app review aligns with the updated Preferred Reporting Items for Systematic Reviews and Meta-Analysis guidelines. The keywords used to search for the apps were "menopause" and "menopausal". Apps were included if they were in English, for individuals or groups and had a lifestyle focus. Apps that looked at other aspects of women's health, required external devices, cost to download, or were symptom-tracking were excluded. The quality and functionality were assessed using the Mobile App Rating Scale and IMS Institute for Healthcare Informatics Functionality score. Data were synthesised descriptively.

RESULTS

Twenty-eight apps were selected and reviewed from the 236 apps screened from the Apple store and Google play store. Only 57% of the apps reviewed (n = 16) had content on osteoporosis which was educational in purpose. The readability of the apps was complex and best understood by university graduates. The average functionality score of the apps reviewed was 4.57 out of 11 and that of quality is 3.1 out of 5, both of which need improvement.

CONCLUSIONS

Existing menopause apps need more input from experts to improve the quality and functionality, using simple language. More emphasis on specific health problems during menopause, including osteoporosis, is required.

TRIAL REGISTRATION

Not relevant.

Conducting a systematic review and evaluation of commercially available mobile applications (apps) on a health-related topic: the TECH approach and a step-by-step methodological guide

OBJECTIVES

To provide an overview of the methodological considerations for conducting commercial smartphone health app reviews (mHealth reviews), with the aim of systematising the process and supporting high-quality evaluations of mHealth apps.

DESIGN

Synthesis of our research team's experiences of conducting and publishing various reviews of mHealth apps available on app stores and hand-searching the top medical informatics journals (eg, The Lancet Digital Health, npj Digital Medicine, Journal of Biomedical Informatics and the Journal of the American Medical Informatics Association) over the last five years (2018-2022) to identify other app reviews to contribute to the discussion of this method and supporting framework for developing a research (review) question and determining the eligibility criteria.

RESULTS

We present seven steps to support rigour in conducting reviews of health apps available on the app market: (1) writing a research question or aims, (2) conducting scoping searches and developing the protocol, (3) determining the eligibility criteria using the TECH framework, (4) conducting the final search and screening of health apps, (5) data extraction, (6) quality, functionality and other assessments and (7) analysis and synthesis of findings. We introduce the novel TECH approach to developing review questions and the eligibility criteria, which considers the Target user, Evaluation focus, Connectedness and the Health domain. Patient and public involvement and engagement opportunities are acknowledged, including co-developing the protocol and undertaking quality or usability assessments.

CONCLUSION

Commercial mHealth app reviews can provide important insights into the health app market, including the availability of apps and their quality and functionality. We have outlined seven key steps for conducting rigorous health app reviews in addition to the TECH acronym, which can support researchers in writing research questions and determining the eligibility criteria. Future work will include a collaborative effort to develop reporting guidelines and a quality appraisal tool to ensure transparency and quality in systematic app reviews.

Patient-facing cancer mobile apps that enable patient reported outcome data to be collected: A systematic review of content, functionality, quality, and ability to integrate with electronic health records

PURPOSE

Enabling cancer patients to self-manage symptoms through mobile applications can result in more informed, autonomous patients who are partners in their care, consequently reducing the burden on health services. Electronic patient reported outcomes completed before a clinical review can increase the frequency and quality of holistic assessments, while integration into electronic health records can maximise clinical utility. The ability of apps to integrate with electronic health records is key to providing a real-time interface between patient reports and healthcare response. This review identifies patient-facing cancer apps which can record patient reported outcomes, and explores their purpose, functionality, quality, and ability to integrate with electronic health records.

METHODS

A systematic app review and content synthesis was conducted on patient-facing cancer apps available in the United Kingdom. Where applicable, the review aligned with the Preferred Reporting Items for Systematic Reviews and meta-Analysis. Two validated scales assessed functionality and quality: The IMS Institute for Healthcare Informatics functionality score and the Mobile App Rating Scale. Flesch-Kincaid metrics explored readability.

RESULTS

Apple App and Google Play stores identified 405 apps, of which 12 met the eligibility criteria. All were free to download, 1 (8%) had in-app purchases/subscriptions. Nine (75%) were affiliated with a professional health body/charity. Six (50%) analysed inputted data and provided medical advice based on answers. The average Flesch Reading Ease score was 42.7 out of 100. The apps had an average of 7.3 functions each and a mean MARS score of 4/5. None integrated with electronic health records.

CONCLUSION

While many cancer apps exist, few enable patient reported outcomes to be recorded and shared with clinicians in real-time. Further research is warranted to explore the feasibility of integrating with electronic health records, as this function can improve patient experience and outcomes, and increase efficiency of hospital resources through more proactive care.

Addressing Complexity in the Pandemic Context: How Systems Thinking Can Facilitate Understanding of Design Aspects for Preventive Technologies

The COVID-19 pandemic constitutes a wicked problem that is defined by rapidly evolving and dynamic conditions, where the physical world changes (e.g., pathogens mutate) and, in parallel, our understanding and knowledge rapidly progress. Various preventive measures have been developed or proposed to manage the situation, including digital preventive technologies to support contact tracing or physical distancing. The complexity of the pandemic and the rapidly evolving nature of the situation pose challenges for the design of effective preventive technologies. The aim of this conceptual paper is to apply a systems thinking model, DSRP (distinctions, systems, relations, perspectives) to explain the underlying assumptions, patterns, and connections of the pandemic domain, as well as to identify potential leverage points for design of preventive technologies. Two different design approaches, contact tracing and nudging for distance, are compared, focusing on how their design and preventive logic are related to system complexity. The analysis explains why a contact tracing technology involves more complexity, which can challenge both implementation and user understanding. A system utilizing nudges can operate using a more distinct system boundary, which can benefit understanding and implementation. However, frequent nudges might pose challenges for user experience. This further implies that these technologies have different contextual requirements and are useful at different levels in society. The main contribution of this work is to show how systems thinking can organize our understanding and guide the design of preventive technologies in the context of epidemics and pandemics.

Using a Proximity-Detection Technology to Nudge for Physical Distancing in a Swedish Workplace During the COVID-19 Pandemic: Retrospective Case Study

BACKGROUND

The recent COVID-19 pandemic has contributed to the emergence of several technologies for infectious disease management. Although much focus has been placed on contact-tracing apps, another promising new tactic is proximity tracing, which focuses on health-related behavior and can be used for primary prevention. Underpinned by theories on behavioral design, a proximity-detection system can be devised that provides a user with immediate nudges to maintain physical distance from others. However, the practical feasibility of proximity detection during an infectious disease outbreak has not been sufficiently investigated.

OBJECTIVE

We aimed to evaluate the feasibility of using a wearable device to nudge for distance and to gather important insights about how functionality and interaction are experienced by users. The results of this study can guide future research and design efforts in this emerging technology.

METHODS

In this retrospective case study, a wearable proximity-detection technology was used in a workplace for 6 weeks during the production of a music competition. The purpose of the technology was to nudge users to maintain their physical distance using auditory feedback. We used a mixed methods sequential approach, including interviews (n=8) and a survey (n=30), to compile the experiences of using wearable technology in a real-life setting.

RESULTS

We generated themes from qualitative analysis based on data from interviews and open-text survey responses. The quantitative data were subsequently integrated into these themes: feasibility (implementation and acceptance-establishing a shared problem; distance tags in context-strategy, environment, and activities; understanding and learning; and accomplishing the purpose) and design aspects (a purposefully annoying device; timing, tone, and proximity; and additional functions).

CONCLUSIONS

This empirical study reports on the feasibility of using wearable technology based on proximity detection to nudge individuals to maintain physical distance in the workplace. The technology supports attention to distance, but the usability of this approach is dependent on the context and situation. In certain situations, the audio signal is frustrating, but most users agree that it needs to be annoying to ensure sufficient behavioral adaption. We proposed a dual nudge that involves vibration followed by sound. There are indications that the technology also facilitates learning how to maintain a greater distance from others, and that this behavior can persist beyond the context of technology use. This study demonstrates that the key value of this technology is that it places the user in control and enables immediate action when the distance to others is not maintained. This study provides insights into the emerging field of personal and wearable technologies used for primary prevention during infectious disease outbreaks. Future research is needed to evaluate the preventive effect on transmission and investigate behavioral changes in detail and in relation to different forms of feedback.

Is virtual reality suitable for hand hygiene training in health care workers? Evaluating an application for acceptability and effectiveness

Background

For effective prevention of nosocomial transmissions continuous training and motivation of health care workers (HCW) are essential to maintain and increase compliance with high rates of hand hygiene. The use of Virtual Reality (VR) seems to be a contemporary and interesting approach for hand hygiene training in HCW. Nevertheless, HCW should be asked for their preferences as intrinsic motivation is essential for compliance with hand hygiene and training success should be evaluated.

Methods

A prospective, cross-controlled trial was conducted at two wards in a tertiary care hospital comparing a conventional lecture for hand hygiene to the use of VR. Both interventions were assigned at ward level. Primary outcome was HCW acceptance, which was verified in a third ward, secondary outcomes were hand rub consumption and compliance to indications for hand hygiene as proposed by WHO.

Results

In summary, 81 trainings were conducted, 48 VR trainings and 33 trainings by lecture. VR training was well accepted by HCW with a mean score in all items from 3.9 to 4.3 (out of 5). While most HCW (69%) would prefer VR teaching rather than a lecture for hand hygiene education, only 4% preferred the traditional lecture. 400 observations of hand hygiene indications were made, 50 before intervention and 50 after each intervention at the three wards. Mean proportion of correct and indication-appropriate performances was 81% before intervention, 87% after VR training (p = 0.12), and 95% after lecture (p = 0.04). Hand rub consumption did not change significantly in any group.

Conclusions

Due to the high acceptance of VR technology among healthcare workers, it can be considered an interesting addition to conventional lectures for teaching hand hygiene. However, the hypothesis that VR teaching has a higher impact on hand rub use and hand hygiene compliance than a conventional lecture cannot be confirmed.

Patient-facing genetic and genomic mobile apps in the UK: a systematic review of content, functionality, and quality

Close relative (consanguineous) marriage is widely practised globally, and it increases the risk of genetic disorders. Mobile apps may increase awareness and education regarding the associated risks in a sensitive, engaging, and accessible manner. This systematic review of patient-facing genetic/genomic mobile apps explores content, function, and quality. We searched the NHS Apps Library and the UK Google Play and Apple App stores for patient-facing genomic/genetic smartphone apps. Descriptive information and information on content was extracted and summarized. Readability was examined using the Flesch–Kincaid metrics. Two raters assessed each app, using the Mobile App Rating Scale (MARS) and the IMS Institute for Healthcare Informatics functionality score. A total of 754 apps were identified, of which 22 met the eligibility criteria. All apps intended to inform/educate users, while 32% analyzed genetic data, and 18% helped to diagnose genetic conditions. Most (68%) were clearly about genetics, but only 14% were affiliated with a medical/health body or charity, and only 36% had a privacy strategy. Mean reading scores were 35 (of 100), with the average reading age being equivalent to US grade 12 (UK year 13). On average, apps had 3.3 of the 11 IMS functionality criteria. The mean MARS quality score was 3.2 ± 0.7. Half met the minimum acceptability score (3 of 5). None had been formally evaluated. It was evident that there are few high-quality genomic/genetic patient-facing apps available in the UK. This demonstrates a need for an accessible, culturally sensitive, evidence-based app to improve genetic literacy within patient populations and specific communities.

How, for Whom, and in Which Contexts or Conditions Augmented and Virtual Reality Training Works in Upskilling Health Care Workers: Realist Synthesis

Background

Using traditional simulators (eg, cadavers, animals, or actors) to upskill health workers is becoming less common because of ethical issues, commitment to patient safety, and cost and resource restrictions. Virtual reality (VR) and augmented reality (AR) may help to overcome these barriers. However, their effectiveness is often contested and poorly understood and warrants further investigation.

Objective

The aim of this review is to develop, test, and refine an evidence-informed program theory on how, for whom, and to what extent training using AR or VR works for upskilling health care workers and to understand what facilitates or constrains their implementation and maintenance.

Methods

We conducted a realist synthesis using the following 3-step process: theory elicitation, theory testing, and theory refinement. We first searched 7 databases and 11 practitioner journals for literature on AR or VR used to train health care staff. In total, 80 papers were identified, and information regarding context-mechanism-outcome (CMO) was extracted. We conducted a narrative synthesis to form an initial program theory comprising of CMO configurations. To refine and test this theory, we identified empirical studies through a second search of the same databases used in the first search. We used the Mixed Methods Appraisal Tool to assess the quality of the studies and to determine our confidence in each CMO configuration.

Results

Of the 41 CMO configurations identified, we had moderate to high confidence in 9 (22%) based on 46 empirical studies reporting on VR, AR, or mixed simulation training programs. These stated that realistic (high-fidelity) simulations trigger perceptions of realism, easier visualization of patient anatomy, and an interactive experience, which result in increased learner satisfaction and more effective learning. Immersive VR or AR engages learners in deep immersion and improves learning and skill performance. When transferable skills and knowledge are taught using VR or AR, skills are enhanced and practiced in a safe environment, leading to knowledge and skill transfer to clinical practice. Finally, for novices, VR or AR enables repeated practice, resulting in technical proficiency, skill acquisition, and improved performance. The most common barriers to implementation were up-front costs, negative attitudes and experiences (ie, cybersickness), developmental and logistical considerations, and the complexity of creating a curriculum. Facilitating factors included decreasing costs through commercialization, increasing the cost-effectiveness of training, a cultural shift toward acceptance, access to training, and leadership and collaboration.

Conclusions

Technical and nontechnical skills training programs using AR or VR for health care staff may trigger perceptions of realism and deep immersion and enable easier visualization, interactivity, enhanced skills, and repeated practice in a safe environment. This may improve skills and increase learning, knowledge, and learner satisfaction. The future testing of these mechanisms using hypothesis-driven approaches is required. Research is also required to explore implementation considerations.

Upskilling health and care workers with augmented and virtual reality: protocol for a realist review to develop an evidence-informed programme theory

INTRODUCTION

Augmented reality (AR) and virtual reality (VR) are increasingly used to upskill health and care providers, including in surgical, nursing and acute care settings. Many studies have used AR/VR to deliver training, providing mixed evidence on their effectiveness and limited evidence regarding contextual factors that influence effectiveness and implementation. This review will develop, test and refine an evidence-informed programme theory on what facilitates or constrains the implementation of AR or VR programmes in health and care settings and understand how, for whom and to what extent they 'work'.

METHODS AND ANALYSIS

This realist review adheres to the Realist And Meta-narrative Evidence Syntheses: Evolving Standards (RAMESES) standards and will be conducted in three steps: theory elicitation, theory testing and theory refinement. First, a search will identify practitioner, academic and learning and technology adoption theories from databases (MEDLINE, Scopus, CINAHL, Embase, Education Resources Information Center, PsycINFO and Web of Science), practitioner journals, snowballing and grey literature. Information regarding contexts, mechanisms and outcomes will be extracted. A narrative synthesis will determine overlapping configurations and form an initial theory. Second, the theory will be tested using empirical evidence located from the above databases and identified from the first search. Quality will be assessed using the Mixed Methods Appraisal Tool (MMAT), and relevant information will be extracted into a coding sheet. Third, the extracted information will be compared with the initial programme theory, with differences helping to make refinements. Findings will be presented as a narrative summary, and the MMAT will determine our confidence in each configuration.

ETHICS AND DISSEMINATION

Ethics approval is not required. This review will develop an evidence-informed programme theory. The results will inform and support AR/VR interventions from clinical educators, healthcare providers and software developers. Upskilling through AR/VR learning interventions may improve quality of care and promote evidence-based practice and continued learning. Findings will be disseminated through conference presentations and peer-reviewed journal articles.