Assessing the Quality and Impact of eHealth Tools: Systematic Literature Review and Narrative Synthesis

Telehealth development in the WHO European region: Results from a quantitative survey and insights from Norway

BACKGROUND

The COVID-19 pandemic sent shock waves through societies, economies, and health systems of Member States in the WHO European Region and beyond. During the pandemic, most countries transitioned from a slow to a rapid adoption of telehealth solutions, to accommodate the public health and social measures introduced to mitigate the spread of the disease. As countries shift to a post-pandemic world, the question remains whether telehealth's importance as a mode of care provision in Europe continues to be significant.

OBJECTIVE

This paper aims to present, synthesize, and interpret results from the Telehealth Programmes section of the 2022 WHO Survey on Digital Health (2022 WHO/Europe DH Survey). We specifically analyze the implementation and use of teleradiology, telemedicine, and telepsychiatry. Norwegian telehealth experiences will be used to illustrate survey findings, and we discuss some of the relevant barriers and facilitators that impact the use of telehealth services.

METHODS

The survey tool was revised from the 2015 WHO Global Survey on eHealth, updated to reflect recent progress and policy priorities.The 2022 WHO/Europe DH Survey was conducted by WHO and circulated to Member States in its European Region from April to October 2022.

RESULTS

The data analysis revealed that teleradiology, telemedicine, and telepsychiatry are the telehealth services most commonly used in the WHO European Region in 2022. Funding remains the most significant barrier to the implementation of telehealth in the Region, followed by infrastructure and capacity/human resources. The survey results highlighted in this study are presented in the following sections: (1) telehealth strategies and financing, (2) telehealth programmes and services offered by Member States of the WHO European Region, (3) barriers to implementing telehealth services, and (4) monitoring and evaluation of telehealth.

CONCLUSION

Based on WHO's 2022 survey, the use of telehealth in the WHO European Region is on the rise. However, merely having telehealth in place is not sufficient for its successful and sustained use for care provision. Responses also uncovered regional differences and barriers that need to be overcome. Successful implementation and scaling of telehealth requires rethinking the design of health and social care systems to create robust, trustworthy, and person-centred digital health and care services.

Clinical Validation of Digital Healthcare Solutions: State of the Art, Challenges and Opportunities

Digital health technologies (DHTs) at the intersection of health, medical informatics, and business aim to enhance patient care through personalised digital approaches. Ensuring the efficacy and reliability of these innovations demands rigorous clinical validation. A PubMed literature review (January 2006 to July 2023) identified 1250 papers, highlighting growing academic interest. A focused narrative review (January 2018 to July 2023) delved into challenges, highlighting issues such as diverse regulatory landscapes, adoption issues in complex healthcare systems, and a plethora of evaluation frameworks lacking pragmatic guidance. Existing frameworks often omit crucial criteria, neglect empirical evidence, and clinical effectiveness is rarely included as a criterion for DHT quality. The paper underscores the urgency of addressing challenges in accreditation, adoption, business models, and integration to safeguard the quality, efficacy, and safety of DHTs. A pivotal illustration of collaborative efforts to address these challenges is exemplified by the Digital Health Validation Center, dedicated to generating clinical evidence of innovative healthcare technologies and facilitating seamless technology transfer. In conclusion, it is necessary to harmonise evaluation approaches and frameworks, improve regulatory clarity, and commit to collaboration to integrate rigorous clinical validation and empirical evidence throughout the DHT life cycle.

Exploring the Role of Complexity in Health Care Technology Bottom-Up Innovations: Multiple-Case Study Using the Nonadoption, Abandonment, Scale-Up, Spread, and Sustainability Complexity Assessment Tool

BACKGROUND

New digital technology presents new challenges to health care on multiple levels. There are calls for further research that considers the complex factors related to digital innovations in complex health care settings to bridge the gap when moving from linear, logistic research to embracing and testing the concept of complexity. The nonadoption, abandonment, scale-up, spread, and sustainability (NASSS) framework was developed to help study complexity in digital innovations.

OBJECTIVE

This study aims to investigate the role of complexity in the development and deployment of innovations by retrospectively assessing challenges to 4 digital health care innovations initiated from the bottom up.

METHODS

A multicase retrospective, deductive, and explorative analysis using the NASSS complexity assessment tool LONG was conducted. In total, 4 bottom-up innovations developed in Region Västra Götaland in Sweden were explored and compared to identify unique and shared complexity-related challenges.

RESULTS

The analysis resulted in joint insights and individual learning. Overall, the complexity was mostly found outside the actual innovation; more specifically, it related to the organization's readiness to integrate new innovations, how to manage and maintain innovations, and how to finance them. The NASSS framework sheds light on various perspectives that can either facilitate or hinder the adoption, scale-up, and spread of technological innovations. In the domain of condition or diagnosis, a well-informed understanding of the complexity related to the condition or illness (diabetes, cancer, bipolar disorders, and schizophrenia disorders) is of great importance for the innovation. The value proposition needs to be clearly described early to enable an understanding of costs and outcomes. The questions in the NASSS complexity assessment tool LONG were sometimes difficult to comprehend, not only from a language perspective but also due to a lack of understanding of the surrounding organization's system and its setting.

CONCLUSIONS

Even when bottom-up innovations arise within the same support organization, the complexity can vary based on the developmental phase and the unique characteristics of each project. Identifying, defining, and understanding complexity may not solve the issues but substantially improves the prospects for successful deployment. Successful innovation within complex organizations necessitates an adaptive leadership and structures to surmount cultural resistance and organizational impediments. A rigid, linear, and stepwise approach risks disregarding interconnected variables and dependencies, leading to suboptimal outcomes. Success lies in embracing the complexity with its uncertainty, nurturing creativity, and adopting a nonlinear methodology that accommodates the iterative nature of innovation processes within complex organizations.

Health Care Professionals' Experiences With a Mobile Self-Care Solution for Low Complex Orthopedic Injuries: Mixed Methods Study

BACKGROUND

To cope with the rising number of patients with trauma in an already constrained Dutch health care system, Direct Discharge (DD) has been introduced in over 25 hospitals in the Netherlands since 2019. With DD, no routine follow-up appointments are scheduled after the emergency department (ED) visit, and patients are supported through information leaflets, a smartphone app, and a telephone helpline. DD reduces secondary health care use, with comparable patient satisfaction and primary health care use. Currently, little is known about the experiences of in-hospital health care professionals with DD.

OBJECTIVE

The aim of this study was to explore the experiences of health care professionals with the DD protocol to enhance durable adoption and improve the protocol.

METHODS

We conducted a mixed methods study parallel to the implementation of DD in 3 hospitals. Data were collected through a preimplementation survey, a postimplementation survey, and semistructured interviews. Quantitative data were reported descriptively, and qualitative data were reported using thematic analysis. Outcomes included the Bowen feasibility parameters: implementation, acceptability, preliminary efficacy, demand, and applicability. Preimplementation expectations were compared with postimplementation experiences. Health care professionals involved in the daily clinical care of patients with low-complex, stable injuries were eligible for this study.

RESULTS

Of the 217 eligible health care professionals, 128 started the primary survey, 37 completed both surveys (response rate of 17%), and 15 participated in semistructured interviews. Health care professionals expressed satisfaction with the DD protocol (median 7.8, IQR 6.8-8.9) on a 10-point scale, with 82% (30/37) of participants noting improved information quality and uniformity and 73% (27/37) of patients perceiving reduced outpatient follow-up and imaging. DD was perceived as safe by 79% (28/37) of participants in its current form, but a feedback system to reassure health care professionals that patients had recovered adequately was suggested to improve DD. The introduction of DD had varying effects on workload and job satisfaction among different occupations. Health care professionals expressed intentions to continue using DD due to increased efficiency, patient empowerment, and self-management.

CONCLUSIONS

Health care professionals perceive DD as an acceptable, applicable, safe, and efficacious alternative to traditional treatment. A numerical in-app feedback system (eg, in-app communication tools or recovery scores) could alleviate health care professionals' concerns about adequate recovery and further improve DD protocols. DD can reduce health care use, which is important in times of constrained resources. Nonetheless, both advantages and disadvantages should be considered while evaluating this type of treatment. In the future, clinicians and policy makers can use these insights to further optimize and implement DD in clinical practice and guidelines.

The Effectiveness of Preoperative Outpatient and Home Rehabilitation and the Impact on the Results of Hip Arthroplasty: Introductory Report

A total hip replacement is the treatment of choice for end-stage hip osteoarthritis. Rehabilitation performed before surgery (called prehabilitation) is used to improve the results of surgical treatment. However, the results of studies have not unquestionably confirmed the effectiveness of preoperative rehabilitation and its impact on the outcome of surgery. The aim of this study is to assess the effectiveness of preoperative outpatient and home rehabilitation in relation to a control group not subject to these forms of influence. A total of 61 patients qualified for primary hip arthroplasty were randomly assigned to a group with outpatient rehabilitation before surgery, exercises performed at home, or a group without any intervention before surgery. Three weeks after surgery, the patients were re-qualified and underwent three weeks of outpatient rehabilitation in the day rehabilitation department. The patients from all three groups were evaluated in terms of functionality and pain using point scales upon enrolment in the study, on admission to the day rehabilitation department, and after 3 weeks of rehabilitation in the department. A total of 50 subjects completed the study. The study results did not reveal statistically significant differences between preoperative rehabilitation and no intervention. Patients rehabilitated at home gave up self-therapy more often than those undergoing outpatient rehabilitation.

Home-based exercise interventions delivered by technology in older adults: A scoping review of technological tools usage

BACKGROUND

Despite technology-based systems being considered promising tools to stimulate and increase physical function at home, most older adults are unfamiliar with technology, which may pose some difficulties. Technology-related parameters, such as adherence, acceptance, and acceptability, are crucial to achieving higher efficacy levels of home-based exercise interventions delivered by technology. In this scoping review, we aimed to revise the use of home-based technological tools to improve physical function in the older population, focusing on the user's experience and perspective. Methods This scoping review was conducted following PRISMA guidelines. The search was conducted in April 2022 and updated in April 2023. A total of 45 studies were included in the review. Results Most studies (95.5%) met the technology usage levels defined by the research team or reported satisfactory technology usage levels. Positive health-related outcomes were reported in 80% of studies. Although the existence of guidelines to correctly define and use measures associated with technology use, including adherence, acceptance and acceptability, some terms are still being used interchangeably. Some concerns related to the lack of an international consensus regarding technology usage measures and the exclusion of older adults who did not own or have previous experience with technology in a large percentage of the included studies may have limited the results obtained. Conclusions Altogether, home-based exercise interventions delivered through technology were associated with positive health-related outcomes in older adults, and technology usage levels are considered satisfactory. Older adults are willing and able to use technology autonomously if adequate support is provided.

A sociotechnical framework to assess patient-facing eHealth tools: results of a modified Delphi process

Among the thousands of eHealth tools available, the vast majority do not get past pilot phases because they cannot prove value, and only a few have been systematically assessed. Although multiple eHealth assessment frameworks have been developed, these efforts face multiple challenges. This study aimed to address some of these challenges by validating and refining an initial list of 55 assessment criteria based on previous frameworks through a two-round modified Delphi process with in-between rounds of interviews. The expert panel (n = 57) included participants from 18 countries and 9 concerned parties. A consensus was reached on 46 criteria that were classified into foundational and contextual criteria. The 36 foundational criteria focus on evaluating the eHealth tool itself and were grouped into nine clusters: technical aspects, clinical utility and safety, usability and human centricity, functionality, content, data management, endorsement, maintenance, and developer. The 10 contextual criteria focus on evaluating the factors that vary depending on the context the tool is being evaluated for and were grouped into seven clusters: data-protection compliance, safety regulatory compliance, interoperability and data integration, cultural requirements, affordability, cost-benefit, and implementability. The classification of criteria into foundational and contextual helps us assess not only the quality of an isolated tool, but also its potential fit in a specific setting. Criteria subscales may be particularly relevant when determining the strengths and weaknesses of the tool being evaluated. This granularity enables different concerned parties to make informed decisions about which tools to consider according to their specific needs and priorities.

Electronic Medication Reconciliation Tools Aimed at Healthcare Professionals to Support Medication Reconciliation: a Systematic Review

The development of health information technology available and accessible to professionals is increasing in the last few years. However, a low number of electronic health tools included some kind of information about medication reconciliation. To identify all the electronic medication reconciliation tools aimed at healthcare professionals and summarize their main features, availability, and clinical impact on patient safety. A systematic review of studies that included a description of an electronic medication reconciliation tool (web-based or mobile app) aimed at healthcare professionals was conducted. The review protocol was registered with PROSPERO: registration number CRD42022366662, and followed PRISMA guidelines. The literature search was performed using four healthcare databases: PubMed, EMBASE, Cochrane Library, and Scopus with no language or publication date restrictions. We identified a total of 1227 articles, of which only 12 met the inclusion criteria.Through these articles,12 electronic tools were detected. Viewing and comparing different medication lists and grouping medications into multiple categories were some of the more recurring features of the tools. With respect to the clinical impact on patient safety, a reduction in adverse drug events or medication discrepancies was detected in up to four tools, but no significant differences in emergency room visits or hospital readmissions were found. 12 e-MedRec tools aimed at health professionals have been developed to date but none was designed as a mobile app. The main features that healthcare professionals requested to be included in e-MedRec tools were interoperability, "user-friendly" information, and integration with the ordering process.

Assessment of healthcare application quality: Development of a standardized methods for healthcare professionals

The advancement of the mobile app market is reshaping healthcare, emphasizing the imperative for quality and efficacy in health applications. This methodology has been devised to assess mobile health applications, aiming to assist healthcare professionals in selecting apps for e-healthcare consumers. Key facets of this approach are: •A stringent selection process within mobile app stores•A standardized assessment using the Mobile App Rating Scale to achieve consistent and replicable evaluations, systematically organizing app evaluations•A comprehensive framework guiding healthcare practitioners in determining which apps to integrate into clinical practice and which to exclude Central to this method is the emphasis on distinguishing apps that enhance clinical practice from those that fall short in important areas such as the effectiveness of proposed health features, data accuracy, adherence to clinical guidelines, data security, and user privacy. With heightened attention to usability and accessibility, the methodology also addresses the common risks inherent in mHealth implementation, ensuring that selected apps not only meet technical criteria but also align with the broader healthcare ecosystem's needs and challenges.

The eHealth usage during COVID-19 pandemic 2020 year-Case of Poland

According to the DESI 2022 digital economy and society ranking, Poland still ranks in one of the last position. Although, in digitising healthcare Poland has made significant progress over the last five years, some inequities in the usage of eHealth have been recognised. This has become an especially important topic after the COVID-19 pandemic. Suddenly, eHealth innovations were much needed to maintain the accessibility of healthcare. Thus, the aim of this study was to explore determinants of eHealth usage by Poles and identify existing potential barriers. Data was collected from the databases of Statistic Poland and statistical methods were employed in this research. The results showed that five variables such as Internet access, Internet use, Internet skills and average monthly disposable income per capita in PLN, along with the number of practicing physicians per capita were important determinants explaining eHealth usage by the analysed Poles between the age of 16-74. The findings showed to increase the usage of eHealth, health policy makers should ensure that Poles acquire and improve Internet skills. Based on results of the research an extended model of eHealth development in Poland, consisting of a central governmental institution and local facilities coordinating remote electronic services, collecting statistical data and providing educational campaigns, was proposed as well.

International practices in health technology assessment and public financing of digital health technologies: recommendations for Hungary

Background

Evaluating and integrating digital health technologies is a critical component of a national healthcare ecosystem in the 2020s and is expected to even increase in significance.

Design

The paper gives an overview of international practices on public financing and health technology assessment of digital health technologies (DHTs) in five European Union (EU) countries and outlines recommendations for country-level action that relevant stakeholders can consider in order to support uptake of digital health solutions in Hungary. A scoping review was carried out to identify and gather country-specific classifications and international practices on the financing DHTs in five pioneering EU countries: Germany, France, Belgium, the United Kingdom and Finland.

Results

Several frameworks have been developed for DHTs, however there is no single, unified framework or method for classification, evaluation, and financing of digital health technologies in European context. European countries apply different taxonomy, use different assessment domains and regulations for the reimbursement of DHTs. The Working Group of the Hungarian Health Economic Society recommends eight specific points for stakeholders, importantly taking active role in shaping common clinical evidence standards and technical quality criteria across in order for common standards to be developed in the European Union single market.

Conclusion

Specificities of national healthcare contexts must be taken into account in decisions to allocate public funds to certain therapies rather than others.