Toolkits for implementing and evaluating digital health: A systematic review of rigor and reporting

Semantic Interoperability of Electronic Health Records: Systematic Review of Alternative Approaches for Enhancing Patient Information Availability

Background

Semantic interoperability facilitates the exchange of and access to health data that are being documented in electronic health records (EHRs) with various semantic features. The main goals of semantic interoperability development entail patient data availability and use in diverse EHRs without a loss of meaning. Internationally, current initiatives aim to enhance semantic development of EHR data and, consequently, the availability of patient data. Interoperability between health information systems is among the core goals of the European Health Data Space regulation proposal and the World Health Organization's Global Strategy on Digital Health 2020-2025.

Objective

To achieve integrated health data ecosystems, stakeholders need to overcome challenges of implementing semantic interoperability elements. To research the available scientific evidence on semantic interoperability development, we defined the following research questions: What are the key elements of and approaches for building semantic interoperability integrated in EHRs? What kinds of goals are driving the development? and What kinds of clinical benefits are perceived following this development?

Methods

Our research questions focused on key aspects and approaches for semantic interoperability and on possible clinical and semantic benefits of these choices in the context of EHRs. Therefore, we performed a systematic literature review in PubMed by defining our study framework based on previous research.

Results

Our analysis consisted of 14 studies where data models, ontologies, terminologies, classifications, and standards were applied for building interoperability. All articles reported clinical benefits of the selected approach to enhancing semantic interoperability. We identified 3 main categories: increasing the availability of data for clinicians (n=6, 43%), increasing the quality of care (n=4, 29%), and enhancing clinical data use and reuse for varied purposes (n=4, 29%). Regarding semantic development goals, data harmonization and developing semantic interoperability between different EHRs was the largest category (n=8, 57%). Enhancing health data quality through standardization (n=5, 36%) and developing EHR-integrated tools based on interoperable data (n=1, 7%) were the other identified categories. The results were closely coupled with the need to build usable and computable data out of heterogeneous medical information that is accessible through various EHRs and databases (eg, registers).

Conclusions

When heading toward semantic harmonization of clinical data, more experiences and analyses are needed to assess how applicable the chosen solutions are for semantic interoperability of health care data. Instead of promoting a single approach, semantic interoperability should be assessed through several levels of semantic requirements A dual model or multimodel approach is possibly usable to address different semantic interoperability issues during development. The objectives of semantic interoperability are to be achieved in diffuse and disconnected clinical care environments. Therefore, approaches for enhancing clinical data availability should be well prepared, thought out, and justified to meet economically sustainable and long-term outcomes.

Physical activity for people with chronic conditions: a systematic review of toolkits to promote adherence

PURPOSE

The World Health Organisation (WHO) recommends that health professionals develop and implement "adherence counselling toolkits" to promote adherence to long-term therapies in people with chronic conditions. This prospectively registered review aimed to systematically identify and evaluate existing toolkits developed to promote adherence to physical activity in people with chronic conditions.

MATERIALS AND METHODS

Grey literature and six e-databases were searched for studies investigating the use of "toolkits" to promote adherence to physical activity or exercise recommendations in people with chronic conditions (Medline, PsycInfo, EmCare, Cochrane, CINAHL Plus, Pedro). A two-stage screening process was completed by two independent reviewers.

RESULTS

Five studies describing five toolkits were included. Three toolkits displayed all WHO recommended features, including information on adherence, a clinically useful way of using this information, and behavioural tools for maintaining habits. The included toolkits featured "adherence" to the intervention; however, this was not their primary aim. There were trends towards improved physical activity with some of the included toolkits.

CONCLUSIONS

There are a lack of rigorously developed toolkits that focus on adherence to physical activity in people with chronic conditions. Toolkits should be developed, tested, and implemented to improve adherence and outcomes for people with chronic conditions.

Defining and Scoping Participatory Health Informatics: An eDelphi Study

BACKGROUND

Health care has evolved to support the involvement of individuals in decision making by, for example, using mobile apps and wearables that may help empower people to actively participate in their treatment and health monitoring. While the term "participatory health informatics" (PHI) has emerged in literature to describe these activities, along with the use of social media for health purposes, the scope of the research field of PHI is not yet well defined.

OBJECTIVE

This article proposes a preliminary definition of PHI and defines the scope of the field.

METHODS

We used an adapted Delphi study design to gain consensus from participants on a definition developed from a previous review of literature. From the literature we derived a set of attributes describing PHI as comprising 18 characteristics, 14 aims, and 4 relations. We invited researchers, health professionals, and health informaticians to score these characteristics and aims of PHI and their relations to other fields over three survey rounds. In the first round participants were able to offer additional attributes for voting.

RESULTS

The first round had 44 participants, with 28 participants participating in all three rounds. These 28 participants were gender-balanced and comprised participants from industry, academia, and health sectors from all continents. Consensus was reached on 16 characteristics, 9 aims, and 6 related fields.

DISCUSSION

The consensus reached on attributes of PHI describe PHI as a multidisciplinary field that uses information technology and delivers tools with a focus on individual-centered care. It studies various effects of the use of such tools and technology. Its aims address the individuals in the role of patients, but also the health of a society as a whole. There are relationships to the fields of health informatics, digital health, medical informatics, and consumer health informatics.

CONCLUSION

We have proposed a preliminary definition, aims, and relationships of PHI based on literature and expert consensus. These can begin to be used to support development of research priorities and outcomes measurements.

Translation, cultural adaptation and validity assessment of the Dutch version of the eHealth Literacy Questionnaire: a mixed-method approach

BACKGROUND

The digitalization of healthcare requires users to have sufficient competence in using digital health technologies. In the Netherlands, as well as in other countries, there is a need for a comprehensive, person-centered assessment of eHealth literacy to understand and address eHealth literacy related needs, to improve equitable uptake and use of digital health technologies.

OBJECTIVE

We aimed to translate and culturally adapt the original eHealth Literacy Questionnaire (eHLQ) to Dutch and to collect initial validity evidence.

METHODS

The eHLQ was translated using a systematic approach with forward translation, an item intent matrix, back translation, and consensus meetings with the developer. A validity-driven and multi-study approach was used to collect validity evidence on 1) test content, 2) response processes and 3) internal structure. Cognitive interviews (n = 14) were held to assess test content and response processes (Study 1). A pre-final eHLQ version was completed by 1650 people participating in an eHealth study (Study 2). A seven-factor Confirmatory Factor Analysis (CFA) model was fitted to the data to assess the internal structure of the eHLQ. Invariance testing was performed across gender, age, education and current diagnosis.

RESULTS

Cognitive interviews showed some problems in wording, phrasing and resonance with individual's world views. CFA demonstrated an equivalent internal structure to the hypothesized (original) eHLQ with acceptable fit indices. All items loaded substantially on their corresponding latent factors (range 0.51-0.81). The model was partially metric invariant across all subgroups. Comparison of scores between groups showed that people who were younger, higher educated and who had a current diagnosis generally scored higher across domains, however effect sizes were small. Data from both studies were triangulated, resulting in minor refinements to eight items and recommendations on use, score interpretation and reporting.

CONCLUSION

The Dutch version of the eHLQ showed strong properties for assessing eHealth literacy in the Dutch context. While ongoing collection of validity evidence is recommended, the evidence presented indicate that the eHLQ can be used by researchers, eHealth developers and policy makers to identify eHealth literacy needs and inform the development of eHealth interventions to ensure that people with limited digital access and skills are not left behind.

Paving the Way Forward for Evidence-Based Continuing Professional Development

ABSTRACT

Continuing professional development (CPD) fosters lifelong learning and enables health care providers to keep their knowledge and skills current with rapidly evolving health care practices. Instructional methods promoting critical thinking and decision making contribute to effective CPD interventions. The delivery methods influence the uptake of content and the resulting changes in knowledge, skills, attitudes, and behavior. Educational approaches are needed to ensure that CPD meets the changing needs of health care providers. This article examines the development approach and key recommendations embedded in a CE Educator's toolkit created to evolve CPD practice and foster a learning experience that promotes self-awareness, self-reflection, competency, and behavioral change. The Knowledge-to-Action framework was used in designing the toolkit. The toolkit highlighted three intervention formats: facilitation of small group learning, case-based learning, and reflective learning. Strategies and guidelines to promote active learning principles in CPD activities within different modalities and learning contexts were included. The goal of the toolkit is to assist CPD providers to design educational activities that optimally support health care providers' self-reflection and knowledge translation into their clinical environment and contribute to practice improvement, thus achieving the outcomes of the quintuple aim.

Digital health and capability maturity models-a critical thematic review and conceptual synthesis of the literature

OBJECTIVE

A literature review of capability maturity models (MMs) to inform the conceptualization, development, implementation, evaluation, and mainstreaming of MMs in digital health (DH).

METHODS

Electronic databases were searched using "digital health," "maturity models," and related terms based on the Digital Health Profile and Maturity Assessment Toolkit Maturity Model (DHPMAT-MM). Covidence was used to screen, identify, capture, and achieve consensus on data extracted by the authors. Descriptive statistics were generated. A thematic analysis and conceptual synthesis were conducted.

FINDINGS

Diverse domain-specific MMs and model development, implementation, and evaluation methods were found. The spread and pattern of different MMs verified the essential DH foundations and five maturity stages of the DHPMAT-MM. An unanticipated finding was the existence of a new category of community-facing MMs. Common characteristics included:1. A dynamic lifecycle approach to digital capability maturity, which is:a. responsive to environmental changes and may improve or worsen over time;b. accumulative, incorporating the attributes of the preceding stage; andc. sequential, where no maturity stage must be skipped.2. Sociotechnical quality improvement of the DH ecosystem and MM, which includes:a. investing in the organization's human, hardware, and software resources andb. a need to engage and improve the DH competencies of citizens.

CONCLUSIONS

The diversity in MMs and variability in methods and content can create cognitive dissonance. A metamodel like the DHPMAT-MM can logically unify the many domain-specific MMs and guide the overall implementation and evaluation of DH ecosystems and MMs over the maturity lifecycle.

Evaluation in Life Cycle of Information Technology (ELICIT) framework: Supporting the innovation life cycle from business case assessment to summative evaluation

OBJECTIVE

Our objective was to develop an evaluation framework for electronic health record (EHR)-integrated innovations to support evaluation activities at each of four information technology (IT) life cycle phases: planning, development, implementation, and operation.

METHODS

The evaluation framework was developed based on a review of existing evaluation frameworks from health informatics and other domains (human factors engineering, software engineering, and social sciences); expert consensus; and real-world testing in multiple EHR-integrated innovation studies.

RESULTS

The resulting Evaluation in Life Cycle of IT (ELICIT) framework covers four IT life cycle phases and three measure levels (society, user, and IT). The ELICIT framework recommends 12 evaluation steps: (1) business case assessment; (2) stakeholder requirements gathering; (3) technical requirements gathering; (4) technical acceptability assessment; (5) user acceptability assessment; (6) social acceptability assessment; (7) social implementation assessment; (8) initial user satisfaction assessment; (9) technical implementation assessment; (10) technical portability assessment; (11) long-term user satisfaction assessment; and (12) social outcomes assessment.

DISCUSSION

Effective evaluation requires a shared understanding and collaboration across disciplines throughout the entire IT life cycle. In contrast with previous evaluation frameworks, the ELICIT framework focuses on all phases of the IT life cycle across the society, user, and IT levels. Institutions seeking to establish evaluation programs for EHR-integrated innovations could use our framework to create such shared understanding and justify the need to invest in evaluation.

CONCLUSION

As health care undergoes a digital transformation, it will be critical for EHR-integrated innovations to be systematically evaluated. The ELICIT framework can facilitate these evaluations.

"Digital Health Diplomacy" in Global Digital Health? A call for critique and discourse

Actualizing the vision of Global Digital Health is a central issue on the Global Health Diplomacy agenda. The COVID-reinforced need for accelerated digital health progress will require political structures and processes to build a foundation for Global Digital Health. Simultaneously, Global Health Diplomacy uses digital technologies in its enactment. Both phenomena have driven interest in the term "Digital Health Diplomacy." A review of the literature revealed 2 emerging but distinct definitions that have been published very recently, each with its associated discourse and practice. This multiplicity of ideas demonstrates the myriad ways in which global digital and political systems are becoming increasingly entangled. Untangling these, this paper proposes and discusses 3 dimensions of Digital Health Diplomacy: "Diplomacy for digital health," "Digital health for diplomacy," and "Digital health in diplomacy." It calls upon digital health professionals, diplomats, political and social scientists, epidemiologists, and clinicians to discuss, critique, and advance this emerging domain.

Evaluation of a Digital Health Initiative in Illicit Substance Use: Cross-sectional Survey Study

Background

The Cracks in the Ice (CITI) community toolkit was developed to provide evidence-based, up-to-date information and resources about crystal methamphetamine to Australians. Given the high rates of internet use in the community and the potential for misinformation, CITI has the potential to play an important role in improving knowledge and challenging misconceptions surrounding crystal methamphetamine.

Objective

This study aims to determine (1) whether the CITI toolkit is achieving its aim of disseminating evidence-based information and resources to people who use crystal methamphetamine, their family and friends, health professionals, and the general community and (2) examine the association between the use of CITI and the knowledge and attitudes about crystal methamphetamine.

Methods

A cross-sectional web-based survey, open to Australian residents (aged ≥18 years), was conducted from November 2018 to March 2019. People who had previously visited the website (referred to as “website visitors” in this study) and those who had not (“naïve”) were recruited. At baseline, knowledge, attitudes, and demographics were assessed. CITI website visitors then completed a series of site evaluation questions, including the System Usability Scale (SUS), and naïve participants were asked to undertake a guided site tour of a replicated version of the site before completing the evaluation questions and repeating knowledge and attitude scales.

Results

Of a total 2108 participants, 564 (26.7%) reported lifetime use of crystal methamphetamine, 434 (20.6%) were family/friends, 288 (13.7%) were health professionals, and 822 (38.9%) were community members. The average SUS score was 73.49 (SD 13.30), indicating good site usability. Health professionals reported significantly higher SUS scores than community members (P=.02) and people who used crystal methamphetamine (P<.001). Website visitors had significantly higher baseline knowledge than naïve participants (P<.001). Among naïve participants, knowledge scores increased following exposure to the website (mean 15.2, SE 0.05) compared to baseline (mean 14.4, SE 0.05; P<.001). The largest shifts in knowledge were observed for items related to prevalence, legal issues, and the effects of the drug. Stigmatizing attitude scores among the naïve group were significantly lower following exposure to CITI (mean 41.97, SE 0.21) compared to baseline (mean 44.3, SE 0.21; P<.001).

Conclusions

This study provides an innovative evaluation of a national eHealth resource. CITI is achieving its aim of disseminating evidence-based, nonstigmatizing, and useful information and resources about crystal methamphetamine to key end user groups and has received good usability scores across its target groups. Interaction with CITI led to immediate improvements in knowledge about crystal methamphetamine and a decrease in stigmatizing attitudes. CITI demonstrates the important role of digital information and support platforms for translating evidence into practice and improving knowledge and reducing stigma.