Digital health and capability maturity models-a critical thematic review and conceptual synthesis of the literature

OBJECTIVE

A literature review of capability maturity models (MMs) to inform the conceptualization, development, implementation, evaluation, and mainstreaming of MMs in digital health (DH).

METHODS

Electronic databases were searched using "digital health," "maturity models," and related terms based on the Digital Health Profile and Maturity Assessment Toolkit Maturity Model (DHPMAT-MM). Covidence was used to screen, identify, capture, and achieve consensus on data extracted by the authors. Descriptive statistics were generated. A thematic analysis and conceptual synthesis were conducted.

FINDINGS

Diverse domain-specific MMs and model development, implementation, and evaluation methods were found. The spread and pattern of different MMs verified the essential DH foundations and five maturity stages of the DHPMAT-MM. An unanticipated finding was the existence of a new category of community-facing MMs. Common characteristics included:1. A dynamic lifecycle approach to digital capability maturity, which is:a. responsive to environmental changes and may improve or worsen over time;b. accumulative, incorporating the attributes of the preceding stage; andc. sequential, where no maturity stage must be skipped.2. Sociotechnical quality improvement of the DH ecosystem and MM, which includes:a. investing in the organization's human, hardware, and software resources andb. a need to engage and improve the DH competencies of citizens.

CONCLUSIONS

The diversity in MMs and variability in methods and content can create cognitive dissonance. A metamodel like the DHPMAT-MM can logically unify the many domain-specific MMs and guide the overall implementation and evaluation of DH ecosystems and MMs over the maturity lifecycle.

National monitoring and evaluation of eHealth: a scoping review

OBJECTIVE

There has been substantial growth in eHealth over the past decade, driven by expectations of improved healthcare system performance. Despite substantial eHealth investment, little is known about the monitoring and evaluation strategies for gauging progress in eHealth availability and use. This scoping review aims to map the existing literature and depict the predominant approaches and methodological recommendations to national and regional monitoring and evaluation of eHealth availability and use, to advance national strategies for monitoring and evaluating eHealth.

METHODS

Peer-reviewed and grey literature on monitoring and evaluation of eHealth availability and use published between January 1, 2009, and March 11, 2019, were eligible for inclusion. A total of 2354 publications were identified and 36 publications were included after full-text review. Data on publication type (eg, empirical research), country, level (national or regional), publication year, method (eg, survey), and domain (eg, provider-centric electronic record) were charted.

RESULTS

The majority of publications monitored availability alone or applied a combination of availability and use measures. Surveys were the most common data collection method (used in 86% of the publications). Organization for Economic Co-operation and Development (OECD), European Commission, Canada Health Infoway, and World Health Organization (WHO) have developed comprehensive eHealth monitoring and evaluation methodology recommendations.

DISCUSSION

Establishing continuous national eHealth monitoring and evaluation, based on international approaches and recommendations, could improve the ability for cross-country benchmarking and learning. This scoping review provides an overview of the predominant approaches to and recommendations for national and regional monitoring and evaluation of eHealth. It thereby provides a starting point for developing national eHealth monitoring strategies.

Developing a Workflow Composite Score to Measure Clinical Information Logistics. A Top-down Approach

BACKGROUND

Availability and usage of individual IT applications have been studied intensively in the past years. Recently, IT support of clinical processes is attaining increasing attention. The underlying construct that describes the IT support of clinical workflows is clinical information logistics. This construct needs to be better understood, operationalised and measured.

OBJECTIVES

It is therefore the aim of this study to propose and develop a workflow composite score (WCS) for measuring clinical information logistics and to examine its quality based on reliability and validity analyses.

METHODS

We largely followed the procedural model of MacKenzie and colleagues (2011) for defining and conceptualising the construct domain, for developing the measurement instrument, assessing the content validity, pretesting the instrument, specifying the model, capturing the data and computing the WCS and testing the reliability and validity.

RESULTS

Clinical information logistics was decomposed into the descriptors data and information, function, integration and distribution, which embraced the framework validated by an analysis of the international literature. This framework was refined selecting representative clinical processes. We chose ward rounds, pre- and post-surgery processes and discharge as sample processes that served as concrete instances for the measurements. They are sufficiently complex, represent core clinical processes and involve different professions, departments and settings. The score was computed on the basis of data from 183 hospitals of different size, ownership, location and teaching status. Testing the reliability and validity yielded encouraging results: the reliability was high with r(split-half) = 0.89, the WCS discriminated between groups; the WCS correlated significantly and moderately with two EHR models and the WCS received good evaluation results by a sample of chief information officers (n = 67). These findings suggest the further utilisation of the WCS.

CONCLUSION

As the WCS does not assume ideal workflows as a gold standard but measures IT support of clinical workflows according to validated descriptors a high portability of the WCS to other hospitals in other countries is very likely. The WCS will contribute to a better understanding of the construct clinical information logistics.

From bed to bench: bridging from informatics practice to theory: an exploratory analysis

BACKGROUND

In 2009, Applied Clinical Informatics (ACI)--focused on applications in clinical informatics--was launched as a companion journal to Methods of Information in Medicine (MIM). Both journals are official journals of the International Medical Informatics Association.

OBJECTIVES

To explore which congruencies and interdependencies exist in publications from theory to practice and from practice to theory and to determine existing gaps. Major topics discussed in ACI and MIM were analyzed. We explored if the intention of publishing companion journals to provide an information bridge from informatics theory to informatics practice and vice versa could be supported by this model. In this manuscript we will report on congruencies and interdependences from practice to theory and on major topics in MIM.

METHODS

Retrospective, prolective observational study on recent publications of ACI and MIM. All publications of the years 2012 and 2013 were indexed and analyzed.

RESULTS

Hundred and ninety-six publications were analyzed (ACI 87, MIM 109). In MIM publications, modelling aspects as well as methodological and evaluation approaches for the analysis of data, information, and knowledge in biomedicine and health care were frequently raised - and often discussed from an interdisciplinary point of view. Important themes were ambient-assisted living, anatomic spatial relations, biomedical informatics as scientific discipline, boosting, coding, computerized physician order entry, data analysis, grid and cloud computing, health care systems and services, health-enabling technologies, health information search, health information systems, imaging, knowledge-based decision support, patient records, signal analysis, and web science. Congruencies between journals could be found in themes, but with a different focus on content. Interdependencies from practice to theory, found in these publications, were only limited.

CONCLUSIONS

Bridging from informatics theory to practice and vice versa remains a major component of successful research and practice as well as a major challenge.

What Are Complex eHealth Innovations and How Do You Measure Them? Position Paper

OBJECTIVES

eHealth and innovation are often regarded as synonyms - not least because eHealth technologies and applications are new to their users. This position paper challenges this view and aims at exploring the nature of eHealth innovation against the background of common definitions of innovation and facts from the biomedical and health informatics literature. A good understanding of what constitutes innovative eHealth developments allows the degree of innovation to be measured and interpreted.

METHODS

To this end, relevant biomedical and health informatics literature was searched mainly in Medline and ACM digital library. This paper presents seven facts about implementing and applying new eHealth developments hereby drawing on the experience published in the literature.

RESULTS

The facts are: 1. eHealth innovation is relative. 2. Advanced clinical practice is the yardstick. 3. Only used and usable eHealth technology can give birth to eHealth innovatio. 4. One new single eHealth function does not make a complex eHealth innovation. 5. eHealth innovation is more evolution than revolution. 6. eHealth innovation is often triggered behind the scenes; and 7. There is no eHealth innovation without sociocultural change.

CONCLUSIONS

The main conclusion of the seven facts is that eHealth innovations have many ingredients: newness, availability, advanced clinical practice with proven outcomes, use and usability, the supporting environment, other context factors and the stakeholder perspectives. Measuring eHealth innovation is thus a complex matter. To this end we propose the development of a composite score that expresses comprehensively the nature of eHealth innovation and that breaks down its complexity into the three dimensions: i) eHealth adoption, ii) partnership with advanced clinical practice, and iii) use and usability of eHealth. In order to better understand the momentum and mechanisms behind eHealth innovation the fourth dimension, iv) eHealth supporting services and means, needs to be studied. Conceptualising appropriate measurement instruments also requires eHealth innovation to be distinguished from eHealth sophistication, performance and quality, although innovation is intertwined with these concepts. The demanding effort for defining eHealth innovation and measuring it properly seem worthwhile and promise advances in creating better systems. This paper thus intends to stimulate the necessary discussion.