Developing evaluation capacities in integrated care projects: Lessons from a scientific support mission implemented in Belgium

The capacity of self-assessment, to learn from experience, to make information-based decisions, and to adapt over time are essential drivers of success for any project aiming at healthcare system change. Yet, many of those projects are managed by healthcare providers' teams with little evaluation capacity. In this article, we describe the support mission delivered by an interdisciplinary scientific team to 12 integrated care pilot projects in Belgium, mobilizing a set of tools and methods: a dashboard gathering population health indicators, a significant event reporting method, an annual report, and the development of a sustainable "learning community." The article provides a reflexive return on the design and implementation of such interventions aimed at building organizational evaluation capacity. Some lessons were drawn from our experience, in comparison with the broader evaluation literature: The provided support should be adapted to the various needs and contexts of the beneficiary organizations, and it has to foster experience-based learning and requires all stakeholders to adopt a learning posture. A long-time, secure perspective should be provided for organizations, and the availability of data and other resources is an essential precondition for successful work.

Telemedicine as a Means to an End, Not an End in Itself

Telemedicine (TM)—the management of disease at a distance—has potential usefulness for patients with advanced respiratory disease. Underscoring this potential is the dramatic expansion of its applications in clinical medicine. However, since clinical studies testing this intervention often provide heterogeneous results, its role in the medical management of respiratory disorders remains inconclusive. A major problem in establishing TM’s effectiveness is that it is not a single intervention; rather, it includes a number of divergent diagnostic and therapeutic modalities—and each must be tested separately. Reflecting the discord between the need for further documentation of its approaches and effectiveness and its rapid utilization without this needed information, a major challenge is the lack of international guidelines for its integration, regulation, operational plans, and guidance for professionals. Tailored TM, with increased flexibility to address differing healthcare contexts, has the potential to improve access to and quality of services while reducing costs and direct input by health professionals. We should view TM as a tool to aid healthcare professionals in managing their patients with respiratory diseases rather than as a stand-alone substitute to traditional medical care. As such, TM is a means rather than an end.

Assessment of the implementation fidelity of a strategy to scale up integrated care in five European regions: a multimethod study

OBJECTIVE

The SCaling IntegRated Care in COntext (SCIROCCO) project tested a step-based scaling up strategy to explore what and how to scale up integrated care initiatives in five European regions. To gain a profound understanding of which factors influence the implementation of this strategy, the objective of this study was to assess the extent to which the SCIROCCO strategy was implemented as planned.

DESIGN

Multimethod study.

METHODS

The extended version of the conceptual framework for implementation fidelity was used to evaluate what factors influence the implementation of the scaling up strategy. Data were collected in the five participating European regions during the intervention period. Data collection methods included: key informant interviews, focus groups, questionnaire studies and project documents.

RESULTS

All three main steps of the scaling up strategy were implemented with acceptable fidelity. Variations were observed in the duration of implementing the steps in the regions. Also, variations were observed in the coverage of experts to participate in the steps of the strategy. Several factors were observed to influence the implementation: facilitation conditions (ie, good coordination for implementation) and participant responsiveness (ie, a positive experience of participants in the organised study visits). Factors that may have moderated adherence to the original plan of the strategy were found in facilitating conditions (ie, in the flexible approach), participant recruitment factors (ie, adaptions of the procedure by the regions) and contextual factors (ie, the level of development of integrated care).

CONCLUSION

This was the first study to assess implementation fidelity of a European project that used a step-based scaling up strategy in five European regions. Similar European projects that are based on collaboration between several European regions can learn from the lessons captured in SCIROCCO and can become more aware of the facilitating factors and pitfalls of implementing such projects.

General practitioners' perspectives on care coordination in primary health care: A qualitative study

Introduction

To explore Norwegian general practitioners' experiences with care coordination in primary health care.

Methods

Qualitative study using data from five focus groups with 32 general practitioners in Norway. We analysed the data using systematic text condensation, a descriptive and explorative method for thematic cross-case analysis of qualitative data.

Results

The general practitioners had different notions of care pathways. They expressed a wish and an obligation to be involved in planning and coordination of primary health-care services, but they experienced organisational and financial barriers that limited their involvement and contribution. General practitioners reported lack of information about and few opportunities for involvement in formal coordination initiatives, and they missed informal arenas for dialogue with other primary health-care professionals. They argued that the general practitioner's role as coordinator should be recognised by other parties and that they needed financial compensation for contributions and attendance in meetings with the municipality.

Discussion

General practitioners need informal arenas for dialogue with other primary health-care professionals and access to relevant information to promote coordinated care. There might be an untapped potential for improving patient care involving general practitioners more in planning and coordinating services at the system level. Financial compensation of general practitioners contribution may promote increased involvement by general practitioners.

An Instrument to Measure Maturity of Integrated Care: A First Validation Study

INTRODUCTION

Lessons captured from interviews with 12 European regions are represented in a new instrument, the B3-Maturity Model (B3-MM). B3-MM aims to assess maturity along 12 dimensions reflecting the various aspects that need to be managed in order to deliver integrated care. The objective of the study was to test the content validity of B3-MM as part of SCIROCCO (Scaling Integrated Care into Context), a European Union funded project.

METHODS

A literature review was conducted to compare B3-MM's 12 dimensions and their measurement scales with existing measures and instruments that focus on assessing the development of integrated care. Subsequently, a three-round survey conducted through a Delphi study with international experts in the field of integrated care was performed to test the relevance of: 1) the dimensions, 2) the maturity indicators and 3) the assessment scale used in B3-MM.

RESULTS

The 11 articles included in the literature review confirmed all the dimensions described in the original version of B3-MM. The Delphi study rounds resulted in various phrasing amendments of indicators and assessment scale. Full agreement among the experts on the relevance of the 12 B3-MM dimensions, their indicators, and assessment scale was reached after the third Delphi round.

CONCLUSION AND DISCUSSION

The B3-MM dimensions, maturity indicators and assessment scale showed satisfactory content validity. While the B3-MM is a unique instrument based on existing knowledge and experiences of regions in integrated care, further testing is needed to explore other measurement properties of B3-MM.

Integration as a scientific paradigm

Purpose

Integration is policy, practice as well as object of systematic investigation. What we do not know is whether or not integration can be understood as a science. In his book The Structure of Scientific Revolutions, Thomas Kuhn formulated a notion of (natural) sciences based on the emergence of commitments amongst a community of scientists to a set of logics, model and exemplars. He called this a paradigm. The purpose of this paper is to assess the scientific nature of integration by perceiving it as a paradigm in Kuhn’s sense.

Design/methodology/approach

The paper proceeds by conceptual reflection through matching existing components, theories and exemplifications of integration to Kuhn’s model of a scientific paradigm. Integration is understood broadly, either vertical or horizontal, and located within the practical domains of policy formulation, policy implementation and evaluation research. The nature, scope and depth of group commitments amongst students and practitioners of integration receive particular attention in line with Kuhn’s social interactionist approach.

Findings

Employing Kuhn’s notion of paradigm in the context of integration highlights the fundamental tension between integration efforts and integration outcomes. Whilst integration defines itself in contradistinction to professional boundaries and fragmentation, the paper argues that it fails to develop a strong theoretical and empirical foundation for a robust and stable group commitment. The reason is that the key motivational force that may create a stable group commitment amongst those engaged in integration, the patient perspective, remains outside the integration paradigm. This leaves integration as a practice and policy model underdeveloped, mainly paradigmatically illustrated by singular exemplars and rooted in aspirational policy vocabulary, while clustered around a near dogmatic belief that working together between services must lead to improved quality of care. To become a scientific paradigm the group commitment in integration would have to coalesce around a clear ontology (symbolic generalisations), epistemology (models of knowledge) and manifestations in practice (exemplars).

Research limitations/implications

At present both the ontology and epistemological foundations of integration practice and research are insufficiently clear. This hampers the development of integration practice as well as a better understanding of how to evaluate integration outcomes. Future studies should focus on the depth, nature and subject of group commitments to assess whether integration is a viable candidate for scientific paradigm or an assorted construct of policy aspirations.

Originality/value

The paper questions the rigour and trajectory of integration practice, policy and research. It identifies a tension at the centre of the field between group commitments to scientific exemplars (case studies) and symbolic generalisations, encapsulated in the desire to improve patient care. The notion of a scientific paradigm thus helps to re-frame the discussion about research and practice in integration.