A Realist Evaluation of Local Networks Designed to Achieve More Integrated Care

How did New Zealand's regional District Health Board groupings work to improve service integration and health outcomes: a realist evaluation

OBJECTIVES

In Aotearoa New Zealand (NZ), integration across the healthcare continuum has been a key approach to strengthening the health system and improving health outcomes. A key example has been four regional District Health Board (DHB) groupings, which, from 2011 to 2022, required the country's 20 DHBs to work together regionally. This research explores how this initiative functioned, examining how, for whom and in what circumstances regional DHB groupings worked to deliver improvements in system integration and health outcomes and equity.

DESIGN

We used a realist-informed evaluation study design. We used documentary analysis to develop programme logic models to describe the context, structure, capabilities, implementation activities and impact of each of the four regional groupings and then conducted interviews with stakeholders. We developed a generalised context-mechanisms-outcomes model, identifying key commonalities explaining how regional work 'worked' across NZ while noting important regional differences.

SETTING

NZ's four regional DHB groupings.

PARTICIPANTS

Forty-nine stakeholders from across the four regional groupings. These included regional DHB governance groups and coordinating regional agencies, DHB senior leadership, Māori and Pasifika leadership and lead clinicians for regional work streams.

RESULTS

Regional DHB working was layered on top of an already complex DHB environment. Organisational heterogeneity and tensions between local and regional priorities were key contextual factors. In response, regional DHB groupings leveraged a combination of 'hard' policy and planning processes, as well as 'soft', relationship-based mechanisms, aiming to improve system integration, population health outcomes and health equity.

CONCLUSION

The complexity of DHB regional working meant that success hinged on building relationships, leadership and trust, alongside robust planning and process mechanisms. As NZ reorients its health system towards a more centralised model underpinned by collaborations between local providers, our findings point to a need to align policy expectations and foster environments that support connection and collegiality across the health system.

Mecanismos de evaluaciones realistas en intervenciones de atención primaria en salud en poblaciones rurales y urbanas marginales

Objetivo.

Identificar los mecanismos reportados en intervenciones de atención primaria en salud (APS) en poblaciones rurales y urbanas marginales entre los años 1997 y 2019.

Métodos.

Se utilizó el método de revisión sistemática exploratoria para identificar la literatura. La búsqueda de literatura fue realizada en las bases de datos Medline (Pubmed y Ovid), Global health, Embase, Web of science, Scopus y Lilacs. La inclusión de literatura consideró estudios de intervención primaria, observacionales con metodologías cualitativas, cuantitativas o mixtas. Se incluyeron artículos en inglés, portugués y español publicados entre 1997 y 2019. Para la selección de documentos definitivos se utilizó Rayyan QCRI, se excluyeron fuentes de información de literatura gris o investigación en progreso.

Resultados.

A nivel individual se identificaron tres grupos de mecanismos: niveles de relaciones entre las intervenciones y los sujetos, transformaciones que pueden generar en los individuos y las relaciones recíprocas. A nivel institucional se relacionaron con la temporalidad, el dinero, relaciones de poder, relaciones de confianza, el sistema de salud, las expectativas y las condiciones administrativas.

Conclusiones.

Los mecanismos individuales son similares en los diferentes contextos, los mecanismos institucionales son susceptibles a la variación del contexto, se representan en la temporalidad, recursos económicos, relaciones de poder y de confianza, el sistema de salud, las expectativas y condiciones administrativas.

Health and Social Care Reform in Scotland - What Next?

INTRODUCTION

This paper analyses the important enablers, barriers and impacts of country-wide implementation of integrated health and social care in Scotland. It offers insights for other systems seeking to advance similar policy and practice.

DESCRIPTION

Landmark legislation was based on a shared vision and narrative about improving outcomes for people and communities. Implementation has involved coordination of multiple policies and interventions for different life stages, care groups, care settings and local context within a dynamic and complex system.

DISCUSSION

Relational and citizen led approaches are critical for success, but it takes time to build trusting relationships, influence organisational and professional cultures and cede power. Assessing national impacts is challenging and progress at a national level can seem slower than local experience suggests, due in part to the relative immaturity of national datasets for community interventions. Five years on there are many examples of innovation and positive outcomes despite increasing demographic, workforce, and financial challenges. However, inequalities continue to increase.

CONCLUSION

Realising the true value from integration will require a stronger focus on place-based prevention and early intervention to achieve a fairer Scotland where everybody thrives. Solidarity, equity, and human rights must guide the next phase of Scotland's story.

Integrated Care in Aotearoa New Zealand 2008-2020

INTRODUCTION

Ten years ago, progress towards integrated care in Aotearoa New Zealand was characterised as slow. Since then, there has been a patchwork of practices occurring under the broad umbrella of integrated care. These include: collective planning approaches (i.e., alliancing), agreed pathways of care, chronic care management initiatives, shared patient information systems, co-located centres and indigenous models of holistic care (e.g., Whānau Ora).

DESCRIPTION

Although integrated care is often mentioned in national policy documents, implementation has been left to regional and local decision making, and very few initiatives have spread beyond their initial locations.

DISCUSSION

System incentives that preserve organisational "sovereignty" and path-dependent funding have slowed progress towards more integrated care in some areas. There is some evidence about specific initiatives and their impact, but it is difficult to discern significant trends and commonalities around the country.

CONCLUSION

In the last ten years, the broad range of initiatives designed to achieve integrated care has absorbed regional and local attention and produced some evidence of progress, but the national picture of change is mixed.

Advancing integrated care evaluation in shifting contexts: blending implementation research with case study design in project SUSTAIN

BACKGROUND

Despite many studies evaluating the effectiveness of integrated care, evidence remains inconsistent. There is increasing commentary pointing out the mismatch between the ability to capture the somewhat 'illusive' impact of integrated care initiatives and programmes, and the most appropriate way to do this. Focusing on methodology, this paper describes and critically reviews the experiences of SUSTAIN, a Horizon 2020 funded project (2015-2019) with the purpose of advancing knowledge and understanding of cross-European integrated care evaluation. SUSTAIN sought to improve integrated care initiatives for older people in seven countries, and to maximise the potential for knowledge transfer and application across Europe. The methods approach drew from implementation research, employing the participative Evidence Integration Triangle (EIT) and incorporating a mixed method, multiple embedded case study design. A core set of qualitative and quantitative indicators, alongside context and process data, were created and tested within four key project domains (person-centredness, prevention-orientation, safety and efficiency). The paper critically discusses the overall approach, highlighting the value of the EIT and case study design, and signalling the challenges of data collection with frail older people and stakeholder involvement at the sites, as well as difficulties developing the core set of indicators.

CONCLUSIONS

Lessons learned and recommendations for advancing integrated care evaluation are put forward that focus on the status of integrated care as a complex intervention and a process. The use of implementation research methods and case study design are recommended as an additional evaluation approach for researchers to consider, alongside suggested ways of improving methods of data collection with frail populations and cost analysis.

(Dis)Integrated Care? Lessons from East London

INTRODUCTION

This paper examines one of the NHS England Pioneers programmes of Integrated Care, which was implemented in three localities in East London, covering the area served by one of the largest hospital groups in the UK and bringing together commissioners, providers and local authorities. The partners agreed to build a model of integrated care that focused on the whole person. This qualitative and participatory evaluation looked at how an ambitious vision translated into the delivery of integrated care on the ground. The study explored the micro-mechanisms of integrated care relationships based on the experience of health and social care professionals working in acute and community care settings.

METHODS

We employed a participatory approach, the Researcher in Residence model, whereby the researcher was embedded in the organisations she evaluated and worked alongside managers and clinicians to build collaboration across the full range of stakeholders, develop shared learning, and find common ground through competing interests, while trying to address power imbalances. A number of complementary qualitative methods of data generation were used, including documentary analysis, participant observations, semi-structured interviews, and coproduction workshops with frontline health and social care professionals to interpret the data and develop recommendations.

RESULTS

Our fieldwork exposed persistent organisational fragmentation, despite the dominant rhetoric of integration and efforts to build a shared vision at senior governance levels. The evaluation identified several important themes, including: a growing barrier between acute and community services; a persisting difficulty experienced by health and social care staff in working together because of professional and cultural differences, as well as conflicting organisational priorities and guidelines; and a lack of capacity and support to deliver a genuine multidisciplinary approach in practice, despite the ethos of multiagency being embraced widely.

DISCUSSION

By focusing on professionals' working routines, we detailed how and why action taken by organisational leaders failed to have tangible impact. The inability to align organisational priorities and guidelines on the ground, as well as a failure to acknowledge the impact of structural incentives for organisations to compete at the expense of cooperation, in a context of limited financial and human resources, acted as barriers to more coordinated working. Within an environment of continuous reconfigurations, staff were often confused about the functions of new services and did not feel they had influence on change processes. Investing in a genuine bottom-up approach could ensure that the range of activities needed to generate system-wide cultural transformation reflect the capacity of the organisations and systems and address genuine local needs.

LIMITATIONS

The authors acknowledge several limitations of this study, including the focus on one geographical area, East London, and the timing of the evaluation, with several new interventions and programmes introduced more or less simultaneously. Some of the intermediate care services under evaluation were still at pilot stage and some teams were undergoing new reconfigurations, reflecting the fast-pace of change of the past decade. This created confusion at times, for instance when discussing specific roles and activities with participants. We tried to address some of these challenges by organising several workshops with different teams to co-interpret and discuss the findings.

Evaluating Large-Scale Integrated Care Projects: The Development of a Protocol for a Mixed Methods Realist Evaluation Study in Belgium

BACKGROUND

The twelve Integrated Care Program pilot projects (ICPs) created by the government plan 'Integrated Care for Better Health' aim to achieve four outcome types (the Quadruple Aim) for people with chronic diseases in Belgium: improved population health, improved patient and provider experiences and improved cost efficiency. The aim of this article is to present the development of a mixed methods realist evaluation of this large-scale, whole system change programme.

METHODS

A scientific team was commissioned to co-design and implement an evaluation protocol in close collaboration with the government, the ICPs and several other involved stakeholders.

RESULTS

A protocol for a mixed methods realist evaluation was developed to gain insights into the mechanisms that foster successful results in ICPs. The qualitative evaluation proposed will be based on the document analysis of yearly ICP progress reports, selected case studies and focus group interviews with stakeholders. Processes and outcomes of all the projects will be monitored using indicators based on administrative data on population health and the quality and costs of care. A yearly survey will be organized to collect data on patient-reported outcomes and experiences and on provider-reported measures of inter-professional collaboration and proper wellbeing. Using both quantitative and qualitative data, we will develop theories about the mechanisms and the associated contextual factors that lead to integrated care and the Quadruple Aim outcomes.

DISCUSSION

The objective of this study is to deliver policy recommendations on strategies and best practices to improve care integration in Belgium and to implement a sustainable monitoring system that serves both policy makers and the stakeholders within the ICPs. Some challenges due to the large scale of the project and the multiple stakeholders involved may impede the successful implementation of this proposal.

Relationships Among Structures, Team Processes, and Outcomes for Service Users in Quebec Mental Health Service Networks

Background:

Few studies have identified and compared profiles of mental health service networks (MHSN) in terms of structures, processes, and outcomes, based on cluster analyses and perceptions of team managers, MH professionals and service users. This study assessed these associations in Quebec metropolitan, urban and semi-urban MHSN.

Methods:

A framework adapted from the Donabedian model guided data management, and cluster analyses were used to identify categories. Study participants included team managers (n = 45), MH professionals (n = 311) and service users (n = 327).

Results:

For all three MHSN, a common outcome category emerged: service users with complex MH problems and negative outcomes. The Metropolitan network reported two categories for structures (specialized MH teams, primary care MH teams) and processes (senior medical professional, psychosocial professionals), and outcomes (middle-age men with positive outcomes, older women with few MH problems). The Urban and Semi-urban networks revealed one category for structures (all teams) and service user (young service users with drug disorders), but two for processes (psychosocial professionals: urban, all professionals: semi-urban).

Conclusion:

The Metropolitan MHSN showed greater heterogeneity regarding structures and team processes than the other two MHSN. Service user outcomes were largely associated with clinical characteristics, regardless of network configurations for structures and team processes.