Implementation of large, multi-site hospital interventions: a realist evaluation of strategies for developing capability

BACKGROUND

This study presents guidelines for implementation distilled from the findings of a realist evaluation. The setting was local health districts in New South Wales, Australia that implemented three clinical improvement initiatives as part of a state-wide program. We focussed on implementation strategies designed to develop health professionals' capability to deliver value-based care initiatives for multisite programs. Capability, which increases implementers' ability to cope with unexpected scenarios is key to managing change.

METHODS

We used a mixed methods realist evaluation which tested and refined program theories elucidating the complex dynamic between context (C), mechanism (M) and outcome (O) to determine what works, for whom, under what circumstances. Data was drawn from program documents, a realist synthesis, informal discussions with implementation designers, and interviews with 10 key informants (out of 37 identified) from seven sites. Data analysis employed a retroductive approach to interrogate the causal factors identified as contributors to outcomes.

RESULTS

CMO statements were refined for four initial program theories: Making it Relevant- where participation in activities was increased when targeted to the needs of the staff; Investment in Quality Improvement- where engagement in capability development was enhanced when it was valued by all levels of the organisation; Turnover and Capability Loss- where the effects of staff turnover were mitigated; and Community-Wide Priority- where there was a strategy of spanning sites. From these data five guiding principles for implementers were distilled: (1) Involve all levels of the health system to effectively implement large-scale capability development, (2) Design capability development activities in a way that supports a learning culture, (3) Plan capability development activities with staff turnover in mind, (4) Increased capability should be distributed across teams to avoid bottlenecks in workflows and the risk of losing key staff, (5) Foster cross-site collaboration to focus effort, reduce variation in practice and promote greater cohesion in patient care.

CONCLUSIONS

A key implementation strategy for interventions to standardise high quality practice is development of clinical capability. We illustrate how leadership support, attention to staff turnover patterns, and making activities relevant to current issues, can lead to an emergent learning culture.

Strategies to Improve Care in the Emergency Department for Culturally and Linguistically Diverse Adults: a Systematic Review

BACKGROUND

The emergency department (ED) is an important gateway into the health system for people from culturally and linguistically diverse (CALD) backgrounds; their experience in the ED is likely to impact the way they access care in the future. Our review aimed to describe interventions used to improve ED health care delivery for adults from a CALD background.

METHODS

An electronic search of four databases was conducted to identify empirical studies that reported interventions with a primary focus of improving ED care for CALD adults (aged ≥ 18 years), with measures relating to ED system performance, patient outcomes, patient experience, or staff experience. Studies published from inception to November 2022 were included. We excluded non-empirical studies, studies where an intervention was not provided in ED, papers where the full text was unavailable, or papers published in a language other than English. The intervention strategies were categorised thematically, and measures were tabulated.

RESULTS

Following the screening of 3654 abstracts, 89 articles underwent full text review; 16 articles met the inclusion criteria. Four clear strategies for targeting action tailored to the CALD population of interest were identified: improving self-management of health issues, improving communication between patients and providers, adhering to good clinical practice, and building health workforce capacity.

CONCLUSIONS

The four strategies identified provide a useful framework for targeted action tailored to the population and outcome of interest. These detailed examples show how intervention design must consider intersecting socio-economic barriers, so as not to perpetuate existing disparity.

REGISTRATION

PROSPERO registration number: CRD42022379584.

Audit and feedback to reduce unwarranted clinical variation at scale: a realist study of implementation strategy mechanisms

BACKGROUND

Unwarranted clinical variation in hospital care includes the underuse, overuse, or misuse of services. Audit and feedback is a common strategy to reduce unwarranted variation, but its effectiveness varies widely across contexts. We aimed to identify implementation strategies, mechanisms, and contextual circumstances contributing to the impact of audit and feedback on unwarranted clinical variation.

METHODS

Realist study examining a state-wide value-based healthcare program implemented between 2017 and 2021 in New South Wales, Australia. Three initiatives within the program included audit and feedback to reduce unwarranted variation in inpatient care for different conditions. Multiple data sources were used to formulate the initial audit and feedback program theory: a systematic review, realist review, program document review, and informal discussions with key program stakeholders. Semi-structured interviews were then conducted with 56 participants to refute, refine, or confirm the initial program theories. Data were analysed retroductively using a context-mechanism-outcome framework for 11 transcripts which were coded into the audit and feedback program theory. The program theory was validated with three expert panels: senior health leaders (n = 19), Agency for Clinical Innovation (n = 11), and Ministry of Health (n = 21) staff.

RESULTS

The program's audit and feedback implementation strategy operated through eight mechanistic processes. The strategy worked well when clinicians (1) felt ownership and buy-in, (2) could make sense of the information provided, (3) were motivated by social influence, and (4) accepted responsibility and accountability for proposed changes. The success of the strategy was constrained when the audit process led to (5) rationalising current practice instead of creating a learning opportunity, (6) perceptions of unfairness and concerns about data integrity, 7) development of improvement plans that were not followed, and (8) perceived intrusions on professional autonomy.

CONCLUSIONS

Audit and feedback strategies may help reduce unwarranted clinical variation in care where there is engagement between auditors and local clinicians, meaningful audit indicators, clear improvement plans, and respect for clinical expertise. We contribute theoretical development for audit and feedback by proposing a Model for Audit and Feedback Implementation at Scale. Recommendations include limiting the number of audit indicators, involving clinical staff and local leaders in feedback, and providing opportunities for reflection.

Implementation of consensus-based perioperative care pathways to reduce clinical variation for elective surgery in an Australian private hospital: a mixed-methods pre-post study protocol

INTRODUCTION

Addressing clinical variation in elective surgery is challenging. A key issue is how to gain consensus between largely autonomous clinicians. Understanding how the consensus process works to develop and implement perioperative pathways and the impact of these pathways on reducing clinical variation can provide important insights into the effectiveness of the consensus process. The primary objective of this study is to understand the implementation of an organisationally supported, consensus approach to implement perioperative care pathways in a private healthcare facility and to determine its impact.

METHODS

A mixed-methods Effectiveness-Implementation Hybrid (type III) pre-post study will be conducted in one Australian private hospital. Five new consensus-based perioperative care pathways will be developed and implemented for specific patient cohorts: spinal surgery, radical prostatectomy, cardiac surgery, bariatric surgery and total hip and knee replacement. The individual components of these pathways will be confirmed as part of a consensus-building approach and will follow a four-stage implementation process using the Exploration, Preparation, Implementation and Sustainment framework. The process of implementation, as well as barriers and facilitators, will be evaluated through semistructured interviews and focus groups with key clinical and non-clinical staff, and participant observation. We anticipate completing 30 interviews and 15-20 meeting observations. Administrative and clinical end-points for at least 152 participants will be analysed to assess the effectiveness of the pathways.

ETHICS AND DISSEMINATION

This study received ethical approval from Macquarie University Human Research Ethics Medical Sciences Committee (Reference No: 520221219542374). The findings of this study will be disseminated through peer-reviewed publications, conference presentations and reports for key stakeholders.

Guiding principles for effective collaborative implementation strategies for multisite hospital improvement initiatives: a mixed-method realist evaluation of collaborative strategies used in four multisite initiatives at public hospitals in New South Wales, Australia

OBJECTIVE

Large-scale, multisite hospital improvement initiatives can advance high-quality care for patients. Implementation support is key to adoption of change in this context. Strategies that foster collaboration within local teams, across sites and between initiative developers and users are important. However not all implementation strategies are successful in all settings, sometimes realising poor or unintended outcomes. Our objective here is to develop guiding principles for effective collaborative implementation strategies for multi-site hospital initiatives.

DESIGN

Mixed-method realist evaluation. Realist studies aim to examine the underlying theories that explain differing outcomes, identifying mechanisms and contextual factors that may trigger them.

SETTING

We report on collaborative strategies used in four multi-site initiatives conducted in all public hospitals in New South Wales, Australia (n>100).

PARTICIPANTS

Using an iterative process, information was gathered on collaborative implementation strategies used, then initial programme theories hypothesised to underlie the strategies' outcomes were surfaced using a realist dialogic approach. A realist interview schedule was developed to elicit evidence for the posited initial programme theories. Fourteen participants from 20 key informants invited participated. Interviews were conducted via Zoom, transcribed and analysed. From these data, guiding principles of fostering collaboration were developed.

RESULTS

Six guiding principles were distilled: (1) structure opportunities for collaboration across sites; (2) facilitate meetings to foster learning and problem-solving across sites; (3) broker useful long-term relationships; (4) enable support agencies to assist implementers by giving legitimacy to their efforts in the eyes of senior management; (5) consider investment in collaboration as effective well beyond the current projects; (6) promote a shared vision and build momentum for change by ensuring inclusive networks where everyone has a voice.

CONCLUSION

Structuring and supporting collaboration in large-scale initiatives is a powerful implementation strategy if contexts described in the guiding principles are present.

Making implementation science more real

Implementation science in healthcare aims to understand how to get evidence into practice. Once this is achieved in one setting, it becomes increasingly difficult to replicate elsewhere. The problem is often attributed to differences in context that influence how and whether implementation strategies work. We argue that realist research paradigms provide a useful framework to express the effect of contextual factors within implementation strategy causal processes. Realist studies are theory-driven evaluations that focus on understanding how and why interventions work under different circumstances. They consider the interaction between contextual circumstances, theoretical mechanisms of change and the outcomes they produce, to arrive at explanations of conditional causality (i.e., what tends to work, for whom, under what circumstances). This Commentary provides example applications using preliminary findings from a large realist implementation study of system-wide value-based healthcare initiatives in New South Wales, Australia. If applied judiciously, realist implementation studies may represent a sound approach to help optimise delivery of the right care in the right setting and at the right time.

Implementing large-system, value-based healthcare initiatives: a realist study protocol for seven natural experiments

INTRODUCTION

Value-based healthcare delivery models have emerged to address the unprecedented pressure on long-term health system performance and sustainability and to respond to the changing needs and expectations of patients. Implementing and scaling the benefits from these care delivery models to achieve large-system transformation are challenging and require consideration of complexity and context. Realist studies enable researchers to explore factors beyond 'what works' towards more nuanced understanding of 'what tends to work for whom under which circumstances'. This research proposes a realist study of the implementation approach for seven large-system, value-based healthcare initiatives in New South Wales, Australia, to elucidate how different implementation strategies and processes stimulate the uptake, adoption, fidelity and adherence of initiatives to achieve sustainable impacts across a variety of contexts.

METHODS AND ANALYSIS

This exploratory, sequential, mixed methods realist study followed RAMESES II (Realist And Meta-narrative Evidence Syntheses: Evolving Standards) reporting standards for realist studies. Stage 1 will formulate initial programme theories from review of existing literature, analysis of programme documents and qualitative interviews with programme designers, implementation support staff and evaluators. Stage 2 envisages testing and refining these hypothesised programme theories through qualitative interviews with local hospital network staff running initiatives, and analyses of quantitative data from the programme evaluation, hospital administrative systems and an implementation outcome survey. Stage 3 proposes to produce generalisable middle-range theories by synthesising data from context-mechanism-outcome configurations across initiatives. Qualitative data will be analysed retroductively and quantitative data will be analysed to identify relationships between the implementation strategies and processes, and implementation and programme outcomes. Mixed methods triangulation will be performed.

ETHICS AND DISSEMINATION

Ethical approval has been granted by Macquarie University (Project ID 23816) and Hunter New England (Project ID 2020/ETH02186) Human Research Ethics Committees. The findings will be published in peer-reviewed journals. Results will be fed back to partner organisations and roundtable discussions with other health jurisdictions will be held, to share learnings.

Avoiding unnecessary hospitalisation for patients with chronic conditions: a systematic review of implementation determinants for hospital avoidance programmes

Background

Studies of clinical effectiveness have demonstrated the many benefits of programmes that avoid unnecessary hospitalisations. Therefore, it is imperative to examine the factors influencing implementation of these programmes to ensure these benefits are realised across different healthcare contexts and settings. Numerous factors may act as determinants of implementation success or failure (facilitators and barriers), by either obstructing or enabling changes in healthcare delivery. Understanding the relationships between these determinants is needed to design and tailor strategies that integrate effective programmes into routine practice. Our aims were to describe the implementation determinants for hospital avoidance programmes for people with chronic conditions and the relationships between these determinants.

Methods

An electronic search of four databases was conducted from inception to October 2019, supplemented by snowballing for additional articles. Data were extracted using a structured data extraction tool and risk of bias assessed using the Hawker Tool. Thematic synthesis was undertaken to identify determinants of implementation success or failure for hospital avoidance programmes for people with chronic conditions, which were categorised according to the Consolidated Framework for Implementation Research (CFIR). The relationships between these determinants were also mapped.

Results

The initial search returned 3537 articles after duplicates were removed. After title and abstract screening, 123 articles underwent full-text review. Thirteen articles (14 studies) met the inclusion criteria. Thematic synthesis yielded 23 determinants of implementation across the five CFIR domains. ‘Availability of resources’, ‘compatibility and fit’, and ‘engagement of interprofessional team’ emerged as the most prominent determinants across the included studies. The most interconnected implementation determinants were the ‘compatibility and fit’ of interventions and ‘leadership influence’ factors.

Conclusions

Evidence is emerging for how chronic condition hospital avoidance programmes can be successfully implemented and scaled across different settings and contexts. This review provides a summary of key implementation determinants and their relationships. We propose a hypothesised causal loop diagram to represent the relationship between determinants within a complex adaptive system.

Trial registration

PROSPERO 162812 

Identification and referral of patients with refractory epilepsy from the primary to the tertiary care interface in New South Wales, Australia

OBJECTIVES

This mixed-method feasibility study conducted in New South Wales (NSW), Australia, aimed to explore clinical practices around the identification of patients with refractory epilepsy and referral from primary care to Tertiary Epilepsy Centers. The perceptions of general practitioners, neurologists, and adults living with refractory epilepsy were considered.

METHODS

Fifty-two data collection events were achieved through 22 semi-structured interviews with six neurologists and 12 adults who currently have, or have had refractory epilepsy, and four family members, 10 clinical observations of patient consultations and 20 surveys with general practitioners. A thematic analysis was conducted on the qualitative data alongside assessment of observational fieldnotes and survey data.

FINDINGS

Two main themes emerged: 1) Patient healthcare pathways and care experiences highlighted the complex and deeply contextualized experiences of both patients and healthcare professionals, from first identification of people's seizures, in primary and community care settings, to referral to Tertiary Epilepsy Centers, shedding light on a fragmented, nonstandardized referral process, influenced by both individual and shared-care practices. 2) Factors impacting referrals and patient pathways indicated that onward referral to a Tertiary Epilepsy Center is affected by the knowledge, or the lack thereof, of healthcare professionals regarding treatment options. Barriers include limited person-centered care, shared decision-making, and refractory epilepsy education for healthcare professionals, which can delay patients' disease identification and can hinder speedy referral pathways and processes, in Australia for up to 17 years. In addition, person-centered communication around care pathways is affected by relationships between clinicians, patients, and family members.

CONCLUSION

This study has identified a noticeable lack of standardized care across epilepsy-related healthcare sectors, which recognizes a need for developing and implementing clearer epilepsy-related guidelines and Continuing Professional Development in the primary and community care settings. This, however, requires greater collaboration and commitment in the primary, community, and tertiary care sectors to address the ongoing misconceptions around professional roles and responsibilities to optimize shared-care practices. Ultimately, prioritizing person-centered care on both patients' and professionals' agendas, in order to improve satisfaction with care experiences of people living with complex epilepsy.

A mobile methods pilot study of surgical spaces: 'fit for purpose? Organisational productivity and workforce wellbeing in workspaces in hospital' (FLOURISH)

Background

Good workspace design is key to the quality of work, safety, and wellbeing for workers, yet we lack vital knowledge about optimal hospital design to meet healthcare workforce needs. This study used novel mobile methods to examine the concept of Work-as-Done and the effect of workspace-use on healthcare professional practice, productivity, health and safety in an Australian university hospital.

Methods

This pilot study took place in one gastroenterological surgical unit between 2018 and 2019. Data collection involved 50 h of observations and informal conversations, followed by interpretation of five architectural plans and 45 photographs. Fieldnotes were thematically analysed and corroborated by analysis of visual data using a predefined taxonomy.

Results

Six themes were identified, revealing spaces that both support and hinder Work-as-Done. Fit-for-purpose spaces facilitated effective communication between staff, patients and families, conferred relative comfort and privacy, and supported effective teamwork. Unfit-for-purpose spaces were characterised by disruptions to work practices, disharmony among team members, and physical discomfort for staff. Staff employed workarounds to manage unfit-for-purpose spaces.

Conclusion

The results identified negative impacts of negotiating unfit-for-purpose workspaces on the work and wellbeing of staff. While the use of workarounds and adaptations enable staff to maintain everyday working practices, they can also lead to unexpected consequences. Results indicated the need to identify and support fit-for-purpose spaces and minimize the detrimental qualities of unfit-for-purpose spaces. This study showed that mobile methods were suitable for examining Work-as-Done in a fast-moving, adaptive hospital setting.

Exploring the recruitment, ethical considerations, conduct and information dissemination of an audiology trial: a pretrial qualitative study (q-COACH)

INTRODUCTION

Randomised controlled trials (RCTs), while still considered the gold standard approach in medical research, can encounter impediments to their successful conduct and the dissemination of results. Pretrial qualitative research can usefully address some of these impediments, including recruitment and retention, ethical conduct, and preferred methods of dissemination. However, pretrial qualitative work is rarely undertaken in audiology. The Comparison of outcomes with hearing aids and cochlear implants in adults with moderately severe-to-profound bilateral sensorineural hearing loss (COACH) is a proposed RCT aiming to clarify when hearing aids (HAs) or cochlear implants (CIs) are the most suitable for different degrees of hearing loss and for which kinds of patients. q-COACH is a pretrial, qualitative study examining stakeholders' experiences of HAs and CIs, current clinical practices and stakeholders' perspectives of the design, conduct and dissemination plans for the proposed COACH study.

METHODS

Twenty-four participants including general practitioners, audiologists, adult HA users, and adult support networks undertook either semi-structured individual or paired interviews and completed demographic questionnaires. Data were analysed thematically.

RESULTS

Four key themes arose from this study: 1) rethinking sampling and recruitment strategies, 2) ethical considerations, 3) refining trial conduct, and 4) interconnected, appropriate and accessible methods of results dissemination.

CONCLUSIONS

This qualitative investigation identified key considerations for the proposed RCT design, conduct and dissemination to help with successful implementation of COACH, and to indicate a plan of action at all RCT stages that would be acceptable to potential participants. By drawing on the perspectives of multiple key stakeholders and including a more general discussion of their experience and opinions of hearing loss, hearing device use and service availability, the study revealed experiential and ethical paradigms in which stakeholders operate. In so doing, q-COACH has exposed the benefits of preliminary qualitative investigations that enable detailed and rich understandings of the phenomenon at stake, forestalling problems and improving the quality of trial design, conduct and dissemination, while informing future RCT development discussions.