Implementing health research through academic and clinical partnerships: a realistic evaluation of the Collaborations for Leadership in Applied Health Research and Care (CLAHRC)

Characterizing research partnerships in child health research: A scoping review

Research partnerships between researchers and knowledge users (KUs) in child health are understudied. This study examined the scope of KU engagement reported in published child health research, inclusive of health research partnership approaches and KU groups. Search strategies were developed by a health research librarian. Studies had to be in English, published since 2007, and were not excluded based on design. A two-step, multiple-person hybrid screening approach was used for study inclusion. Data on study and engagement characteristics, barriers and facilitators, and effects were extracted by one reviewer, with 10% verified by a second reviewer. Three hundred fifteen articles were included, with 243 (77.1%) published between 2019 and 2021. Community-based participatory research was the most common approach used (n = 122, 38.3%). Most studies (n = 235, 74.6%) engaged multiple KU groups (range 1-11), with children/youth, healthcare professionals, and parents/families being most frequently engaged. Reporting of barriers and facilitators and effects were variable, reported in 170 (53.8%) and 197 (62.5%) studies, respectively. Publications have increased exponentially over time. There is ongoing need to optimize evaluation and reporting consistency to facilitate growth in the field. Additional studies are needed to further our understanding of research partnerships in child health.

Series: Public engagement with research. Part 4: Maximising the benefits of involving the public in research implementation

This final article in the four-part series focuses on the often neglected yet important role of the public in implementing research in General Practice and Primary Care more broadly. Experience in implementation of findings from research with public engagement in Primary Care has highlighted how partnership working with patients and the public is important in transitioning from 'what we know' from the evidence-base to 'what we do' in practice. Factors related to Primary Care research that make public engagement important are highlighted e.g. implementing complex interventions, implementing interventions that increase health equity, implementing interventions in countries with different primary healthcare system strengths. Involvement of patients and public can enhance the development of modelling and simulation included in studies on systems modelling for improving health services. We draw on the emerging evidence base to describe public engagement in implementation and offer some guiding principles for engaging with the public in the implementation in General Practice and Primary Care in general. Illustrative case studies are included to support others wishing to offer meaningful engagement in implementing research evidence.

Understanding how and under what circumstances integrated knowledge translation works for people engaged in collaborative research: metasynthesis of IKTRN casebooks

INTRODUCTION AND AIMS

Interaction and collaboration between researchers, patients/public, clinicians, managers and policy-makers are necessary to enhance the relevance and use of research, improve planning, and optimize healthcare delivery and outcomes. The Integrated Knowledge Translation Research Network (IKTRN) published four casebooks from 2019 to 2021, describing varied approaches to research co-production. Our aim was to examine the case studies to extend existing theoretical and empirical perspectives about how co-production works.

METHODS

We used metasynthesis, a qualitative research design that includes seven iterative steps (clarify the purpose, delineate the case studies included, extract and code the data, derive themes from the coded data, determine the relationships of the themes to research co-production, synthesize the concepts, and build theory).

RESULTS

A total of 35 cases was reviewed. The aggregate findings of this metasynthesis identified multiple contextual and process factors, barriers, and facilitators that influence integrated knowledge translation (IKT), and a range of IKT activities that increased the likelihood of success of co-production during research. In comparing the findings from the metasynthesis with existing literature, we found a number of consistencies, but also new information about barriers, facilitators, IKT activities and outcomes, thereby adding to our understanding about factors that influence co-production.

CONCLUSIONS

This metasynthesis provided concrete examples to optimize co-produced clinical and health system research. More research is needed to fully understand how to overcome some challenging modifiable barriers, establish relationships, facilitate communication, overcome power differentials and create processes for knowledge-users working across boundaries (clinical practice and research) to stay engaged and participate fully in research endeavours.

Aligning intuition and theory: a novel approach to identifying the determinants of behaviours necessary to support implementation of evidence into practice

BACKGROUND

Disentangling the interplay between experience-based intuition and theory-informed implementation is crucial for identifying the direct contribution theory can make for generating behaviour changes needed for successful evidence translation. In the context of 'clinicogenomics', a complex and rapidly evolving field demanding swift practice change, we aimed to (a) describe a combined clinician intuition- and theory-driven method for identifying determinants of and strategies for implementing clinicogenomics, and (b) articulate a structured approach to standardise hypothesised behavioural pathways and make potential underlying theory explicit.

METHODS

Interview data from 16 non-genetic medical specialists using genomics in practice identified three target behaviour areas across the testing process: (1) identifying patients, (2) test ordering and reporting, (3) communicating results. The Theoretical Domains Framework (TDF) was used to group barriers and facilitators to performing these actions. Barriers were grouped by distinct TDF domains, with 'overarching' TDF themes identified for overlapping barriers. Clinician intuitively-derived implementation strategies were matched with corresponding barriers, and retrospectively coded against behaviour change techniques (BCTs). Where no intuitive strategies were provided, theory-driven strategies were generated. An algorithm was developed and applied to articulate how implementation strategies address barriers to influence behaviour change.

RESULTS

Across all target behaviour areas, 32 identified barriers were coded across seven distinct TDF domains and eight overarching TDF themes. Within the 29 intuitive strategies, 21 BCTs were represented and used on 49 occasions to address 23 barriers. On 10 (20%) of these occasions, existing empirical links were found between BCTs and corresponding distinct TDF-coded barriers. Twenty additional theory-driven implementation strategies (using 19 BCTs on 31 occasions) were developed to address nine remaining barriers.

CONCLUSION

Clinicians naturally generate their own solutions when implementing clinical interventions, and in this clinicogenomics example these intuitive strategies aligned with theoretical recommendations 20% of the time. We have matched intuitive strategies with theory-driven BCTs to make potential underlying theory explicit through proposed structured hypothesised causal pathways. Transparency and efficiency are enhanced, providing a novel method to identify determinants of implementation. Operationalising this approach to support the design of implementation strategies may optimise practice change in response to rapidly evolving scientific advances requiring swift translation into healthcare.

Practices to support co-design processes: A case-study of co-designing a program for children with parents with a mental health problem in the Austrian region of Tyrol

Forms of collaborative knowledge production, such as community-academic partnerships (CAP), have been increasingly used in health care. However, instructions on how to deliver such processes are lacking. We aim to identify practice ingredients for one element within a CAP, a 6-month co-design process, during which 26 community- and 13 research-partners collaboratively designed an intervention programme for children whose parent have a mental illness. Using 22 published facilitating and hindering factors for CAP as the analytical framework, eight community-partners reflected on the activities which took place during the co-design process. From a qualitative content analysis of the data, we distilled essential practices for each CAP factor. Ten community- and eight research-partners revised the results and co-authored this article. We identified 36 practices across the 22 CAP facilitating or hindering factors. Most practices address more than one factor. Many practices relate to workshop design, facilitation methods, and relationship building. Most practices were identified for facilitating 'trust among partners', 'shared visions, goals and/or missions', 'effective/frequent communication', and 'well-structured meetings'. Fewer practices were observed for 'effective conflict resolution', 'positive community impact' and for avoiding 'excessive funding pressure/control struggles' and 'high burden of activities'. Co-designing a programme for mental healthcare is a challenging process that requires skills in process management and communication. We provide practice steps for delivering co-design activities. However, practitioners may have to adapt them to different cultural contexts. Further research is needed to analyse whether co-writing with community-partners results in a better research output and benefits for participants.

What do we want to get out of this? a critical interpretive synthesis of the value of process evaluations, with a practical planning framework

BACKGROUND

Process evaluations aim to understand how complex interventions bring about outcomes by examining intervention mechanisms, implementation, and context. While much attention has been paid to the methodology of process evaluations in health research, the value of process evaluations has received less critical attention. We aimed to unpack how value is conceptualised in process evaluations by identifying and critically analysing 1) how process evaluations may create value and 2) what kind of value they may create.

METHODS

We systematically searched for and identified published literature on process evaluation, including guidance, opinion pieces, primary research, reviews, and discussion of methodological and practical issues. We conducted a critical interpretive synthesis and developed a practical planning framework.

RESULTS

We identified and included 147 literature items. From these we determined three ways in which process evaluations may create value or negative consequences: 1) through the socio-technical processes of 'doing' the process evaluation, 2) through the features/qualities of process evaluation knowledge, and 3) through using process evaluation knowledge. We identified 15 value themes. We also found that value varies according to the characteristics of individual process evaluations, and is subjective and context dependent.

CONCLUSION

The concept of value in process evaluations is complex and multi-faceted. Stakeholders in different contexts may have very different expectations of process evaluations and the value that can and should be obtained from them. We propose a planning framework to support an open and transparent process to plan and create value from process evaluations and negotiate trade-offs. This will support the development of joint solutions and, ultimately, generate more value from process evaluations to all.

Does place matter in the implementation of an evidence-based program policy in an Australian place-based initiative for children?

Policy-mandated requirements for use of evidence-based programs (EBP) in place-based initiatives are becoming more common. Little attention has been paid to the geographic aspects of uneven market development and urbanicity in implementing EBPs in large place-based initiatives. The aim of this study was to explore geographic variation in knowledge, attitudes, and experiences of service providers who implemented an EBP policy in Australia's largest place-based initiative for children, Communities for Children. A cross-sectional online survey of Communities for Children service providers was conducted in 2018-2019, yielding 197 participants from all of Australia's eight states and territories. Relationships between two measures of 'place' (thick and thin market states; urbanicity: urban, regional and remote) and study-designed measures of knowledge, attitudes, and implementation experiences were analyzed using adjusted logistic and multinomial regressions. Participants from thin market states (outside the Eastern Seaboard) were more resistant to the policy and experienced greater implementation challenges than those from thick market states (Eastern Seaboard). Regional participants reported greater knowledge about EBPs but experienced greater dissatisfaction and implementation challenges with the policy than both urban and remote participants. Our study found that place does matter when implementing EBPs in a place-based initiative.

What shapes research and research capacity building in rural health services? Context matters

Objective

To determine the contextual factors influencing research and research capacity building in rural health settings.

Design

Qualitative study using semi‐structured telephone interviews to collect data regarding health professionals’ research education and capacity building. Analysis involved inductive coding using Braun and Clark’s thematic analysis; and deductive mapping to the Consolidated Framework for Implementation Research (CFIR).

Setting

Victorian rural health services and university campuses.

Participants

Twenty senior rural health managers, academics and/or research coordinators. Participants had at least three years’ experience in rural public health, health‐related research or health education settings.

Main outcome measures

Contextual factors influencing the operationalisation and prioritisation of research capacity building in rural health services.

Results

Findings reflected the CFIR domains and constructs: intervention characteristics (relative advantage); outer setting (cosmopolitanism, external policies and incentives); inner setting (implementation climate, readiness for implementation); characteristics of individuals (self‐efficacy); and process (planning, engaging). Findings illustrated the implementation context and the complex contextual tensions, which either prevent or enhance research capacity building in rural health services.

Conclusions

Realising the Australian Government’s vision for improved health service provision and health outcomes in rural areas requires a strong culture of research and research capacity building in rural health services. Low levels of rural research funding, chronic workforce shortages and the tension between undertaking research and delivering health care, all significantly impact the operationalisation and prioritisation of research capacity building in rural health services. Effective policy and investment addressing these contextual factors is crucial for the success of research capacity building in rural health services.

Context-specific adaptation of an eHealth-facilitated, integrated care model and tailoring its implementation strategies-A mixed-methods study as a part of the SMILe implementation science project

Background

Contextually adapting complex interventions and tailoring their implementation strategies is key to a successful and sustainable implementation. While reporting guidelines for adaptations and tailoring exist, less is known about how to conduct context-specific adaptations of complex health care interventions.

Aims

To describe in methodological terms how the merging of contextual analysis results (step 1) with stakeholder involvement, and considering overarching regulations (step 2) informed our adaptation of an Integrated Care Model (ICM) for SteM cell transplantatIon faciLitated by eHealth (SMILe) and the tailoring of its implementation strategies (step 3).

Methods

Step 1: We used a mixed-methods design at University Hospital Basel, guided by the Basel Approach for coNtextual ANAlysis (BANANA). Step 2: Adaptations of the SMILe-ICM and tailoring of implementation strategies were discussed with an interdisciplinary team (n = 28) by considering setting specific and higher-level regulatory scenarios. Usability tests were conducted with patients (n = 5) and clinicians (n = 4). Step 3: Adaptations were conducted by merging our results from steps 1 and 2 using the Framework for Reporting Adaptations and Modifications-Enhanced (FRAME). We tailored implementation strategies according to the Expert Recommendations for Implementing Change (ERIC) compilation.

Results

Step 1: Current clinical practice was mostly acute-care-driven. Patients and clinicians valued eHealth-facilitated ICMs to support trustful patient-clinician relationships and the fitting of eHealth components to context-specific needs. Step 2: Based on information from project group meetings, adaptations were necessary on the organizational level (e.g., delivery of self-management information). Regulations informed the tailoring of SMILe-ICM`s visit timepoints and content; data protection management was adapted following Swiss regulations; and steering group meetings supported infrastructure access. The usability tests informed further adaptation of technology components. Step 3: Following FRAME and ERIC, SMILe-ICM and its implementation strategies were contextually adapted and tailored to setting-specific needs.

Discussion

This study provides a context-driven methodological approach on how to conduct intervention adaptation including the tailoring of its implementation strategies. The revealed meso-, and macro-level differences of the contextual analysis suggest a more targeted approach to enable an in-depth adaptation process. A theory-guided adaptation phase is an important first step and should be sufficiently incorporated and budgeted in implementation science projects.

Large-scale implementation of stroke early supported discharge: the WISE realist mixed-methods study

Background

In England, the provision of early supported discharge is recommended as part of an evidence-based stroke care pathway.

Objectives

To investigate the effectiveness of early supported discharge services when implemented at scale in practice and to understand how the context within which these services operate influences their implementation and effectiveness.

Design

A mixed-methods study using a realist evaluation approach and two interlinking work packages was undertaken. Three programme theories were tested to investigate the adoption of evidence-based core components, differences in urban and rural settings, and communication processes.

Setting and interventions

Early supported discharge services across a large geographical area of England, covering the West and East Midlands, the East of England and the North of England.

Participants

Work package 1: historical prospective patient data from the Sentinel Stroke National Audit Programme collected by early supported discharge and hospital teams. Work package 2: NHS staff (n = 117) and patients (n = 30) from six purposely selected early supported discharge services.

Data and main outcome

Work package 1: a 17-item early supported discharge consensus score measured the adherence to evidence-based core components defined in an international consensus document. The effectiveness of early supported discharge was measured with process and patient outcomes and costs. Work package 2: semistructured interviews and focus groups with NHS staff and patients were undertaken to investigate the contextual determinants of early supported discharge effectiveness.

Results

A variety of early supported discharge service models had been adopted, as reflected by the variability in the early supported discharge consensus score. A one-unit increase in early supported discharge consensus score was significantly associated with a more responsive early supported discharge service and increased treatment intensity. There was no association with stroke survivor outcome. Patients who received early supported discharge in their stroke care pathway spent, on average, 1 day longer in hospital than those who did not receive early supported discharge. The most rural services had the highest service costs per patient. NHS staff identified core evidence-based components (e.g. eligibility criteria, co-ordinated multidisciplinary team and regular weekly multidisciplinary team meetings) as central to the effectiveness of early supported discharge. Mechanisms thought to streamline discharge and help teams to meet their responsiveness targets included having access to a social worker and the quality of communications and transitions across services. The role of rehabilitation assistants and an interdisciplinary approach were facilitators of delivering an intensive service. The rurality of early supported discharge services, especially when coupled with capacity issues and increased travel times to visit patients, could influence the intensity of rehabilitation provision and teams’ flexibility to adjust to patients’ needs. This required organising multidisciplinary teams and meetings around the local geography. Findings also highlighted the importance of good leadership and communication. Early supported discharge staff highlighted the need for collaborative and trusting relationships with patients and carers and stroke unit staff, as well as across the wider stroke care pathway.

Limitations

Work package 1: possible influence of unobserved variables and we were unable to determine the effect of early supported discharge on patient outcomes. Work package 2: the pragmatic approach led to ‘theoretical nuggets’ rather than an overarching higher-level theory.

Conclusions

The realist evaluation methodology allowed us to address the complexity of early supported discharge delivery in real-world settings. The findings highlighted the importance of context and contextual features and mechanisms that need to be either addressed or capitalised on to improve effectiveness.

Trial registration

Current Controlled Trials ISRCTN15568163.

Funding

This project was funded by the National Institute for Health Research (NIHR) Health Services and Delivery Research programme and will be published in full in Health Services and Delivery Research; Vol. 9, No. 22. See the NIHR Journals Library website for further project information.

Value of stakeholder engagement in improving newborn care in Kenya: a qualitative description of perspectives and lessons learned

OBJECTIVE

Embedding researchers within health systems results in more socially relevant research and more effective uptake of evidence into policy and practice. However, the practice of embedded health service research remains poorly understood. We explored and assessed the development of embedded participatory approaches to health service research by a health research team in Kenya highlighting the different ways multiple stakeholders were engaged in a neonatal research study.

METHODS

We conducted semistructured qualitative interviews with key stakeholders. Data were analysed thematically using both inductive and deductive approaches.

SETTING

Over recent years, the Health Services Unit within the Kenya Medical Research Institute (KEMRI)-Wellcome Trust Research Programme in Nairobi Kenya, has been working closely with organisations and technical stakeholders including, but not limited to, medical and nursing schools, frontline health workers, senior paediatricians, policymakers and county officials, in developing and conducting embedded health research. This involves researchers embedding themselves in the contexts in which they carry out their research (mainly in county hospitals, local universities and other training institutions), creating and sustaining social networks. Researchers collaboratively worked with stakeholders to identify clinical, operational and behavioural issues related to routine service delivery, formulating and exploring research questions to bring change in practice PARTICIPANTS: We purposively selected 14 relevant stakeholders spanning policy, training institutions, healthcare workers, regulatory councils and professional associations.

RESULTS

The value of embeddedness is highlighted through the description of a recently completed project, Health Services that Deliver for Newborns (HSD-N). We describe how the HSD-N research process contributed to and further strengthened a collaborative research platform and illustrating this project's role in identifying and generating ideas about how to tackle health service delivery problems CONCLUSIONS: We conclude with a discussion about the experiences, challenges and lessons learned regarding engaging stakeholders in the coproduction of research.

Integrated knowledge translation to advance noncommunicable disease policy and practice in South Africa: application of the Exploration, Preparation, Implementation, and Sustainment (EPIS) framework

Background

In response to the “know–do” gap, several initiatives have been implemented to enhance evidence-informed decision-making (EIDM). These include individual training, organizational culture change management, and legislative changes. The importance of relationships and stakeholder engagement in EIDM has led to an evolution of models and approaches including integrated knowledge translation (IKT). IKT has emerged as a key strategy for ensuring that engagement is equitable, demand-driven, and responsive. As a result, the African-German Collaboration for Evidence-Based Healthcare and Public Health in Africa (CEBHA+) incorporated an IKT approach to influence noncommunicable diseases (NCD) policy and practice. We documented the phased process of developing, implementing, and monitoring the IKT approach in South Africa; and explored the appropriateness of using the exploration, preparation, implementation, and sustainment (EPIS) framework for this purpose.

Methods

We mapped the South Africa IKT approach onto the EPIS framework using a framework analysis approach. Notes of team meetings, stakeholder matrices, and engagement strategies were analysed and purposefully plotted against the four phases of the framework in order to populate the different constructs. We discussed and finalized the analysis in a series of online iterations until consensus was reached.

Results

The mapping exercise revealed an IKT approach that was much more iterative, dynamic, and engaging than initially thought. Several constructs (phase-agnostic) remained important and stable across EPIS phases: stable and supportive funding; committed and competent leadership; skilled and dedicated IKT champions; diverse and established personal networks; a conducive and enabling policy environment; and boundary-spanning intermediaries. Constructs such as “innovations” constantly evolved and adapted to the changing inner and outer contexts (phase-specific).

Conclusions

Using the EPIS framework to interrogate, reflect on, and document our IKT experiences proved extremely relevant and useful. Phase-agnostic constructs proved critical to ensure resilience and agility of NCD deliberations and policies in the face of highly dynamic and changing local contexts, particularly in view of the current coronavirus disease 2019 (COVID-19) pandemic. Bridging IKT with a framework from implementation science helps to reflect on this process and can guide the development and planning of similar interventions and strategies.

Forms and functions of bridging factors: specifying the dynamic links between outer and inner contexts during implementation and sustainment

BACKGROUND

Bridging factors are relational ties, formal arrangements, and processes that connect outer system and inner organizational contexts. They may be critical drivers of evidence-based practice (EBP) implementation and sustainment. Yet, the complex interplay between outer and inner contexts is often not considered. Bridging factors were recently defined in the updated Exploration, Preparation, Implementation, Sustainment (EPIS) framework. Further identification and specification of this construct will advance implementation models, measures, and methods. Our goal is to advance bridging factor research by identifying relevant dimensions and exemplifying these dimensions through illustrative case studies.

METHODS

We used a multiple case study design. Each case (n = 10) represented different contexts, EBPs, and bridging factor types. Inclusion criteria were the presence of clearly distinguishable outer and inner contexts, identifiable bridging factor, sufficient information to describe how the bridging factor affected implementation, and variation from other cases. We used an iterative qualitative inquiry process to develop and refine a list of dimensions. Case data were entered into a matrix. Dimensions comprised the rows and case details comprised the columns. After a review of all cases, we collectively considered and independently coded each dimension as function or form.

RESULTS

We drew upon the concepts of functions and forms, a distinction originally proposed in the complex health intervention literature. Function dimensions help define the bridging factor and illustrate its purpose as it relates to EBP implementation. Form dimensions describe the specific structures and activities that illustrate why and how the bridging factor has been customized to a local implementation experience. Function dimensions can help researchers and practitioners identify the presence and purpose of bridging factors, whereas form dimensions can help us understand how the bridging factor may be designed or modified to support EBP implementation in a specific context. We propose five function and three form bridging factor dimensions.

CONCLUSIONS

Bridging factors are described in many implementation models and studies, but without explicit reference or investigation. Bridging factors are an understudied and critical construct that requires further attention to facilitate implementation research and practice. We present specific recommendations for a bridging factors research agenda.

Understanding and investing in healthcare innovation and collaboration

PURPOSE

To understand the partnership between clinicians and academics who come together to provide high-quality care alongside research and innovation, identifying challenges and productive conditions for innovation and collaboration across multi-disciplinary teams.

DESIGN/METHODOLOGY/APPROACH

An explorative action research methodology was adopted. Semi-structured interviews were conducted with 15 clinical, academic and executive leads at a large metropolitan tertiary care hospital with an academic health services portfolio in the UK.

FINDINGS

Clinical leaders recognise the division of limited resource, restrictive employment contracts and the divergent priorities of each organisation as challenges hindering the collaborative process and derailing innovation. Developing a culture of respect, valuing and investing in individuals and allowing time and space for interaction help facilitate successful innovation and collaboration. Successfully leading collaborative innovation requires a combination of kindness, conviction and empowerment, alongside the articulation of a vision and accountability.

RESEARCH LIMITATIONS/IMPLICATIONS

Action research continues at this site, and further enquiry into the experiences, challenges and solutions of non-leaders when collaborating and innovating will be captured to present views across the organisation.

PRACTICAL IMPLICATIONS

Clinical and academic collaboration and innovation are essential to the continued success of healthcare. To ensure hospitals can continue to facilitate this in increasingly challenging circumstances, they must ensure longevity and stability of teams, devote time and resource to research and innovation, nurture interpersonal skills and develop kind and empowering leaders.

ORIGINALITY/VALUE

This work uniquely focuses on a real-time collaborative and innovative development. By employing action research while this development was happening, we were able to access the real time views of those at the centre of that collaboration. We offer insight into the challenges and effective solutions that consultant-level clinical leaders encounter when attempting to innovate and collaborate in practice.

Programme theory and linked intervention strategy for large-scale change to improve hospital care in a low and middle-income country - A Study Pre-Protocol

In low and middle-income countries (LMIC) general hospitals are important for delivering some key acute care services. Neonatal care is emblematic of these acute services as averting deaths requires skilled care over many days from multiple professionals with at least basic equipment. However, hospital care is often of poor quality and large-scale change is needed to improve outcomes. In this manuscript we aim to show how we have drawn upon our understanding of contexts of care in Kenyan general hospital NBUs, and on social and behavioural theories that offer potential mechanisms of change in these settings, to develop an initial programme theory guiding a large scale change intervention to improve neonatal care and outcomes.  Our programme theory is an expression of our assumptions about what actions will be both useful and feasible.  It incorporates a recognition of our strengths and limitations as a research-practitioner partnership to influence change. The steps we employ represent the initial programme theory development phase commonly undertaken in many Realist Evaluations. However, unlike many Realist Evaluations that develop initial programme theories focused on pre-existing interventions or programmes, our programme theory informs the design of a new intervention that we plan to execute. Within this paper we articulate briefly how we propose to operationalise this new intervention. Finally, we outline the quantitative and qualitative research activities that we will use to address specific questions related to the delivery and effects of this new intervention, discussing some of the challenges of such study designs. We intend that this research on the intervention will inform future efforts to revise the programme theory and yield transferable learning.

Programme theory and linked intervention strategy for large-scale change to improve hospital care in a low and middle-income country - A Study Pre-Protocol

In low and middle-income countries (LMIC) general hospitals are important for delivering some key acute care services. Neonatal care is emblematic of these acute services as averting deaths requires skilled care over many days from multiple professionals with at least basic equipment. However, hospital care is often of poor quality and large-scale change is needed to improve outcomes. In this manuscript we aim to show how we have drawn upon our understanding of contexts of care in Kenyan general hospital NBUs, and on social and behavioural theories that offer potential mechanisms of change in these settings, to develop an initial programme theory guiding a large scale change intervention to improve neonatal care and outcomes.  Our programme theory is an expression of our assumptions about what actions will be both useful and feasible.  It incorporates a recognition of our strengths and limitations as a research-practitioner partnership to influence change. The steps we employ represent the initial programme theory development phase commonly undertaken in many Realist Evaluations. However, unlike many Realist Evaluations that develop initial programme theories focused on pre-existing interventions or programmes, our programme theory informs the design of a new intervention that we plan to execute. Within this paper we articulate briefly how we propose to operationalise this new intervention. Finally, we outline the quantitative and qualitative research activities that we will use to address specific questions related to the delivery and effects of this new intervention, discussing some of the challenges of such study designs. We intend that this research on the intervention will inform future efforts to revise the programme theory and yield transferable learning.

Knowledge mobilization in bridging patient-practitioner-researcher boundaries: A systematic integrative review

AIM

To review when, how, and in what context knowledge mobilization (KMb) has crossed patient-practitioner-researcher boundaries.

BACKGROUND

KMb is essential in contemporary health care, yet little is known about how patients are engaged.

DESIGN

Integrative review.

DATA SOURCES

Ten academic databases and grey literature.

REVIEW METHODS

We followed integrative review methodology to identify publications from 2006-2019 which contributed to understanding of cross-boundary KMb. We extracted data using a bespoke spreadsheet and the Template for Intervention Description and Replication (TIDieR) framework. We used meta-summary to organize key findings.

RESULTS

Thirty-three papers collectively provide new insights into 'when' and 'how' KMb has crossed patient-researcher-practitioner boundaries and the impact this has achieved. Knowledge is mobilized to improve care, promote health, or prevent ill health. Most studies focus on creating or re-shaping knowledge to make it more useful. Knowledge is mobilized in small community groups, in larger networks, and intervention studies. Finding the right people to engage in activities is crucial, as activities can be demanding and time-consuming. Devolving power to communities and using local people to move knowledge can be effective. Few studies report definitive outcomes of KMb.

CONCLUSION

Cross-boundary KMb can and does produce new and shared knowledge for health care. Positive outcomes can be achieved using diverse public engagement strategies. KMb process and theory is an emerging discipline, further research is needed on effective cross-boundary working and on measuring the impact of KMb.

IMPACT

This review provides new and nuanced understandings of how KMb theory has been used to bridge patient-researcher-practitioner boundaries. We have assessed 'how', 'when', and in what context patients, practitioners and researchers have attempted to mobilize knowledge and identified impact. We have developed a knowledge base about good practice and what can and potentially should be avoided in cross-boundary KMb.

Bridging the research-practice gap in healthcare: a rapid review of research translation centres in England and Australia

Background

Large-scale partnerships between universities and health services are widely seen as vehicles for bridging the evidence–practice gap and for accelerating the adoption of new evidence in healthcare. Recently, different versions of these partnerships – often called academic health science centres – have been established across the globe. Although they differ in structure and processes, all aim to improve the integration of research and education with health services. Collectively, these entities are often referred to as Research Translation Centres (RTCs) and both England and Australia have developed relatively new and funded examples of these collaborative centres.

Methods

This paper presents findings from a rapid review of RTCs in Australia and England that aimed to identify their structures, leadership, workforce development and strategies for involving communities and service users. The review included published academic and grey literature with a customised search of the Google search engine and RTC websites.

Results

RTCs are complex system-level interventions that will need to disrupt the current paradigms and silos inherent in healthcare, education and research in order to meet their aims. This will require vision, leadership, collaborations and shared learnings, alongside structures, processes and strategies to deliver impact in the face of complexity. The impact of RTCs in overcoming the deeply entrenched silos across organisations, disciplines and sectors needs to be captured at the systems, organisation and individual levels. This includes workforce capacity and public and patient involvement that are vital to understanding the evolution of RTCs. In addition, new models of leadership are needed to support the brokering and mobilisation of knowledge in complex organisations.

Conclusions

The development and funding of RTCs represents one of the most significant shifts in the health research landscape and it is imperative that we continue to explore how we can progress the integration of research and healthcare and ensure research meets stakeholder needs and is translated via the collaborations supported by these organisations. Because RTCs are a recent addition to the healthcare landscape in Australia, it is instructive to review the processes and infrastructure needed to support their implementation and applied health research in England.

The use of the PARIHS framework in implementation research and practice-a citation analysis of the literature

BACKGROUND

The Promoting Action on Research Implementation in Health Services (PARIHS) framework was developed two decades ago and conceptualizes successful implementation (SI) as a function (f) of the evidence (E) nature and type, context (C) quality, and the facilitation (F), [SI = f (E,C,F)]. Despite a growing number of citations of theoretical frameworks including PARIHS, details of how theoretical frameworks are used remains largely unknown. This review aimed to enhance the understanding of the breadth and depth of the use of the PARIHS framework.

METHODS

This citation analysis commenced from four core articles representing the key stages of the framework's development. The citation search was performed in Web of Science and Scopus. After exclusion, we undertook an initial assessment aimed to identify articles using PARIHS and not only referencing any of the core articles. To assess this, all articles were read in full. Further data extraction included capturing information about where (country/countries and setting/s) PARIHS had been used, as well as categorizing how the framework was applied. Also, strengths and weaknesses, as well as efforts to validate the framework, were explored in detail.

RESULTS

The citation search yielded 1613 articles. After applying exclusion criteria, 1475 articles were read in full, and the initial assessment yielded a total of 367 articles reported to have used the PARIHS framework. These articles were included for data extraction. The framework had been used in a variety of settings and in both high-, middle-, and low-income countries. With regard to types of use, 32% used PARIHS in planning and delivering an intervention, 50% in data analysis, 55% in the evaluation of study findings, and/or 37% in any other way. Further analysis showed that its actual application was frequently partial and generally not well elaborated.

CONCLUSIONS

In line with previous citation analysis of the use of theoretical frameworks in implementation science, we also found a rather superficial description of the use of PARIHS. Thus, we propose the development and adoption of reporting guidelines on how framework(s) are used in implementation studies, with the expectation that this will enhance the maturity of implementation science.

Conceptualising the initiation of researcher and research user partnerships: a meta-narrative review

Background

Integrated knowledge translation refers to researcher and research user partnerships to co-generate and implement knowledge. This type of partnership may be critical to success in increasing knowledge use and impact, but the conceptualisation of its initiation has not been fully developed. Initiating this type of partnership has proven to be challenging but crucial to its success. The purpose of this study was to conduct a meta-narrative review of partnership initiation concepts, processes, enablers, barriers and outcomes in the disciplines of healthcare and social sciences where examples of researcher and research user partnerships were found.

Methods

Seven research traditions were identified. Three were in the discipline of social sciences (including psychology, education and business) and five were in the discipline of healthcare (including medicine, nursing, public health, health services research). Searches were conducted in MEDLINE, EMBASE, CINAHL, ABI Inform, ERIC, PsychInfo and the Cochrane Library on June 9, 2017. Fifty titles and abstracts were screened in triplicate; data were extracted from three records in duplicate. Narratives comprised of study characteristics and conceptual and empirical findings across traditions were tabulated, summarised and compared.

Results

A total of 7779 unique results were identified and 17 reviews published from 1998 to 2017 were eligible. All reviews identified a partnership initiation phase referred to as ‘early’ or ‘developmental’, or more vaguely as ‘fuzzy’, across six traditions – integrated knowledge translation, action research, stakeholder engagement, knowledge transfer, team initiation and shared mental models. The partnership initiation processes, enablers, barriers and outcomes were common to multiple narratives and summarised in a Partnership Initiation Conceptual Framework. Our review revealed limited use or generation of theory in most included reviews, and little empirical evidence testing the links between partnership initiation processes, enablers or barriers, and outcomes for the purpose of describing successful researcher and research user partnership initiation.

Conclusions

Narratives across multiple research traditions revealed similar integrated knowledge translation initiation processes, enablers, barriers and outcomes, which were captured in a conceptual framework that can be employed by researchers and research users to study and launch partnerships. While partnership initiation was recognised, it remains vaguely conceptualised despite lengthy research in several fields of study. Ongoing research of partnership initiation is needed to identify or generate relevant theory, and to empirically establish outcomes and the determinants of those outcomes.

Exploring the evolution of engagement between academic public health researchers and decision-makers: from initiation to dissolution

CONTEXT

Relationships between researchers and decision-makers have demonstrated positive potential to influence research, policy and practice. Over time, interest in better understanding the relationships between the two parties has grown as demonstrated by a plethora of studies globally. However, what remains elusive is the evolution of these vital relationships and what can be learned from them with respect to advancing evidence-informed decision-making. We therefore explored the nuances around the initiation, maintenance and dissolution of academic-government relationships.

METHODS

We conducted in-depth interviews with 52 faculty at one school of public health and 24 government decision-makers at city, state, federal and global levels. Interviews were transcribed and coded deductively and inductively using Atlas.Ti. Responses across codes and respondents were extracted into an Excel matrix and compared in order to identify key themes.

FINDINGS

Eight key drivers to engagement were identified, namely (1) decision-maker research needs, (2) learning, (3) access to resources, (4) student opportunities, (5) capacity strengthening, (6) strategic positioning, (7) institutional conditionalities, and (8) funder conditionalities. There were several elements that enabled initiation of relationships, including the role of faculty members in the decision-making process, individual attributes and reputation, institutional reputation, social capital, and the role of funders. Maintenance of partnerships was dependent on factors such as synergistic collaboration (i.e. both benefit), mutual trust, contractual issues and funding. Dissolution of relationships resulted from champions changing/leaving positions, engagement in transactional relationships, or limited mutual trust and respect.

CONCLUSIONS

As universities and government agencies establish relationships and utilise opportunities to share ideas, envision change together, and leverage their collaborations to use evidence to inform decision-making, a new modus operandi becomes possible. Embracing the individual, institutional, networked and systems dynamics of relationships can lead to new practices, alternate approaches and transformative change. Government agencies, schools of public health and higher education institutions more broadly, should pay deliberate attention to identifying and managing the various drivers, enablers and disablers for relationship initiation and resilience in order to promote more evidence-informed decision-making.

Knowledge mobilisation in practice: an evaluation of the Australian Prevention Partnership Centre

Background

Cross-sector collaborative partnerships are a vital strategy in efforts to strengthen research-informed policy and practice and may be particularly effective at addressing the complex problems associated with chronic disease prevention. However, there is still a limited understanding of how such partnerships are implemented in practice and how their implementation contributes to outcomes. This paper explores the operationalisation and outcomes of knowledge mobilisation strategies within the Australian Prevention Partnership Centre — a research collaboration between policy-makers, practitioners and researchers.

Methods

The Centre’s programme model identifies six knowledge mobilisation strategies that are hypothesised to be essential for achieving its objectives. Using a mixed methods approach combining stakeholder interviews, surveys, participant feedback forms and routine process data over a 5-year period, we describe the structures, resources and activities used to operationalise these strategies and explore if and how they have contributed to proximal outcomes.

Results

Results showed that Centre-produced research, resources, tools and methods were impacting policy formation and funding. Policy-makers reported using new practical methodologies that were helping them to design, implement, evaluate and obtain funding for scaled-up policies and programmes, and co-creating compelling prevention narratives. Some strategies were better implemented and more impactful than others in supporting these outcomes, with variation in who they worked for. The activities used to effect engagement, capacity-building and partnership formation were mostly generating positive results, but co-production could be enhanced by greater shared decision-making. Considerably more work is needed to successfully operationalise knowledge integration and adaptive learning.

Conclusions

Describing how collaborative cross-sector research partnerships are operationalised in practice, and with what effects, can provide important insights into practical strategies for establishing and growing such partnerships and for maximising their contributions to policy. Findings suggest that the Centre has many strengths but could benefit from more inclusive and transparent governance and internal processes that facilitate dialogue about roles, expectations and co-production practices.

From what we know to what we do: enhancing absorptive capacity in translational health research

Background

Globally, evidence about what works is slow to translate into frontline healthcare delivery. As a response, government policy has focused on translational health initiatives, such as the National Institute for Health Research funded Applied Research Collaborations in England. Concepts from organisation science prove useful to support such translational initiatives. We critique the application of two organisation science concepts linked to the broad domain of what is commonly termed ‘knowledge mobilisation’ in healthcare settings, specifically ‘knowledge brokers’ and ‘absorptive capacity’, to provide lessons for leaders of translational initiatives.

Results

The presence of knowledge brokers to ‘move from what we know to what we do’ in healthcare delivery appears necessary but insufficient to have a system level effect. To embed knowledge brokers in the wider healthcare system so they draw on various sources of evidence to discharge their role with greatest effect, we encourage leaders of translational health research initiatives to take account of the concept of absorptive capacity (ACAP) from the organisation science literature. Leaders should focus on enhancing ACAP though development of ‘co-ordination capabilities’. Such co-ordination capability should aim not just to acquire different types of evidence, but to ensure that all types of evidence are used to develop, implement and scale up healthcare delivery that best benefits patients. Specific co-ordination capabilities that support translation of evidence are: clinician involvement in research and its implementation; patient and public involvement in research and its implementation; business intelligence structures and processes at organisational and system level.

Conclusion

Attention to the dimensions and antecedents of ACAP, alongside the implementation of the knowledge brokering solution, in translational health research initiatives, is likely to better ensure the latter’s success.

What is the impact of large-scale implementation of stroke Early Supported Discharge? A mixed methods realist evaluation study protocol

BACKGROUND

Stroke Early Supported Discharge (ESD) is a service innovation that facilitates discharge from hospital and delivery of specialist rehabilitation in patients' homes. There is currently widespread implementation of ESD services in many countries, driven by robust clinical trial evidence. In England, the type of ESD service patients receive on the ground is variable, and in some regions, ESD is still not offered at all. This protocol presents a study designed to investigate the mechanisms and outcomes of implementing ESD at scale in real-world conditions. This will help to establish which models of ESD are most effective and in what context.

METHODS

A realist evaluation approach composed of two interlinking work packages will be adopted to investigate how and why ESD works, for whom and in what circumstances. Work package 1 (WP1) will begin with a rapid evidence synthesis to formulate preliminary realist hypotheses. Quantitative analyses of historical prospective Sentinel Stroke National Audit Programme (SSNAP) data will be performed to evaluate service outcomes based on the degree to which evidence-based ESD has been implemented. Work package 2 (WP2) will involve the qualitative investigation of purposively selected case study sites featuring in WP1 and covering different regions in England. The perspectives of clinicians, managers, commissioners, and service users will be explored qualitatively. Cost implications of ESD models will be examined using a cost-consequence analysis. Cross-case comparisons and triangulation of the data sources from both work packages will be performed to test, revise, and refine initial programme theories and address research aims.

DISCUSSION

This study will investigate whether and how current large-scale implementation of ESD is achieving the outcomes suggested by the evidence base. The theory-driven evaluation approach will highlight key mechanisms and contextual conditions necessary to optimise outcomes and allow us to draw transferable lessons to inform the effective implementation and sustainability of ESD in clinical practice. In addition, the methodological framework will progress the theoretical understanding of implementation and evaluation of complex rehabilitation interventions in stroke care.

TRIAL REGISTRATION

ISRCTN: 15568163, registration date: 26 October 2018.

How was a national moving and handling people guideline intended to work? The underlying programme theory

In healthcare, moving and handling people (MHP) often cause musculoskeletal disorders. To prevent musculoskeletal disorders due to MHP, many national evidence-based guidelines have been developed. However, little is known about how these guidelines were intended to work, i.e. their 'programme theory', how implementation by intended users is influenced by contextual factors and mechanisms to produce outcomes. This paper identifies the programme theory of a national MHP guideline (MHPG) using thematic analysis of the MHPG document, three organisational planning documents, and interviews with MHPG developers. The analysis identified the intended users of the MHPG as health and safety managers and MHP coordinators. The programme theory comprised contextual factors, potentially hindering (e.g. budget constraints) or facilitating (e.g. changing demographics) implementation, being influenced by mechanisms mainly based on ethical (quality of care, evidence-based practices), and economic reasoning (reducing cost of MHP, return on investment) to reduce injuries caused by MHP - the intended outcome.

Evidence use in equity focused health impact assessment: a realist evaluation

BACKGROUND

Equity-focused health impact assessment (EFHIA) can function as a framework and tool that supports users to collate data, information, and evidence related to health equity in order to identify and mitigate the impact of a current or proposed initiative on health inequities. Despite education efforts in both the clinical and public health settings, practitioners have found implementation and the use of evidence in completing equity focussed assessment tools to be challenging.

METHODS

We conducted a realist evaluation of evidence use in EFHIA in three phases: 1) developing propositions informed by a literature scan, existing theoretical frameworks, and stakeholder engagement; 2) data collection at four case study sites using online surveys, semi-structured interviews, document analysis, and observation; and 3) a realist analysis and identification of context-mechanism-outcome patterns and demi-regularities.

RESULTS

We identified limited use of academic evidence in EFHIA with two explanatory demi-regularities: 1) participants were unable to "identify with" academic sources, acknowledging that evidence based practice and use of academic literature was valued in their organization, but seen as less likely to provide answers needed for practice and 2) use of academic evidence was not associated with a perceived "positive return on investment" of participant energy and time. However, we found that knowledge brokering at the local site can facilitate evidence familiarity and manageability, increase user confidence in using evidence, and increase the likelihood of evidence use in future work.

CONCLUSIONS

The findings of this study provide a realist perspective on evidence use in practice, specifically for EFHIA. These findings can inform ongoing development and refinement of various knowledge translation interventions, particularly for practitioners delivering front-line public health services.

Can oral healthcare for older people be embedded into routine community aged care practice? A realist evaluation using normalisation process theory

BACKGROUND

An intervention 'Better Oral Health in Home Care' was introduced (2012-2014) to improve the oral health of older people receiving community aged care services. Implementation of the intervention was theoretically framed by the Promoting Action on Research Implementation in Health Services framework. Process outcomes demonstrated significant improvements in older people's oral health.

OBJECTIVE

To evaluate the extent to which the intervention has been embedded and sustained into routine community aged care practice 3 years after the initial implementation project.

DESIGN

A Realist Evaluation applying Normalisation Process Theory within a single case study setting.

SETTING

Community aged care (home care) provider in South Australia, Australia.

PARTICIPANTS

Purposeful sampling was undertaken. Twelve staff members were recruited from corporate, management and direct care positions. Two consumers representing high and low care recipients also participated.

METHODS

Qualitative methods were applied in two subcases, reflecting different contextual settings. Data were collected via semi-structured interviews and analysed deductively by applying the Normalisation Process Theory core constructs (with the recommended phases of the Realist Evaluation cycle). Retrospective and prospective analytic methods investigated how the intervention has been operationalised by comparing two timeframes: Time 1 (Implementation June 2012-December 2014) and Time 2 (Post-implementation July 2017-July 2018).

RESULTS

At Time 1, the initial program theory proposed that multi-level facilitation contributed to a favourable context that triggered positive mechanisms supportive of building organisational and workforce oral healthcare capacity. At Time 2, an alternative program theory of how the intervention has unfolded in practice described a changed context following the withdrawal of the project facilitation processes with the triggering of alternative mechanisms that have made it difficult for staff to embed sustainable practice.

CONCLUSION

Findings concur with the literature that successful implementation outcomes do not necessarily guarantee sustainability. The study has provided a deeper explanation of how contextual characteristics have contributed to the conceptualisation of oral healthcare as a low priority, basic work-ready personal care task and how this, in turn, hindered the embedding of sustainable oral healthcare into routine community aged care practice. This understanding can be used to better inform the development of strategies, such as multi-level facilitation, needed to navigate contextual barriers so that sustainable practice can be achieved.

Reshaping public hospitals: an agenda for reform in Asia and the Pacific

Hospitals in the Asia-Pacific today face the 'triple aim' challenge, proposed by the Institute for Healthcare Improvement, of how to improve quality of care and population health, while at the same time controlling healthcare costs. Yet, pursuing these challenges in combination is presently a remote prospect for many hospitals and, indeed, in a majority of countries in the region. The roles and functions of the public hospital sector within local health systems need redefinition and reform in the context of demographic and epidemiological transitions. Policymakers, managers and health professionals have an obligation to reshape the future of public hospitals. This article outlines actions for how public hospitals can be reshaped from a health system perspective. First, hospitals should be integrated into the fabric of the local health system; they can lead in this through working in alliances with other healthcare facilities, including primary care and private hospitals. Policymakers have a role in facilitating this as it contributes to health improvement of the population. Second, investments in system innovation, management improvement and information systems are required and their impact assessed. Such investments can contribute to cost control and efficiency. Public hospital sector investments should be strategic, efficient and should not bias investment in broader determinants of health. Third, reorienting health workforce competencies and appropriate skills should be central to hospital sector reforms, from policy to frontline services delivery. Creative thinking is needed to build and support flexible care delivery arrangements for services designed to respond to patients ' and providers' needs. Pivotal to achievement of each of these three areas of reform is good governance and leadership.

What is a research derived actionable tool, and what factors should be considered in their development? A Delphi study

BACKGROUND

Research findings should be disseminated appropriately to generate maximum impact. The development of research derived 'actionable' tools (RDAT) as research outputs may contribute to impact in health services and health systems research. However there is little agreement on what is meant by actionable tool or what can make them useful. We set out to develop a consensus definition of what is meant by a RDAT and to identify characteristics of a RDAT that would support its use across the research-practice boundary.

METHODS

A modified Delphi method was used with a panel of 33 experts comprising of researchers, research funders, policy makers and practitioners. Three rounds were administered including an initial workshop, followed by two online surveys comprising of Likert scales supplemented with open-ended questions. Consensus was defined at 75% agreement.

RESULTS

Consensus was reached for the definition and characteristics of RDATs, and on considerations that might maximize their use. The panel also agreed how RDATs could become integral to primary research methods, conduct and reporting. A typology of RDATs did not reach consensus.

CONCLUSIONS

A group of experts agreed a definition and characteristics of RDATs that are complementary to peer reviewed publications. The importance of end users shaping such tools was seen as of paramount importance. The findings have implications for research funders to resource such outputs in funding calls. The research community might consider developing and applying skills to coproduce RDATs with end users as part of the research process. Further research is needed on tracking the impact of RDATs, and defining a typology with a range of end-users.

Factors influencing early stage healthcare-academia partnerships

Purpose The purpose of this paper is to explore factors influencing early implementation and intermediate outcomes of a healthcare-academia partnership in a primary healthcare setting. Design/methodology/approach The Academic Primary Healthcare Network (APHN) initiative was launched in 2011 in Stockholm County, Sweden and included 201 primary healthcare centres. Semi-structured interviews were conducted in 2013-2014 with all coordinating managers ( n=8) and coordinators ( n=4). A strategic change model framework was used to collect and analyse data. Findings Several factors were identified to aid early implementation: assignment and guidelines that allowed flexibility; supportive management; dedicated staff; facilities that enabled APHN actions to be integrated into healthcare practice; and positive experiences from research and educational activities. Implementation was hindered by: discrepancies between objectives and resources; underspecified guidelines that trigger passivity; limited research and educational activities; a conflicting non-supportive reimbursement system; limited planning; and organisational fragmentation. Intermediate outcomes revealed that various actions, informed by the APHN assignment, were launched in all APHNs. Practical implications The findings can be rendered applicable by preparing stakeholders in healthcare services to optimise early implementation of healthcare-academia partnerships. Originality/value This study increases understanding of interactions between factors that influence early stage partnerships between healthcare services and academia in primary healthcare settings.

Learning from the emergence of NIHR Collaborations for Leadership in Applied Health Research and Care (CLAHRCs): a systematic review of evaluations

Background

Collaborations for Leadership in Applied Health Research and Care (CLAHRCs) were funded by NIHR in England in 2008 and 2014 as partnerships between universities and surrounding health service organisations, focused on improving the quality of healthcare through the conduct and application of applied health research. The aim of this review is to synthesise learning from evaluations of the CLAHRCs.

Methods

Fifteen databases including CINAHL, MEDLINE, EMBASE and PsycINFO were searched to identify any evaluations of CLAHRCs. Current and archived CLAHRC websites and the reference lists of retrieved articles were scanned to identify any additional evaluations. Searches were restricted to English language only. Any publications from evaluations of the CLAHRCs were eligible for inclusion if they fulfilled at least one of three pre-specified inclusion criteria. A narrative synthesis was undertaken.

Results

Twenty-six evaluations (reported in 37 papers) were deemed eligible for inclusion. Evaluations focused on describing and exploring the formative partnerships, vision, values, structures and processes of CLAHRCs; the nature and role of boundaries; the deployment of knowledge brokers and hybrid roles to support knowledge mobilisation; patient and public involvement; and capacity building. The relative lack of data about the early impact of CLAHRCs on health care provision or outcomes is notable.

Conclusions

Much of the evaluative focus on CLAHRCs has been on how they have been organised and on the development of theory around their emergent properties. Evidence is lacking on the impact of CLAHRCs particularly in relation to the knowledge mobilisation processes and practices adopted. Further evaluation of CLAHRCs and other similar research and practice partnerships is warranted and should focus on which knowledge mobilisation approaches work where, how and why.

Trial registration

PROSPERO (Registration number: CRD42016042945).

Electronic supplementary material

The online version of this article (10.1186/s13012-018-0805-y) contains supplementary material, which is available to authorized users.

Leading top-down implementation processes: a qualitative study on the role of managers

Background

Leadership has been identified as an influential factor in implementation processes in healthcare organizations. However, the processes through which leaders affect implementation outcomes are largely unknown. The purpose of this study is to analyse how managers interpret and make sense of a large scale top-down implementation initiative and what implications this has for the implementation process. This was studied at the implementation of an academic primary healthcare initiative covering 210 primary healthcare centres in central Sweden. The aim of the initiative was to integrate research and education into regular primary healthcare services.

Methods

The study builds on 16 in-depth individual semi-structured interviews with all managers (n = 8) who had operative responsibility for the implementation. Each manager was interviewed twice during the initial phase of the implementation. Data were analysed using a thematic approach guided by theory on managerial role taking based on the Transforming Experience Framework.

Results

How the managers interpreted and made sense of the implementation task built on three factors: how they perceived the different parts of the initiative, how they perceived themselves in relation to these parts, and the resources available for the initiative. Based on how they combined these three factors the managers chose to integrate or separate the different parts of the initiative in their management of the implementation process.

Conclusions

This research emphasizes that managers in healthcare seem to have a substantial impact on how and to what extent different tasks are addressed and prioritized in top-down implementation processes. This has policy implications. To achieve intended implementation outcomes, the authors recognize the necessity of an early and on-going dialogue about how the implementation is perceived by the managers responsible for the implementation.

Electronic supplementary material

The online version of this article (10.1186/s12913-018-3360-y) contains supplementary material, which is available to authorized users.

Collaborative and partnership research for improvement of health and social services: researcher's experiences from 20 projects

BACKGROUND

Getting research into policy and practice in healthcare is a recognised, world-wide concern. As an attempt to bridge the gap between research and practice, research funders are requesting more interdisciplinary and collaborative research, while actual experiences of such processes have been less studied. Accordingly, the purpose of this study was to gain more knowledge on the interdisciplinary, collaborative and partnership research process by investigating researchers' experiences of and approaches to the process, based on their participation in an inventive national research programme. The programme aimed to boost collaborative and partnership research and build learning structures, while improving ways to lead, manage and develop practices in Swedish health and social services.

METHODS

Interviews conducted with project leaders and/or lead researchers and documentation from 20 projects were analysed using directed and conventional content analysis.

RESULTS

Collaborative approaches were achieved by design, e.g. action research, or by involving practitioners from several levels of the healthcare system in various parts of the research process. The use of dual roles as researcher/clinician or practitioner/PhD student or the use of education designed especially for practitioners or 'student researchers' were other approaches. The collaborative process constituted the area for the main lessons learned as well as the main problems. Difficulties concerned handling complexity and conflicts between different expectations and demands in the practitioner's and researcher's contexts, and dealing with human resource issues and group interactions when forming collaborative and interdisciplinary research teams. The handling of such challenges required time, resources, knowledge, interactive learning and skilled project management.

CONCLUSIONS

Collaborative approaches are important in the study of complex phenomena. Results from this study show that allocated time, arenas for interactions and skills in project management and communication are needed during research collaboration to ensure support and build trust and understanding with involved practitioners at several levels in the healthcare system. For researchers, dealing with this complexity takes time and energy from the scientific process. For practitioners, this puts demands on understanding a research process and how it fits with on-going organisational agendas and activities and allocating time. Some of the identified factors may be overlooked by funders and involved stakeholders when designing, performing and evaluating interdisciplinary, collaborative and partnership research.

Models and applications for measuring the impact of health research: update of a systematic review for the Health Technology Assessment programme

BACKGROUND

This report reviews approaches and tools for measuring the impact of research programmes, building on, and extending, a 2007 review.

OBJECTIVES

(1) To identify the range of theoretical models and empirical approaches for measuring the impact of health research programmes; (2) to develop a taxonomy of models and approaches; (3) to summarise the evidence on the application and use of these models; and (4) to evaluate the different options for the Health Technology Assessment (HTA) programme.

DATA SOURCES

We searched databases including Ovid MEDLINE, EMBASE, Cumulative Index to Nursing and Allied Health Literature and The Cochrane Library from January 2005 to August 2014.

REVIEW METHODS

This narrative systematic literature review comprised an update, extension and analysis/discussion. We systematically searched eight databases, supplemented by personal knowledge, in August 2014 through to March 2015.

RESULTS

The literature on impact assessment has much expanded. The Payback Framework, with adaptations, remains the most widely used approach. It draws on different philosophical traditions, enhancing an underlying logic model with an interpretative case study element and attention to context. Besides the logic model, other ideal type approaches included constructionist, realist, critical and performative. Most models in practice drew pragmatically on elements of several ideal types. Monetisation of impact, an increasingly popular approach, shows a high return from research but relies heavily on assumptions about the extent to which health gains depend on research. Despite usually requiring systematic reviews before funding trials, the HTA programme does not routinely examine the impact of those trials on subsequent systematic reviews. The York/Patient-Centered Outcomes Research Institute and the Grading of Recommendations Assessment, Development and Evaluation toolkits provide ways of assessing such impact, but need to be evaluated. The literature, as reviewed here, provides very few instances of a randomised trial playing a major role in stopping the use of a new technology. The few trials funded by the HTA programme that may have played such a role were outliers.

DISCUSSION

The findings of this review support the continued use of the Payback Framework by the HTA programme. Changes in the structure of the NHS, the development of NHS England and changes in the National Institute for Health and Care Excellence's remit pose new challenges for identifying and meeting current and future research needs. Future assessments of the impact of the HTA programme will have to take account of wider changes, especially as the Research Excellence Framework (REF), which assesses the quality of universities' research, seems likely to continue to rely on case studies to measure impact. The HTA programme should consider how the format and selection of case studies might be improved to aid more systematic assessment. The selection of case studies, such as in the REF, but also more generally, tends to be biased towards high-impact rather than low-impact stories. Experience for other industries indicate that much can be learnt from the latter. The adoption of researchfish^® (researchfish Ltd, Cambridge, UK) by most major UK research funders has implications for future assessments of impact. Although the routine capture of indexed research publications has merit, the degree to which researchfish will succeed in collecting other, non-indexed outputs and activities remains to be established.

LIMITATIONS

There were limitations in how far we could address challenges that faced us as we extended the focus beyond that of the 2007 review, and well beyond a narrow focus just on the HTA programme.

CONCLUSIONS

Research funders can benefit from continuing to monitor and evaluate the impacts of the studies they fund. They should also review the contribution of case studies and expand work on linking trials to meta-analyses and to guidelines.

FUNDING

The National Institute for Health Research HTA programme.

An integrated care programme in London: qualitative evaluation

PURPOSE

A well-funded, four-year integrated care programme was implemented in south London. The programme attempted to integrate care across primary, acute, community, mental health and social care. The purpose of this paper is to reduce hospital admissions and nursing home placements. Programme evaluation aimed to identify what worked well and what did not; lessons learnt; the value of integrated care investment.

DESIGN/METHODOLOGY/APPROACH

Qualitative data were obtained from documentary analysis, stakeholder interviews, focus groups and observational data from programme meetings. Framework analysis was applied to stakeholder interview and focus group data in order to generate themes.

FINDINGS

The integrated care project had not delivered expected radical reductions in hospital or nursing home utilisation. In response, the scheme was reformulated to focus on feasible service integration. Other benefits emerged, particularly system transformation. Nine themes emerged: shared vision/case for change; interventions; leadership; relationships; organisational structures and governance; citizens and patients; evaluation and monitoring; macro level. Each theme was interpreted in terms of "successes", "challenges" and "lessons learnt".

RESEARCH LIMITATIONS/IMPLICATIONS

Evaluation was hampered by lack of a clear evaluation strategy from programme inception to conclusion, and of the evidence required to corroborate claims of benefit.

PRACTICAL IMPLICATIONS

Key lessons learnt included: importance of strong clinical leadership, shared ownership and inbuilt evaluation.

ORIGINALITY/VALUE

Primary care was a key player in the integrated care programme. Initial resistance delayed implementation and related to concerns about vertical integration and scepticism about unrealistic goals. A focus on clinical care and shared ownership contributed to eventual system transformation.

How to Improve Integrated Care for People with Chronic Conditions: Key Findings from EU FP-7 Project INTEGRATE and Beyond

BACKGROUND

Political and public health leaders increasingly recognize the need to take urgent action to address the problem of chronic diseases and multi-morbidity. European countries are facing unprecedented demand to find new ways to deliver care to improve patient-centredness and personalization, and to avoid unnecessary time in hospitals. People-centred and integrated care has become a central part of policy initiatives to improve the access, quality, continuity, effectiveness and sustainability of healthcare systems and are thus preconditions for the economic sustainability of the EU health and social care systems.

PURPOSE

This study presents an overview of lessons learned and critical success factors to policy making on integrated care based on findings from the EU FP-7 Project Integrate, a literature review, other EU projects with relevance to this study, a number of best practices on integrated care and our own experiences with research and policy making in integrated care at the national and international level.

RESULTS

Seven lessons learned and critical success factors to policy making on integrated care were identified.

CONCLUSION

The lessons learned and critical success factors to policy making on integrated care show that a comprehensive systems perspective should guide the development of integrated care towards better health practices, education, research and policy.

Exploring Proxy Measures of Mutuality for Strategic Partnership Development: A Case Study

Partnerships between academic and clinical-based health organizations are becoming increasingly important in improving health outcomes. Mutuality is recognized as a vital component of these partnerships. If partnerships are to achieve mutuality, there is a need to define what it means to partnering organizations. Few studies have described the elements contributing to mutuality, particularly in new relationships between academic and clinical partners. This study seeks to identify how mutuality is expressed and to explore potential proxy measures of mutuality for an alliance consisting of a hospital system and a School of Public Health. Key informant interviews were conducted with faculty and hospital representatives serving on the partnership steering committee. Key informants were asked about perceived events that led to the development of the Alliance; perceived goals, expectations, and outcomes; and current/future roles with the Alliance. Four proxy measures of mutuality for an academic–clinical partnership were identified: policy directives, community beneficence, procurement of human capital, and partnership longevity. Findings can inform the development of tools for assisting in strengthening relationships and ensuring stakeholders’ interests align with the mission and goal of the partnership by operationalizing elements necessary to evaluate the progress of the partnership.

Evidence-based policymaking is not like evidence-based medicine, so how far should you go to bridge the divide between evidence and policy?

There is extensive health and public health literature on the ‘evidence-policy gap’, exploring the frustrating experiences of scientists trying to secure a response to the problems and solutions they raise and identifying the need for better evidence to reduce policymaker uncertainty. We offer a new perspective by using policy theory to propose research with greater impact, identifying the need to use persuasion to reduce ambiguity, and to adapt to multi-level policymaking systems.

We identify insights from secondary data, namely systematic reviews, critical analysis and policy theories relevant to evidence-based policymaking. The studies are drawn primarily from countries such as the United States, United Kingdom, Canada, Australia and New Zealand. We combine empirical and normative elements to identify the ways in which scientists can, do and could influence policy.

We identify two important dilemmas, for scientists and researchers, that arise from our initial advice. First, effective actors combine evidence with manipulative emotional appeals to influence the policy agenda – should scientists do the same, or would the reputational costs outweigh the policy benefits? Second, when adapting to multi-level policymaking, should scientists prioritise ‘evidence-based’ policymaking above other factors? The latter includes governance principles such the ‘co-production’ of policy between local public bodies, interest groups and service users. This process may be based primarily on values and involve actors with no commitment to a hierarchy of evidence.

We conclude that successful engagement in ‘evidence-based policymaking’ requires pragmatism, combining scientific evidence with governance principles, and persuasion to translate complex evidence into simple stories. To maximise the use of scientific evidence in health and public health policy, researchers should recognise the tendency of policymakers to base judgements on their beliefs, and shortcuts based on their emotions and familiarity with information; learn ‘where the action is’, and be prepared to engage in long-term strategies to be able to influence policy; and, in both cases, decide how far you are willing to go to persuade policymakers to act and secure a hierarchy of evidence underpinning policy. These are value-driven and political, not just ‘evidence-based’, choices.

Partnering to improve care: the case of the Veterans’ Health Administration’s Quality Enhancement Research Initiative

Background

Within many large health care organizations, researchers and operations partners (i.e., policymakers, managers, clinical leaders) join to conduct studies to improve the quality of patient care. Yet optimal approaches to conducting partnership research and evaluation are only beginning to be clearly defined. The Veterans’ Health Administration (VA) Quality Enhancement Research Initiative (QUERI), funded by operations leaders and administered by the VA’s research service, now has nearly two decades of experience in fostering research–operations partnerships for improving quality of VA care. The work reported here is part of a national evaluation of QUERI. Because individuals in research and operations often have differing backgrounds and perspectives, we aim to identify the main sources of tension in research–operations partnerships and strategies for maximizing partnership success, through the eyes of QUERI participants.

Methods

We conducted semi-structured interviews with 116 researchers and operations partners chosen randomly from within pre-identified key participant groups. We conducted inductive qualitative analysis of verbatim interview transcripts, limited to the 89 interviews of individuals reporting at least some familiarity with QUERI.

Results

Tensions in research–operations partnerships were primarily related to diverging incentives and to differing values placed on scientific rigor or integrity versus quick timelines. To alleviate these tensions, operations’ partners highlighted the importance of ‘perspective-taking’ (i.e., putting themselves into the shoes of the researchers) to ensure a mutually beneficial and attractive partnership, whereas researchers identified the importance of overcoming the need for recognition to be apportioned between either research or operations for achieved results. Both researchers and operations participants identified jointly designing each partnership from the beginning, minimizing research bureaucracy burdens, and prioritizing in-person communication and long-term relationships as key partnership building blocks.

Conclusions

QUERI research and operations participants had largely concordant views on partnership tensions and approaches for improving partnership success. The fact that only researchers mentioned moving beyond recognition for the results achieved and only operations staff mentioned the importance of ‘perspective-taking’ suggests, however, that there may be unresolved tensions. These results suggest that researchers may benefit from better aligning of academic incentives with contributions to the health care organization and establishing formal recognition of operational impacts of research, while preserving some flexibility and independence of the research process.

What do we think we are doing? How might a clinical information network be promoting implementation of recommended paediatric care practices in Kenyan hospitals?

Background

The creation of a clinical network was proposed as a means to promote implementation of a set of recommended clinical practices targeting inpatient paediatric care in Kenya. The rationale for selecting a network as a strategy has been previously described. Here, we aim to describe network activities actually conducted over its first 2.5 years, deconstruct its implementation into specific components and provide our ‘insider’ interpretation of how the network is functioning as an intervention.

Methods

We articulate key activities that together have constituted network processes over 2.5 years and then utilise a recently published typology of implementation components to give greater granularity to this description from the perspective of those delivering the intervention. Using the Behaviour Change Wheel we then suggest how the network may operate to achieve change and offer examples of change before making an effort to synthesise our understanding in the form of a realist context–mechanism–outcome configuration.

Results

We suggest our network is likely to comprise 22 from a total of 73 identifiable intervention components, of which 12 and 10 we consider major and minor components, respectively. At the policy level, we employed clinical guidelines, marketing and communication strategies with intervention characteristics operating through incentivisation, persuasion, education, enablement, modelling and environmental restructuring. These might influence behaviours by enhancing psychological capability, creating social opportunity and increasing motivation largely through a reflective pathway.

Conclusions

We previously proposed a clinical network as a solution to challenges implementing recommended practices in Kenyan hospitals based on our understanding of theory and context. Here, we report how we have enacted what was proposed and use a recent typology to deconstruct the intervention into its elements and articulate how we think the network may produce change. We offer a more generalised statement of our theory of change in a context–mechanism–outcome configuration. We hope this will complement a planned independent evaluation of ‘how things work’, will help others interpret results of change reported more formally in the future and encourage others to consider further examination of networks as means to scale up improvement practices in health in lower income countries.

Clinical interventions, implementation interventions, and the potential greyness in between -a discussion paper

BACKGROUND

There is increasing awareness that regardless of the proven value of clinical interventions, the use of effective strategies to implement such interventions into clinical practice is necessary to ensure that patients receive the benefits. However, there is often confusion between what is the clinical intervention and what is the implementation intervention. This may be caused by a lack of conceptual clarity between 'intervention' and 'implementation', yet at other times by ambiguity in application. We suggest that both the scientific and the clinical communities would benefit from greater clarity; therefore, in this paper, we address the concepts of intervention and implementation, primarily as in clinical interventions and implementation interventions, and explore the grey area in between.

DISCUSSION

To begin, we consider the similarities, differences and potential greyness between clinical interventions and implementation interventions through an overview of concepts. This is illustrated with reference to two examples of clinical interventions and implementation intervention studies, including the potential ambiguity in between. We then discuss strategies to explore the hybridity of clinical-implementation intervention studies, including the role of theories, frameworks, models, and reporting guidelines that can be applied to help clarify the clinical and implementation intervention, respectively.

CONCLUSION

Semantics provide opportunities for improved precision in depicting what is 'intervention' and what is 'implementation' in health care research. Further, attention to study design, the use of theory, and adoption of reporting guidelines can assist in distinguishing between the clinical intervention and the implementation intervention. However, certain aspects may remain unclear in analyses of hybrid studies of clinical and implementation interventions. Recognizing this potential greyness can inform further discourse.

Getting the most out of knowledge and innovation transfer agents in health care: a qualitative study

Background

Knowledge and innovation transfer (KIT) is recognised internationally as a complex, dynamic process that is difficult to embed in organisations. There is growing use of health service–academic–industry collaborations in the UK, with knowledge brokers linking producers with the users of knowledge and innovation.

Aim

Focusing on KIT ‘agent’ roles within Academic Health Science Networks in England and Partnerships in Wales, we show how individual dispositions, processes and content contribute to desired outcomes.

Methods

We studied the KIT intentions of all Academic Health Science Networks in England, and the South East Wales Academic Health Science Partnership. Using a qualitative case study design, we studied the work of 13 KIT agents purposively sampled from five networks, by collecting data from observation of meetings, documentation, KIT agent audio-diaries, and semistructured interviews with KIT agents, their line managers and those they supported (‘Links’). We also used a consensus method in a meeting of experts (nominal group technique) to discuss the measurement of outcomes of KIT agent activity.

Findings

The case study KIT agents were predominantly from a clinical background with differing levels of experience and expertise, with the shared aim of improving services and patient care. Although outside of recognised career structures, the flexibility afforded to KIT agents to define their role was an enabler of success. Other helpful factors included (1) time and resources to devote to KIT activity; (2) line manager support and a team to assist in the work; and (3) access and the means to use data for improvement projects. The organisational and political context could be challenging. KIT agents not only tackled local barriers such as siloed working, but also navigated shifting regional and national policies. Board-level support for knowledge mobilisation together with a culture of reflection (listening to front-line staff), openness to challenges and receptivity to research all enabled KIT agents to achieve desired outcomes. Nominal group findings underscored the importance of relating measures to specific intended outcomes. However, the case studies highlighted that few measures were employed by KIT agents and their managers. Using social marketing theory helped to show linkages between processes, outcomes and impact, and drew attention to how KIT agents developed insight into their clients’ needs and tailored work accordingly.

Limitations

Level of KIT agent participation varied; line managers and Links were interviewed only once; and outcomes were self-reported.

Conclusions

Social marketing theory provided a framework for analysing KIT agent activity. The preparatory work KIT agents do in listening, understanding local context and building relationships enabled them to develop ‘insight’ and adapt their ‘offer’ to clients to achieve desired outcomes.

Future work

The complexity of the role and the environment in which it is played out justifies more research on KIT agents. Suggestions include (1) longitudinal study of career pathways; (2) how roles are negotiated within teams and how competing priorities are managed; (3) how success is measured; (4) the place of improvement methodologies within KIT work; (5) the application of social marketing theory to comparative study of similar roles; and (6) patients as KIT agents.

Funding

The National Institute for Health Research Health Services and Delivery Research programme.

Protocol for process evaluation of a randomised controlled trial of family-led rehabilitation post stroke (ATTEND) in India

INTRODUCTION

We are undertaking a randomised controlled trial (fAmily led rehabiliTaTion aftEr stroke in INDia, ATTEND) evaluating training a family carer to enable maximal rehabilitation of patients with stroke-related disability; as a potentially affordable, culturally acceptable and effective intervention for use in India. A process evaluation is needed to understand how and why this complex intervention may be effective, and to capture important barriers and facilitators to its implementation. We describe the protocol for our process evaluation to encourage the development of in-process evaluation methodology and transparency in reporting.

METHODS AND ANALYSIS

The realist and RE-AIM (Reach, Effectiveness, Adoption, Implementation and Maintenance) frameworks informed the design. Mixed methods include semistructured interviews with health providers, patients and their carers, analysis of quantitative process data describing fidelity and dose of intervention, observations of trial set up and implementation, and the analysis of the cost data from the patients and their families perspective and programme budgets. These qualitative and quantitative data will be analysed iteratively prior to knowing the quantitative outcomes of the trial, and then triangulated with the results from the primary outcome evaluation.

ETHICS AND DISSEMINATION

The process evaluation has received ethical approval for all sites in India. In low-income and middle-income countries, the available human capital can form an approach to reducing the evidence practice gap, compared with the high cost alternatives available in established market economies. This process evaluation will provide insights into how such a programme can be implemented in practice and brought to scale. Through local stakeholder engagement and dissemination of findings globally we hope to build on patient-centred, cost-effective and sustainable models of stroke rehabilitation.

TRIAL REGISTRATION NUMBER

CTRI/2013/04/003557.

The Human Capital of Knowledge Brokers: An analysis of attributes, capacities and skills of academic teaching and research faculty at Kenyan schools of public health

BACKGROUND

Academic faculty involved in public health teaching and research serve as the link and catalyst for knowledge synthesis and exchange, enabling the flow of information resources, and nurturing relations between 'two distinct communities' - researchers and policymakers - who would not otherwise have the opportunity to interact. Their role and their characteristics are of particular interest, therefore, in the health research, policy and practice arena, particularly in low- and middle-income countries. We investigated the individual attributes, capacities and skills of academic faculty identified as knowledge brokers (KBs) in schools of public health (SPH) in Kenya with a view to informing organisational policies around the recruitment, retention and development of faculty KBs.

METHODS

During April 2013, we interviewed 12 academics and faculty leadership (including those who had previously been identified as KBs) from six SPHs in Kenya, and 11 national health policymakers with whom they interact. Data were qualitatively analyzed using inductive thematic analysis to unveil key characteristics.

RESULTS

Key characteristics of KBs fell into five categories: sociodemographics, professional competence, experiential knowledge, interactive skills and personal disposition. KBs' reputations benefitted from their professional qualifications and content expertise. Practical knowledge in policy-relevant situations, and the related professional networks, allowed KB's to navigate both the academic and policy arenas and also to leverage the necessary connections required for policy influence. Attributes, such as respect and a social conscience, were also important KB characteristics.

CONCLUSION

Several changes in Kenya are likely to compel academics to engage increasingly with policymakers at an enhanced level of debate, deliberation and discussion in the future. By recognising existing KBs, supporting the emergence of potential KBs, and systematically hiring faculty with KB-specific characteristics, SPHs can enhance their collective human capital and influence on public health policy and practice. Capacity strengthening of tangible skills and recognition of less tangible personality characteristics could contribute to enhanced academic-policymaker networks. These, in turn, could contribute to the relevance of SPH research and teaching programs as well as evidence-informed public health policies.

RAMESES II reporting standards for realist evaluations

BACKGROUND

Realist evaluation is increasingly used in health services and other fields of research and evaluation. No previous standards exist for reporting realist evaluations. This standard was developed as part of the RAMESES II project. The project's aim is to produce initial reporting standards for realist evaluations.

METHODS

We purposively recruited a maximum variety sample of an international group of experts in realist evaluation to our online Delphi panel. Panel members came from a variety of disciplines, sectors and policy fields. We prepared the briefing materials for our Delphi panel by summarising the most recent literature on realist evaluations to identify how and why rigour had been demonstrated and where gaps in expertise and rigour were evident. We also drew on our collective experience as realist evaluators, in training and supporting realist evaluations, and on the RAMESES email list to help us develop the briefing materials. Through discussion within the project team, we developed a list of issues related to quality that needed to be addressed when carrying out realist evaluations. These were then shared with the panel members and their feedback was sought. Once the panel members had provided their feedback on our briefing materials, we constructed a set of items for potential inclusion in the reporting standards and circulated these online to panel members. Panel members were asked to rank each potential item twice on a 7-point Likert scale, once for relevance and once for validity. They were also encouraged to provide free text comments.

RESULTS

We recruited 35 panel members from 27 organisations across six countries from nine different disciplines. Within three rounds our Delphi panel was able to reach consensus on 20 items that should be included in the reporting standards for realist evaluations. The overall response rates for all items for rounds 1, 2 and 3 were 94 %, 76 % and 80 %, respectively.

CONCLUSION

These reporting standards for realist evaluations have been developed by drawing on a range of sources. We hope that these standards will lead to greater consistency and rigour of reporting and make realist evaluation reports more accessible, usable and helpful to different stakeholders.

Does the 'diffusion of innovations' model enrich understanding of research use? Case studies of the implementation of thrombolysis services for stroke

OBJECTIVE

To test whether the model of 'diffusion of innovations' enriches understanding of the implementation of evidence-based thrombolysis services for stroke patients.

METHODS

Four case studies of the implementation of evidence on thrombolysis in stroke services in England and Sweden. Semistructured interviews with 95 staff including doctors, nurses and managers working in stroke units, emergency medicine, radiology, the ambulance service, community rehabilitation services and commissioners.

RESULTS

The implementation of thrombolysis in acute stroke management benefited from a critical mass of the factors featured in the model including: the support of national and local opinion leaders; a strong evidence base and financial incentives. However, while the model provided a starting point as an organizational framework for mapping the critical factors influencing implementation, to understand properly the process of implementation and the importance of the different factors identified, more detailed analyses of context and, in particular, of the human and social dimensions of change was needed.

CONCLUSIONS

While recognising the usefulness of the model of diffusion of innovations in mapping the processes by which diffusion occurs, the use of methods that lend themselves to in-depth analysis, such as ethnography and the application of relevant bodies of social theory, are needed.

Figuring out fidelity: a worked example of the methods used to identify, critique and revise the essential elements of a contextualised intervention in health policy agencies

BACKGROUND

In this paper, we identify and respond to the fidelity assessment challenges posed by novel contextualised interventions (i.e. interventions that are informed by composite social and psychological theories and which incorporate standardised and flexible components in order to maximise effectiveness in complex settings). We (a) describe the difficulties of, and propose a method for, identifying the essential elements of a contextualised intervention; (b) provide a worked example of an approach for critiquing the validity of putative essential elements; and (c) demonstrate how essential elements can be refined during a trial without compromising the fidelity assessment. We used an exploratory test-and-refine process, drawing on empirical evidence from the process evaluation of Supporting Policy In health with Research: an Intervention Trial (SPIRIT). Mixed methods data was triangulated to identify, critique and revise how the intervention's essential elements should be articulated and scored.

RESULTS

Over 50 provisional elements were refined to a final list of 20 and the scoring rationalised. Six (often overlapping) challenges to the validity of the essential elements were identified. They were (1) redundant-the element was not essential; (2) poorly articulated-unclear, too specific or not specific enough; (3) infeasible-it was not possible to implement the essential element as intended; (4) ineffective-the element did not effectively deliver the change principles; (5) paradoxical-counteracting vital goals or change principles; or (6) absent or suboptimal-additional or more effective ways of operationalising the theory were identified. We also identified potentially valuable 'prohibited' elements that could be used to help reduce threats to validity.

CONCLUSIONS

We devised a method for critiquing the construct validity of our intervention's essential elements and modifying how they were articulated and measured, while simultaneously using them as fidelity indicators. This process could be used or adapted for other contextualised interventions, taking evaluators closer to making theoretically and contextually sensitive decisions upon which to base fidelity assessments.

Collective action for implementation: a realist evaluation of organisational collaboration in healthcare

BACKGROUND

Increasingly, it is being suggested that translational gaps might be eradicated or narrowed by bringing research users and producers closer together, a theory that is largely untested. This paper reports a national study to fill a gap in the evidence about the conditions, processes and outcomes related to collaboration and implementation.

METHODS

A longitudinal realist evaluation using multiple qualitative methods case studies was conducted with three Collaborations for Leadership in Applied Health Research in Care (England). Data were collected over four rounds of theory development, refinement and testing. Over 200 participants were involved in semi-structured interviews, non-participant observations of events and meetings, and stakeholder engagement. A combined inductive and deductive data analysis process was focused on proposition refinement and testing iteratively over data collection rounds.

RESULTS

The quality of existing relationships between higher education and local health service, and views about whether implementation was a collaborative act, created a path dependency. Where implementation was perceived to be removed from service and there was a lack of organisational connections, this resulted in a focus on knowledge production and transfer, rather than co-production. The collaborations' architectures were counterproductive because they did not facilitate connectivity and had emphasised professional and epistemic boundaries. More distributed leadership was associated with greater potential for engagement. The creation of boundary spanning roles was the most visible investment in implementation, and credible individuals in these roles resulted in cross-boundary work, in facilitation and in direct impacts. The academic-practice divide played out strongly as a context for motivation to engage, in that 'what's in it for me' resulted in variable levels of engagement along a co-operation-collaboration continuum. Learning within and across collaborations was patchy depending on attention to evaluation.

CONCLUSIONS

These collaborations did not emerge from a vacuum, and they needed time to learn and develop. Their life cycle started with their position on collaboration, knowledge and implementation. More impactful attempts at collective action in implementation might be determined by the deliberate alignment of a number of features, including foundational relationships, vision, values, structures and processes and views about the nature of the collaboration and implementation.