Learning from the emergence of NIHR Collaborations for Leadership in Applied Health Research and Care (CLAHRCs): a systematic review of evaluations

If health organisations and staff engage in research, does healthcare improve? Strengthening the evidence base through systematic reviews

BACKGROUND

There is an often-held assumption that the engagement of clinicians and healthcare organizations in research improves healthcare performance at various levels. Previous reviews found up to 28 studies suggesting a positive association between the engagement of individuals and healthcare organizations in research and improvements in healthcare performance. The current study sought to provide an update.

METHODS

We updated our existing published systematic review by again addressing the question: Does research engagement (by clinicians and organizations) improve healthcare performance? The search covered the period 1 January 2012 to March 2024, in two phases. First, the formal updated search ran from 1 January 2012 to 31 May 2020, in any healthcare setting or country and focussed on English language publications. In this phase two searches identified 66 901 records. Later, a further check of key journals and citations to identified papers ran from May 2020 to March 2024. In total, 168 papers progressed to full-text appraisal; 62 were identified for inclusion in the update. Then we combined papers from our original and updated reviews.

RESULTS

In the combined review, the literature is dominated by papers from the United States (50/95) and mostly drawn from the Global North. Papers cover various clinical fields, with more on cancer than any other field; 86 of the 95 papers report positive results, of which 70 are purely positive and 16 positive/mixed, meaning there are some negative elements (i.e. aspects where there is a lack of healthcare improvement) in their findings.

CONCLUSIONS

The updated review collates a substantial pool of studies, especially when combined with our original review, which are largely positive in terms of the impact of research engagement on processes of care and patient outcomes. Of the potential engagement mechanisms, the review highlights the important role played by research networks. The review also identifies various papers which consider how far there is a "dose effect" from differing amounts of research engagement. Additional lessons come from analyses of equity issues and negative papers. This review provides further evidence of contributions played by systems level research investments such as research networks on processes of care and patient outcomes.

"We all see things through a different lens based on our life experiences": co-production of a web-based implementation toolkit with stakeholders across the health and social care system

Background

Implementing new innovations across the health and social care system is complex, involving many factors that in recent years have been compounded by Covid-19. While a plethora of implementation tools and frameworks are available, there are limitations in terms of their design and accessibility. Co-production is a valuable mechanism for developing tools that have utility and accessibility for those tasked with using them in health and social care organisations and there is growing acknowledgement of increasing the role of co-production in implementation science. This paper provides novel insight into co-production practices and relevance to implementation science by reporting findings from a study to co-produce a web-based implementation toolkit (WIT) that is accessible, usable and designed to support adaptive implementation across health and social care systems. Key themes relating to the process of co-production are outlined and the value of using co-production in implementation processes are discussed.

Methods

A web-based survey (n = 36) was conducted with a range of stakeholders across health and social care. Findings identified a need for WIT. Survey respondents were invited to express interest in becoming part of a co-production group and to take part in three online interactive workshops to co-produce WIT. Workshops took place with the group (n = 12) and focused on key developmental stages of WIT.

Results

Online co-production workshops were integral to the development and refinement of WIT. Benefits of using this process identified three interrelated themes: (i) Co-designing key features of the toolkit, (ii) Co-producing a toolkit with utility for users across health and social care settings, (iii) Co-producing a toolkit to support the implementation journey. Our approach of undertaking co-production as a dialogic process enabled generation of these themes. To illuminate discussion of these themes we draw upon iterative co-development of the "active ingredients" of key components (e.g., interactive Implementation Wheel) and functions (e.g., interactive "pop-up" definitions of keyword) and features (e.g., case studies) of WIT.

Conclusion

Using a co-production approach with a range of end-users across health and social care systems, highlights the benefits of understanding implementation processes for users in these settings. User-centred design and processes for ensuring accessibility readily support the translation of implementation into rapidly changing health and social care systems to benefit outcomes for patients, their families, carers, service users and practitioners.

Exploring Elinor Ostrom's principles for collaborative group working within a user-led project: lessons from a collaboration between researchers and a user-led organisation

BACKGROUND

Some research has been undertaken into the mechanisms that shape successful participatory approaches in the context of efforts to improve health and social care. However, greater attention needs to be directed to how partnerships between researchers and user-led organisations (ULOs) might best be formed, practiced, managed, and assessed. We explored whether political economist Elinor Ostrom's Nobel prize winning analysis of common pool resource management-specifically eight principles to enhance collaborative group working as derived from her empirical research-could be usefully applied within a user-led project aiming to co-design new services to support more inclusive involvement of Disabled people in decision-making processes in policy and practice.

METHODS

Participant observation and participatory methods over a 16-month period comprising observational notes of online user-led meetings (26 h), online study team meetings (20 h), online Joint Interpretive Forum meetings (8 h), and semi-structured one-to-one interviews with project participants (44 h) at two time points (months 6 and 10).

RESULTS

Initially it proved difficult to establish working practices informed by Ostrom's principles for collaborative group working within the user-led project. Several attempts were made to put a structure in place that met the needs of both the research study and the aims of the user-led project, but this was not straightforward. An important shift saw a move away from directly applying the principles to the working practices of the group and instead applying them to specific tasks the group were undertaking. This was a helpful realisation which enabled the principles to become-for most but not all participants-a useful facilitation device in the latter stages of the project. Eventually we applied the principles in a way that was useful and enabled collaboration between researchers and a ULO (albeit in unexpected ways).

CONCLUSIONS

Our joint reflections emphasise the importance of being reflexive and responsive when seeking to apply theories of collaboration (the principles) within user-led work. At an early stage, it is important to agree shared definitions and understanding of what 'user-led' means in practice. It is crucial to actively adapt and translate the principles in ways that make them more accessible and applicable within groups where prior knowledge of their origins is both unlikely and unnecessary.

Organisational benefits of undertaking research in healthcare: an approach to uncover impact

There is increasing focus to review the societal impact of research through assessment and research excellence frameworks. These often link to financial and reputational incentives within the academic community. However, timeframes to demonstrate impact using these approaches are often long and are not designed to show benefit to service collaborators who require evidence of improvement and change to their services more immediately. Impacts that are measured this way may also miss out on unintended and positive impacts that occur as by-products of research, or through the 'ripple effect' that research may have on practice. Importantly, demonstrating how research makes a difference can improve the research culture in services, and motivations in service partners to become, and stay involved in research. This article describes, and provides access to, a tool called VICTOR (making Visible the ImpaCT Of Research) that was developed by a community of practice involving 12 NHS organisations through blending evidence from the literature, practice and service users. We describe the types of impact that have been collected by VICTOR and explore how collecting impact in this way might help research-practice partnerships and inform research methodologies and may be useful to show impacts alongside, and shortly after the research process.

Characterizing Canadian funded partnered health research projects between 2011 and 2019: a retrospective analysis

BACKGROUND AND AIMS

Involving research users in collaborative research approaches may increase the relevance and utility of research findings. Our primary objectives were to (i) identify and describe characteristics of Canadian federally and provincially funded health research projects that included research users and were funded between 2011 and 2019; (ii) explore changes over time; and (iii) compare characteristics between funder required and optional partnerships.

METHODS

Retrospective analysis. Inclusion criteria were projects that included research users. We analyzed publicly available project variables, and coded field and type of research using established classification systems. We summarized data with descriptive statistics and compared variables across three funding year blocks and partnership requirement status.

RESULTS

We identified 1153 partnered health research projects, representing 137 fields of research and 37 types of research categories. Most projects included a required partnership (80%) and fell into health and social care services research (66%). Project length and funding amount increased from average of 24.8 months and $266 248 CAD in 2011-2013 to 31.6 months and $438 766 CAD in 2017-2019. There were significantly fewer required partnerships in 2017-2019.

CONCLUSIONS

Between 2011 and 2019 Canadian federally and provincially funded partnered health research reflected primarily care services research across many fields. The observed breadth suggests that partnered health research approaches are applicable in many fields of research. Additional work to support partnered research across all types of health research (especially biomedical research) is warranted. The administration of larger grants that are funded for longer time periods may address previously identified concerns among research teams engaging in partnered research but may mean that fewer teams receive funding and risk delaying responding to time-sensitive data needs for users. Our process and findings can be used as a starting point for international comparison.

Why Is It So Hard to Evaluate Knowledge Exchange? Comment on "Sustaining Knowledge Translation Practices: A Critical Interpretive Synthesis"

Despite a growth in knowledge translation (KT) or exchange activities, and a smaller growth in their evaluations, it remains challenging to identify evidence of efficacy. This could be due to well-documented political and logistical difficulties involved in evaluating knowledge exchange interventions. By bringing in theory from science and technology studies (STS), Borst et al1 offer a new way of thinking about this problem. Most KT evaluations draw on health research traditions; centralising comparability, efficacy, and so on. Borst et al propose focusing on the work it takes to move knowledge over boundaries between these communities, seeing relationships as interactions, not just conduits for evidence. They show how 'context' can be understood as a mutual creation, not a static environment; and that institutions shape behaviours, rather than merely being sites or platforms for evidence mobilisation. Seeing KT as a creative, active practice opens new ways to design and evaluate KT mechanisms.

Leading co-production in five UK collaborative research partnerships (2008-2018): responses to four tensions from senior leaders using auto-ethnography

BACKGROUND

Despite growing enthusiasm for co-production in healthcare services and research, research on co-production practices is lacking. Multiple frameworks, guidelines and principles are available but little empirical research is conducted on 'how to do' co-production of research to improve healthcare services. This paper brings together insights from UK-based collaborative research partnerships on leading co-production. Its aim is to inform practical guidance for new partnerships planning to facilitate the co-production of applied health research in the future.

METHODS

Using an auto-ethnographic approach, experiential evidence was elicited through collective sense making from recorded conversations between the research team and senior leaders of five UK-based collaborative research partnerships. This approach applies a cultural analysis and interpretation of the leaders' behaviours, thoughts and experiences of co-production taking place in 2008-2018 and involving academics, health practitioners, policy makers and representatives of third sector organisations.

RESULTS

The findings highlight a variety of practices across CLAHRCs, whereby the intersection between the senior leaders' vision and local organisational context in which co-production occurs largely determines the nature of co-production process and outcomes. We identified four tensions in doing co-production: (1) idealistic, tokenistic vs realistic narratives, (2) power differences and (lack of) reciprocity, (3) excluding vs including language and communication, (4) individual motivation vs structural issues.

CONCLUSIONS

The tensions were productive in helping collaborative research partnerships to tailor co-production practices to their local needs and opportunities. Resulting variation in co-production practices across partnerships can therefore be seen as highly advantageous creative adaptation, which makes us question the utility of seeking a unified 'gold standard' of co-production. Strategic leadership is an important starting point for finding context-tailored solutions; however, development of more distributed forms of leadership over time is needed to facilitate co-production practices between partners. Facilitating structures for co-production can enable power sharing and boost capacity and capability building, resulting in more inclusive language and communication and, ultimately, more credible practices of co-production in research. We provide recommendations for creating more realistic narratives around co-production and facilitating power sharing between partners.

Describing the evidence-base for research engagement by health care providers and health care organisations: a scoping review

BACKGROUND

Having a research-engaged health and medical workforce is associated with improvements in clinical outcomes for patients. As such, there has been significant government investment internationally to support health care organisations and services to increase staff engagement with research.

OBJECTIVES

This scoping review sought to provide an overview of the literature describing strategies employed to increase research engagement by health care providers and organisations, and to undertake a qualitative analysis to generate a list of research engagement strategies.

METHODS

A scoping review using systematic search strategies was undertaken to locate peer-review publications and grey literature related to research engagement by health care providers and organisations. Research engagement was defined as a 'deliberate set of intellectual and practical activities undertaken by health care staff and organisations to conduct research'. A database search of electronic records was performed with no limit on publication date. Publications were included regardless of study type (excluding systematic reviews) and categorised as either databased (presenting data or new analysis of existing data) and non-databased (no new data or analyses). Databased publications were further classified according to study type, study design and setting. A qualitative synthesis using a Framework Approach was undertaken with all studies that described a strategy to improve research engagement.

RESULTS

A total of 152 publications were included in this study with 54% categorised as non-databased. Of the databased articles, the majority (72%) were descriptive studies describing prevalence of correlates of research engagement, 17 (25%) described intervention studies where only two were controlled studies. The following research engagement strategies were identified: i) dual skilled team/staff, ii) resources or physical infrastructure, iii) incentives, iv) leadership support of research, v) education/training, vi) networks, vii) forming partnerships or collaborations and viii) overall leadership structure of entity.

CONCLUSIONS

The literature on research engagement is primarily opinion-based and descriptive in nature. To provide the evidence needed to inform strategies, this needs to progress beyond descriptive to more rigorous well-designed intervention research.

Too much theory and not enough practice? The challenge of implementation science application in healthcare practice

BACKGROUND

Implementation science (IS) should contribute to maintaining high standards of care across healthcare systems and enhancing care practices. However, despite the evident need for greater and more rapid uptake and integration of evidence in practice, IS design and methodology fall short of the needs of effective translation.

AIM

In this paper we examine what it is about IS that makes it so appealing for effective uptake of interventions in routine practice, and yet so difficult to achieve. We propose a number of ways that implementation scientists could build mutual relationships with healthcare practitioners and other stakeholders including public members to ensure greater shared care practices, and highlight the value of IS training, collaborative educational events, and co-designed research.

DISCUSSION

More consideration should be given to IS applications in healthcare contexts. Implementation scientists can make a valuable contribution by mobilizing theory and improving practice. However, goals for an evidence-based system may be more appropriately achieved through greater outreach and collaboration, with methods that are flexible to support rapid implementation in complex adaptive systems. Collective learning and mutual trust can be cultivated by embedding researchers into healthcare services while offering greater opportunities for practitioners to learn about, and engage in, implementation research.

CONCLUSION

To bridge the worlds of healthcare practice and IS, researchers could be more consistent in the relationships they build with professionals and the public, communicating through a shared language and co-joining practical approaches to effective implementation. This will build capacity for improved collaboration and foster respectful, interdisciplinary relationships.

Advocacy, activism, and lobbying: How variations in interpretation affects ability for academia to engage with public policy

Research and teaching are considered core-responsibilities for academic researchers. "Practice" activities however are viewed as ancillary, despite university emphasis on their importance. As funders, governments, and academia address the role of research in social impact, the deliberations on researcher activism, advocacy and lobbying have seen a resurgence. This study explores the perceptions of 52 faculty and 24 government decisionmakers on the roles, responsibilities, and restrictions of an academic to proactively engage in efforts that can be interpreted under these three terms. Data was coded through inductive thematic analysis using Atlas.Ti and a framework approach. We found that discordant perceptions about how much activism, advocacy and lobbying faculty should be engaging in, results from how each term is defined, interpreted, supported and reported by the individuals, the School of Public Health (SPH), and government agencies. Influential faculty factors included: seniority, previous experiences, position within the institution, and being embedded in a research center with an advocacy focus. Faculty views on support for advocacy were often divergent. We surmise therefore, that for effective and mutually beneficial collaboration to occur, academic institutions need to align rhetoric with reality with respect to encouraging modes and support for government engagement. Similarly, government agencies need to provide more flexible modes of engagement. This will contribute to alleviating confusion as well as tension leading to more effective engagement and consequently opportunity for evidence-informed decision making in public health globally.

Identity reconciliation and knowledge mobilization in a mandated community of practice

Purpose

Communities of Practice (CoPs) are increasingly being created to facilitate knowledge mobilization in organizations. This paper aims to elucidate an underexplored aspect of participation in mandated CoPs – identity reconciliation. Specifically, the authors explore how actors reconcile their existing identities with becoming members of new knowledge mobilization CoPs.

Design/methodology/approach

The authors conducted a longitudinal qualitative case study over a 12-month period to explore identity reconciliation practices during the formation of the “ED CoP” – mandated by policymakers to mobilize knowledge between process improvement advisors and clinicians from various hospitals. Observation and interviews allowed us to uncover “front stage” and “backstage” practices of identity reconciliation.

Findings

The findings reveal two key unexpected modes of identity reconciliation – “distancing” and “peripheral lurking”. These modes resulted in different trajectories of participation of two of the key participant groups – “veteran” improvement advisors and “veteran” clinicians.

Practical implications

Different modes of identity reconciliation of different participants impact the formation of CoPs and how knowledge mobilization occurs within them. This paper offers a sensitizing lens for practitioners creating CoPs which enhances awareness of hidden identity practices, and recommendations to enable practitioners to effectively facilitate CoP formation.

Originality/value

This study suggests that identity reconciliation is an integral aspect of CoP formation, and essential for knowledge mobilization within CoPs. Whereas studies on CoPs in the knowledge management literature have mostly assumed that collaboration produces beneficial knowledge mobilization outcomes, the findings build a more nuanced picture of the processes involved in producing these outcomes.

Trialists perspectives on sustaining, spreading, and scaling-up of quality improvement interventions

BACKGROUND

Quality improvement (QI) evaluations rarely consider how a successful intervention can be sustained long term, nor how to spread or scale to other locations. A survey of authors of randomized trials of diabetes QI interventions included in an ongoing systematic review found that 78% of trials reported improved quality of care, but 40% of these trials were not sustained. This study explores why and how the effective interventions were sustained, spread, or scaled.

METHODS

A qualitative approach was used, focusing on case examples. Diabetes QI program trial authors were purposefully sampled and recruited for telephone interviews. Authors were eligible if they had completed the author survey, agreed to follow-up, and had a completed a diabetes QI trial they deemed "effective." Snowball sampling was used if the participant identified someone who could provide a different perspective on the same trial. Interviews were transcribed verbatim. Inductive thematic analysis was conducted to identify barriers and facilitators to sustainability, spread, and/or scale of the QI program, using case examples to show trajectories across projects and people.

RESULTS

Eleven of 44 eligible trialists participated in an interview. Four reported that the intervention was "sustained" and nine were "spread," however, interviews highlighted that these terms were interpreted differently over time and between participants. Participant stories highlighted the varied trajectories of how projects evolved and how some research careers adapted to increase impact. Three interacting themes, termed the "3C's," helped explain the variation in sustainability, spread, and scale: (i) understanding the concepts of implementation, sustainability, sustainment, spread, and scale; (ii) having the appropriate competencies; and (iii) the need for individual, organizational, and system capacity.

CONCLUSIONS

Challenges in defining sustainability, spread and scale make it difficult to fully understand impact. However, it is clear that from the beginning of intervention design, trialists need to understand the concepts and have the competency and capacity to plan for feasible and sustainable interventions that have potential to be sustained, spread and/or scaled if found to be effective.

A realist evaluation of a collaborative model to support research co-production in long-term care settings in England: the ExCHANGE protocol

BACKGROUND

Collaborative working between academic institutions and those who provide health and social care has been identified as integral in order to produce acceptable, relevant, and timely research, and for outputs to be useful and practical to implement. The ExCHANGE Collaboration aims to bring together researchers and people working, living in and visiting care homes to build capacity, share and mobilise knowledge, and identify key areas for future research. This paper describes an embedded, formative, realist and theory-driven evaluation which aims to gather information about how successful the ExCHANGE Collaboration is perceived to be in achieving its aims. An existing realist programme theory from the literature - Closer Collaboration - will be supplemented by two substantive theories: Co-production and Knowledge Brokering. This will result in an initial programme theory which will be tested by this formative evaluation to refine understanding of how the ExCHANGE Collaboration works.

METHODS

The evaluation will employ mixed qualitative methods, including: analysis of documents such as feedback forms, Knowledge Broker journal/diary, event attendance records, risk and issues logs and other relevant paperwork gathered as part of project delivery; observations of events/activities; and interviews with care home providers and staff, care home residents, residents' family members, and researchers who are involved in the project (both project design/delivery, and also attendance or involvement in project activities/events). Framework Analysis will be used to interpret the data collected; analysis will be strategic, by focusing on particular key areas of importance in the developing theory of how the ExCHANGE Collaboration might achieve change.

RESULTS

The results of this study are expected to be published in 2022.

DISCUSSION

This evaluation will investigate how successful the ExCHANGE Collaboration is perceived to be in achieving its aims, in what way, in which contexts, and how this may differ for those involved. It will do this by testing an initial programme theory about how the collaboration works, for whom, under which circumstances, and in what way. Findings will be shared through written publication, an end of project learning event for those involved/interested in the project, and a lay summary to be made publically available.

How Do Decision Makers and Service Providers Experience Participatory Approaches to Developing and Implementing Physical Activity Interventions with Older Adults? A Thematic Analysis

BACKGROUND

Physical activity has numerous health and well-being benefits for older adults, but many older adults are inactive. Interventions designed to increase physical activity in older adults have typically only produced small effects and have not achieved long-term changes. There is increasing interest in participatory approaches to promoting physical activity, such as co-production, co-design and place-based approaches, but they have typically involved researchers as participants. This study aimed to understand the experiences of decision-makers and service developers with the introduction of such participatory approaches when developing new physical activity programmes outside of a research setting.

METHODS

Semi-structured, qualitative interviews were conducted with 20 individuals who were involved in commissioning or developing the Greater Manchester Active Ageing Programme. This programme involved funding eight local authorities within Greater Manchester, England, to produce physical activity projects for older adults, involving participatory approaches. An inductive thematic analysis was conducted, structured using the Framework approach.

RESULTS

Interviewees identified important benefits of the participatory approaches. The increased involvement of older adults led to older adults contributing valuable ideas, becoming involved in and taking ownership of projects. Interviewees identified the need to move away from traditional emphases on increasing physical activity to improve health, towards focussing on social and fun elements. The accessibility of the session location and information was considered important. Challenges were also identified. In particular, it was recognised that the new approaches require significant time investment to do well, as trusting relationships with older adults and partner organisations need to be developed. Ensuring the sustainability of projects in the context of short-term funding cycles was a concern.

CONCLUSIONS

Incorporating participatory approaches was perceived to yield important benefits. Interviewees highlighted that to ensure success, sufficient time needs to be provided to develop good working relationships with older adults and partner organisations. They also emphasised that sufficient funding to ensure adequate staffing and the sustainability of projects is required to allow benefits to be gained. Importantly, the implementation of these approaches appears feasible across a range of local authorities.

Managing Complex Healthcare Change: A Qualitative Exploration of Current Practice in New South Wales, Australia

PURPOSE

As the cost of healthcare continues to rise, healthcare organizations internationally are seeking long-term solutions to eradicate inefficiency, achieve value-based healthcare, and minimize hospital inpatient services. This requires transformational change in healthcare organizations, and associated change management and leadership capability at multiple levels. Despite the critical need for effective change leadership and management in healthcare, limited evidence exists that this currently occurs in addition to the capability and capacity for managing change in health systems.

METHODS

Semi-structured interviews were undertaken with 16 healthcare managers and leaders at a range of levels in nine healthcare organizations across the public health system of one Australian state (New South Wales), including metropolitan, regional and rural geographical areas. Thematic content analysis was undertaken with the emergent data.

RESULTS

Four key themes emerged from the data: 1) lack of adoption of frameworks and methods for change management for any scope or scale of change, 2) inadequate resources for delivering, managing and leading change, 3) insufficient leadership, capacity and capability in managing change, and 4) the need for support and culture that supports change at all levels of the system.

CONCLUSION

Ensuring dedicated resources for change and sufficient capacity and capability amongst health professionals and managers at every level in a health system are required for effective management of change. An enabling culture for change, supported by adequate education and training in change leadership and management are critical in order for the benefits of health service and system changes to be realised.

Applying systems thinking to knowledge mobilisation in public health

CONTEXT

Knowledge mobilisation (KM) is a vital strategy in efforts to improve public health policy and practice. Linear models describing knowledge transfer and translation have moved towards multi-directional and complexity-attuned approaches where knowledge is produced and becomes meaningful through social processes. There are calls for systems approaches to KM but little guidance on how this can be operationalised. This paper describes the contribution that systems thinking can make to KM and provides guidance about how to put it into action.

METHODS

We apply a model of systems thinking (which focuses on leveraging change in complex systems) to eight KM practices empirically identified by others. We describe how these models interact and draw out some key learnings for applying systems thinking practically to KM in public health policy and practice. Examples of empirical studies, tools and targeted strategies are provided.

FINDINGS

Systems thinking can enhance and fundamentally transform KM. It upholds a pluralistic view of knowledge as informed by multiple parts of the system and reconstituted through use. Mobilisation is conceived as a situated, non-prescriptive and potentially destabilising practice, no longer conceptualised as a discrete piece of work within wider efforts to strengthen public health but as integral to and in continual dialogue with those efforts. A systems approach to KM relies on contextual understanding, collaborative practices, addressing power imbalances and adaptive learning that responds to changing interactions between mobilisation activities and context.

CONCLUSION

Systems thinking offers valuable perspectives, tools and strategies to better understand complex problems in their settings and for strengthening KM practice. We make four suggestions for further developing empirical evidence and debate about how systems thinking can enhance our capacity to mobilise knowledge for solving complex problems - (1) be specific about what is meant by 'systems thinking', (2) describe counterfactual KM scenarios so the added value of systems thinking is clearer, (3) widen conceptualisations of impact when evaluating KM, and (4) use methods that can track how and where knowledge is mobilised in complex systems.

Bridging the research-practice gap in healthcare: a rapid review of research translation centres in England and Australia

Background

Large-scale partnerships between universities and health services are widely seen as vehicles for bridging the evidence–practice gap and for accelerating the adoption of new evidence in healthcare. Recently, different versions of these partnerships – often called academic health science centres – have been established across the globe. Although they differ in structure and processes, all aim to improve the integration of research and education with health services. Collectively, these entities are often referred to as Research Translation Centres (RTCs) and both England and Australia have developed relatively new and funded examples of these collaborative centres.

Methods

This paper presents findings from a rapid review of RTCs in Australia and England that aimed to identify their structures, leadership, workforce development and strategies for involving communities and service users. The review included published academic and grey literature with a customised search of the Google search engine and RTC websites.

Results

RTCs are complex system-level interventions that will need to disrupt the current paradigms and silos inherent in healthcare, education and research in order to meet their aims. This will require vision, leadership, collaborations and shared learnings, alongside structures, processes and strategies to deliver impact in the face of complexity. The impact of RTCs in overcoming the deeply entrenched silos across organisations, disciplines and sectors needs to be captured at the systems, organisation and individual levels. This includes workforce capacity and public and patient involvement that are vital to understanding the evolution of RTCs. In addition, new models of leadership are needed to support the brokering and mobilisation of knowledge in complex organisations.

Conclusions

The development and funding of RTCs represents one of the most significant shifts in the health research landscape and it is imperative that we continue to explore how we can progress the integration of research and healthcare and ensure research meets stakeholder needs and is translated via the collaborations supported by these organisations. Because RTCs are a recent addition to the healthcare landscape in Australia, it is instructive to review the processes and infrastructure needed to support their implementation and applied health research in England.

Flipping the paradigm: a qualitative exploration of research translation centres in the United Kingdom and Australia

Background

Over the past decade, Research Translation Centres (RTCs) have been established in many countries. These centres (sometimes referred to as Academic Health Science Centres) are designed to bring universities and healthcare providers together in order to accelerate the generation and translation of new evidence that is responsive to health service and community priorities. This has the potential to effectively ‘flip’ the traditional research and education paradigms because it requires active participation and continuous engagement with stakeholders (especially service users, the community and frontline clinicians). Although investment and expectations of RTCs are high, the literature confirms a need to better understand the processes that RTCs use to mobilise knowledge, build workforce capacity, and co-produce research with patients and the public to ensure population impact and drive healthcare improvement.

Methods

Semi-structured interviews were conducted with selected leaders and members from select RTCs in England and Australia. Convenience sampling was utilised to identify RTCs, based on their geography, accessibility and availability. Purposive sampling and a snowballing approach were employed to recruit individual participants for interviews, which were conducted face to face or via videoconferencing. Interviews were recorded, transcribed verbatim and analysed using a reflexive and inductive approach. This involved two researchers comparing codes and interrogating themes that were analysed inductively against the study aims and through meetings with the research team.

Results

A total of 41 participants, 22 from England and 19 from Australia were interviewed. Five major themes emerged, including (1) dissonant metrics, (2) different models of leadership, (3) public and patient involvement and research co-production, (4) workforce development and (5) barriers to collaboration.

Conclusions

Participants identified the need for performance measures that capture community impact. Better aligned success metrics, enhanced leadership, strategies to partner with patients and the public, enhanced workforce development and strategies to enhance collaboration were all identified as crucial for RTCs to succeed.

Fiddling while Rome burns? Conducting research with healthcare staff when the NHS is in crisis

PURPOSE

Health research in the UK is being impeded by a stretched NHS system. The purpose of this paper is to use the Great Fire of Rome as an allegory to understand the difficulties encountered by health researchers when attempting to conduct research within a healthcare system that is currently in crisis.

DESIGN/METHODOLOGY/APPROACH

The paper draws on both the authors' own and other research teams' experiences from the published literature in order to demonstrate that this difficulty is a widespread problem for the health research community in the UK.

FINDINGS

Recruitment and engagement issues across different research studies and clinical environments are often ascribed as being related to individual contexts or settings. Rather, the authors propose that these problems are actually writ large across nearly the entire NHS. The authors offer ideas for what can be done to alleviate the worst of this situation - a change in culture and ways of working alongside employing more pragmatic, rapid methods to engage exceptionally busy healthcare staff.

ORIGINALITY/VALUE

The paper offers a provocative viewpoint that instead of seeking to individualise recruitment and engagement issues in relation to the local context, the research community should publicly acknowledge the universality of this problem in order to bring about meaningful change.

Evaluating stakeholder involvement in building a decision support tool for NHS health checks: co-producing the WorkHORSE study

Background

Stakeholder engagement is being increasingly recognised as an important way to achieving impact in public health. The WorkHORSE (Working Health Outcomes Research Simulation Environment) project was designed to continuously engage with stakeholders to inform the development of an open access modelling tool to enable commissioners to quantify the potential cost-effectiveness and equity of the NHS Health Check Programme.

An objective of the project was to evaluate the involvement of stakeholders in co-producing the WorkHORSE computer modelling tool and examine how they perceived their involvement in the model building process and ultimately contributed to the strengthening and relevance of the modelling tool.

Methods

We identified stakeholders using our extensive networks and snowballing techniques. Iterative development of the decision support modelling tool was informed through engaging with stakeholders during four workshops. We used detailed scripts facilitating open discussion and opportunities for stakeholders to provide additional feedback subsequently. At the end of each workshop, stakeholders and the research team completed questionnaires to explore their views and experiences throughout the process.

Results

30 stakeholders participated, of which 15 attended two or more workshops. They spanned local (NHS commissioners, GPs, local authorities and academics), third sector and national organisations including Public Health England.

Stakeholders felt valued, and commended the involvement of practitioners in the iterative process. Major reasons for attending included: being able to influence development, and having insight and understanding of what the tool could include, and how it would work in practice. Researchers saw the process as an opportunity for developing a common language and trust in the end product, and ensuring the support tool was transparent. The workshops acted as a reality check ensuring model scenarios and outputs were relevant and fit for purpose.

Conclusions

Computational modellers rarely consult with end users when developing tools to inform decision-making. The added value of co-production (continuing collaboration and iteration with stakeholders) enabled modellers to produce a “real-world” operational tool. Likewise, stakeholders had increased confidence in the decision support tool’s development and applicability in practice.

Mainstreaming public involvement in a complex research collaboration: A theory-informed evaluation

INTRODUCTION

There is an extensive literature on public involvement (PI) in research, but this has focused primarily on experiences for researchers and public contributors and factors enabling or restricting successful involvement in specific projects. There has been less consideration of a 'whole system' approach to embedding PI across an organization from governance structures through to research projects.

OBJECTIVE

To investigate how a combination of two theoretical frameworks, one focused on mainstreaming and the other conceptualizing quality, can illuminate the embedding of positive and influential PI throughout a research organization.

METHODS

The study used data from the evaluation of a large UK research collaboration. Primary data were collected from 131 respondents (including Public Advisers, university, NHS and local government staff) via individual and group interviews/workshops. Secondary sources included monitoring data and internal documents.

FINDINGS

CLAHRC-NWC made real progress in mainstreaming PI. An organizational vision and infrastructure to embed PI at all levels were created, and the number and range of opportunities increased; PI roles became more clearly defined and increasingly public contributors felt able to influence decisions. However, the aspiration to mainstream PI throughout the collaboration was not fully achieved: a lack of staff 'buy-in' meant that in some areas, it was not experienced as positively or was absent.

CONCLUSION

The two theoretical frameworks brought a novel perspective, facilitating the investigation of the quality of PI in structures and processes across the whole organization. We propose that combining these frameworks can assist the evaluation of PI research.

Value and learning from carer involvement in a cluster randomised controlled trial and process evaluation - Organising Support for Carers of Stroke Survivors (OSCARSS)

BACKGROUND

Patient, Carer and Public Involvement (PCPI) should be embedded in health care research. Delivering PCPI can be challenging, but even when PCPI is carried out it is rarely reported resulting in lost opportunities for learning. This paper aims to describe PCPI in the OSCARSS study, a pragmatic-cluster randomised controlled trial with an embedded economic and process evaluation.

METHODS

A carer research user group (RUG) co-developed OSCARSS to evaluate how to best deliver support to caregivers of stroke survivors. The PCPI activity involved regular meetings and preparatory work, from the initial conceptualisation of the study through to dissemination. Written reports, structured group discussions and individual interviews were carried out with the RUG and researchers to capture the added value and learning. This paper was co-authored by two of the RUG members with contributions from the wider RUG and researchers.

RESULTS

The core six members of the caregiver RUG attended the majority of the meetings alongside three researchers, one of whom was the co-chief investigator. PCPI was instrumental in changing many aspects of the research protocol, design and delivery and contributed to dissemination and sharing of good practice. There were challenges due to the emotional toll when PCPI members shared their stories and the extensive time commitment. Positive experiences of learning and fulfilment were reported by the individual researchers and PCPI members. Wider organisational administrative and financial support facilitated the PCPI. The researchers' existing positive regard for PCPI and the clear focus of the group were key to the successful co-design of this research.

CONCLUSIONS

The value and learning from the PCPI collaborative work with the researchers was of benefit to the study and the individuals involved. Specific PCPI influences were a challenge to pinpoint as successful co-design meant the researchers' and carers' contributions were intertwined and decision-making shared.

Multi-level analysis of the learning health system: Integrating contributions from research on organizations and implementation

Introduction

Organizations and systems that deliver health care may better adapt to rapid change in their environments by acting as learning organizations and learning health systems (LHSs). Despite widespread recognition that multilevel forces shape capacity for learning within care delivery organizations, there is no agreed-on, comprehensive, multilevel framework to inform LHS research and practice.

Methods

We develop such a framework, which can enhance both research on LHSs and practical steps toward their development. We draw on existing frameworks and research within organization and implementation science and synthesize contributions from three influential frameworks: the Consolidated Framework for Implementation Research, the social-ecological framework, and the organizational change framework. These frameworks come, respectively, from the fields of implementation science, public health, and organization science.

Results

Our proposed integrative framework includes both intraorganizational levels (individual, team, mid-management, organization) and the operating and general environments in which delivery organizations operate. We stress the importance of examining interactions among influential factors both within and across system levels and focus on the effects of leadership, incentives, and culture. Additionally, we indicate that organizational learning depends substantially on internal and cross-level alignment of these factors. We illustrate the contribution of our multilevel perspective by applying it to the analysis of three diverse implementation initiatives that aimed at specific care improvements and enduring system learning.

Conclusions

The framework and perspective developed here can help investigators and practitioners broadly scan and then investigate forces influencing improvement and learning and may point to otherwise unnoticed interactions among influential factors. The framework can also be used as a planning tool by managers and practitioners.

Learning from a Feasibility Trial of a Simple Intervention: Is Research a Barrier to Service Delivery, or is Service Delivery a Barrier to Research?

(1) Background: Applied health services research (AHSR) relies upon coordination across multiple organizational boundaries. Our aim was to understand how competing organizational and professional goals enhance or impede the conduct of high quality AHSR. (2) Methods: A qualitative study was conducted in two local health care systems in the UK, linked to a feasibility trial of a clinic-based intervention in secondary care. Data collection involved 24 semi-structured interviews with research managers, clinical research staff, health professionals, and patients. (3) Results: This study required a dynamic network of interactions between heterogeneous health and social care stakeholders, each characterized by differing ways of organizing activities which constitute their core functions; cultures of collaboration and interaction and understanding of what research involves and how it contributes to patient care. These interrelated factors compounded the occupational and organizational boundaries that hindered communication and coordination. (4) Conclusions: Despite the strategic development of multiple organizations to foster inter-professional collaboration, the competing goals of research and clinical practice can impede the conduct of high quality AHSR. To remedy this requires the alignment and streamlining of organizational goals, so that all agencies involved in AHSR develop a shared understanding and mutual respect for the progress of evidence-based medicine and the complex and often nuanced environments in which it is created and practiced.

What Can Policy-Makers Get Out of Systems Thinking? Policy Partners' Experiences of a Systems-Focused Research Collaboration in Preventive Health

BACKGROUND

There is increasing interest in using systems thinking to tackle 'wicked' policy problems in preventive health, but this can be challenging for policy-makers because the literature is amorphous and often highly theoretical. Little is known about how best to support health policy-makers to gain skills in understanding and applying systems thinking for policy action.

METHODS

In-depth interviews were conducted with 18 policy-makers who are participating in an Australian research collaboration that uses a systems approach. Our aim was to explore factors that support policy-makers to use systems approaches, and to identify any impacts of systems thinking on policy thinking or action, including the pathways through which these impacts occurred.

RESULTS

All 18 policy-makers agreed that systems thinking has merit but some questioned its practical policy utility. A small minority were confused about what systems thinking is or which approaches were being used in the collaboration. The majority were engaged with systems thinking and this group identified concrete impacts on their work. They reported using systems-focused research, ideas, tools and resources in policy work that were contributing to the development of practical methodologies for policy design, scaling up, implementation and evaluation; and to new prevention narratives. Importantly, systems thinking was helping some policy-makers to reconceptualise health problems and contexts, goals, potential policy solutions and methods. In short, they were changing how they think about preventive health.

CONCLUSION

These results show that researchers and policy-makers can put systems thinking into action as part of a research collaboration, and that this can result in discernible impacts on policy processes. In this case, action-oriented collaboration and capacity development over a 5-year period facilitated mutual learning and practical application. This indicates that policy-makers can get substantial applied value from systems thinking when they are involved in extended co-production processes that target policy impact and are supported by responsive capacity strategies.

How do organisations implement research impact assessment (RIA) principles and good practice? A narrative review and exploratory study of four international research funding and administrative organisations

Background

Public research funding agencies and research organisations are increasingly accountable for the wider impacts of the research they support. While research impact assessment (RIA) frameworks and tools exist, little is known and shared of how these organisations implement RIA activities in practice.

Methods

We conducted a review of academic literature to search for research organisations’ published experiences of RIAs. We followed this with semi-structured interviews from a convenience sample (n = 7) of representatives of four research organisations deploying strategies to support and assess research impact.

Results

We found only five studies reporting empirical evidence on how research organisations put RIA principles into practice. From our interviews, we observed a disconnect between published RIA frameworks and tools, and the realities of organisational practices, which tended not to be reported.

We observed varying maturity and readiness with respect to organisations’ structural set ups for conducting RIAs, particularly relating to leadership, skills for evaluation and automating RIA data collection. Key processes for RIA included efforts to engage researcher communities to articulate and plan for impact, using a diversity of methods, frameworks and indicators, and supporting a learning approach. We observed outcomes of RIAs as having supported a dialogue to orient research to impact, underpinned shared learning from analyses of research, and provided evidence of the value of research in different domains and to different audiences.

Conclusions

Putting RIA principles and frameworks into practice is still in early stages for research organisations. We recommend that organisations (1) get set up by considering upfront the resources, time and leadership required to embed impact strategies throughout the organisation and wider research ‘ecosystem’, and develop methodical approaches to assessing impact; (2) work together by engaging researcher communities and wider stakeholders as a core part of impact pathway planning and subsequent assessment; and (3) recognise the benefits that RIA can bring about as a means to improve mutual understanding of the research process between different actors with an interest in research.

Experience of Health Leadership in Partnering With University-Based Researchers in Canada - A Call to "Re-imagine" Research

BACKGROUND

Emerging evidence that meaningful relationships with knowledge users are a key predictor of research use has led to promotion of partnership approaches to health research. However, little is known about health system experiences of collaborations with university-based researchers, particularly with research partnerships in the area of health system design and health service organization. The purpose of the study was to explore the experience and perspectives of senior health managers in health service organizations, with health organization-university research partnerships.

METHODS

In-depth, semi-structured interviews (n = 25) were conducted with senior health personnel across Canada to explore their perspectives on health system research; experiences with health organization-university research partnerships; challenges to partnership research; and suggested actions for improving engagement with knowledge users and promoting research utilization. Participants, recruited from organizations with regional responsibilities, were responsible for system-wide planning and support functions.

RESULTS

Research is often experienced as unhelpful or irrelevant to decision-making by many within the system. Research, quality improvement (QI) and evaluation are often viewed as separate activities and coordinated by different responsibility areas. Perspectives of senior managers on barriers to partnership differed from those identified in the literature: organizational stress and restructuring, and limitations in readiness of researchers to work in the fast-paced healthcare environment, were identified as major barriers. Although the need for strong executive leadership was emphasized, "multi-system action" is needed for effective partnerships.

CONCLUSION

Common approaches to research and knowledge translation are often not appropriate for addressing issues of health service design and health services organization. Nor is the research community providing expertise to many important activities that the healthcare system is taking to improve health services. A radical rethinking of how we prepare health service researchers; position research within the health system; and fund research activities and infrastructure is needed if the potential benefits of research are to be achieved. Lack of response to health system needs may contribute to research and 'evidence-informed' practice being further marginalized from healthcare operations. Interventions to address barriers must respond to the perspectives and experience of health leadership.

Initiation is recognized as a fundamental early phase of integrated knowledge translation (IKT): qualitative interviews with researchers and research users in IKT partnerships

Background

Health care researcher-research user partnerships, referred to as integrated knowledge translation (IKT), have been adopted on an international basis, and are an effective means of co-generating and implementing evidence into policy and practice. Prior research suggests that an initiation period is essential for establishing functional partnerships. To characterize IKT initiation and describe determinants of IKT initiation success, this study explored IKT initiation processes, enablers, and barriers among researchers and research users involved in IKT partnerships.

Methods

A descriptive qualitative approach was used compliant with COREQ standards. Canadian researchers and research users in research collaborations were identified on publicly-available directories and web sites, and referred by those interviewed. They were asked to describe how partnerships were initiated, influencing factors, the length of initiation, and interventions needed to support initiation. Sampling was concurrent with data collection and analysis to achieve thematic saturation. Data were analyzed using constant comparative technique by all members of the research team.

Results

In total, 22 individuals from 6 provinces were interviewed (9 researchers, 11 research users, 2 connectors). They confirmed that IKT initiation is a distinct early phase of partnerships. The period ranged from 6 months to 2 years for 75.0% of participants in pre-existing partnerships, to 6 years for newly-formed partnerships. High-level themes were: Newly identifying and securing partners is an intensive process; Processes and activities take place over a protracted period through multiple interactions; Identifying and engaging committed partners is reliant on funding; and Partnership building is challenged by maintaining continuity and enthusiasm. Participants underscored the need for an IKT partner matching forum, IKT initiation toolkit, and funding for non-research activities required during IKT initiation to establish functional researcher-research user partnerships. Themes were largely similar regardless of participant years of experience with IKT or being involved in a new versus pre-existing partnership.

Conclusions

IKT initiation is a recognized and important early phase of IKT that establishes functional partnerships, and once established, ongoing partnership for subsequent projects is likely. Further research is needed to develop and evaluate approaches recommended by participants for stimulating IKT initiation.

Turning conceptual systems maps into dynamic simulation models: An Australian case study for diabetes in pregnancy

BACKGROUND

System science approaches are increasingly used to explore complex public health problems. Quantitative methods, such as participatory dynamic simulation modelling, can mobilise knowledge to inform health policy decisions. However, the analytic and practical steps required to turn collaboratively developed, qualitative system maps into rigorous and policy-relevant quantified dynamic simulation models are not well described. This paper reports on the processes, interactions and decisions that occurred at the interface between modellers and end-user participants in an applied health sector case study focusing on diabetes in pregnancy.

METHODS

An analysis was conducted using qualitative data from a participatory dynamic simulation modelling case study in an Australian health policy setting. Recordings of participatory model development workshops and subsequent meetings were analysed and triangulated with field notes and other written records of discussions and decisions. Case study vignettes were collated to illustrate the deliberations and decisions made throughout the model development process.

RESULTS

The key analytic objectives and decision-making processes included: defining the model scope; analysing and refining the model structure to maximise local relevance and utility; reviewing and incorporating evidence to inform model parameters and assumptions; focusing the model on priority policy questions; communicating results and applying the models to policy processes. These stages did not occur sequentially; the model development was cyclical and iterative with decisions being re-visited and refined throughout the process. Storytelling was an effective strategy to both communicate and resolve concerns about the model logic and structure, and to communicate the outputs of the model to a broader audience.

CONCLUSION

The in-depth analysis reported here examined the application of participatory modelling methods to move beyond qualitative conceptual mapping to the development of a rigorously quantified and policy relevant, complex dynamic simulation model. The analytic objectives and decision-making themes identified provide guidance for interpreting, understanding and reporting future participatory modelling projects and methods.

The dark side of coproduction: do the costs outweigh the benefits for health research?

Background

Coproduction, a collaborative model of research that includes stakeholders in the research process, has been widely advocated as a means of facilitating research use and impact. We summarise the arguments in favour of coproduction, the different approaches to establishing coproductive work and their costs, and offer some advice as to when and how to consider coproduction.

Debate

Despite the multiplicity of reasons and incentives to coproduce, there is little consensus about what coproduction is, why we do it, what effects we are trying to achieve, or the best coproduction techniques to achieve policy, practice or population health change. Furthermore, coproduction is not free risk or cost. Tensions can arise throughout coproduced research processes between the different interests involved. We identify five types of costs associated with coproduced research affecting the research itself, the research process, professional risks for researchers and stakeholders, personal risks for researchers and stakeholders, and risks to the wider cause of scholarship. Yet, these costs are rarely referred to in the literature, which generally calls for greater inclusion of stakeholders in research processes, focusing exclusively on potential positives. There are few tools to help researchers avoid or alleviate risks to themselves and their stakeholders.

Conclusions

First, we recommend identifying specific motivations for coproduction and clarifying exactly which outcomes are required for whom for any particular piece of research. Second, we suggest selecting strategies specifically designed to enable these outcomes to be achieved, and properly evaluated. Finally, in the absence of strong evidence about the impact and process of coproduction, we advise a cautious approach to coproduction. This would involve conscious and reflective research practice, evaluation of how coproduced research practices change outcomes, and exploration of the costs and benefits of coproduction. We propose some preliminary advice to help decide when coproduction is likely to be more or less useful.