Knowledge exchange in health-care commissioning: case studies of the use of commercial, not-for-profit and public sector agencies, 2011–14

The effectiveness of knowledge-sharing techniques and approaches in research funded by the National Institute for Health and Care Research (NIHR): a systematic review

BACKGROUND

The National Institute of Health and Care Research (NIHR), funds, enables and delivers world-leading health and social care research to improve people's health and wellbeing. To achieve this aim, effective knowledge sharing (two-way knowledge sharing between researchers and stakeholders to create new knowledge and enable change in policy and practice) is needed. To date, it is not known which knowledge sharing techniques and approaches are used or how effective these are in creating new knowledge that can lead to changes in policy and practice in NIHR funded studies.

METHODS

In this restricted systematic review, electronic databases [MEDLINE, The Health Management Information Consortium (including the Department of Health's Library and Information Services and King's Fund Information and Library Services)] were searched for published NIHR funded studies that described knowledge sharing between researchers and other stakeholders. One researcher performed title and abstract, full paper screening and quality assessment (Critical Appraisal Skills Programme qualitative checklist) with a 20% sample independently screened by a second reviewer. A narrative synthesis was adopted.

RESULTS

In total 9897 records were identified. After screening, 17 studies were included. Five explicit forms of knowledge sharing studies were identified: embedded models, knowledge brokering, stakeholder engagement and involvement of non-researchers in the research or service design process and organisational collaborative partnerships between universities and healthcare organisations. Collectively, the techniques and approaches included five types of stakeholders and worked with them at all stages of the research cycle, except the stage of formation of the research design and preparation of funding application. Seven studies (using four of the approaches) gave examples of new knowledge creation, but only one study (using an embedded model approach) gave an example of a resulting change in practice. The use of a theory, model or framework to explain the knowledge sharing process was identified in six studies.

CONCLUSIONS

Five knowledge sharing techniques and approaches were reported in the included NIHR funded studies, and seven studies identified the creation of new knowledge. However, there was little investigation of the effectiveness of these approaches in influencing change in practice or policy.

The usage of data in NHS primary care commissioning: a realist review

BACKGROUND

Primary care has been described as the 'bedrock' of the National Health Service (NHS) accounting for approximately 90% of patient contacts but is facing significant challenges. Against a backdrop of a rapidly ageing population with increasingly complex health challenges, policy-makers have encouraged primary care commissioners to increase the usage of data when making commissioning decisions. Purported benefits include cost savings and improved population health. However, research on evidence-based commissioning has concluded that commissioners work in complex environments and that closer attention should be paid to the interplay of contextual factors and evidence use. The aim of this review was to understand how and why primary care commissioners use data to inform their decision making, what outcomes this leads to, and understand what factors or contexts promote and inhibit their usage of data.

METHODS

We developed initial programme theory by identifying barriers and facilitators to using data to inform primary care commissioning based on the findings of an exploratory literature search and discussions with programme implementers. We then located a range of diverse studies by searching seven databases as well as grey literature. Using a realist approach, which has an explanatory rather than a judgemental focus, we identified recurrent patterns of outcomes and their associated contexts and mechanisms related to data usage in primary care commissioning to form context-mechanism-outcome (CMO) configurations. We then developed a revised and refined programme theory.

RESULTS

Ninety-two studies met the inclusion criteria, informing the development of 30 CMOs. Primary care commissioners work in complex and demanding environments, and the usage of data are promoted and inhibited by a wide range of contexts including specific commissioning activities, commissioners' perceptions and skillsets, their relationships with external providers of data (analysis), and the characteristics of data themselves. Data are used by commissioners not only as a source of evidence but also as a tool for stimulating commissioning improvements and as a warrant for convincing others about decisions commissioners wish to make. Despite being well-intentioned users of data, commissioners face considerable challenges when trying to use them, and have developed a range of strategies to deal with 'imperfect' data.

CONCLUSIONS

There are still considerable barriers to using data in certain contexts. Understanding and addressing these will be key in light of the government's ongoing commitments to using data to inform policy-making, as well as increasing integrated commissioning.

The commissioning of infant mental health services in the United Kingdom: A study of stakeholder views

CONTEXT

Infant regulatory disturbances are common and stable over time and can compromise infant outcomes across a range of developmental domains. Many such problems have their origins within the parent-infant relationship and specialized parent-infant relationship teams provide support and intervention that is explicitly aimed at addressing such relationship difficulties. However, there are currently only around 27 such teams across the United Kingdom, and just under half of CAMHS do not accept referrals of children under 2 years of age.

AIM

The current research aimed to examine the views of commissioners of children's services regarding the reasons for commissioning (or not) infant mental health services.

METHOD

Fourteen in-depth interviews were conducted with a range of stakeholders involved in commissioning children's services across 14 areas of England, half of which were commissioning specialized infant mental health services. A thematic analysis was undertaken.

RESULTS

A total of five themes emerged from the data as being key factors in the commissioning of infant mental health services: pressure from local practitioners, policy transfer through policy networks, opportunity for long-term cost reduction, potential to embed the service model within existing services and perinatal mental health funding.

CONCLUSION

As with commissioning more widely, the commissioning of infant mental health services is a complex process, with a range of factors influencing whether such services are commissioned or not, and data to suggest that the process is currently driven by informal and contingent factors, as much as by the evidence regarding what works.

Components, impacts and costs of dementia home support: a research programme including the DESCANT RCT

Background

Over half of people with dementia live at home. We know little about what home support could be clinically effective or cost-effective in enabling them to live well.

Objectives

We aimed to (1) review evidence for components of home support, identify their presence in the literature and in services in England, and develop an appropriate economic model; (2) develop and test a practical memory support package in early-stage dementia, test the clinical effectiveness and cost-effectiveness of routine home support in later-stage dementia and design a toolkit based on this evidence; and (3) elicit the preferences of staff, carers and people with dementia for home support inputs and packages, and evaluate the cost-effectiveness of these approaches in early- and later-stage dementia.

Design

We undertook (1) an evidence synthesis, national surveys on the NHS and social care and an economic review; (2) a multicentre pragmatic randomised trial [Dementia Early Stage Cognitive Aids New Trial (DESCANT)] to estimate the clinical effectiveness and cost-effectiveness of providing memory aids and guidance to people with early-stage dementia (the DESCANT intervention), alongside process evaluation and qualitative analysis, an observational study of existing care packages in later-stage dementia along with qualitative analysis, and toolkit development to summarise this evidence; and (3) consultation with experts, staff and carers to explore the balance between informal and paid home support using case vignettes, discrete choice experiments to explore the preferences of people with dementia and carers between home support packages in early- and later-stage dementia, and cost–utility analysis building on trial and observational study.

Setting

The national surveys described Community Mental Health Teams, memory clinics and social care services across England. Recruitment to the trial was through memory services in nine NHS trusts in England and one health board in Wales. Recruitment to the observational study was through social services in 17 local authorities in England. Recruitment for the vignette and preference studies was through memory services, community centres and carers’ organisations.

Participants

People aged > 50 years with dementia within 1 year of first attendance at a memory clinic were eligible for the trial. People aged > 60 years with later-stage dementia within 3 months of a review of care needs were eligible for the observational study. We recruited staff, carers and people with dementia for the vignette and preference studies. All participants had to give written informed consent.

Main outcome measures

The trial and observational study used the Bristol Activities of Daily Living Scale as the primary outcome and also measured quality of life, capability, cognition, general psychological health and carers’ sense of competence.

Methods

Owing to the heterogeneity of interventions, methods and outcome measures, our evidence and economic reviews both used narrative synthesis. The main source of economic studies was the NHS Economic Evaluation Database. We analysed the trial and observational study by linear mixed models. We analysed the trial by ‘treatment allocated’ and used propensity scores to minimise confounding in the observational study.

Results

Our reviews and surveys identified several home support approaches of potential benefit. In early-stage dementia, the DESCANT trial had 468 randomised participants (234 intervention participants and 234 control participants), with 347 participants analysed. We found no significant effect at the primary end point of 6 months of the DESCANT intervention on any of several participant outcome measures. The primary outcome was the Bristol Activities of Daily Living Scale, for which scores range from 0 to 60, with higher scores showing greater dependence. After adjustment for differences at baseline, the mean difference was 0.38, slightly but not significantly favouring the comparator group receiving treatment as usual. The 95% confidence interval ran from –0.89 to 1.65 (p = 0.56). There was no evidence that more intensive care packages in later-stage dementia were more effective than basic care. However, formal home care appeared to help keep people at home. Staff recommended informal care that cost 88% of formal care, but for informal carers this ratio was only 62%. People with dementia preferred social and recreational activities, and carers preferred respite care and regular home care. The DESCANT intervention is probably not cost-effective in early-stage dementia, and intensive care packages are probably not cost-effective in later-stage dementia. From the perspective of the third sector, intermediate intensity packages were cheaper but less effective. Certain elements may be driving these results, notably reduced use of carers’ groups.

Limitations

Our chosen outcome measures may not reflect subtle outcomes valued by people with dementia.

Conclusions

Several approaches preferred by people with dementia and their carers have potential. However, memory aids aiming to affect daily living activities in early-stage dementia or intensive packages compared with basic care in later-stage dementia were not clinically effective or cost-effective.

Future work

Further work needs to identify what people with dementia and their carers prefer and develop more sensitive outcome measures.

Study registration

Current Controlled Trials ISRCTN12591717. The evidence synthesis is registered as PROSPERO CRD42014008890.

Funding

This project was funded by the National Institute for Health Research (NIHR) Programme Grants for Applied Research programme and will be published in full in Programme Grants for Applied Research; Vol. 9, No. 6. See the NIHR Journals Library website for further project information.

Improving the capabilities of NHS organisations to use evidence: a qualitative study of redesign projects in Clinical Commissioning Groups

Background

Innovation driven by authoritative evidence is critical to the survival of England’s NHS. Clinical Commissioning Groups (CCGs) are central in NHS efforts to do more with less. Although decisions should be based on the ‘best available evidence’, this is often problematic, with frequent mismatches between the evidence ‘pushed’ by producers and that used in management work. Our concern, then, is to understand practices and conditions (which we term ‘capabilities’) that enable evidence use in commissioning work. We consider how research gets into CCGs (‘push’), how CCGs use evidence (‘pull’) and how this can be supported (toolkit development). We aim to contribute to evidence-based NHS innovation, and, more generally, to improved health-care service provision.

Method

Supported by the National Institute for Health Research (NIHR), we conducted semistructured ethnographic interviews in eight CCGs. We also conducted observations of redesign meetings in two of the CCGs. We used inductive and deductive coding to identify evidence used and capabilities for use from the qualitative data. We then compared across cases to understand variations in outcomes as a function of capabilities. To help improvements in commissioning, we collated our findings into a toolkit for use by stakeholders. We also conducted a small-scale case study of the production of evidence-based guidance to understand evidence ‘push’.

Results

Fieldwork indicated that different evidences inform CCG decision-making, which we categorise as ‘universal’, ‘local’, ‘expertise-based’ and ‘trans-local’. Fieldwork also indicated that certain practices and conditions (‘capabilities’) enable evidence use, including ‘sourcing and evaluating evidence’, ‘engaging experts’, ‘effective framing’, ‘managing roles and expectations’ and ‘managing expert collaboration’. Importantly, cases in which fewer capabilities were recorded tended to report more problems, relative to cases in which needed capabilities were applied. These latter cases were more likely to effectively use evidence, achieve objectives and maintain stakeholder satisfaction. We also found that various understandings of end-users are inscribed into products by evidence producers, which seems to reflect the evolving landscape of the production of authoritative evidence.

Conclusions

This was exploratory research on evidence use capabilities in commissioning decisions. The findings suggest that commissioning stakeholders need support to identify, understand and apply evidence. Support to develop capabilities for evidence may be one means of ensuring effective, evidence-based innovations in commissioning. Our work with evidence producers also shows variation in their perceptions of end users, which may inform the ‘push’/’pull’ gap between research and practice. There were also some limitations to our project, including a smaller than expected sample size and a time frame that did not allow us to capture full redesign projects in all CCGs.

Future work

With these findings in mind, future work may look more closely at how information comes to be treated as evidence and at the relationships of capabilities to project outcomes. Going forward, knowledge, especially that related to generalisability, may be built by means of a longer time and the study of redesign projects in different settings.

Funding

The NIHR Health Services and Delivery Research programme.

Clinical and cost-effectiveness, safety and acceptability of community intravenous antibiotic service models: CIVAS systematic review

OBJECTIVE

Evaluate evidence of the efficacy, safety, acceptability and cost-effectiveness of outpatient parenteral antimicrobial therapy (OPAT) models.

DESIGN

A systematic review.

DATA SOURCES

MEDLINE, EMBASE, CINAHL, Cochrane Library, National Health Service (NHS) Economic Evaluation Database (EED), Research Papers in Economics (RePEc), Tufts Cost-Effectiveness Analysis (CEA) Registry, Health Business Elite, Health Information Management Consortium (HMIC), Web of Science Proceedings, International Pharmaceutical Abstracts, British Society for Antimicrobial Chemotherapy website. Searches were undertaken from 1993 to 2015.

STUDY SELECTION

All studies, except case reports, considering adult patients or practitioners involved in the delivery of OPAT were included. Studies combining outcomes for adults and children or non-intravenous (IV) and IV antibiotic groups were excluded, as were those focused on process of delivery or clinical effectiveness of 1 antibiotic over another. Titles/abstracts were screened by 1 reviewer (20% verified). 2 authors independently screened studies for inclusion.

RESULTS

128 studies involving >60 000 OPAT episodes were included. 22 studies (17%) did not indicate the OPAT model used; only 29 involved a comparator (23%). There was little difference in duration of OPAT treatment compared with inpatient therapy, and overall OPAT appeared to produce superior cure/improvement rates. However, when models were considered individually, outpatient delivery appeared to be less effective, and self-administration and specialist nurse delivery more effective. Drug side effects, deaths and hospital readmissions were similar to those for inpatient treatment, but there were more line-related complications. Patient satisfaction was high, with advantages seen in being able to resume daily activities and having greater freedom and control. However, most professionals perceived challenges in providing OPAT.

CONCLUSIONS

There were no systematic differences related to the impact of OPAT on treatment duration or adverse events. However, evidence of its clinical benefit compared with traditional inpatient treatment is lacking, primarily due to the dearth of good quality comparative studies. There was high patient satisfaction with OPAT use but the few studies considering practitioner acceptability highlighted organisational and logistic barriers to its delivery.

Effects of a demand-led evidence briefing service on the uptake and use of research evidence by commissioners of health services: a controlled before-and-after study

Background

The Health and Social Care Act 2012 (Great Britain.Health and Social Care Act 2012. London: The Stationery Office; 2012) has mandated research use as a core consideration of health service commissioning arrangements. We evaluated whether or not access to a demand-led evidence briefing service improved the use of research evidence by commissioners, compared with less intensive and less targeted alternatives.

Design

Controlled before-and-after study.

Setting

Clinical Commissioning Groups (CCGs) in the north of England.

Main outcome measures

Change at 12 months from baseline of a CCG’s ability to acquire, assess, adapt and apply research evidence to support decision-making. Secondary outcomes measured individual clinical leads’ and managers’ intentions to use research evidence in decision-making.

Methods

Nine CCGs received one of three interventions: (1) access to an evidence briefing service; (2) contact plus an unsolicited push of non-tailored evidence; or (3) an unsolicited push of non-tailored evidence. Data for the primary outcome measure were collected at baseline and 12 months post intervention, using a survey instrument devised to assess an organisation’s ability to acquire, assess, adapt and apply research evidence to support decision-making. In addition, documentary and observational evidence of the use of the outputs of the service was sought and interviews with CCG participants were undertaken.

Results

Most of the requests were conceptual; they were not directly linked to discrete decisions or actions but were intended to provide knowledge about possible options for future actions. Symbolic use to justify existing decisions and actions were less frequent and included a decision to close a walk-in centre and to lend weight to a major initiative to promote self-care already under way. The opportunity to impact directly on decision-making processes was limited to work to establish disinvestment policies. In terms of impact overall, the evidence briefing service was not associated with increases in CCGs’ capacity to acquire, assess, adapt and apply research evidence to support decision-making, individual intentions to use research findings or perceptions of CCGs’ relationships with researchers. Regardless of the intervention received, at baseline participating CCGs indicated that they felt that they were inconsistent in their research-seeking behaviours and their capacity to acquire research remained so at follow-up. The informal nature of decision-making processes meant that there was little or no traceability of the use of evidence.

Limitations

Low baseline and follow-up response rates (of 68% and 44%, respectively) and missing data limit the reliability of these findings.

Conclusions

Access to a demand-led evidence briefing service did not improve the uptake and use of research evidence by NHS commissioners compared with less intensive and less targeted alternatives. Commissioners appear to be well intentioned but ad hoc users of research.

Future work

Further research is required on the effects of interventions and strategies to build individual and organisational capacity to use research. Resource-intensive approaches to providing evidence may best be employed to support instrumental decision-making. Comparative evaluation of the impact of less intensive but targeted strategies on the uptake and use of research by commissioners is warranted.

Funding

The National Institute for Health Research Health Services and Delivery Research programme.

Community governance in primary health care: towards an international Ideal Type

Against a global background of increased resource management responsibilities for primary health care agencies, general medical practices, in particular, are increasingly being required to demonstrate the legitimacy of their decision making in market oriented environments. In this context a scoping review explores the potential utility for health managers in primary health care of community governance as a policy concept. The review of recent research suggests that applied learning from international health systems with enhanced approaches to public and patient involvement may contribute to meeting this requirement. Such approaches often characterise local health systems in Latin America and North West Europe where innovative models are beginning to respond effectively to the growing demands on general practice. The study design draws on documentary and secondary data analyses to identify common components of community governance from the countries in these regions, supplemented by other relevant international studies and sources where appropriate. Within a comprehensive framework of collaborative governance the components are aggregated in an Ideal Type format to provide a point of reference for possible adaptation and transferable learning across market oriented health systems. Each component is illustrated with international exemplars from recent organisational practices in primary health care. The application of community governance is considered for the particular contexts of GP led Clinical Commissioning Groups in England and Primary Health Networks in Australia. Some components of the Ideal Type possess potentially powerful negative as well as positive motivational effects, with PPI at practice levels sometimes hindering the development of effective local governance. This highlights the importance of careful and competent management of the growing resources attributed to primary health care agencies, which possess an increasingly diverse range of non-governmental status. Future policy and research priorities are outlined. Copyright © 2016 John Wiley & Sons, Ltd.

Evidence based policy making and the 'art' of commissioning - how English healthcare commissioners access and use information and academic research in 'real life' decision-making: an empirical qualitative study

BACKGROUND

Policymakers such as English healthcare commissioners are encouraged to adopt 'evidence-based policy-making', with 'evidence' defined by researchers as academic research. To learn how academic research can influence policy, researchers need to know more about commissioning, commissioners' information seeking behaviour and the role of research in their decisions.

METHODS

In case studies of four commissioning organisations, we interviewed 52 people including clinical and managerial commissioners, observed 14 commissioning meetings and collected documentation e.g. meeting minutes and reports. Using constant comparison, data were coded, summarised and analysed to facilitate cross case comparison.

RESULTS

The 'art of commissioning' entails juggling competing agendas, priorities, power relationships, demands and personal inclinations to build a persuasive, compelling case. Policymakers sought information to identify options, navigate ways through, justify decisions and convince others to approve and/or follow the suggested course. 'Evidence-based policy-making' usually meant pragmatic selection of 'evidence' such as best practice guidance, clinicians' and users' views of services and innovations from elsewhere. Inconclusive or negative research was unhelpful in developing policymaking plans and did not inform disinvestment decisions. Information was exchanged through conversations and stories, which were fast, flexible and suited the rapidly changing world of policymaking. Local data often trumped national or research-based evidence. Local evaluations were more useful than academic research.

DISCUSSION

Commissioners are highly pragmatic and will only use information that helps them create a compelling case for action.Therefore, researchers need to start producing more useful information.

CONCLUSIONS

To influence policymakers' decisions, researchers need to 1) learn more about local policymakers' priorities 2) develop relationships of mutual benefit 3) use verbal instead of writtencommunication 4) work with intermediaries such as public health consultants and 5) co-produce local evaluations.