Research that is both rigorous and useful

How hard can it be to include research evidence and evaluation in local health policy implementation? Results from a mixed methods study

Background

Although an evidence-based approach is the ideal model for planning and delivering healthcare, barriers exist to using research evidence to implement and evaluate service change. This paper aims to inform policy implementation and evaluation by understanding the role of research evidence at the local level through implementation of a national chronic conditions management policy.

Methods

We conducted a national email survey of health service commissioners at the most devolved level of decision-making in Wales (Local Health Boards – LHBs) followed by in-depth interviews with representatives of LHBs, purposively selecting five to reflect geographic and economic characteristics. Survey data were analysed descriptively; we used thematic analysis for interview data.

Results

All LHBs (n = 22) completed questionnaires. All reported they routinely assessed the research literature before implementing interventions, but free-text answers revealed wide variation in approach. Most commonly reported information sources included personal contacts, needs assessments, information or research databases. No consistent approach to evaluation was reported. Frequently reported challenges were: insufficient staff capacity (17/22); limited skills, cost, limited time, competing priorities (16/22); availability and quality of routine data (15/22). Respondents reported they would value central guidance on evaluation.

Five interviews were held with managers from the five LHBs contacted. Service delivery decisions were informed by Welsh Government initiatives and priorities, budgets, perceived good practice, personal knowledge, and local needs, but did not include formal research evidence, they reported. Decision making was a collaborative process including clinical staff, patient representatives, and partner organization managers with varying levels of research experience. Robust evaluation data were required, but they were constrained by a lack of skills, time, and resources. They viewed evaluation as a means of demonstrating that targets had been met.

Conclusions

There is a gap between evidence-based aims of national health policy and how health services are commissioned, implemented, and evaluated at local level. Commissioners and managers are unable to routinely incorporate research evidence. If health services research is to identify most effective ways to implement high quality care, it should be incorporated into commissioning and service delivery. Local commissioners and managers need to build the critical use of research evidence and evaluation into health policy implementation at local level in order to provide consistent and effective healthcare services.