Describing the impact of health services and policy research

Objectives

In an essentially applied area of research, there are particular pressures on health services and policy researchers to describe the impact of their work. However, specialized research impact assessments often require skills and resources beyond those available to individual researchers, and ad hoc accounts impose a considerable burden to generate. Further, these idiosyncratic accounts may not facilitate comparative analysis to inform research management, practice and assessment. This paper describes an initial attempt to develop a methodical approach to identify and describe research impact.

Methods

A Research Impact Framework was developed, drawing on the literature and interviews with researchers at the London School of Hygiene and Tropical Medicine, and was used to structure impact narratives of selected research projects. These narratives were based on semi-structured interviews with principal investigators and documentary analysis of the projects.

Results

Using the framework as a guide, researchers were relatively easily and methodically able to identify and present impacts of their work. Researchers’ narratives contained verifiable evidence and highlighted a wide range of areas in which health services and policy research has impact. The standardized structure of the narratives also facilitated analysis across projects. Factors thought to positively influence the impact of research included researchers’ continued involvement in research and policy networks, established track records in the field, and the ability to identify and use key influencing events, such as ‘policy windows’.

Conclusions

The framework helped develop research impact narratives and facilitated comparisons across projects, highlighting issues for research management and assessment.

International changes in end-of-life practices over time: a systematic review

Background

End-of-life policies are hotly debated in many countries, with international evidence frequently used to support or oppose legal reforms. Existing reviews are limited by their focus on specific practices or selected jurisdictions. The objective is to review international time trends in end-of-life practices.

Methods

We conducted a systematic review of empirical studies on medical end-of-life practices, including treatment withdrawal, the use of drugs for symptom management, and the intentional use of lethal drugs. A search strategy was conducted in MEDLINE, EMBASE, Web of Science, Sociological Abstracts, PAIS International, Worldwide Political Science Abstracts, International Bibliography of the Social Sciences and CINAHL. We included studies that described physicians’ actual practices and estimated annual frequency at the jurisdictional level. End-of-life practice frequencies were analyzed for variations over time, using logit regression.

Results

Among 8183 references, 39 jurisdiction-wide surveys conducted between 1990 and 2010 were identified. Of those, 22 surveys used sufficiently similar research methods to allow further statistical analysis. Significant differences were found across surveys in the frequency of treatment withdrawal, use of opiates or sedatives and the intentional use of lethal drugs (X² > 1000, p < 0.001 for all). Regression analyses showed increased use of opiates and sedatives over time (p < 0.001), which could reflect more intense symptom management at the end of life, or increase in these drugs to intentionally cause patients’ death.

Conclusion

The use of opiates and sedatives appears to have significantly increased over time between 1990 and 2010. Better distinction between practices with different legal status is required to properly interpret the policy significance of these changes. Research on the effects of public policies should take a comprehensive look at trends in end-of-life practice patterns and their associations with policy changes.

"It's a complex mesh"- how large-scale health system reorganisation affected the delivery of the immunisation programme in England: a qualitative study

BACKGROUND

The English health system experienced a large-scale reorganisation in April 2013. A national tri-partite delivery framework involving the Department of Health, NHS England and Public Health England was agreed and a new local operational model applied. Evidence about how health system re-organisations affect constituent public health programmes is sparse and focused on low and middle income countries. We conducted an in-depth analysis of how the English immunisation programme adapted to the April 2013 health system reorganisation, and what facilitated or hindered the delivery of immunisation services in this context.

METHODS

A qualitative case study methodology involving interviews and observations at national and local level was applied. Three sites were selected to represent different localities, varying levels of immunisation coverage and a range of changes in governance. Study participants included 19 national decision-makers and 56 local implementers. Two rounds of interviews and observations (immunisation board/committee meetings) occurred between December 2014 and June 2015, and September and December 2015. Interviews were audio recorded and transcribed verbatim and written accounts of observed events compiled. Data was imported into NVIVO 10 and analysed thematically.

RESULTS

The new immunisation programme in the new health system was described as fragmented, and significant effort was expended to regroup. National tripartite arrangements required joint working and accountability; a shift from the simpler hierarchical pre-reform structure, typical of many public health programmes. New local inter-organisational arrangements resulted in ambiguity about organisational responsibilities and hindered data-sharing. Whilst making immunisation managers responsible for larger areas supported equitable resource distribution and strengthened service commissioning, it also reduced their ability to apply clinical expertise, support and evaluate immunisation providers' performance. Partnership working helped staff adapt, but the complexity of the health system hindered the development of consistent approaches for training and service evaluation.

CONCLUSION

The April 2013 health system reorganisation in England resulted in significant fragmentation in the way the immunisation programme was delivered. Some of this was a temporary by-product of organisational change, other more persistent challenges were intrinsic to the complex architecture of the new health system. Partnership working helped immunisation leaders and implementers reconnect and now the challenge is to assess how inter-agency collaboration can be strengthened.

The effectiveness of clinical networks in improving quality of care and patient outcomes: a systematic review of quantitative and qualitative studies

BACKGROUND

Reorganisation of healthcare services into networks of clinical experts is increasing as a strategy to promote the uptake of evidence based practice and to improve patient care. This is reflected in significant financial investment in clinical networks. However, there is still some question as to whether clinical networks are effective vehicles for quality improvement. The aim of this systematic review was to ascertain the effectiveness of clinical networks and identify how successful networks improve quality of care and patient outcomes.

METHODS

A systematic search was undertaken in accordance with the PRISMA approach in Medline, Embase, CINAHL and PubMed for relevant papers between 1 January 1996 and 30 September 2014. Established protocols were used separately to examine and assess the evidence from quantitative and qualitative primary studies and then integrate findings.

RESULTS

A total of 22 eligible studies (9 quantitative; 13 qualitative) were included. Of the quantitative studies, seven focused on improving quality of care and two focused on improving patient outcomes. Quantitative studies were limited by a lack of rigorous experimental design. The evidence indicates that clinical networks can be effective vehicles for quality improvement in service delivery and patient outcomes across a range of clinical disciplines. However, there was variability in the networks' ability to make meaningful network- or system-wide change in more complex processes such as those requiring intensive professional education or more comprehensive redesign of care pathways. Findings from qualitative studies indicated networks that had a positive impact on quality of care and patients outcomes were those that had adequate resources, credible leadership and efficient management coupled with effective communication strategies and collaborative trusting relationships.

CONCLUSIONS

There is evidence that clinical networks can improve the delivery of healthcare though there are few high quality quantitative studies of their effectiveness. Our findings can provide policymakers with some insight into how to successfully plan and implement clinical networks by ensuring strong clinical leadership, an inclusive organisational culture, adequate resourcing and localised decision-making authority.

Evaluating the implementation of a national clinical programme for diabetes to standardise and improve services: a realist evaluation protocol

BACKGROUND

Over the last three decades in response to the growing burden of diabetes, countries worldwide have developed national and regional multifaceted programmes to improve the monitoring and management of diabetes and to enhance the coordination of care within and across settings. In Ireland in 2010, against a backdrop of limited dedicated strategic planning and engrained variation in the type and level of diabetes care, a national programme was established to standardise and improve care for people with diabetes in Ireland, known as the National Diabetes Programme (NDP). The NDP comprises a range of organisational and service delivery changes to support evidence-based practices and policies. This realist evaluation protocol sets out the approach that will be used to identify and explain which aspects of the programme are working, for whom and in what circumstances to produce the outcomes intended.

METHODS/DESIGN

This mixed method realist evaluation will develop theories about the relationship between the context, mechanisms and outcomes of the diabetes programme. In stage 1, to identify the official programme theories, documentary analysis and qualitative interviews were conducted with national stakeholders involved in the design, development and management of the programme. In stage 2, as part of a multiple case study design with one case per administrative region in the health system, qualitative interviews are being conducted with frontline staff and service users to explore their responses to, and reasoning about, the programme's resources (mechanisms). Finally, administrative data will be used to examine intermediate implementation outcomes such as service uptake, acceptability, and fidelity to models of care.

DISCUSSION

This evaluation is using the principles of realist evaluation to examine the implementation of a national programme to standardise and improve services for people with diabetes in Ireland. The concurrence of implementation and evaluation has enabled us to produce formative feedback for the NDP while also supporting the refinement and revision of initial theories about how the programme is being implemented in the dynamic and unstable context of the Irish healthcare system.

Explaining the Persistent Dominance of the Greek Medical Profession Across Successive Health Care System Reforms from 1983 to the Present

-The Greek medical profession played an important role at the start of the Greek National Health System (NHS) in 1983 and became intrinsic to its later development. In particular, junior hospital doctors firmly established their position and rights as a result of the new NHS. Using archival sources and interviews with elite participants, this article investigates the specific patterns of power and influence that Greek NHS doctors have exerted from the establishment of the Greek NHS through the latest major attempt at reform in 2001 to the present. Hospital doctors, in particular, have been able consistently to resist any health care system reforms that might affect their dominant position. Their unchallenged position in the system derives from both the particularities of the Greek state and society (in particular, the former's founding institutional arrangements and the latter's clientelistic social relations) and the key role that junior doctors played in the early stages of the Greek NHS. As a result, the system is highly path dependent in that the initial implementation of the NHS during the 1980s ensured that subsequent reforms consistently favored the self-interest of medical doctors. Though challenges to the unaccountable power of the medical profession have emerged in Greece following the financial crisis of 2009, including the beginnings of a popular critique of the medical profession, it is too soon to tell whether these will succeed in bringing about significant change.

The experience of potentially vulnerable people during cold weather: implications for policy and practice

OBJECTIVES

To examine the experience of potentially vulnerable people during cold weather to inform interventions aimed at improving well-being.

STUDY DESIGN

Qualitative study.

METHODS

Telephone interviews with 35 individuals who could be considered to be potentially vulnerable during cold weather. Individuals were interviewed on two occasions during the winter of 2012-13, one or two days after a level 3 cold weather alert, as defined by the Cold Weather Plan for England, had been issued.

RESULTS

Participants were largely unaware of the health risks associated with low temperatures, especially cardiovascular risk. There was a clear distinction between the thermal experience of people in social housing, which was newer, had efficient heating, was well insulated and well-maintained, and owner occupiers living in older, harder to heat homes. Most participants relied on public transport, and many faced arduous journeys to reach basic facilities. Vulnerability to cold was mediated to a significant extent by practical social support from family members.

CONCLUSIONS

Resources should be targeted at people in hard to heat homes and those that are socially isolated. The repertoire of initiatives aimed at reducing cold-related mortality and morbidity could usefully be augmented by efforts to reduce social isolation and build community resilience.

Health information on alcoholic beverage containers: has the alcohol industry's pledge in England to improve labelling been met?

AIMS

In the United Kingdom, alcohol warning labels are the subject of a voluntary agreement between industry and government. In 2011, as part of the Public Health Responsibility Deal in England, the industry pledged to ensure that 80% of products would have clear, legible health warning labelling, although an analysis commissioned by Portman found that only 57.1% met best practice. We assessed what proportion of alcohol products now contain the required health warning information, and its clarity and placement.

DESIGN

Survey of alcohol labelling data.

SETTING

United Kingdom.

PARTICIPANTS

Analysis of the United Kingdom's 100 top-selling alcohol brands (n = 156 individual products).

MEASUREMENTS

We assessed the product labels in relation to the presence of five labelling elements: information on alcohol units, government consumption guidelines, pregnancy warnings, reference to the Drinkaware website and a responsibility statement. We also assessed the size, colour and placement of text, and the size and colouring of the pregnancy warning logo.

FINDINGS

The first three (required) elements were present on 77.6% of products examined. The mean font size of the Chief Medical Officer's (CMO) unit guidelines (usually on the back of the product) was 8.17-point. The mean size of pregnancy logos was 5.95 mm. The pregnancy logo was on average smaller on wine containers.

CONCLUSIONS

The UK Public Health Responsibility Deal alcohol labelling pledge has not been fully met. Labelling information frequently falls short of best practice, with font and logos smaller than would be accepted on other products with health effects.