What are the relative merits of the sources used to identify potential research priorities for the NHS HTA programme?

Is Co-production Just a Pipe Dream for Applied Health Research Commissioning? An Exploratory Literature Review

Background and Rationale: Internationally, the idea of "co-production' has become more popular in health research because of the promise of partnership between researchers and patients to create research that focuses on patients' needs. Patient and public involvement (PPI) at an early stage in deciding what research should be funded, can improve the quality and impact of research. However, professional power over the process places limits on the public practising their participatory rights for involvement in commissioning research that affects them and can leave members of the public feeling unheard or excluded, particularly within the context of early phase applied health research. Aim: This article explores whether and how the public can be involved in the co-production of research commissioning early on in the process, with a focus on the power relations that pervade basic and early phase translational applied health research. Methods: An exploratory literature review of international peer-reviewed and gray health research literature using structured searches of electronic databases and key search terms. Results: There is very little literature that critically evaluates how PPI is embedded into the early phases of the commissioning process. The field of basic or early translational applied research appear to be particularly challenging. Four themes which emerged from the review are: reasons for PPI in research commissioning; benefits of PPI at strategic levels of research commissioning; contributions of patients and members of the public; improving PPI in research commissioning. Conclusion: Although the public are being consulted at some stages of the research commissioning process, it is evident that the process of determining research priorities and agendas is far from being widely co-produced. Moving PPI from a consultative paternalistic model to a collaborative partnership model should be a priority for commissioners. Significant changes to communication, practices, systems, structures, or cultures that exclude patients and the public from contributing in meaningful ways, are needed to fulfill the potential of co-produced models of research commissioning.

Identifying research priorities for public health research to address health inequalities: use of Delphi-like survey methods

Background

In the funding of health research and public health research it is vital that research questions posed are important and that funded research meets a research need or a gap in evidence. Many methods are used in the identification of research priorities, however, these can be resource intensive, costly and logistically challenging. Identifying such research priorities can be particularly challenging for complex public health problems as there is a need to consult a number of experts across disciplines and with a range of expertise. This study investigated the use of Delphi-like survey methods in identifying important research priorities relating to health inequalities and framing tractable research questions for topic areas identified.

Methods

The study was conducted in two phases, both using Delphi-like survey methods. Firstly, public health professionals with an interest in health inequalities were asked to identify research priorities. Secondly academic researchers were asked to frame tractable research questions relating to the priorities identified. These research priorities identified using Delphi-like survey methods were subsequently compared to those identified using different methods.

Results

A total of 52 public health professionals and 21 academics across the United Kingdom agreed to take part. The response rates were high, from public health professionals across three survey rounds (69%, 50% and 40%) and from academics across one round (52%), indicating that participants were receptive to the method and motivated to respond. The themes identified as encompassing the most important research priorities were mental health, healthy environment and health behaviours. Within these themes, the topic areas that emerged most strongly included community interventions for prevention of mental health problems and the food and alcohol environment. Some responses received from academic researchers were (as requested) in the form of tractable research questions, whereas others contributed further potential topic areas instead.

Conclusions

Delphi-like survey methods are practical and productive as a means of obtaining opinions from a wide number of relevant experts identifying potential priority topic areas for research; however, this method is less appropriate for framing tractable research questions.

Electronic supplementary material

The online version of this article (doi:10.1186/s12961-017-0252-2) contains supplementary material, which is available to authorized users.

Using a co-production prioritization exercise involving South Asian children, young people and their families to identify health priorities requiring further research and public awareness

OBJECTIVES

To facilitate South Asian (SA) families and health-care professionals (HCPs) participation in a prioritization exercise to co-produce child health research and public awareness agendas.

DESIGN

A three-stage process was adopted involving the following: (i) systematic literature review, (ii) HCP scoping survey and (iii) focus groups of SA adolescents and families. A Punjabi- and Urdu-speaking community facilitator moderated focus groups. A British Sign Language interpreter assisted in the hard of hearing group. Concordant and discordant themes between HCPs and SAs were identified.

SETTING

National survey of HCPs. Leicestershire for SA families.

PARTICIPANTS

A total of 27 HCPs and 35 SAs. SAs varied by descent, age (16-74), UK stay length (3-57 years) religion and disability.

RESULTS

Ranked by submission frequency in the survey, HCPs prioritized (i) public awareness on obesity, mental health, health-care access, vitamin D and routine health checks and (ii) research on nutrition, diabetes, health education and parenting methods.

DISCUSSION

South Asians prioritized research into the effectiveness of alternative medicines, a theme not identified by HCPs. Both HCPs and SAs prioritized increased research or public awareness on mental health illness, blood and organ donation, obesity and diet. Whilst HCPs identified diabetes, vitamin D and rickets together with parenting methods were important priorities requiring increased public awareness, and these views were not shared by SAs.

CONCLUSIONS

Minority groups are not always included in priority setting exercises due to concerns about language and perceived difficulty with accessing communities. Through this co-production exercise, we showed that it is possible and essential.

Clinical trial metadata: defining and extracting metadata on the design, conduct, results and costs of 125 randomised clinical trials funded by the National Institute for Health Research Health Technology Assessment programme

BACKGROUND

By 2011, the Health Technology Assessment (HTA) programme had published the results of over 100 trials with another 220 in progress. The aim of the project was to develop and pilot 'metadata' on clinical trials funded by the HTA programme.

OBJECTIVES

The aim of the project was to develop and pilot questions describing clinical trials funded by the HTA programme in terms of it meeting the needs of the NHS with scientifically robust studies. The objectives were to develop relevant classification systems and definitions for use in answering relevant questions and to assess their utility.

DATA SOURCES

Published monographs and internal HTA documents.

REVIEW METHODS

A database was developed, 'populated' using retrospective data and used to answer questions under six prespecified themes. Questions were screened for feasibility in terms of data availability and/or ease of extraction. Answers were assessed by the authors in terms of completeness, success of the classification system used and resources required. Each question was scored to be retained, amended or dropped.

RESULTS

One hundred and twenty-five randomised trials were included in the database from 109 monographs. Neither the International Standard Randomised Controlled Trial Number nor the term 'randomised trial' in the title proved a reliable way of identifying randomised trials. Only limited data were available on how the trials aimed to meet the needs of the NHS. Most trials were shown to follow their protocols but updates were often necessary as hardly any trials recruited as planned. Details were often lacking on planned statistical analyses, but we did not have access to the relevant statistical plans. Almost all the trials reported on cost-effectiveness, often in terms of both the primary outcome and quality-adjusted life-years. The cost of trials was shown to depend on the number of centres and the duration of the trial. Of the 78 questions explored, 61 were well answered, 33 fully with 28 requiring amendment were the analysis updated. The other 17 could not be answered with readily available data.

LIMITATIONS

The study was limited by being confined to 125 randomised trials by one funder.

CONCLUSIONS

Metadata on randomised controlled trials can be expanded to include aspects of design, performance, results and costs. The HTA programme should continue and extend the work reported here.

FUNDING

The National Institute for Health Research HTA programme.

Health research, development and innovation in England from 1988 to 2013: from research production to knowledge mobilization

This paper presents a critical analysis of the development of government policy and practice on health research, development and innovation over the last 25 years - starting from the publication of a seminal report from the House of Lords Science and Technology Committee in 1988. We first set out to map and analyse the trends in ideas and thinking that have shaped research policy and practice over this period, and to put the development of health research, development and innovation in the wider context of health system reforms and changes. We argue that though this has been a transformative period for health research, rather less progress has been made in the domains of development and innovation, and we offer an analysis of why this might be the case. Drawing on advances in our understanding about how research informs practice, we then make the case for a more integrative model of research, development and innovation. This leads us to conclude that recent experiments with Collaborations for Leadership in Applied Health Research and Care and Academic Health Science Centres and Networks offer some important lessons for future policy directions.

Health technology assessment: Research trends and future priorities in Europe

Objective:

To provide an overview of health services research related to health technology assessment (HTA) and to identify research priorities from a European perspective.

Methods:

Several methods were used: systematic review of articles indexed with the MeSH term ‘technology assessment’ in PubMed from February 1999-2009; online survey among experts; and conference workshop discussions.

Results:

Research activity in HTA varies considerably across Europe. The research was categorised into six areas: (1) the breadth of analysis in HTA (such as economic, organizational and social aspects); (2) HTA products developed to meet the needs of policy-makers (such as horizon scanning, mini-HTA, and core HTA); (3) handling life-cycle perspectives in relation to technologies; (4) topics that challenge existing methods and for which HTA should be developed to address the themes more comprehensively (such as public health interventions and organizational interventions); (5) development of HTA capacity and programmes; and (6) links between policy and HTA. An online survey showed that the three areas that were given priority were the relationship between HTA and policy-making (71%), the impact of HTA (62%) and incorporating patient aspects in HTA (50%). Policy-makers highlighted HTA and innovation processes as their main research priority (42%). Areas that the systematic review identified as future priorities include issues within the six existing research areas such as disinvestment, developing evidence for new technologies, assessing the wider effects of technology use, and determining how HTA affects decision-making. In addition, relative effectiveness and individualized treatments are areas of growing interest.

Conclusions:

The research priorities identified are important for obtaining high quality and cost-effective health care in Europe. Managing the introduction, use and phasing out of technologies challenges health services throughout Europe, and these processes need to be improved to successfully manage future more general challenges. An ageing population and a diminishing workforce both require strong efforts to ensure effective and well-organized use of human resources and technologies. Furthermore, Europe needs to focus on innovation. This is closely linked to use of technologies and calls for future research.

Emerging health technologies: informing and supporting health policy early

OBJECTIVES

All western healthcare systems are confronted with a rising number of new health technologies. To support decision-making processes with sound information about new health technologies, some countries have established "Horizon Scanning Systems (HSS)". This paper gives an overview of processes and practices of HSS.

METHOD

The paper is based on a literature review (Medline and Embase) and on unpublished information gathered from HSS-agencies.

RESULTS

The 13 current HSS have been collaborating in the EuroScan network since 1999. EuroScan has agreed on a common terminology, classification and understanding of their activities. All their activities consist of 5 sequenced main components: identification and filtering, prioritization, early assessment, dissemination and monitoring the assessed technologies. Although there is a common understanding with regard to function and processes there are some differences in the scope of the national/regional HSS.

CONCLUSION

EuroScan has played an important role in the harmonization process so that effective collaboration, reduction of duplication and the further development of procedures have become possible. Because of the common understanding there is a certain stability and integration across the functions of HSS. Nonetheless there are some obvious "blank spots" susceptible to subjectivity, such as an implicit prioritization process.

Priority setting for health technology assessments: a systematic review of current practical approaches

OBJECTIVES

This study sought to identify and compare various practical and current approaches of health technology assessment (HTA) priority setting.

METHODS

A literature search was performed across PubMed, MEDLINE, EMBASE, BIOSIS, and Cochrane. Given an earlier review conducted by European agencies (EUR-ASSESS project), the search was limited to literature indexed from 1996 onward. We also searched Web sites of HTA agencies as well as HTAi and ISTAHC conference abstracts. Agency representatives were contacted for information about their priority-setting processes. Reports on practical approaches selected through these sources were identified independently by two reviewers.

RESULTS

A total of twelve current priority-setting frameworks from eleven agencies were identified. Ten countries were represented: Canada, Denmark, England, Hungary, Israel, Scotland, Spain, Sweden, The Netherlands, and United States. Fifty-nine unique HTA priority-setting criteria were divided into eleven categories (alternatives; budget impact; clinical impact; controversial nature of proposed technology; disease burden; economic impact; ethical, legal, or psychosocial implications; evidence; interest; timeliness of review; variation in rates of use). Differences across HTA agencies were found regarding procedures for categorizing, scoring, and weighing of policy criteria.

CONCLUSIONS

Variability exists in the methods for priority setting of health technology assessment across HTA agencies. Quantitative rating methods and consideration of cost benefit for priority setting were seldom used. These study results will assist HTA agencies that are re-visiting or developing their prioritization methods.

Improving the use of research evidence in guideline development: 16. Evaluation

BACKGROUND

The World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the last of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this.

OBJECTIVES

We reviewed the literature on evaluating guidelines and recommendations, including their quality, whether they are likely to be up-to-date, and their implementation. We also considered the role of guideline developers in undertaking evaluations that are needed to inform recommendations.

METHODS

We searched PubMed and three databases of methodological studies for existing systematic reviews and relevant methodological research. We did not conduct systematic reviews ourselves. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments.

KEY QUESTIONS AND ANSWERS

Our answers to these questions were informed by a review of instruments for evaluating guidelines, several studies of the need for updating guidelines, discussions of the pros and cons of different research designs for evaluating the implementation of guidelines, and consideration of the use of uncertainties identified in systematic reviews to set research priorities. How should the quality of guidelines or recommendations be appraised? WHO should put into place processes to ensure that both internal and external review of guidelines is undertaken routinely. A checklist, such as the AGREE instrument, should be used. The checklist should be adapted and tested to ensure that it is suitable to the broad range of recommendations that WHO produces, including public health and health policy recommendations, and that it includes questions about equity and other items that are particularly important for WHO guidelines. When should guidelines or recommendations be updated? Processes should be put into place to ensure that guidelines are monitored routinely to determine if they are in need of updating. People who are familiar with the topic, such as Cochrane review groups, should do focused, routine searches for new research that would require revision of the guideline. Periodic review of guidelines by experts not involved in developing the guidelines should also be considered. Consideration should be given to establishing guideline panels that are ongoing, to facilitate routine updating, with members serving fixed periods with a rotating membership. How should the impact of guidelines or recommendations be evaluated? WHO headquarters and regional offices should support member states and those responsible for policy decisions and implementation to evaluate the impact of their decisions and actions by providing advice regarding impact assessment, practical support and coordination of efforts. Before-after evaluations should be used cautiously and when there are important uncertainties regarding the effects of a policy or its implementation, randomised evaluations should be used when possible. What responsibility should WHO take for ensuring that important uncertainties are addressed by future research when the evidence needed to inform recommendations is lacking? Guideline panels should routinely identify important uncertainties and research priorities. This source of potential priorities for research should be used systematically to inform priority-setting processes for global research.

Health technology assessment in England and Wales

In this study, we describe the current state of the Health Technology Assessment (HTA) system in England and Wales. This system rests on a distinction between assessment and appraisal and has three main strands: researcher-led HTA, the research and development program, and the HTA-NICE (National Institute for Clinical Excellence) process. We outline the pressures for HTA and how it has evolved in the British National Health Service. We discuss how HTA priorities are chosen, how HTA information is collected and assessed, how HTA evidence is used, and we make some observations about its impact. In our discussion, we consider some limitations of the HTA system, its possible divergence from evidence-based health care, its centralization, and some of the key challenges for managing HTA-driven policy. But we remain hopeful that HTA can contribute to better and more explicit decision-making within England and Wales.