An international survey of the public engagement practices of health technology assessment organizations

Clinical Practice Guideline Recommendations For Pediatric Multisystem Trauma Care: A Systematic Review

OBJECTIVE

To systematically review clinical practice guidelines (CPGs) for pediatric multisystem trauma, appraise their quality, synthesize the strength of recommendations and quality of evidence, and identify knowledge gaps.

BACKGROUND

Traumatic injuries are the leading cause of death and disability in children, who require a specific approach to injury care. Difficulties integrating CPG recommendations may cause observed practice and outcome variation in pediatric trauma care.

METHODS

We conducted a systematic review using Medline, Embase, Cochrane Library, Web of Science, ClinicalTrials, and grey literature, from January 2007 to November 2022. We included CPGs targeting pediatric multisystem trauma with recommendations on any acute care diagnostic or therapeutic interventions. Pairs of reviewers independently screened articles, extracted data, and evaluated the quality of CPGs using "Appraisal of Guidelines, Research, and Evaluation II."

RESULTS

We reviewed 19 CPGs, and 11 were considered high quality. Lack of stakeholder engagement and implementation strategies were weaknesses in guideline development. We extracted 64 recommendations: 6 (9%) on trauma readiness and patient transfer, 24 (38%) on resuscitation, 22 (34%) on diagnostic imaging, 3 (5%) on pain management, 6 (9%) on ongoing inpatient care, and 3 (5%) on patient and family support. Forty-two (66%) recommendations were strong or moderate, but only 5 (8%) were based on high-quality evidence. We did not identify recommendations on trauma survey assessment, spinal motion restriction, inpatient rehabilitation, mental health management, or discharge planning.

CONCLUSIONS

We identified 5 recommendations for pediatric multisystem trauma with high-quality evidence. Organizations could improve CPGs by engaging all relevant stakeholders and considering barriers to implementation. There is a need for robust pediatric trauma research, to support recommendations.

A triangulated approach for understanding scientists' perceptions of public engagement with science

Scientists are expected to engage with the public, especially when society faces challenges like the COVID-19 pandemic or climate change, but what public engagement means to scientists is not clear. We use a triangulated, mixed-methods approach combining survey and focus group data to gain insight into how pre-tenure and tenured scientists personally conceptualize public engagement. Our findings indicate that scientists' understanding of public engagement is similarly complex and diverse as the scholarly literature. While definitions and examples of one-way forms of engagement are the most salient for scientists, regardless of tenure status, scientists also believe public engagement with science includes two-way forms of engagement, such as citizen and community involvement in research. These findings suggest that clear definitions of public engagement are not necessarily required for its application but may be useful to guide scientists in their engagement efforts, so they align with what is expected of them.

Potential Barriers of Patient Involvement in Health Technology Assessment in Central and Eastern European Countries

Patients' perspectives are important to identify preferences, estimate values and appreciate unmet medical needs in the process of research and development and subsequent assessment of new health technologies. Patient and public involvement in health technology assessment (HTA) is essential in understanding and assessing wider implications of coverage and reimbursement decisions for patients, their relatives, caregivers, and the general population. There are two approaches to incorporating the patients' voice in HTA, preferably used in a mix. In the first one, patients, caregivers and/or their representatives directly participate at discussions in different stages of the HTA process, often at the same table with other stakeholders. Secondly, patient involvement activities can be supported by evidence on patient value and experience collected directly from patients, caregivers and/or their representatives often by patient groups Patient involvement practices, however, are limited in Central and Eastern European (CEE) countries without clear methodology or regulatory mechanisms to guide patient involvement in the HTA process. This poses the question of transferability of practices used in other countries, and might call for the development of new CEE-specific guidelines and methods. In this study we aim to map potential barriers of patient involvement in HTA in countries of the CEE region.

Values, principles, strategies, and frameworks underlying patient and public involvement in health technology assessment and guideline development: a scoping review

The importance of patient and public involvement (PPI) is recognized by agencies involved in health technology assessment (HTA) and guideline development. However, a comprehensive overview of the underlying PPI principles, values, strategies, and frameworks is lacking. This scoping review aimed to summarize the available evidence on principles, values, frameworks, and strategies underpinning PPI carried out by agencies involved in HTA and guideline development. A total of twelve records were included, of which four referred to guidelines and eight to HTA. Overall, this review demonstrated a lack of consistency in the definition and application of the concepts of values and principles to PPI in the context of guideline development and HTA. There was significant overlap between values and principles, with some broad themes emerging, such as representation, transparency, relevance, equity, fairness, and reconciling different types of knowledge. Frameworks were typically based on the stages of guideline development or HTA, despite heterogeneity in how stages were labeled and described. Strategies were also mapped to the stages of guideline development and HTA and varied substantially depending on the context and setting. Both strategies and frameworks demonstrated patients and the public can be involved, albeit to a variable extent, throughout the stages of guideline development and HTA. However, frameworks often failed to explicitly link the values and principles with the HTA and guideline development stages through actionable PPI strategies. Further research is warranted to better understand the values, principles, and frameworks underpinning PPI in guideline development and HTA.

A framework for action to improve patient and public involvement in health technology assessment

Background

Patient and public involvement (PPI) in the Brazilian Health Technology Assessment (HTA) process occurs in response to a legislative mandate for “social participation.” This resulted in some limited patient participation activities, and, therefore, a more systematic approach was needed. The study describes the development of a suggested framework for action to improve PPI in HTA.

Methods

This work used formal methodology to develop a PPI framework based on three-phase mixed-methods research with desktop review of Brazilian PPI activities in HTA; workshop, survey, and interviews with Brazilian stakeholders; and a rapid review of international practices to enact effective patient involvement. Patient partners reviewed the draft framework.

Results

According to patient group representatives, their involvement in the Brazilian HTA process is important but could be improved. Different stakeholders perceived barriers, identified values, and made suggestions for improvement, such as expansion of communication, capacity building, and transparency, to support more meaningful patient involvement. The international practices identified opportunities for earlier, more active, and collaborative PPI during all HTA stages, based on values and principles that are relevant for Brazilian patients and the public. These findings were synthesized to design a framework that defines and systematizes actions to support PPI in Brazil, highlighting the importance of evaluating these strategies.

Conclusions

Since the publication of this framework, some of its suggestions are being implemented in the Brazilian HTA process to improve PPI. We encourage other HTA organizations to consider a systematic and planned approach with regular evaluation when pursuing or strengthening involvement practices.

Role of the Patient‘s Viewpoint in Health Technologies Assessment in Bulgaria

Objectives: The aim of this study was to determine the role of the patient’s point of view when assessing health technologies as well as the patients’ willingness to participate in this process in our country.

Methods: A pilot study included 165 patients across Bulgaria recruited from December 2019 to March 2020. Appropriate descriptive and statistical methods were used.

Results: The patient’s point of view was an essential element in HTA. Patients in Bulgaria have the desire and willingness to share their personal experience with institutions, but insufficient knowledge of the process hinders their full participation in HTA. Our results showed that the need for higher awareness in the field and familiarity with HTA methodology are critical factors for the inclusion of patients in the HTA procedures in Bulgaria.

Conclusion: This study was one of the few ones focused on the impact of patient participation in HTA in Bulgaria. The patient’s role in the development of medical technology must be recognized and considered a critical factor in the future of HTA.

Public engagement in health technology assessment in Brazil: the case of the public consultation on National Clinical Guidelines for Care in Normal Birth

BACKGROUND

There is a growing body of literature that recognizes the importance of public engagement in health technology assessment. However, there is still uncertainty regarding how the results should be recorded, analyzed, and used by decision makers.

OBJECTIVE

Synthesize the contributions of the Brazilian public (women, health professionals, managers, educational institutions, and companies) about the implementation of the National Clinical Guidelines for Care in Normal Birth from the public consultation carried out in Brazil.

METHOD

IRaMuTeQ software was used to organize and summarize the corpus based on three types of analysis: descriptive statistics; descending hierarchical classification; and specificities analysis. The public consultation was conducted in 2016 by the National Committee for Health Technology Incorporation (CONITEC) in the Brazilian public health system as part of the guideline development process.

RESULTS

The corpus consisted of 303 texts, separated into 1233 text segments, 1081 of which were used, corresponding to retention of 87.67%. Five classes emerged from our analyses: mandatory presence of an obstetrician during labor and delivery in hospital settings; barriers and facilitators for guideline implementation; use of evidence-based practices by health professionals; progression of labor and delivery and women's rights; and mobilization to promote the guideline For each class, the most frequent words and sentences with the highest chi-squared scores were presented. Barriers were associated with lack of financial resources, training and professional motivation, and facilitators with training to change the practices of health professionals. Obstetric nurses emerged as an alternative for supervising normal births as well as the mandatory presence of an obstetrician during childbirth in hospital settings.

CONCLUSION

Our findings summarize the contributions provided by the Brazilian public and shed some light on the barriers and facilitators of clinical guidelines for care in normal birth. These topics are not typically explored by quantitative studies. Including this information in the decision-making process would not only increase public engagement, but provide greater evidence for implementing the clinical guidelines nationwide.

Limited role of patient input in specialty drug coverage policies

BACKGROUND: Despite increased financial contributions towards care, consumers' role in shaping their insurance benefits is unclear. OBJECTIVE: To examine the role played by patient input when US commercial health plans formulate specialty drug coverage policies, along with the benefits and challenges of considering this input. METHODS: We employed a parallel, mixed-methods approach. First, we reviewed health plans' policy development processes as reported on their websites. Second, we reviewed a data set of private health plan coverage decisions for specialty drugs and examined whether the evidence cited in policies included patient-reported outcomes (eg, health-related quality of life endpoints) and patient-based methodological designs (eg, interviews or surveys of patients). Third, we performed a survey (N = 21 respondents) and interviews (N = 5 interviewees) with plan decision-makers to determine the current role of patient input in plan decision-making, and the benefits and challenges of incorporating this data when formulating specialty drug coverage policies. RESULTS: We found that plans do not commonly solicit patient input when developing coverage policies, with only two instances of limited interaction between plans and patients or members. 1,316 (9%) of the studies plans cited in their specialty drug coverage policies included at least one patient-reported endpoint, and 0.4% (N = 62) used a patient-based methodological design. Of studies with patient-based designs, 40 used interviews, 26 included surveys/questionnaires, and one concerned shared decision-making (design categories not mutually exclusive). Almost half of the survey respondents reported having never engaged with patients or members when developing coverage policies. Among respondents who had engaged with patients or members, most reported doing so only rarely. The survey and interviews highlighted various benefits of soliciting patient input, including the value of obtaining a humanistic perspective, and several challenges, including resource requirements and the quality of obtained information. CONCLUSIONS: We found a notable lack of patient and member engagement by commercial health plans when formulating drug coverage policies. Survey respondents and interviewees identified benefits of accounting for patients' and plan members' values and preferences in specialty drug coverage policies, but also reported a number of important challenges to doing so. DISCLOSURES: National Pharmaceutical Council provided funding for this research.

Public values and plurality in health priority setting: What to do when people disagree and why we should care about reasons as well as choices

CONTEXT

'What does 'The Public' think?' is a question often posed by researchers and policy makers, and public values are regularly invoked to justify policy decisions. Over time there has been a participatory turn in the social and health sciences, including health technology assessment and priority setting in health, towards citizen participation such that public policies reflect public values. It is one thing to agree that public values are important, however, and another to agree on how public values should be elicited, deliberated upon and integrated into decision-making. Surveys of public values rarely deliver unanimity, and preference heterogeneity, or plurality, is to be expected.

METHODS

This paper examines the role of public values in health policy and how to elicit, analyse, and present values, in the face of plurality. We delineate the strengths and weaknesses of aggregative and deliberative methods before setting out a new empirical framework, drawing on Sunstein's Incompletely Theorised Agreements, based on three levels: principles, policies and patients. The framework is illustrated using a recognised policy dilemma - the provision of high cost, limited-effect medicines intended to extend life for people with terminal illnesses.

FINDINGS

Application of the multi-level framework to public values permits transparent consideration of plurality, including analysis of coherence and consensus, in a way that offers routes to policy recommendations that are based on public values and justified in those terms.

CONCLUSIONS

Using the new framework and eliciting quantitative and qualitative data across levels of abstraction has the potential to inform policy recommendations grounded in public values, where values are plural. This is not to suggest that one solution will magically emerge, but rather that choices between policies can be explicitly justified in relation to the properties of public values, and a much clearer understanding of (in)consistencies and areas of consensus.

Patient and public involvement in health technology assessment: update of a systematic review of international experiences

OBJECTIVE

To summarize current evidence on patient and public involvement (PPI) in health technology assessment (HTA) in order to synthesize the barriers and facilitators, and to propose a framework to assess its impact.

METHODS

We conducted an update of a systematic review published in 2011 considering the recent scientific literature (qualitative, quantitative, and mixed-methods studies). We searched papers published between March 2009 (end of the initial search) and December 2019 in five databases using specific search strategies. We identified other publications through citation tracking and contacting authors of previous related studies. Reviewers independently selected relevant studies based on prespecified inclusion and exclusion criteria. We extracted information using a pre-established grid.

RESULTS

We identified a total of 7872 publications from the main search strategy. Ultimately, thirty-one distinct new studies met the inclusion criteria, whereas seventeen studies were included in the previous systematic review. PPI is realized through two main strategies: (i) patients and public members participate directly in decision-making processes (participation) and (ii) patients or public perspectives are solicited to inform decisions (consultation or indirect participation). This review synthesizes the barriers and facilitators to PPI in HTA, and a framework to assess its impact is proposed.

CONCLUSION

The number of studies on patients or public involvement in HTA has dramatically increased in recent years. Findings from this updated systematic review show that PPI is done mostly through consultation and that direct involvement is less frequent. Several barriers to PPI in HTA exist, notably the lack of information to patients and public about HTA and the lack of guidance and policies to support PPI in HTA.

Social participation in the health technology incorporation process into Unified Health System

OBJECTIVE

To describe the current process of social participation in the incorporation of health technologies in Brazil, within the context of the Unified Health System (SUS).

METHODS

A descriptive study was conducted based on the analysis of official records of the actions of the National Committee for Health Technology Incorporation into Unified Health System and its website, from the beginning of its activities in January 2012 until December 2017.

RESULTS

The findings indicate that, in Brazil, there are legal instruments related to social participation in health, including the health technology assessment (HTA) field. However, its implementation is relatively recent and has been carried out gradually. In addition to the legal instruments (National Health Council representative, public consultation and public hearing forecast), other information and transparency strategies have been shown to be allied to social participation in the incorporation of health technologies. However, activities such as legally provided public hearings have not yet been carried out.

CONCLUSIONS

Several actions to foster social participation were developed over the analyzed period, but they need to be evaluated in order to maintain or improve them. In addition, there is a need for more qualified social participation in the various existing spaces, including those prescribed by law.

Defining the role of the public in Health Technology Assessment (HTA) and HTA-informed decision-making processes

Objectives

The terminology used to describe community participation in Health Technology Assessment (HTA) is contested and frequently confusing. The terms patients, consumers, public, lay members, customers, users, citizens, and others have been variously used, sometimes interchangeably. Clarity in the use of terms and goals for including the different groups is needed to mitigate existing inconsistencies in the application of patient and public involvement (PPI) across HTA processes around the world.

Methods

We drew from a range of literature sources in order to conceptualize (i) an operational definition for the “public” and other stakeholders in the context of HTA and (ii) possible goals for their involvement. Draft definitions were tested and refined in an iterative consensus-building process with stakeholders from around the world.

Results

The goals, terminology, interests, and roles for PPI in HTA processes were clarified. The research provides rationales for why the role of the public should be distinguished from that of patients, their families, and caregivers. A definition for the public in the context of HTA was developed: A community member who holds the public interest and has no commercial, personal, or professional interest in the HTA process

Conclusions

There are two distinct aspects to the interests held by the public which should be explicitly included in the HTA process: the first lies in ensuring democratic accountability and the second in recognising the importance of including public values in decision making.

Healthcare Technology Transfer in Sub-Saharan Africa: An Inductive Approach

The research addressed within this paper sets out to develop a framework towards facilitating health-related technology transfer (TT) to and within sub-Saharan African countries. In turn, this framework will attempt to alleviate healthcare burdens in developing nations through a combination of acquisitions and collaborative technology development. Systematic conceptual and comparative literature reviews have been conducted to identify the major characteristics of TT. The conceptual review has outlined the universal characteristics of TT such as TT methods, prominent stakeholders and the importance of knowledge transfer while the systematic comparative review exclusively evaluated sub-Saharan African healthcare TT characteristics such as infrastructure barriers and the marketability of the transfer object. The outcomes of the literature reviews have been clustered into five phases, forming the basis of the conceptual framework. This framework aims to guide a user through the phases of technology development, technology analysis, technology transfer method application, change management and commercialization by providing managerial best practices at each phase. The conceptual framework has been evaluated by incorporating the outcomes of 16 semi-structured interviews conducted with healthcare and TT industry experts. The final framework aims to provide guidelines for any stakeholder involved in healthcare technology transfer regardless of the healthcare implementation by highlighting best practices surrounding stakeholder co-creation, transfer method application and constructing a sustainable healthcare technology transfer venture.

What Is Next for Patient Preferences in Health Technology Assessment? A Systematic Review of the Challenges

BACKGROUND

Integrating patient preferences in Health Technology Assessment (HTA) is argued to improve uptake, adherence, and patient satisfaction. However, how to elicit and incorporate these preferences in HTA in a systematic and scientifically valid manner is subject to debate.

OBJECTIVE

This article provides a systematic review of the challenges to integrating patient preferences in HTA that have been raised in the literature about patient preferences in HTA.

METHODS

A systematic review of articles published between 2013 and 2017 addressing challenges to the integration of patient preferences in HTA was conducted in 7 databases. All issues with respect to the integration of patient preferences in HTA were extracted and divided into 5 categories: conceptual, normative, procedural, methodological, and practical issues. The issues were ranked according to how often they were mentioned.

RESULTS

Of 2147 retrieved articles, 67 were included in the analysis. Thirty-seven unique research issues were identified. In the majority of the articles, methodological issues were posed (82%), followed by procedural (73%), normative (51%), practical (24%), and conceptual (9%) issues. Frequently posed methodological issues concerned preference heterogeneity and choice of method. Common procedural issues concerned how to evaluate the impact of preference studies and their degree of being evidence based.

CONCLUSIONS

This article provides an overview of issues with respect to the integration of patient preferences in HTA procedures. Most issues were of a methodological or procedural nature; yet, the large number of different issues points to the overall importance of further researching the different aspects concerned with patient preferences in HTA. Through its ranking of how many articles mention particular issues, this article proposes an implicit research agenda.

Practical applications of rapid review methods in the development of Australian health policy

Rapid reviews (RRs) are a method of evidence synthesis that can provide robust evidence to support policy decisions in a timely manner. Herein we describe the methods used to conduct RRs and present an illustrative case study to describe how RRs can be used to inform contemporary Australian health policy. The aim of the present study was to explore several important aspects of how RRs can inform decision makers. RRs are conducted within limited time frames of as little as 4 weeks. Policy questions may focus on issues of efficacy, service delivery and service organisation rather than reimbursement of new services, which is better answered by a more comprehensive assessment. RRs use flexible and pragmatic methods, which aim to balance the objectivity and rigour required of the reviews within limited time frames. This flexibility allows for great variation across products with regard to length, depth of analysis and methods used. As a result, RRs can be specifically tailored to address targeted policy questions and are a useful tool in the development of Australian health policy.

Leaving No One Behind: Participatory Technology Appraisal as a Platform for Agenda Setting to Address Disparities in Access to Health Services in Thailand

OBJECTIVES

This article discusses how participatory technology appraisal as part of the Universal Coverage Scheme (UCS) in Thailand contributes to improving access to essential health services among vulnerable populations.

METHODS

Document review was conducted on health technology appraisal approaches introduced by the UCS. The review involves health benefit proposals advanced by stakeholders and also meeting minutes of relevant working groups and committees published between 2010 and 2015.

RESULTS

From the establishment of the UCS participatory technology appraisal mechanism in 2010 until 2015, a total of 133 health interventions have been nominated. Some nominations highlight problems in access to care among vulnerable populations. As policy advocates continue to be involved in the latter stages of coverage decisions, they have opportunities to persuade policy makers and other stakeholders to agree to the rationales of their proposals. Some interventions were rejected because they did not meet value for money, affordability, and feasibility criteria; however, topic nominations from stakeholders as well as relevant deliberation throughout the technology appraisal processes have a potential to improve accessibility of health care among the disadvantaged.

CONCLUSIONS

Through participation in the UCS policy-making processes, key stakeholders are able to direct the attention of decision makers to significant gaps in access to services among vulnerable citizens, a health system problem rarely brought to discussion by policy elites and experts. The Thai experience reaffirms participatory technology appraisal as a supportive measure to providing universal health coverage.

AGING AND HEALTH TECHNOLOGY ASSESSMENT: AN IDEA WHOSE TIME HAS COME

OBJECTIVES

With the increase in technologies to support an aging population, health technology assessment (HTA) of aging-related technologies warrants special consideration. At Health Technology Assessment international (HTAi) 2016 and HTAi 2017, an international panel explored interests in HTA focused on aging.

METHODS

Panelists from five countries shared the state of aging and HTA in their countries. Opportunities were provided for participants to discuss and rate the themes identified by the panelists.

RESULTS

In 2016, the highest ranked themes were: (i) identifying unmet needs of older adults that could be met by technology-how can HTA help?; (ii) differences in assessment of aging-related technologies-what is the scope?; and (iii) involvement of older adults and caregivers. These themes became the starting point for discussion in 2017, for which the highest ranked themes were: (i) identification of challenges in HTA and aging; and (ii) approaches to advancing effectiveness of HTA for aging.

CONCLUSION

These discussions allowed for examination of future directions for HTA and aging: engagement of older adults to inform the agenda of HTA and the broader public policy enterprise; a systems approach to thinking about needs of older persons should support the type and level of care desired by the individual rather than the health institutions, and HTA should reflect these desires when evaluating technological aides; and there is potential for health information systems and "big data" to support HTA activities that assess usability of technologies for older adults. We hope to build on the momentum of this community to continue exploring opportunities for aging and HTA.

Financial interests of patient organisations contributing to technology assessment at England's National Institute for Health and Care Excellence: policy review

OBJECTIVE

To investigate the prevalence of financial interests among patient organisations contributing to health technology assessment at the National Institute for Health and Care Excellence (NICE) in England and the extent to which NICE's disclosure policy ensures that decision making committees are aware of these interests.

DESIGN

Policy review using accounts, annual reports, and websites of patient organisations; payments declared by pharmaceutical manufacturers on their websites and a centralised database (Disclosure UK); declarations of interests by nominated representatives of patient organisations; and responses from patient organisations.

SETTING

Appraisals of medicines and treatments (technologies) for use in the English and Welsh National Health Service.

PARTICIPANTS

53 patient organisations contributing to 41 NICE technology appraisals published in 2015 and 2016, with 117 separate occasions that a patient organisation contributed to the appraisal of a technology.

MAIN OUTCOME MEASURES

Prevalence of specific interests (that is, funding from manufacturer(s) of a technology under appraisal or competitor products); proportion of specific interests of which NICE's decision making committees were aware; proportion of unknown specific interests for which disclosure was not required by NICE's policy RESULTS: 38/53 (72%) patient organisations had accepted funding from the manufacturer(s) of a technology or a competitor product in the same year that they had contributed to the appraisal of that technology or the previous year. Specific interests were present on 92/117 (79%) occasions that patient organisations contributed to appraisals in 2015 and 2016. NICE's decision making committees were aware of less than a quarter of specific interests (30/144; 21%). For nearly two thirds of the specific interests not known to committees (71/114; 62%), disclosure by patient organisations was not required by NICE's policy.

CONCLUSIONS

Financial interests are highly prevalent among patient organisations contributing to health technology assessment. NICE should review its disclosure policy to ensure that decision making committees are aware of all relevant interests.

Patient and Public Involvement in the Development of Healthcare Guidance: An Overview of Current Methods and Future Challenges

Clinical guidelines and health technology assessments are valuable instruments to improve the quality of healthcare delivery and aim to integrate the best available evidence with real-world, expert context. The role of patient and public involvement in their development has grown in recent decades, and this article considers the international literature exploring aspects of this participation, including the integration of experiential and scientific knowledge, recruitment strategies, models of involvement, stages of involvement, and methods of evaluation. These developments have been underpinned by the parallel rise of public involvement and evidence-based medicine as important concepts in health policy. Improving the recruitment of guideline group chairs, widening evidence reviews to include patient preference studies, adapting guidance presentation to highlight patient preference points and providing clearer instructions on how patient organisations can submit their intelligence are emerging proposals that may further enhance patient and public involvement in their processes.

EVALUATION OF PATIENT AND PUBLIC INVOLVEMENT INITIATIVES IN HEALTH TECHNOLOGY ASSESSMENT: A SURVEY OF INTERNATIONAL AGENCIES

Objectives: Although there is increased awareness of patient and public involvement (PPI) among health technology assessment (HTA) organizations, evaluations of PPI initiatives are relatively scarce. Our objective as members of Health Technology Assessment International's (HTAi's) Patient and Citizen Involvement Group (PCIG) was to advance understanding of the range of evaluation strategies adopted by HTA organizations and their potential usefulness.

Methods: In March 2016, a survey was sent to fifty-four HTA organizations through the International Network of Agencies for Health Technology Assessment (INAHTA) and contacts of members of HTAi's PCIG. Respondents were asked about their organizational structure; how patients and members of the public are involved; whether and how PPI initiatives have been evaluated, and, if so, which facilitators and challenges to evaluation were found and how results were used and disseminated.

Results: Fifteen (n = 15) programs from twelve countries responded (response rate 27.8 percent) that involved patients (14/15) and members of the public (10/15) in HTA activities. Seven programs evaluated their PPI activities, including participant satisfaction (5/7), process (5/7) and impact evaluations (4/7). Evaluation results were used to improve PPI activities, identify education and training needs, and direct strategic priorities. Facilitators and challenges revolved around the need for stakeholder buy-in, sufficient resources, senior leadership, and including patients in evaluations.

Conclusions: A small but diverse set of HTA organizations evaluate their PPI activities using a range of strategies that reflect the range of rationales and approaches to PPI in HTA. It will be important for HTA organizations to draw on evaluation theories and methods.

Envolvimento dos usuários de sistemas de saúde na Avaliação de Tecnologias em Saúde (ATS): uma revisão narrativa de estratégias internacionais

O objetivo foi descrever e analisar modelos e estratégias internacionais de envolvimento de usuários de sistemas de saúde nos processos de avaliação para incorporação de tecnologias de saúde registrados na literatura científica. Realizou-se levantamento da literatura em sete bases científicas, seleção de artigos, identificação e descrição dos modelos adotados em diferentes países. De 392 artigos identificados, oito foram selecionados segundo critérios de elegibilidade. As estratégias e modelos identificados foram classificados segundo o domínio do envolvimento; tipo de público e nível de envolvimento. A descrição dos modelos permitiu identificar uma ampla diversidade de experiências para envolvimento do público em processos de ATS. Os resultados apontam para a troca de conhecimentos e informações como forma de reduzir o distanciamento entre os usuários e os processos de avaliação mediante estratégias diversificadas de incentivo à participação.

What factors determine the choice of public engagement undertaken by health technology assessment decision-making organizations?

Purpose Public engagement in health technology assessment (HTA) is increasingly considered crucial for good decision making. Determining the "right" type of engagement activity is key in achieving the appropriate consideration of public values. Little is known about the factors that determine how HTA organizations (HTAOs) decide on their method of public engagement, and there are a number of possible factors that might shape these decisions. The purpose of this paper is to understand the potential drivers of public engagement from an organizational perspective. Design/methodology/approach The published HTA literature is reviewed alongside existing frameworks of public engagement in order to elucidate key factors influencing the choice of public engagement process undertaken by HTAOs. A conceptual framework is then developed to illustrate the factors identified from the literature that appear to influence public engagement choice. Findings Determining the type of public engagement undertaken in HTA is based on multiple factors, some of which are not always explicitly acknowledged. These factors included the: perceived complexity of the policy-making issue, perceived impact of the decision, transparency and opportunities for public involvement in governance, as well as time and resource constraints. The influences of these factors vary depending on the context, indicating that a one size fits all approach to public engagement may not be effective. Originality/value Awareness of the various factors that might influence the type of public engagement undertaken would enable decision makers to reflect on their choices and be more accountable and transparent about their choice of engagement process in eliciting public values and preferences in a HTAO.

Sustainability in health care by allocating resources effectively (SHARE) 4: exploring opportunities and methods for consumer engagement in resource allocation in a local healthcare setting

BACKGROUND

This is the fourth in a series of papers reporting a program of Sustainability in Health care by Allocating Resources Effectively (SHARE) in a local healthcare setting. Healthcare decision-makers have sought to improve the effectiveness and efficiency of services through removal or restriction of practices that are unsafe or of little benefit, often referred to as 'disinvestment'. A systematic, integrated, evidence-based program for disinvestment was being established within a large Australian health service network. Consumer engagement was acknowledged as integral to this process. This paper reports the process of developing a model to integrate consumer views and preferences into an organisation-wide approach to resource allocation.

METHODS

A literature search was conducted and interviews and workshops were undertaken with health service consumers and staff. Findings were drafted into a model for consumer engagement in resource allocation which was workshopped and refined.

RESULTS

Although consumer engagement is increasingly becoming a requirement of publicly-funded health services and documented in standards and policies, participation in organisational decision-making is not widespread. Several consistent messages for consumer engagement in this context emerged from the literature and consumer responses. Opportunities, settings and activities for consumer engagement through communication, consultation and participation were identified within the resource allocation process. Sources of information regarding consumer values and perspectives in publications and locally-collected data, and methods to use them in health service decision-making, were identified. A model bringing these elements together was developed.

CONCLUSION

The proposed model presents potential opportunities and activities for consumer engagement in the context of resource allocation.

Value to Whom? The Patient Voice in the Value Discussion

BACKGROUND

Professional societies and other organizations have recently taken a visible role trying to define treatment value via value frameworks and assessments, providing payer or provider recommendations, and potentially impacting patient access. Patient perspectives routinely differ from those of other stakeholders. Yet, it is not always apparent that patients were engaged in value framework development or assessment.

OBJECTIVES

To describe the development and content of the National Health Council's (NHC's) Rubric, a tool that includes criteria for evaluation of value frameworks specifically with regard to patient-centeredness and meaningful patient engagement.

METHODS

The NHC held a multistakeholder, invitational roundtable in Washington, DC, in 2016. Participants reviewed existing patient-engagement rubrics, discussed experiences with value frameworks, debated and thematically grouped hallmark patient-centeredness characteristics, and developed illustrative examples of the characteristics. These materials were organized into the rubric, and subsequently vetted via multistakeholder peer review.

RESULTS

The resulting rubric describes six domains of patient-centered value frameworks: partnership, transparency, inclusiveness, diversity, outcomes, and data sources. Each domain includes specific examples illustrating how patient engagement and patient-centeredness can be operationalized in value framework processes.

CONCLUSIONS

The NHC multistakeholder roundtable's recommendations are captured in the NHC's Rubric to assess value framework and model patient-centeredness and patient engagement. The Rubric is a tool that will be refined over time on the basis of feedback from patient, patient group, framework developer, and other stakeholder-use experiences.

Patient advocate perspectives on involvement in HTA: an international snapshot

PLAIN ENGLISH SUMMARY

A number of health technology assessment (HTA) organisations have developed processes to engage patients in the assessment of new health technologies such as pharmaceuticals, diagnostic tests, devices or medical procedures. Typically, this involves the HTA agency providing an opportunity for patient advocates and their patient organisations (support groups for patients with a specific disease or condition) to provide submissions detailing experiences with the disease and the health technology that is being assessed. While some literature exists about how HTA agencies view the engagement of patients in the HTA process, it is not yet clear how the patient advocates and patient organisations themselves view this engagement. To answer this question, we surveyed the views of patient advocates who were members of patient organisations known to be engaged in the process of HTA or evidence-based practice. Snowballing - that is, passing on the survey invitation from individuals invited to take part in the survey to other individuals - occurred in one of the countries. The responses in this country provided a very useful comparison between the views of people who were appointed as the 'patient representatives' on an HTA committee with those who contributed input as part of the general patient organisation engagement process. Our findings identify gaps in understanding of the purpose of patient involvement and whether patient organisations felt their input made a difference, the information and support provided, and if and how feedback is given to the patient organisations. Our work can help inform further research as well as continuing improvements in HTA patient engagement processes.

ABSTRACT

Background Patient involvement in health technology assessment (HTA) processes is becoming more frequent. However, it is not clear how patient advocates and their disease-based patient organisations that are involved in HTA view their involvement. We report on the results of an international survey of patient advocates and members of patient organisations about their experiences and perceptions of that involvement. Methods A 16-question survey was sent out to patient advocates and members of patient groups known to be involved in HTA processes or evidence-based practice. The survey consisted of open-ended questions focusing on respondent characteristics, stage and nature of involvement, support from HTA agencies for involvement, purpose of involvement, feedback on involvement, and whether the respondents felt that their input made a difference. Results Of 16 individuals who received the survey, 15 responded. Three, from Italy, Israel and Japan, were not involved in HTA in their country. Respondents from the following countries reported involvement in HTA processes: Canada, England, Scotland, and Wales, The Netherlands, Australia, Taiwan. The respondents indicated that HTA agencies reach out to them either actively or passively, and that their involvement is often at the appraisal stage of HTA. Typically, they reported involvement as either participants in committees or providers of submissions to HTA agencies. A wide range of approaches to supporting patient involvement by the HTA agencies was identified by respondents - including personal and telephone support, online resources, training and provision of information - but the level and type of support reported was uneven across jurisdictions. Not all respondents were clear on the purpose of their involvement in HTA, although some were able to cite specific examples of how their input made a difference; members of an HTA decision-making committee appeared to have a better understanding and were able to give examples. Feedback from HTA agencies to the patient groups on their submissions is often not provided. Conclusions Although considerable progress has been made in terms of engaging patients and patient groups in HTA, gaps remain in how involvement is supported, including facilitating involvement, clarity on roles, two-way flow of information, and methods for enhancing communication between patient organisations and HTA agencies.

CHANGING HEALTH TECHNOLOGY ASSESSMENT PARADIGMS?

Objectives: Health technology assessment (HTA) has to innovate to best support changing health system environments and to help provide access to valuable innovation under fiscal constraint.

Methods: Issues associated with changing HTA paradigms were identified through scoping and explored through deliberation at a meeting of industry and HTA leaders.

Results: Five broad areas of change (engagement, scientific dialogue, research prioritization, adaptive approaches, and real world data) were identified. The meeting focused on two themes derived from these: re-thinking scientific dialogue and multi-stakeholder engagement, and re-thinking value, affordability, and access. Earlier and ongoing engagement to steer the innovation process and help achieve appropriate use across the technology lifecycle was perceived as important but would be resource intensive and would require priority setting. Patients need to be involved throughout, and particularly at the early stages. Further discussion is needed on the type of body best suited to convening the dialogue required. There was agreement that HTA must continue to assess value, but views differed on the role that HTA should play in assessing affordability and on appropriate responses to challenges around affordability. Enhanced horizon scanning could play an important role in preparing for significant future investments.

Conclusions: Early and ongoing multi-stakeholder engagement and revisiting approaches to valuing innovation are required. Questions remain as to the most appropriate role for HTA bodies. Changing HTA paradigms extend HTA's traditional remit of being responsive to decision-makers demands to being more proactive and considering whole system value.

Mapping capacity to conduct health technology assessment in Central, Eastern and South-Eastern Europe

Aim

To provide insights into the capacity to conduct health technology assessment (HTA) in Central, Eastern, and South-Eastern Europe (CESEE), taking account of technical, financial, networking, and human resources.

Methods

An e-mail survey of 257 CESEE key informants involved in HTA was undertaken between March and April 2014. Contact e-mail addresses were identified from the internet. The survey questionnaire consisted of 3 sections: i) characteristics of the organization performing HTA, (ii) networking in HTA, and (iii) resources allocated for HTA.

Results

The survey was completed by 41 respondents representing a wide range of institutions from CESEE countries (response rate of 19.8%). Less than a quarter of respondents reported that their institutions had HTA-specific budgets, whereas the majority indicated that their institutions participated in HTA networks either at domestic or international levels. Although almost half of respondents indicated that their institutions offered HTA training, a shortage in skills training was suggested as the main barrier to HTA.

Conclusion

This is the first survey to thoroughly assess the state of HTA capacity in the CESEE region. To strengthen HTA capacity, CESEE countries should increase financial, technical, and training resources. To strengthen collaboration, the European Union and other international bodies should assist existing HTA networks in fulfilling their regional activities through leadership, advocacy to local policymakers, funding, and technical assistance.

CJCheck Stage 1: development and testing of a checklist for reporting community juries - Delphi process and analysis of studies published in 1996-2015

Background

Opportunities for community members to actively participate in policy development are increasing. Community/citizen's juries (CJs) are a deliberative democratic process aimed to illicit informed community perspectives on difficult topics. But how comprehensive these processes are reported in peer‐reviewed literature is unknown. Adequate reporting of methodology enables others to judge process quality, compare outcomes, facilitate critical reflection and potentially repeat a process. We aimed to identify important elements for reporting CJs, to develop an initial checklist and to review published health and health policy CJs to examine reporting standards.

Design

Using the literature and expertise from CJ researchers and policy advisors, a list of important CJ reporting items was suggested and further refined. We then reviewed published CJs within the health literature and used the checklist to assess the comprehensiveness of reporting.

Results

CJCheck was developed and examined reporting of CJ planning, juror information, procedures and scheduling. We screened 1711 studies and extracted data from 38. No studies fully reported the checklist items. The item most consistently reported was juror numbers (92%, 35/38), while least reported was the availability of expert presentations (5%, 2/38). Recruitment strategies were described in 66% of studies (25/38); however, the frequency and timing of deliberations was inadequately described (29%, 11/38).

Conclusions

Currently CJ publications in health and health policy literature are inadequately reported, hampering their use in policy making. We propose broadening the CJCheck by creating a reporting standards template in collaboration with international CJ researchers, policy advisors and consumer representatives to ensure standardized, systematic and transparent reporting.

PUBLIC AND PATIENT INVOLVEMENT IN HEALTH TECHNOLOGY ASSESSMENT: A FRAMEWORK FOR ACTION

OBJECTIVE

As health technology assessment (HTA) organizations in Canada and around the world seek to involve the public and patients in their activities, frameworks to guide decisions about whom to involve, through which mechanisms, and at what stages of the HTA process have been lacking. The aim of this study was to describe the development and outputs of a comprehensive framework for involving the public and patients in a government agency's HTA process.

METHODS

The framework was informed by a synthesis of international practice and published literature, a dialogue with local, national and international stakeholders, and the deliberations of a government agency's public engagement subcommittee in Ontario, Canada.

RESULTS

The practice and literature synthesis failed to identify a single, optimal approach to involving the public and patients in HTA. Choice of methods should be considered in the context of each HTA stage, goals for incorporating societal and/or patient perspectives into the process, and relevant societal and/or patient values at stake. The resulting framework is structured around four actionable elements: (i) guiding principles and goals for public and patient involvement (PPI) in HTA, (ii) the establishment of a common language to support PPI efforts, (iii) a flexible array of PPI approaches, and (iv) on-going evaluation of PPI to inform adjustments over time.

CONCLUSIONS

A public and patient involvement framework has been developed for implementation in a government agency's HTA process. Core elements of this framework may apply to other organizations responsible for HTA and health system quality improvement.

Patterns of public participation

Purpose

– The paper summarizes data from 12 countries, chosen to exhibit wide variation, on the role and place of public participation in the setting of priorities. The purpose of this paper is to exhibit cross-national patterns in respect of public participation, linking those differences to institutional features of the countries concerned.

Design/methodology/approach

– The approach is an example of case-orientated qualitative assessment of participation practices. It derives its data from the presentation of country case studies by experts on each system. The country cases are located within the historical development of democracy in each country.

Findings

– Patterns of participation are widely variable. Participation that is effective through routinized institutional processes appears to be inversely related to contestatory participation that uses political mobilization to challenge the legitimacy of the priority setting process. No system has resolved the conceptual ambiguities that are implicit in the idea of public participation.

Originality/value

– The paper draws on a unique collection of country case studies in participatory practice in prioritization, supplementing existing published sources. In showing that contestatory participation plays an important role in a sub-set of these countries it makes an important contribution to the field because it broadens the debate about public participation in priority setting beyond the use of minipublics and the observation of public representatives on decision-making bodies.

Steps toward improving ethical evaluation in health technology assessment: a proposed framework

Background

While evaluation of ethical aspects in health technology assessment (HTA) has gained much attention during the past years, the integration of ethics in HTA practice still presents many challenges. In response to the increasing demand for expansion of health technology assessment (HTA) methodology to include ethical issues more systematically, this article reports on a multi-stage study that aimed at construction of a framework for improving the integration of ethics in HTA.

Methods

The framework was developed through the following phases: 1) a systematic review and content analysis of guidance documents for ethics in HTA; 2) identification of factors influencing the integration of ethical considerations in HTA; 3) preparation of an action-oriented framework based on the key elements of the existing guidance documents and identified barriers to and facilitators of their implementation; and 4) expert consultation and revision of the framework.

Results

The proposed framework consists of three main components: an algorithmic flowchart, which exhibits the different steps of an ethical inquiry throughout the HTA process, including: defining the objectives and scope of the evaluation, stakeholder analysis, assessing organizational capacity, framing ethical evaluation questions, ethical analysis, deliberation, and knowledge translation; a stepwise guide, which focuses on the task objectives and potential questions that are required to be addressed at each step; and a list of some commonly recommended or used tools to help facilitate the evaluation process.

Conclusions

The proposed framework can be used to support and promote good practice in integration of ethics into HTA. However, further validation of the framework through case studies and expert consultation is required to establish its utility for HTA practice.

Preferences for engagement in health technology assessment decision-making: a nominal group technique with members of the public

OBJECTIVE

To identify characteristics (factors) about health technology assessment (HTA) decisions that are important to the public in determining whether public engagement should be undertaken and the reasons for these choices.

DESIGN

Focus groups using a nominal group technique to identify and rank factors relevant to public engagement in HTA decision-making. Thematic analysis was also undertaken to describe reasons underpinning participants' choices and rankings.

SETTING

Members of the Australian general public.

PARTICIPANTS

58 people, aged 19-71 years participated in 6 focus groups.

RESULTS

24 factors were identified by participants that were considered important in determining whether public engagement should be undertaken. These factors were individually ranked and grouped into 4 themes to interpret preferences for engagement. Members of the public were more likely to think public engagement was needed when trade-offs between benefits and costs were required to determine 'value', uncertainties in the evidence were present, and family members and/or carers were impacted. The role of public engagement was also seen as important if the existent system lacked transparency and did not provide a voice for patients, particularly for conditions less known in the community.

CONCLUSIONS

Members of the public considered value, impact, uncertainty, equity and transparency in determining when engagement should be undertaken. This indicates that the public's preferences on when to undertake engagement relate to both the content of the HTA itself as well as the processes in place to support HTA decision-making. By understanding these preferences, decision-makers can work towards more effective, meaningful public engagement by involving the public in issues that are important to them and/or improving the processes around decision-making.

It's not evidence, it's insight: bringing patients' perspectives into health technology appraisal at NICE

PLAIN ENGLISH SUMMARY

Health technology appraisal involves reviewing the findings from clinical trials and economic data to produce guidance on how health technology should be used. This task is carried out by appraisal committees in NICE. One of the several ways in which patients can feed their views into these committees is via a written patient statement. We asked nine committee members about what difference the information from patients makes to their decision-making. The Committee members reported that written patient statements offer a different perspective when reviewing the clinical and economic data. This can have a profound impact when a committee draws conclusions based solely on the data, which may not reflect the reality of patients' lives. The patients' and carers' input provides meaning to the data, 'bringing the numbers to life'. It identifies if the technology has any wider impacts than what's been reported in the clinical trial, and also if the trial has measured what's important to patients. We conclude that the written patient statement adds value to the decision-making process by helping Committee members to make sense of the clinical and economic data-it makes them look at the evidence 'in a different light'. Patients' stories are very effective in this context, because they have the power to communicate and to challenge Committee members' assumptions. Understanding this difference between analysing research evidence and drawing on patients' insights is important in thinking about what's needed in a written patient statement and the best way to obtain it.

ABSTRACT

Background Health technology appraisal involves reviewing clinical and economic data to inform guidance on the use of technology. In England this task is carried out by appraisal committees within the National Institute for Health and Care Excellence (NICE). Patients are not committee members as they have a vested interest in the outcome, but one of the several ways they are involved is through submitting a written patient statement, which is considered by the committee during its deliberations. We aimed to find out how the written patient statement adds value to the decision-making process by exploring how it is used in practice. Methods Semi-structured interviews were conducted with nine members of NICE appraisal committees. The interviews were transcribed and analysed thematically. We drew on published evidence of the impact of patient involvement on clinical research and our experience of supporting organisations to produce written patient statements to analyse the findings. Results Committee members reported that written patient statements offer a different perspective when evaluating clinical and economic data. This can have a profound impact when a committee draws conclusions based on data that may not reflect the reality of the patient experience. Information from patients and carers also provides context and meaning to the data, by explaining its real-life implications. It identifies wider impacts of a technology that may not have been assessed in a clinical trial, as well as commenting on whether what has been measured in a trial is relevant to patients. The main barrier to using the written patient statement is the misperception that it is a form of research 'evidence', when in fact it takes the form of experiential knowledge - or insight. Conclusions The written patient statement adds value by aiding Committee members in their interpretation of existing evidence - it enables them to consider this evidence 'in a different light'. In this context, patients' experiential knowledge is effective because it is subjective, emotional and anecdotal. It then has the power to communicate and to challenge assumptions based on the data alone. Understanding this difference between using evidence and insights has implications for the content of a written patient statement and the approaches used to obtain it.

Prioritising patients for bariatric surgery: building public preferences from a discrete choice experiment into public policy

OBJECTIVES

To derive priority weights for access to bariatric surgery for obese adults, from the perspective of the public.

SETTING

Australian public hospital system.

PARTICIPANTS

Adults (N=1994), reflecting the age and gender distribution of Queensland and South Australia.

PRIMARY AND SECONDARY OUTCOME MEASURES

A discrete choice experiment in which respondents indicated which of two individuals with different characteristics should be prioritised for surgery in repeated hypothetical choices. Potential surgery recipients were described by seven key characteristics or attributes: body mass index (BMI), presence of comorbid conditions, age, family history, commitment to lifestyle change, time on the surgical wait list and chance of maintaining weight loss following surgery. A multinomial logit model was used to evaluate preferences and derive priority weights (primary analysis), with a latent class model used to explore respondent characteristics that were associated with variation in preference across the sample (see online supplementary analysis).

RESULTS

A preference was observed to prioritise individuals who demonstrated a strong commitment to maintaining a healthy lifestyle as well as individuals categorised with very severe (BMI≥50 kg/m2) or (to a lesser extent) severe (BMI≥40 kg/m2) obesity, those who already have obesity-related comorbidity, with a family history of obesity, with a greater chance of maintaining weight loss or who had spent a longer time on the wait list. Lifestyle commitment was considered to be more than twice as important as any other criterion. There was little tendency to prioritise according to the age of the recipient. Respondent preferences were dependent on their BMI, previous experience with weight management surgery, current health state and education level.

CONCLUSIONS

This study extends our understanding of the publics' preferences for priority setting to the context of bariatric surgery, and derives priority weights that could be used to assist bodies responsible for commissioning bariatric services.

WHAT IS THE ROLE OF COMMUNITY PREFERENCE INFORMATION IN HEALTH TECHNOLOGY ASSESSMENT DECISION MAKING? A CASE STUDY OF COLORECTAL CANCER SCREENING

OBJECTIVES

The aim of this study was to determine the role of community preference information from discrete choice studies of colorectal cancer (CRC) screening in health technology assessment (HTA) reports and subsequent policy decisions.

METHODS

We undertook a systematic review of discrete choice studies of CRC screening. Included studies were reviewed to assess the policy context of the research. For those studies that cited a recent or pending review of CRC screening, further searches were undertaken to determine the extent to which community preference information was incorporated into the HTA decision-making process.

RESULTS

Eight discrete choice studies that evaluated preferences for CRC screening were identified. Four of these studies referred to a national or local review of CRC screening in three countries: Australia, Canada, and the Netherlands. Our review of subsequently released health policy documents showed that while consideration was given to community views on CRC, policy was not informed by discrete choice evidence.

CONCLUSIONS

Preferences and values of patients are increasingly being considered "evidence" to be incorporated into HTA reports. Discrete choice methodology is a rigorous quantitative method for eliciting preferences and while as a methodology it is growing in profile, it would appear that the results of such research are not being systematically translated or integrated into HTA reports. A formalized approach is needed to incorporate preference literature into the HTA decision-making process.

BARRIERS AND FACILITATORS INFLUENCING ETHICAL EVALUATION IN HEALTH TECHNOLOGY ASSESSMENT

Objectives: The objective of this study was to explore barriers and facilitators influencing the integration of ethical considerations in health technology assessment (HTA).

Methods: The study consisted of two complementary approaches: (a) a systematic review of the literature; and (b) an eighteen-item online survey that was distributed to fifty-six HTA agencies affiliated with the International Network of Agencies for Health Technology Assessment.

Results: The review identified twenty-six relevant articles. The most often cited barriers in the literature were: scarcity, heterogeneity and complexity of ethical analysis methods; challenges in translating ethical analysis results into knowledge that is useful for decision makers; and lack of organizational support in terms of required expertise, time and financial resources. The most frequently cited facilitators included: usage of value-based appraisal methods, stakeholder and public engagement, enhancement of practice guidelines, ethical expertise, and educational interventions.

Representatives of twenty-six (46.5 percent) agencies from nineteen countries completed the survey. A median of 10 percent (interquartile range, 5 percent to 50 percent) of the HTA products produced by the agencies was reported to include an assessment of ethical aspects. The most commonly perceived barriers were: limited ethical knowledge and expertise, insufficient time and resources, and difficulties in finding ethical evidence or using ethical guidelines. Educational interventions, demand by policy makers, and involvement of ethicists in HTA were the most commonly perceived facilitators.

Conclusions: Our results emphasize the importance of simplification of ethics methodology and development of good practice guidelines in HTA, as well as capacity building for engaging HTA practitioners in ethical analyses.

Public preferences for engagement in Health Technology Assessment decision-making: protocol of a mixed methods study

BACKGROUND

Much attention in recent years has been given to the topic of public engagement in health technology assessment (HTA) decision-making. HTA organizations spend substantial resources and time on undertaking public engagement, and numerous studies have examined challenges and barriers to engagement in the decision-making process however uncertainty remains as to optimal methods to incorporate the views of the public in HTA decision-making. Little research has been done to ascertain whether current engagement processes align with public preferences and to what extent their desire for engagement is dependent on the question being asked by decision-makers or the characteristics of the decision. This study will examine public preferences for engagement in Australian HTA decision-making using an exploratory mixed methods design.

METHODS/DESIGN

The aims of this study are to: 1) identify characteristics about HTA decisions that are important to the public in determining whether public engagement should be undertaken on a particular topic, 2) determine which decision characteristics influence public preferences for the extent, or type of public engagement, and 3) describe reasons underpinning these preferences. Focus group participants from the general community, aged 18-70 years, will be purposively sampled from the Australian population to ensure a wide range of demographic groups. Each focus group will include a general discussion on public engagement as well as a ranking exercise using a modified nominal group technique (NGT). The NGT will inform the design of a discrete choice study to quantitatively assess public preferences for engagement in HTA decision-making.

DISCUSSION

The proposed research seeks to investigate under what circumstances and how the public would like their views and preferences to be considered in health technology assessments. HTA organizations regularly make decisions about when and how public engagement should occur but without consideration of the public's preferences on the method and extent of engagement. This information has the potential to assist decision-makers in tailoring engagement approaches, and may be particularly useful in decisions with potential for conflict where clarification of public values and preferences could strengthen the decision-making process.

Public attitudes and values in priority setting

There is growing recognition that critical decisions concerning investments in new health care technologies and services should incorporate society’s values along with the scientific evidence. From a normative perspective, public engagement can help realize the democratic ideals of legitimacy, transparency, and accountability. On a more pragmatic level, public engagement can help stakeholders understand the degree of popular support for policy options, and may enhance public trust in decision-making processes. To better understand public attitudes and values relating to priority setting in health care, researchers and decision-makers will have to employ a range of quantitative and qualitative approaches, drawing on different disciplines and methodological traditions.

Evaluation of patient involvement in a health technology assessment

OBJECTIVES

We sought to evaluate patient involvement (consultation and direct participation) in the assessment of alternative measures to restraint and seclusion among adults in short-term hospital wards (in psychiatry) and long-term care facilities for the elderly.

METHODS

We conducted individual semi-structured interviews with thirteen stakeholders: caregivers, healthcare managers, patient representatives, health technology assessment (HTA) unit members, researchers, and members of the local HTA scientific committee. Data were collected until saturation. We carried out content analysis of two HTA reports and four other documents that were produced in relation with this HTA. We also used field notes taken during formal meetings and informal discussions with stakeholders. We performed thematic analysis based on a framework for assessing patient involvement in HTA. We then triangulated data.

RESULTS

For the majority of interviewees, patient consultation enriched the content of the HTA report and its recommendations. This also made it possible to suggest other alternatives that could reduce the use of restraint and seclusion and helped confirm some views and comments from healthcare professionals consulted in this HTA. The direct participation of patient representatives enabled rephrasing of some findings so as to bring the patient perspective to the HTA report.

CONCLUSIONS

Patient consultation was seen as having directly influenced the content of the HTA report while direct participation made it possible to rephrase some findings. This is one of few studies to assess the impact of patient involvement in HTA and more such studies are needed to identify the best ways to improve the input of such involvement.

Involving patients in health technology funding decisions: stakeholder perspectives on processes used in Australia

BACKGROUND

Governments use a variety of processes to incorporate public perspectives into policymaking, but few studies have evaluated these processes from participants' point of view.

OBJECTIVE

The objective of this study was twofold: to understand the perspectives of selected stakeholders with regard to involvement processes used by Australian Advisory Committees to engage the public and patients; and to identify barriers and facilitators to participation.

DESIGN

Twelve semi-structured interviews were conducted with representatives of different stakeholder groups involved in health technology funding decisions in Australia. Data were collected and analysed using a theoretical framework created by Rowe and Frewer, but adapted to more fully acknowledge issues of power and influence.

RESULTS

Stakeholder groups disagreed as to what constitutes effective and inclusive patient involvement. Barriers reported by interviewees included poor communication, a lack of transparency, unworkable deadlines, and inadequate representativeness. Also described were problems associated with defining the task for patients and their advocates and with the timing of patient input in the decision-making process. Interviewees suggested that patient participation could be improved by increasing the number of patient organizations engaged in processes and including those organizations at different stages of decision making, especially earlier.

CONCLUSIONS

The different evaluations made by stakeholder groups appear to be underpinned by contrasting conceptions of public involvement and its value, in line with Graham Martin's work which distinguishes between 'technocratic' and 'democratic' public involvement. Understanding stakeholders' perspectives and the contrasting conceptions of public involvement could foster future agreement on which processes should be used to involve the public in decision making.

Understanding the role of patient organizations in health technology assessment

BACKGROUND

The involvement of patient representatives in health technology assessment is increasingly seen by policy makers and researchers as key for the deployment of patient-centred health care, but there is uncertainty and a lack of theoretical understanding regarding the knowledge and expertise brought by patient representatives and organisations to HTA processes.

OBJECTIVE

To propose a conceptually-robust typological model of the knowledge and expertise held by patient organisations.

DESIGN, DATA COLLECTION AND ANALYSIS

The study followed a case-study design. Data were collected within an international research project on patient organisations' engagement with knowledge, and included archival and documentary data, in-depth interviews with key members of the organisation and participant observation. Data analysis followed standard procedure of qualitative analysis anchored in an analytic induction approach.

RESULTS

Analysis identified three stages in the history of the patient organisation under analysis - Alzheimer's Society. In a first period, the focus is on 'caring knowledge' and an emphasis on its volunteer membership. In a transition stage, a combination of experiential, clinical and scientific knowledge is proposed in an attempt to expand its field of activism into HTA. In the most recent phase, there is a deepening of its network of associations to secure its role in the production of evidence.

CONCLUSIONS

Analysis identified an important relationship between the forms of knowledge deployed by patient organisations and the networks of expertise and policy they mobilise to pursue their activities. A model of this relationship is outlined, for the use of further research and practice on patient involvement.

A descriptive review on methods to prioritize outcomes in a health care context

BACKGROUND

Evidence synthesis has seen major methodological advances in reducing uncertainty and estimating the sizes of the effects. Much less is known about how to assess the relative value of different outcomes.

OBJECTIVE

To identify studies that assessed preferences for outcomes in health conditions.

METHODS

SEARCH STRATEGY

we searched MEDLINE, EMBASE, PsycINFO and the Cochrane Library in February 2014.

INCLUSION CRITERIA

eligible studies investigated preferences of patients, family members, the general population or healthcare professionals for health outcomes. The intention of this review was to include studies which focus on theoretical alternatives; studies which assessed preferences for distinct treatments were excluded.

DATA EXTRACTION

study characteristics as study objective, health condition, participants, elicitation method, and outcomes assessed in the study were extracted.

MAIN RESULTS

One hundred and twenty-four studies were identified and categorized into four groups: (1) multi criteria decision analysis (MCDA) (n = 71), (2) rating or ranking (n = 25), (3) utility eliciting (n = 5) and (4) studies comparing different methods (n = 23). The number of outcomes assessed by method group varied. The comparison of different methods or subgroups within one study often resulted in different hierarchies of outcomes.

CONCLUSIONS

A dominant method most suitable for application in evidence syntheses was not identified. As preferences of patients differ from those of other stakeholders (especially medical professionals), the choice of the group to be questioned is consequential. Further research needs to focus on validity and applicability of the identified methods.

Australian Public Preferences for the Funding of New Health Technologies

Background. Ethical, economic, political, and legitimacy arguments support the consideration of public preferences in health technology decision making. The objective was to assess public preferences for funding new health technologies and to compare a profile case best-worst scaling (BWS) and traditional discrete choice experiment (DCE) method. Methods. An online survey consisting of a DCE and BWS task was completed by 930 adults recruited via an Internet panel. Respondents traded between 7 technology attributes. Participation quotas broadly reflected the population of Queensland, Australia, by gender and age. Choice data were analyzed using a generalized multinomial logit model. Results. The findings from both the BWS and DCE were generally consistent in that respondents exhibited stronger preferences for technologies offering prevention or early diagnosis over other benefit types. Respondents also prioritized technologies that benefit younger people, larger numbers of people, those in rural areas, or indigenous Australians; that provide value for money; that have no available alternative; or that upgrade an existing technology. However, the relative preference weights and consequent preference orderings differed between the DCE and BWS models. Further, poor correlation between the DCE and BWS weights was observed. While only a minority of respondents reported difficulty completing either task (22.2% DCE, 31.9% BWS), the majority (72.6%) preferred the DCE over BWS task. Conclusions. This study provides reassurance that many criteria routinely used for technology decision making are considered to be relevant by the public. The findings clearly indicate the perceived importance of prevention and early diagnosis. The dissimilarity observed between DCE and profile case BWS weights is contrary to the findings of previous comparisons and raises uncertainty regarding the comparative merits of these stated preference methods in a priority-setting context.