Economic evaluations conducted by Canadian health technology assessment agencies: Where do we stand?

Adherence to country-specific guidelines among economic evaluations undertaken in three high-income and middle-income countries: a systematic review

OBJECTIVE

To assess the adherence of economic evaluations to the recommendations on principles of economic evaluation as stated in the country-specific guidelines for three countries across different income groups, namely, Canada, South Africa, and Egypt.

METHODS

Searches were undertaken in three databases to identify economic evaluations meeting predefined inclusion criteria. Methodological and reporting standards listed in the country-specific guidelines were converted into discrete binary variables to calculate mean adherence scores. Quality appraisal was done using Drummond's checklist. Stratified analysis was undertaken to identify independent variables affecting adherence.

RESULTS

We identified forty-four, seventy-nine, and sixteen economic evaluations for Canada, South Africa, and Egypt, respectively. The mean adherence score was the highest for Canada (71%), followed by South Africa (65%) and Egypt (60%). Adherence to guidelines was positively correlated with quality of studies, r = .72. Furthermore, the mean adherence score was significantly (p < .05) higher for studies using a cost-utility analysis design (72%), having local/national funding aid (72%), undertaken by a health economist (71%) and for pharmacoeconomic evaluations (70%).

CONCLUSION

The quality of economic evaluations improves with adherence to country-specific guidelines. Locally funded and health-economist led health technology assessments (HTAs) should be encouraged for greater adherence to the guidelines. The HTA researchers and the HTA bodies should lay emphasis on adherence to the country-specific guidelines for improving the quality of HTA evidence.

Applying an Implementation Framework to the Use of Evidence from Economic Evaluations in Making Healthcare Decisions

BACKGROUND AND OBJECTIVE

There is a need for the application of theory in understanding the use of evidence from economic evaluations in healthcare decision making. The purpose of this study is to review the published literature on the use of evidence from economic evaluations for healthcare decision making and to map the findings to the Consolidated Framework for Implementation Research (CFIR).

METHODS

A systematic search strategy was used to identify studies investigating the factors that determine the use of evidence from economic evaluation in healthcare decision making. Barriers and facilitators identified in the included studies were mapped across the five CFIR domains, with the "intervention" referring to the use of economic evaluations in decision making. Gaps, inconsistencies and emergent relations were identified through the mapping process.

RESULTS

Fifty-three studies met eligibility criteria and were included in the review. The CFIR constructs associated with the Intervention Characteristics and those associated with the knowledge and beliefs of users of economic evaluations were widely cited in the identified barriers and facilitators. Other constructs from the CFIR had not been reported in the literature, such as 'organisational networks' and 'individual stage of change'. Most of the stages in the implementation process as described by the CFIR were reflected in the identified barriers and facilitators.

DISCUSSION

By categorising barriers and facilitators into domains, the CFIR provides a systematic approach to assess how these factors interact. Literature gaps in the literature regarding the use of economic evaluation in healthcare decision making were identified, specifically issues regarding organisational networks and the role of feedback.

CONCLUSIONS

Through mapping findings from studies of the use of evidence from economic evaluations in healthcare decision making, we present an implementation framework based on the CFIR for understanding the use of economic evaluations into practice.

BARRIERS AND FACILITATORS INFLUENCING ETHICAL EVALUATION IN HEALTH TECHNOLOGY ASSESSMENT

Objectives: The objective of this study was to explore barriers and facilitators influencing the integration of ethical considerations in health technology assessment (HTA).

Methods: The study consisted of two complementary approaches: (a) a systematic review of the literature; and (b) an eighteen-item online survey that was distributed to fifty-six HTA agencies affiliated with the International Network of Agencies for Health Technology Assessment.

Results: The review identified twenty-six relevant articles. The most often cited barriers in the literature were: scarcity, heterogeneity and complexity of ethical analysis methods; challenges in translating ethical analysis results into knowledge that is useful for decision makers; and lack of organizational support in terms of required expertise, time and financial resources. The most frequently cited facilitators included: usage of value-based appraisal methods, stakeholder and public engagement, enhancement of practice guidelines, ethical expertise, and educational interventions.

Representatives of twenty-six (46.5 percent) agencies from nineteen countries completed the survey. A median of 10 percent (interquartile range, 5 percent to 50 percent) of the HTA products produced by the agencies was reported to include an assessment of ethical aspects. The most commonly perceived barriers were: limited ethical knowledge and expertise, insufficient time and resources, and difficulties in finding ethical evidence or using ethical guidelines. Educational interventions, demand by policy makers, and involvement of ethicists in HTA were the most commonly perceived facilitators.

Conclusions: Our results emphasize the importance of simplification of ethics methodology and development of good practice guidelines in HTA, as well as capacity building for engaging HTA practitioners in ethical analyses.

Bridging the gap: exploring the barriers to using economic evidence in healthcare decision making and strategies for improving uptake

Evidence from economic evaluations is often not used to inform healthcare policy despite being well regarded by policy makers and physicians. This article employs the accessibility and acceptability framework to review the barriers to using evidence from economic evaluation in healthcare policy and the strategies used to overcome these barriers. Economic evaluations are often inaccessible to policymakers due to the absence of relevant economic evaluations, the time and cost required to conduct and interpret economic evaluations, and lack of expertise to evaluate quality and interpret results. Consistently reported factors that limit the translation of findings from economic evaluations into healthcare policy include poor quality of research informing economic evaluations, assumptions used in economic modelling, conflicts of interest, difficulties in transferring resources between sectors, negative attitudes to healthcare rationing, and the absence of equity considerations. Strategies to overcome these barriers have been suggested in the literature, including training, structured abstract databases, rapid evaluation, reporting checklists for journals, and considering factors other than cost effectiveness in economic evaluations, such as equity or budget impact. The factors that prevent or encourage decision makers to use evidence from economic evaluations have been identified, but the relative importance of these factors to decision makers is uncertain.

Ethics, economics and the regulation and adoption of new medical devices: case studies in pelvic floor surgery

BACKGROUND

Concern has been growing in the academic literature and popular media about the licensing, introduction and adoption of surgical devices before full effectiveness and safety evidence is available to inform clinical practice. Our research will seek empirical survey evidence about the roles, responsibilities, and information and policy needs of the key stakeholders in the introduction into clinical practice of new surgical devices for pelvic floor surgery, in terms of the underlying ethical principals involved in the economic decision-making process, using the example of pelvic floor procedures.

METHODS/DESIGN

Our study involves three linked case studies using, as examples, selected pelvic floor surgery devices representing Health Canada device safety risk classes: low, medium and high risk. Data collection will focus on stakeholder roles and responsibilities, information and policy needs, and perceptions of those of other key stakeholders, in seeking and using evidence about new surgical devices when licensing and adopting them into practice. For each class of device, interviews will be used to seek the opinions of stakeholders. The following stakeholders and ethical and economic principles provide the theoretical framework for the study: Stakeholders--federal regulatory body, device manufacturers, clinicians, patients, health care institutions, provincial health departments, and professional societies. Clinical settings in two centres (in different provinces) will be included. Ethics--beneficence, non-maleficence, autonomy, justice. Economics--scarcity of resources, choices, opportunity costs.For each class of device, responses will be analysed to compare and contrast between stakeholders. Applied ethics and economic theory, analysis and critical interpretation will be used to further illuminate the case study material.

DISCUSSION

The significance of our research in this new area of ethics will lie in providing recommendations for regulatory bodies, device manufacturers, clinicians, health care institutions, policy makers and professional societies, to ensure surgical patients receive sufficient information before providing consent for pelvic floor surgery. In addition, we shall provide a wealth of information for future study in other areas of surgery and clinical management, and provide suggestions for changes to health policy.

A bibliometric study of scientific literature on obesity research in PubMed (1988-2007)

This article describes a bibliometric review of the publications on obesity research in PubMed over the last 20 years. We used Medline via the PubMed online service of the US National Library of Medicine from 1988 to 2007. The search strategy was: ([obesity] in MesH). A total of 58,325 references were retrieved, 25.5% in 1988-1997, and 74.5% in 1998-2007. The growth in the number of publications showed an exponential increase. The references were published in 3613 different journals, with 20 journals contributing 25% of obesity literature. The two journals contributing most were the International Journal of Obesity (5.1%), Obesity-Obesity Research (2.9%). North America and Europe were the most productive world areas with 44.1% and 37.9% of the literature, respectively. The US was the predominant country in number of publications, followed by the United Kingdom, Japan and Italy. The ranking of production changed when the number of publications was normalized by population, gross domestic product and obesity prevalence by countries. The great increase of publications on obesity during the period 1988-2007 was particularly evident in the second decade of the period which is concordant with the worldwide obesity epidemic. USA and Europe were leaders in the production of scientific articles on obesity.

Impact of the 1997 Canadian guidelines on the conduct of Canadian-based economic evaluations in the published literature

OBJECTIVE

To assess the impact of the 1997 Canadian guidelines on the methods and presentation of economic evaluations conducted from a Canadian perspective in the published literature.

METHODS

A systematic literature review was conducted to identify health technology economic evaluations conducted from a Canadian perspective published in peer-reviewed journals between 2001 and 2006. To investigate the impact of the 1997 Canadian Coordinating Office of Health Technology Assessment guidelines, each included study was assessed against 17 of the 25 recommendations.

RESULTS

Of the 153 included studies, a base set of 9 methodological standards, as outlined by the 1997 guidelines, were followed by over 50% of the studies including: indications, outcomes for cost utility analysis, outcomes for cost benefit analysis, discounting future cost and outcomes, cost identification and valuation, evaluating uncertainty and disclosing funding relationships. Main divergences from the guidelines were found for analytic technique (38%), study perspective (23%), source of preferences (8%), equity (7%), and cost measurement (24%).

CONCLUSION

The current assessment has shown that the 1997 Canadian guidelines have set a minimum methodological standard within the community of "doers" conducting economic analyses from a Canadian perspective. Although there was divergence from some of the recommendations, the majority were reflected as changes in the 2006 Canadian guidelines.

In search of the evidentiary foundation of published Canadian economic evaluations (2001–06)

Objectives:The aim of this study was to present a review of economic evaluations conducted from a Canadian perspective and to characterize sources of evidence and statistical methods to analyze effectiveness measures, resource utilization, and uncertainty.

Methods:A search strategy was developed to identify Canadian economic evaluations published between January 2001 and June 2006. A standardized abstraction form was used to extract key data (e.g., study design, data sources, statistical methods).

Results:A total of 153 unique studies were included for review, of which 75 were evaluations of drug therapies and less than half were funded by industry. Cost-effectiveness analysis was the most common type of economic evaluation and 80 percent of the studies used modeling techniques. A single source of evidence for effectiveness measures was used in half of the studies. Statistical methods were commonly reported to compare effectiveness measures when the economic evaluation was conducted alongside a clinical trial but less commonly when determining effectiveness input parameters in model-based economic evaluations, or to analyze resource utilization data. Authors relied mostly on univariate sensitivity analyses to explore uncertainty.

Conclusions:This review identifies the need to improve the conduct and reporting of statistical methods for economic evaluations to improve confidence in the results.