The GRADE approach for assessing new technologies as applied to apheresis devices in ulcerative colitis

DEFINING THE RELEVANT OUTCOME MEASURES IN MEDICAL DEVICE ASSESSMENTS: AN ANALYSIS OF THE DEFINITION PROCESS IN HEALTH TECHNOLOGY ASSESSMENT

BACKGROUND

Defining relevant outcome measures for clinical trials on medical devices (MD) is complex, as there is a large variety of potentially relevant outcomes. The chosen outcomes vary widely across clinical trials making the assessment in evidence syntheses very challenging. The objective is to provide an overview on the current common procedures of health technology assessment (HTA) institutions in defining outcome measures in MD trials.

METHODS

In 2012-14, the Web pages of 126 institutions involved in HTA were searched for methodological manuals written in English or German that describe methods for the predefinition process of outcome measures. Additionally, the institutions were contacted by email. Relevant information was extracted. All process steps were performed independently by two reviewers.

RESULTS

Twenty-four manuals and ten responses from the email request were included in the analysis. Overall, 88.5 percent of the institutions describe the type of outcomes that should be considered in detail and 84.6 percent agree that the main focus should be on patient relevant outcomes. Specifically related to MD, information could be obtained in 26 percent of the included manuals and email responses. Eleven percent of the institutions report a particular consideration of MD related outcomes.

CONCLUSIONS

This detailed analysis on common procedures of HTA institutions in the context of defining relevant outcome measures for the assessment of MD shows that standardized procedures for MD from the perspective of HTA institutions are not widespread. This leads to the question if a homogenous approach should be implemented in the field of HTA on MD.

A Critical Review of Grading Systems: Implications for Public Health Policy

Grading instruments are an important part of evidence-based medicine and are used to inform health policy and the development of clinical practice guidelines. They are extensively used in the development of clinical guidelines and the assessment of research publications, having particular impact on health care and policy sectors. The positive effects of using grading instruments are, however, potentially undermined by their misuse and a number of shortcomings. This review found eight key concerns about grading instruments: (1) lack of information on validity and reliability, (2) poor concurrent validity, (3) may not account for external validity, (4) may not be inherently logical, (5) susceptibility to subjectivity, (6) complex systems with inadequate instructions, (7) may be biased toward randomized controlled trial (RCT) studies, and (8) may not adequately address the variety of non-RCTs. This narrative review concludes that there is a need to take into account these criticisms and domain-specific limitations, to enable the use and development of the most appropriate grading instruments. Grading systems need to be matched to both the research question being asked and the type of evidence being used.

Nutrition, a health technology that deserves increasing interest among HTA doers. A systematic review

The increasing interest for evaluating indirect consequences of health care interventions and their interaction with patients' behavior have put the focus on health promotion interventions including nutrition and the need to measure and evaluate them.

OBJECTIVE

In this review we have aimed to analyze current status of written and published reports on nutrition and nutrition interventions by HTA doers, how assessment has been approached and which metrics and designs have been proposed.

METHODS

For that purpose, we searched the Center for Reviews and Dissemination databases (CRD) comprising the International Network of Agencies for HTA database (INAHTA), the database of effects (DARE) and the National Health Service Economic Evaluation Database (NHS EED). The words used include nutrition and nutrition interventions and there was no limit on data coverage. We complemented the search by manually seek for further reports on INAHTA's agencies webpages. We extracted the reports for their classification and analysis.

RESULTS

We found 82 reports from different sources and after applying inclusion and exclusion criteria, we finally included 42. All the reports correspond to High income Countries (HiC) including agencies from Europe, North America and Oceania. The agencies or programs most represented correspond to the NIHR (UK) and AHRQ (USA). There were general reports around the role of functional foods and specific reports on the impact of establishing nutrition specific strategies in hospitals. 6 out of 42 analyzed the economic consequences of nutrition interventions and 4 reports were related to the methodologies used or the appliance of systematic review methods to the field of nutrition.

CONCLUSIONS

the reports included correspond to HiC while those HTA agencies established in Low and Middle Income countries (LMiC) have no reported or written activities on the role of nutrition and nutrition interventions. Retrieved reports written by HTA doers/producers confirm the use and utility of systematic reviews and economic analysis methods and its applicability for nutrition interventions. However, some measurements such as Quality Adjusted Life Years (QALY) need to be refined to better reflect the impact of these interventions.

Adding a "GRADE" to the quality appraisal of rheumatoid arthritis guidelines identifies limitations beyond AGREE-II

OBJECTIVES

To assess how well treatment recommendations for rheumatoid arthritis (RA) address Grading of Recommendations Assessment, Development and Evaluation (GRADE) steps and determine whether these steps can be adequately appraised using Appraisal of Guidelines Research & Evaluation II (AGREE-II).

STUDY DESIGN AND SETTING

We systematically reviewed English-language treatment recommendations for the pharmacologic management of RA since 2000, assessed how well GRADE steps were addressed, rated AGREE-II quality, and compared the findings.

RESULTS

GRADE steps were poorly addressed by the 44 included guidelines. Few guidelines discussed study limitations and/or risk of bias (23%), inconsistency (50%), indirectness (39%), imprecision (23%), or potential for publication bias (0%). Observational evidence was cited in 96% but rarely evaluated systematically. Only one guideline considered evidence on patients' preferences for health outcomes, and few provided an explicit justification for the strength of evidence or recommendation. The five GRADE steps that overlapped with AGREE-II questions were addressed more frequently (by 54-100% of guidelines) than the 13 GRADE steps not directly assessed by AGREE-II (0-50%). Among the nine guidelines rated as "Recommended for use" by AGREE-II, 8 of 13 GRADE steps were not addressed consistently by any guideline.

CONCLUSION

GRADE's steps are poorly addressed by RA recommendations. AGREE-II provides a broad assessment of quality but lacks sufficient granularity to assess how well a guideline addresses GRADE's steps.

Development, implementation, and evaluation of a structured reporting web tool for abdominal aortic aneurysms

BACKGROUND

The majority of radiological reports are lacking a standard structure. Even within a specialized area of radiology, each report has its individual structure with regards to details and order, often containing too much of non-relevant information the referring physician is not interested in. For gathering relevant clinical key parameters in an efficient way or to support long-term therapy monitoring, structured reporting might be advantageous.

OBJECTIVE

Despite of new technologies in medical information systems, medical reporting is still not dynamic. To improve the quality of communication in radiology reports, a new structured reporting system was developed for abdominal aortic aneurysms (AAA), intended to enhance professional communication by providing the pertinent clinical information in a predefined standard.

METHODS

Actual state analysis was performed within the departments of radiology and vascular surgery by developing a Technology Acceptance Model. The SWOT (strengths, weaknesses, opportunities, and threats) analysis focused on optimization of the radiology reporting of patients with AAA. Definition of clinical parameters was achieved by interviewing experienced clinicians in radiology and vascular surgery. For evaluation, a focus group (4 radiologists) looked at the reports of 16 patients. The usability and reliability of the method was validated in a real-world test environment in the field of radiology.

RESULTS

A Web-based application for radiological "structured reporting" (SR) was successfully standardized for AAA. Its organization comprises three main categories: characteristics of pathology and adjacent anatomy, measurements, and additional findings. Using different graphical widgets (eg, drop-down menus) in each category facilitate predefined data entries. Measurement parameters shown in a diagram can be defined for clinical monitoring and be adducted for quick adjudications. Figures for optional use to guide and standardize the reporting are embedded. Analysis of variance shows decreased average time required with SR to obtain a radiological report compared to free-text reporting (P=.0001). Questionnaire responses confirm a high acceptance rate by the user.

CONCLUSIONS

The new SR system may support efficient radiological reporting for initial diagnosis and follow-up for AAA. Perceived advantages of our SR platform are ease of use, which may lead to more accurate decision support. The new system is open to communicate not only with clinical partners but also with Radiology Information and Hospital Information Systems.

Workshop Report: concepts and methods in the economics of nutrition--gateways to better economic evaluation of nutrition interventions

Improving health through better nutrition of the population may contribute to enhanced efficiency and sustainability of healthcare systems. A recent expert meeting investigated in detail a number of methodological aspects related to the discipline of nutrition economics. The role of nutrition in health maintenance and in the prevention of non-communicable diseases is now generally recognised. However, the main scope of those seeking to contain healthcare expenditures tends to focus on the management of existing chronic diseases. Identifying additional relevant dimensions to measure and the context of use will become increasingly important in selecting and developing outcome measurements for nutrition interventions. The translation of nutrition-related research data into public health guidance raises the challenging issue of carrying out more pragmatic trials in many areas where these would generate the most useful evidence for health policy decision-making. Nutrition exemplifies all the types of interventions and policy which need evaluating across the health field. There is a need to start actively engaging key stakeholders in order to collect data and to widen health technology assessment approaches for achieving a policy shift from evidence-based medicine to evidence-based decision-making in the field of nutrition.

Using a SWOT analysis to inform healthy eating and physical activity strategies for a remote First Nations community in Canada

PURPOSE

To plan community-driven health promotion strategies based on a strengths, weaknesses, opportunities, and threats (SWOT) analysis of the healthy eating and physical activity patterns of First Nation (FN) youth.

DESIGN

Cross-sectional qualitative and quantitative data used to develop SWOT themes and strategies.

SETTING

Remote, subarctic FN community of Fort Albany, Ontario, Canada.

SUBJECTS

Adult (n  =  25) and youth (n  =  66, grades 6-11) community members.

MEASURES

Qualitative data were collected using five focus groups with adults (two focus groups) and youth (three focus groups), seven individual interviews with adults, and an environmental scan of 13 direct observations of events/locations (e.g., the grocery store). Quantitative data on food/physical activity behaviors were collected using a validated Web-based survey with youth.

ANALYSIS

Themes were identified from qualitative and quantitative data and were analyzed and interpreted within a SWOT matrix.

RESULTS

Thirty-two SWOT themes were identified (e.g., accessibility of existing facilities, such as the gymnasium). The SWOT analysis showed how these themes could be combined and transformed into 12 strategies (e.g., expanding and enhancing the school snack/breakfast program) while integrating suggestions from the community.

CONCLUSION

SWOT analysis was a beneficial tool that facilitated the combination of local data and community ideas in the development of targeted health promotion strategies for the FN community of Fort Albany.

Pre-coverage assessments of new hospital interventions on Austria: methodology and 3 years of experience

OBJECTIVES

A new decision-making process was set up by the Austrian Ministry of Health to regulate coverage of new proposed Extra Medical Services (EMS; German: Medizinische Einzel-Leistung [MEL]) in 2008. As part of the annual decision-making process an independent academic institution (LBI-HTA) is evaluating relevant evidence on these new technologies and provides HTAs, including evidence-based recommendations for decision makers.

METHODS

About ten EMS assessments are performed annually by the LBI-HTA simultaneously between January and March. Each peer-reviewed report consists of a systematic literature review and critical appraisal of evidence using the GRADE methodology. The generation of numerous reports of good quality standards within the short timeframe is achieved by a standardized workflow with predefined assignment of tasks for all participants.

RESULTS

In total, the LBI-HTA performed twenty-five EMS assessments on thirty-three different interventions in the last three years. Coverage was recommended with limitation for eleven (33%) interventions, and not recommended for twenty-two (66%) interventions. The federal health commission decided on acceptance or preliminary acceptance of coverage in seven (22%) cases, rejection in eighteen (55%) cases and changed the status to "subject to approval" in seven (24%) cases.

CONCLUSIONS

Pre-coverage assessment of new hospital interventions was implemented successfully in Austria. It has proved to be a useful tool to support decision makers with objective evidence when deciding whether or not to reimburse medical services.