Which value aspects are relevant for the evaluation of medical devices? Exploring stakeholders' views through a Web-Delphi process

BACKGROUND

Implementation and uptake of health technology assessment for evaluating medical devices require including aspects that different stakeholders consider relevant, beyond cost and effectiveness. However, the involvement of stakeholders in sharing their views still needs to be improved.

OBJECTIVE

This article explores the relevance of distinct value aspects for evaluating different types of medical devices according to stakeholders' views.

METHODS

Thirty-four value aspects collected through literature review and expert validation were the input for a 2-round Web-Delphi process. In the Web-Delphi, a panel of participants from five stakeholders' groups (healthcare professionals, buyers and policymakers, academics, industry, and patients and citizens) judged the relevance of each aspect, by assigning a relevance-level ('Critical', 'Fundamental', 'Complementary', or 'Irrelevant'), for two types of medical devices separately: 'Implantable' and 'In vitro tests based on biomarkers'. Opinions were analysed at the panel and group level, and similarities across devices were identified.

RESULTS

One hundred thirty-four participants completed the process. No aspects were considered 'Irrelevant', neither for the panel nor for stakeholder groups, in both types of devices. The panel considered effectiveness and safety-related aspects 'Critical' (e.g., 'Adverse events for the patient'), and costs-related aspects 'Fundamental' (e.g., 'Cost of the medical device'). Several additional aspects not included in existing frameworks' literature, e.g., related to environmental impact and devices' usage by the healthcare professional, were deemed as relevant by the panel. A moderate to substantial agreement across and within groups was observed.

CONCLUSION

Different stakeholders agree on the relevance of including multiple aspects in medical devices' evaluation. This study produces key information to inform the development of frameworks for valuing medical devices, and to guide evidence collection.

Advancing tools to promote health equity across European Union regions: the EURO-HEALTHY project

BACKGROUND

Population health measurements are recognised as appropriate tools to support public health monitoring. Yet, there is still a lack of tools that offer a basis for policy appraisal and for foreseeing impacts on health equity. In the context of persistent regional inequalities, it is critical to ascertain which regions are performing best, which factors might shape future health outcomes and where there is room for improvement.

METHODS

Under the EURO-HEALTHY project, tools combining the technical elements of multi-criteria value models and the social elements of participatory processes were developed to measure health in multiple dimensions and to inform policies. The flagship tool is the Population Health Index (PHI), a multidimensional measure that evaluates health from the lens of equity in health determinants and health outcomes, further divided into sub-indices. Foresight tools for policy analysis were also developed, namely: (1) scenarios of future patterns of population health in Europe in 2030, combining group elicitation with the Extreme-World method and (2) a multi-criteria evaluation framework informing policy appraisal (case study of Lisbon). Finally, a WebGIS was built to map and communicate the results to wider audiences.

RESULTS

The Population Health Index was applied to all European Union (EU) regions, indicating which regions are lagging behind and where investments are most needed to close the health gap. Three scenarios for 2030 were produced - (1) the 'Failing Europe' scenario (worst case/increasing inequalities), (2) the 'Sustainable Prosperity' scenario (best case/decreasing inequalities) and (3) the 'Being Stuck' scenario (the EU and Member States maintain the status quo). Finally, the policy appraisal exercise conducted in Lisbon illustrates which policies have higher potential to improve health and how their feasibility can change according to different scenarios.

CONCLUSIONS

The article makes a theoretical and practical contribution to the field of population health. Theoretically, it contributes to the conceptualisation of health in a broader sense by advancing a model able to integrate multiple aspects of health, including health outcomes and multisectoral determinants. Empirically, the model and tools are closely tied to what is measurable when using the EU context but offering opportunities to be upscaled to other settings.

Scenarios for population health inequalities in 2030 in Europe: the EURO-HEALTHY project experience

Background

Health inequalities have been consistently reported across and within European countries and continue to pose major challenges to policy-making. The development of scenarios regarding what could affect population health (PH) inequalities across Europe in the future is considered critical. Scenarios can help policy-makers prepare and better cope with fast evolving challenges.

Objective

This paper describes the three 2030 time-horizon scenarios developed under the EURO-HEALTHY project, depicting the key factors that may affect the evolution of PH inequalities across European regions.

Methods

A three-stage socio-technical approach was applied: i) identification of drivers (key factors expected to affect the evolution of PH inequalities across European regions until 2030) – this stage engaged in a Web-Delphi process a multidisciplinary panel of 51 experts and other stakeholders representing the different perspectives regarding PH inequalities; ii) generation of scenario structures – different drivers’ configurations (i.e. their hypotheses for evolution) were organized into coherent scenario structures using the Extreme-World Method; and iii) validation of scenario structures and generation of scenario narratives. Stages ii) and iii) were conducted in two workshops with a strategic group of 13 experts with a wide view about PH inequalities. The scenario narratives were elaborated with the participants’ insights from both the Web-Delphi process and the two workshops, together with the use of evidence (both current and future-oriented) on the different areas within the PH domain.

Results

Three scenarios were developed for the evolution of PH inequalities in Europe until 2030: ‘Failing Europe’ (worst-case but plausible picture of the future), ‘Sustainable Prosperity’ (best-case but plausible picture of the future), and an interim scenario ‘Being Stuck’ depicting a ‘to the best of our knowledge’ evolution. These scenarios show the extent to which a combination of Political, Economic, Social, Technological, Legal and Environmental drivers shape future health inequalities, providing information for European policy-makers to reflect upon whether and how to design robust policy solutions to tackle PH inequalities.

Conclusions

The EURO-HEALTHY scenarios were designed to inform both policy design and appraisal. They broaden the scope, create awareness and generate insights regarding the evolution of PH inequalities across European regions.

Indicators for evaluating European population health: a Delphi selection process

Background

Indicators are essential instruments for monitoring and evaluating population health. The selection of a multidimensional set of indicators should not only reflect the scientific evidence on health outcomes and health determinants, but also the views of health experts and stakeholders. The aim of this study is to describe the Delphi selection process designed to promote agreement on indicators considered relevant to evaluate population health at the European regional level.

Methods

Indicators were selected in a Delphi survey conducted using a web-platform designed to implement and monitor participatory processes. It involved a panel of 51 experts and 30 stakeholders from different areas of knowledge and geographies. In three consecutive rounds the panel indicated their level of agreement or disagreement with indicator’s relevance for evaluating population health in Europe. Inferential statistics were applied to draw conclusions on observed level of agreement (Scott’s Pi interrater reliability coefficient) and opinion change (McNemar Chi-square test). Multivariate analysis of variance was conducted to check if the field of expertise influenced the panellist responses (Wilk’s Lambda test).

Results

The panel participated extensively in the study (overall response rate: 80%). Eighty indicators reached group agreement for selection in the areas of: economic and social environment (12); demographic change (5); lifestyle and health behaviours (8); physical environment (6); built environment (12); healthcare services (11) and health outcomes (26). Higher convergence of group opinion towards agreement on the relevance of indicators was seen for lifestyle and health behaviours, healthcare services, and health outcomes. The panellists’ field of expertise influenced responses: statistically significant differences were found for economic and social environment (p < 0.05 in round 1 and 2), physical environment (p < 0.01 in round 1) and health outcomes (p < 0.01 in round 3).

Conclusions

The high levels of participation observed in this study, by involving experts and stakeholders and ascertaining their views, underpinned the added value of using a transparent Web-Delphi process to promote agreement on what indicators are relevant to appraise population health.

Electronic supplementary material

The online version of this article (10.1186/s12889-018-5463-0) contains supplementary material, which is available to authorized users.