1
|
Gomes M, Turner AJ, Sammon C, Dawoud D, Ramagopalan S, Simpson A, Siebert U. Acceptability of Using Real-World Data to Estimate Relative Treatment Effects in Health Technology Assessments: Barriers and Future Steps. VALUE IN HEALTH : THE JOURNAL OF THE INTERNATIONAL SOCIETY FOR PHARMACOECONOMICS AND OUTCOMES RESEARCH 2024; 27:623-632. [PMID: 38369282 DOI: 10.1016/j.jval.2024.01.020] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/04/2023] [Revised: 01/19/2024] [Accepted: 01/24/2024] [Indexed: 02/20/2024]
Abstract
OBJECTIVES Evidence about the comparative effects of new treatments is typically collected in randomized controlled trials (RCTs). In some instances, RCTs are not possible, or their value is limited by an inability to capture treatment effects over the longer term or in all relevant population subgroups. In these cases, nonrandomized studies (NRS) using real-world data (RWD) are increasingly used to complement trial evidence on treatment effects for health technology assessment (HTA). However, there have been concerns over a lack of acceptability of this evidence by HTA agencies. This article aims to identify the barriers to the acceptance of NRS and steps that may facilitate increases in the acceptability of NRS in the future. METHODS Opinions of the authorship team based on their experience in real-world evidence research in academic, HTA, and industry settings, supported by a critical assessment of existing studies. RESULTS Barriers were identified that are applicable to key stakeholder groups, including HTA agencies (eg, the lack of comprehensive methodological guidelines for using RWD), evidence generators (eg, avoidable deviations from best practices), and external stakeholders (eg, data controllers providing timely access to high-quality RWD). Future steps that may facilitate future acceptability of NRS include improvements in the quality, integration, and accessibility of RWD, wider use of demonstration projects to highlight the value and applicability of nonrandomized designs, living, and more detailed HTA guidelines, and improvements in HTA infrastructure relating to RWD. CONCLUSION NRS can represent a crucial source of evidence on treatment effects for use in HTA when RCT evidence is limited.
Collapse
Affiliation(s)
- Manuel Gomes
- Department of Applied Health Research, University College London, London, England, UK
| | | | | | - Dalia Dawoud
- Science, Policy and Research Programme, National Institute for Health and Care Excellence, London, England, UK; Faculty of Pharmacy, Cairo University, Cairo, Egypt
| | | | - Alex Simpson
- Global Access, F. Hoffmann-La Roche Ltd, Grenzacherstrasse, Basel, Switzerland.
| | - Uwe Siebert
- Department of Public Health, Health Services Research and Health Technology Assessment, UMIT TIROL - University for Health Sciences and Technology, Hall in Tirol, Austria; Center for Health Decision Science and Department of Health Policy and Management, Harvard T.H Chan School of Public Health, Boston, MA, USA; Institute for Technology Assessment and Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| |
Collapse
|
2
|
Turner AJ, Sammon C, Latimer N, Adamson B, Beal B, Subbiah V, Abrams KR, Ray J. Transporting Comparative Effectiveness Evidence Between Countries: Considerations for Health Technology Assessments. PHARMACOECONOMICS 2024; 42:165-176. [PMID: 37891433 PMCID: PMC10811184 DOI: 10.1007/s40273-023-01323-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 10/03/2023] [Indexed: 10/29/2023]
Abstract
Internal validity is often the primary concern for health technology assessment agencies when assessing comparative effectiveness evidence. However, the increasing use of real-world data from countries other than a health technology assessment agency's target population in effectiveness research has increased concerns over the external validity, or "transportability", of this evidence, and has led to a preference for local data. Methods have been developed to enable a lack of transportability to be addressed, for example by accounting for cross-country differences in disease characteristics, but their consideration in health technology assessments is limited. This may be because of limited knowledge of the methods and/or uncertainties in how best to utilise them within existing health technology assessment frameworks. This article aims to provide an introduction to transportability, including a summary of its assumptions and the methods available for identifying and adjusting for a lack of transportability, before discussing important considerations relating to their use in health technology assessment settings, including guidance on the identification of effect modifiers, guidance on the choice of target population, estimand, study sample and methods, and how evaluations of transportability can be integrated into health technology assessment submission and decision processes.
Collapse
Affiliation(s)
| | | | - Nick Latimer
- School of Health and Related Research, University of Sheffield, Sheffield, UK
- Delta Hat, Nottingham, UK
| | | | | | | | - Keith R Abrams
- Department of Statistics, University of Warwick, Coventry, UK
- Centre for Health Economics, University of York, York, UK
| | - Joshua Ray
- Global Access, F. Hoffmann-La Roche Ltd, Grenzacherstrasse 124, 4070 Basel, Switzerland.
| |
Collapse
|
3
|
Leahy TP, Durand-Zaleski I, Sampietro-Colom L, Kent S, Zöllner Y, Coyle D, Casadei G. The role of quantitative bias analysis for nonrandomized comparisons in health technology assessment: recommendations from an expert workshop. Int J Technol Assess Health Care 2023; 39:e68. [PMID: 37981828 DOI: 10.1017/s0266462323002702] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2023]
Abstract
The use of treatment effects derived from nonrandomized studies (NRS) in health technology assessment (HTA) is growing. NRS carry an inherently greater risk of bias than randomized controlled trials (RCTs). Although bias can be mitigated to some extent through appropriate approaches to study design and analysis, concerns around data availability and quality and the absence of randomization mean residual biases typically render the interpretation of NRS challenging. Quantitative bias analysis (QBA) methods are a range of methods that use additional, typically external, data to understand the potential impact that unmeasured confounding and other biases including selection bias and time biases can have on the results (i.e., treatment effects) from an NRS. QBA has the potential to support HTA bodies in using NRS to support decision-making by quantifying the magnitude, direction, and uncertainty of biases. However, there are a number of key aspects of the use of QBA in HTA which have received limited discussion. This paper presents recommendations for the use of QBA in HTA developed using a multi-stakeholder workshop of experts in HTA with a focus on QBA for unmeasured confounding.
Collapse
Affiliation(s)
| | - Isabelle Durand-Zaleski
- AP-HP, Health Economics Research Unit, Department of Public Health, Henri Mondor Hospital, Paris, France
- Methods, UMRS 1153, French National Institute of Health and Medical Research, Paris, France
- Faculty of Medicine, Université Paris Est Creteil, Creteil, France
| | - Laura Sampietro-Colom
- Health Technology Assessment (HTA) Unit, Hospital Clinic of Barcelona, Barcelona, Spain
| | | | - York Zöllner
- Department of Health Sciences, HAW Hamburg, Hamburg, Germany
| | - Doug Coyle
- School of Epidemiology and Public Health, University of Ottawa, Ottawa, ON, Canada
| | - Gianluigi Casadei
- Medical Biotechnology and Translational Medicine, University of Milan, Milan, Italy
| |
Collapse
|
4
|
Simpson A, Ramagopalan SV. R WE ready for reimbursement? A round up of developments in real-world evidence relating to health technology assessment: part 9. J Comp Eff Res 2022; 11:1147-1149. [PMID: 35998008 DOI: 10.2217/cer-2022-0145] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022] Open
Abstract
In this latest update we highlight a recent International Society of Pharmacoeconomics and Outcomes Research Good Practice Report on machine learning (ML) for health economics and outcomes research. We specifically discuss use cases of ML that offer opportunities in the generation of evidence using real-world data, including improvements in the identification of study cohorts, confounder identification and adjustment and estimating treatment effect heterogeneity. Barriers to the wider adoption of ML methods are also discussed.
Collapse
Affiliation(s)
- Alex Simpson
- Global Access, F Hoffmann-La Roche, Basel, Switzerland
| | | |
Collapse
|
5
|
Leahy TP, Duffield S, Kent S, Sammon C, Tzelis D, Ray J, Groenwold RH, Gomes M, Ramagopalan S, Grieve R. Application of quantitative bias analysis for unmeasured confounding in cost-effectiveness modelling. J Comp Eff Res 2022; 11:861-870. [PMID: 35678168 DOI: 10.2217/cer-2022-0030] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/20/2022] Open
Abstract
Due to uncertainty regarding the potential impact of unmeasured confounding, health technology assessment (HTA) agencies often disregard evidence from nonrandomized studies when considering new technologies. Quantitative bias analysis (QBA) methods provide a means to quantify this uncertainty but have not been widely used in the HTA setting, particularly in the context of cost-effectiveness modelling (CEM). This study demonstrated the application of an aggregate and patient-level QBA approach to quantify and adjust for unmeasured confounding in a simulated nonrandomized comparison of survival outcomes. Application of the QBA output within a CEM through deterministic and probabilistic sensitivity analyses and under different scenarios of knowledge of an unmeasured confounder demonstrates the potential value of QBA in HTA.
Collapse
Affiliation(s)
| | - Stephen Duffield
- National Institute for Health & Care Excellence, Manchester, M1 4BT, UK
| | - Seamus Kent
- National Institute for Health & Care Excellence, Manchester, M1 4BT, UK
| | | | | | - Joshua Ray
- Global Access, F. Hoffmann-La Roche, Grenzacherstrasse 124, CH-4070, Basel, Switzerland
| | - Rolf Hh Groenwold
- Leiden University Medical Centre, Department of Clinical Epidemiology & Department of Biomedical Data Sciences, Einthovenweg 20, 2333, ZC Leiden, The Netherlands
| | | | - Sreeram Ramagopalan
- Global Access, F. Hoffmann-La Roche, Grenzacherstrasse 124, CH-4070, Basel, Switzerland
| | - Richard Grieve
- London School of Hygiene & Tropical Medicine, London, WC1H 9SH, UK
| |
Collapse
|