1
|
DiStefano MJ, Pearson SD, Rind DM, Zemplenyi A. How Do the Institute for Clinical and Economic Review's Assessments of Comparative Effectiveness Compare With the German Federal Joint Committee's Assessments of Added Benefit? A Qualitative Study. VALUE IN HEALTH : THE JOURNAL OF THE INTERNATIONAL SOCIETY FOR PHARMACOECONOMICS AND OUTCOMES RESEARCH 2024; 27:1066-1072. [PMID: 38679288 DOI: 10.1016/j.jval.2024.04.015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/13/2023] [Revised: 04/08/2024] [Accepted: 04/11/2024] [Indexed: 05/01/2024]
Abstract
OBJECTIVES We compared the Institute for Clinical and Economic Review's (ICER) ratings of comparative clinical effectiveness with the German Federal Joint Committee's (G-BA) added benefit ratings, and explored what factors may explain the disagreement between the 2 organizations. METHODS We included drugs if they were assessed by ICER under its 2020 to 2023 Value Assessment Framework and had a corresponding assessment by G-BA as of January 2024 for the same indication, patient population, and comparator drug. To compare assessments, we modified ICER's proposed crosswalk between G-BA and ICER benefit ratings to account for G-BA's certainty ratings. We also determined whether each pair was based on similar evidence. Assessment pairs exhibiting disagreement based on the modified crosswalk despite a similar evidence base were qualitatively analyzed to identify reasons for disagreement. RESULTS Out of 15 drug assessment pairs matched on indication, patient subgroup, and comparator, none showed agreement in their assessments when based on similar evidence. Disagreement was attributed to differences in evidence evaluation, including evaluations of safety, generalizability, and study design, as well as G-BA's rejection of the available evidence in 4 cases as unsuitable. CONCLUSIONS The findings demonstrate that even under conditions where populations and comparators are identical and the evidence base is consistent, different assessors may arrive at divergent conclusions about comparative effectiveness, thus underscoring the presence of value judgments within assessments of clinical effectiveness. To support initiatives that seek to facilitate the exchange of value assessments between countries, these value judgments should always be transparently presented and justified in assessment summaries.
Collapse
Affiliation(s)
- Michael J DiStefano
- Center for Pharmaceutical Outcomes Research, Department of Clinical Pharmacy, Skaggs School of Pharmacy and Pharmaceutical Sciences, University of Colorado Anschutz Medical Campus, Aurora, CO, USA.
| | | | - David M Rind
- Institute for Clinical and Economic Review, Boston, MA, USA
| | - Antal Zemplenyi
- Center for Pharmaceutical Outcomes Research, Department of Clinical Pharmacy, Skaggs School of Pharmacy and Pharmaceutical Sciences, University of Colorado Anschutz Medical Campus, Aurora, CO, USA
| |
Collapse
|
2
|
Cardoso MMDA, Machado-Rugolo J, Thabane L, da Rocha NC, Barbosa AMP, Komoda DS, de Almeida JTC, Curado DDSP, Weber SAT, de Andrade LGM. Application of natural language processing to predict final recommendation of Brazilian health technology assessment reports. Int J Technol Assess Health Care 2024; 40:e19. [PMID: 38605654 DOI: 10.1017/s0266462324000163] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/13/2024]
Abstract
INTRODUCTION Health technology assessment (HTA) plays a vital role in healthcare decision-making globally, necessitating the identification of key factors impacting evaluation outcomes due to the significant workload faced by HTA agencies. OBJECTIVES The aim of this study was to predict the approval status of evaluations conducted by the Brazilian Committee for Health Technology Incorporation (CONITEC) using natural language processing (NLP). METHODS Data encompassing CONITEC's official report summaries from 2012 to 2022. Textual data was tokenized for NLP analysis. Least Absolute Shrinkage and Selection Operator, logistic regression, support vector machine, random forest, neural network, and extreme gradient boosting (XGBoost), were evaluated for accuracy, area under the receiver operating characteristic curve (ROC AUC) score, precision, and recall. Cluster analysis using the k-modes algorithm categorized entries into two clusters (approved, rejected). RESULTS The neural network model exhibited the highest accuracy metrics (precision at 0.815, accuracy at 0.769, ROC AUC at 0.871, and recall at 0.746), followed by XGBoost model. The lexical analysis uncovered linguistic markers, like references to international HTA agencies' experiences and government as demandant, potentially influencing CONITEC's decisions. Cluster and XGBoost analyses emphasized that approved evaluations mainly concerned drug assessments, often government-initiated, while non-approved ones frequently evaluated drugs, with the industry as the requester. CONCLUSIONS NLP model can predict health technology incorporation outcomes, opening avenues for future research using HTA reports from other agencies. This model has the potential to enhance HTA system efficiency by offering initial insights and decision-making criteria, thereby benefiting healthcare experts.
Collapse
Affiliation(s)
- Marilia Mastrocolla de Almeida Cardoso
- Health Technology Assessment Unit, Hospital das Clínicas da Faculdade de Medicina de Botucatu, Botucatu, Brazil
- Laboratory of Data Science and Predictive Analysis in Health, Hospital das Clínicas da Faculdade de Medicina de Botucatu, Botucatu, Brazil
| | - Juliana Machado-Rugolo
- Health Technology Assessment Unit, Hospital das Clínicas da Faculdade de Medicina de Botucatu, Botucatu, Brazil
- Laboratory of Data Science and Predictive Analysis in Health, Hospital das Clínicas da Faculdade de Medicina de Botucatu, Botucatu, Brazil
| | - Lehana Thabane
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada
- Biostatistics Unit, St Joseph's Healthcare Hamilton, Hamilton, ON, Canada
- Faculty of Health Sciences, University of Johannesburg, Johannesburg, South Africa
| | - Naila Camila da Rocha
- Laboratory of Data Science and Predictive Analysis in Health, Hospital das Clínicas da Faculdade de Medicina de Botucatu, Botucatu, Brazil
| | - Abner Mácula Pacheco Barbosa
- Laboratory of Data Science and Predictive Analysis in Health, Hospital das Clínicas da Faculdade de Medicina de Botucatu, Botucatu, Brazil
- Department of Ophthalmology, Otorhinolaryngology and Head and Neck Surgery, Medical School (FMB) of São Paulo State University, Botucatu, Brazil
| | | | - Juliana Tereza Coneglian de Almeida
- Health Technology Assessment Unit, Hospital das Clínicas da Faculdade de Medicina de Botucatu, Botucatu, Brazil
- Laboratory of Data Science and Predictive Analysis in Health, Hospital das Clínicas da Faculdade de Medicina de Botucatu, Botucatu, Brazil
| | - Daniel da Silva Pereira Curado
- Department of Management and Incorporation of Health Technologies, Ministry of Health, Brasilia, Distrito Federal, Brazil
| | - Silke Anna Theresa Weber
- Health Technology Assessment Unit, Hospital das Clínicas da Faculdade de Medicina de Botucatu, Botucatu, Brazil
- Department of Ophthalmology, Otorhinolaryngology and Head and Neck Surgery, Medical School (FMB) of São Paulo State University, Botucatu, Brazil
| | - Luis Gustavo Modelli de Andrade
- Laboratory of Data Science and Predictive Analysis in Health, Hospital das Clínicas da Faculdade de Medicina de Botucatu, Botucatu, Brazil
- Department of Internal Medicine, Medical School (FMB) of São Paulo State University, Botucatu, Brazil
| |
Collapse
|
3
|
Bolous NS, Chokwenda-Makore N, Bonilla M, Chingo G, Kambugu J, Mulindwa JM, Noleb M, Chitsike I, Bhakta N. Addressing the gap in health economics data to support national cancer control plans in low- and middle-income countries: The Childhood Cancers Budgeting Rapidly to Incorporate Disadvantaged Groups for Equity (CC-BRIDGE) tool. Cancer 2024; 130:1112-1124. [PMID: 38100617 DOI: 10.1002/cncr.35146] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2023] [Revised: 11/09/2023] [Accepted: 11/14/2023] [Indexed: 12/17/2023]
Abstract
BACKGROUND National cancer control plans (NCCPs) are complex public health programs that incorporate evidence-based cancer control strategies to improve health outcomes for all individuals in a country. Given the scope of NCCPs, small and vulnerable populations, such as patients with childhood cancer, are often missed. To support planning efforts, a rapid, modifiable tool was developed that estimates a context-specific national budget to fund pediatric cancer programs, provides 5-year scale-up scenarios, and calculates annual cost-effectiveness. METHODS The tool was codeveloped by teams of policymakers, clinicians, and public health advocates in Zimbabwe, Zambia, and Uganda. The 11 costing categories included real-world data, modeled data, and data from the literature. A base-case and three 5-year scale-up scenarios were created using modifiable inputs. The cost-effectiveness of the disability-adjusted life years averted was calculated. Results were compared with each country's projected gross domestic product per capita for 2022 through 2026. RESULTS The number of patients/total budget for year 1 was 250/$1,109,366 for Zimbabwe, 280/$1,207,555 for Zambia, and 1000/$2,277,397 for Uganda. In year 5, these values were assumed to increase to 398/$5,545,445, 446/$4,926,150, and 1594/$9,059,331, respectively. Base-case cost per disability-adjusted life year averted/ratio to gross domestic product per capita for year 1, assuming 20% survival, was: $807/0.5 for Zimbabwe, $785/0.7 for Zambia, and $420/0.5 for Uganda. CONCLUSIONS This costing tool provided a framework to forecast a budget for childhood-specific cancer services. By leveraging minimal primary data collection with existing secondary data, local teams obtained rapid results, ensuring that childhood cancer budgeting is not neglected once in every 5 to 6 years of planning processes.
Collapse
Affiliation(s)
- Nancy S Bolous
- Department of Global Pediatric Medicine, St. Jude Children's Research Hospital, Memphis, Tennessee, USA
| | - Nester Chokwenda-Makore
- Department of Paediatrics and Child Health, Faculty of Medicine and Health Sciences, University of Zimbabwe, Harare, Zimbabwe
| | - Miguel Bonilla
- Department of Global Pediatric Medicine, St. Jude Children's Research Hospital, Memphis, Tennessee, USA
| | - Grace Chingo
- Department of Pediatric Oncology, Cancer Disease Hospital, Lusaka, Zambia
| | - Joyce Kambugu
- Department of Pediatric Oncology, Uganda Cancer Institute, Kampala, Uganda
| | - Justin M Mulindwa
- Department of Pediatric Oncology, Cancer Disease Hospital, Lusaka, Zambia
| | - Mugisha Noleb
- Department of Pediatric Oncology, Uganda Cancer Institute, Kampala, Uganda
| | - Inam Chitsike
- Department of Paediatrics and Child Health, Faculty of Medicine and Health Sciences, University of Zimbabwe, Harare, Zimbabwe
| | - Nickhill Bhakta
- Department of Global Pediatric Medicine, St. Jude Children's Research Hospital, Memphis, Tennessee, USA
| |
Collapse
|
4
|
Di Bidino R, Piaggio D, Andellini M, Merino-Barbancho B, Lopez-Perez L, Zhu T, Raza Z, Ni M, Morrison A, Borsci S, Fico G, Pecchia L, Iadanza E. Scoping Meta-Review of Methods Used to Assess Artificial Intelligence-Based Medical Devices for Heart Failure. Bioengineering (Basel) 2023; 10:1109. [PMID: 37892839 PMCID: PMC10604154 DOI: 10.3390/bioengineering10101109] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2023] [Revised: 09/13/2023] [Accepted: 09/17/2023] [Indexed: 10/29/2023] Open
Abstract
Artificial intelligence and machine learning (AI/ML) are playing increasingly important roles, permeating the field of medical devices (MDs). This rapid progress has not yet been matched by the Health Technology Assessment (HTA) process, which still needs to define a common methodology for assessing AI/ML-based MDs. To collect existing evidence from the literature about the methods used to assess AI-based MDs, with a specific focus on those used for the management of heart failure (HF), the International Federation of Medical and Biological Engineering (IFMBE) conducted a scoping meta-review. This manuscript presents the results of this search, which covered the period from January 1974 to October 2022. After careful independent screening, 21 reviews, mainly conducted in North America and Europe, were retained and included. Among the findings were that deep learning is the most commonly utilised method and that electronic health records and registries are among the most prevalent sources of data for AI/ML algorithms. Out of the 21 included reviews, 19 focused on risk prediction and/or the early diagnosis of HF. Furthermore, 10 reviews provided evidence of the impact on the incidence/progression of HF, and 13 on the length of stay. From an HTA perspective, the main areas requiring improvement are the quality assessment of studies on AI/ML (included in 11 out of 21 reviews) and their data sources, as well as the definition of the criteria used to assess the selection of the most appropriate AI/ML algorithm.
Collapse
Affiliation(s)
- Rossella Di Bidino
- Fondazione Policlinico Universitario Agostino Gemelli IRCCS—The Graduate School of Health Economics and Management (ALTEMS), 00168 Rome, Italy
| | - Davide Piaggio
- School of Engineering, University of Warwick, Coventry CV4 7AL, UK; (D.P.); (M.A.); (Z.R.); (L.P.)
| | - Martina Andellini
- School of Engineering, University of Warwick, Coventry CV4 7AL, UK; (D.P.); (M.A.); (Z.R.); (L.P.)
| | - Beatriz Merino-Barbancho
- Life Supporting Technologies, Photonics Technology and Bioengineering Department, School of Telecommunication Engineering, Universidad Politécnica de Madrid, 28040 Madrid, Spain (L.L.-P.); (G.F.)
| | - Laura Lopez-Perez
- Life Supporting Technologies, Photonics Technology and Bioengineering Department, School of Telecommunication Engineering, Universidad Politécnica de Madrid, 28040 Madrid, Spain (L.L.-P.); (G.F.)
| | - Tianhui Zhu
- NIHR London In-Vitro Diagnostics Cooperative, Imperial College of London, London W2 1NY, UK
| | - Zeeshan Raza
- School of Engineering, University of Warwick, Coventry CV4 7AL, UK; (D.P.); (M.A.); (Z.R.); (L.P.)
| | - Melody Ni
- NIHR London In-Vitro Diagnostics Cooperative, Imperial College of London, London W2 1NY, UK
| | - Andra Morrison
- Canadian Agency for Drugs and Technologies in Health, Ottawa, ON K1S 5S8, Canada;
| | - Simone Borsci
- NIHR London In-Vitro Diagnostics Cooperative, Imperial College of London, London W2 1NY, UK
- Department of Learning, Data Analysis, and Technology, Cognition, Data and Education (CODE) Group, Faculty of Behavioural Management and Social Sciences, University of Twente, 7522 Enschede, The Netherlands
| | - Giuseppe Fico
- Life Supporting Technologies, Photonics Technology and Bioengineering Department, School of Telecommunication Engineering, Universidad Politécnica de Madrid, 28040 Madrid, Spain (L.L.-P.); (G.F.)
| | - Leandro Pecchia
- School of Engineering, University of Warwick, Coventry CV4 7AL, UK; (D.P.); (M.A.); (Z.R.); (L.P.)
- School of Engineering, University Campus Bio-Medico, 00128 Rome, Italy
- International Federation of Medical and Biological Engineering, B-1090 Brussels, Belgium
| | - Ernesto Iadanza
- International Federation of Medical and Biological Engineering, B-1090 Brussels, Belgium
- Department of Medical Biotechnologies, University of Siena, 53100 Siena, Italy
| |
Collapse
|
5
|
Hollingworth SA, Leaupepe GA, Nonvignon J, Fenny AP, Odame EA, Ruiz F. Economic evaluations of non-communicable diseases conducted in Sub-Saharan Africa: a critical review of data sources. COST EFFECTIVENESS AND RESOURCE ALLOCATION 2023; 21:57. [PMID: 37641087 PMCID: PMC10463745 DOI: 10.1186/s12962-023-00471-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2023] [Accepted: 08/23/2023] [Indexed: 08/31/2023] Open
Abstract
BACKGROUND Policymakers in sub-Saharan Africa (SSA) face challenging decisions regarding the allocation of health resources. Economic evaluations can help decision makers to determine which health interventions should be funded and or included in their benefits package. A major problem is whether the evaluations incorporated data from sources that are reliable and relevant to the country of interest. We aimed to review the quality of the data sources used in all published economic evaluations for cardiovascular disease and diabetes in SSA. METHODS We systematically searched selected databases for all published economic evaluations for CVD and diabetes in SSA. We modified a hierarchy of data sources and used a reference case to measure the adherence to reporting and methodological characteristics, and descriptively analysed author statements. RESULTS From 7,297 articles retrieved from the search, we selected 35 for study inclusion. Most were modelled evaluations and almost all focused on pharmacological interventions. The studies adhered to the reporting standards but were less adherent to the methodological standards. The quality of data sources varied. The quality level of evidence in the data domains of resource use and costs were generally considered of high quality, with studies often sourcing information from reliable databases within the same jurisdiction. The authors of most studies referred to data sources in the discussion section of the publications highlighting the challenges of obtaining good quality and locally relevant data. CONCLUSIONS The data sources in some domains are considered high quality but there remains a need to make substantial improvements in the methodological adherence and overall quality of data sources to provide evidence that is sufficiently robust to support decision making in SSA within the context of UHC and health benefits plans. Many SSA governments will need to strengthen and build their capacity to conduct economic evaluations of interventions and health technology assessment for improved priority setting. This capacity building includes enhancing local infrastructures for routine data production and management. If many of the policy makers are using economic evaluations to guide resource allocation, it is imperative that the evidence used is of the feasibly highest quality.
Collapse
Affiliation(s)
| | | | | | - Ama Pokuaa Fenny
- Institute of Social, Statistical and Economic Research, University of Ghana, Accra, Ghana
| | - Emmanuel A Odame
- Dept of Medical Affairs, Korle Bu Teaching Hospital, Accra, Ghana
| | - Francis Ruiz
- Department of Global Health and Development, London School of Hygiene and Tropical Medicine, London, UK
| |
Collapse
|