Modeling using discrete event simulation: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force--4

A Systematic Literature Review of Modelling Approaches to Evaluate the Cost Effectiveness of PET/CT for Therapy Response Monitoring in Oncology

BACKGROUND AND OBJECTIVE

This systematic literature review addresses model-based cost-effectiveness studies for therapy response monitoring with positron emission tomography (PET) generally combined with low-dose computed tomography (CT) for various cancer types. Given the known heterogeneity in therapy response events, studies should consider patient-level modelling rather than cohort-based modelling because of its flexibility in handling these events and the time to events. This review aims to identify the modelling methods used and includes a systematic assessment of the assumptions made in the current literature.

METHODS

This study was conducted and reported following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 statement. Information sources included electronic bibliographic databases, reference lists of review articles and contact with experts in the fields of nuclear medicine, health technology assessment and health economics. Eligibility criteria included peer-reviewed scientific publications and published grey literature. Literature searches, screening and critical appraisal were conducted by two reviewers independently. The Consolidated Health Economic Evaluation Reporting Standards (CHEERS) were used to assess the methodological quality. The Bias in Economic Evaluation (ECOBIAS) checklist was used to determine the risk of bias in the included publications.

RESULTS

The search results included 2959 publications. The number of publications included for data extraction and synthesis was ten, representing eight unique studies. These studies addressed patients with lymphoma, advanced head and neck cancers, brain tumours, non-small cell lung cancer and cervical cancer. All studies addressed response to chemotherapy. No study evaluated response to immunotherapy. Most studies positioned PET/CT as an add-on modality and one study positioned PET/CT as a replacement for conventional imaging (X-ray and contrast-enhanced CT). Three studies reported decision-tree structures, four studies reported cohort-level state-transition models and one study reported a partitioned survival model. No patient-level models were reported. The simulation horizons adopted ranged from 1 year to lifetime. Most studies reported a probabilistic analysis, whereas two studies reported a deterministic analysis only. Two studies conducted a value of information analysis. Multiple studies did not adequately discuss model-specific aspects of bias. Most importantly and regularly observed were a high risk of structural assumptions bias, limited simulation horizon bias and wrong model bias.

CONCLUSIONS

Model-based cost-effectiveness analysis for therapy response monitoring with PET/CT was based on cohorts of patients instead of individual patients in the current literature. Therefore, the heterogeneity in therapy response events was commonly not addressed appropriately. Further research should include more advanced and patient-level modelling approaches to accurately represent the complex context of clinical practice and, therefore, to be meaningful to support decision making.

REGISTRATION

This review is registered in PROSPERO, the international prospective register of systematic reviews funded by the National Institute for Health Research, with CRD42023402581.

In Silico Evaluation of Algorithm-Based Clinical Decision Support Systems: Protocol for a Scoping Review

BACKGROUND

Integrating algorithm-based clinical decision support (CDS) systems poses significant challenges in evaluating their actual clinical value. Such CDS systems are traditionally assessed via controlled but resource-intensive clinical trials.

OBJECTIVE

This paper presents a review protocol for preimplementation in silico evaluation methods to enable broadened impact analysis under simulated environments before clinical trials.

METHODS

We propose a scoping review protocol that follows an enhanced Arksey and O'Malley framework and PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) guidelines to investigate the scope and research gaps in the in silico evaluation of algorithm-based CDS models-specifically CDS decision-making end points and objectives, evaluation metrics used, and simulation paradigms used to assess potential impacts. The databases searched are PubMed, Embase, CINAHL, PsycINFO, Cochrane, IEEEXplore, Web of Science, and arXiv. A 2-stage screening process identified pertinent articles. The information extracted from articles was iteratively refined. The review will use thematic, trend, and descriptive analyses to meet scoping aims.

RESULTS

We conducted an automated search of the databases above in May 2023, with most title and abstract screenings completed by November 2023 and full-text screening extended from December 2023 to May 2024. Concurrent charting and full-text analysis were carried out, with the final analysis and manuscript preparation set for completion in July 2024. Publication of the review results is targeted from July 2024 to February 2025. As of April 2024, a total of 21 articles have been selected following a 2-stage screening process; these will proceed to data extraction and analysis.

CONCLUSIONS

We refined our data extraction strategy through a collaborative, multidisciplinary approach, planning to analyze results using thematic analyses to identify approaches to in silico evaluation. Anticipated findings aim to contribute to developing a unified in silico evaluation framework adaptable to various clinical workflows, detailing clinical decision-making characteristics, impact measures, and reusability of methods. The study's findings will be published and presented in forums combining artificial intelligence and machine learning, clinical decision-making, and health technology impact analysis. Ultimately, we aim to bridge the development-deployment gap through in silico evaluation-based potential impact assessments.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/63875.

Different Models, Same Results: Considerations When Choosing Between Approaches to Model Cost Effectiveness of Chimeric-Antigen Receptor T-Cell Therapy Versus Standard of Care

OBJECTIVE

Chimeric antigen-receptor T-cell therapy (CAR-T) is characterised by early phase data at the time of registration, high upfront cost and a complex manufacturing and administration process compared with standard therapies. Our objective was to compare the performance of different models to assess the cost effectiveness of CAR-T using a state-transition model (STM), partitioned survival model (PSM) and discrete event simulation (DES).

METHODS

Individual data for tisagenlecleucel for the treatment of young patients with acute lymphoblastic leukaemia (ALL) were used to populate the models. Costs and benefits were measured over a lifetime to generate a cost per quality-adjusted life-year (QALY). Model performance was compared quantitatively on the outcomes generated and a checklist developed summarising the components captured by each model type relevant to assessing cost effectiveness of CAR-T.

RESULTS

Models generated similar results with base-case analyses ranging from an incremental cost per QALY of $96,074-$99,625. DES was the only model to specifically capture CAR-T wait time, demonstrating a substantial loss of benefit of CAR-T with increased wait time.

CONCLUSION

Although model type did not meaningfully impact base-case results, the ability to incorporate an outcome-based payment arrangement (OBA) and wait time are important elements to consider when selecting a model for CAR-T. DES provided greater flexibility compared with STM and PSM approaches to deal with the complex manufacturing and administration process that can lead to extended wait times and substantially reduce the benefit of CAR-T. This is an important consideration when selecting a model type for CAR-T, so major drivers of uncertainty are considered in funding decisions.

Next-generation sequencing in oncology: challenges in economic evaluations

INTRODUCTION

Next-generation sequencing (NGS) identifies genetic variants to inform personalized treatment plans. Insufficient evidence of cost-effectiveness impedes the integration of NGS into routine cancer care. The complexity of personalized treatment challenges conventional economic evaluation. Clearly delineating challenges informs future cost-effectiveness analyses to better value and contextualize health, preference-, and equity-based outcomes.

AREAS COVERED

We conducted a scoping review to characterize the applied methods and outcomes of economic evaluations of NGS in oncology and identify existing challenges. We included 27 articles published since 2016 from a search of PubMed, Embase, and Web of Science. Identified challenges included defining the evaluative scope, managing evidentiary limitations including lack of causal evidence, incorporating preference-based utility, and assessing distributional and equity-based impacts. These challenges reflect the difficulty of generating high-quality clinical effectiveness and real-world evidence (RWE) for NGS-guided interventions.

EXPERT OPINION

Adapting methodological approaches and developing life-cycle health technology assessment (HTA) guidance using RWE is crucial for implementing NGS in oncology. Healthcare systems, decision-makers, and HTA organizations are facing a pivotal opportunity to adapt to an evolving clinical paradigm and create innovative regulatory and reimbursement processes that will enable more sustainable, equitable, and patient-oriented healthcare.

The PHEM-B toolbox of methods for incorporating the influences on Behaviour into Public Health Economic Models

BACKGROUND

It is challenging to predict long-term outcomes of interventions without understanding how they work. Health economic models of public health interventions often do not incorporate the many determinants of individual and population behaviours that influence long term effectiveness. The aim of this paper is to draw on psychology, sociology, behavioural economics, complexity science and health economics to: (a) develop a toolbox of methods for incorporating the influences on behaviour into public health economic models (PHEM-B); and (b) set out a research agenda for health economic modellers and behavioural/ social scientists to further advance methods to better inform public health policy decisions.

METHODS

A core multidisciplinary group developed a preliminary toolbox from a published review of the literature and tested this conceptually using a case study of a diabetes prevention simulation. The core group was augmented by a much wider group that covered a broader range of multidisciplinary expertise. We used a consensus method to gain agreement of the PHEM-B toolbox. This included a one-day workshop and subsequent reviews of the toolbox.

RESULTS

The PHEM-B toolbox sets out 12 methods which can be used in different combinations to incorporate influences on behaviours into public health economic models: collaborations between modellers and behavioural scientists, literature reviewing, application of the Behaviour Change Intervention Ontology, systems mapping, agent-based modelling, differential equation modelling, social network analysis, geographical information systems, discrete event simulation, theory-informed statistical and econometric analyses, expert elicitation, and qualitative research/process tracing. For each method, we provide a description with key references, an expert consensus on the circumstances when they could be used, and the resources required.

CONCLUSIONS

This is the first attempt to rigorously and coherently propose methods to incorporate the influences on behaviour into health economic models of public health interventions. It may not always be feasible or necessary to model the influences on behaviour explicitly, but it is essential to develop an understanding of the key influences. Changing behaviour and maintaining that behaviour change could have different influences; thus, there could be benefits in modelling these separately. Future research is needed to develop, collaboratively with behavioural scientists, a suite of more robust health economic models of health-related behaviours, reported transparently, including coding, which would allow model reuse and adaptation.

Cost-Effectiveness of 9-Valent HPV Vaccination for Patients Treated for High-Grade Cervical Intraepithelial Neoplasia in the UK

Importance

Patients who have been treated for high-grade cervical intraepithelial neoplasia (CIN grade ≥2) are at a high risk for subsequent CIN and other cancers and diseases related to human papillomavirus (HPV). HPV vaccination can reduce the risk of subsequent disease in patients surgically treated for grade 2 or greater CIN; however, there is no formal recommendation for prophylactic HPV vaccination in this high-risk population, and the cost-effectiveness is unknown.

Objective

To assess the incremental lifetime outcomes, costs, and cost-effectiveness of integrating peritreatment 9-valent HPV (9vHPV) vaccination in combination with posttreatment surveillance for the prevention of cervical cancer and other HPV-attributable diseases in patients surgically treated for grade 2 or greater CIN vs posttreatment surveillance alone from a UK payer perspective.

Design, Setting, and Participants

This economic evaluation used 3 independent Markov model structures. Model inputs for vaccine efficacy, utilities, and costs were obtained from published sources, and cervical cancer screening data were obtained from the National Health Service Cervical Screening Program. Costs were adjusted to 2022 to 2023 reference years. Data were analyzed from October 2022 to September 2023.

Exposure

Peritreatment vaccination with 9vHPV in combination with posttreatment surveillance compared with posttreatment surveillance alone.

Main Outcomes and Measures

Clinical outcomes included grade 1, 2, or 3 CIN; cervical cancer; vaginal cancer; vulvar cancer; anal cancer; head and neck cancer; genital warts; and recurrent respiratory papillomatosis. Incremental cost-effectiveness ratios (ICERs) using a willingness-to-pay threshold (WTP) of £20 000 (US $26 200) per quality-adjusted life-year (QALY) were estimated. Deterministic sensitivity analysis and probabilistic sensitivity analysis were performed.

Results

Vaccination with 9vHPV in conjunction with posttreatment surveillance was cost-effective, with a favorable ICER of £13 789.07 (US $18 064.68) per QALY gained (ie, below the WTP of £20 000 per QALY) vs posttreatment surveillance alone. The resulting ICER was £52 358.01 (US $68 588.99) per HPV-related cancer averted and £64 090 (US $83 958.18) per HPV-related cancer death averted. The ICER was most sensitive to discount rate, incidence of HPV infection, vaccine price, and age at initial treatment for grade 2 or greater CIN. Results of the probabilistic sensitivity analysis showed peritreatment 9vHPV vaccination was cost-effective at the WTP recommended by the UK's Joint Committee on Vaccination and Immunisation (90% of iterations <£30 000 [US $39 300] per QALY) in 100% of iterations.

Conclusions and Relevance

These findings suggest that peritreatment prophylactic 9vHPV vaccination is a cost-effective option for preventing subsequent HPV-attributable diseases in patients surgically treated for grade 2 or greater CIN.

Predicting the Population Health Economic Impact of Current and New Cancer Treatments for Colorectal Cancer: A Data-Driven Whole Disease Simulation Model for Predicting the Number of Patients with Colorectal Cancer by Stage and Treatment Line in Australia

OBJECTIVES

Effective healthcare planning, resource allocation, and budgeting require accurate predictions of the number of patients needing treatment at specific cancer stages and treatment lines. The Predicting the Population Health Economic Impact of Current and New Cancer Treatments (PRIMCAT) for Colorectal Cancer (CRC) simulation model (PRIMCAT-CRC) was developed to meet this requirement for all CRC stages and relevant molecular profiles in Australia.

METHODS

Real-world data were used to estimate treatment utilization and time-to-event distributions. This populated a discrete-event simulation, projecting the number of patients receiving treatment across all disease stages and treatment lines for CRC and forecasting the number of patients likely to utilize future treatments. Illustrative analyses were undertaken, estimating treatments across disease stages and treatment lines over a 5-year period (2022-2026). We demonstrated the model's applicability through a case study introducing pembrolizumab as a first-line treatment for mismatch-repair-deficient stage IV.

RESULTS

Clinical registry data from 7163 patients informed the model. The model forecasts 15 738 incident and 2821 prevalent cases requiring treatment in 2022, rising to 15 921 and 2871, respectively, by 2026. Projections show that over 2022 to 2026, there will be a total of 116 752 treatments initiated, with 43% intended for stage IV disease. The introduction of pembrolizumab is projected for 706 patients annually, totaling 3530 individuals starting treatment with pembrolizumab over the forecasted period, without significantly altering downstream utilization of subsequent treatments.

CONCLUSIONS

PRIMCAT-CRC is a versatile tool that can be used to estimate the eligible patient populations for novel cancer therapies, thereby reducing uncertainty for policymakers in decisions to publicly reimburse new treatments.

Cost-effectiveness of monitoring ocular hypertension based on a risk prediction tool

BACKGROUND/AIMS

To assess the cost-effectiveness of making treatment decisions for patients with ocular hypertension (OHT) based on a risk prediction (RP) tool in the United Kingdom.

METHODS

A discrete event simulation model was constructed to compare the cost-effectiveness of an alternative care pathway in which the treatment decision was guided by a validated RP tool in secondary care against decision-making based on the standard care (SC). Individual patient sampling was used. Patients diagnosed with OHT and with an intraocular pressure of 24 mm Hg or over entered the model with a set of predefined individual characteristics related to their risk of conversion to glaucoma. These characteristics were retrieved from electronic medical records (n=5740). Different stages of glaucoma were modelled following conversion to glaucoma.

RESULTS

Almost all (99%) patients were treated using the RP strategy, and less than half (47%) of the patients were treated using the SC strategy. The RP strategy produced higher cost but also higher quality-adjusted life years (QALYs) than the SC strategy. The RP strategy was cost-effective compared with the SC strategy in the base-case analysis, with an incremental cost-effectiveness ratio value of £11 522. The RP strategy had a 96% probability of being cost-effective under a £20 000 per QALY threshold.

CONCLUSIONS

The use of an RP tool for the management of patients with OHT is likely to be cost-effective. However, the generalisability of the result might be limited due to the high-risk nature of this cohort and the specific RP threshold used in the study.

Integrating decision modeling and machine learning to inform treatment stratification

There is increasing interest in moving away from "one size fits all (OSFA)" approaches toward stratifying treatment decisions. Understanding how expected effectiveness and cost-effectiveness varies with patient covariates is a key aspect of stratified decision making. Recently proposed machine learning (ML) methods can learn heterogeneity in outcomes without pre-specifying subgroups or functional forms, enabling the construction of decision rules ('policies') that map individual covariates into a treatment decision. However, these methods do not yet integrate ML estimates into a decision modeling framework in order to reflect long-term policy-relevant outcomes and synthesize information from multiple sources. In this paper, we propose a method to integrate ML and decision modeling, when individual patient data is available to estimate treatment-specific survival time. We also propose a novel implementation of policy tree algorithms to define subgroups using decision model output. We demonstrate these methods using the SPRINT (Systolic Blood Pressure Intervention Trial), comparing outcomes for "standard" and "intensive" blood pressure targets. We find that including ML into a decision model can impact the estimate of incremental net health benefit (INHB) for OSFA policies. We also find evidence that stratifying treatment using subgroups defined by a tree-based algorithm can increase the estimates of the INHB.

Anti-Vascular Endothelial Growth Factor Drugs Compared With Panretinal Photocoagulation for the Treatment of Proliferative Diabetic Retinopathy: A Cost-Effectiveness Analysis

OBJECTIVES

This study aimed to evaluate the cost-effectiveness of anti-vascular endothelial growth factor drugs (anti-VEGFs) compared with panretinal photocoagulation (PRP) for treating proliferative diabetic retinopathy (PDR) in the United Kingdom.

METHODS

A discrete event simulation model was developed, informed by individual participant data meta-analysis. The model captures treatment effects on best corrected visual acuity in both eyes, and the occurrence of diabetic macular edema and vitreous hemorrhage. The model also estimates the value of undertaking further research to resolve decision uncertainty.

RESULTS

Anti-VEGFs are unlikely to generate clinically meaningful benefits over PRP. The model predicted anti-VEGFs be more costly and similarly effective as PRP, generating 0.029 fewer quality-adjusted life-years at an additional cost of £3688, with a net health benefit of -0.214 at a £20 000 willingness-to-pay threshold. Scenario analysis results suggest that only under very select conditions may anti-VEGFs offer potential for cost-effective treatment of PDR. The consequences of loss to follow-up were an important driver of model outcomes.

CONCLUSIONS

Anti-VEGFs are unlikely to be a cost-effective treatment for early PDR compared with PRP. Anti-VEGFs are generally associated with higher costs and similar health outcomes across various scenarios. Although anti-VEGFs were associated with lower diabetic macular edema rates, the number of cases avoided is insufficient to offset the additional treatment costs. Key uncertainties relate to the long-term comparative effectiveness of anti-VEGFs, particularly considering the real-world rates and consequences of treatment nonadherence. Further research on long-term visual acuity and rates of vision-threatening complications may be beneficial in resolving uncertainties.

Early Economic Modeling to Inform a Target Product Profile: A Case Study of a Novel Rapid Test for Clostridioides difficile Infection

Background. Target product profiles (TPPs) specify the essential properties tests must have to be able to address an unmet clinical need. Aim. To explore how early economic modeling can help to define TPP specifications based on cost-effectiveness considerations using the example of a new rapid diagnostic for Clostridioides difficile infection (CDI), a contagious health care-associated infection causing potentially fatal diarrhea. Methods. A resource-constrained simulation model was developed to compare a hypothetical test for CDI with current practice (i.e., test with glutamate dehydrogenase enzyme immunoassay first; if positive, test with polymerase chain reaction and cytotoxicity assay) for adult individuals with suspected CDI at the Leeds Teaching Hospital National Health System (NHS) Trust in the United Kingdom. Parameters are taken from UK-based observational data collected between 2018 and 2021, published literature, and expert opinion. A methodological framework was developed 1) to derive minimum diagnostic sensitivity and specificity and maximum price for different test turnaround-time values based on cost-effectiveness considerations from the health care perspective using the National Institute of Health Care Excellence willingness-to-pay threshold of £20,000 per quality-adjusted life-years and 2) to test their robustness using a series of sensitivity analyses. Results. A new rapid test for CDI with a 15-min turnaround time would require a minimum diagnostic sensitivity and specificity both equal to 96% and a maximum price of £44 to maintain cost-effectiveness compared with standard of care. Conclusions. This study provides a framework to inform the essential test properties based on cost-effectiveness considerations and to isolate the most influential model parameters and scenarios via a series of sensitivity analyses. These specifications, in turn, could be used to inform future TPPs for tests.

Highlights

Target product profiles (TPPs) for new medical tests provide test developers with performance benchmarks and technical requirements for new tests. Early economic evaluation has already been used to identify acceptable ranges for certain performance requirements for new tests. Currently, however, early economic evaluation methods are yet to be used in the context of TPP development, and there is no guidance as to how this could and should be done.A de novo approach was developed to identify the minimum performance requirements and maximum costs for new tests, based on cost-effectiveness considerations, while also isolating most influential parameters. The added value of this framework lies in structuring early economic evaluation methods as a means of informing transparent, evidence-based minimum TPP performance specifications while also accounting as much as possible for the (inevitable) uncertainty surrounding the minimum performance requirements.This study represents the first application of early economic modeling as a means of deriving the minimum performance specifications for a novel point-of-care test for Clostridioides difficile infection as set out in a future TPP.

A structured process for the validation of a decision-analytic model: application to a cost-effectiveness model for risk-stratified national breast screening

BACKGROUND

Decision-makers require knowledge of the strengths and weaknesses of decision-analytic models used to evaluate healthcare interventions to be able to confidently use the results of such models to inform policy. A number of aspects of model validity have previously been described, but no systematic approach to assessing the validity of a model has been proposed. This study aimed to consolidate the different aspects of model validity into a step-by-step approach to assessing the strengths and weaknesses of a decision-analytic model.

METHODS

A pre-defined set of steps were used to conduct the validation process of an exemplar early decision-analytic-model-based cost-effectiveness analysis of a risk-stratified national breast cancer screening programme [UK healthcare perspective; lifetime horizon; costs (£; 2021)]. Internal validation was assessed in terms of descriptive validity, technical validity and face validity. External validation was assessed in terms of operational validation, convergent validity (or corroboration) and predictive validity.

RESULTS

The results outline the findings of each step of internal and external validation of the early decision-analytic-model and present the validated model (called 'MANC-RISK-SCREEN'). The positive aspects in terms of meeting internal validation requirements are shown together with the remaining limitations of MANC-RISK-SCREEN.

CONCLUSION

Following a transparent and structured validation process, MANC-RISK-SCREEN has been shown to have satisfactory internal and external validity for use in informing resource allocation decision-making. We suggest that MANC-RISK-SCREEN can be used to assess the cost-effectiveness of exemplars of risk-stratified national breast cancer screening programmes (NBSP) from the UK perspective.

IMPLICATIONS

A step-by-step process for conducting the validation of a decision-analytic model was developed for future use by health economists. Using this approach may help researchers to fully demonstrate the strengths and limitations of their model to decision-makers.

Cost-Effectiveness of Lovotibeglogene Autotemcel (Lovo-Cel) Gene Therapy for Patients with Sickle Cell Disease and Recurrent Vaso-Occlusive Events in the United States

BACKGROUND AND OBJECTIVE

Gene therapies for sickle cell disease (SCD) may offer meaningful benefits for patients and society. This study evaluated the cost-effectiveness of lovotibeglogene autotemcel (lovo-cel), a one-time gene therapy administered via autologous hematopoietic stem cell transplantation, compared with common care for patients in the United States (US) with SCD aged ≥ 12 years with ≥ 4 vaso-occlusive events (VOEs) in the past 24 months.

METHODS

We developed a patient-level simulation model accounting for lovo-cel and SCD-related events, complications, and mortality over a lifetime time horizon. The pivotal phase 1/2 HGB-206 clinical trial (NCT02140554) served as the basis for lovo-cel efficacy and safety. Cost, quality-of-life, and other clinical data were sourced from HGB-206 data and the literature. Analyses were conducted from US societal and third-party payer perspectives. Uncertainty was assessed through probabilistic sensitivity analysis and extensive scenario analyses.

RESULTS

Patients treated with lovo-cel were predicted to survive 23.84 years longer on average (standard deviation [SD], 12.80) versus common care (life expectancy, 62.24 versus 38.40 years), with associated discounted patient quality-adjusted life-year (QALY) gains of 10.20 (SD, 4.10) and direct costs avoided of $1,329,201 (SD, $1,346,446) per patient. Predicted societal benefits included discounted caregiver QALY losses avoided of 1.19 (SD, 1.38) and indirect costs avoided of $540,416 (SD, $262,353) per patient. Including lovo-cel costs ($3,282,009 [SD, $29,690] per patient) resulted in incremental cost-effectiveness ratios of $191,519 and $124,051 per QALY gained from third-party payer and societal perspectives, respectively. In scenario analyses, the predicted cost-effectiveness of lovo-cel also was sensitive to baseline age and VOE frequency and to the proportion of patients achieving and maintaining complete resolution of VOEs.

CONCLUSIONS

Our analysis of lovo-cel gene therapy compared with common care for patients in the US with SCD with recurrent VOEs estimated meaningful improvements in survival, quality of life, and other clinical outcomes accompanied by increased overall costs for the health care system and for broader society. The predicted economic value of lovo-cel gene therapy was influenced by uncertainty in long-term clinical effects and by positive spillover effects on patient productivity and caregiver burden.

Improving resource allocation in the precision medicine Era: a simulation-based approach using R

The application of personalized medicine in developing countries is a major challenge, especially for those with poor economic status. A critical factor in improving the application of personalized medicine is the efficient allocation of resources. In healthcare systems, optimizing resource allocation without compromising patient care is paramount. This tutorial employs a simulation-based approach to evaluate the efficiency of bed allocation within a hospital setting. Utilizing a patient arrival model with an exponential distribution, we simulated patient trajectories to examine system bottlenecks, particularly focusing on waiting times. Initial simulations painted a scenario of an 'unstable' system, where waiting times and queue lengths surged due to the limited number of available beds. This research offers insights for hospital management on resource optimization leading to improved patient care.

Discrete Event Simulation to Incorporate Infusion Wait-Time When Assessing Cost-Effectiveness of a Chimeric-Antigen Receptor T Cell Therapy

OBJECTIVES

The main objective was to use discrete event simulation to model the impact of wait-time, defined as the time between leukapheresis and chimeric antigen receptor (CAR-T) infusion, when assessing the cost-effectiveness of tisagenlecleucel in young patients with relapsed/refractory acute lymphoblastic leukemia.

METHODS

The movement of patients through the model was determined by parametric time-to-event distributions, with the competing risk of an event determining the costs and quality-adjusted life-years (QALYs) assigned. Cost-effectiveness was expressed using the incremental cost-effectiveness ratio (ICER) for tisagenlecleucel compared with chemotherapy over the lifetime.

RESULTS

The base case generated a total of 5.79 QALYs and $622 872 for tisagenlecleucel and 1.19 QALYs and $181 219 for blinatumomab, resulting in an ICER of $96 074 per QALY. An increase in mean CAR-T wait-time to 6.20 months reduced the benefit and costs of tisagenlecleucel to 2.78 QALYs and $294 478 because of fewer patients proceeding to infusion, reducing the ICER to $71 112 per QALY. Alternatively, when the cost of tisagenlecleucel was assigned pre-infusion in sensitivity analysis, the ICER increased with increasing wait-time.

CONCLUSIONS

Under a payment arrangement where CAR-T cost is incurred post-infusion, the loss of benefit to patients is not reflected in the ICER. This may be misguiding to decision makers, where cost-effectiveness ratios are used to guide resource allocation. discrete event simulation is an important tool for economic modeling of CAR-T as it is amenable to capturing the impact of wait-time, facilitating better understanding of factors affecting service delivery and consequently informed decision making to deliver faster access to CAR-T for patients.

Research trends in contemporary health economics: a scientometric analysis on collective content of specialty journals

INTRODUCTION

Health economics is a thriving sub-discipline of economics. Applied health economics research is considered essential in the health care sector and is used extensively by public policy makers. For scholars, it is important to understand the history and status of health economics-when it emerged, the rate of research output, trending topics, and its temporal evolution-to ensure clarity and direction when formulating research questions.

METHODS

Nearly 13,000 articles were analysed, which were found in the collective publications of the ten most specialised health economic journals. We explored this literature using patterns of term co-occurrence and document co-citation.

RESULTS

The research output in this field is growing exponentially. Five main research divisions were identified: (i) macroeconomic evaluation, (ii) microeconomic evaluation, (iii) measurement and valuation of outcomes, (iv) monitoring mechanisms (evaluation), and (v) guidance and appraisal. Document co-citation analysis revealed eighteen major research streams and identified variation in the magnitude of activities in each of the streams. A recent emergence of research activities in health economics was seen in the Medicaid Expansion stream. Established research streams that continue to show high levels of activity include Child Health, Health-related Quality of Life (HRQoL) and Cost-effectiveness. Conversely, Patient Preference, Health Care Expenditure and Economic Evaluation are now past their peak of activity in specialised health economic journals. Analysis also identified several streams that emerged in the past but are no longer active.

CONCLUSIONS

Health economics is a growing field, yet there is minimal evidence of creation of new research trends. Over the past 10 years, the average rate of annual increase in internationally collaborated publications is almost double that of domestic collaborations (8.4% vs 4.9%), but most of the top scholarly collaborations remain between six countries only.

Optimizing the management of electrophysiology labs in Chinese hospitals using a discrete event simulation tool

BACKGROUND

The growing demand for electrophysiology (EP) treatment in China presents a challenge for current EP care delivery systems. This study constructed a discrete event simulation (DES) model of an inpatient EP care delivery process, simulating a generalized inpatient journey of EP patients from admission to discharge in the cardiology department of a tertiary hospital in China. The model shows how many more patients the system can serve under different resource constraints by optimizing various phases of the care delivery process.

METHODS

Model inputs were based on and validated using real-world data, simulating the scheduling of limited resources among competing demands from different patient types. The patient stay consists of three stages, namely: the pre-operative stay, the EP procedure, and the post-operative stay. The model outcome was the total number of discharges during the simulation period. The scenario analysis presented in this paper covers two capacity-limiting scenarios (CLS): (1) fully occupied ward beds and (2) fully occupied electrophysiology laboratories (EP labs). Within each CLS, we investigated potential throughput when the length of stay or operative time was reduced by 10%, 20%, and 30%. The reductions were applied to patients with atrial fibrillation, the most common indication accounting for almost 30% of patients.

RESULTS

Model validation showed simulation results approximated actual data (137.2 discharges calculated vs. 137 observed). With fully occupied wards, reducing pre- and/or post-operative stay time resulted in a 1-7% increased throughput. With fully occupied EP labs, reduced operative time increased throughput by 3-12%.

CONCLUSIONS

Model validation and scenario analyses demonstrated that the DES model reliably reflects the EP care delivery process. Simulations identified which phases of the process should be optimized under different resource constraints, and the expected increases in patients served.

Reflecting Parameter Uncertainty in Addition to Variability in Constrained Healthcare Resource Discrete Event Simulations: Worth Going the Extra Mile or a Road to Nowhere?

OBJECTIVES

Probabilistic sensitivity analysis (PSA) has been shown to reduce bias in outcomes of health economic models. However, only 1 existing study has been identified that incorporates PSA within a resource-constrained discrete event simulation (DES) model. This article aims to assess whether it is feasible and appropriate to use PSA to characterize parameter uncertainty in DES models that are primarily constructed to explore the impact of constrained resources.

METHODS

PSA is incorporated into a new case study of an Emergency Department DES. Structured expert elicitation is used to derive the variability and uncertainty input distributions associated with length of time taken to complete key activities within the Emergency Department. Potential challenges of implementation and analysis are explored.

RESULTS

The results of a trial of the model, which used the best estimates of the elicited means and variability around the time taken to complete activities, provided a reasonable fit to the data for length of time within the Emergency Department. However, there was substantial and skewed uncertainty around the activity times estimated from the elicitation exercise. This led to patients taking almost 3 weeks to leave the Emergency Department in some PSA runs, which would not occur in practice.

CONCLUSIONS

Structured expert elicitation can be used to derive plausible estimates of activity times and their variability, but experts' uncertainty can be substantial. For parameters that have an impact on interactions within a resource-constrained simulation model, PSA can lead to implausible model outputs; hence, other methods may be needed.

Using stochastic simulation modelling to study occupancy levels of decentralised admission avoidance units in Norway

Identifying alternatives to acute hospital admission is a priority for many countries. Over 200 decentralised municipal acute units (MAUs) were established in Norway to divert low-acuity patients away from hospitals. MAUs have faced criticism for low mean occupancy and not relieving pressures on hospitals. We developed a discrete time simulation model of admissions and discharges to MAUs to test scenarios for increasing absolute mean occupancy. We also used the model to estimate the number of patients turned away as historical data was unavailable. Our experiments suggest that mergers alone are unlikely to substantially increase MAU absolute mean occupancy as unmet demand is generally low. However, merging MAUs offers scope for up to 20% reduction in bed capacity, without affecting service provision. Our work has relevance for other admissions avoidance units and provides a method for estimating unconstrained demand for beds in the absence of historical data.

Assessment of the Effectiveness, Socio-Economic Impact and Implementation of a Digital Solution for Patients with Advanced Chronic Diseases: The ADLIFE Study Protocol

Due to population ageing and medical advances, people with advanced chronic diseases (ACD) live longer. Such patients are even more likely to face either temporary or permanent reduced functional reserve, which typically further increases their healthcare resource use and the burden of care on their caregiver(s). Accordingly, these patients and their caregiver(s) may benefit from integrated supportive care provided via digitally supported interventions. This approach may either maintain or improve their quality of life, increase their independence, and optimize the healthcare resource use from early stages. ADLIFE is an EU-funded project, aiming to improve the quality of life of older people with ACD by providing integrated personalized care via a digitally enabled toolbox. Indeed, the ADLIFE toolbox is a digital solution which provides patients, caregivers, and health professionals with digitally enabled, integrated, and personalized care, supporting clinical decisions, and encouraging independence and self-management. Here we present the protocol of the ADLIFE study, which is designed to provide robust scientific evidence on the assessment of the effectiveness, socio-economic, implementation, and technology acceptance aspects of the ADLIFE intervention compared to the current standard of care (SoC) when applied in real-life settings of seven different pilot sites across six countries. A quasi-experimental trial following a multicenter, non-randomized, non-concurrent, unblinded, and controlled design will be implemented. Patients in the intervention group will receive the ADLIFE intervention, while patients in the control group will receive SoC. The assessment of the ADLIFE intervention will be conducted using a mixed-methods approach.

Development and validation of a decision model for the evaluation of novel lung cancer treatments in the Netherlands

Recent discoveries in molecular diagnostics and drug treatments have improved the treatment of patients with advanced (inoperable) non-squamous non-small cell lung cancer (NSCLC) from solely platinum-based chemotherapy to more personalized treatment, including targeted therapies and immunotherapies. However, these improvements come at considerable costs, highlighting the need to assess their cost-effectiveness in order to optimize lung cancer care. Traditionally, cost-effectiveness models for the evaluation of new lung cancer treatments were based on the findings of the randomized control trials (RCTs). However, the strict RCT inclusion criteria make RCT patients not representative of patients in the real-world. Patients in RCTs have a better prognosis than patients in a real-world setting. Therefore, in this study, we developed and validated a diagnosis-treatment decision model for patients with advanced (inoperable) non-squamous NSCLC based on real-world data in the Netherlands. The model is a patient-level microsimulation model implemented as discrete event simulation with five health events. Patients are simulated from diagnosis to death, including at most three treatment lines. The base-model (non-personalized strategy) was populated using real-world data of patients treated with platinum-based chemotherapy between 2008 and 2014 in one of six Dutch teaching hospitals. To simulate personalized care, molecular tumor characteristics were incorporated in the model based on the literature. The impact of novel targeted treatments and immunotherapies was included based on published RCTs. To validate the model, we compared survival under a personalized treatment strategy with observed real-world survival. This model can be used for health-care evaluation of personalized treatment for patients with advanced (inoperable) NSCLC in the Netherlands.

Modeling the Natural History and Screening Effects of Colorectal Cancer Using Both Adenoma and Serrated Neoplasia Pathways: The Development, Calibration, and Validation of a Discrete Event Simulation Model

Background. Existing colorectal cancer (CRC) screening models mostly focus on the adenoma pathway of CRC development, overlooking the serrated neoplasia pathway, which might result in overly optimistic screening predictions. In addition, Bayesian inference methods have not been widely used for model calibration. We aimed to develop a CRC screening model accounting for both pathways, calibrate it with approximate Bayesian computation (ABC) methods, and validate it with large CRC screening trials. Methods. A discrete event simulation (DES) of the CRC natural history (DECAS) was constructed using the adenoma and serrated pathways in R software. The model simulates CRC-related events in a specific birth cohort through various natural history states. Calibration took advantage of 74 prevalence data points from the German screening colonoscopy program of 5.2 million average-risk participants using an ABC method. CRC incidence outputs from DECAS were validated with the German national cancer registry data; screening effects were validated using 17-y data from the UK Flexible Sigmoidoscopy Screening sigmoidoscopy trial and a German screening colonoscopy cohort study. Results. The Bayesian calibration rendered 1,000 sets of posterior parameter samples. With the calibrated parameters, the observed age- and sex-specific CRC prevalences from the German registries were within the 95% DECAS-predicted intervals. Regarding screening effects, DECAS predicted a 41% (95% intervals 30%-51%) and 62% (95% intervals 55%-68%) reduction in 17-y cumulative CRC mortality for a single screening sigmoidoscopy and colonoscopy, respectively, falling within 95% confidence intervals reported in the 2 clinical studies used for validation. Conclusions. We presented DECAS, the first Bayesian-calibrated DES model for CRC natural history and screening, accounting for 2 CRC tumorigenesis pathways. The validated model can serve as a valid tool to evaluate the (cost-)effectiveness of CRC screening strategies.

Highlights

This article presents a new discrete event simulation model, DECAS, which models both adenoma-carcinoma and serrated neoplasia pathways for colorectal cancer (CRC) development and CRC screening effects.DECAS is calibrated based on a Bayesian inference method using the data from German screening colonoscopy program, which consists of more than 5 million first-time average-risk participants aged 55 years and older in 2003 to 2014.DECAS is flexible for evaluating various CRC screening strategies and can differentiate screening effects in different parts of the colon.DECAS is validated with large screening sigmoidoscopy and colonoscopy clinical study data and can be further used to evaluate the (cost-)effectiveness of German colorectal cancer screening strategies.

Cost-effectiveness of guideline-based care provision for patients with diabetes-related foot ulcers: A modelled analysis using discrete event simulation

AIMS

The provision of guideline-based care for patients with diabetes-related foot ulcers (DFU) in clinical practice is suboptimal. We estimated the cost-effectiveness of higher rates of guideline-based care, compared with current practice.

METHODS

The costs and quality-adjusted life-years (QALYs) associated with current practice (30% of patients receiving guideline-based care) were compared with seven hypothetical scenarios with increasing proportion of guideline-based care (40%, 50%, 60%, 70%, 80%, 90% and 100%). Comparisons were made using discrete event simulations reflecting the natural history of DFU over a 3-year time horizon from the Australian healthcare perspective. Incremental cost-effectiveness ratios were calculated for each scenario and compared to a willingness-to-pay of AUD 28,000 per QALY. Probabilistic sensitivity analyses were conducted to incorporate joint parameter uncertainty.

RESULTS

All seven scenarios with higher rates of guideline-based care were likely cheaper and more effective than current practice. Increased proportions compared with current practice resulted in between AUD 0.28 and 1.84 million in cost savings and 11-56 additional QALYs per 1000 patients. Probabilistic sensitivity analyses indicated that the finding is robust to parameter uncertainty.

CONCLUSIONS

Higher proportions of patients receiving guideline-based care are less costly and improve patient outcomes. Strategies to increase the proportion of patients receiving guideline-based care are warranted.

Long-term cost-effectiveness of matrix-associated chondrocyte implantation in the German health care system: a discrete event simulation

INTRODUCTION

Cartilage defects in the knee can be caused by injury, various types of arthritis, or degeneration. As a long-term consequence of cartilage defects, osteoarthritis can develop over time, often leading to the need for a total knee replacement (TKR). The treatment alternatives of chondral defects include, among others, microfracture, and matrix-associated autologous chondrocyte implantation (M-ACI). The purpose of this study was to determine cost-effectiveness of M-ACI in Germany with available mid- and long-term outcome data, with special focus on the avoidance of TKR.

MATERIALS AND METHODS

We developed a discrete-event simulation (DES) that follows up individuals with cartilage defects of the knee over their lifetimes. The DES was conducted with a status-quo scenario in which M-ACI is available and a comparison scenario with no M-ACI available. The model included 10,000 patients with articular cartilage defects. We assumed Weibull distributions for short- and long-term effects for implant failures. Model outcomes were costs, number of TKRs, and quality-adjusted life years (QALYs). All analyses were performed from the perspective of the German statutory health insurance.

RESULTS

The majority of patients was under 45 years old, with defect sizes between 2 and 7 cm² (mean: 4.5 cm²); average modeled lifetime was 48 years. In the scenario without M-ACI, 26.4% of patients required a TKR over their lifetime. In the M-ACI scenario, this was the case in only 5.5% of cases. Thus, in the modeled cohort of 10,000 patients, 2700 TKRs, including revisions, could be avoided. Patients treated with M-ACI experienced improved quality of life (22.53 vs. 21.21 QALYs) at higher treatment-related costs (18,589 vs. 14,134 € /patient) compared to those treated without M-ACI, yielding an incremental cost-effectiveness ratio (ICER) of 3376 € /QALY.

CONCLUSION

M-ACI is projected to be a highly cost-effective treatment for chondral defects of the knee in the German healthcare setting.

Validating the Assumptions of Population Adjustment: Application of Multilevel Network Meta-regression to a Network of Treatments for Plaque Psoriasis

BACKGROUND

Network meta-analysis (NMA) and indirect comparisons combine aggregate data (AgD) from multiple studies on treatments of interest but may give biased estimates if study populations differ. Population adjustment methods such as multilevel network meta-regression (ML-NMR) aim to reduce bias by adjusting for differences in study populations using individual patient data (IPD) from 1 or more studies under the conditional constancy assumption. A shared effect modifier assumption may also be necessary for identifiability. This article aims to demonstrate how the assumptions made by ML-NMR can be assessed in practice to obtain reliable treatment effect estimates in a target population.

METHODS

We apply ML-NMR to a network of evidence on treatments for plaque psoriasis with a mix of IPD and AgD trials reporting ordered categorical outcomes. Relative treatment effects are estimated for each trial population and for 3 external target populations represented by a registry and 2 cohort studies. We examine residual heterogeneity and inconsistency and relax the shared effect modifier assumption for each covariate in turn.

RESULTS

Estimated population-average treatment effects were similar across study populations, as differences in the distributions of effect modifiers were small. Better fit was achieved with ML-NMR than with NMA, and uncertainty was reduced by explaining within- and between-study variation. We found little evidence that the conditional constancy or shared effect modifier assumptions were invalid.

CONCLUSIONS

ML-NMR extends the NMA framework and addresses issues with previous population adjustment approaches. It coherently synthesizes evidence from IPD and AgD studies in networks of any size while avoiding aggregation bias and noncollapsibility bias, allows for key assumptions to be assessed or relaxed, and can produce estimates relevant to a target population for decision-making.

HIGHLIGHTS

Multilevel network meta-regression (ML-NMR) extends the network meta-analysis framework to synthesize evidence from networks of studies providing individual patient data or aggregate data while adjusting for differences in effect modifiers between studies (population adjustment). We apply ML-NMR to a network of treatments for plaque psoriasis with ordered categorical outcomes.We demonstrate for the first time how ML-NMR allows key assumptions to be assessed. We check for violations of conditional constancy of relative effects (such as unobserved effect modifiers) through residual heterogeneity and inconsistency and the shared effect modifier assumption by relaxing this for each covariate in turn.Crucially for decision making, population-adjusted treatment effects can be produced in any relevant target population. We produce population-average estimates for 3 external target populations, represented by the PsoBest registry and the PROSPECT and Chiricozzi 2019 cohort studies.

Causal evidence in health decision making: methodological approaches of causal inference and health decision science

Objectives

Public health decision making is a complex process based on thorough and comprehensive health technology assessments involving the comparison of different strategies, values and tradeoffs under uncertainty. This process must be based on best available evidence and plausible assumptions. Causal inference and health decision science are two methodological approaches providing information to help guide decision making in health care. Both approaches are quantitative methods that use statistical and modeling techniques and simplifying assumptions to mimic the complexity of the real world. We intend to review and lay out both disciplines with their aims, strengths and limitations based on a combination of textbook knowledge and expert experience.

Methods

To help understanding and differentiating the methodological approaches of causal inference and health decision science, we reviewed both methods with the focus on aims, research questions, methods, assumptions, limitations and challenges, and software. For each methodological approach, we established a group of four experts from our own working group to carefully review and summarize each method, followed by structured discussion rounds and written reviews, in which the experts from all disciplines including HTA and medicine were involved. The entire expert group discussed objectives, strengths and limitations of both methodological areas, and potential synergies. Finally, we derived recommendations for further research and provide a brief outlook on future trends.

Results

Causal inference methods aim for drawing causal conclusions from empirical data on the relationship of pre-specified interventions on a specific target outcome and apply a counterfactual framework and statistical techniques to derive causal effects of exposures or interventions from these data. Causal inference is based on a causal diagram, more specifically, a directed acyclic graph (DAG), which encodes the assumptions regarding the causal relations between variables. Depending on the type of confounding and selection bias, traditional statistical methods or more complex g-methods are needed to derive valid causal effects. Besides the correct specification of the DAG and the statistical model, assumptions such as consistency, positivity, and exchangeability must be checked when aiming at causal inference. Health decision science aims for guiding policy decision making regarding health interventions considering and balancing multiple competing objectives of a decision based on data from multiple sources and studies, for example prevalence studies, clinical trials and long-term observational routine effectiveness studies, and studies on preferences and costs. It involves decision analysis, a systematic, explicit and quantitative framework to guide decisions under uncertainty. Decision analyses are based on decision-analytic models to mimic the course of disease as well as aspects and consequences of the intervention in order to quantitatively optimize the decision. Depending on the type of decision problem, decision trees, state-transition models, discrete event simulation models, dynamic transmission models, or other model types are applied. Models must be validated against observed data, and comprehensive sensitivity analyses must be performed to assess uncertainty. Besides the appropriate choice of the model type and the valid specification of the model structure, it must be checked if input parameters of effects can be interpreted as causal parameters in the model. Otherwise results will be biased.

Conclusions

Both causal inference and health decision science aim for providing best causal evidence for informed health decision making. The strengths and limitations of both methods differ and a good understanding of both methods is essential for correct application but also for correct interpretation of findings from the described methods. Importantly, decision-analytic modeling should be combined with causal inference when developing guidance and recommendations regarding decisions on health care interventions.

Use of Resource Modeling to Quantify the Organizational Impact of Subcutaneous Formulations for the Treatment of Oncologic Patients: The Case of Daratumumab in Multiple Myeloma

PURPOSE

Resource modeling aims to explicitly quantify the effects of adopting new health care technologies in settings with capacity-related constraints. The aim of this analysis was to use resource modeling to explore the effects of the uptake of first-line treatment with daratumumab on wait lists and wait times in patients with untreated multiple myeloma. Two formulations were compared: the standard IV formulation (DARA-IV) and a recently approved SC formulation (DARA-SC).

METHODS

First, semi-structured interviews at six oncologic centers were used to retrieve data on the management of patients given a DARA-IV regimen. Second, a discrete event simulation (DES) model was built to estimate the effects on resource consumption, wait lists, and wait times in scenarios with different incident numbers of patients treated with either DARA-IV or DARA-SC.

FINDINGS

In all of the simulated scenarios with more incident patients initiated on first-line treatment with DARA-IV, the actual capacity of infusion chairs was not enough to meet the demand, leading to increases in wait times and wait lists. In the highest-demand scenario, 17 more infusion chairs per center would be required to avoid such increases. Treatment with DARA-SC would allow centers to meet the demand with their actual capacity.

IMPLICATIONS

DES modeling can effectively be used to formally explore the effects of different formulations on the use of limited resources, wait lists, and wait times at the facility level. Based on the findings from this analysis, DARA-SC may free up resources and prevent short- and long-term costs to infusion centers.

Discrete Event Simulation Model for Cost-Effectiveness Evaluation of Screening for Asymptomatic Patients with Lower Extremity Arterial Disease

Lower limb ischemic disease (LEAD) affects a significant portion of the population, with most patients being asymptomatic. Patient screening is necessary because LEAD patients have an increased risk of occurrence of other cardiovascular events and manifestations of disease, in terms of leg symptoms such as intermittent claudication, critical limb ischemia, or amputation. The aim of this work was to evaluate the cost-effectiveness of screening using ABI diagnostics in asymptomatic patients and its impact on limb symptoms associated with LEAD. A discrete event simulation model was created to capture lifetime costs and effects. Costs were calculated from the perspective of the health care payer, and the effects were calculated as QALYs. A cost-effectiveness analysis was performed to compare ABI screening examination and the situation without such screening. A probabilistic sensitivity analysis and scenario analysis were carried out to evaluate the robustness of the results. In the basic setting, the screening intervention was a more expensive intervention, at a cost of CZK 174,010, compared to CZK 70,177 for the strategy without screening. The benefits of screening were estimated at 14.73 QALYs, with 14.46 QALYs without screening. The final ICER value of CZK 389,738 per QALY is below the willingness to pay threshold. Likewise, the results of the probabilistic sensitivity analysis and of the scenario analysis were below the threshold of willingness to pay, thus confirming the robustness of the results. In conclusion, ABI screening appears to be a cost-effective strategy for asymptomatic patients aged 50 years when compared to the no-screening option.

The Use of Expert Elicitation among Computational Modeling Studies in Health Research: A Systematic Review

BACKGROUND

Expert elicitation (EE) has been used across disciplines to estimate input parameters for computational modeling research when information is sparse or conflictual.

OBJECTIVES

We conducted a systematic review to compare EE methods used to generate model input parameters in health research.

DATA SOURCES

PubMed and Web of Science.

STUDY ELIGIBILITY

Modeling studies that reported the use of EE as the source for model input probabilities were included if they were published in English before June 2021 and reported health outcomes.

DATA ABSTRACTION AND SYNTHESIS

Studies were classified as "formal" EE methods if they explicitly reported details of their elicitation process. Those that stated use of expert opinion but provided limited information were classified as "indeterminate" methods. In both groups, we abstracted citation details, study design, modeling methodology, a description of elicited parameters, and elicitation methods. Comparisons were made between elicitation methods.

STUDY APPRAISAL

Studies that conducted a formal EE were appraised on the reporting quality of the EE. Quality appraisal was not conducted for studies of indeterminate methods.

RESULTS

The search identified 1520 articles, of which 152 were included. Of the included studies, 40 were classified as formal EE and 112 as indeterminate methods. Most studies were cost-effectiveness analyses (77.6%). Forty-seven indeterminate method studies provided no information on methods for generating estimates. Among formal EEs, the average reporting quality score was 9 out of 16.

LIMITATIONS

Elicitations on nonhealth topics and those reported in the gray literature were not included.

CONCLUSIONS

We found poor reporting of EE methods used in modeling studies, making it difficult to discern meaningful differences in approaches. Improved quality standards for EEs would improve the validity and replicability of computational models.

HIGHLIGHTS

We find extensive use of expert elicitation for the development of model input parameters, but most studies do not provide adequate details of their elicitation methods.Lack of reporting hinders greater discussion of the merits and challenges of using expert elicitation for model input parameter development.There is a need to establish expert elicitation best practices and reporting guidelines.

Modelling Granular Process Flow Information to Reduce Bottlenecks in the Emergency Department

Increasing demand and changing case-mix have resulted in bottlenecks and longer waiting times in emergency departments (ED). However, many process improvement efforts addressing the bottlenecks have limitations, as they lack accurate models of the real system as input accounting for operational complexities. To understand the limitation, this research modelled granular procedural information, to analyse processes in a Level-1 ED of a 1200-bed teaching hospital in the UK. Semi-structured interviews with 21 clinicians and direct observations provided the necessary information. Results identified Majors as the most crowded area, hence, a systems modelling technique, role activity diagram, was used to derive highly granular process maps illustrating care in Majors which were further validated by 6 additional clinicians. Bottlenecks observed in Majors included awaiting specialist input, tests outside the ED, awaiting transportation, bed search, and inpatient handover. Process mapping revealed opportunities for using precedence information to reduce repeat tests; informed alerting; and provisioning for operational complexity into ED processes as steps to potentially alleviate bottlenecks. Another result is that this is the first study to map care processes in Majors, the area within the ED that treats complex patients whose care journeys are susceptible to variations. Findings have implications on the development of improvement approaches for managing bottlenecks.

Improving emergency treatment for patients with acute stroke: the PEARS research programme, including the PASTA cluster RCT

Background

Intravenous thrombolysis and intra-arterial thrombectomy are proven emergency treatments for acute ischaemic stroke, but they require rapid delivery to selected patients within specialist services. National audit data have shown that treatment provision is suboptimal.

Objectives

The aims were to (1) determine the content, clinical effectiveness and day 90 cost-effectiveness of an enhanced paramedic assessment designed to facilitate thrombolysis delivery in hospital and (2) model thrombectomy service configuration options with optimal activity and cost-effectiveness informed by expert and public views.

Design

A mixed-methods approach was employed between 2014 and 2019. Systematic reviews examined enhanced paramedic roles and thrombectomy effectiveness. Professional and service user groups developed a thrombolysis-focused Paramedic Acute Stroke Treatment Assessment, which was evaluated in a pragmatic multicentre cluster randomised controlled trial and parallel process evaluation. Clinicians, patients, carers and the public were surveyed regarding thrombectomy service configuration. A decision tree was constructed from published data to estimate thrombectomy eligibility of the UK stroke population. A matching discrete-event simulation predicted patient benefits and financial consequences from increasing the number of centres.

Setting

The paramedic assessment trial was hosted by three regional ambulance services (in north-east England, north-west England and Wales) serving 15 hospitals.

Participants

A total of 103 health-care representatives and 20 public representatives assisted in the development of the paramedic assessment. The trial enrolled 1214 stroke patients within 4 hours of symptom onset. Thrombectomy service provision was informed by a Delphi exercise with 64 stroke specialists and neuroradiologists, and surveys of 147 patients and 105 public respondents.

Interventions

The paramedic assessment comprised additional pre-hospital information collection, structured hospital handover, practical assistance up to 15 minutes post handover, a pre-departure care checklist and clinician feedback.

Main outcome measures

The primary outcome was the proportion of patients receiving thrombolysis. Secondary outcomes included day 90 health (poor status was a modified Rankin Scale score of > 2). Economic outputs reported the number of cases treated and cost-effectiveness using quality-adjusted life-years and Great British pounds.

Data sources

National registry data from the Sentinel Stroke National Audit Programme and the Scottish Stroke Care Audit were used.

Review methods

Systematic searches of electronic bibliographies were used to identify relevant literature. Study inclusion and data extraction processes were described using Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines.

Results

The paramedic assessment trial found a clinically important but statistically non-significant reduction in thrombolysis among intervention patients, compared with standard care patients [197/500 (39.4%) vs. 319/714 (44.7%), respectively] (adjusted odds ratio 0.81, 95% confidence interval 0.61 to 1.08; p = 0.15). The rate of poor health outcomes was not significantly different, but was lower in the intervention group than in the standard care group [313/489 (64.0%) vs. 461/690 (66.8%), respectively] (adjusted odds ratio 0.86, 95% confidence interval 0.60 to 1.2; p = 0.39). There was no difference in the quality-adjusted life-years gained between the groups (0.005, 95% confidence interval –0.004 to 0.015), but total costs were significantly lower for patients in the intervention group than for those in the standard care group (–£1086, 95% confidence interval –£2236 to –£13). It has been estimated that, in the UK, 10,140–11,530 patients per year (i.e. 12% of stroke admissions) are eligible for thrombectomy. Meta-analysis of published data confirmed that thrombectomy-treated patients were significantly more likely to be functionally independent than patients receiving standard care (odds ratio 2.39, 95% confidence interval 1.88 to 3.04; n = 1841). Expert consensus and most public survey respondents favoured selective secondary transfer for accessing thrombectomy at regional neuroscience centres. The discrete-event simulation model suggested that six new English centres might generate 190 quality-adjusted life-years (95% confidence interval –6 to 399 quality-adjusted life-years) and a saving of £1,864,000 per year (95% confidence interval –£1,204,000 to £5,017,000 saving per year). The total mean thrombectomy cost up to 72 hours was £12,440, mostly attributable to the consumables. There was no significant cost difference between direct admission and secondary transfer (mean difference –£368, 95% confidence interval –£1016 to £279; p = 0.26).

Limitations

Evidence for paramedic assessment fidelity was limited and group allocation could not be masked. Thrombectomy surveys represented respondent views only. Simulation models assumed that populations were consistent with published meta-analyses, included limited parameters reflecting underlying data sets and did not consider the capital costs of setting up new services.

Conclusions

Paramedic assessment did not increase the proportion of patients receiving thrombolysis, but outcomes were consistent with improved cost-effectiveness at day 90, possibly reflecting better informed treatment decisions and/or adherence to clinical guidelines. However, the health difference was non-significant, small and short term. Approximately 12% of stroke patients are suitable for thrombectomy and widespread provision is likely to generate health and resource gains. Clinician and public views support secondary transfer to access treatment.

Future work

Further evaluation of emergency care pathways will determine whether or not enhanced paramedic assessment improves hospital guideline compliance. Validation of the simulation model post reconfiguration will improve precision and describe wider resource implications.

Trial registration

This trial is registered as ISRCTN12418919 and the systematic review protocols are registered as PROSPERO CRD42014010785 and PROSPERO CRD42015016649.

Funding

The project was funded by the National Institute for Health and Care Research (NIHR) Programme Grants for Applied Research programme and will be published in full in Programme Grants for Applied Research; Vol. 10, No. 4. See the NIHR Journals Library website for further project information.

Aspirin to target arterial events in chronic kidney disease (ATTACK): study protocol for a multicentre, prospective, randomised, open-label, blinded endpoint, parallel group trial of low-dose aspirin vs. standard care for the primary prevention of cardiovascular disease in people with chronic kidney disease

BACKGROUND

Chronic kidney disease (CKD) is a very common long-term condition and powerful risk factor for cardiovascular disease (CVD). Low-dose aspirin is of proven benefit in the secondary prevention of myocardial infarction (MI) and stroke in people with pre-existing CVD. However, in people without CVD, the rates of MI and stroke are much lower, and the benefits of aspirin in the primary prevention of CVD are largely balanced by an increased risk of bleeding. People with CKD are at greatly increased risk of CVD and so the absolute benefits of aspirin are likely to be greater than in lower-risk groups, even if the relative benefits are the same. Post hoc evidence suggests the relative benefits may be greater in the CKD population but the risk of bleeding may also be higher. A definitive study of aspirin for primary prevention in this high-risk group, recommended by the National Institute for Health and Care Excellence (NICE) in 2014, has never been conducted. The question has global significance given the rising burden of CKD worldwide and the low cost of aspirin.

METHODS

ATTACK is a pragmatic multicentre, prospective, randomised, open-label, blinded endpoint adjudication superiority trial of aspirin 75 mg daily vs. standard care for the primary prevention of CVD in 25,210 people aged 18 years and over with CKD recruited from UK Primary Care. Participants aged 18 years and over with CKD (GFR category G1-G4) will be identified in Primary Care and followed up using routinely collected data and annual questionnaires for an average of 5 years. The primary outcome is the time to first major vascular event (composite of non-fatal MI, non-fatal stroke and cardiovascular death [excluding confirmed intracranial haemorrhage and other fatal cardiovascular haemorrhage]). Deaths from other causes (including fatal bleeding) will be treated as competing events. The study will continue until 1827 major vascular events have occurred. The principal safety outcome is major intracranial and extracranial bleeding; this is hypothesised to be increased in those randomised to take aspirin. The key consideration is then whether and to what extent the benefits of aspirin from the expected reduction in CVD events exceed the risks of major bleeding.

DISCUSSION

This will be the first definitive trial of aspirin for primary CVD prevention in CKD patients. The research will be of great interest to clinicians, guideline groups and policy-makers, in the UK and globally, particularly given the high and rising prevalence of CKD that is driven by population ageing and epidemics of obesity and diabetes. The low cost of aspirin means that a positive result would be of relevance to low- and middle-income countries and the impact in the developed world less diluted by any inequalities in health care access.

TRIAL REGISTRATION

ISRCTN: ISRCTN40920200 . EudraCT: 2018-000644-26 .

CLINICALTRIALS

gov: NCT03796156.

Being Transparent About Brilliant Failures: An Attempt to Use Real-World Data in a Disease Model for Patients with Castration-Resistant Prostate Cancer

BACKGROUND

Real-world disease models spanning multiple treatment lines can provide insight into the (cost) effectiveness of treatment sequences in clinical practice.

OBJECTIVE

Our objective was to explore whether a disease model based solely on real-world data (RWD) could be used to estimate the effectiveness of treatments for patients with castration-resistant prostate cancer (CRPC) that could then be suitably used in a cost-effectiveness analysis.

METHODS

We developed a patient-level simulation model using patient-level data from the Dutch CAPRI registry as input parameters. Time to event (TTE) and overall survival (OS) were estimated with multivariate regression models, and type of event (i.e., next treatment or death) was estimated with multivariate logistic regression models. To test internal validity, TTE and OS from the simulation model were compared with the observed outcomes in the registry.

RESULTS

Although patient characteristics and survival outcomes of the simulated data were comparable to those in the observed data (median OS 20.6 vs. 19.8 months, respectively), the disease model was less accurate in estimating differences between treatments (median OS simulated vs. observed population: 18.6 vs. 17.9 [abiraterone acetate plus prednisone], 24.0 vs. 25.0 [enzalutamide], 20.2 vs. 18.7 [docetaxel], and 20.0 vs. 23.8 months [radium-223]).

CONCLUSIONS

Overall, the disease model accurately approximated the observed data in the total CRPC population. However, the disease model was unable to predict differences in survival between treatments due to unobserved differences. Therefore, the model is not suitable for cost-effectiveness analysis of CRPC treatment. Using a combination of RWD and data from randomised controlled trials to estimate treatment effectiveness may improve the model.

Comparing Modeling Approaches for Discrete Event Simulations With Competing Risks Based on Censored Individual Patient Data: A Simulation Study and Illustration in Colorectal Cancer

OBJECTIVES

This study aimed to provide detailed guidance on modeling approaches for implementing competing events in discrete event simulations based on censored individual patient data (IPD).

METHODS

The event-specific distributions (ESDs) approach sampled times from event-specific time-to-event distributions and simulated the first event to occur. The unimodal distribution and regression approach sampled a time from a combined unimodal time-to-event distribution, representing all events, and used a (multinomial) logistic regression model to select the event to be simulated. A simulation study assessed performance in terms of relative absolute event incidence difference and relative entropy of time-to-event distributions for different types and levels of right censoring, numbers of events, distribution overlap, and sample sizes. Differences in cost-effectiveness estimates were illustrated in a colorectal cancer case study.

RESULTS

Increased levels of censoring negatively affected the modeling approaches' performance. A lower number of competing events and higher overlap of distributions improved performance. When IPD were censored at random times, ESD performed best. When censoring occurred owing to a maximum follow-up time for 2 events, ESD performed better for a low level of censoring (ie, 10%). For 3 or 4 competing events, ESD better represented the probabilities of events, whereas unimodal distribution and regression better represented the time to events. Differences in cost-effectiveness estimates, both compared with no censoring and between approaches, increased with increasing censoring levels.

CONCLUSIONS

Modelers should be aware of the different modeling approaches available and that selection between approaches may be informed by data characteristics. Performing and reporting extensive validation efforts remains essential to ensure IPD are appropriately represented.

A Dynamic Approach to Economic Priority Setting to Invest in Youth Mental Health and Guide Local Implementation: Economic Protocol for Eight System Dynamics Policy Models

BACKGROUND

Mental illness costs the world economy over US2.5 Bn each year, including premature mortality, morbidity, and productivity losses. Multisector approaches are required to address the systemic drivers of mental health and ensure adequate service provision. There is an important role for economics to support priority setting, identify best value investments and inform optimal implementation. Mental health can be defined as a complex dynamic system where decision makers are challenged to prospectively manage the system over time. This protocol describes the approach to equip eight system dynamics (SD) models across Australia to support priority setting and guide portfolio investment decisions, tailored to local implementation context.

METHODS

As part of a multidisciplinary team, three interlinked protocols are developed; (i) the participatory process to codesign the models with local stakeholders and identify interventions for implementation, (ii) the technical protocol to develop the SD models to simulate the dynamics of the local population, drivers of mental health, the service system and clinical outcomes, and (iii) the economic protocol to detail how the SD models will be equipped to undertake a suite of economic analysis, incorporating health and societal perspectives. Models will estimate the cost of mental illness, inclusive of service costs (health and other sectors, where necessary), quality-adjusted life years (QALYs) lost, productivity costs and carer costs. To assess the value of investing (disinvesting) in interventions, economic analysis will include return-on-investment, cost-utility, cost benefit, and budget impact to inform affordability. Economic metrics are expected to be dynamic, conditional upon changing population demographics, service system capacities and the mix of interventions when synergetic or antagonistic interactions. To support priority setting, a portfolio approach will identify best value combinations of interventions, relative to a defined budget(s). User friendly dashboards will guide decision makers to use the SD models to inform resource allocation and generate business cases for funding.

DISCUSSION

Equipping SD models to undertake economic analysis is intended to support local priority setting and help optimise implementation regarding the best value mix of investments, timing and scale. The objectives are to improve allocative efficiency, increase mental health and economic productivity.

OUP accepted manuscript

Objective

The study sought to investigate the disease state–dependent risk profiles of patient demographics and medical comorbidities associated with adverse outcomes of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infections.

Materials and Methods

A covariate-dependent, continuous-time hidden Markov model with 4 states (moderate, severe, discharged, and deceased) was used to model the dynamic progression of COVID-19 during the course of hospitalization. All model parameters were estimated using the electronic health records of 1362 patients from ProMedica Health System admitted between March 20, 2020 and December 29, 2020 with a positive nasopharyngeal PCR test for SARS-CoV-2. Demographic characteristics, comorbidities, vital signs, and laboratory test results were retrospectively evaluated to infer a patient’s clinical progression.

Results

The association between patient-level covariates and risk of progression was found to be disease state dependent. Specifically, while being male, being Black or having a medical comorbidity were all associated with an increased risk of progressing from the moderate disease state to the severe disease state, these same factors were associated with a decreased risk of progressing from the severe disease state to the deceased state.

Discussion

Recent studies have not included analyses of the temporal progression of COVID-19, making the current study a unique modeling-based approach to understand the dynamics of COVID-19 in hospitalized patients.

Conclusion

Dynamic risk stratification models have the potential to improve clinical outcomes not only in COVID-19, but also in a myriad of other acute and chronic diseases that, to date, have largely been assessed only by static modeling techniques.

More from less: Study on increasing throughput of COVID-19 screening and testing facility at an apex tertiary care hospital in New Delhi using discrete-event simulation software

Background

One of the challenges has been coping with an increasing need for COVID-19 testing. A COVID-19 screening and testing facility was created. There was a need for increasing throughput of the facility within the existing space and limited resources. Discrete event simulation was used to address this challenge.

Methodology

A cross-sectional interventional study was done from September 2020 to October 2020. Detailed process mapping with all micro-processes was done. Patient arrival patterns and time taken at each step were measured by two independent observers at random intervals over two weeks. The existing system was simulated and a bottleneck was identified. Two possible alternatives to the problem were simulated and evaluated.

Results

Scenario 1 showed a maximum throughput of 316. The average milestone times of all the processes after the step of “Preparation of sampling kits” jumped 62%; from 82 to 133 min. Staff state times also showed that staff at this step was stretched and medical lab technicians were underutilized. Scenario 2 simulated the alternative with lesser time spent on sampling kit preparation with a 22.4% increase in throughput, but could have led to impaired quality check. Scenario 3 simulated with increased manpower at the stage of bottleneck with 26.5% increase in throughput and was implemented on-ground.

Conclusion

Discrete event simulation helped to identify the bottleneck, simulate possible alternative solutions without disturbing the ongoing work, and finally choose the most suitable intervention to increase throughput, without the need for additional space allocation. It therefore helped to optimally utilize resources and get “more from less.”

A review of simulation models for the long-term management of type 2 diabetes in low-and-middle income countries

INTRODUCTION

The burden of type 2 diabetes is steadily increasing in low-and-middle-income countries, thereby posing a major threat from both a treatment, and funding standpoint. Although simulation modelling is generally relied upon for evaluating long-term costs and consequences associated with diabetes interventions, no recent article has reviewed the characteristics and capabilities of available models used in low-and-middle-income countries. We review the use of computer simulation modelling for the management of type 2 diabetes in low-and-middle-income countries.

METHODS

A search for studies reporting computer simulation models of the natural history of individuals with type 2 diabetes and/or decision models to evaluate the impact of treatment strategies on these populations was conducted in PubMed. Data were extracted following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines and assessed using modelling checklists. Publications before the year 2000, from high-income countries, studies involving animals and analyses that did not use mathematical simulations were excluded. The full text of eligible articles was sourced and information about the intervention and population being modelled, type of modelling approach and the model structure was extracted.

RESULTS

Of the 79 articles suitable for full text review, 44 studies met the inclusion criteria. All were cost-effectiveness/utility studies with the majority being from the East Asia and Pacific region (n = 29). Of the included studies, 34 (77.3%) evaluated the cost-effectiveness of pharmacological interventions and approximately 75% of all included studies used HbA1c as one of the treatment effects of the intervention. 32 (73%) of the publications were microsimulation models, and 29 (66%) were state-transition models. Most of the studies utilised annual cycles (n = 29, 71%), and accounted for costs and outcomes over 20 years or more (n = 38, 86.4%).

CONCLUSIONS

While the use of simulation modelling in the management of type 2 diabetes has been steadily increasing in low-and-middle-income countries, there is an urgent need to invest in evaluating therapeutic and policy interventions related to type 2 diabetes in low-and-middle-income countries through simulation modelling, especially with local research data. Moreover, it is important to improve transparency and credibility in the reporting of input data underlying model-based economic analyses, and studies.

Limitations of Available Blood Products for Massive Transfusion During Mass Casualty Events at US Level 1 Trauma Centers

INTRODUCTION

Exsanguination remains a leading cause of preventable death in traumatically injured patients. To better treat hemorrhagic shock, hospitals have adopted massive transfusion protocols (MTPs) which accelerate the delivery of blood products to patients. There has been an increase in mass casualty events (MCE) worldwide over the past two decades. These events can overwhelm a responding hospital's supply of blood products. Using a computerized model, this study investigated the ability of US trauma centers (TCs) to meet the blood product requirements of MCEs.

METHODS

Cross-sectional survey data of on-hand blood products were collected from 16 US level-1 TCs. A discrete event simulation model of a TC was developed based on historic data of blood product consumption during MCEs. Each hospital's blood bank was evaluated across increasingly more demanding MCEs using modern MTPs to guide resuscitation efforts in massive transfusion (MT) patients.

RESULTS

A total of 9,000 simulations were performed on each TC's data. Under the least demanding MCE scenario, the median size MCE in which TCs failed to adequately meet blood product demand was 50 patients (IQR 20-90), considering platelets. Ten TCs exhaust their supply of platelets prior to red blood cells (RBCs) or plasma. Disregarding platelets, five TCs exhausted their supply of O- packed RBCs, six exhausted their AB plasma supply, and five had a mixed exhaustion picture.

CONCLUSION

Assuming a TC's ability to treat patients is limited only by their supply of blood products, US level-1 TCs lack the on-hand blood products required to adequately treat patients following a MCE. Use of non-traditional blood products, which have a longer shelf life, may allow TCs to better meet the blood product requirement needs of patients following larger MCEs.

Lifetime Health and Economic Outcomes of Active Surveillance, Radical Prostatectomy, and Radiotherapy for Favorable-Risk Localized Prostate Cancer

OBJECTIVES

To estimate the lifetime health and economic outcomes of selecting active surveillance (AS), radical prostatectomy (RP), or radiation therapy (RT) as initial management for low- or favorable-risk localized prostate cancer.

METHODS

A discrete-event simulation model was developed using evidence from published randomized trials. Health outcomes were measured in life-years and quality-adjusted life-years (QALYs). Costs were included from a public payer perspective in Australian dollars. Outcomes were discounted at 5% over a lifetime horizon. Probabilistic and scenario analyses quantified parameter and structural uncertainty.

RESULTS

A total of 60% of patients in the AS arm eventually received radical treatment (surgery or radiotherapy) compared with 90% for RP and 91% for RT. Although AS resulted in fewer treatment-related complications, it led to increased clinical progression (AS 40.7%, RP 17.6%, RT 19.9%) and metastatic disease (AS 13.4%, RP 6.1%, RT 7.0%). QALYs were 10.88 for AS, 11.10 for RP, and 11.13 for RT. Total costs were A$17 912 for AS, A$15 609 for RP, and A$15 118 for RT. At a willingness to pay of A$20 000/QALY, RT had a 61.4% chance of being cost-effective compared to 38.5% for RP and 0.1% for AS.

CONCLUSIONS

Although AS resulted in fewer and delayed treatment-related complications, it was not found to be a cost-effective strategy for favorable-risk localized prostate cancer over a lifetime horizon because of an increase in the number of patients developing metastatic disease. RT was the dominant strategy yielding higher QALYs at lower cost although differences compared with RP were small.

Optimal Triage for COVID-19 Patients Under Limited Health Care Resources With a Parsimonious Machine Learning Prediction Model and Threshold Optimization Using Discrete-Event Simulation: Development Study

Background

The COVID-19 pandemic has placed an unprecedented burden on health care systems.

Objective

We aimed to effectively triage COVID-19 patients within situations of limited data availability and explore optimal thresholds to minimize mortality rates while maintaining health care system capacity.

Methods

A nationwide sample of 5601 patients confirmed with COVID-19 until April 2020 was retrospectively reviewed. Extreme gradient boosting (XGBoost) and logistic regression analysis were used to develop prediction models for the maximum clinical severity during hospitalization, classified according to the World Health Organization Ordinal Scale for Clinical Improvement (OSCI). The recursive feature elimination technique was used to evaluate the maintenance of model performance when clinical and laboratory variables were eliminated. Using populations based on hypothetical patient influx scenarios, discrete-event simulation was performed to find an optimal threshold within limited resource environments that minimizes mortality rates.

Results

The cross-validated area under the receiver operating characteristic curve (AUROC) of the baseline XGBoost model that utilized all 37 variables was 0.965 for OSCI ≥6. Compared to the baseline model’s performance, the AUROC of the feature-eliminated model that utilized 17 variables was maintained at 0.963 with statistical insignificance. Optimal thresholds were found to minimize mortality rates in a hypothetical patient influx scenario. The benefit of utilizing an optimal triage threshold was clear, reducing mortality up to 18.1%, compared with the conventional Youden index.

Conclusions

Our adaptive triage model and its threshold optimization capability revealed that COVID-19 management can be achieved via the cooperation of both the medical and health care management sectors for maximum treatment efficacy. The model is available online for clinical implementation.

Framework for the development and evaluation of complex interventions: gap analysis, workshop and consultation-informed update

BACKGROUND

The Medical Research Council published the second edition of its framework in 2006 on developing and evaluating complex interventions. Since then, there have been considerable developments in the field of complex intervention research. The objective of this project was to update the framework in the light of these developments. The framework aims to help research teams prioritise research questions and design, and conduct research with an appropriate choice of methods, rather than to provide detailed guidance on the use of specific methods.

METHODS

There were four stages to the update: (1) gap analysis to identify developments in the methods and practice since the previous framework was published; (2) an expert workshop of 36 participants to discuss the topics identified in the gap analysis; (3) an open consultation process to seek comments on a first draft of the new framework; and (4) findings from the previous stages were used to redraft the framework, and final expert review was obtained. The process was overseen by a Scientific Advisory Group representing the range of relevant National Institute for Health Research and Medical Research Council research investments.

RESULTS

Key changes to the previous framework include (1) an updated definition of complex interventions, highlighting the dynamic relationship between the intervention and its context; (2) an emphasis on the use of diverse research perspectives: efficacy, effectiveness, theory-based and systems perspectives; (3) a focus on the usefulness of evidence as the basis for determining research perspective and questions; (4) an increased focus on interventions developed outside research teams, for example changes in policy or health services delivery; and (5) the identification of six 'core elements' that should guide all phases of complex intervention research: consider context; develop, refine and test programme theory; engage stakeholders; identify key uncertainties; refine the intervention; and economic considerations. We divide the research process into four phases: development, feasibility, evaluation and implementation. For each phase we provide a concise summary of recent developments, key points to address and signposts to further reading. We also present case studies to illustrate the points being made throughout.

LIMITATIONS

The framework aims to help research teams prioritise research questions and design and conduct research with an appropriate choice of methods, rather than to provide detailed guidance on the use of specific methods. In many of the areas of innovation that we highlight, such as the use of systems approaches, there are still only a few practical examples. We refer to more specific and detailed guidance where available and note where promising approaches require further development.

CONCLUSIONS

This new framework incorporates developments in complex intervention research published since the previous edition was written in 2006. As well as taking account of established practice and recent refinements, we draw attention to new approaches and place greater emphasis on economic considerations in complex intervention research. We have introduced a new emphasis on the importance of context and the value of understanding interventions as 'events in systems' that produce effects through interactions with features of the contexts in which they are implemented. The framework adopts a pluralist approach, encouraging researchers and research funders to adopt diverse research perspectives and to select research questions and methods pragmatically, with the aim of providing evidence that is useful to decision-makers.

FUTURE WORK

We call for further work to develop relevant methods and provide examples in practice. The use of this framework should be monitored and the move should be made to a more fluid resource in the future, for example a web-based format that can be frequently updated to incorporate new material and links to emerging resources.

FUNDING

This project was jointly funded by the Medical Research Council (MRC) and the National Institute for Health Research (Department of Health and Social Care 73514).

Appraisal of patient-level health economic models of severe mental illness: systematic review

BACKGROUND

Healthcare decision makers require accurate long-term economic models to evaluate the cost-effectiveness of new mental health interventions.

AIMS

To assess the suitability of current patient-level economic models to estimate long-term economic outcomes in severe mental illness.

METHOD

We undertook pre-specified systematic searches in MEDLINE, Embase and PsycINFO to identify reviews and stand-alone publications of economic models of interventions for schizophrenia, bipolar disorder and major depressive disorder (PROSPERO: CRD42020158243). We screened paper titles and abstracts to identify unique patient-level economic models. We conducted a structured extraction of identified models, recording the presence of key predefined model features. Model quality and validation were appraised using the 2014 ISPOR and 2016 AdViSHE model checklists.

RESULTS

We identified 15 unique patient-level models for psychosis and major depressive disorder from 1481 non-duplicate records. Models addressed schizophrenia (n = 6), bipolar disorder (n = 2) and major depressive disorder (n = 7). The predominant model type was discrete event simulation (n = 9). Model complexity and incorporation of patient heterogeneity varied considerably, and only five models extrapolated costs and outcomes over a lifetime horizon. Key model parameters were often based on low-quality evidence, and checklist quality assessment revealed weak model verification procedures.

CONCLUSIONS

Existing patient-level economic models of interventions for severe mental illness have considerable limitations. New modelling efforts must be supplemented by the generation of good-quality, contemporary evidence suitable for model building. Combined effort across the research community is required to build and validate economic extrapolation models suitable for accurately assessing the long-term value of new interventions from short-term clinical trial data.

Users' Guide to Medical Decision Analysis

Clinicians regularly have to trade benefits and harms to choose between testing and treatment strategies. This process is often done by making global and implicit judgments. A decision analysis is an analytic method that makes this process more explicit, reproducible, and evidence-based. While clinicians are unlikely to conduct their own decision analysis, they will read publications of such analyses or use guidelines based on them. This review outlines the anatomy of a decision tree and provides clinicians with the tools to critically appraise a decision analysis and apply its results to medical decision making. Clinicians reading about a decision analysis can make two judgments. The first judgment is about the credibility of the methods, such as whether the decision analysis addressed a relevant clinical question, included all important outcomes, used the current best evidence to derive variables in the model, and adopted the appropriate time horizon. The second judgment is about rating confidence in the preferred course of action by determining the certainty in the model variables, whether the results are robust in sensitivity analyses and if the results are applicable to a specific patient. Results from a valid and robust decision analysis can inform both guideline panels and the patient-clinician dyad engaged in shared decision-making.

Cost Effectiveness of Ranibizumab vs Aflibercept vs Bevacizumab for the Treatment of Macular Oedema Due to Central Retinal Vein Occlusion: The LEAVO Study

BACKGROUND

We aimed to assess the cost effectiveness of intravitreal ranibizumab (Lucentis), aflibercept (Eylea) and bevacizumab (Avastin) for the treatment of macular oedema due to central retinal vein occlusion.

METHODS

We calculated costs and quality-adjusted life-years from the UK National Health Service and Personal Social Services perspective. We performed a within-trial analysis using the efficacy, safety, resource use and health utility data from a randomised controlled trial (LEAVO) over 100 weeks. We built a discrete event simulation to model long-term outcomes. We estimated utilities using the Visual-Functioning Questionnaire-Utility Index, EQ-5D and EQ-5D with an additional vision question. We used standard UK costs sources for 2018/19 and a cost of £28 per bevacizumab injection. We discounted costs and quality-adjusted life-years at 3.5% annually.

RESULTS

Bevacizumab was the least costly intervention followed by ranibizumab and aflibercept in both the within-trial analysis (bevacizumab: £6292, ranibizumab: £13,014, aflibercept: £14,328) and long-term model (bevacizumab: £18,353, ranibizumab: £30,226, aflibercept: £35,026). Although LEAVO did not demonstrate bevacizumab to be non-inferior for the visual acuity primary outcome, the three interventions generated similar quality-adjusted life-years in both analyses. Bevacizumab was always the most cost-effective intervention at a threshold of £30,000 per quality-adjusted life-year, even using the list price of £243 per injection.

CONCLUSIONS

Wider adoption of bevacizumab for the treatment of macular oedema due to central retinal vein occlusion could result in substantial savings to healthcare systems and deliver similar health-related quality of life. However, patients, funders and ophthalmologists should be fully aware that LEAVO could not demonstrate that bevacizumab is non-inferior to the licensed agents.

Patient Flows, Patient Distribution Computations and Medicines Accounting in the Pharmacoeconomic Models Through Procurement Perspective

Purpose

Stimulating cost reduction of pharmaceutical companies to optimize the structure of distribution of patients by the level of treatment costs in various programs.

Patients and Methods

In this article, we rise up the issues of pharmacoeconomic modeling related to the description of the patient flows in the pharmacoeconomic model and methods to determining the course dose of drugs under the restriction of integer computations. We established two possible ways of distributing patients through treatment regimens in pharmacoeconomic models, also analyzed the effects of simultaneous and uniform entry of patients into the model. Also, we considered the limitations and possibilities of calculations based on the active substance and packaging, as well as the transition factor of the remainder of the drug in the next time period.

Results

A mathematical model of the analysis of the system assessment of patients by the level of risk of abandoning a healthy lifestyle in connection with the growing problems of the difficult-to-control process is developed. The use of a rational data convolution mode allowed us to obtain a criterion for the optimality of the process and a logical point of stability of the pharmaceutical company by rationally applying treatment methods according to established standards (percentage base). This approach makes it possible to influence the management of private clinics through clear ideas on the algorithms for prescribing drugs in each group of patients and their zoning in the vector recovery mode.

Conclusion

Initial data and sample size: 552 measurements of the intervals of changes in the subject's indicators in seconds (smoothing and scaling the data to the level of the base (analytical) period or the final (barrier) period). Regular use of this approach makes it possible to reserve the resources of the body of a healthy and physically active person in a timely manner for a very reliable functioning of all body systems, taking into account the dosed intake of prescribed drugs and the conditions of comfortable (decent) maintenance of patients during the course of treatment according to the method chosen by the doctor.

Developing a reference protocol for structured expert elicitation in health-care decision-making: a mixed-methods study

BACKGROUND

Many decisions in health care aim to maximise health, requiring judgements about interventions that may have higher health effects but potentially incur additional costs (cost-effectiveness framework). The evidence used to establish cost-effectiveness is typically uncertain and it is important that this uncertainty is characterised. In situations in which evidence is uncertain, the experience of experts is essential. The process by which the beliefs of experts can be formally collected in a quantitative manner is structured expert elicitation. There is heterogeneity in the existing methodology used in health-care decision-making. A number of guidelines are available for structured expert elicitation; however, it is not clear if any of these are appropriate for health-care decision-making.

OBJECTIVES

The overall aim was to establish a protocol for structured expert elicitation to inform health-care decision-making. The objectives are to (1) provide clarity on methods for collecting and using experts' judgements, (2) consider when alternative methodology may be required in particular contexts, (3) establish preferred approaches for elicitation on a range of parameters, (4) determine which elicitation methods allow experts to express uncertainty and (5) determine the usefulness of the reference protocol developed.

METHODS

A mixed-methods approach was used: systemic review, targeted searches, experimental work and narrative synthesis. A review of the existing guidelines for structured expert elicitation was conducted. This identified the approaches used in existing guidelines (the 'choices') and determined if dominant approaches exist. Targeted review searches were conducted for selection of experts, level of elicitation, fitting and aggregation, assessing accuracy of judgements and heuristics and biases. To sift through the available choices, a set of principles that underpin the use of structured expert elicitation in health-care decision-making was defined using evidence generated from the targeted searches, quantities to elicit experimental evidence and consideration of constraints in health-care decision-making. These principles, including fitness for purpose and reflecting individual expert uncertainty, were applied to the set of choices to establish a reference protocol. An applied evaluation of the developed reference protocol was also undertaken.

RESULTS

For many elements of structured expert elicitation, there was a lack of consistency across the existing guidelines. In almost all choices, there was a lack of empirical evidence supporting recommendations, and in some circumstances the principles are unable to provide sufficient justification for discounting particular choices. It is possible to define reference methods for health technology assessment. These include a focus on gathering experts with substantive skills, eliciting observable quantities and individual elicitation of beliefs. Additional considerations are required for decision-makers outside health technology assessment, for example at a local level, or for early technologies. Access to experts may be limited and in some circumstances group discussion may be needed to generate a distribution.

LIMITATIONS

The major limitation of the work conducted here lies not in the methods employed in the current work but in the evidence available from the wider literature relating to how appropriate particular methodological choices are.

CONCLUSIONS

The reference protocol is flexible in many choices. This may be a useful characteristic, as it is possible to apply this reference protocol across different settings. Further applied studies, which use the choices specified in this reference protocol, are required.

FUNDING

This project was funded by the NIHR Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 25, No. 37. See the NIHR Journals Library website for further project information. This work was also funded by the Medical Research Council (reference MR/N028511/1).

Quantitative Evidence Synthesis Methods for the Assessment of the Effectiveness of Treatment Sequences for Clinical and Economic Decision Making: A Review and Taxonomy of Simplifying Assumptions

Sequential use of alternative treatments for chronic conditions represents a complex intervention pathway; previous treatment and patient characteristics affect both the choice and effectiveness of subsequent treatments. This paper critically explores the methods for quantitative evidence synthesis of the effectiveness of sequential treatment options within a health technology assessment (HTA) or similar process. It covers methods for developing summary estimates of clinical effectiveness or the clinical inputs for the cost-effectiveness assessment and can encompass any disease condition. A comprehensive review of current approaches is presented, which considers meta-analytic methods for assessing the clinical effectiveness of treatment sequences and decision-analytic modelling approaches used to evaluate the effectiveness of treatment sequences. Estimating the effectiveness of a sequence of treatments is not straightforward or trivial and is severely hampered by the limitations of the evidence base. Randomised controlled trials (RCTs) of sequences were often absent or very limited. In the absence of sufficient RCTs of whole sequences, there is no single best way to evaluate treatment sequences; however, some approaches could be re-used or adapted, sharing ideas across different disease conditions. Each has advantages and disadvantages, and is influenced by the evidence available, extent of treatment sequences (number of treatment lines or permutations), and complexity of the decision problem. Due to the scarcity of data, modelling studies applied simplifying assumptions to data on discrete treatments. A taxonomy for all possible assumptions was developed, providing a unique resource to aid the critique of existing decision-analytic models.

Quantifying Dynamic Flow of Emergency Department (ED) Patient Managements: A Multistate Model Approach

Background

Emergency department (ED) crowding and prolonged lengths of stay continue to be important medical issues. It is difficult to apply traditional methods to analyze multiple streams of the ED patient management process simultaneously. The aim of this study was to develop a statistical model to delineate the dynamic patient flow within the ED and to analyze the effects of relevant factors on different patient movement rates.

Methods

This study used a retrospective cohort available with electronic medical data. Important time points and relevant covariates of all patients between January and December 2013 were collected. A new five-state Markov model was constructed by an expert panel, including three intermediate states: triage, physician management, and observation room and two final states: admission and discharge. A day was further divided into four six-hour periods to evaluate dynamics of patient movement over time.

Results

A total of 149,468 patient records were analyzed with a median total length of stay being 2.12 (interquartile range = 6.51) hours. The patient movement rates between states were estimated, and the effects of the age group and triage level on these movements were also measured. Patients with lower acuity go home more quickly (relative rate (RR): 1.891, 95% CI: 1.881–1.900) but have to wait longer for physicians (RR: 0.962, 95% CI: 0.956–0.967) and admission beds (RR: 0.673, 95% CI: 0.666–0.679). While older patients were seen more quickly by physicians (RR: 1.134, 95% CI: 1.131–1.139), they spent more time waiting for the final state (for admission RR: 0.830, 95% CI: 0.821–0.839; for discharge RR: 0.773, 95% CI: 0.769–0.776). Comparing the differences in patient movement rates over a 24-hour day revealed that patients wait longer before seen by physicians during the evening and that they usually move from the ED to admission afternoon. Predictive dynamic illustrations show that six hours after the patients' entry, the probability of still in the ED system ranges from 28% in the evening to 38% in the morning.

Conclusions

The five-state model well described the dynamic ED patient flow and analyzed the effects of relevant influential factors at different states. The model can be used in similar medical settings or incorporate different important covariates to develop individually tailored approaches for the improvement of efficiency within the health professions.