Emulator-based Bayesian calibration of the CISNET colorectal cancer models

Purpose

To calibrate Cancer Intervention and Surveillance Modeling Network (CISNET) 's SimCRC, MISCAN-Colon, and CRC-SPIN simulation models of the natural history colorectal cancer (CRC) with an emulator-based Bayesian algorithm and internally validate the model-predicted outcomes to calibration targets.

Methods

We used Latin hypercube sampling to sample up to 50,000 parameter sets for each CISNET-CRC model and generated the corresponding outputs. We trained multilayer perceptron artificial neural networks (ANN) as emulators using the input and output samples for each CISNET-CRC model. We selected ANN structures with corresponding hyperparameters (i.e., number of hidden layers, nodes, activation functions, epochs, and optimizer) that minimize the predicted mean square error on the validation sample. We implemented the ANN emulators in a probabilistic programming language and calibrated the input parameters with Hamiltonian Monte Carlo-based algorithms to obtain the joint posterior distributions of the CISNET-CRC models' parameters. We internally validated each calibrated emulator by comparing the model-predicted posterior outputs against the calibration targets.

Results

The optimal ANN for SimCRC had four hidden layers and 360 hidden nodes, MISCAN-Colon had 4 hidden layers and 114 hidden nodes, and CRC-SPIN had one hidden layer and 140 hidden nodes. The total time for training and calibrating the emulators was 7.3, 4.0, and 0.66 hours for SimCRC, MISCAN-Colon, and CRC-SPIN, respectively. The mean of the model-predicted outputs fell within the 95% confidence intervals of the calibration targets in 98 of 110 for SimCRC, 65 of 93 for MISCAN, and 31 of 41 targets for CRC-SPIN.

Conclusions

Using ANN emulators is a practical solution to reduce the computational burden and complexity for Bayesian calibration of individual-level simulation models used for policy analysis, like the CISNET CRC models. In this work, we present a step-by-step guide to constructing emulators for calibrating three realistic CRC individual-level models using a Bayesian approach.

Black-White disparities in colorectal cancer outcomes: a simulation study of screening benefit

The US Black population has higher colorectal cancer (CRC) incidence rates and worse CRC survival than the US White population, as well as historically lower rates of CRC screening. The Surveillance, Epidemiology, and End Results incidence rate data in people diagnosed between the ages of 20 and 45 years, before routine CRC screening is recommended, were analyzed to estimate temporal changes in CRC risk in Black and White populations. There was a rapid rise in rectal and distal colon cancer incidence in the White population but not the Black population, and little change in proximal colon cancer incidence for both groups. In 2014-2018, CRC incidence per 100 000 was 17.5 (95% confidence interval [CI] = 15.3 to 19.9) among Black individuals aged 40-44 years and 16.6 (95% CI = 15.6 to 17.6) among White individuals aged 40-44 years; 42.3% of CRCs diagnosed in Black patients were proximal colon cancer, and 41.1% of CRCs diagnosed in White patients were rectal cancer. Analyses used a race-specific microsimulation model to project screening benefits, based on life-years gained and lifetime reduction in CRC incidence, assuming these Black-White differences in CRC risk and location. The projected benefits of screening (via either colonoscopy or fecal immunochemical testing) were greater in the Black population, suggesting that observed Black-White differences in CRC incidence are not driven by differences in risk. Projected screening benefits were sensitive to survival assumptions made for Black populations. Building racial disparities in survival into the model reduced projected screening benefits, which can bias policy decisions.

Identifying Optimal Methods for Addressing Confounding Bias When Estimating the Effects of State-level Policies

BACKGROUND

Policy evaluation studies that assess how state-level policies affect health-related outcomes are foundational to health and social policy research. The relative ability of newer analytic methods to address confounding, a key source of bias in observational studies, has not been closely examined.

METHODS

We conducted a simulation study to examine how differing magnitudes of confounding affected the performance of 4 methods used for policy evaluations: (1) the two-way fixed effects difference-in-differences model; (2) a 1-period lagged autoregressive model; (3) augmented synthetic control method; and (4) the doubly robust difference-in-differences approach with multiple time periods from Callaway-Sant'Anna. We simulated our data to have staggered policy adoption and multiple confounding scenarios (i.e., varying the magnitude and nature of confounding relationships).

RESULTS

Bias increased for each method: (1) as confounding magnitude increases; (2) when confounding is generated with respect to prior outcome trends (rather than levels), and (3) when confounding associations are nonlinear (rather than linear). The autoregressive model and augmented synthetic control method had notably lower root mean squared error than the two-way fixed effects and Callaway-Sant'Anna approaches for all scenarios; the exception is nonlinear confounding by prior trends, where Callaway-Sant'Anna excels. Coverage rates were unreasonably high for the augmented synthetic control method (e.g., 100%), reflecting large model-based standard errors and wide confidence intervals in practice.

CONCLUSIONS

In our simulation study, no single method consistently outperformed the others, but a researcher's toolkit should include all methodologic options. Our simulations and associated R package can help researchers choose the most appropriate approach for their data.

Projected long-term effects of colorectal cancer screening disruptions following the COVID-19 pandemic

The aftermath of the initial phase of the COVID-19 pandemic may contribute to the widening of disparities in colorectal cancer (CRC) outcomes due to differential disruptions to CRC screening. This comparative microsimulation analysis uses two CISNET CRC models to simulate the impact of ongoing screening disruptions induced by the COVID-19 pandemic on long-term CRC outcomes. We evaluate three channels through which screening was disrupted: delays in screening, regimen switching, and screening discontinuation. The impact of these disruptions on long-term CRC outcomes was measured by the number of life-years lost due to CRC screening disruptions compared to a scenario without any disruptions. While short-term delays in screening of 3-18 months are predicted to result in minor life-years loss, discontinuing screening could result in much more significant reductions in the expected benefits of screening. These results demonstrate that unequal recovery of screening following the pandemic can widen disparities in CRC outcomes and emphasize the importance of ensuring equitable recovery to screening following the pandemic.

Robustness Analysis of Colorectal Cancer Colonoscopy Screening Strategies

Colorectal Cancer (CRC) is a leading cause of cancer deaths in the United States. Despite significant overall declines in CRC incidence and mortality, there has been an alarming increase in CRC among people younger than 50. This study uses an established microsimulation model, CRC-SPIN, to perform a 'stress test' of colonoscopy screening strategies. First, we expand CRC-SPIN to include birth-cohort effects. Second, we estimate natural history model parameters via Incremental Mixture Approximate Bayesian Computation (IMABC) for two model versions to characterize uncertainty while accounting for increased early CRC onset. Third, we simulate 26 colonoscopy screening strategies across the posterior distribution of estimated model parameters, assuming four different colonoscopy sensitivities (104 total scenarios). We find that model projections of screening benefit are highly dependent on natural history and test sensitivity assumptions, but in this stress test, the policy recommendations are robust to the uncertainties considered.

Optimal non-pharmaceutical pandemic response strategies depend critically on time horizons and costs

The COVID-19 pandemic has called for swift action from local governments, which have instated non-pharmaceutical interventions (NPIs) to curb the spread of the disease. The swift implementation of social distancing policies has raised questions about the costs and benefits of strategies that either aim to keep cases as low as possible (suppression) or aim to reach herd immunity quickly (mitigation) to tackle the COVID-19 pandemic. While curbing COVID-19 required blunt instruments, it is unclear whether a less-transmissible and less-deadly emerging pathogen would justify the same response. This paper illuminates this question using a parsimonious transmission model by formulating the social distancing lives vs. livelihoods dilemma as a boundary value problem using calculus of variations. In this setup, society balances the costs and benefits of social distancing contingent on the costs of reducing transmission relative to the burden imposed by the disease. We consider both single-objective and multi-objective formulations of the problem. To the best of our knowledge, our approach is distinct in the sense that strategies emerge from the problem structure rather than being imposed a priori. We find that the relative time-horizon of the pandemic (i.e., the time it takes to develop effective vaccines and treatments) and the relative cost of social distancing influence the choice of the optimal policy. Unsurprisingly, we find that the appropriate policy response depends on these two factors. We discuss the conditions under which each policy archetype (suppression vs. mitigation) appears to be the most appropriate.

Unequal Recovery in Colorectal Cancer Screening Following the COVID-19 Pandemic: A Comparative Microsimulation Analysis

The aftermath of the initial phase of the COVID-19 pandemic may contribute to the widening of disparities in access to colorectal cancer (CRC) screening due to differential disruptions to CRC screening. This comparative microsimulation analysis uses two CISNET CRC models to simulate the impact of ongoing screening disruptions induced by the COVID-19 pandemic on long-term CRC outcomes. We evaluate three channels through which screening was disrupted: delays in screening, regimen switching, and screening discontinuation. The impact of these disruptions on long-term colorectal cancer (CRC) outcomes was measured by the number of Life-years lost due to CRC screening disruptions compared to a scenario without any disruptions. While short-term delays in screening of 3-18 months are predicted to result in minor life-years loss, discontinuing screening could result in much more significant reductions in the expected benefits of screening. These results demonstrate that unequal recovery of screening following the pandemic can widen disparities in colorectal cancer outcomes and emphasize the importance of ensuring equitable recovery to screening following the pandemic.

Reopening Under Uncertainty: Stress-Testing California's COVID-19 Exit Strategy

The coronavirus disease 2019 pandemic required significant public health interventions from local governments. Early in the pandemic, RAND researchers developed a decision support tool to provide policymakers with insight into the trade-offs they might face when choosing among nonpharmaceutical intervention levels. Using an updated version of the model, the researchers performed a stress-test of a variety of alternative reopening plans, using California as an example. This article presents the general lessons learned from these experiments and discusses four characteristics of the best reopening strategies.

Too Good to Be True? Evaluation of Colonoscopy Sensitivity Assumptions Used in Policy Models

BACKGROUND

Models can help guide colorectal cancer (CRC) screening policy. While models are carefully calibrated and validated, there is less scrutiny of assumptions about test performance.

METHODS

We examined the validity of the CRC-SPIN model and colonoscopy sensitivity assumptions. Standard sensitivity assumptions, consistent with published decision analyses, assume sensitivity equal to 0.75 for diminutive adenomas (<6mm), 0.85 for small adenomas (6-10mm), 0.95 for large adenomas ( {greater than or equal to} 10mm), and 0.95 for preclinical cancer. We also selected adenoma sensitivity that resulted in more accurate predictions. Targets were drawn from the Wheat Bran Fiber study. We examined how well the model predicted outcomes measured over a three-year follow-up period, including: the number of adenomas detected, the size of the largest adenoma detected, and incident CRC.

RESULTS

Using standard sensitivity assumptions, the model predicted adenoma prevalence that was too low (42.5% versus 48.9% observed, with 95% confidence interval 45.3%-50.7%) and detection of too few large adenomas (5.1% versus 14.% observed, with 95% confidence interval 11.8%-17.4%). Predictions were close to targets when we set sensitivities to 0.20 for diminutive adenomas, 0.60 for small adenomas, 0.80 for 10-20mm adenomas, and 0.98 for adenomas 20mm and larger.

CONCLUSIONS

Colonoscopy may be less accurate than currently assumed, especially for diminutive adenomas. Alternatively, the CRC-SPIN model may not accurately simulate onset and progression of adenomas in higher-risk populations.

IMPACT

Misspecification of either colonoscopy sensitivity or disease progression in high-risk populations may impact the predicted effectiveness of CRC screening. When possible, decision analyses used to inform policy should address these uncertainties.

Reopening California: Seeking robust, non-dominated COVID-19 exit strategies

The COVID-19 pandemic required significant public health interventions from local governments. Although nonpharmaceutical interventions often were implemented as decision rules, few studies evaluated the robustness of those reopening plans under a wide range of uncertainties. This paper uses the Robust Decision Making approach to stress-test 78 alternative reopening strategies, using California as an example. This study uniquely considers a wide range of uncertainties and demonstrates that seemingly sensible reopening plans can lead to both unnecessary COVID-19 deaths and days of interventions. We find that plans using fixed COVID-19 case thresholds might be less effective than strategies with time-varying reopening thresholds. While we use California as an example, our results are particularly relevant for jurisdictions where vaccination roll-out has been slower. The approach used in this paper could also prove useful for other public health policy problems in which policymakers need to make robust decisions in the face of deep uncertainty.

Reopening California : Seeking Robust, Non-Dominated COVID-19 Exit Strategies

Amid global scarcity of COVID-19 vaccines and the threat of new variant strains, California and other jurisdictions face the question of when and how to implement and relax COVID-19 Nonpharmaceutical Interventions (NPIs). While policymakers have attempted to balance the health and economic impacts of the pandemic, decentralized decision-making, deep uncertainty, and the lack of widespread use of comprehensive decision support methods can lead to the choice of fragile or inefficient strategies. This paper uses simulation models and the Robust Decision Making (RDM) approach to stress-test California's reopening strategy and other alternatives over a wide range of futures. We find that plans which respond aggressively to initial outbreaks are required to robustly control the pandemic. Further, the best plans adapt to changing circumstances, lowering their stringent requirements to reopen over time or as more constituents are vaccinated. While we use California as an example, our results are particularly relevant for jurisdictions where vaccination roll-out has been slower.

Modeling COVID-19 Nonpharmaceutical Interventions: Exploring periodic NPI strategies

We developed a COVID-19 transmission model used as part of RAND's web-based COVID-19 decision support tool that compares the effects of nonpharmaceutical public health interventions (NPIs) on health and economic outcomes. An interdisciplinary approach informed the selection and use of multiple NPIs, combining quantitative modeling of the health/economic impacts of interventions with qualitative assessments of other important considerations (e.g., cost, ease of implementation, equity). This paper provides further details of our model, describes extensions, presents sensitivity analyses, and analyzes strategies that periodically switch between a base NPI level and a higher NPI level. We find that a periodic strategy, if implemented with perfect compliance, could have produced similar health outcomes as static strategies but might have produced better outcomes when considering other measures of social welfare. Our findings suggest that there are opportunities to shape the tradeoffs between economic and health outcomes by carefully evaluating a more comprehensive range of reopening policies.

Could periodic nonpharmaceutical intervention strategies produce better COVID-19 health and economic outcomes?

We developed a COVID-19 transmission model to compare the effects of nonpharmaceutical public health interventions (NPIs) on health and economic outcomes. An interdisciplinary approach informed the selection and use of multiple NPIs, combining quantitative modeling of the health and economic impacts of interventions with qualitative assessments of other important considerations (e.g., cost, ease of implementation, equity). We used our model to analyzed strategies that periodically switch between a base NPI and a high NPI level. We find that this systematic strategy could have produced similar health outcomes as static strategies but better social welfare and economic outcomes. Our findings suggest that there are opportunities to shape the tradeoffs between economic and health outcomes by carefully evaluating a more comprehensive range of reopening policies.

Inducing Brazilian manufacturing SMEs productivity with Lean tools

Purpose

The purpose of this paper is to present a method for assisting micro and small companies of the industrial sector with the adoption of Lean practices.

Design/methodology/approach

The paper outlines the method construction steps, which used a design science research approach.

Findings

This research led to the structuring of a method for implementing Lean Manufacturing tools in micro and small companies of the industrial sector. The developed method contributed to the knowledge in Lean Manufacturing by systematizing its tools in a heuristic approach that can be applied to an operation using overall equipment effectiveness (OEE) as a guiding indicator.

Practical implications

This method can be used to guide the implementation of Lean tools in SMEs industries.

Originality/value

The originality of this paper lies in the adoption of an operation-focused approach only (rather than an approach that begins with the mapping of an entire process) and the use of OEE as the basis for prioritization of improvements to be performed and operational control.

Lean office e BPM: proposição e aplicação de método para a redução de desperdícios em áreas administrativas

A necessidade de eliminar desperdícios em áreas administrativas amplifica-se à medida que aumenta a competitividade nos setores, pressionando assim a redução de custos e despesas fixas. Neste contexto, além de mostrarem-se relevantes para as operações diretas, os conceitos e ferramentas Lean também ganham importância no escritório e em áreas de apoio. Apesar disto, os métodos disponíveis para a utilização do Lean em áreas administrativas carecem de integração com técnicas consolidadas pela literatura em Gerenciamento de Processos de Negócio. Frente a necessidade das organizações em eliminar desperdícios, e às lacunas identificadas na literatura existente sobre Lean Office, o presente artigo apresenta a proposição de um método para a implantação do Lean em áreas administrativas, incorporando o uso de técnicas de Gerenciamento de Processos. Além disto, este trabalho apresenta a aplicação do método no Sindicato das Indústrias Metalúrgica, Mecânica e de Material Elétrico e Eletrônico de São Leopoldo - SINDIMETAL. Para este objetivo, o trabalho revisa os conceitos Lean e de Gerenciamento de Processos de Negócio, utilizando como método de pesquisa a Design Science Research. Por fim são discutidas implicações a partir destas proposições bem como perspectivas para o desenvolvimento do Lean como alternativa para melhoria em operações administrativas.