Is probabilistic bias analysis approximately Bayesian?

Does recall bias explain the association of mood disorders with workplace harassment?

Purpose

To determine the contribution of recall bias to the observed excess in mental ill-health in those reporting harassment at work.

Methods

A prospective cohort of 1885 workers in welding and electrical trades was contacted every six months for up to 5 years, asking whether they were currently anxious or depressed and whether this was made worse by work. Only at the end of the study did we ask about any workplace harassment they had experienced at work. We elicited sensitivity and specificity of self-reported bullying from published reliability studies and formulated priors that reflect the possibility of over-reporting of workplace harassment (exposure) by those whose anxiety or depression was reported to be made worse by work (cases). We applied the resulting misclassification models to probabilistic bias analysis (PBA) of relative risks.

Results

We observe that PBA implies that it is unlikely that biased misclassification due to the study subjects' states of mind could have caused the entire observed association. Indeed, the results demonstrated that doubling of risk of anxiety or depression following workplace harassment is plausible, with the unadjusted relative risk attenuated with understated uncertainty.

Conclusions

It seems unlikely that risk of anxiety or depression following workplace harassment can be explained by the form of recall bias that we proposed.

Does adjustment for non-differential misclassification of dichotomous exposure induce positive bias if there is no true association?

This article is a response to an off-the-record discussion that I had at an international meeting of epidemiologists more than decade ago. It centered on a concern, perhaps widely spread, that adjustment for exposure misclassification can induce a false positive result. I trace the possible history of this supposition and test it in a simulated case-control study under the assumption of non-differential misclassification of binary exposure, in which a Bayesian adjustment is applied. Probabilistic bias analysis is also briefly considered. The main conclusion is that adjustment for the presumed non-differential exposure misclassification of dichotomous does not "induce" positive associations, especially if the focus of the interpretation of the result is taken away from the point estimate. The misconception about positive bias induced by adjustment for exposure misclassification, if more clearly explained during the training of epidemiologists, may promote appropriate (and wider) use of the adjustment techniques. The simple message that can be derived from this paper is: "Exposure misclassification as a tractable problem that deserves much more attention than just a typical qualitative throw-away discussion".

Bias analyses to investigate the impact of differential participation: Application to a birth defects case-control study

BACKGROUND

Certain associations observed in the National Birth Defects Prevention Study (NBDPS) contrasted with other research or were from areas with mixed findings, including no decrease in odds of spina bifida with periconceptional folic acid supplementation, moderately increased cleft palate odds with ondansetron use and reduced hypospadias odds with maternal smoking.

OBJECTIVES

To investigate the plausibility and extent of differential participation to produce effect estimates observed in NBDPS.

METHODS

We searched the literature for factors related to these exposures and participation and conducted deterministic quantitative bias analyses. We estimated case-control participation and expected exposure prevalence based on internal and external reports, respectively. For the folic acid-spina bifida and ondansetron-cleft palate analyses, we hypothesized the true odds ratio (OR) based on prior studies and quantified the degree of exposure over- (or under-) representation to produce the crude OR (cOR) in NBDPS. For the smoking-hypospadias analysis, we estimated the extent of selection bias needed to nullify the association as well as the maximum potential harmful OR.

RESULTS

Under our assumptions (participation, exposure prevalence, true OR), there was overrepresentation of folic acid use and underrepresentation of ondansetron use and smoking among participants. Folic acid-exposed spina bifida cases would need to have been ≥1.2× more likely to participate than exposed controls to yield the observed null cOR. Ondansetron-exposed cleft palate cases would need to have been 1.6× more likely to participate than exposed controls if the true OR is null. Smoking-exposed hypospadias cases would need to have been ≥1.2 times less likely to participate than exposed controls for the association to falsely appear protective (upper bound of selection bias adjusted smoking-hypospadias OR = 2.02).

CONCLUSIONS

Differential participation could partly explain certain associations observed in NBDPS, but questions remain about why. Potential impacts of other systematic errors (e.g. exposure misclassification) could be informed by additional research.

SAS and R code for probabilistic quantitative bias analysis for misclassified binary variables and binary unmeasured confounders

Systematic error from selection bias, uncontrolled confounding, and misclassification is ubiquitous in epidemiologic research but is rarely quantified using quantitative bias analysis (QBA). This gap may in part be due to the lack of readily modifiable software to implement these methods. Our objective is to provide computing code that can be tailored to an analyst's dataset. We briefly describe the methods for implementing QBA for misclassification and uncontrolled confounding and present the reader with example code for how such bias analyses, using both summary-level data and individual record-level data, can be implemented in both SAS and R. Our examples show how adjustment for uncontrolled confounding and misclassification can be implemented. Resulting bias-adjusted point estimates can then be compared to conventional results to see the impact of this bias in terms of its direction and magnitude. Further, we show how 95% simulation intervals can be generated that can be compared to conventional 95% confidence intervals to see the impact of the bias on uncertainty. Having easy to implement code that users can apply to their own datasets will hopefully help spur more frequent use of these methods and prevent poor inferences drawn from studies that do not quantify the impact of systematic error on their results.

Impact of Alcohol Consumption on Multiple Sclerosis Using Model-based Standardization and Misclassification Adjustment Via Probabilistic Bias Analysis

BACKGROUND

The etiology of multiple sclerosis (MS) is still not well-demonstrated, and assessment of some risk factors like alcohol consumption has problems like confounding and measurement bias. To determine the causal effect of alcohol consumption on MS after adjusting for alcohol consumption misclassification bias and confounders.

METHODS

In a population-based incident case-control study, 547 patients with MS and 1057 healthy people were recruited. A minimally sufficient adjustment set of confounders was derived using the causal directed acyclic graph. The probabilistic bias analysis method (PBAM) using beta, logit-logistic, and triangular probability distributions for sensitivity/specificity to adjust for misclassification bias in self-reporting alcohol consumption and model-based standardization (MBS) to estimate the causal effect of alcohol consumption were used. Population attributable fraction (PAF) estimates with 95% Monte Carlo sensitivity analysis (MCSA) intervals were calculated using PBAM and MBS analysis. Bootstrap was used to deal with random errors.

RESULTS

The adjusted risk ratio (95% MCSA interval) from the probabilistic bias analysis and MBS between alcohol consumption and MS using the three distribution was in the range of 1.93 (1.07 to 4.07) to 2.02 (1.15 to 4.69). The risk difference (RD) in all three scenarios was 0.0001 (0.0000 to 0.0005) and PAF was in the range of 0.15 (0.010 to 0.50) to 0.17 (0.001 to 0.47).

CONCLUSION

After adjusting for measurement bias, confounding, and random error alcohol consumption had a positive causal effect on the incidence of MS.

Effect of alcohol consumption on breast cancer: probabilistic bias analysis for adjustment of exposure misclassification bias and confounders

PURPOSE

This study was conducted to evaluate the effect of alcohol consumption on breast cancer, adjusting for alcohol consumption misclassification bias and confounders.

METHODS

This was a case-control study of 932 women with breast cancer and 1000 healthy control. Using probabilistic bias analysis method, the association between alcohol consumption and breast cancer was adjusted for the misclassification bias of alcohol consumption as well as a minimally sufficient set of adjustment of confounders derived from a causal directed acyclic graph. Population attributable fraction was estimated using the Miettinen's Formula.

RESULTS

Based on the conventional logistic regression model, the odds ratio estimate between alcohol consumption and breast cancer was 1.05 (95% CI: 0.57, 1.91). However, the adjusted estimates of odds ratio based on the probabilistic bias analysis ranged from 1.82 to 2.29 for non-differential and from 1.93 to 5.67 for differential misclassification. Population attributable fraction ranged from 1.51 to 2.57% using non-differential bias analysis and 1.54-3.56% based on differential bias analysis.

CONCLUSION

A marked measurement error was in self-reported alcohol consumption so after correcting misclassification bias, no evidence against independence between alcohol consumption and breast cancer changed to a substantial positive association.

The impact of omitting confounders in parallel process latent growth curve mediation models: Three sensitivity analysis approaches

Parallel process latent growth curve mediation models (PP-LGCMMs) are frequently used to longitudinally investigate the mediation effects of treatment on the level and change of outcome through the level and change of mediator. An important but often violated assumption in empirical PP-LGCMM analysis is the absence of omitted confounders of the relationships among treatment, mediator, and outcome. In this study, we analytically examined how omitting pretreatment confounders impacts the inference of mediation from the PP-LGCMM. Using the analytical results, we developed three sensitivity analysis approaches for the PP-LGCMM, including the frequentist, Bayesian, and Monte Carlo approaches. The three approaches help investigate different questions regarding the robustness of mediation results from the PP-LGCMM, and handle the uncertainty in the sensitivity parameters differently. Applications of the three sensitivity analyses are illustrated using a real-data example. A user-friendly Shiny web application is developed to conduct the sensitivity analyses.

Opioid-related Mortality in United States Death Certificate Data: A Quantitative Bias Analysis With Expert Elicitation of Bias Parameters

BACKGROUND

Opioid-related mortality is an important public health problem in the United States. Incidence estimates rely on death certificate data generated by health care providers and medical examiners. Opioid overdoses may be underreported when other causes of death appear plausible. We applied physician-elicited death certificate bias parameters to quantitative bias analyses assessing potential age-related differential misclassification in US opioid-related mortality estimates.

METHODS

We obtained cause-of-death data (US, 2017) from the National Center for Health Statistics and calculated crude opioid-related outpatient death counts by age category (25-54, 55-64, 65+). We elicited beliefs from 10 primary care physicians on sensitivity of opioid-related death classification from death certificates. We summarized elicited sensitivity estimates, calculated plausible specificity values, and applied resulting parameters in a probabilistic bias analysis.

RESULTS

Physicians estimated wide sensitivity ranges for classification of opioid-related mortality by death certificates, with lower estimated sensitivities among older age groups. Probabilistic bias analyses adjusting for physician-estimated misclassification indicated 3.1 times more (95% uncertainty interval: 1.2-23.5) opioid-related deaths than the observed death count in the 65+ age group. All age groups had substantial increases in bias-adjusted death counts.

CONCLUSIONS

We developed and implemented a feasible method of eliciting physician expert opinion on bias parameters for sensitivity of a medical record-based death indicator and applied findings in quantitative bias analyses adjusting for differential misclassification. Our findings are consistent with the hypothesis that opioid-related mortality rates may be substantially underestimated, particularly among older adults, due to misclassification in cause-of-death data from death certificates.

Accounting for Misclassification and Selection Bias in Estimating Effectiveness of Self-managed Medication Abortion

BACKGROUND

Studies on the effectiveness of self-managed medication abortion may suffer from misclassification and selection bias due to self-reported outcomes and loss of follow-up. Monte Carlo sensitivity analysis can estimate self-managed abortion effectiveness accounting for these potential biases.

METHODS

We conducted a Monte Carlo sensitivity analysis based on data from the Studying Accompaniment model Feasibility and Effectiveness Study (the SAFE Study), to generate bias-adjusted estimates of the effectiveness of self-managed abortion with accompaniment group support. Between July 2019 and April 2020, we enrolled a total of 1051 callers who contacted accompaniment groups in Argentina and Nigeria for self-managed abortion information; 961 took abortion medications and completed at least one follow-up. Using these data, we calculated measures of effectiveness adjusted for ineligibility, misclassification, and selection bias across 50,000 simulations with bias parameters drawn from pre-specified Beta distributions in R.

RESULTS

After accounting for the potential influence of various sources of bias, bias-adjusted estimates of effectiveness were similar to observed estimates, conditional on chosen bias parameters: 92.68% (95% simulation interval: 87.80%, 95.74%) for mifepristone in combination with misoprostol (versus 93.7% in the observed data) and 98.47% (95% simulation interval: 96.79%, 99.39%) for misoprostol alone (versus 99.3% in the observed data).

CONCLUSIONS

After adjustment for multiple potential sources of bias, estimates of self-managed medication abortion effectiveness remain high. Monte Carlo sensitivity analysis may be useful in studies measuring an epidemiologic proportion (i.e., effectiveness, prevalence, cumulative incidence) while accounting for possible selection or misclassification bias.

Sensitivity analysis for random measurement error using regression calibration and simulation-extrapolation

Objective

Sensitivity analysis for random measurement error can be applied in the absence of validation data by means of regression calibration and simulation-extrapolation. These have not been compared for this purpose.

Study design and setting

A simulation study was conducted comparing the performance of regression calibration and simulation-extrapolation for linear and logistic regression. The performance of the two methods was evaluated in terms of bias, mean squared error (MSE) and confidence interval coverage, for various values of reliability of the error-prone measurement (0.05-0.91), sample size (125-4000), number of replicates (2-10), and R-squared (0.03-0.75). It was assumed that no validation data were available about the error-free measures, while correct information about the measurement error variance was available.

Results

Regression calibration was unbiased while simulation-extrapolation was biased: median bias was 0.8% (interquartile range (IQR): -0.6;1.7%), and -19.0% (IQR: -46.4;-12.4%), respectively. A small gain in efficiency was observed for simulation-extrapolation (median MSE: 0.005, IQR: 0.004;0.006) versus regression calibration (median MSE: 0.006, IQR: 0.005;0.009). Confidence interval coverage was at the nominal level of 95% for regression calibration, and smaller than 95% for simulation-extrapolation (median coverage: 85%, IQR: 73;93%). The application of regression calibration and simulation-extrapolation for a sensitivity analysis was illustrated using an example of blood pressure and kidney function.

Conclusion

Our results support the use of regression calibration over simulation-extrapolation for sensitivity analysis for random measurement error.

Bias Analysis Gone Bad

Quantitative bias analysis comprises the tools used to estimate the direction, magnitude, and uncertainty from systematic errors affecting epidemiologic research. Despite the availability of methods and tools, and guidance for good practices, few reports of epidemiologic research incorporate quantitative estimates of bias impacts. The lack of familiarity with bias analysis allows for the possibility of misuse, which is likely most often unintentional but could occasionally include intentional efforts to mislead. We identified 3 examples of suboptimal bias analysis, one for each common bias. For each, we describe the original research and its bias analysis, compare the bias analysis with good practices, and describe how the bias analysis and research findings might have been improved. We assert no motive to the suboptimal bias analysis by the original authors. Common shortcomings in the examples were lack of a clear bias model, computed example, and computing code; poor selection of the values assigned to the bias model's parameters; and little effort to understand the range of uncertainty associated with the bias. Until bias analysis becomes more common, community expectations for the presentation, explanation, and interpretation of bias analyses will remain unstable. Attention to good practices should improve quality, avoid errors, and discourage manipulation.

Invited Commentary: Dealing With the Inevitable Deficiencies of Bias Analysis-and All Analyses

Lash et al. (Am J Epidemiol. 2021;190(8):1604-1612) have presented detailed critiques of 3 bias analyses that they identify as "suboptimal." This identification raises the question of what "optimal" means for bias analysis, because it is practically impossible to do statistically optimal analyses of typical population studies-with or without bias analysis. At best the analysis can only attempt to satisfy practice guidelines and account for available information both within and outside the study. One should not expect a full accounting for all sources of uncertainty; hence, interval estimates and distributions for causal effects should never be treated as valid uncertainty assessments-they are instead only example analyses that follow from collections of often questionable assumptions. These observations reinforce those of Lash et al. and point to the need for more development of methods for judging bias-parameter distributions and utilization of available information.

Bias in Self-reported Prepregnancy Weight Across Maternal and Clinical Characteristics

OBJECTIVES

Prepregnancy body mass index (BMI) and gestational weight gain (GWG) are known determinants of maternal and child health; calculating both requires an accurate measure of prepregnancy weight. We compared self-reported prepregnancy weight to measured weights to assess reporting bias by maternal and clinical characteristics.

METHODS

We conducted a retrospective cohort study among pregnant women using electronic health records (EHR) data from Kaiser Permanente Northwest, a non-profit integrated health care system in Oregon and southwest Washington State. We identified women age ≥ 18 years who were pregnant between 2000 and 2010 with self-reported prepregnancy weight, ≥ 2 measured weights between ≤ 365-days-prior-to and ≤ 42-days-after conception, and measured height in their EHR. We compared absolute and relative difference between self-reported weight and two "gold-standards": (1) weight measured closest to conception, and (2) usual weight (mean of weights measured 6-months-prior-to and ≤ 42-days-after conception). Generalized-estimating equations were used to assess predictors of misreport controlling for covariates, which were obtained from the EHR or linkage to birth certificate.

RESULTS

Among the 16,227 included pregnancies, close agreement (± 1 kg or ≤ 2%) between self-reported and closest-measured weight was 44% and 59%, respectively. Overall, self-reported weight averaged 1.3 kg (SD 3.8) less than measured weight. Underreporting was higher among women with elevated BMI category, late prenatal care entry, and pregnancy outcome other than live/stillbirth (p < .05). Using self-reported weight, BMI was correctly classified for 91% of pregnancies, but ranged from 70 to 98% among those with underweight or obesity, respectively. Results were similar using usual weight as gold standard. CONCLUSIONS FOR PRACTICE: Accurate measure of prepregnancy weight is essential for clinical guidance and surveillance efforts that monitor maternal health and evaluate public-health programs. Identification of characteristics associated with misreport of self-reported weight can inform understanding of bias when assessing the influence of prepregnancy BMI or GWG on health outcomes.

A systematic review of quantitative bias analysis applied to epidemiological research

BACKGROUND

Quantitative bias analysis (QBA) measures study errors in terms of direction, magnitude and uncertainty. This systematic review aimed to describe how QBA has been applied in epidemiological research in 2006-19.

METHODS

We searched PubMed for English peer-reviewed studies applying QBA to real-data applications. We also included studies citing selected sources or which were identified in a previous QBA review in pharmacoepidemiology. For each study, we extracted the rationale, methodology, bias-adjusted results and interpretation and assessed factors associated with reproducibility.

RESULTS

Of the 238 studies, the majority were embedded within papers whose main inferences were drawn from conventional approaches as secondary (sensitivity) analyses to quantity-specific biases (52%) or to assess the extent of bias required to shift the point estimate to the null (25%); 10% were standalone papers. The most common approach was probabilistic (57%). Misclassification was modelled in 57%, uncontrolled confounder(s) in 40% and selection bias in 17%. Most did not consider multiple biases or correlations between errors. When specified, bias parameters came from the literature (48%) more often than internal validation studies (29%). The majority (60%) of analyses resulted in >10% change from the conventional point estimate; however, most investigators (63%) did not alter their original interpretation. Degree of reproducibility related to inclusion of code, formulas, sensitivity analyses and supplementary materials, as well as the QBA rationale.

CONCLUSIONS

QBA applications were rare though increased over time. Future investigators should reference good practices and include details to promote transparency and to serve as a reference for other researchers.

Substantial underestimation of SARS-CoV-2 infection in the United States

Accurate estimates of the burden of SARS-CoV-2 infection are critical to informing pandemic response. Confirmed COVID-19 case counts in the U.S. do not capture the total burden of the pandemic because testing has been primarily restricted to individuals with moderate to severe symptoms due to limited test availability. Here, we use a semi-Bayesian probabilistic bias analysis to account for incomplete testing and imperfect diagnostic accuracy. We estimate 6,454,951 cumulative infections compared to 721,245 confirmed cases (1.9% vs. 0.2% of the population) in the United States as of April 18, 2020. Accounting for uncertainty, the number of infections during this period was 3 to 20 times higher than the number of confirmed cases. 86% (simulation interval: 64-99%) of this difference is due to incomplete testing, while 14% (0.3-36%) is due to imperfect test accuracy. The approach can readily be applied in future studies in other locations or at finer spatial scale to correct for biased testing and imperfect diagnostic accuracy to provide a more realistic assessment of COVID-19 burden.

Effect of Smoking on Breast Cancer by Adjusting for Smoking Misclassification Bias and Confounders Using a Probabilistic Bias Analysis Method

Purpose

The aim of this study was to determine the association between smoking and breast cancer after adjusting for smoking misclassification bias and confounders.

Methods

In this case–control study, 1000 women with breast cancer and 1000 healthy controls were selected. Using a probabilistic bias analysis method, the association between smoking and breast cancer was adjusted for the bias resulting from misclassification of smoking secondary to self-reporting as well as a minimally sufficient adjustment set of confounders derived from a causal directed acyclic graph (cDAG). Population attributable fraction (PAF) for smoking was calculated using Miettinen’s formula.

Results

While the odds ratio (OR) from the conventional logistic regression model between smoking and breast cancer was 0.64 (95% CI: 0.36–1.13), the adjusted ORs from the probabilistic bias analysis were in the ranges of 2.63–2.69 and 1.73–2.83 for non-differential and differential misclassification, respectively. PAF ranges obtained were 1.36–1.72% and 0.62–2.01% using the non-differential bias analysis and differential bias analysis, respectively.

Conclusion

After misclassification correction for smoking, the non-significant negative-adjusted association between smoking and breast cancer changed to a significant positive-adjusted association.

A Bayesian Sensitivity Analysis to Partition Body Mass Index Into Components of Body Composition: An Application to Head and Neck Cancer Survival

Previous studies have suggested a "J-shaped" relationship between body mass index (BMI, calculated as weight (kg)/height (m)2) and survival among head and neck cancer (HNC) patients. However, BMI is a vague measure of body composition. To provide greater resolution, we used Bayesian sensitivity analysis, informed by external data, to model the relationship between predicted fat mass index (FMI, adipose tissue (kg)/height (m)2), lean mass index (LMI, lean tissue (kg)/height (m)2), and survival. We estimated posterior median hazard ratios and 95% credible intervals for the BMI-mortality relationship in a Bayesian framework using data from 1,180 adults in North Carolina with HNC diagnosed between 2002 and 2006. Risk factors were assessed by interview shortly after diagnosis and vital status through 2013 via the National Death Index. The relationship between BMI and all-cause mortality was convex, with a nadir at 28.6, with greater risk observed throughout the normal weight range. The sensitivity analysis indicated that this was consistent with opposing increases in risk with FMI (per unit increase, hazard ratio = 1.04 (1.00, 1.08)) and decreases with LMI (per unit increase, hazard ratio = 0.90 (0.85, 0.95)). Patterns were similar for HNC-specific mortality but associations were stronger. Measures of body composition, rather than BMI, should be considered in relation to mortality risk.

The impact of maternal smoking during pregnancy on childhood asthma: adjusted for exposure misclassification; results from the National Health and Nutrition Examination Survey, 2011-2012

PURPOSE

We sought to examine the association between childhood asthma and self-reported maternal smoking during pregnancy (MSDP) after adjusting for a range of exposure misclassification scenarios using a Bayesian approach that incorporated exposure misclassification probability estimates from the literature.

METHODS

Self-reported MSDP and asthma data were extracted from National Health and Nutrition Examination Survey 2011-2012. The association between self-reported MSDP and asthma was adjusted for exposure misclassification using a Bayesian bias model approach.

RESULTS

We included 3074 subjects who were 1-15 years of age, including 492 asthma cases. The mean (SD) of age of the participants was 8.5 (4.1) and 7.1 (4.2) years and the number (percentage) of female was 205 (42%) and 1314 (51%) among asthmatic and nonasthmatic groups, respectively. The odds ratio (OR) for the association between self-reported MSDP and asthma in logistic regression adjusted for confounders was 1.28 (95% confidence interval: 0.92, 1.77). In a Bayesian analysis that adjusted for exposure misclassification using external data, we found different ORs between MSDP and asthma by applying different priors (posterior ORs 0.90 [95% credible interval {CRI}: 0.47, 1.60] to 3.05 [95% CRI: 1.73, 5.53] in differential and 1.22 [CRI 95%: 0.62, 2.25] to 1.60 CRI: 1.18, 2.19) in nondifferential misclassification settings.

CONCLUSIONS

Given the assumptions and the accuracy of the bias model, the estimated effect of MSDP on asthma after adjusting for misclassification was strengthened in many scenarios.

Multiple bias analysis using logistic regression: an example from the National Birth Defects Prevention Study

PURPOSE

Exposure misclassification, selection bias, and confounding are important biases in epidemiologic studies, yet only confounding is routinely addressed quantitatively. We describe how to combine two previously described methods and adjust for multiple biases using logistic regression.

METHODS

Weights were created from selection probabilities and predictive values for exposure classification and applied to multivariable logistic regression models in a case-control study of prepregnancy obesity (body mass index ≥30 vs. <30 kg/m²) and cleft lip with or without cleft palate (CL/P) using data from the National Birth Defects Prevention Study (2523 cases, 10,605 controls).

RESULTS

Adjusting for confounding by race/ethnicity, prepregnancy obesity, and CL/P were weakly associated (odds ratio [OR]: 1.10; 95% confidence interval: 0.98, 1.23). After weighting the data to account for exposure misclassification, missing exposure data, selection bias, and confounding, multiple bias-adjusted ORs ranged from 0.94 to 1.03 in nonprobabilistic bias analyses and median multiple bias-adjusted ORs ranged from 0.93 to 1.02 in probabilistic analyses.

CONCLUSIONS

This approach, adjusting for multiple biases using a logistic regression model, suggested that the observed association between obesity and CL/P could be due to the presence of bias.

Hierarchical Semi-Bayes Methods for Misclassification in Perinatal Epidemiology

BACKGROUND

Validation data are used to estimate the extent of misclassification in epidemiologic studies. In the Penn MOMS cohort, prepregnancy body mass index is subject to misclassification, and validation data are available to estimate the extent of misclassification. We use these data to estimate the association between maternal prepregnancy body mass index and early preterm (<32 weeks) birth using a semi-Bayes hierarchical model, allowing for more flexible adjustment for misclassification.

METHODS

We propose a two-stage model that first fits a Bayesian hierarchical model for the bias parameters in the validation study. This model shrinks bias parameters in different groups toward one another in an effort to gain precision and improve mean squared error. In the second stage, we draw random samples from the posterior distribution of the bias parameters to implement a probabilistic bias analysis adjusting for exposure misclassification in a frequentist outcome model.

RESULTS

Bias parameters from the hierarchical model were often more substantively reasonable and often had smaller variance. Adjusting results for misclassification generally attenuated the strength of the unadjusted associations. After adjusting for misclassification, underweight mothers were not at increased risk of early preterm birth relative to normal weight mothers. Severely obese mothers had an increased risk of early preterm birth relative to normal weight mothers.

CONCLUSIONS

The two-stage semi-Bayesian hierarchical model borrowed strength between group-specific bias parameters to adjust for exposure misclassification. Model results support evidence of an increased risk of early preterm birth among severely obese mothers, relative to normal weight mothers.

Applying quantitative bias analysis to estimate the plausible effects of selection bias in a cluster randomised controlled trial: secondary analysis of the Primary care Osteoarthritis Screening Trial (POST)

Background

Selection bias is a concern when designing cluster randomised controlled trials (c-RCT). Despite addressing potential issues at the design stage, bias cannot always be eradicated from a trial design. The application of bias analysis presents an important step forward in evaluating whether trial findings are credible. The aim of this paper is to give an example of the technique to quantify potential selection bias in c-RCTs.

Methods

This analysis uses data from the Primary care Osteoarthritis Screening Trial (POST). The primary aim of this trial was to test whether screening for anxiety and depression, and providing appropriate care for patients consulting their GP with osteoarthritis would improve clinical outcomes. Quantitative bias analysis is a seldom-used technique that can quantify types of bias present in studies. Due to lack of information on the selection probability, probabilistic bias analysis with a range of triangular distributions was also used, applied at all three follow-up time points; 3, 6, and 12 months post consultation. A simple bias analysis was also applied to the study.

Results

Worse pain outcomes were observed among intervention participants than control participants (crude odds ratio at 3, 6, and 12 months: 1.30 (95% CI 1.01, 1.67), 1.39 (1.07, 1.80), and 1.17 (95% CI 0.90, 1.53), respectively). Probabilistic bias analysis suggested that the observed effect became statistically non-significant if the selection probability ratio was between 1.2 and 1.4. Selection probability ratios of > 1.8 were needed to mask a statistically significant benefit of the intervention.

Conclusions

The use of probabilistic bias analysis in this c-RCT suggested that worse outcomes observed in the intervention arm could plausibly be attributed to selection bias. A very large degree of selection of bias was needed to mask a beneficial effect of intervention making this interpretation less plausible.

The Harm Done to Reproducibility by the Culture of Null Hypothesis Significance Testing

In the last few years, stakeholders in the scientific community have raised alarms about a perceived lack of reproducibility of scientific results. In reaction, guidelines for journals have been promulgated and grant applicants have been asked to address the rigor and reproducibility of their proposed projects. Neither solution addresses a primary culprit, which is the culture of null hypothesis significance testing that dominates statistical analysis and inference. In an innovative research enterprise, selection of results for further evaluation based on null hypothesis significance testing is doomed to yield a low proportion of reproducible results and a high proportion of effects that are initially overestimated. In addition, the culture of null hypothesis significance testing discourages quantitative adjustments to account for systematic errors and quantitative incorporation of prior information. These strategies would otherwise improve reproducibility and have not been previously proposed in the widely cited literature on this topic. Without discarding the culture of null hypothesis significance testing and implementing these alternative methods for statistical analysis and inference, all other strategies for improving reproducibility will yield marginal gains at best.

On the Need for Quantitative Bias Analysis in the Peer-Review Process

Peer review is central to the process through which epidemiologists generate evidence to inform public health and medical interventions. Reviewers thereby act as critical gatekeepers to high-quality research. They are asked to carefully consider the validity of the proposed work or research findings by paying careful attention to the methodology and critiquing the importance of the insight gained. However, although many have noted problems with the peer-review system for both manuscripts and grant submissions, few solutions have been proposed to improve the process. Quantitative bias analysis encompasses all methods used to quantify the impact of systematic error on estimates of effect in epidemiologic research. Reviewers who insist that quantitative bias analysis be incorporated into the design, conduct, presentation, and interpretation of epidemiologic research could substantially strengthen the process. In the present commentary, we demonstrate how quantitative bias analysis can be used by investigators and authors, reviewers, funding agencies, and editors. By utilizing quantitative bias analysis in the peer-review process, editors can potentially avoid unnecessary rejections, identify key areas for improvement, and improve discussion sections by shifting from speculation on the impact of sources of error to quantification of the impact those sources of bias may have had.

Cytochrome P-450 2D6 (CYP2D6) Genotype and Breast Cancer Recurrence in Tamoxifen-Treated Patients: Evaluating the Importance of Loss of Heterozygosity

Tamoxifen therapy for estrogen receptor-positive breast cancer reduces the risk of recurrence by approximately one-half. Cytochrome P-450 2D6, encoded by the polymorphic cytochrome P-450 2D6 gene (CYP2D6), oxidizes tamoxifen to its most active metabolites. Steady-state concentrations of endoxifen (4-hydroxy-N-desmethyltamoxifen), the most potent antiestrogenic metabolite, are reduced in women whose CYP2D6 genotypes confer poor enzyme function. Thirty-one studies of the association of CYP2D6 genotype with breast cancer survival have yielded heterogeneous results. Some influential studies genotyped DNA from tumor-infiltrated tissues, and their results may have been susceptible to germline genotype misclassification from loss of heterozygosity at the CYP2D6 locus. We systematically reviewed 6 studies of concordance between genotypes obtained from paired nonneoplastic and breast tumor-infiltrated tissues, all of which showed excellent CYP2D6 genotype agreement. We applied these concordance data to a quantitative bias analysis of the subset of the 31 studies that were based on genotypes from tumor-infiltrated tissue to examine whether genotyping errors substantially biased estimates of association. The bias analysis showed negligible bias by discordant genotypes. Summary estimates of association, with or without bias adjustment, indicated no clinically important association between CYP2D6 genotype and breast cancer survival in tamoxifen-treated women.

A probabilistic bias analysis for misclassified categorical exposures, with application to oral anti-hyperglycaemic drugs

PURPOSE

The effect of drug exposure misclassification generally receives little attention in pharmacoepidemiological research. In this paper, we illustrate a probabilistic bias analysis approach for misclassified categorical exposures and apply it in a database study of oral anti-hyperglycaemic drugs (OADs).

METHODS

A cohort study based on the Health Search Database general-practice database was carried out by including 12 640 adult (≥40 years) patients newly treated with OADs during 2003-2010. The proportion of days covered by OADs prescriptions during the first year of follow-up was evaluated for each individual, either by means of the prescribed daily dose or the defined daily dose. The effect of misclassification on hypothetical OAD-outcome association profiles was assessed through the proposed probabilistic bias analysis approach, taking advantage of available exposure validation data.

RESULTS

During the first year of follow-up, the average (SD) number of months with OADs available was 7 (4) months and 5 (3) months according to the prescribed daily dose and defined daily dose metrics, respectively. Probabilistic bias analysis results based on validation data suggest that the effect of misclassification is complex, as conventional exposure-outcome association estimates may be of greater or lower magnitude than their misclassification-adjusted values.

CONCLUSIONS

Misclassification should be taken into account in database studies on the safety of prescribed medications. To this aim, investigators should take advantage of external exposure validation data in sensitivity analysis approaches such as ours. Copyright © 2016 John Wiley & Sons, Ltd.

Probabilistic bias analysis in pharmacoepidemiology and comparative effectiveness research: a systematic review

PURPOSE

We systematically reviewed pharmacoepidemiologic and comparative effectiveness studies that use probabilistic bias analysis to quantify the effects of systematic error including confounding, misclassification, and selection bias on study results.

METHODS

We found articles published between 2010 and October 2015 through a citation search using Web of Science and Google Scholar and a keyword search using PubMed and Scopus. Eligibility of studies was assessed by one reviewer. Three reviewers independently abstracted data from eligible studies.

RESULTS

Fifteen studies used probabilistic bias analysis and were eligible for data abstraction-nine simulated an unmeasured confounder and six simulated misclassification. The majority of studies simulating an unmeasured confounder did not specify the range of plausible estimates for the bias parameters. Studies simulating misclassification were in general clearer when reporting the plausible distribution of bias parameters. Regardless of the bias simulated, the probability distributions assigned to bias parameters, number of simulated iterations, sensitivity analyses, and diagnostics were not discussed in the majority of studies.

CONCLUSION

Despite the prevalence and concern of bias in pharmacoepidemiologic and comparative effectiveness studies, probabilistic bias analysis to quantitatively model the effect of bias was not widely used. The quality of reporting and use of this technique varied and was often unclear. Further discussion and dissemination of the technique are warranted. Copyright © 2016 John Wiley & Sons, Ltd.

Validity of birth certificate-derived maternal weight data in twin pregnancies

Birth certificates are an important source of pre-pregnancy body mass index (BMI) and gestational weight gain (GWG) data for surveillance and aetiologic studies, but little is known about their validity in twin pregnancies. Twins experience high rates of adverse perinatal outcomes that have been associated with BMI and GWG in singletons. Our objective was to evaluate the accuracy of birth certificate-derived pre-pregnancy BMI and GWG compared with medical record-derived data in a sample of 186 twin pregnancies at a teaching hospital in Pennsylvania (2003-2010). Twelve strata were created by simultaneous stratification on pre-pregnancy BMI (underweight, normal weight/overweight, obese class 1, obese classes 2 and 3) and GWG (<20th, 20-80th, >80th percentile). The agreement of birth certificate-derived pre-pregnancy BMI category with medical record BMI category was lowest among underweight mothers [75% (95% confidence interval 51-91%) ] and highest among normal/overweight [97% (90-99%) ] and obese classes 2 and 3 mothers [97% (85-99%) ]. Agreement for GWG category from the birth certificate varied from 57% (41-70%) for GWG >80th percentile to 80% (65-91%) and 82% (72-89%) for GWG <20th and 20th-80th percentiles, respectively. The misclassification of BMI and GWG was primarily due to error in pre-pregnancy weight rather than weight at delivery or height. Agreement proportions for twins were not meaningfully different from the proportions in a comparable sample of singleton pregnancies. These data suggest that birth certificate-based BMI and GWG data are prone to error in twin pregnancies. Those who use these data should conduct internal validation studies and adjust their results using bias analyses.

Robust bayesian sensitivity analysis for case-control studies with uncertain exposure misclassification probabilities

Exposure misclassification in case-control studies leads to bias in odds ratio estimates. There has been considerable interest recently to account for misclassification in estimation so as to adjust for bias as well as more accurately quantify uncertainty. These methods require users to elicit suitable values or prior distributions for the misclassification probabilities. In the event where exposure misclassification is highly uncertain, these methods are of limited use, because the resulting posterior uncertainty intervals tend to be too wide to be informative. Posterior inference also becomes very dependent on the subjectively elicited prior distribution. In this paper, we propose an alternative "robust Bayesian" approach, where instead of eliciting prior distributions for the misclassification probabilities, a feasible region is given. The extrema of posterior inference within the region are sought using an inequality constrained optimization algorithm. This method enables sensitivity analyses to be conducted in a useful way as we do not need to restrict all of our unknown parameters to fixed values, but can instead consider ranges of values at a time.

Methods to Explore Uncertainty and Bias Introduced by Job Exposure Matrices

Job exposure matrices (JEMs) are used to measure exposures based on information about particular jobs and tasks. JEMs are especially useful when individual exposure data cannot be obtained. Nonetheless, there may be other workplace exposures associated with the study disease that are not measured in available JEMs. When these exposures are also associated with the exposures measured in the JEM, biases due to uncontrolled confounding will be introduced. Furthermore, individual exposures differ from JEM measurements due to differences in job conditions and worker practices. Uncertainty may also be present at the assessor level since exposure information for each job may be imprecise or incomplete. Assigning individuals a fixed exposure determined by the JEM ignores these uncertainty sources. We examine the uncertainty displayed by bias analyses in a study of occupational electric shocks, occupational magnetic fields, and amyotrophic lateral sclerosis.

Quantifying and Adjusting for Disease Misclassification Due to Loss to Follow-Up in Historical Cohort Mortality Studies

The purpose of this analysis was to quantify and adjust for disease misclassification from loss to follow-up in a historical cohort mortality study of workers where exposure was categorized as a multi-level variable. Disease classification parameters were defined using 2008 mortality data for the New Zealand population and the proportions of known deaths observed for the cohort. The probability distributions for each classification parameter were constructed to account for potential differences in mortality due to exposure status, gender, and ethnicity. Probabilistic uncertainty analysis (bias analysis), which uses Monte Carlo techniques, was then used to sample each parameter distribution 50,000 times, calculating adjusted odds ratios (ORDM-LTF) that compared the mortality of workers with the highest cumulative exposure to those that were considered never-exposed. The geometric mean ORDM-LTF ranged between 1.65 (certainty interval (CI): 0.50-3.88) and 3.33 (CI: 1.21-10.48), and the geometric mean of the disease-misclassification error factor (εDM-LTF), which is the ratio of the observed odds ratio to the adjusted odds ratio, had a range of 0.91 (CI: 0.29-2.52) to 1.85 (CI: 0.78-6.07). Only when workers in the highest exposure category were more likely than those never-exposed to be misclassified as non-cases did the ORDM-LTF frequency distributions shift further away from the null. The application of uncertainty analysis to historical cohort mortality studies with multi-level exposures can provide valuable insight into the magnitude and direction of study error resulting from losses to follow-up.

Misclassification in administrative claims data: quantifying the impact on treatment effect estimates

Misclassification is present in nearly every epidemiologic study, yet is rarely quantified in analysis in favor of a focus on random error. In this review, we discuss past and present wisdom on misclassification and what measures should be taken to quantify this influential bias, with a focus on bias in pharmacoepidemiologic studies. To date, pharmacoepidemiology primarily utilizes data obtained from administrative claims, a rich source of prescription data but susceptible to bias from unobservable factors including medication sample use, medications filled but not taken, health conditions that are not reported in the administrative billing data, and inadequate capture of confounders. Due to the increasing focus on comparative effectiveness research, we provide a discussion of misclassification in the context of an active comparator, including a demonstration of treatment effects biased away from the null in the presence of nondifferential misclassification. Finally, we highlight recently developed methods to quantify bias and offer these methods as potential options for strengthening the validity and quantifying uncertainty of results obtained from pharmacoepidemiologic research.

Comparison of bias analysis strategies applied to a large data set

BACKGROUND

Epidemiologic data sets continue to grow larger. Probabilistic-bias analyses, which simulate hundreds of thousands of replications of the original data set, may challenge desktop computational resources.

METHODS

We implemented a probabilistic-bias analysis to evaluate the direction, magnitude, and uncertainty of the bias arising from misclassification of prepregnancy body mass index when studying its association with early preterm birth in a cohort of 773,625 singleton births. We compared 3 bias analysis strategies: (1) using the full cohort, (2) using a case-cohort design, and (3) weighting records by their frequency in the full cohort.

RESULTS

Underweight and overweight mothers were more likely to deliver early preterm. A validation substudy demonstrated misclassification of prepregnancy body mass index derived from birth certificates. Probabilistic-bias analyses suggested that the association between underweight and early preterm birth was overestimated by the conventional approach, whereas the associations between overweight categories and early preterm birth were underestimated. The 3 bias analyses yielded equivalent results and challenged our typical desktop computing environment. Analyses applied to the full cohort, case cohort, and weighted full cohort required 7.75 days and 4 terabytes, 15.8 hours and 287 gigabytes, and 8.5 hours and 202 gigabytes, respectively.

CONCLUSIONS

Large epidemiologic data sets often include variables that are imperfectly measured, often because data were collected for other purposes. Probabilistic-bias analysis allows quantification of errors but may be difficult in a desktop computing environment. Solutions that allow these analyses in this environment can be achieved without new hardware and within reasonable computational time frames.

Misclassification in administrative claims data: quantifying the impact on treatment effect estimates

Misclassification is present in nearly every epidemiologic study, yet is rarely quantified in analysis in favor of a focus on random error. In this review, we discuss past and present wisdom on misclassification and what measures should be taken to quantify this influential bias, with a focus on bias in pharmacoepidemiologic studies. To date, pharmacoepidemiology primarily utilizes data obtained from administrative claims, a rich source of prescription data but susceptible to bias from unobservable factors including medication sample use, medications filled but not taken, health conditions that are not reported in the administrative billing data, and inadequate capture of confounders. Due to the increasing focus on comparative effectiveness research, we provide a discussion of misclassification in the context of an active comparator, including a demonstration of treatment effects biased away from the null in the presence of nondifferential misclassification. Finally, we highlight recently developed methods to quantify bias and offer these methods as potential options for strengthening the validity and quantifying uncertainty of results obtained from pharmacoepidemiologic research.

Validity of birth certificate-derived maternal weight data

BACKGROUND

Studies using vital records-based maternal weight data have become more common, but the validity of these data is uncertain.

METHODS

We evaluated the accuracy of prepregnancy body mass index (BMI) and gestational weight gain (GWG) reported on birth certificates using medical record data in 1204 births at a teaching hospital in Pennsylvania from 2003 to 2010. Deliveries at this hospital were representative of births statewide with respect to BMI, GWG, race/ethnicity, and preterm birth. Forty-eight strata were created by simultaneous stratification on prepregnancy BMI (underweight, normal weight/overweight, obese class 1, obese classes 2 and 3), GWG (<20th, 20-80th, >80th percentile), race/ethnicity (non-Hispanic white, non-Hispanic black), and gestational age (term, preterm).

RESULTS

The agreement of birth certificate-derived prepregnancy BMI category with medical record BMI category was highest in the normal weight/overweight and obese class 2 and 3 groups. Agreement varied from 52% to 100% across racial/ethnic and gestational age strata. GWG category from the birth registry agreed with medical records for 41-83% of deliveries, and agreement tended to be the poorest for very low and very high GWG. The misclassification of GWG was driven by errors in reported prepregnancy weight rather than maternal weight at delivery, and its magnitude depended on prepregnancy BMI category and gestational age at delivery.

CONCLUSIONS

Maternal weight data, particularly at the extremes, are poorly reported on birth certificates. Investigators should devote resources to well-designed validation studies, the results of which can be used to adjust for measurement errors by bias analysis.

What do measures of agreement (κ) tell us about quality of exposure assessment? Theoretical analysis and numerical simulation

BACKGROUND

The reliability of binary exposure classification methods is routinely reported in occupational health literature because it is viewed as an important component of evaluating the trustworthiness of the exposure assessment by experts. The Kappa statistics (κ) are typically employed to assess how well raters or classification systems agree in a variety of contexts, such as identifying exposed participants in a population-based epidemiological study of risks due to occupational exposures. However, the question we are really interested in is not so much the reliability of an exposure assessment method, although this holds value in itself, but the validity of the exposure estimates. The validity of binary classifiers can be expressed as a method's sensitivity (SN) and specificity (SP), estimated from its agreement with the error-free classifier.

METHODS AND RESULTS

We describe a simulation-based method for deriving information on SN and SP that can be derived from κ and the prevalence of exposure, since an analytic solution is not possible without restrictive assumptions. This work is illustrated in the context of comparison of job-exposure matrices assessing occupational exposures to polycyclic aromatic hydrocarbons.

DISCUSSION

Our approach allows the investigators to evaluate how good their exposure-assessment methods truly are, not just how well they agree with each other, and should lead to incorporation of information of validity of expert assessment methods into formal uncertainty analyses in epidemiology.

Bias analysis to improve monitoring an HIV epidemic and its response: approach and application to a survey of female sex workers in Iran

BACKGROUND

We present probabilistic and Bayesian techniques to correct for bias in categorical and numerical measures and empirically apply them to a recent survey of female sex workers (FSW) conducted in Iran.

METHODS

We used bias parameters from a previous validation study to correct estimates of behaviours reported by FSW. Monte-Carlo Sensitivity Analysis and Bayesian bias analysis produced point and simulation intervals (SI).

RESULTS

The apparent and corrected prevalence differed by a minimum of 1% for the number of 'non-condom use sexual acts' (36.8% vs 35.8%) to a maximum of 33% for 'ever associated with a venue to sell sex' (35.5% vs 68.0%). The negative predictive value of the questionnaire for 'history of STI' and 'ever associated with a venue to sell sex' was 36.3% (95% SI 4.2% to 69.1%) and 46.9% (95% SI 6.3% to 79.1%), respectively. Bias-adjusted numerical measures of behaviours increased by 0.1 year for 'age at first sex act for money' to 1.5 for 'number of sexual contacts in last 7 days'.

CONCLUSIONS

The 'true' estimates of most behaviours are considerably higher than those reported and the related SIs are wider than conventional CIs. Our analysis indicates the need for and applicability of bias analysis in surveys, particularly in stigmatised settings.