Case Definition and Design Sensitivity

Differential recall bias in estimating treatment effects in observational studies

Observational studies are frequently used to estimate the effect of an exposure or treatment on an outcome. To obtain an unbiased estimate of the treatment effect, it is crucial to measure the exposure accurately. A common type of exposure misclassification is recall bias, which occurs in retrospective cohort studies when study subjects may inaccurately recall their past exposure. Particularly challenging is differential recall bias in the context of self-reported binary exposures, where the bias may be directional rather than random and its extent varies according to the outcomes experienced. This paper makes several contributions: (1) it establishes bounds for the average treatment effect even when a validation study is not available; (2) it proposes multiple estimation methods across various strategies predicated on different assumptions; and (3) it suggests a sensitivity analysis technique to assess the robustness of the causal conclusion, incorporating insights from prior research. The effectiveness of these methods is demonstrated through simulation studies that explore various model misspecification scenarios. These approaches are then applied to investigate the effect of childhood physical abuse on mental health in adulthood.

Quantitative bias analysis in practice: review of software for regression with unmeasured confounding

BACKGROUND

Failure to appropriately account for unmeasured confounding may lead to erroneous conclusions. Quantitative bias analysis (QBA) can be used to quantify the potential impact of unmeasured confounding or how much unmeasured confounding would be needed to change a study's conclusions. Currently, QBA methods are not routinely implemented, partly due to a lack of knowledge about accessible software. Also, comparisons of QBA methods have focused on analyses with a binary outcome.

METHODS

We conducted a systematic review of the latest developments in QBA software published between 2011 and 2021. Our inclusion criteria were software that did not require adaption (i.e., code changes) before application, was still available in 2022, and accompanied by documentation. Key properties of each software tool were identified. We provide a detailed description of programs applicable for a linear regression analysis, illustrate their application using two data examples and provide code to assist researchers in future use of these programs.

RESULTS

Our review identified 21 programs with [Formula: see text] created post 2016. All are implementations of a deterministic QBA with [Formula: see text] available in the free software R. There are programs applicable when the analysis of interest is a regression of binary, continuous or survival outcomes, and for matched and mediation analyses. We identified five programs implementing differing QBAs for a continuous outcome: treatSens, causalsens, sensemakr, EValue, and konfound. When applied to one of our illustrative examples, causalsens incorrectly indicated sensitivity to unmeasured confounding whereas the other four programs indicated robustness. sensemakr performs the most detailed QBA and includes a benchmarking feature for multiple unmeasured confounders.

CONCLUSIONS

Software is now available to implement a QBA for a range of different analyses. However, the diversity of methods, even for the same analysis of interest, presents challenges to their widespread uptake. Provision of detailed QBA guidelines would be highly beneficial.

The association between critical shoulder angle and revision following anatomic total shoulder arthroplasty: a matched case-control study

HYPOTHESIS

The concept of the critical shoulder angle (CSA) was introduced in 2013, with studies showing that larger CSA is associated with rotator cuff tears (RCTs) and smaller CSA with glenohumeral osteoarthritis. We hypothesized outcomes following total shoulder arthroplasty (TSA) would differ depending on CSA.

METHODS

We conducted a matched case-control study using Kaiser Permanente's Shoulder Arthroplasty Registry to identify patients who underwent primary elective anatomic TSA for the diagnosis of osteoarthritis from 2009-2018. Seventy-eight adult patients who underwent revision following the primary TSA due to glenoid component failure or rotator cuff tear comprised the case group. A control group of nonrevised patients were identified from the same source population. Two controls were matched to each case by age, gender, body mass index, American Society of Anesthesiologists classification, surgeon who performed the index TSA, and post-TSA follow-up time. The relationship between revision and CSA as measured on radiographs were analyzed as a 1:2 matched-pairs case-control study with use of multiple conditional multivariable logistic regression.

RESULTS

Revised cases had a higher likelihood of a CSA ≥35° (odds ratio [OR] = 2.41, 95% confidence interval [CI] = 1.27-4.59). A higher likelihood of CSA ≥35° was observed for those revised for glenoid loosening (OR = 4.58, 95% CI = 1.20-17.50) and revised for rotator cuff tear (OR = 2.41, 95% CI = 1.18-4.92) compared with nonrevised controls. Every 5° increase in CSA had higher odds of overall revision (OR = 1.62, 95% CI = 1.18-2.21), glenoid loosening (OR = 2.50, 95% CI = 1.27-4.92), and rotator cuff tear (OR = 1.51, 95% CI = 1.07-2.14).

CONCLUSION

In a matched case-control study of primary anatomic TSA, individuals who were revised for aseptic glenoid loosening and superior cuff failure had a higher CSA compared with nonrevised individuals. These data suggest that surgeons may consider using reverse arthroplasty in cases of primary shoulder arthritis with a CSA of 35° or greater.

A conditional test with demonstrated insensitivity to unmeasured bias in matched observational studies

In an observational study matched for observed covariates, an association between treatment received and outcome exhibited may indicate not an effect caused by the treatment, but merely some bias in the allocation of treatments to individuals within matched pairs. The evidence that distinguishes moderate biases from causal effects is unevenly dispersed among possible comparisons in an observational study: some comparisons are insensitive to larger biases than others. Intuitively, larger treatment effects tend to be insensitive to larger unmeasured biases, and perhaps matched pairs can be grouped using covariates, doses or response patterns so that groups of pairs with larger treatment effects may be identified. Even if an investigator has a reasoned conjecture about where to look for insensitive comparisons, that conjecture might prove mistaken, or, when not mistaken, it might be received sceptically by other scientists who doubt the conjecture or judge it to be too convenient in light of its success with the data at hand. In this article a test is proposed that searches for insensitive findings over many comparisons, but controls the probability of falsely rejecting a true null hypothesis of no treatment effect in the presence of a bias of specified magnitude. An example is studied in which the test considers many comparisons and locates an interpretable comparison that is insensitive to larger biases than a conventional comparison based on Wilcoxon’s signed rank statistic applied to all pairs. A simulation examines the power of the proposed test. The method is implemented in the R package dstat, which contains the example and reproduces the analysis.

EVIDENCE FACTORS IN A CASE-CONTROL STUDY WITH APPLICATION TO THE EFFECT OF FLEXIBLE SIGMOIDOSCOPY SCREENING ON COLORECTAL CANCER

As in any observational study, in a case-control study a primary concern is potential unmeasured confounders. Bias, due to unmeasured confounders, can result in a false discovery of an apparent treatment effect when there is none. Replication of an observational study, which tries to provide multiple analyses of the data where the biases affecting each analysis are thought to be different, is one way to strengthen the evidence from an observational study. Evidence factors allow for internal replication by testing a hypothesis using multiple comparisons in a way that the comparisons yield independent evidence and differ in the sources of potential bias. We construct evidence factors in a case-control study in which there are two types of cases, "narrow" cases which are thought to be potentially more affected by the exposure and "marginal" cases which are thought to have more heterogeneous causes. We develop and study an inference procedure for using such evidence factors and apply it to a study of the effect of sigmoidoscopy screening on colorectal cancer.

The Association Between Tibial Slope and Revision Anterior Cruciate Ligament Reconstruction in Patients ≤21 Years Old: A Matched Case-Control Study Including 317 Revisions

BACKGROUND

There is evidence that tibial slope may play a role in revision risk after anterior cruciate ligament reconstruction (ACLR); however, prior studies are inconsistent.

PURPOSE

To determine (1) whether there is a difference in lateral tibial posterior slope (LTPS) or medial tibial posterior slope (MTPS) between patients undergoing revised ACLR and those not requiring revision and (2) whether the medial-to-lateral slope difference is different between these 2 groups.

STUDY DESIGN

Case-control study; Level of evidence, 3.

METHODS

We conducted a matched case-control study (2006-2015). Cases were patients aged ≤21 years who underwent revision surgery after primary unilateral ACLR; controls were patients aged ≤21 years without revision who were identified from the same source population. Controls were matched to cases by age, sex, body mass index, race, graft type, femoral fixation device, and post-ACLR follow-up time. Tibial slope measurements were made by a single blinded reviewer using magnetic resonance imaging. The Wilcoxon signed rank test and McNemar test were used for continuous and categorical variables, respectively.

RESULTS

No difference was observed between revised and nonrevised ACLR groups for LTPS (median: 6° vs 6°, P = .973) or MTPS (median: 4° vs 5°, P = .281). Furthermore, no difference was found for medial-to-lateral slope difference (median: -1 vs -1, P = .289). A greater proportion of patients with revised ACLR had an LTPS ≥12° (7.6% vs 3.8%) and ≥13° (4.7% vs 1.3%); however, this was not statistically significant after accounting for multiple testing.

CONCLUSION

We failed to observe an association between revision ACLR surgery and LTPS, MTPS, or medial-to-lateral slope difference. However, there was a greater proportion of patients in the revision ACLR group with an LTPS ≥12°, suggesting that a minority of patients who have more extreme values of LTPS have a higher revision risk after primary ACLR. A future cohort study evaluating the angle that best differentiates patients at highest risk for revision is needed.

An adaptive Mantel-Haenszel test for sensitivity analysis in observational studies

In a sensitivity analysis in an observational study with a binary outcome, is it better to use all of the data or to focus on subgroups that are expected to experience the largest treatment effects? The answer depends on features of the data that may be difficult to anticipate, a trade-off between unknown effect-sizes and known sample sizes. We propose a sensitivity analysis for an adaptive test similar to the Mantel-Haenszel test. The adaptive test performs two highly correlated analyses, one focused analysis using a subgroup, one combined analysis using all of the data, correcting for multiple testing using the joint distribution of the two test statistics. Because the two component tests are highly correlated, this correction for multiple testing is small compared with, for instance, the Bonferroni inequality. The test has the maximum design sensitivity of two component tests. A simulation evaluates the power of a sensitivity analysis using the adaptive test. Two examples are presented. An R package, sensitivity2x2xk, implements the procedure.

The cross-cut statistic and its sensitivity to bias in observational studies with ordered doses of treatment

A common practice with ordered doses of treatment and ordered responses, perhaps recorded in a contingency table with ordered rows and columns, is to cut or remove a cross from the table, leaving the outer corners--that is, the high-versus-low dose, high-versus-low response corners--and from these corners to compute a risk or odds ratio. This little remarked but common practice seems to be motivated by the oldest and most familiar method of sensitivity analysis in observational studies, proposed by Cornfield et al. (1959), which says that to explain a population risk ratio purely as bias from an unobserved binary covariate, the prevalence ratio of the covariate must exceed the risk ratio. Quite often, the largest risk ratio, hence the one least sensitive to bias by this standard, is derived from the corners of the ordered table with the central cross removed. Obviously, the corners use only a portion of the data, so a focus on the corners has consequences for the standard error as well as for bias, but sampling variability was not a consideration in this early and familiar form of sensitivity analysis, where point estimates replaced population parameters. Here, this cross-cut analysis is examined with the aid of design sensitivity and the power of a sensitivity analysis.