Self-controlled case series analyses: Small-sample performance

Inclusion of unexposed subjects improves the precision and power of self-controlled case series method

The self-controlled case series is an important method in the studies of the safety of biopharmaceutical products. It uses the conditional Poisson model to make comparison within persons. In models without adjustment for age (or other time-varying covariates), cases who are never exposed to the product do not contribute any information to the estimation. We provide analytic proof and simulation results that the inclusion of unexposed cases in the conditional Poisson model with age adjustment reduces the asymptotic variance of the estimator of the exposure effect and increases power. We re-analysed a vaccine safety dataset to illustrate.

The Case Time Series Design

Modern data linkage and technologies provide a way to reconstruct detailed longitudinal profiles of health outcomes and predictors at the individual or small-area level. Although these rich data resources offer the possibility to address epidemiologic questions that could not be feasibly examined using traditional studies, they require innovative analytical approaches. Here we present a new study design, called case time series, for epidemiologic investigations of transient health risks associated with time-varying exposures. This design combines a longitudinal structure and flexible control of time-varying confounders, typical of aggregated time series, with individual-level analysis and control-by-design of time-invariant between-subject differences, typical of self-matched methods such as case-crossover and self-controlled case series. The modeling framework is highly adaptable to various outcome and exposure definitions, and it is based on efficient estimation and computational methods that make it suitable for the analysis of highly informative longitudinal data resources. We assess the methodology in a simulation study that demonstrates its validity under defined assumptions in a wide range of data settings. We then illustrate the design in real-data examples: a first case study replicates an analysis on influenza infections and the risk of myocardial infarction using linked clinical datasets, while a second case study assesses the association between environmental exposures and respiratory symptoms using real-time measurements from a smartphone study. The case time series design represents a general and flexible tool, applicable in different epidemiologic areas for investigating transient associations with environmental factors, clinical conditions, or medications.

Self-controlled case series studies: Just how rare does a rare non-recurrent outcome need to be?

The self-controlled case series method assumes that adverse outcomes arise according to a non-homogeneous Poisson process. This implies that it is applicable to independent recurrent outcomes. However, the self-controlled case series method may also be applied to unique, non-recurrent outcomes or first outcomes only, in the limit where these become rare. We investigate this rare outcome assumption when the self-controlled case series method is applied to non-recurrent outcomes. We study this requirement analytically and by simulation, and quantify what is meant by 'rare' in this context. In simulations we also apply the self-controlled risk interval design, a special case of the self-controlled case series design. To illustrate, we extract data on the incidence rate of some recurrent and non-recurrent outcomes within a defined study population to check whether outcomes are sufficiently rare for the rare outcome assumption to hold when applying the self-controlled case series method to first or unique outcomes. The main findings are that the relative bias should be no more than 5% when the cumulative incidence over total time observed is less than 0.1 per individual. Inclusion of age (or calendar time) effects will further reduce bias. Designs that begin observation with exposure maximise bias, whereas little or no bias will be apparent when there is no time trend in the distribution of exposures, or when exposure is central within time observed.

Analysis of methods commonly used in biomedicine for treatment versus control comparison of very small samples

BACKGROUND AND OBJECTIVE

A rough estimate indicated that use of samples of size not larger than ten is not uncommon in biomedical research and that many of such studies are limited to strong effects due to sample sizes smaller than six. For data collected from biomedical experiments it is also often unknown if mathematical requirements incorporated in the sample comparison methods are satisfied.

METHODS

Computer simulated experiments were used to examine performance of methods for qualitative sample comparison and its dependence on the effectiveness of exposure, effect intensity, distribution of studied parameter values in the population, and sample size. The Type I and Type II errors, their average, as well as the maximal errors were considered.

RESULTS

The sample size 9 and the t-test method with p = 5% ensured error smaller than 5% even for weak effects. For sample sizes 6-8 the same method enabled detection of weak effects with errors smaller than 20%. If the sample sizes were 3-5, weak effects could not be detected with an acceptable error; however, the smallest maximal error in the most general case that includes weak effects is granted by the standard error of the mean method. The increase of sample size from 5 to 9 led to seven times more accurate detection of weak effects. Strong effects were detected regardless of the sample size and method used.

CONCLUSIONS

The minimal recommended sample size for biomedical experiments is 9. Use of smaller sizes and the method of their comparison should be justified by the objective of the experiment.

A comparison of estimators from self-controlled case series, case-crossover design, and sequence symmetry analysis for pharmacoepidemiological studies

Background

Despite the frequent use of self-controlled methods in pharmacoepidemiological studies, the factors that may bias the estimates from these methods have not been adequately compared in real-world settings. Here, we comparatively examined the impact of a time-varying confounder and its interactions with time-invariant confounders, time trends in exposures and events, restrictions, and misspecification of risk period durations on the estimators from three self-controlled methods. This study analyzed self-controlled case series (SCCS), case-crossover (CCO) design, and sequence symmetry analysis (SSA) using simulated and actual electronic medical records datasets.

Methods

We evaluated the performance of the three self-controlled methods in simulated cohorts for the following scenarios: 1) time-invariant confounding with interactions between the confounders, 2) time-invariant and time-varying confounding without interactions, 3) time-invariant and time-varying confounding with interactions among the confounders, 4) time trends in exposures and events, 5) restricted follow-up time based on event occurrence, and 6) patient restriction based on event history. The sensitivity of the estimators to misspecified risk period durations was also evaluated. As a case study, we applied these methods to evaluate the risk of macrolides on liver injury using electronic medical records.

Results

In the simulation analysis, time-varying confounding produced bias in the SCCS and CCO design estimates, which aggravated in the presence of interactions between the time-invariant and time-varying confounders. The SCCS estimates were biased by time trends in both exposures and events. Erroneously short risk periods introduced bias to the CCO design estimate, whereas erroneously long risk periods introduced bias to the estimates of all three methods. Restricting the follow-up time led to severe bias in the SSA estimates. The SCCS estimates were sensitive to patient restriction. The case study showed that although macrolide use was significantly associated with increased liver injury occurrence in all methods, the value of the estimates varied.

Conclusions

The estimations of the three self-controlled methods depended on various underlying assumptions, and the violation of these assumptions may cause non-negligible bias in the resulting estimates. Pharmacoepidemiologists should select the appropriate self-controlled method based on how well the relevant key assumptions are satisfied with respect to the available data.

Electronic supplementary material

The online version of this article (10.1186/s12874-017-0457-7) contains supplementary material, which is available to authorized users.

Risk of Acute Asthma Attacks Associated With Nonsteroidal Anti-inflammatory Drugs: A Self-Controlled Case Series

BACKGROUND

Although asthma attacks are known adverse events associated with nonsteroidal anti-inflammatory drug (NSAID) use, few studies have quantified these risks. The objectives of this study were to utilize an epidemiological approach to quantitatively evaluate the risk of acute asthma attacks associated with NSAID prescription in Japan and to compare the risks among NSAIDs according to their cyclooxygenase (COX)-2 selectivity.

METHODS

We conducted a self-controlled case series study using Japanese health insurance claims data. Exposed cases were identified as those who had experienced both NSAID prescription and acute asthma attack, which was defined as the combination of an inhalation procedure and the prescription of any inhaled β₂-agonist. The incidence rate ratios (IRRs) and 95% confidence intervals (CIs) for NSAID prescription periods compared with baseline periods were calculated using conditional Poisson regression models; COX-2 selective and nonselective NSAIDs were similarly compared.

RESULTS

We identified 9769 subjects, more than 95% of whom were younger than 60 years. There was a significantly higher risk of acute asthma attacks during the NSAID prescription period when compared with the baseline period. The quantified IRRs were, in descending order, 93.94 (95% CI, 90.10-97.95) for the prescription start date, 3.96 (95% CI, 3.63-4.33) for 1 to 9 days after the prescription start date, 3.01 (95% CI, 2.78-3.25) for 7 days after the prescription end date, 2.19 (95% CI, 1.82-2.65) for >9 days after the prescription start date, and 1.44 (95% CI, 1.29 -1.61) for 7 days before the prescription start date. There were lower asthmatic risks for COX-2 selective NSAIDs compared with nonselective NSAIDs.

CONCLUSIONS

The use of NSAIDs in Japan was associated with an increased risk of acute asthma attacks. However, this risk was lower in COX-2 selective NSAIDs.

Bias correction of risk estimates in vaccine safety studies with rare adverse events using a self-controlled case series design

The self-controlled case series (SCCS) method is often used to examine the temporal association between vaccination and adverse events using only data from patients who experienced such events. Conditional Poisson regression models are used to estimate incidence rate ratios, and these models perform well with large or medium-sized case samples. However, in some vaccine safety studies, the adverse events studied are rare and the maximum likelihood estimates may be biased. Several bias correction methods have been examined in case-control studies using conditional logistic regression, but none of these methods have been evaluated in studies using the SCCS design. In this study, we used simulations to evaluate 2 bias correction approaches-the Firth penalized maximum likelihood method and Cordeiro and McCullagh's bias reduction after maximum likelihood estimation-with small sample sizes in studies using the SCCS design. The simulations showed that the bias under the SCCS design with a small number of cases can be large and is also sensitive to a short risk period. The Firth correction method provides finite and less biased estimates than the maximum likelihood method and Cordeiro and McCullagh's method. However, limitations still exist when the risk period in the SCCS design is short relative to the entire observation period.

Use of the self-controlled case-series method in vaccine safety studies: review and recommendations for best practice

The self-controlled case-series method was originally developed to investigate potential associations between vaccines and adverse events, and is now commonly used for this purpose. This study reviews applications of the method to vaccine safety investigations in the period 1995–2010. In total, 40 studies were reviewed. The application of the self-controlled case-series method in these studies is critically examined, with particular reference to the definition of observation and risk periods, control of confounders, assumptions and potential biases, methodological and presentation issues, power and sample size, and software. Comparisons with other study designs undertaken in the papers reviewed are also highlighted. Some recommendations are presented, with the emphasis on promoting good practice.

The methodology of self-controlled case series studies

The self-controlled case series method is increasingly being used in pharmacoepidemiology, particularly in vaccine safety studies. This method is typically used to evaluate the association between a transient exposure and an acute event, using only cases. We present both parametric and semiparametric models using a motivating example on MMR vaccine and bleeding disorders. We briefly describe approaches for interferent events and a sequential version of the method for prospective surveillance of drug safety. The efficiency of the self-controlled case series method is compared to the that of cohort and case control studies. Some further extensions, to long or indefinite exposures and to bivariate counts, are described.