Instrumental variables in influenza vaccination studies: mission impossible?!

Machine Learning in Causal Inference: Application in Pharmacovigilance

Monitoring adverse drug events or pharmacovigilance has been promoted by the World Health Organization to assure the safety of medicines through a timely and reliable information exchange regarding drug safety issues. We aim to discuss the application of machine learning methods as well as causal inference paradigms in pharmacovigilance. We first reviewed data sources for pharmacovigilance. Then, we examined traditional causal inference paradigms, their applications in pharmacovigilance, and how machine learning methods and causal inference paradigms were integrated to enhance the performance of traditional causal inference paradigms. Finally, we summarized issues with currently mainstream correlation-based machine learning models and how the machine learning community has tried to address these issues by incorporating causal inference paradigms. Our literature search revealed that most existing data sources and tasks for pharmacovigilance were not designed for causal inference. Additionally, pharmacovigilance was lagging in adopting machine learning-causal inference integrated models. We highlight several currently trending directions or gaps to integrate causal inference with machine learning in pharmacovigilance research. Finally, our literature search revealed that the adoption of causal paradigms can mitigate known issues with machine learning models. We foresee that the pharmacovigilance domain can benefit from the progress in the machine learning field.

Estimation of Relative Vaccine Effectiveness in Influenza: A Systematic Review of Methodology

BACKGROUND

When new vaccine components or platforms are developed, they will typically need to demonstrate noninferiority or superiority over existing products, resulting in the assessment of relative vaccine effectiveness (rVE). This review aims to identify how rVE evaluation is being performed in studies of influenza to inform a more standardized approach.

METHODS

We conducted a systematic search on PubMed, Google Scholar, and Web of Science for studies reporting rVE comparing vaccine components, dose, or vaccination schedules. We screened titles, abstracts, full texts, and references to identify relevant articles. We extracted information on the study design, relative comparison made, and the definition and statistical approach used to estimate rVE in each study.

RESULTS

We identified 63 articles assessing rVE in influenza virus. Studies compared multiple vaccine components (n = 38), two or more doses of the same vaccine (n = 17), or vaccination timing or history (n = 9). One study compared a range of vaccine components and doses. Nearly two-thirds of all studies controlled for age, and nearly half for comorbidities, region, and sex. Assessment of 12 studies presenting both absolute and relative effect estimates suggested proportionality in the effects, resulting in implications for the interpretation of rVE effects.

CONCLUSIONS

Approaches to rVE evaluation in practice is highly varied, with improvements in reporting required in many cases. Extensive consideration of methodologic issues relating to rVE is needed, including the stability of estimates and the impact of confounding structure on the validity of rVE estimates.

Impact of influenza vaccination on amoxicillin prescriptions in older adults: A retrospective cohort study using primary care data

Background

Bacterial infections of the upper and lower respiratory tract are a frequent complication of influenza and contribute to the widespread use of antibiotics. Influenza vaccination may help reduce both appropriate and inappropriate prescribing of antibiotics. Electronic health records provide a rich source of information for assessing secondary effects of influenza vaccination.

Methods

We conducted a retrospective study to estimate effects of influenza vaccine on antibiotic (amoxicillin) prescription in the elderly based on data from the Clinical Practice Research Datalink. The introduction of UK policy to recommend the influenza vaccine to older adults in 2000 led to a substantial increase in uptake, creating a natural experiment. Of 259,753 eligible patients that were unvaccinated in 1999 and aged≥65y by January 2000, 88,519 patients received influenza vaccination in 2000. These were propensity score matched 1:1 to unvaccinated patients. Time-to-amoxicillin was analysed using the Prior Event Rate Ratio (PERR) Pairwise method to address bias from time-invariant measured and unmeasured confounders. A simulation study and negative control outcome were used to help strengthen the validity of results.

Results

Compared to unvaccinated patients, those from the vaccinated group were more likely to be prescribed amoxicillin in the year prior to vaccination: hazard ratio (HR) 1.90 (95% confidence interval 1.83, 1.98). Following vaccination, the vaccinated group were again more likely to be prescribed amoxicillin, HR 1.64 (1.58,1.71). After adjusting for prior differences between the two groups using PERR Pairwise, overall vaccine effectiveness was 0.86 (0.81, 0.92). Additional analyses suggested that provided data meet the PERR assumptions, these estimates were robust.

Conclusions

Once differences between groups were taken into account, influenza vaccine had a beneficial effect, lowering the frequency of amoxicillin prescribing in the vaccinated group. Ensuring successful implementation of national programmes of vaccinating older adults against influenza may help contribute to reducing antibiotic resistance.

The Dutch influenza vaccination policy and medication use, outpatient visits, hospitalization and mortality at age 65

BACKGROUND

Our objective was to obtain estimates of the impact of the Dutch vaccination programme on medication use, outpatient visits, hospitalization and mortality at age 65.

METHODS

We linked population-wide mortality, hospitalization and municipality registries to identify influenza-related deaths and hospitalizations, and used health interview surveys to identify medication use and outpatient visits during 1996-2008. We applied a regression discontinuity design to estimate the intention-to-treat effect of the personal invitation for a free influenza vaccination sent to every Dutch inhabitant at age 65 years on each of the outcomes, separately in influenza-epidemic and non-epidemic months.

RESULTS

Invitation receipt for free influenza vaccination at age 65 led to a 9.8 percentage points [95% confidence interval (CI) = 3.5 to16.1; P < 0.01] rise in influenza vaccination. During influenza-epidemic months, it was associated with 1.5 fewer influenza/pneumonia deaths per 100 000 individuals (95% CI = -3.1 to -0.0; P = 0.05), a 15 percentage point lower probability to use prescribed medicines (95% CI = -28 to -3; P = 0.02) and 0.13 fewer General Practitioner (GP) visits per month (95% CI = -0.28 to 0.02; P = 0.09), while the association with hospitalizations due to influenza/pneumonia was small and imprecisely estimated (seven more hospitalizations per 100 000 individuals, 95% CI = -20 to 33; P = 0.63). No associations were found with any outcomes during non-epidemic months.

CONCLUSIONS

Personal invitations for a free influenza vaccination sent to every Dutch inhabitant at age 65 took pressure off primary health care but had small effects on hospitalizations and mortality.

Analysis of relative effectiveness of high-dose versus standard-dose influenza vaccines using an instrumental variable method

BACKGROUND

Observational studies of the relative effectiveness of influenza vaccines are essential for public health decision making. Their estimates, however, are subject to bias due to unmeasured confounders. Instrumental variable (IV) methods can control for observed and unobserved confounders.

METHODS

We used linked electronic medical record databases in the Veterans Health Administration (VHA) as well as Medicare administrative files to examine the relative vaccine effectiveness (rVE) of high-dose influenza vaccine (HD) versus standard-dose influenza vaccines (SD) in preventing hospitalizations among VHA-enrolled Veterans ≥65 years of age during 5 influenza seasons (2010-2011 through 2014-2015). Using multivariable IV Poisson regression modeling to address unmeasured confounding and bias, we analyzed the data by each season and through longitudinal analysis of all five seasons.

FINDINGS

We included 3,638,924 person-influenza seasons of observation where 158,636 (4%) were among HD vaccine recipients and 3,480,288 (96%) were among SD vaccine recipients. Of the 1,728,562 Veterans, 1,702,824 (98.5%) were male and 1,299,412 (75%) were non-Hispanic white. Based on the longitudinal analysis of all five seasons, the IV-adjusted rVE estimate of HD vs. SD was 10% (95% CI, 8-12%) against all-cause hospitalization; 18% (95% CI, 15-21%) against cardiorespiratory-associated hospitalization; and 14% (95% CI, 6-22%) against influenza/pneumonia-associated hospitalization. The findings by season were similar.

INTERPRETATION

Our analysis of VHA clinical data collected from approximately 1.7 million Veterans 65 years and older during five seasons demonstrates that high-dose influenza vaccine is more effective than standard-dose influenza vaccines in preventing influenza- or pneumonia-associated hospitalizations, cardiorespiratory hospitalizations, and all-cause hospitalizations.

Instrumental Variable Analyses in Pharmacoepidemiology: What Target Trials Do We Emulate?

Purpose of review

When leveraging observational data to estimate treatment effects, it is useful to explicitly specify the “target trial” the investigators aspire to emulate. One concern is whether a proposed analysis plan can address the realities of the differences between the available non-randomized observational study and the target trial. When large or unknown sources of unmeasured confounding are suspected, investigators might consider turning to instrumental variable (IV) methods. Of course, the interpretation and appropriateness of IV analyses need to be considered carefully. The purpose of this review is to summarize recent methodologic advancements in how epidemiologists weigh the validity of an IV analysis and to place these methodologic advancements in the context of the feasible target trial’s protocol components.

Recent findings

There have been increased development and application of tools for sensitivity analyses, falsification strategies, and the identification of previously overlooked problems with IV analyses as applied in pharmacoepidemiology. Many of these recent insights can be seen as articulating restrictions on or tradeoffs between the types of target trials that can be validly emulated when using a classical IV analysis.

Summary

Putting classical IV methods in the context of target trials underscores the importance of recent methodologic developments and, more generally, when and how an IV analysis would be appropriate. We see that some tradeoffs in defining the target trials are unavoidable, that some tradeoffs may be offset or explored via sensitivity analyses, and that this serves as a framework for scientific discourse regarding IV and non-IV results emulating potentially different trials with different tradeoffs.

Evaluating different physician's prescribing preference based instrumental variables in two primary care databases: a study of inhaled long-acting beta2-agonist use and the risk of myocardial infarction

PURPOSE

Instrumental variable (IV) analysis with physician's prescribing preference (PPP) as IV is increasingly used in pharmacoepidemiology. However, it is unclear whether this IV performs consistently across databases. We aimed to evaluate the validity of different PPPs in a study of inhaled long-acting beta2-agonist (LABA) use and myocardial infarction (MI).

METHODS

Information on adults with asthma and/or COPD and at least one prescription of beta2-agonist, or muscarinic antagonist was extracted from the CPRD (UK) and the Mondriaan (Netherlands) databases. LABA exposure was considered time-fixed or time-varying. We measured PPPs using previous LABA prescriptions of physicians or proportion of LABA prescriptions per practice. Correlation (r) and standardized difference (SDif) were used to assess assumption of IV analysis.

RESULTS

For time-fixed LABA, the IV based on 10 previous prescriptions outperformed the other IVs regarding strength of the IV (r ≥ 0.15) and balance of confounders between IV categories (SDif < 0.10). None of the IVs we considered appeared to be valid for time-varying LABA. In CPRD (n = 490,499), which included approximately 18 times more subjects than Mondriaan (n = 27,459), IVs appeared more valid. LABA was not associated with MI; hazard ratios ranged from 0.86 to 1.18 for conventional analysis, and from 0.61 to 1.24 for the IV analyses with apparent valid IVs.

CONCLUSIONS

The validity of physician's prescribing preference as IV strongly depends on how this IV is defined and in which database it is applied. Hence, general recommendations cannot be made, other than to generate several plausible IVs, assess their validity, and report the estimate(s) from apparently valid IVs.

Using claims data to predict dependency in activities of daily living as a proxy for frailty

PURPOSE

Estimating drug effectiveness and safety among older adults in population-based studies using administrative health care claims can be hampered by unmeasured confounding as a result of frailty. A claims-based algorithm that identifies patients likely to be dependent, a proxy for frailty, may improve confounding control. Our objective was to develop an algorithm to predict dependency in activities of daily living (ADL) in a sample of Medicare beneficiaries.

METHODS

Community-dwelling respondents to the 2006 Medicare Current Beneficiary Survey, >65 years old, with Medicare Part A, B, home health, and hospice claims were included. ADL dependency was defined as needing help with bathing, eating, walking, dressing, toileting, or transferring. Potential predictors were demographics, International Classification of Diseases, Ninth Revision Clinical Modification diagnosis/procedure and durable medical equipment codes for frailty-associated conditions. Multivariable logistic regression was used to predict ADL dependency. Cox models estimated hazard ratios for death as a function of observed and predicted ADL dependency.

RESULTS

Of 6391 respondents, 57% were female, 88% white, and 38% were ≥80. The prevalence of ADL dependency was 9.5%. Strong predictors of ADL dependency were charges for a home hospital bed (OR = 5.44, 95%CI = 3.28-9.03) and wheelchair (OR = 3.91, 95%CI = 2.78-5.51). The c-statistic of the final model was 0.845. Model-predicted ADL dependency of 20% or greater was associated with a hazard ratio for death of 3.19 (95%CI: 2.78, 3.68).

CONCLUSIONS

An algorithm for predicting ADL dependency using health care claims was developed to measure some aspects of frailty. Accounting for variation in frailty among older adults could lead to more valid conclusions about treatment use, safety, and effectiveness.

Seasonal Influenza Vaccine Effectiveness (SIVE): an observational retrospective cohort study – exploitation of a unique community-based national-linked database to determine the effectiveness of the seasonal trivalent influenza vaccine

Background

Globally, seasonal influenza is responsible for an estimated 3 to 5 million cases of severe illness and 250,000 to 500,000 deaths per year. It is uncertain to what extent national vaccination programmes can prevent this morbidity and mortality.

Objective

To determine the effectiveness of the seasonal trivalent inactivated influenza vaccine.

Design

We undertook a retrospective observational cohort study. A propensity score model was constructed and adjusted odds ratios (ORs) were calculated to assess differences in vaccine uptake according to a number of patient characteristics. Adjusted illness and mortality hazard ratios (HRs) were estimated from a Cox proportional hazards model adjusted for sex, age, socioeconomic status, smoking status, urban/rural location, clinical at-risk groups (i.e. patients with chronic respiratory, heart, kidney, liver or neurological disease, immunosuppression and diabetes), Charlson comorbidity index, previous pneumococcal and influenza vaccination, and number of previous primary care consultations, prescribed drugs and hospital admissions. We also included nursing home residence and social care support. Vaccine effectiveness (VE) was expressed as a percentage, and represents a reduction in risk provided by the vaccine for a given outcome (e.g. laboratory-confirmed influenza). This was calculated as 1 − HR, where HR is that of the measured clinical outcome in vaccinated compared with unvaccinated individuals. For estimates of VE derived from linked virological swab data, we carried out a nested case–control study design.

Setting

A national linkage of patient-level primary care, hospital, death certification and virological swab-linked data across nine influenza seasons (2000–9).

Participants

A nationally representative sample of the Scottish population during 1,767,919 person-seasons of observation. Cases of influenza were confirmed using reverse transcription-polymerase chain reaction (RT-PCR) in a subset of the population (n = 3323).

Interventions

Trivalent inactivated seasonal influenza vaccination (n = 274,071).

Main outcome measures

VE, pooled across seasons and adjusting for confounders, was estimated by determining laboratory-confirmed influenza, influenza-related morbidity and mortality including primary care influenza-like illnesses, hospitalisation and death from influenza and pneumonia.

Results

Most vaccines (93.6%;n = 256,474 vaccines) were administered to at-risk patients targeted for vaccination, with a 69.3% uptake among those aged ≥ 65 years (178,754 vaccinations during 258,100 person-seasons). For at-risk patients aged < 65 years there was a 26.2% uptake (77,264 vaccinations during 295,116 person-seasons). VE in preventing RT-PCR laboratory-confirmed influenza was 57.1% [95% confidence interval (CI) 31.3% to 73.3%]. VE was 18.8% (95% CI –103.7% to 67.6%) in patients aged ≥ 65 years and 59.6% (95% CI 21.9% to 79.1%) in those aged < 65 years at risk of serious complications from influenza. In the matched analysis (156,096 person-seasons), adjusted VE for reducing primary care consultations for influenza-like illnesses was 16.3% (95% CI 5.7% to 26.0%). VE in reducing hospitalisations was 19.3% for influenza and pneumonia (95% CI 8.3% to 29.1%) and 26.7% for pneumonia and chronic obstructive pulmonary disease (95% CI 19.8% to 32.9%). VE in reducing death due to influenza and pneumonia was 37.9% (95% CI 29.5% to 45.4%).

Conclusions

Few countries' health systems allow for the integrated and accessible data recording that made this study possible and made it feasible to collate centrally almost all hospitalisations and deaths attributed to influenza, thereby allowing completeness of reporting. Using these data, we found most influenza vaccines were administered to those at risk of serious complications from influenza. In a nationally representative cohort we found that the vaccine was associated with a significant decrease in the risk of RT-PCR-confirmed influenza (the decrease was substantial particularly for at-risk patients aged < 65 years) and complications arising from influenza (where more modest decreases were found). Although the modest size of our cohort made it possible to collate centrally almost all cases of influenza-related disease, analysis of subgroups (in particular older age groups) or by individual season resulted in poorer precision and wide CIs. Any future work should therefore aim to address this issue by ensuring adequate power to test VE in these subgroups of patients, while minimising the effect of bias, such as health-seeking behaviour.

Funding

The National Institute for Health Research Health Services and Delivery Research programme.

Seasonal Influenza Vaccine Effectiveness in the community (SIVE): protocol for a cohort study exploiting a unique national linked data set

INTRODUCTION

Seasonal influenza vaccination is recommended for all individuals aged 65 years and over and in individuals younger than 65 years with comorbidities. There is good evidence of vaccine effectiveness (VE) in young healthy individuals but less robust evidence for effectiveness in the populations targeted for influenza vaccination. Undertaking a randomised controlled trial to assess VE is now impractical due to the presence of national vaccination programmes. Quasi-experimental designs offer the potential to advance the evidence base in such scenarios, and the authors have therefore been commissioned to undertake a naturalistic national evaluation of seasonal influenza VE by using data derived from linkage of a number of Scottish health databases. The aim of this study is to examine the effectiveness of the seasonal influenza vaccination in the Scottish population.

METHODS AND ANALYSIS

A cohort study design will be used pooling data over nine seasons. A primary care database covering 4% of the Scottish population for the period 2000-2009 has been linked to the national database of hospital admissions and the death register and is being linked to the Health Protection Scotland virology database. The primary outcome is VE measured in terms of rate of hospital admissions due to respiratory illness. Multivariable regression will be used to produce estimates of VE adjusted for confounders. The major challenge of this approach is addressing the strong effect of confounding due to vaccinated individuals being systematically different from unvaccinated individuals. Analyses using propensity scores and instrumental variables will be undertaken, and the effect of an unknown confounder will be modelled in a sensitivity analysis to assess the robustness of the estimates.

ETHICS AND DISSEMINATION

The West of Scotland Research Ethics Committee has classified this project as surveillance. The study findings will be disseminated in peer-reviewed publications and presented at international conferences.

Influenza vaccine effectiveness in the elderly based on administrative databases: change in immunization habit as a marker for bias

BACKGROUND

Administrative databases provide efficient methods to estimate influenza vaccine effectiveness (IVE) against severe outcomes in the elderly but are prone to intractable bias. This study returns to one of the linked population databases by which IVE against hospitalization and death in the elderly was first assessed. We explore IVE across six more recent influenza seasons, including periods before, during, and after peak activity to identify potential markers for bias.

METHODS AND FINDINGS

Acute respiratory hospitalization and all-cause mortality were compared between immunized/non-immunized community-dwelling seniors ≥65 years through administrative databases in Manitoba, Canada between 2000-01 and 2005-06. IVE was compared during pre-season/influenza/post-season periods through logistic regression with multivariable adjustment (age/sex/income/residence/prior influenza or pneumococcal immunization/medical visits/comorbidity), stratification based on prior influenza immunization history, and propensity scores. Analysis during pre-season periods assessed baseline differences between immunized and unimmunized groups. The study population included ∼140,000 seniors, of whom 50-60% were immunized annually. Adjustment for key covariates and use of propensity scores consistently increased IVE. Estimates were paradoxically higher pre-season and for all-cause mortality vs. acute respiratory hospitalization. Stratified analysis showed that those twice consecutively and currently immunized were always at significantly lower hospitalization/mortality risk with odds ratios (OR) of 0.60 [95%CI0.48-0.75] and 0.58 [0.53-0.64] pre-season and 0.77 [0.69-0.86] and 0.71 [0.66-0.77] during influenza circulation, relative to the consistently unimmunized. Conversely, those forgoing immunization when twice previously immunized were always at significantly higher hospitalization/mortality risk with OR of 1.41 [1.14-1.73] and 2.45 [2.21-2.72] pre-season and 1.21 [1.03-1.43] and 1.78 [1.61-1.96] during influenza circulation.

CONCLUSIONS

The most pronounced IVE estimates were paradoxically observed pre-season, indicating bias tending to over-estimate vaccine protection. Change in immunization habit from that of the prior two years may be a marker for this bias in administrative data sets; however, no analytic technique explored could adjust for its influence. Improved methods to achieve valid interpretation of protection in the elderly are needed.

Use of instrumental variable in prescription drug research with observational data: a systematic review

OBJECTIVE

Instrumental variable (IV) analysis may offer a useful approach to the problem of unmeasured confounding in prescription drug research if the IV is: (1) strongly and unbiasedly associated to treatment assignment; and (2) uncorrelated with factors predicting the outcome (key assumptions).

STUDY DESIGN AND METHODS

We conducted a systematic review of the use of IV methods in prescription drug research to identify the major types of IVs and the evidence for meeting IV assumptions. We searched MEDLINE, OVID, PsychoInfo, EconLit, and economic databases from 1961 to 2009.

RESULTS

We identified 26 studies. Most (n=16) were published after 2007. We identified five types of IVs: regional variation (n=8), facility-prescribing patterns (n=5), physician preference (n=8), patient history/financial status (n=3), and calendar time (n=4). Evidence supporting the validity of IV was inconsistent. All studies addressed the first IV assumption; however, there was no standard for demonstrating that the IV sufficiently predicted treatment assignment. For the second assumption, 23 studies provided explicit argument that IV was uncorrelated with the outcome, and 16 supported argument with empirical evidence.

CONCLUSIONS

Use of IV methods is increasing in prescription drug research. However, we did not find evidence of a dominant IV. Future research should develop standards for reporting the validity and strength of IV according to key assumptions.