Assessing recording delays in general practice records to inform near real-time vaccine safety surveillance using the Clinical Practice Research Datalink (CPRD)

Current Approaches in Postapproval Vaccine Safety Studies Using Real-World Data: A Systematic Review of Published Literature

PURPOSE

Well-designed observational postmarketing studies using real-world data (RWD) are critical in supporting an evidence base and bolstering public confidence in vaccine safety. This systematic review presents current research methodologies in vaccine safety research in postapproval settings, technological advancements contributing to research resources and capabilities, and their major strengths and limitations.

METHODS

A comprehensive search was conducted using PubMed to identify relevant articles published from January 1, 2019, to December 31, 2022. Eligible studies were summarized overall by study design and other study characteristics (eg, country, vaccine studied, types of data source, and study population). An in-depth review of select studies representative of conventional or new designs, analytical approaches, or data collection methods was conducted to summarize current methods in vaccine safety research.

FINDINGS

Out of 977 articles screened for inclusion, 135 were reviewed. The review shows that recent advancements in scientific methods, digital technology, and analytic approaches have significantly contributed to postapproval vaccine safety studies using RWD. "Near real-time surveillance" using large datasets (via collaborative or distributed databases) has been used to facilitate rapid signal detection that complements passive surveillance. There was increasing appreciation for self-controlled case-only designs (self-controlled case series and self-controlled risk interval) to assess acute-onset safety outcomes, artificial intelligence, and natural language processing to improve outcome accuracy and study timeliness and emerging artificial intelligence-based analysis to capture adverse events from social media platforms.

IMPLICATIONS

Continued development in the area of vaccine safety research methodologies using RWD is warranted. The future of successful vaccine safety research, especially evaluation of rare safety events, is likely to comprise digital technologies including linking RWD networks, machine learning, and advanced analytic methods to generate rapid and robust real-world safety information.

Safety of COVID-19 vaccination and acute neurological events: A self-controlled case series in England using the OpenSAFELY platform

INTRODUCTION

We investigated the potential association of COVID-19 vaccination with three acute neurological events: Guillain-Barré syndrome (GBS), transverse myelitis and Bell's palsy.

METHODS

With the approval of NHS England we analysed primary care data from >17 million patients in England linked to emergency care, hospital admission and mortality records in the OpenSAFELY platform. Separately for each vaccine brand, we used a self-controlled case series design to estimate the incidence rate ratio for each outcome in the period following vaccination (4-42 days for GBS, 4-28 days for transverse myelitis and Bell's palsy) compared to a within-person baseline, using conditional Poisson regression.

RESULTS

Among 7,783,441 ChAdOx1 vaccinees, there was an increased rate of GBS (N = 517; incidence rate ratio 2·85; 95% CI2·33-3·47) and Bell's palsy (N = 5,350; 1·39; 1·27-1·53) following a first dose of ChAdOx1 vaccine, corresponding to 11.0 additional cases of GBS and 17.9 cases of Bell's palsy per 1 million vaccinees if causal. For GBS this applied to the first, but not the second, dose. There was no clear evidence of an association of ChAdOx1 vaccination with transverse myelitis (N = 199; 1·51; 0·96-2·37). Among 5,729,152 BNT162b2 vaccinees, there was no evidence of any association with GBS (N = 283; 1·09; 0·75-1·57), transverse myelitis (N = 109; 1·62; 0·86-3·03) or Bell's palsy (N = 3,609; 0·89; 0·76-1·03). Among 255,446 mRNA-1273 vaccine recipients there was no evidence of an association with Bell's palsy (N = 78; 0·88, 0·32-2·42).

CONCLUSIONS

COVID-19 vaccines save lives, but it is important to understand rare adverse events. We observed a short-term increased rate of Guillain-Barré syndrome and Bell's palsy after first dose of ChAdOx1 vaccine. The absolute risk, assuming a causal effect attributable to vaccination, was low.

Immortal time bias for life-long conditions in retrospective observational studies using electronic health records

Background

Immortal time bias is common in observational studies but is typically described for pharmacoepidemiology studies where there is a delay between cohort entry and treatment initiation.

Methods

This study used the Clinical Practice Research Datalink (CPRD) and linked national mortality data in England from 2000 to 2019 to investigate immortal time bias for a specific life-long condition, intellectual disability. Life expectancy (Chiang’s abridged life table approach) was compared for 33,867 exposed and 980,586 unexposed individuals aged 10+ years using five methods: (1) treating immortal time as observation time; (2) excluding time before date of first exposure diagnosis; (3) matching cohort entry to first exposure diagnosis; (4) excluding time before proxy date of inputting first exposure diagnosis (by the physician); and (5) treating exposure as a time-dependent measure.

Results

When not considered in the design or analysis (Method 1), immortal time bias led to disproportionately high life expectancy for the exposed population during the first calendar period (additional years expected to live: 2000–2004: 65.6 [95% CI: 63.6,67.6]) compared to the later calendar periods (2005–2009: 59.9 [58.8,60.9]; 2010–2014: 58.0 [57.1,58.9]; 2015–2019: 58.2 [56.8,59.7]). Date of entry of diagnosis (Method 4) was unreliable in this CPRD cohort. The final methods (Method 2, 3 and 5) appeared to solve the main theoretical problem but residual bias may have remained.

Conclusions

We conclude that immortal time bias is a significant issue for studies of life-long conditions that use electronic health record data and requires careful consideration of how clinical diagnoses are entered onto electronic health record systems.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-022-01581-1.

Evaluation of Covid-19 vaccines: Pharmacoepidemiological aspects

The marketing authorization granted to SARS-Cov-2 vaccines was accompanied by reinforced safety monitoring plans. These plans' implementation was part of the usual logic of post-marketing surveillance of new and innovative health products. It was especially adapted to the context of post-marketing monitoring of drugs developed according to the usual scientific quality standards but in an accelerated schedule. In Europe, the reinforced surveillance system relies on the complementary strengths of pharmacovigilance and pharmacoepidemiology. If the performances of pharmacovigilance monitoring are incomparable for the detection of safety signals relating to rare events of atypical presentation, it needs to be completed with pharmacoepidemiology activities for more common events, either multifactorial or frequently classified as idiopathic. The pharmacoepidemiological monitoring developed in Europe was elaborated before the first SARS-Cov-2 vaccines where marketed, taking into account the lessons learned from the vaccination campaign against 2009 A (H1N1) influenza. It includes numerous academic studies as well as studies performed within vaccines risk management plans. In terms of safety, those defined a priori mostly concerns a list of pre-established health events of specific interest. Aside of these planned activities, ad-hoc studies will be latter developed on purpose to investigate safety signals or potential signals that could be identified as the result of pharmacovigilance activities. Aside of these regulated activities, as for today, very few studies have been published regarding SARS-Cov-2 vaccines; most of the existing consist in preprints that should be considered with caution. Pharmacoepidemiology of vaccines is thought to allow near-real time monitoring that needs sufficient time to provide with valid results. In the constant urge for information that accompanies COVID-related science, it is important not to make haste the enemy of speed and to let pharmacoepidemiology provides with what it is expected to do: rock-solid scientific information contributing to evidence-based decision-making.

Should free-text data in electronic medical records be shared for research? A citizens' jury study in the UK

BACKGROUND

Use of routinely collected patient data for research and service planning is an explicit policy of the UK National Health Service and UK government. Much clinical information is recorded in free-text letters, reports and notes. These text data are generally lost to research, due to the increased privacy risk compared with structured data. We conducted a citizens' jury which asked members of the public whether their medical free-text data should be shared for research for public benefit, to inform an ethical policy.

METHODS

Eighteen citizens took part over 3 days. Jurors heard a range of expert presentations as well as arguments for and against sharing free text, and then questioned presenters and deliberated together. They answered a questionnaire on whether and how free text should be shared for research, gave reasons for and against sharing and suggestions for alleviating their concerns.

RESULTS

Jurors were in favour of sharing medical data and agreed this would benefit health research, but were more cautious about sharing free-text than structured data. They preferred processing of free text where a computer extracted information at scale. Their concerns were lack of transparency in uses of data, and privacy risks. They suggested keeping patients informed about uses of their data, and giving clear pathways to opt out of data sharing.

CONCLUSIONS

Informed citizens suggested a transparent culture of research for the public benefit, and continuous improvement of technology to protect patient privacy, to mitigate their concerns regarding privacy risks of using patient text data.

Use of routinely collected electronic healthcare data for postlicensure vaccine safety signal detection: a systematic review

Background

Concerns regarding adverse events following vaccination (AEFIs) are a key challenge for public confidence in vaccination. Robust postlicensure vaccine safety monitoring remains critical to detect adverse events, including those not identified in prelicensure studies, and to ensure public safety and public confidence in vaccination. We summarise the literature examined AEFI signal detection using electronic healthcare data, regarding data sources, methodological approach and statistical analysis techniques used.

Methods

We performed a systematic review using the Preferred Reporting Items for Systematic Reviews and Meta-analyses guidelines. Five databases (PubMed/Medline, EMBASE, CINAHL, the Cochrane Library and Web of Science) were searched for studies on AEFIs monitoring published up to 25 September 2017. Studies were appraised for methodological quality, and results were synthesised narratively.

Result

We included 47 articles describing AEFI signal detection using electronic healthcare data. All studies involved linked diagnostic healthcare data, from the emergency department, inpatient and outpatient setting and immunisation records. Statistical analysis methodologies used included non-sequential analysis in 33 studies, group sequential analysis in two studies and 12 studies used continuous sequential analysis. Partially elapsed risk window and data accrual lags were the most cited barriers to monitor AEFIs in near real-time.

Conclusion

Routinely collected electronic healthcare data are increasingly used to detect AEFI signals in near real-time. Further research is required to check the utility of non-coded complaints and encounters, such as telephone medical helpline calls, to enhance AEFI signal detection.

Trial registration number

CRD42017072741.

Benefit-Risk Monitoring of Vaccines Using an Interactive Dashboard: A Methodological Proposal from the ADVANCE Project

INTRODUCTION

New vaccines are launched based on their benefit-risk (B/R) profile anticipated from clinical development. Proactive post-marketing surveillance is necessary to assess whether the vaccination uptake and the B/R profile are as expected and, ultimately, whether further public health or regulatory actions are needed. There are several, typically not integrated, facets of post-marketing vaccine surveillance: the surveillance of vaccination coverage, vaccine safety, effectiveness and impact.

OBJECTIVE

With this work, we aim to assess the feasibility and added value of using an interactive dashboard as a potential methodology for near real-time monitoring of vaccine coverage and pre-specified health benefits and risks of vaccines.

METHODS

We developed a web application with an interactive dashboard for B/R monitoring. The dashboard is demonstrated using simulated electronic healthcare record data mimicking the introduction of rotavirus vaccination in the UK. The interactive dashboard allows end users to select certain parameters, including expected vaccine effectiveness, age groups, and time periods and allows calculation of the incremental net health benefit (INHB) as well as the incremental benefit-risk ratio (IBRR) for different sets of preference weights. We assessed the potential added value of the dashboard by user testing amongst a range of stakeholders experienced in the post-marketing monitoring of vaccines.

RESULTS

The dashboard was successfully implemented and demonstrated. The feedback from the potential end users was generally positive, although reluctance to using composite B/R measures was expressed.

CONCLUSION

The use of interactive dashboards for B/R monitoring is promising and received support from various stakeholders. In future research, the use of such an interactive dashboard will be further tested with real-life data as opposed to simulated data.

Do GPs accurately record date of death? A UK observational analysis

OBJECTIVE

To examine the concordance between dates of death recorded in UK primary care and national mortality records.

METHODS

UK primary care data from the Clinical Practice Research Datalink were linked to Office for National Statistics (ONS) data, for 118 571 patients who died between September 2010 and September 2015. Logistic regression was used to examine factors associated with discrepancy in death dates between data sets.

RESULTS

Death dates matched in 76.8% of cases with primary care dates preceding ONS date in 2.9%, and following in 20.3% of cases; 92.2% of cases differed by <2 weeks. Primary care date was >4 weeks later than ONS in 1.5% of cases and occurred more frequently with deaths categorised as 'external' (15.8% vs 0.8% for cancer), and in younger patients (15.9% vs 1% for 18-29 and 80-89 years, respectively). General practices with the greatest discrepancies (97.5th percentile) had around 200 times higher odds of recording substantially discordant dates than practices with the lowest discrepancies (2.5th percentile).

CONCLUSION

Dates of death in primary care records often disagree with national records and should be treated with caution. There is marked variation between practices, and studies involving young patients, unexplained deaths and where precise date of death is important are particularly vulnerable to these issues.

Do delays in data availability limit the implementation of near real-time vaccine safety surveillance using the Clinical Practice Research Datalink?

Purpose

Near real‐time vaccine safety surveillance (NRTVSS) using electronic health records has been used to detect timely vaccine safety signals. Trial implementation of NRTVSS using the Clinical Practice Research Datalink (CPRD) has shown that there is limited power to detect safety signals for rare events. Delays in recording outcomes and receiving data influence the power and timeliness to identify a signal. Our work aimed to compare how different sources of delays influence power and expected time to signal to implement NRTVSS using CPRD.

Methods

We studied seasonal influenza vaccine/Guillain‐Barré syndrome and performed power and expected time to signal calculations for the 2013‐2014/2014‐2015 seasons. We used the Poisson‐based maximised sequential probability ratio test, which compares observed‐to‐expected events. For each study season, we obtained an average Guillain‐Barré syndrome/seizures age‐sex–adjusted rate from the 5 previous seasons and then used this rate to calculate the expected number of events, assuming a 42‐day risk‐window. Calculations were performed for detecting rate ratios of 1.5 to 10. We compared power and timeliness considering combinations of the presence/absence of delays in recording outcomes and in receiving data. The R‐package Sequential was used.

Results

In general, there was ≥80% power to detect increases in risk of ≥4 at the end of the season. Assuming absence of delays slightly improved power (a maximum increase of 4%) but did not noticeably reduce time to detect a signal.

Conclusion

Removing delays in data availability is insufficient to significantly improve the performance of a NRTVSS system using CPRD. Expansion of CPRD data is required.

Implementing near real-time vaccine safety surveillance using the Clinical Practice Research Datalink (CPRD)

INTRODUCTION

Near real-time vaccine safety surveillance (NRTVSS) using electronic health records is increasingly used to rapidly detect vaccine safety signals. NRTVSS has not been fully implemented in the UK. We assessed the feasibility of implementing this surveillance using the UK Clinical Practice Research Datalink (CPRD).

METHODS

We selected seasonal influenza vaccine/Guillain-Barré Syndrome (GBS) as an example of a rare outcome and measles-mumps-rubella (MMR) vaccine/febrile seizures as a positive control. For influenza/GBS we implemented a system for the 2013/2014 and 2014/2015 influenza seasons; for MMR/seizures the surveillance period was July 2014-June 2015. We used the continuous Poisson-based maximized sequential probability ratio test (PMaxSPRT), comparing observed-to-expected events, for both pairs. We calculated an age-sex-adjusted rate using 5years of historic data and used this rate to calculate the expected number of events in pre-specified post-vaccination risk-window (GBS: 0-42days, seizures: 6-21days). For MMR/seizures we also implemented the system using the Binominal-based maximized sequential probability ratio test (BMaxSPRT). For this, we compared seizures in the risk-window (6-21days) to a control window (0-5 and 22-32days). Delays in recording outcomes influence the data available, so we adjusted the expected number of events using a historical distribution of delays in recording GBS/febrile seizures. Analyses were run using data up to each CPRD monthly release. We also performed power calculations for detecting increases in relative risk (RR) from 1.5 to 10.

RESULTS

For influenza/GBS we implemented a system in both seasons with no signal. Power to detect a signal was >80% for RR≥4. For MMR/seizures we were able to identify a signal with PMaxSPRT but not with BMaxSPRT. Power≥80% for RR≥2.5 for both tests.

CONCLUSION

CPRD is a potential data source to implement NRTVSS to exclude large increases in the risk of rare outcomes after seasonal influenza and lower increases in risk for more frequent outcomes.

Assessing recording delays in general practice records to inform near real-time vaccine safety surveillance using the Clinical Practice Research Datalink (CPRD)

Purpose

Near real‐time vaccine safety surveillance (NRTVSS) is an option for post‐licensure vaccine safety assessment. NRTVSS requires timely recording of outcomes in the database used. Our main objective was to examine recording delays in the Clinical Practice Research Datalink (CPRD) for outcomes of interest for vaccine safety to inform the feasibility of NRTVSS using these data. We also evaluated completeness of recording and further assessed reporting delays for hospitalized events in CPRD.

Methods

We selected Guillain–Barré syndrome (GBS), Bell's palsy (BP), optic neuritis (ON) and febrile seizures (FS), from January 2005 to June 2014. We assessed recording delays (e.g. due to feedback from specialist referral) in stand‐alone CPRD by comparing the event and system dates and excluding delays >1 year. We used linked CPRD‐hospitalization data to further evaluate delays and completeness of recording in CPRD.

Results

Among 51 220 patients for the stand‐alone CPRD analysis (GBS: n = 830; BP: n = 12 602; ON: n = 1720; and FS: n = 36 236), most had a record entered within 1 month of the event date (GBS: 73.6%; BP: 93.4%; ON: 76.2%; and FS: 85.6%). A total of 13 482 patients, with a first record in hospital, were included for the analysis of linked data (GBS: n = 678; BP: n = 4060; ON: n = 485; and FS: n = 8321). Of these, <50% had a record in CPRD after 1 year (GBS: 41.3%; BP: 22.1%; ON: 22.4%; and FS: 41.8%).

Conclusion

This work shows that most diagnoses in CPRD for the conditions examined were recorded with delays of ≤30 days, making NRTVSS possible. The pattern of delays was condition‐specific and could be used to adjust for delays in the NRTVSS analysis. Despite low sensitivity of recording, implementing NRTVSS in CPRD is worthwhile and could be carried out, at least on a trial basis, for events of interest. © 2017 The Authors. Pharmacoepidemiology & Drug Safety Published by John Wiley & Sons Ltd.