Prediction of post-vaccination Guillain-Barré syndrome using data from a passive surveillance system

AE-GPT: Using Large Language Models to extract adverse events from surveillance reports-A use case with influenza vaccine adverse events

Though Vaccines are instrumental in global health, mitigating infectious diseases and pandemic outbreaks, they can occasionally lead to adverse events (AEs). Recently, Large Language Models (LLMs) have shown promise in effectively identifying and cataloging AEs within clinical reports. Utilizing data from the Vaccine Adverse Event Reporting System (VAERS) from 1990 to 2016, this study particularly focuses on AEs to evaluate LLMs' capability for AE extraction. A variety of prevalent LLMs, including GPT-2, GPT-3 variants, GPT-4, and Llama2, were evaluated using Influenza vaccine as a use case. The fine-tuned GPT 3.5 model (AE-GPT) stood out with a 0.704 averaged micro F1 score for strict match and 0.816 for relaxed match. The encouraging performance of the AE-GPT underscores LLMs' potential in processing medical data, indicating a significant stride towards advanced AE detection, thus presumably generalizable to other AE extraction tasks.

Guillain-Barre syndrome following COVID-19 vaccines: A review of literature

Objective

This study aimed to retrospectively analyze reported Guillain-Barré syndrome (GBS) cases that occurred after COVID-19 vaccination.

Methods

Case reports of GBS following COVID-19 vaccination that were published before May 14, 2022, were retrieved from PubMed. The cases were retrospectively analyzed for their basic characteristics, vaccine types, the number of vaccination doses before onset, clinical manifestations, laboratory test results, neurophysiological examination results, treatment, and prognosis.

Results

Retrospective analysis of 60 case reports revealed that post-COVID-19 vaccination GBS occurred mostly after the first dose of the vaccination (54 cases, 90%) and was common for DNA vaccination (38 cases, 63%), common in middle-aged and elderly people (mean age: 54.5 years), and also common in men (36 cases, 60%). The mean time from vaccination to onset was 12.3 days. The classical GBS (31 cases, 52%) was the major clinical classification and the AIDP subtype (37 cases, 71%) was the major neurophysiological subtype, but the positive rate of anti-ganglioside antibodies was low (7 cases, 20%). Bilateral facial nerve palsy (76% vs 18%) and facial palsy with distal paresthesia (38% vs 5%) were more common for DNA vaccination than for RNA vaccination.

Conclusion

After reviewing the literature, we proposed a possible association between the risk of GBS and the first dose of the COVID-19 vaccines, especially DNA vaccines. The higher rate of facial involvement and a lower positive rate of anti-ganglioside antibodies may be a characteristic feature of GBS following COVID-19 vaccination. The causal relationship between GBS and COVID-19 vaccination remains speculative, more research is needed to establish an association between GBS and COVID-19 vaccination. We recommend surveillance for GBS following vaccination, because it is important in determining the true incidence of GBS following COVID-19 vaccination, as well as in the development of a more safer vaccine.

Vaccine Vigilance System: Considerations on the Effectiveness of Vigilance Data Use in COVID-19 Vaccination

(1) Background: The safety of medicines has been receiving increased attention to ensure that the risks of taking medicines do not outweigh the benefits. This is the reason why, over several decades, the pharmacovigilance system has been developed. The post-authorization pharmacovigilance system is based on reports from healthcare professionals and patients on observed adverse reactions. The reports are collected in databases and progressively evaluated. However, there are emerging concerns about the effectiveness of the established passive pharmacovigilance system in accelerating circumstances, such as the COVID-19 pandemic, when billions of doses of new vaccines were administered without a long history of use. Currently, health professionals receive fragmented new information on the safety of medicines from competent authorities after a lengthy evaluation process. Simultaneously, in the context of accelerated mass vaccination, health professionals need to have access to operational information-at least on organ systems at higher risk. Therefore, the aim of this study was to perform a primary data analysis of publicly available data on suspected COVID-19 vaccine-related adverse reactions in Europe, in order to identify the predominant groups of reported medical conditions after vaccination and their association with vaccine groups, as well as to evaluate the data accessibility on specific syndromes. (2) Methods: To achieve the objectives, the data publicly available in the EudraVigilance European Database for Suspected Adverse Drug Reaction Reports were analyzed. The following tasks were defined to: (1) Identify the predominant groups of medical conditions mentioned in adverse reaction reports; (2) determine the relative frequency of reports within vaccine groups; (3) assess the feasibility of obtaining information on a possibly associated syndrome-myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS). (3) Results: The data obtained demonstrate that the predominant medical conditions induced after vaccination are relevant to the following categories: (1) "General disorders and administration site conditions", (2) "nervous system disorders", and (3) "musculoskeletal and connective tissue disorders". There are more reports for mRNA vaccines, but the relative frequency of reports per dose administered, is lower for this group of vaccines. Information on ME/CFS was not available, but reports of "chronic fatigue syndrome" are included in the database and accessible for primary analysis. (4) Conclusions: The information obtained on the predominantly reported medical conditions and the relevant vaccine groups may be useful for health professionals, patients, researchers, and medicine manufacturers. Policymakers could benefit from reflecting on the design of an active pharmacovigilance model, making full use of modern information technologies, including big data analysis of social media and networks for the detection of primary signals and building an early warning system.

Usefulness of Vaccine Adverse Event Reporting System for Machine-Learning Based Vaccine Research: A Case Study for COVID-19 Vaccines

Usefulness of Vaccine-Adverse Event-Reporting System (VAERS) data and protocols required for statistical analyses were pinpointed with a set of recommendations for the application of machine learning modeling or exploratory analyses on VAERS data with a case study of COVID-19 vaccines (Pfizer-BioNTech, Moderna, Janssen). A total of 262,454 duplicate reports (29%) from 905,976 reports were identified, which were merged into a total of 643,522 distinct reports. A customized online survey was also conducted providing 211 reports. A total of 20 highest reported adverse events were first identified. Differences in results after applying various machine learning algorithms (association rule mining, self-organizing maps, hierarchical clustering, bipartite graphs) on VAERS data were noticed. Moderna reports showed injection-site-related AEs of higher frequencies by 15.2%, consistent with the online survey (12% higher reporting rate for pain in the muscle for Moderna compared to Pfizer-BioNTech). AEs {headache, pyrexia, fatigue, chills, pain, dizziness} constituted >50% of the total reports. Chest pain in male children reports was 295% higher than in female children reports. Penicillin and sulfa were of the highest frequencies (22%, and 19%, respectively). Analysis of uncleaned VAERS data demonstrated major differences from the above (7% variations). Spelling/grammatical mistakes in allergies were discovered (e.g., ~14% reports with incorrect spellings for penicillin).

Comparability of clinical trials and spontaneous reporting data regarding COVID-19 vaccine safety

Severe adverse events (AEs) after COVID-19 vaccination are not well studied in randomized controlled trials (RCTs) due to rarity and short follow-up. To monitor the safety of COVID-19 vaccines ("Pfizer" vaccine dose 1 and 2, "Moderna" vaccine dose 1 and 2, and "Janssen" vaccine single dose) in the U.S., especially regarding severe AEs, we compare the relative rankings of these vaccines using both RCT and the Vaccine Adverse Event Reporting System (VAERS) data. The risks of local and systemic AEs were assessed from the three pivotal COVID-19 vaccine trials and also calculated in the VAERS cohort consisting of 559,717 reports between December 14, 2020 and September 17, 2021. AE rankings of the five vaccine groups calculated separately by RCT and VAERS were consistent, especially for systemic AEs. For severe AEs reported in VAERS, the reported risks of thrombosis and GBS after Janssen vaccine were highest. The reported risk of shingles after the first dose of Moderna vaccine was highest, followed by the second dose of the Moderna vaccine. The reported risk of myocarditis was higher after the second dose of Pfizer and Moderna vaccines. The reported risk of anaphylaxis was higher after the first dose of Pfizer vaccine. Limitations of this study are the inherent biases of the spontaneous reporting system data, and only including three pivotal RCTs and no comparison with other active vaccine safety surveillance systems.

Prediction of post-vaccination Guillain-Barré syndrome using data from a passive surveillance system

Purpose

Severe adverse events (AEs), such as Guillain‐Barré syndrome (GBS) occur rarely after influenza vaccination. We identify highly associated AEs with GBS and develop prediction models for GBS using the US Vaccine Adverse Event Reporting System (VAERS) reports following trivalent influenza vaccination (FLU3).

Methods

This study analyzed 80 059 reports from the US VAERS between 1990 and 2017. Several AEs were identified as highly associated with GBS and were used to develop the prediction model. Some common and mild AEs that were suspected to be underreported when GBS occurred simultaneously were removed from the final model. The analyses were validated using European influenza vaccine AEs data from EudraVigilance.

Results

Of the 80 059 reports, 1185 (1.5%) were annotated as GBS related. Twenty‐four AEs were identified as having strong association with GBS. The full prediction model, using age, sex, and all 24 AEs achieved an area under the receiver operating characteristic (ROC) curve (AUC) of 85.4% (90% CI: [83.8%, 86.9%]). After excluding the nine (e.g., pruritus, rash, injection site pain) likely underreported AEs, the final AUC became 77.5% (90% CI: [75.5%, 79.6%]). Two hundred and one (0.25%) reports were predicted as of high risk of GBS (predicted probability >25%) and 84 actually developed GBS.

Conclusion

The prediction performance demonstrated the potential of developing risk‐prediction models utilizing the VAERS cohort. Excluding the likely underreported AEs sacrificed some prediction power but made the model more interpretable and feasible. The high absolute risk of even a small number of AE combinations suggests the promise of GBS prediction within the VAERS dataset.

Bayesian Network to Predict the Risk of Post Influenza Vaccination Guillain-Barré Syndrome: A Development and Validation Study (Preprint)

Background

Identifying the key factors of Guillain-Barré syndrome (GBS) and predicting its occurrence are vital for improving the prognosis of patients with GBS. However, there are scarcely any publications on a forewarning model of GBS. A Bayesian network (BN) model, which is known to be an accurate, interpretable, and interaction-sensitive graph model in many similar domains, is worth trying in GBS risk prediction.

Objective

The aim of this study is to determine the most significant factors of GBS and further develop and validate a BN model for predicting GBS risk.

Methods

Large-scale influenza vaccine postmarketing surveillance data, including 79,165 US (obtained from the Vaccine Adverse Event Reporting System between 1990 and 2017) and 12,495 European (obtained from the EudraVigilance system between 2003 and 2016) adverse events (AEs) reports, were extracted for model development and validation. GBS, age, gender, and the top 50 prevalent AEs were included for initial BN construction using the R package bnlearn.

Results

Age, gender, and 10 AEs were identified as the most significant factors of GBS. The posttest probability of GBS suggested that male vaccinees aged 50-64 years and without erythema should be on the alert or be warned by clinicians about an increased risk of GBS, especially when they also experience symptoms of asthenia, hypesthesia, muscular weakness, or paresthesia. The established BN model achieved an area under the receiver operating characteristic curve of 0.866 (95% CI 0.865-0.867), sensitivity of 0.752 (95% CI 0.749-0.756), specificity of 0.882 (95% CI 0.879-0.885), and accuracy of 0.882 (95% CI 0.879-0.884) for predicting GBS risk during the internal validation and obtained values of 0.829, 0.673, 0.854, and 0.843 for area under the receiver operating characteristic curve, sensitivity, specificity, and accuracy, respectively, during the external validation.

Conclusions

The findings of this study illustrated that a BN model can effectively identify the most significant factors of GBS, improve understanding of the complex interactions among different postvaccination symptoms through its graphical representation, and accurately predict the risk of GBS. The established BN model could further assist clinical decision-making by providing an estimated risk of GBS for a specific vaccinee or be developed into an open-access platform for vaccinees’ self-monitoring.