An Implementation and Visualization of the Tree-Based Scan Statistic for Safety Event Monitoring in Longitudinal Electronic Health Data

Optimal Significance Levels and Sample Sizes for Signal Detection Methods Based on Non-constant Hazards

BACKGROUND AND OBJECTIVES

Statistical methods for signal detection of adverse drug reactions (ADRs) in electronic health records (EHRs) need information about optimal significance levels and sample sizes to achieve sufficient power. Sauzet and Cornelius proposed tests for signal detection based on the hazard functions of Weibull type distributions (WSP tests) which use the time-to-event information available in EHRs. Optimal significance levels and sample sizes for the application of the WPS tests are derived.

METHOD

A simulation study was performed with a range of scenarios for sample size, rate of event due (ADRs), and not due to the drug and random time to ADR occurrence. Based on the area under the curve of the receiver operating characteristic graph, we obtain optimal significance levels of the different WSP tests for the implementation in a hypothesis free signal detection setting and approximate sample sizes required to reach a power of 80% or 90%.

RESULTS

The dWSP-pPWSP (combination of double WSP and power WSP) test with a significance level of 0.004 was recommended. Sample sizes needed for a power of 80% were found to start at 60 events for an ADR rate equal to the background rate of 0.1. The number of events required for a background rate of 0.05 and an ADR rate equal to a 20% increase of the background rate was 900.

CONCLUSION

Based on this study, it is recommended to use the dWSP-pWSP test combination for signal detection with a significance level of 0.004 when the same test is applied to all adverse events not depending on rates.

Considerations for practical use of tree-based scan statistics for signal detection using electronic healthcare data: a case study with insulin glargine

BACKGROUND

Hypothesis-free signal detection (HFSD) methods such as tree-based scan statistics (TBSS) applied to longitudinal electronic healthcare data (EHD) are increasingly used in safety monitoring. However, challenges may arise in interpreting HFSD results alongside results from disproportionality analysis of spontaneous reporting.

RESEARCH DESIGN AND METHODS

Using the anti-diabetes drug insulin glargine (Lantus®) we apply two different tree-based scan designs using TreeScan™ software on retrospective EHD and compare the results to one another as well as to results from a disproportionality analysis using SRD.

RESULTS

The self-controlled tree temporal scan method produced the larger number of alerts relative to propensity-score matched approach; however, far fewer alerts were observed when analyses were limited to EHD in inpatient/emergency room settings only. Very few reference adverse events were observed using TBSS methods on EHD relative to disproportionality methods in SRD.

CONCLUSION

Differences in detected alerts between TBSS methods and between TBSS and disproportionality analysis of SRD are likely attributable to differences in data, comparator, and study design. Our results suggest that HFDS methods like TBSS applied to EHD may complement more traditional approaches such as disproportionality analysis of SRD to provide a more complete picture of product safety in the post-approval setting.

A New Drug Safety Signal Detection and Triage System Integrating Sequence Symmetry Analysis and Tree-Based Scan Statistics with Longitudinal Data

Purpose

Development and evaluation of a drug-safety signal detection system integrating data-mining tools in longitudinal data is essential. This study aimed to construct a new triage system using longitudinal data for drug-safety signal detection, integrating data-mining tools, and evaluate adaptability of such system.

Patients and Methods

Based on relevant guidelines and structural frameworks in Taiwan's pharmacovigilance system, we constructed a triage system integrating sequence symmetry analysis (SSA) and tree-based scan statistics (TreeScan) as data-mining tools for detecting safety signals. We conducted an exploratory analysis utilizing Taiwan's National Health Insurance Database and selecting two drug classes (sodium-glucose co-transporter-2 inhibitors (SGLT2i) and non-fluorinated quinolones (NFQ)) as chronic and episodic treatment respectively, as examples to test feasibility of the system.

Results

Under the proposed system, either cohort-based or self-controlled mining with SSA and TreeScan was selected, based on whether the screened drug had an appropriate comparator. All detected alerts were further classified as known adverse drug reactions (ADRs), events related to other causes or potential signals from the triage algorithm, building on existing drug labels and clinical judgement. Exploratory analysis revealed greater numbers of signals for NFQ with a relatively low proportion of known ADRs; most were related to indication, patient characteristics or bias. No safety signals were found. By contrast, most SGLT2i signals were known ADRs or events related to patient characteristics. Four were potential signals warranting further investigation.

Conclusion

The proposed system facilitated active and systematic screening to detect and classify potential safety signals. Countries with real-world longitudinal data could adopt it to streamline drug-safety surveillance.

Machine Learning in Causal Inference: Application in Pharmacovigilance

Monitoring adverse drug events or pharmacovigilance has been promoted by the World Health Organization to assure the safety of medicines through a timely and reliable information exchange regarding drug safety issues. We aim to discuss the application of machine learning methods as well as causal inference paradigms in pharmacovigilance. We first reviewed data sources for pharmacovigilance. Then, we examined traditional causal inference paradigms, their applications in pharmacovigilance, and how machine learning methods and causal inference paradigms were integrated to enhance the performance of traditional causal inference paradigms. Finally, we summarized issues with currently mainstream correlation-based machine learning models and how the machine learning community has tried to address these issues by incorporating causal inference paradigms. Our literature search revealed that most existing data sources and tasks for pharmacovigilance were not designed for causal inference. Additionally, pharmacovigilance was lagging in adopting machine learning-causal inference integrated models. We highlight several currently trending directions or gaps to integrate causal inference with machine learning in pharmacovigilance research. Finally, our literature search revealed that the adoption of causal paradigms can mitigate known issues with machine learning models. We foresee that the pharmacovigilance domain can benefit from the progress in the machine learning field.

OUP accepted manuscript

Aim

Methods

Results

Conclusions

Safety of medicines and vaccines - building next generation capability

The systematic safety surveillance of real-world use of medicinal products and related activities (pharmacovigilance) started in earnest as a scientific field only in the 1960s. While developments have occurred over the past 50 years, adding to its complexity and sophistication, the extent to which some of these advances have positively impacted the capability for ensuring patient safety is questionable. We review how the conduct of safety surveillance has changed, highlight recent scientific advances, and argue how they need to be harnessed to enhance pharmacovigilance in the future. Specifically, we describe five changes that we believe should and will need to happen globally in the coming years: (i) better, more diverse data used for safety; (ii) the switch from manual activities to automation; (iii) removal of limited value, extraneous transactional activities and replacement with sharpened focus on scientific efforts to improve patient safety; (iv) patient-involved and focussed safety; and (v) personalised safety.

A predictive paradigm for COVID-19 prognosis based on the longitudinal measure of biomarkers

Novel coronavirus disease 2019 (COVID-19) is an emerging, rapidly evolving crisis, and the ability to predict prognosis for individual COVID-19 patient is important for guiding treatment. Laboratory examinations were repeatedly measured during hospitalization for COVID-19 patients, which provide the possibility for the individualized early prediction of prognosis. However, previous studies mainly focused on risk prediction based on laboratory measurements at one time point, ignoring disease progression and changes of biomarkers over time. By using historical regression trees (HTREEs), a novel machine learning method, and joint modeling technique, we modeled the longitudinal trajectories of laboratory biomarkers and made dynamically predictions on individual prognosis for 1997 COVID-19 patients. In the discovery phase, based on 358 COVID-19 patients admitted between 10 January and 18 February 2020 from Tongji Hospital, HTREE model identified a set of important variables including 14 prognostic biomarkers. With the trajectories of those biomarkers through 5-day, 10-day and 15-day, the joint model had a good performance in discriminating the survived and deceased COVID-19 patients (mean AUCs of 88.81, 84.81 and 85.62% for the discovery set). The predictive model was successfully validated in two independent datasets (mean AUCs of 87.61, 87.55 and 87.03% for validation the first dataset including 112 patients, 94.97, 95.78 and 94.63% for the second validation dataset including 1527 patients, respectively). In conclusion, our study identified important biomarkers associated with the prognosis of COVID-19 patients, characterized the time-to-event process and obtained dynamic predictions at the individual level.