A Method for the Minimization of Competition Bias in Signal Detection from Spontaneous Reporting Databases

The evolving role of disproportionality analysis in pharmacovigilance

INTRODUCTION

From 2009 to 2015, the IMI PROTECT conducted rigorous studies addressing questions about optimal implementation and significance of disproportionality analyses, leading to the development of Good Signal Detection Practices. The ensuing period witnessed the independent exploration of research paths proposed by IMI PROTECT, accumulating valuable experience and insights that have yet to be seamlessly integrated.

AREAS COVERED

This state-of-the-art review integrates IMI PROTECT recommendations with recent acquisitions and evolving challenges. It deals with defining the object of study, disproportionality methods, subgrouping, masking, drug-drug interaction, duplication, expectedness, the debated use of disproportionality results as risk measures, integration with other types of data.

EXPERT OPINION

Despite the ongoing skepticism regarding the usefulness of disproportionality analyses and individual case safety reports, their ability to timely detect safety signals regarding rare and unpredictable adverse reactions remains unparalleled. Moreover, recent exploration into their potential for characterizing safety signals revealed valuable insights concerning potential risk factors and the patient's perspective. To fully realize their potential beyond hypothesis generation and achieve a comprehensive evidence synthesis with other kinds of data and studies, each with their unique limitations and contributions, we need to investigate methods for more transparently communicating disproportionality results and mapping and addressing pharmacovigilance biases.

Impact of Covid-19 Vaccination on Spontaneous Pharmacovigilance Reporting in France

INTRODUCTION

In 2021, the massive Covid-19 vaccination campaign in France was accompanied by an intensified pharmacovigilance monitoring of their potential adverse drug reactions. The importance of this reporting might have led to an important selective reporting and overloading of Pharmacovigilance Centres, delaying the recording of some reports in the national pharmacovigilance database. In this context, we aimed to evaluate the impact of the Covid-19 vaccination campaign in France and related reports on spontaneous reporting of adverse drug reactions that were not related to the Covid-19 vaccine.

METHODS

We performed time-series analyses considering the monthly number of adverse drug reactions reported between January 1, 2018 and April 30, 2022 using the French Pharmacovigilance database. The impact of the Covid-19 vaccination campaign on the monthly reporting not Covid-19 vaccine related was estimated using interrupted time-series. January 2021, marking the start of the campaign, was the intervention date in the models. Analyses were run globally first considering all adverse drug reaction reports, and second according to notifier type and to case seriousness.

RESULTS

We included 170,294 reports registered in the French Pharmacovigilance database between January 1, 2018 and April 30, 2022 that were not Covid-19 vaccine-related. Among these, 77,067 (45.3%) were serious and 146,683 (86.1%) had been reported by health care professionals. The campaign start was associated with a nearly 35.0% decrease in average monthly reporting that was not Covid-19 vaccine-related, with a significant level decrease in the monthly number of reports of -658.0 (p < 10-3) immediately after the vaccination campaign start and a subsequent slope decrease of -50.0 (p < 10-3). This decrease was mainly due to a significant level and slope decrease (level: -739.2 p < 10-3; slope: -39 [p < 10-2]) for health care professional reports. A similar level decrease was found for the monthly number of both serious and non-serious reports (-402.3, p < 10-3; and -311.9, p = 10-2, respectively). According to the ATC 1 level, the decrease in the monthly number of reports showed similar patterns for all drugs. However, a potential increase in the number of serious reports suspecting antineoplastic and immunomodulating drugs (ATC L) or drugs targeting blood was observed (ATC B).

CONCLUSION

Our study showed a significant impact of the Covid-19 campaign vaccination in the reporting of adverse drug reactions that were not Covid-19 vaccine-related, of roughly 35%. This leads to a loss of information regarding the monitoring of drug safety that could have impacted the system capacity to detect safety signals for drugs other than Covid-19 vaccines.

High-Dimensional Propensity Score-Adjusted Case-Crossover for Discovering Adverse Drug Reactions from Computerized Administrative Healthcare Databases

INTRODUCTION

Increasing availability of medico-administrative databases has prompted the development of automated pharmacovigilance signal detection methodologies. Self-controlled approaches have recently been proposed. They account for time-independent confounding factors that may not be recorded. So far, large numbers of drugs have been screened either univariately or with LASSO penalized regressions.

OBJECTIVE

We propose and assess a new method that combines the case-crossover self-controlled design with propensity scores (propensity score-adjusted case-crossover) built from high-dimensional data-driven variable selection, to account for co-medications or possibly other measured confounders.

METHODS

Comparison with the univariate and LASSO case-crossover was performed from simulations and a real-data study. Multiple regressions (LASSO, propensity score-adjusted case-crossover) accounted for co-medications and no other covariates. For the univariate and propensity score-adjusted case-crossover methods, the detection threshold was based on a false discovery rate procedure, while for LASSO, it relied on the Akaike Information Criterion. For the real-data study, two drug safety experts evaluated the signals generated from the analysis of 4099 patients with acute myocardial infarction from the French national health database.

RESULTS

On simulations, our approach ranked the signals similarly to the LASSO and better than the univariate method while controlling the false discovery rate at the prespecified level, contrary to the univariate method. The LASSO provided the best sensitivity at the cost of larger false discovery rate estimates. On the application, our approach showed similar performances to the LASSO and better performances than the univariate method. It highlighted 43 signals out of 609 drug candidates: 22 (51%) were considered as potentially pharmacologically relevant, including seven (16%) regarded as highly relevant.

CONCLUSIONS

Our findings show the interest of a propensity score combined with a case-crossover for pharmacovigilance. They also confirm that indication bias remains a challenge when mining medico-administrative databases.

Development of a Network-Based Signal Detection Tool: The COVID-19 Adversome in the FDA Adverse Event Reporting System

Introduction: The analysis of pharmacovigilance databases is crucial for the safety profiling of new and repurposed drugs, especially in the COVID-19 era. Traditional pharmacovigilance analyses-based on disproportionality approaches-cannot usually account for the complexity of spontaneous reports often with multiple concomitant drugs and events. We propose a network-based approach on co-reported events to help assessing disproportionalities and to effectively and timely identify disease-, comorbidity- and drug-related syndromes, especially in a rapidly changing low-resources environment such as that of COVID-19. Materials and Methods: Reports on medications administered for COVID-19 were extracted from the FDA Adverse Event Reporting System quarterly data (January-September 2020) and queried for disproportionalities (Reporting Odds Ratio corrected for multiple comparisons). A network (the Adversome) was estimated considering events as nodes and conditional co-reporting as links. Communities of significantly co-reported events were identified. All data and scripts employed are available in a public repository. Results: Among the 7,082 COVID-19 reports extracted, the seven most frequently suspected drugs (remdesivir, hydroxychloroquine, azithromycin, tocilizumab, lopinavir/ritonavir, sarilumab, and ethanol) have shown disproportionalities with 54 events. Of interest, myasthenia gravis with hydroxychloroquine, and cerebrovascular vein thrombosis with azithromycin. Automatic clustering identified 13 communities, including a methanol-related neurotoxicity associated with alcohol-based hand-sanitizers and a long QT/hepatotoxicity cluster associated with azithromycin, hydroxychloroquine and lopinavir-ritonavir interactions. Conclusion: Findings from the Adversome detect plausible new signals and iatrogenic syndromes. Our network approach complements traditional pharmacovigilance analyses, and may represent a more effective signal detection technique to guide clinical recommendations by regulators and specific follow-up confirmatory studies.

New adaptive lasso approaches for variable selection in automated pharmacovigilance signal detection

Background

Adverse effects of drugs are often identified after market introduction. Post-marketing pharmacovigilance aims to detect them as early as possible and relies on spontaneous reporting systems collecting suspicious cases. Signal detection tools have been developed to mine these large databases and counts of reports are analysed with disproportionality methods. To address disproportionality method biases, recent methods apply to individual observations taking into account all exposures for the same patient. In particular, the logistic lasso provides an efficient variable selection framework, yet the choice of the regularization parameter is a challenging issue and the lasso variable selection may give inconsistent results.

Methods

We propose a new signal detection methodology based on the adaptive lasso. We derived two new adaptive weights from (i) a lasso regression using the Bayesian Information Criterion (BIC), and (ii) the class-imbalanced subsampling lasso (CISL), an extension of stability selection. The BIC is used in the adaptive lasso stage for variable selection. We performed an extensive simulation study and an application to real data, where we compared our methods to the existing adaptive lasso, and recent detection approaches based on lasso regression or propensity scores in high dimension. For both studies, we evaluate the methods in terms of false discoveries and sensitivity.

Results

In the simulations and the application, both proposed adaptive weights show equivalent or better performances than the other competitors, with an advantage for the CISL-based adaptive weights. CISL and lasso regression using BIC are solid alternatives.

Conclusion

Our proposed adaptive lasso is an appealing methodology for signal detection in pharmacovigilance. Although we cannot rely on test theory, our approaches show a low and stable False Discovery Rate in all simulation settings. All methods evaluated in this work are implemented in the adapt4pv R package.

Supplementary Information

The online version contains supplementary material available at (10.1186/s12874-021-01450-3).

Thromboembolic Events with Cyclin-Dependent Kinase 4/6 Inhibitors in the FDA Adverse Event Reporting System

Simple Summary

This post-marketing research addressed the role of cyclin-dependent kinase 4/6 inhibitors, relatively new anticancer drugs approved for advanced breast cancer in cancer-associated thrombosis. We used the Food and Drug Administration pharmacovigilance database to retrospectively assess thromboembolic events in the real world. A potential class effect was found for venous thrombosis, whereas distinctive arterial events emerged for ribociclib. These signals call for both prospective research and early proactive monitoring by oncologists, also in patients without apparent risk factors. These findings strengthen the role of timely pharmacovigilance to detect and characterize post-marketing adverse events of special interest, thus supporting patient care.

Abstract

We analyzed thromboembolic events, recognized (AESIs), with cyclin-dependent kinase (CDK)4/6 inhibitors, using the Food and Drug Administration adverse event reporting system. Methods: Thromboembolic events were characterized in terms of spectrum [venous and arterial thromboembolism (VTE; ATE)] and clinical features by combining the disproportionality approach [reporting odds ratio (ROR) with 95% confidence interval (CI)] with individual case assessment. Results: A total of 1722 thromboembolic events were retained. Increased VTE reporting emerged for CDK4/6 inhibitors in the exploratory analyses (n = 659; ROR = 1.51; 95% CI = 1.39–1.63), with consistent disproportionality in the consolidated analyses (e.g., deep vein thrombosis with abemaciclib: 17; 1.98; 1.22–3.19). Higher-than-expected ATE reporting was found for ribociclib, including myocardial infarction (41; 1.82; 1.33–2.48), with rapid onset (median latency 1 vs. 6 months for other CDK4/6 inhibitors). Causality was highly probable or probable in 83.2% of cases, with a negligible proportion of pre-existing drug- and patient-related risk factors except for cardiovascular comorbidities (26%). Conclusions: Although causal association cannot be firmly inferred, oncologists should proactively monitor the occurrence of VTE with CDK4/6 inhibitors. The unexpected distinctive increased ATE reporting with ribociclib deserves urgent clarification though large comparative population-based studies. We support pharmacovigilance for the post-marketing characterization of AESIs, thus promoting real-time safe prescribing in oncology.

A safety signal of somnambulism with the use of antipsychotics and lithium: A pharmacovigilance disproportionality analysis

AIMS

Antipsychotics and lithium are widely used in psychiatry, particularly in schizophrenia and bipolar disorders. Recently, some cases of somnambulism or sleep-related eating disorder (SRED) have been reported in patients treated with these drugs. This study investigated the risk of reporting somnambulism or SRED associated with the use of antipsychotics and lithium.

METHODS

The World Health Organization pharmacovigilance database (VigiBase), comprising >18 million adverse events, was queried. All somnambulism or SRED reports related to antipsychotics or lithium were identified. The association between antipsychotics or lithium and somnambulism or SRED was computed using the proportional reporting ratio (PRR) and information component.

RESULTS

Among the 5784 cases reporting somnambulism or SRED, 508 suspected at least 1 antipsychotic or lithium. Most patients were aged 18-64 years (62.0%), and 37.0% were men. In most cases (77.6%), antipsychotic or lithium were the only drug class involved, and 53.3% of cases suspected quetiapine. Somnambulism was reported in 88.6% of cases and SRED in 18.1%. A significant association was found for second-generation antipsychotics (PRR 3.44, 95% confidence interval 3.13) and lithium (PRR 2.03, [1.22; 3.37]), but not for first-generation antipsychotics (PRR 0.99, [0.68; 1.44]).

CONCLUSIONS

We found a significant signal of somnambulism or SRED related to second-generation antipsychotics and lithium. While case reports mentioned mostly quetiapine and olanzapine, almost all second-generation antipsychotics were associated with somnambulism or SRED.

SGLT-2 inhibitors and atrial fibrillation in the Food and Drug Administration adverse event reporting system

Background

Sodium glucose cotransporter-2 inhibitors (SGLT2i) reduce the risk of heart failure and new data show they can prevent atrial fibrillation (AF). We examined the association between SGLT2i and AF in the Food and Drug Administration adverse event reporting system (FAERS).

Methods

We mined the FAERS from 2014q1 to 2019q4 to compare AF reporting for SGLT-2 i versus reports for other glucose lowering medications (ATC10 class). Several exclusions were sequentially applied for: concomitant medications; diabetes, cardiovascular or renal disease indication; reports for competing adverse events (genitourinary tract infections, ketoacidosis, Fournier’s gangrene, amputation). We provide descriptive statistics and calculated proportional reporting ratios (PRR).

Results

There were 62,098 adverse event reports for SGLT2i and 642,031 reports for other ATC10 drugs. The reporting of AF was significantly lower with SGLT2i than with other ATC10 drugs (4.8 versus 8.7/1000; p < 0.001) with a PRR of 0.55 (0.49–0.62). Results did not change substantially after excluding reports listing insulin (PRR 0.49) or anti-arrhythmics (PRR 0.59) as suspect or concomitant drugs, excluding reports with indications for cardiovascular disease (PRR 0.49) or renal disease (PRR 0.55), and those filed for competing adverse events (PRR 0.63). Results were always statistically significant whether the diabetes indication was specified. Negative and positive controls confirmed internal validity of the database.

Conclusions

In a large pharmacovigilance database, AF was robustly and consistently reported more frequently for diabetes medications other than SGLT2i. This finding complements available evidence from trials supporting a protective role of SGLT2i against the occurrence of AF.

Cyclin-dependent kinase 4/6 inhibitors and interstitial lung disease in the FDA adverse event reporting system: a pharmacovigilance assessment

PURPOSE

We assessed pulmonary toxicity of cyclin-dependent kinase (CDK)4/6 inhibitors by analyzing the publicly available FDA Adverse Event Reporting System (FAERS).

METHODS

Reports of interstitial lung disease (ILD) were characterized in terms of demographic information, including daily dose, latency, concomitant drugs known to be associated with ILD, and causality assessment (adapted WHO system). Disproportionality analyses were carried out by calculating reporting odds ratios (RORs) with 95% confidence interval (CI), accounting for major confounders, including notoriety and competition biases.

RESULTS

ILD reports (N = 161) represented 2.1% and 0.3% of all reports for abemaciclib and palbocilcib/ribociclib, respectively, with negligible proportion of concomitant pneumotoxic drugs. Increased reporting was found for CDK4/6 inhibitors when compared to other drugs (ROR = 1.50; 95%CI = 1.28-1.74), and abemaciclib vs other anticancer agents (4.70; 3.62-5.98). Sensitivity analyses confirmed a strong and consistent disproportionality for abemaciclib. Higher-than-expected reporting emerged for palbociclib (1.38; 1.07-1.77) and ribociclib (2.39; 1.34-3.92) only when removing Japan reports. ILD occurred at recommended daily doses, with median latency ranging from 50 (abemaciclib) to 253 (ribociclib) days. Causality was highly probable in 55% of abemaciclib cases, probable in 68% of palbociclib cases.

CONCLUSIONS

Increased reporting of ILD with CDK4/6 inhibitors calls for further comparative population-based studies to characterize and quantify the actual risk, taking into account drug- and patient-related risk factors. These findings strengthen the role of (a) timely pharmacovigilance to detect post-marketing signals through FAERS and other real-world data, (b) clinicians to assess early, on a case-by-case basis, the potential responsibility of CDK4/6 inhibitors when diagnosing a lung injury.

Adverse drug reaction risks obtained from meta-analyses and pharmacovigilance disproportionality analyses are correlated in most cases

OBJECTIVE

We aimed at testing if a correlation between adverse drug reactions relative risks estimated from meta-analyses and disproportionality analyses calculated from pharmacovigilance spontaneous reporting systems databases exist, and if methodological choices modify this correlation.

STUDY DESIGN

We extracted adverse drug reactions (ADR) odds ratios (ORs) from meta-analyses used as reference and calculated corresponding Reporting Odds Ratios (RORs) from the WHO pharmacovigilance database according to five different designs. We also calculated the relative bias and agreement of ROR compared to ORs.

RESULTS

We selected five meta-analyses which displayed a panel of 13 ADRs. A significant correlation for 7 out of the 13 ADRs studied in the primary analysis was found. The methods for ROR calculation impacted the results but none systematically improved the correlations. Whereas correlation was found between OR and ROR, agreement was poor and relative bias was important.

CONCLUSION

Despite the large variation in disproportionality analyses results due to design specification, this study provides further evidence that relative risks obtained from meta-analyses and from disproportionality analyses correlate in most cases, in particular for objective ADR not associated with the underlying pathology.

Cardiovascular Toxicity of Tyrosine Kinase Inhibitors Used in Chronic Myeloid Leukemia: An Analysis of the FDA Adverse Event Reporting System Database (FAERS)

Tyrosine kinase inhibitors (TKIs), the treatment of choice for chronic myeloid leukemia (CML), can be associated to cardiovascular (CV) adverse events (AEs). A case/non-case study was performed using AE reports registered in the Food and Drug Administration (FDA) Adverse Event Reporting System (FAERS) database to compare the risk of CV event reports related to TKIs indicated in the management of chronic myeloid leukemia (CML). Disproportionality of CV event-related TKIs was computed using the Reporting Odds Ratio (ROR) as a measure of potential risk increase. Nilotinib accounts for more than half of reported cases related to TKIs. Signal of Disproportionate Reporting (SDR) was found for cardiac failure, ischemic heart disease, cardiac arrhythmias, torsade de pointes/QT prolongation, hypertension, and pulmonary hypertension. Dasatinib and bosutinib were related to the highest disproportionality for cardiac failure. Nilotinib was associated with the highest SDR for ischemic heart disease, torsade de pointes/QT prolongation and cardiac arrhythmias. Only ponatinib was related to an SDR for hypertension, while dasatinib and imatinib were related to pulmonary hypertension. In the context of CML, TKIs have different safety profiles related to CV events, among which nilotinib seems particularly related to. These results claim for a revision of its CV safety profile mainly for the risk of torsade de pointes/QT prolongation.

Uses of pharmacovigilance databases: An overview

Over the past decades, assessment of drug safety and of their benefits harms balance has been profoundly modified by the availability of large databases and computerized automated statistical approaches. Improvement of digital data storage capacity has been applied to pharmacovigilance reports. VigiBase, the international pharmacovigilance database, is now aggregating over 21 million individual case safety reports in 2020. Identification and investigation of drug safety signals - concerning notably rare and unknown adverse drug reactions - is one of the major tasks in pharmacovigilance that can be amplified by automated signal detection. Several quantitative statistical methods exist, each with its own strengths and limits. Integrating signal detection, pharmacovigilance databases can be used for a wide variety of retrospective observational studies illustrated here by concrete examples. Confirming these signals by orthogonal validation using pre-clinical platforms and prospective trials is helpful. Pharmacovigilance databases represent a considerable source of information. However, the quality of signal detection and of pharmacoepidemiology studies in the field of adverse drug reaction closely depends on the quality of the individual data recorded.

A stratification method based on clustering for the minimization of data masking effect in signal detection

Background

Data masking is an inborn defect of measures of disproportionality in adverse drug reactions (ADRs) signal detection. Many previous studies can be roughly classified into three categories: data removal, regression and stratification. However, frequency differences of adverse drug events (ADEs) reports, which would be an important factor of masking, were not considered in these methods. The aim of this study is to explore a novel stratification method for minimizing the impact of frequency differences on real signals masking.

Methods

Reports in the Chinese Spontaneous Reporting Database (CSRD) between 2010 and 2011 were selected. The overall dataset was stratified into some clusters by the frequency of drugs, ADRs, and drug-event combinations (DECs) in sequence. K-means clustering was used to conduct stratification according to data distribution characteristics. The Information Component (IC) was adopted for signal detection in each cluster respectively. By extracting ADRs from drug product labeling, a reference database was introduced for performance evaluation based on Recall, Precision and F-measure. In addition, some DECs from the Adverse Drug Reactions Information Bulletin (ADRIB) issued by CFDA were collected for further reliability evaluation.

Results

With stratification, the study dataset was divided into 21 clusters, among which the frequency of DRUGs, ADRs or DECs followed the similar order of magnitude respectively. Recall increased by 34.95% from 29.93 to 40.39%, Precision reduced by 10.52% from 54.56 to 48.82%, while F-measure increased by 14.39% from 38.65 to 44.21%. According to ADRIB after 2011, 5 DECs related to Potassium Magnesium Aspartate, 61 DECs related to Levofloxacin Hydrochloride and 26 DECs related to Cefazolin were highlighted.

Conclusions

The proposed method is effectively and reliably for the minimization of data masking effect in signal detection. Considering the decrease of Precision, it is suggested to be a supplement rather than an alternative to non-stratification method.

Pharmacovigilance of sodium-glucose co-transporter-2 inhibitors: What a clinician should know on disproportionality analysis of spontaneous reporting systems

Sodium-glucose co-transporter-2 inhibitors (SGLT2-Is) have consistently demonstrated a clinically significant reduction of cardiovascular mortality. However, their safety in clinical practice is still incompletely characterized, and post-marketing monitoring is required considering the expected increase in clinical use. Different analyses of international spontaneous reporting systems, known as disproportionality analyses (DAs), have highlighted the occurrence of ketoacidosis, amputations, acute renal failure and skin toxicity. In this viewpoint, we critically appraise these pharmacovigilance data on SGLT2-Is, with the aim of supporting clinicians in proper interpretation of these studies, and discussing their risk-benefit profile. To this aim, we offer a broad perspective on basic technical aspects subtending DAs of spontaneous reporting databases (describing peculiarities of the Food and Drug Administration Adverse Event Reporting System), their common and evolving uses, key pitfalls in presenting study results (in terms of "risk" or "association") and relevant strategies to account for major confounders. This will also facilitate reviewers and editors in proper evaluation of DAs, and prompt pharmacovigilance experts in converging towards a set of minimum requirements in standardization of design, performance and reporting of DAs. A consensus on quality assessment of DAs will finally establish their transferability to clinical practice. It is anticipated that DAs cannot be used per se as a standalone approach to assess a drug-related risk and cannot replace clinical judgment in the individual patient.

False-positive results in pharmacoepidemiology and pharmacovigilance

False-positive constitute an important issue in scientific research. In the domain of drug evaluation, it affects all phases of drug development and assessment, from the very early preclinical studies to the late post-marketing evaluations. The core concern associated with this false-positive is the lack of replicability of the results. Aside from fraud or misconducts, false-positive is often envisioned from the statistical angle, which considers them as a price to pay for type I error in statistical testing, and its inflation in the context of multiple testing. If envisioning this problematic in the context of pharmacoepidemiology and pharmacovigilance however, that both evaluate drugs in an observational settings, information brought by statistical testing and the significance of such should only be considered as additional to the estimates provided and their confidence interval, in a context where differences have to be a clinically meaningful upon everything, and the results appear robust to the biases likely to have affected the studies. In the following article, we consequently illustrate these biases and their consequences in generating false-positive results, through studies and associations between drug use and health outcomes that have been widely disputed.

A mathematical framework to quantify the masking effect associated with the confidence intervals of measures of disproportionality

BACKGROUND

The lower bound of the 95% confidence interval of measures of disproportionality (Lower95CI) is widely used in signal detection. Masking is a statistical issue by which true signals of disproportionate reporting are hidden by the presence of other medicines. The primary objective of our study is to develop and validate a mathematical framework for assessing the masking effect of Lower95CI.

METHODS

We have developed our new algorithm based on the masking ratio (MR) developed for the measures of disproportionality. A MR for the Lower95CI (MRCI) is proposed. A simulation study to validate this algorithm was also conducted.

RESULTS

We have established the existence of a very close mathematical relation between MR and MRCI. For a given drug-event pair, the same product will be responsible for the highest masking effect with the measure of disproportionality and its Lower95CI. The extent of masking is likely to be very similar across the two methods. An important proportion of identical drug-event associations affected by the presence of an important masking effect is revealed by the unmasking exercise, whether the proportional reporting ratio (PRR) or its confidence interval are used.

CONCLUSION

The detection of the masking effect of Lower95CI can be automated. The real benefits of this unmasking in terms of new true-positive signals (rate of true-positive/false-positive) or time gained by the revealing of signals using this method have not been fully assessed. These benefits should be demonstrated in the context of prospective studies.