Online causal inference with application to near real-time post-market vaccine safety surveillance

Streaming data routinely generated by social networks, mobile or web applications, e-commerce, and electronic health records present new opportunities to monitor the impact of an intervention on an outcome via causal inference methods. However, most existing causal inference methods have been focused on and applied to static data, that is, a fixed data set in which observations are pooled and stored before performing statistical analysis. There is thus a pressing need to turn static causal inference into online causal learning to support near real-time monitoring of treatment effects. In this paper, we present a framework for online estimation and inference of treatment effects that can incorporate new information as it becomes available without revisiting prior observations. We show that, under mild regularity conditions, the proposed online estimator is asymptotically equivalent to the offline oracle estimator obtained by pooling all data. Our proposal is motivated by the need for near real-time vaccine effectiveness and safety monitoring, and our proposed method is applied to a case study on COVID-19 vaccine safety surveillance.

A seasonality-adjusted sequential test for vaccine safety surveillance

Post-market active safety monitoring is important for the timely capture of safety signals associated with exposure to a new vaccine or drug. The group sequential analysis is a common method employed in safety surveillance. Specifically, it compares the post-vaccination incidence of adverse event (AE) in a vaccinated population with a pre-specified reference level by sequentially conducting hypothesis testing during the surveillance. When the number of AEs is "too high", a safety signal is identified. If the null hypothesis is never rejected, the vaccine is considered safe. Such an approach does not account for either the variation in determining the reference risk from a control population or the seasonality effect. Furthermore, not rejecting the null could be due to a lack of power and cannot always be interpreted as proof of safety. In this paper, we proposed a new group sequential test procedure fully accounting for both seasonality and variation from the historical controls. More importantly, we proposed to construct a confidence interval for the relative AE risk between the exposed and control groups at the end of the study, which can be used to quantify the safety of the vaccine. The proposed method is illustrated via real-data examples on anaphylaxis and examined by extensive simulation studies.

An application of the Causal Roadmap in two safety monitoring case studies: Causal inference and outcome prediction using electronic health record data

Background

Real-world data, such as administrative claims and electronic health records, are increasingly used for safety monitoring and to help guide regulatory decision-making. In these settings, it is important to document analytic decisions transparently and objectively to assess and ensure that analyses meet their intended goals.

Methods

The Causal Roadmap is an established framework that can guide and document analytic decisions through each step of the analytic pipeline, which will help investigators generate high-quality real-world evidence.

Results

In this paper, we illustrate the utility of the Causal Roadmap using two case studies previously led by workgroups sponsored by the Sentinel Initiative - a program for actively monitoring the safety of regulated medical products. Each case example focuses on different aspects of the analytic pipeline for drug safety monitoring. The first case study shows how the Causal Roadmap encourages transparency, reproducibility, and objective decision-making for causal analyses. The second case study highlights how this framework can guide analytic decisions beyond inference on causal parameters, improving outcome ascertainment in clinical phenotyping.

Conclusion

These examples provide a structured framework for implementing the Causal Roadmap in safety surveillance and guide transparent, reproducible, and objective analysis.

Enhanced Adenovirus Vaccine Safety Surveillance in Military Setting, United States

The US Coast Guard Academy began adenovirus vaccination of incoming cadets in 2022. Of 294 vaccine recipients, 15%-20% had mild respiratory or systemic symptoms within 10 days postvaccination but no serious adverse events after 90 days. Our findings support the continued use of adenovirus vaccines in congregate military settings.

Safety Surveillance of Mass Praziquantel and Albendazole Co-Administration in School Children from Southern Ethiopia: An Active Cohort Event Monitoring

Preventive chemotherapy (PC) with praziquantel and albendazole co-administration to all at-risk populations is the global intervention strategy to eliminate schistosomiasis and soil-transmitted helminth (STH) from being public health problems. Due to weak pharmacovigilance systems, safety monitoring during a mass drug administration (MDA) is lacking, especially in sub-Saharan Africa. We conducted large-scale active safety surveillance to identify the incidence, types, severity, and associated risk factors of adverse events (AEs) following praziquantel and albendazole MDA in 5848 school children (5−15 years old). Before MDA, 1484 (25.4%) children were prescreened for S. mansoni and STH infections, of whom 71.8% were infected with at least one parasite; 34.5% (512/1484) had S. mansoni and 853 (57.5%) had an STH infection. After collecting the baseline socio-demographic, clinical, and medical data, including any pre-existing clinical symptoms, participants received single dose praziquantel and albendazole MDA. Treatment-associated AEs were actively monitored on days 1 and 7 of the MDA. The events reported before and after the MDA were cross-checked and verified to identify MDA-associated AEs. The cumulative incidence of experiencing at least one type of MDA-associated AE was 13.3% (95% CI = 12.5−14.2%); 85.5%, 12.4%, and 1.8% of reported AEs were mild, moderate, and severe, respectively. The proportion of experiencing one, two, or ≥ three types of AEs was 57.7%, 34.1%, and 8.2%, respectively. The cumulative incidence of AEs in S. mansoni- and (17.0%) and STH (14.1%)-infected children was significantly higher (p < 0.001, χ2 = 15.0) than in non-infected children (8.4%). Headache, abdominal pain, vomiting, dizziness, and nausea were the most common AEs. Being female, older age, having S. mansoni or STH infection were significant predictors of MDA-associated AEs. In summary, praziquantel and albendazole co-administration is generally safe and tolerable. MDA-associated AEs are mostly mild-to-moderately severe and transient. The finding of few severe AEs and significantly high rates of AEs in helminth-infected children underscores the need to integrate pharmacovigilance in MDA programs, especially in high schistosomiasis and STH endemic areas.

Enhanced passive safety surveillance of a trivalent and a quadrivalent influenza vaccine in Denmark and Finland during the 2018/2019 season

The European Medicines Agency requires Enhanced Passive Safety Surveillance (EPSS) for all seasonal influenza vaccines. Here, we report the EPSS results for the trivalent inactivated influenza vaccine (IIV3; Vaxigrip®) and the quadrivalent inactivated influenza vaccine (IIV4; VaxigripTetra^TM) during the 2018/19 influenza season in Denmark and Finland. The primary objective was to estimate the rates of suspected adverse reactions (ARs) occurring within 7 days following routine vaccination. Between October and November 2018, 1000 safety report cards (SRCs) for IIV3 were distributed in Denmark, and 996 SRCs for IIV4 were distributed in Finland. Participants were instructed to report any ARs by telephone or e-mail using the information provided on the SRC. All participants vaccinated with IIV3 were aged ≥18 years. Most participants vaccinated with IIV4 (95.5%) were aged 18 − 65 years, 2.2% were aged 6 months to 17 years, and 2.3% were aged >65 years. Fifty-five ARs were reported by 12 participants (1.2%) vaccinated with IIV3 and 162 ARs were reported by 53 participants (5.3%) vaccinated with IIV4. The most frequent ARs were vaccination site pain and fever for IIV3, and vaccination site pain, vaccination site inflammation, myalgia, and headache for IIV4. The 2018/19 AR rates for IIV3 were comparable to 2017/18 rates. The 2018/19 AR rates for IIV4 were higher than those in 2017/18 but were still lower than the expected AR rates listed in the IIV4 Summary of Product Characteristics. In conclusion, the 2018/19 EPSS showed no clinically significant change from the expected safety profiles of IIV3 and IIV4 vaccines.

Challenges Associated with the Safety Signal Detection Process for Medical Devices

Background

Previous safety issues involving medical devices have stressed the need for better safety signal detection. Various European Union (EU) national competent authorities have started to focus on strengthening the analysis of vigilance data. Consequently, article 90 of the new EU regulation states that the European Commission shall put in place systems and processes to actively monitor medical device safety signals.

Methods

A systematic literature review was conducted to synthesize the current state of knowledge and investigate the present tools used for medical device safety signal detection. An electronic literature search was performed in Embase, Medline, Cochrane, Web of science, and Google scholar from inception until January 2017. Articles that included terms related to medical devices and terms associated with safety were selected. A further selection was based on the abstract review. A full review of the remaining articles was conducted to decide on which articles finally to consider relevant for this review. Completeness was assessed based on the content of the articles.

Results

Our search resulted in a total of 20,819 articles, of which 24 met the inclusion criteria and were subject to data extraction and completeness scoring. A wide range of data sources, especially spontaneous reporting systems and registries, used for the detection and assessment of product problems and patient harms associated with the use of medical devices, were studied. Coding is remarkably heterogeneous, no agreement on the preferred methods for signal detection exists, and no gold standard for signal detection has been established thus far.

Conclusion

Data source harmonization, the development of gold standard signal detection methodologies and the standardization of coding dictionaries are amongst the recommendations to support the implementation of a new proactive approach to signal detection. The new safety surveillance system will be able to use real-world evidence to support regulatory decision-making across all jurisdictions.

Enhanced passive safety surveillance of three marketed influenza vaccines in the UK and the Republic of Ireland during the 2017/18 season

Safety surveillance is required for each season’s influenza vaccines to rapidly detect and evaluate potential new safety concerns before the peak period of immunization. Here we report the results of an enhanced passive safety surveillance for a trivalent split-virion inactivated influenza vaccine (IIV3; Vaxigrip®), an intradermal version of this vaccine (IIV3-ID; Intanza® 15 µg), and a recently licensed quadrivalent version (IIV4; VaxigripTetra^TM) during the 2017/18 influenza season in the UK and Republic of Ireland. The primary objective was to determine the rates of adverse reactions (ARs) occurring within 7 days following routine vaccination. Between September and November 2017, 979 safety report cards were distributed to vaccinees receiving IIV3-ID, 1005 to those receiving IIV3, and 957 to those receiving IIV4. At least one AR was reported by 28 participants (2.9%) vaccinated with IIV3-ID, 14 participants (1.4%) vaccinated with IIV3, and 20 participants (2.1%) vaccinated with IIV4. The most frequent ARs were injection-site reactions and headache. One participant vaccinated with IIV3-ID reported two suspected serious ARs (dyskinesia and a shock symptom), although these could not be confirmed as vaccine-related. Rates of ARs for IIV3 and IIV3-ID for 2017/18 did not differ from the 2016/17 rates. For IIV4, in its first season since licensure, AR frequencies were similar to those in the Summary of Product Characteristics. In conclusion, no change was found compared to the known or expected AR rates for IIV3, IIV3-ID, or IIV4 during the 2017/18 season.

Constructing the informatics and information technology foundations of a medical device evaluation system: a report from the FDA unique device identifier demonstration

Objective

The US Food and Drug Administration (FDA) has recognized the need to improve the tracking of medical device safety and performance, with implementation of Unique Device Identifiers (UDIs) in electronic health information as a key strategy. The FDA funded a demonstration by Mercy Health wherein prototype UDIs were incorporated into its electronic information systems. This report describes the demonstration's informatics architecture.

Methods

Prototype UDIs for coronary stents were created and implemented across a series of information systems, resulting in UDI-associated data flow from manufacture through point of use to long-term follow-up, with barcode scanning linking clinical data with UDI-associated device attributes. A reference database containing device attributes and the UDI Research and Surveillance Database (UDIR) containing the linked clinical and device information were created, enabling longitudinal assessment of device performance. The demonstration included many stakeholders: multiple Mercy departments, manufacturers, health system partners, the FDA, professional societies, the National Cardiovascular Data Registry, and information system vendors.

Results

The resulting system of systems is described in detail, including entities, functions, linkage between the UDIR and proprietary systems using UDIs as the index key, data flow, roles and responsibilities of actors, and the UDIR data model.

Conclusion

The demonstration provided proof of concept that UDIs can be incorporated into provider and enterprise electronic information systems and used as the index key to combine device and clinical data in a database useful for device evaluation. Keys to success and challenges to achieving this goal were identified. Fundamental informatics principles were central to accomplishing the system of systems model.

TOY SAFETY SURVEILLANCE FROM ONLINE REVIEWS

Toy-related injuries account for a significant number of childhood injuries and the prevention of these injuries remains a goal for regulatory agencies and manufacturers. Text-mining is an increasingly prevalent method for uncovering the significance of words using big data. This research sets out to determine the effectiveness of text-mining in uncovering potentially dangerous children's toys. We develop a danger word list, also known as a 'smoke word' list, from injury and recall text narratives. We then use the smoke word lists to score over one million Amazon reviews, with the top scores denoting potential safety concerns. We compare the smoke word list to conventional sentiment analysis techniques, in terms of both word overlap and effectiveness. We find that smoke word lists are highly distinct from conventional sentiment dictionaries and provide a statistically significant method for identifying safety concerns in children's toy reviews. Our findings indicate that text-mining is, in fact, an effective method for the surveillance of safety concerns in children's toys and could be a gateway to effective prevention of toy-product-related injuries.

Post-licensure safety surveillance study of routine use of tetanus toxoid, reduced diphtheria toxoid and 5-component acellular pertussis vaccine

An observational post-licensure (Phase IV) retrospective large-database safety study was conducted at Kaiser Permanente, a US integrated medical care organization, to assess the safety of Tetanus Toxoid, Reduced Diphtheria Toxoid and 5-Component Acellular Pertussis Vaccine (Tdap5) administered as part of routine healthcare among adolescents and adults. We evaluated incidence rates of various clinical events resulting in outpatient clinic, emergency department (ED), and hospital visits during various time intervals (windows) following Tdap5 vaccination using 2 pharmacoepidemiological methods (risk interval and historic cohort) and several screening thresholds. Plausible outcomes of interest with elevated incidence rate ratios (IRRs) were further evaluated by reviewing individual patient records to confirm the diagnosis, timing (temporal relationship), alternative etiology, and other health record details to discern possible relatedness of the health events to vaccination. Overall, 124,139 people received Tdap5 vaccine from September 2005 through mid-October 2006, and 203,154 in the comparison cohort received a tetanus and diphtheria toxoid adsorbed vaccine (and no live virus vaccine) during the year prior to initiation of this study. In the outpatient, ED and hospital databases, respectively, we identified 11/26, 179/700 and 187/700 unique health outcomes with IRRs significantly >1.0. Among the same unique health outcomes in the outpatient, ED, and hospital databases, 9, 146, and 385, respectively, had IRRs significantly <1.0. Further scrutiny of the outcomes with elevated IRRs did not reveal unexpected signals of adverse outcomes related to vaccination. In conclusion, Tdap5 vaccine was found to be safe among this large population of adolescents and adults.

Safety surveillance in haemophilia and allied disorders

The introduction of clotting factor concentrates has transformed the lives of persons with inherited bleeding disorders. With the use of prophylactic treatment, it is now possible to prevent bleeding in these individuals. The early concentrates were contaminated with the HIV and hepatitis C viruses (HCV) and resulted in major morbidity and mortality in the recipients. Current products are much safer, especially in terms of infectious agents, but other adverse events such as alloantibodies (inhibitors), allergic reactions and thrombotic risks remain of concern. Approximately 30% of previously untreated patients with severe haemophilia A develop inhibitors, making this the most important issue in haemophilia care today. Recently, it was suggested that one of the most commonly used concentrates was associated with a higher inhibitor risk, but this was not supported by the evidence from all studies. Good safety surveillance systems are essential for all diseases and products but are particularly so in the group of individuals with inherited bleeding disorders treated with clotting factor concentrates who have suffered disproportionately from the adverse effects of their treatment. National and multinational systems are now in place to allow reporting of adverse events in patients with inherited bleeding disorders. All clinicians treating individuals with inherited bleeding disorders should prospectively report adverse events to treatment even if they are believed to be common and well recognized.

The Pioneering Role of the Vaccine Safety Datalink Project (VSD) to Advance Collaborative Research and Distributed Data Networks

Introduction:

Large-scale distributed data networks consisting of diverse stakeholders including providers, patients, and payers are changing health research in terms of methods, speed and efficiency. The Vaccine Safety Datalink (VSD) set the stage for expanded involvement of health plans in collaborative research.

Expanding Surveillance Capacity and Progress Toward a Learning Health System:

From an initial collaboration of four integrated health systems with fewer than 10 million covered lives to 16 diverse health plans with nearly 100 million lives now in the FDA Sentinel, the expanded engagement of health plan researchers has been essential to increase the value and impact of these efforts. The collaborative structure of the VSD established a pathway toward research efforts that successfully engage all stakeholders in a cohesive rather than competitive manner. The scientific expertise and methodology developed through the VSD such as rapid cycle analysis (RCA) to conduct near real-time safety surveillance allowed for the development of the expanded surveillance systems that now exist.

Building on Success and Lessons Learned:

These networks have learned from and built on the knowledge base and infrastructure created by the VSD investigators. This shared technical knowledge and experience expedited the development of systems like the FDA’s Mini-Sentinel and the Patient Centered Outcomes Research Institute (PCORI)’s PCORnet

Conclusion:

This narrative reviews the evolution of the VSD, its contribution to other collaborative research networks, longer-term sustainability of this type of distributed research, and how knowledge gained from the earlier efforts can contribute to a continually learning health system.

Active and passive surveillance of enoxaparin generics: a case study relevant to biosimilars

OBJECTIVE

This retrospective analysis assessed the capability of active and passive safety surveillance systems to track product-specific safety events in the USA for branded and generic enoxaparin, a complex injectable subject to immune-related and other adverse events (AEs).

METHODS

Analysis of heparin-induced thrombocytopenia (HIT) incidence was performed on benefit claims for commercial and Medicare supplemental-insured individuals newly treated with enoxaparin under pharmacy benefit (1 January 2009 - 30 June 2012). Additionally, spontaneous reports from the FDA AE Reporting System were reviewed to identify incidence and attribution of enoxaparin-related reports to specific manufacturers.

RESULTS

Specific, dispensed products were identifiable from National Drug Codes only in pharmacy-benefit databases, permitting sensitive comparison of HIT incidence in nearly a third of patients treated with brand or generic enoxaparin. After originator medicine's loss of exclusivity, only 5% of spontaneous reports were processed by generic manufacturers; reports attributable to specific generics were approximately ninefold lower than expected based on market share.

CONCLUSIONS

Claims data were useful for active surveillance of enoxaparin generics dispensed under pharmacy benefits but not for products administered under medical benefits. These findings suggest that the current spontaneous reporting system will not distinguish product-specific safety signals for products distributed by multiple manufacturers, including biosimilars.

Electronic clinical laboratory test results data tables: lessons from Mini-Sentinel

PURPOSE

Developing electronic clinical data into a common data model posed substantial challenges unique from those encountered with administrative data. We present here the design, implementation, and use of the Mini-Sentinel Distributed Database laboratory results table (LRT).

METHODS

We developed the LRT and guided Mini-Sentinel data partners (DPs) in populating it from their source data. Data sources included electronic health records and internal and contracted clinical laboratory systems databases. We employed the Logical Observation Identifiers, Names, and Codes (LOINC®) results reporting standards. We evaluated transformed results data using data checks and an iterative, ongoing characterization and harmonization process.

RESULTS

Key LRT variables included test name, subcategory, specimen source, LOINC, patient location, specimen date and time, result unit, and unique person identifier. Selected blood and urine chemistry, hematology, coagulation, and influenza tests were included. Twelve DPs with outpatient test results participated; four also contributed inpatient test results. As of September 2013, the LRT included 385,516,239 laboratory test results; data are refreshed at least quarterly. LOINC availability and use varied across DP. Multiple data quality and content issues were identified and addressed.

CONCLUSION

Developing the LRT brought together disparate data sources with no common coding structure. Clinical laboratory test results obtained during routine healthcare delivery are neither uniformly coded nor documented in a standardized manner. Applying a systematic approach with data harmonization efforts and ongoing oversight and management is necessary for a clinical laboratory results data table to remain valid and useful.

Likelihood ratio based tests for longitudinal drug safety data

This article presents longitudinal likelihood ratio test (LongLRT) methods for large databases with exposure information. These methods are applied to a pooled large longitudinal clinical trial dataset for drugs treating osteoporosis with concomitant use of proton pump inhibitors (PPIs). When the interest is in the evaluation of a signal of an adverse event for a particular drug compared with placebo or a comparator, the special case of the LongLRT, referred to as sequential LRT (SeqLRT), is also presented. The results show that there is some possible evidence of concomitant use of PPIs leading to more adverse events associated with osteoporosis. The performance of the proposed LongLRT and SeqLRT methods is evaluated using simulated datasets and shown to be good in terms of (conditional) power and control of type I error over time. The proposed methods can also be applied to large observational databases with exposure information under the US Food and Drug Administration Sentinel Initiative for active surveillance. Published 2014. This article is a US Government work and is in the public domain in the USA.

Automating case definitions using literature-based reasoning

BACKGROUND

Establishing a Case Definition (CDef) is a first step in many epidemiological, clinical, surveillance, and research activities. The application of CDefs still relies on manual steps and this is a major source of inefficiency in surveillance and research.

OBJECTIVE

Describe the need and propose an approach for automating the useful representation of CDefs for medical conditions.

METHODS

We translated the existing Brighton Collaboration CDef for anaphylaxis by mostly relying on the identification of synonyms for the criteria of the CDef using the NLM MetaMap tool. We also generated a CDef for the same condition using all the related PubMed abstracts, processing them with a text mining tool, and further treating the synonyms with the above strategy. The co-occurrence of the anaphylaxis and any other medical term within the same sentence of the abstracts supported the construction of a large semantic network. The 'islands' algorithm reduced the network and revealed its densest region including the nodes that were used to represent the key criteria of the CDef. We evaluated the ability of the "translated" and the "generated" CDef to classify a set of 6034 H1N1 reports for anaphylaxis using two similarity approaches and comparing them with our previous semi-automated classification approach.

RESULTS

Overall classification performance across approaches to producing CDefs was similar, with the generated CDef and vector space model with cosine similarity having the highest accuracy (0.825 ± 0.003) and the semi-automated approach and vector space model with cosine similarity having the highest recall (0.809 ± 0.042). Precision was low for all approaches.

CONCLUSION

The useful representation of CDefs is a complicated task but potentially offers substantial gains in efficiency to support safety and clinical surveillance.

Duloxetine and pregnancy outcomes: safety surveillance findings

BACKGROUND

Duloxetine hydrochloride is approved for the treatment or management of major depressive disorder, generalized anxiety disorder, diabetic peripheral neuropathic pain, chronic musculoskeletal pain, and fibromyalgia in the United States. These conditions affect millions of women, including those of childbearing potential. In pregnancy, pharmacological treatment is justified only if the potential benefits outweigh potential risks to mother and fetus, neonate or infant. There are no adequate and well-controlled studies in pregnant women treated with duloxetine. Post-marketing surveillance is an important tool for the assessment of drug safety in pregnancy in a naturalistic setting.

OBJECTIVE

Using safety surveillance and spontaneous adverse events reporting databases, to provide pregnancy outcomes statistics as they relate to duloxetine exposure.

STUDY DESIGN AND SETTING

This was an analysis of pregnancy outcome data captured in Lilly Safety System (LSS) (a safety database for the collection, storage, and reporting of adverse events involving Lilly Products), through October 31 2011 and the FDA Adverse Events Reporting System (AERS) database through September 30 2011. Both databases provided spontaneous reporting data from the time of first duloxetine marketing authorization in 2004; in addition, the LSS Database includes serious adverse event and pregnancy data from clinical trials since the creation of the database in 1983.

PATIENTS

Patients who had received duloxetine during pregnancy and reported pregnancy outcomes.

MAIN OUTCOME MEASURES

Normal and abnormal pregnancy outcomes. Abnormal outcomes comprised spontaneous abortion, premature/post-term birth, congenital anomaly, perinatal/post-perinatal complication, still birth, and ectopic pregnancy. Descriptive statistics are provided for LSS data. A disproportionality analysis was performed using the Empirical Bayes Geometric Mean (EBGM) for the AERS data. The lower bound of the 90% confidence interval of EBGM (EB05) ≥1 was used as the threshold to determine disproportionality.

RESULTS

In the LSS analysis, 400 pregnancy cases with a known pregnancy outcome were identified. Of the 233 prospectively reported cases, 170 (73%) were spontaneous reports; the remainder were reported from clinical trials (58 [25%]) or post-marketing studies (5 [2%]). In most of these cases (74%), patients received duloxetine for the treatment of depression. Pregnancy outcomes were normal in 143 cases, and abnormal in 90 cases. Abnormal pregnancy outcomes were mainly spontaneous abortions (n=41), post/perinatal conditions (n=25) or premature births (n=19). In patients with abnormal pregnancy outcomes, relevant concomitant medication use and relevant medical history were more frequently reported, compared to those with normal pregnancy outcomes (p<0.05). For the AERS database analysis, EB05 was less than one for all clusters of abnormal pregnancy outcomes; there was no disproportionality of reporting adverse pregnancy outcomes for patients treated with duloxetine versus all other drugs or selected antidepressants.

CONCLUSION

While limitations of these data are recognized, the information available to date from these two data sources suggest that the frequency of abnormal outcomes reported in duloxetine pregnancy cases is generally consistent with the historic control rates in the general population.

An evaluation of a distributed medical device safety surveillance system: the DELTA network study

BACKGROUND

Post-market medical product safety surveillance is a complex task requiring standardized data collection, prompt adverse event reporting mechanisms and appropriate methodologies to identify low frequency safety threats and risk communication.

PURPOSE

To review the design of the DELTA (Data Extraction and Longitudinal Trend Analysis) network study of the medical device safety surveillance.

METHODS

This is a multicenter prospective observational study designed to evaluate the safety of new cardiovascular devices used during percutaneous coronary intervention (PCI) performed through continuous analysis of the routinely collected American College of Cardiology-National Cardiovascular Data Registry (ACC-NCDR) data elements. The primary endpoint of the study is detection of adverse event rates specific to several classes of new medical devices, including drug eluting coronary stents, embolic protection devices, and vascular closure devices in patients undergoing PCI. Secondary endpoints include the time-savings between the DELTA network detection of a true device safety alert and the time taken to detect the same outcome using conventional retrospective data analysis, overall sensitivity, specificity, positive predictive value and negative predictive value of the DELTA network surveillance system.

RESULTS

The details of the study are described including system design, eligibility criteria, methods and components of data collection, data security and statistical methods. In addition, the methods of adjudication and verification following an adverse event alert, overall study outcomes, end points, limitations and potential advantages are discussed.

CONCLUSION

This report describes the first multicenter prospective study of a computerized safety surveillance system to monitor and evaluate the safety of new cardiovascular devices.