Mini-Sentinel's systematic reviews of validated methods for identifying health outcomes using administrative data: summary of findings and suggestions for future research

Validation of diagnosis codes in healthcare databases in Taiwan, a literature review

PURPOSE

To compile validation findings of diagnosis codes and related algorithms for health outcomes of interest from National Health Insurance (NHI) or electronic medical records in Taiwan.

METHODS

We carried out a literature review of English articles in PubMed® and Embase from 2000 through July 2022 with appropriate search terms. Potentially relevant articles were identified through review of article titles and abstracts, full text search of methodology terms "validation", "positive predictive value", and "algorithm" in Subjects & Methods (or Methods) and Results sections of articles, followed by full text review of potentially eligible articles.

RESULTS

We identified 50 published reports with validation findings of diagnosis codes and related algorithms for a wide range of health outcomes of interest in Taiwan, including cardiovascular diseases, stroke, renal impairment, malignancy, diabetes, mental health diseases, respiratory diseases, viral (B and C) hepatitis, and tuberculosis. Most of the reported PPVs were in the 80% ~ 99% range. Assessment of algorithms based on ICD-10 systems were reported in 8 articles, all published in 2020 or later.

CONCLUSIONS

Investigators have published validation reports that may serve as empirical evidence to evaluate the utility of secondary health data environment in Taiwan for research and regulatory purpose.

Development and validation of case-finding algorithms to identify acute pancreatitis in the Veterans Health Administration

PURPOSE

Acute pancreatitis (AP) is a frequently encountered adverse drug reaction. However, the validity of diagnostic codes for AP is unknown. We aimed to determine the positive predictive value (PPV) of a diagnostic code-based algorithm for identifying patients with AP within the US Veterans Health Administration and evaluate the value of adding readily available structured laboratory information.

METHODS

We identified patients with possible AP events first based on the presence of a single hospital discharge ICD-9 or ICD-10 diagnosis of AP (Algorithm 1). We then expanded Algorithm 1 by including relevant laboratory test results (Algorithm 2). Specifically, we considered amylase or lipase serum values obtained between 2 days before admission and the end of the hospitalization. Medical records of a random sample of patients identified by the respective algorithms were reviewed by two separate gastroenterologists to adjudicate AP events. The PPV (95% confidence interval [CI]) for the algorithms were calculated.

RESULTS

Algorithm 2, consisting of one ICD-9 or ICD-10 hospital discharge diagnosis of AP and the addition of lipase serum value ≥200 U/L, had a PPV 89.1% (95% CI 83.0%-95.2%), improving from the PPV of algorithm 1 (57.9% [95% CI 46.8-69.0]).

CONCLUSIONS

An algorithm consisting of an ICD-9 or ICD-10 diagnosis of AP with a lipase value ≥200 U/L achieved high PPV. This simple algorithm can be readily implemented in any electronic health records (EHR) systems and could be useful for future pharmacoepidemiologic studies on AP.

Use of diagnostic likelihood ratio of outcome to evaluate misclassification bias in the planning of database studies

Background

The diagnostic likelihood ratio (DLR) and its utility are well-known in the field of medical diagnostic testing. However, its use has been limited in the context of an outcome validation study. We considered that wider recognition of the utility of DLR would enhance the practices surrounding database studies. This is particularly timely and important since the use of healthcare-related databases for pharmacoepidemiology research has greatly expanded in recent years. In this paper, we aimed to advance the use of DLR, focusing on the planning of a new database study.

Methods

Theoretical frameworks were developed for an outcome validation study and a comparative cohort database study; these two were combined to form the overall relationship. Graphical presentations based on these relationships were used to examine the implications of validation study results on the planning of a database study. Additionally, novel uses of graphical presentations were explored using some examples.

Results

Positive DLR was identified as a pivotal parameter that connects the expected positive-predictive value (PPV) with the disease prevalence in the planned database study, where the positive DLR is equal to sensitivity/(1-specificity). Moreover, positive DLR emerged as a pivotal parameter that links the expected risk ratio with the disease risk of the control group in the planned database study. In one example, graphical presentations based on these relationships provided a transparent and informative summary of multiple validation study results. In another example, the potential use of a graphical presentation was demonstrated in selecting a range of positive DLR values that best represented the relevant validation studies.

Conclusions

Inclusion of the DLR in the results section of a validation study would benefit potential users of the study results. Furthermore, investigators planning a database study can utilize the DLR to their benefit. Wider recognition of the full utility of the DLR in the context of a validation study would contribute meaningfully to the promotion of good practice in planning, conducting, analyzing, and interpreting database studies.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-022-01757-1.

Development of an indicator of smoking status for people with multiple sclerosis in administrative data

Background

Administrative data lack health behavior information.

Methods

We developed an administrative case definition for past or current (‘ever smoking’) in 1320 individuals with MS from Manitoba, Canada. Candidate indicators for ‘ever smoked’ included smoking cessation medications, and diagnosis codes for tobacco use and chronic obstructive pulmonary disease, using variable lookback periods.

Results

When compared to self-reported smoking status, the case definition incorporating all indicators over a lifetime lookback period had a sensitivity of 31.98%, and positive predictive value of 78.26%.

Conclusion

This smoking status definition could only partially control for confounding due to smoking because of the low sensitivity.

Identifying cases of spinal cord injury or disease in a primary care electronic medical record database

OBJECTIVE

To identify cases of spinal cord injury or disease (SCI/D) in an Ontario database of primary care electronic medical records (EMR).

DESIGN

A reference standard of cases of chronic SCI/D was established via manual review of EMRs; this reference standard was used to evaluate potential case identification algorithms for use in the same database.

SETTING

Electronic Medical Records Primary Care (EMRPC) Database, Ontario, Canada.

PARTICIPANTS

A sample of 48,000 adult patients was randomly selected from 213,887 eligible patients in the EMRPC database.

INTERVENTIONS

N/A.

MAIN OUTCOME MEASURE(S)

Candidate algorithms were evaluated using sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and F-score.

RESULTS

126 cases of chronic SCI/D were identified, forming the reference standard. Of these, 57 were cases of traumatic spinal cord injury (TSCI), and 67 were cases of non-traumatic spinal cord injury (NTSCI). The optimal case identification algorithm used free-text keyword searches and a physician billing code, and had 70.6% sensitivity (61.9-78.4), 98.5% specificity (97.3-99.3), 89.9% PPV (82.2-95.0), 94.7% NPV (92.8-96.3), and an F-score of 79.1.

CONCLUSIONS

Identifying cases of chronic SCI/D from a database of primary care EMRs using free-text entries is feasible, relying on a comprehensive case definition. Identifying a cohort of patients with SCI/D will allow for future study of the epidemiology and health service utilization of these patients.

Development and Internal Validation of a Disability Algorithm for Multiple Sclerosis in Administrative Data

Objective: We developed and internally validated an algorithm for disability status in multiple sclerosis (MS) using administrative data.

Methods: We linked administrative data from Manitoba, Canada to a clinical dataset with Expanded Disability Status Scale (EDSS) scores for people with MS. Clinical EDSS scores constituted the reference standard. We created candidate indicators using the administrative data. These included indicators based on use of particular health care services (home care, long-term care, rehabilitation admission), use of specific diagnostic codes (such as spasticity, quadriplegia), and codes based on use of Employment and Income Insurance. We developed algorithms to predict severe disability (EDSS ≥6.0), and to predict disability as a continuous measure. We manually developed algorithms, and also employed regression approaches. After we selected our preferred algorithms for disability, we tested their association with health care use due to any cause and infection after potential confounders.

Results: We linked clinical and administrative data for 1,767 persons with MS, most of whom were women living in urban areas. All individual indicators tested had specificities >90% for severe disability, and all but a diagnosis of visual disturbance had positive predictive values (PPV) >70%. The combination of home care or long-term care use or rehabilitation admission had a sensitivity of 61.9%, specificity of 90.76%, PPV of 70.06% and negative predictive of 87.21%. Based on regression modeling, the best-performing algorithm for predicting the EDSS as a continuous variable included age, home care use, long-term care admission, admission for rehabilitation, visual disturbance, other paralytic syndromes and spasticity. The mean difference between observed and predicted values of the EDSS was −0.0644 (95%CI −0.1632, 0.0304). Greater disability, whether measured using the clinical EDSS or either of the administrative data algorithms was similarly associated with increased hospitalization rates due to any cause and infection.

Conclusion: We developed and internally validated an algorithm for disability in MS using administrative data that may support population-based studies that wish to account for disability status but do not have access to clinical data sources with this information. We also found that more severe disability is associated with increased health care use, including due to infection.

Physician adjudication of angioedema diagnosis codes in a population of patients with heart failure prescribed angiotensin-converting enzyme inhibitor therapy

PURPOSE

Our objective was to calculate the positive predictive value (PPV) of the ICD-9 diagnosis code for angioedema when physicians adjudicate the events by electronic health record review. Our secondary objective was to evaluate the inter-rater reliability of physician adjudication.

METHODS

Patients from the Cardiovascular Research Network previously diagnosed with heart failure who were started on angiotensin-converting enzyme inhibitors (ACEI) during the study period (July 1, 2006 through September 30, 2015) were included. A team of two physicians per participating site adjudicated possible events using electronic health records for all patients coded for angioedema for a total of five sites. The PPV was calculated as the number of physician-adjudicated cases divided by all cases with the diagnosis code of angioedema (ICD-9-CM code 995.1) meeting the inclusion criteria. The inter-rater reliability of physician teams, or kappa statistic, was also calculated.

RESULTS

There were 38 061 adults with heart failure initiating ACEI in the study (21 489 patient-years). Of 114 coded events that were adjudicated by physicians, 98 angioedema events were confirmed for a PPV of 86% (95% CI: 80%, 92%). The kappa statistic based on physician inter-rater reliability was 0.65 (95% CI: 0.47, 0.82).

CONCLUSIONS

ICD-9 diagnosis code of 995.1 (angioneurotic edema, not elsewhere classified) is highly predictive of angioedema in adults with heart failure exposed to ACEI.

Development and Validation of a Model to Identify Alzheimer's Disease and Related Syndromes in Administrative Data

BACKGROUND

Administrative data are used in the field of Alzheimer's Disease and Related Syndromes (ADRS), however their performance to identify ADRS is unknown.

OBJECTIVE

i) To develop and validate a model to identify ADRS prevalent cases in French administrative data (SNDS), ii) to identify factors associated with false negatives.

METHODS

Retrospective cohort of subjects ≥ 65 years, living in South-Western France, who attended a memory clinic between April and December 2013. Gold standard for ADRS diagnosis was the memory clinic specialized diagnosis. Memory clinics' data were matched to administrative data (drug reimbursements, diagnoses during hospitalizations, registration with costly chronic conditions). Prediction models were developed for 1-year and 3-year periods of administrative data using multivariable logistic regression models. Overall model performance, discrimination, and calibration were estimated and corrected for optimism by resampling. Youden index was used to define ADRS positivity and to estimate sensitivity, specificity, positive predictive and negative probabilities. Factors associated with false negatives were identified using multivariable logistic regressions.

RESULTS

3360 subjects were studied, 52% diagnosed with ADRS by memory clinics. Prediction model based on age, all-cause hospitalization, registration with ADRS as a chronic condition, number of anti-dementia drugs, mention of ADRS during hospitalizations had good discriminative performance (c-statistic: 0.814, sensitivity: 76.0%, specificity: 74.2% for 2013 data). 419 false negatives (24.0%) were younger, had more often ADRS types other than Alzheimer's disease, moderate forms of ADRS, recent diagnosis, and suffered from other comorbidities than true positives.

CONCLUSION

Administrative data presented acceptable performance for detecting ADRS. External validation studies should be encouraged.

Development and validation of a predictive model algorithm to identify anaphylaxis in adults with type 2 diabetes in U.S. administrative claims data

PURPOSE

To use medical record adjudication and predictive modeling methods to develop and validate an algorithm to identify anaphylaxis among adults with type 2 diabetes (T2D) in administrative claims.

METHODS

A conventional screening algorithm that prioritized sensitivity to identify potential anaphylaxis cases was developed and consisted of diagnosis codes for anaphylaxis or relevant signs and symptoms. This algorithm was applied to adults with T2D in the HealthCore Integrated Research Database (HIRD) from 2016 to 2018. Clinical experts adjudicated anaphylaxis case status from redacted medical records. We used confirmed case status as an outcome for predictive models developed using lasso regression with 10-fold cross-validation to identify predictors and estimate the probability of confirmed anaphylaxis.

RESULTS

Clinical adjudicators reviewed medical records with sufficient information from 272 adults identified by the anaphylaxis screening algorithm, which had an estimated Positive Predictive Value (PPV) of 65% (95% confidence interval [CI]: 60%-71%). The predictive model algorithm had a c-statistic of 0.95. The model's probability threshold of 0.60 excluded 89% (84/94) of false positives identified by the screening algorithm, with a PPV of 94% (95% CI: 91%-98%). The model excluded very few true positives (15 of 178), and identified 92% (95% CI: 87%-96%) of the cases selected by the screening algorithm.

CONCLUSIONS

Predictive modeling techniques yielded an accurate algorithm with high PPV and sensitivity for identifying anaphylaxis in administrative claims. This algorithm could be considered in future safety studies using similar claims data to reduce potential outcome misclassification.

Intra-database validation of case-identifying algorithms using reconstituted electronic health records from healthcare claims data

BACKGROUND

Diagnosis performances of case-identifying algorithms developed in healthcare database are usually assessed by comparing identified cases with an external data source. When this is not feasible, intra-database validation can present an appropriate alternative.

OBJECTIVES

To illustrate through two practical examples how to perform intra-database validations of case-identifying algorithms using reconstituted Electronic Health Records (rEHRs).

METHODS

Patients with 1) multiple sclerosis (MS) relapses and 2) metastatic castration-resistant prostate cancer (mCRPC) were identified in the French nationwide healthcare database (SNDS) using two case-identifying algorithms. A validation study was then conducted to estimate diagnostic performances of these algorithms through the calculation of their positive predictive value (PPV) and negative predictive value (NPV). To that end, anonymized rEHRs were generated based on the overall information captured in the SNDS over time (e.g. procedure, hospital stays, drug dispensing, medical visits) for a random selection of patients identified as cases or non-cases according to the predefined algorithms. For each disease, an independent validation committee reviewed the rEHRs of 100 cases and 100 non-cases in order to adjudicate on the status of the selected patients (true case/ true non-case), blinded with respect to the result of the corresponding algorithm.

RESULTS

Algorithm for relapses identification in MS showed a 95% PPV and 100% NPV. Algorithm for mCRPC identification showed a 97% PPV and 99% NPV.

CONCLUSION

The use of rEHRs to conduct an intra-database validation appears to be a valuable tool to estimate the performances of a case-identifying algorithm and assess its validity, in the absence of alternative.

Challenges Associated with the Safety Signal Detection Process for Medical Devices

Background

Previous safety issues involving medical devices have stressed the need for better safety signal detection. Various European Union (EU) national competent authorities have started to focus on strengthening the analysis of vigilance data. Consequently, article 90 of the new EU regulation states that the European Commission shall put in place systems and processes to actively monitor medical device safety signals.

Methods

A systematic literature review was conducted to synthesize the current state of knowledge and investigate the present tools used for medical device safety signal detection. An electronic literature search was performed in Embase, Medline, Cochrane, Web of science, and Google scholar from inception until January 2017. Articles that included terms related to medical devices and terms associated with safety were selected. A further selection was based on the abstract review. A full review of the remaining articles was conducted to decide on which articles finally to consider relevant for this review. Completeness was assessed based on the content of the articles.

Results

Our search resulted in a total of 20,819 articles, of which 24 met the inclusion criteria and were subject to data extraction and completeness scoring. A wide range of data sources, especially spontaneous reporting systems and registries, used for the detection and assessment of product problems and patient harms associated with the use of medical devices, were studied. Coding is remarkably heterogeneous, no agreement on the preferred methods for signal detection exists, and no gold standard for signal detection has been established thus far.

Conclusion

Data source harmonization, the development of gold standard signal detection methodologies and the standardization of coding dictionaries are amongst the recommendations to support the implementation of a new proactive approach to signal detection. The new safety surveillance system will be able to use real-world evidence to support regulatory decision-making across all jurisdictions.

A primer on quantitative bias analysis with positive predictive values in research using electronic health data

OBJECTIVE

In health informatics, there have been concerns with reuse of electronic health data for research, including potential bias from incorrect or incomplete outcome ascertainment. In this tutorial, we provide a concise review of predictive value-based quantitative bias analysis (QBA), which comprises epidemiologic methods that use estimates of data quality accuracy to quantify the bias caused by outcome misclassification.

TARGET AUDIENCE

Health informaticians and investigators reusing large, electronic health data sources for research.

SCOPE

When electronic health data are reused for research, validation of outcome case definitions is recommended, and positive predictive values (PPVs) are the most commonly reported measure. Typically, case definitions with high PPVs are considered to be appropriate for use in research. However, in some studies, even small amounts of misclassification can cause bias. In this tutorial, we introduce methods for quantifying this bias that use predictive values as inputs. Using epidemiologic principles and examples, we first describe how multiple factors influence misclassification bias, including outcome misclassification levels, outcome prevalence, and whether outcome misclassification levels are the same or different by exposure. We then review 2 predictive value-based QBA methods and why outcome PPVs should be stratified by exposure for bias assessment. Using simulations, we apply and evaluate the methods in hypothetical electronic health record-based immunization schedule safety studies. By providing an overview of predictive value-based QBA, we hope to bridge the disciplines of health informatics and epidemiology to inform how the impact of data quality issues can be quantified in research using electronic health data sources.

Towards The Automated, Empirical Filtering of Drug-Drug Interaction Alerts in Clinical Decision Support Systems: Historical Cohort Study of Vitamin K Antagonists

Background

Drug-drug interactions (DDIs) involving vitamin K antagonists (VKAs) constitute an important cause of in-hospital morbidity and mortality. However, the list of potential DDIs is long; the implementation of all these interactions in a clinical decision support system (CDSS) results in over-alerting and alert fatigue, limiting the benefits provided by the CDSS.

Objective

To estimate the probability of occurrence of international normalized ratio (INR) changes for each DDI rule, via the reuse of electronic health records.

Methods

An 8-year, exhaustive, population-based, historical cohort study including a French community hospital, a group of Danish community hospitals, and a Bulgarian hospital. The study database included 156,893 stays. After filtering against two criteria (at least one VKA administration and at least one INR laboratory result), the final analysis covered 4047 stays. Exposure to any of the 145 drugs known to interact with VKA was tracked and analyzed if at least 3 patients were concerned. The main outcomes are VKA potentiation (defined as an INR≥5) and VKA inhibition (defined as an INR≤1.5). Groups were compared using the Fisher exact test and logistic regression, and the results were expressed as an odds ratio (95% confidence limits).

Results

The drugs known to interact with VKAs either did not have a statistically significant association regarding the outcome (47 drug administrations and 14 discontinuations) or were associated with significant reduction in risk of its occurrence (odds ratio<1 for 18 administrations and 21 discontinuations).

Conclusions

The probabilities of outcomes obtained were not those expected on the basis of our current body of pharmacological knowledge. The results do not cast doubt on our current pharmacological knowledge per se but do challenge the commonly accepted idea whereby this knowledge alone should be used to define when a DDI alert should be displayed. Real-life probabilities should also be considered during the filtration of DDI alerts by CDSSs, as proposed in SPC-CDSS (statistically prioritized and contextualized CDSS). However, these probabilities may differ from one hospital to another and so should probably be calculated locally.

Assessment of Studies Evaluating Incremental Costs, Effectiveness, or Cost-Effectiveness of Systemic Therapies in Breast Cancer Based on Claims Data: A Systematic Review

OBJECTIVES

Large secondary databases, such as those containing insurance claims data, are increasingly being used to compare the effects and costs of treatments in routine clinical practice. Despite their appeal, however, caution must be exercised when using these data. In this study, we aimed to identify and assess the methodological quality of studies that used claims data to compare the effectiveness, costs, or cost-effectiveness of systemic therapies for breast cancer.

METHODS

We searched Embase, the Cochrane Library, Medline, Web of Science, and Google Scholar for English-language publications and assessed the methodological quality using the Good Research for Comparative Effectiveness principles. This study was registered with the International Prospective Register of Systematic Reviews (PROSPERO) under number CRD42018103992.

RESULTS

We identified 1251 articles, of which 106 met the inclusion criteria. Most studies were conducted in the United States (74%) and Taiwan (9%) and were based on claims data sets (35%) or claims data linked to cancer registries (58%). Furthermore, most included large samples (mean 17 130 patients) and elderly patients, and they covered various outcomes (eg, survival, adverse events, resource use, and costs). Key methodological shortcomings were the lack of information on relevant confounders, the risk of immortal time bias, and the lack of information on the validity of outcomes. Only a few studies performed sensitivity analyses.

CONCLUSIONS

Many comparative studies of cost, effectiveness, and cost-effectiveness have been published in recent decades based on claims data, and the number of publications has increased over time. Despite the availability of guidelines to improve quality, methodological issues persist and are often inappropriately addressed or reported.

Post-approval Safety Surveillance Study of Golimumab in the Treatment of Rheumatic Disease Using a United States Healthcare Claims Database

Background and Objective

Golimumab is a fully human anti-tumor necrosis factor monoclonal antibody approved for the treatment of rheumatoid arthritis (RA), psoriatic arthritis (PsA), and ankylosing spondylitis (AS). This study estimated rates of prespecified outcomes in patients with RA, PsA or AS initiating golimumab versus matched patients initiating non-biologic systemic (NBS) medications.

Methods

Patients enrolled in a US health plan with rheumatic disease who initiated a study medication were accrued between April 2009 and November 2014. Golimumab initiators were matched by propensity score to NBS initiators in a 1:4 ratio. Outcomes were identified through September 2015. As-treated, as-matched, and nested case–control (NCC) analyses were conducted in the matched cohorts. Sensitivity analyses evaluated the impact of residual confounding and nondifferential misclassification of exposure and outcomes.

Results

Risks of outcomes were similar between golimumab and NBS initiators. In the as-treated analysis, the rate ratio (RR) for depression was elevated during current golimumab use versus golimumab non-use in the NBS cohort [RR 1.45, 95% confidence interval (CI) 1.31–1.61]. This finding was not replicated in as-matched (RR 1.08, 95% CI 0.97–1.19) or NCC (odds ratio 1.01, 95% CI 0.78–1.31) analyses, which focused on incident cases. Sensitivity analyses suggest that depression was sensitive to misclassification, and the RR changed from greater than to less than one across a plausible range of specificity.

Conclusions

This study suggests that there is no association between exposure to golimumab and an increased risk of prespecified outcomes. Increased depression risk in the as-treated analysis was not replicated in other analyses and may be associated with residual imbalance in baseline history or severity of depression.

Electronic supplementary material

The online version of this article (10.1007/s40261-020-00959-7) contains supplementary material, which is available to authorized users.

Bridging the Gap Between RCTs and RWE Through Endpoint Selection

This commentary is authored by several industry real-world evidence (RWE) experts, with support from IQVIA, as part of the 'RWE Leadership Forum': a group of Industry Leaders who have come together as non-competitive partners to understand and respond to RWD/E challenges and opportunities with a single expert voice. Here, the forum discusses the value in bridging the industry disconnect between RTCs and RWE, with a view to promoting the use of RWE in the RCT environment. RCT endpoints are explored along several axes including their clinical relevance and their measure of direct patient benefit, and then compared with their real-world counterparts to identify suitable paths, or gaps, for assimilating RWE endpoints into the RCT environment.

Active Surveillance of Adverse Events Following Human Papillomavirus Vaccination: Feasibility Pilot Study Based on the Regional Health Care Information Platform in the City of Ningbo, China

BACKGROUND

Comprehensive safety data for vaccines from post-licensure surveillance, especially active surveillance, could guide administrations and individuals to make reasonable decisions on vaccination. Therefore, we designed a pilot study to assess the capability of a regional health care information platform to actively monitor the safety of a newly licensed vaccine.

OBJECTIVE

This study aimed to conduct active surveillance of human papillomavirus (HPV) vaccine safety based on this information platform.

METHODS

In 2017, one of China's most mature information platforms with superior data linkage was selected. A structured questionnaire and open-ended interview guidelines were developed to investigate the feasibility of active surveillance following HPV vaccination using the regional health care information platform in Ningbo. The questionnaire was sent to participants via email, and a face-to-face interview was conducted to confirm details or resolve discrepancies.

RESULTS

Five databases that could be considered essential to active surveillance of vaccine safety were integrated into the platform starting in 2015. Except for residents' health records, which had a coverage rate of 87%, the data sources covered more than 95% of the records that were documented in Ningbo. All the data could be inherently linked using the national identity card. There were 19,328 women who received the HPV vaccine, and 37,988 doses were administered in 2017 and 2018. Women aged 30-40 years accounted for the largest proportion. Quadrivalent vaccination accounted for 73.1% of total vaccination, a much higher proportion than that of bivalent vaccination. Of the first doses, 60 (60/19,328, 0.31%) occurred outside Ningbo. There were no missing data for vaccination-relevant variables, such as identity card, vaccine name, vaccination doses, vaccination date, and manufacturer. ICD-10 coding could be used to identify 9,180 cases using a predefined list of the outcomes of interest, and 1.88% of these cases were missing the identity card. During the 90 days following HPV vaccination, 4 incident cases were found through the linked vaccination history and electronic medical records. The combined incident rate of rheumatoid arthritis, optic neuritis, and Henoch-Schonlein purpura was 8.84/100,000 doses of bivalent HPV, and the incidence rate of rheumatoid arthritis was 3.75/100,000 doses of quadrivalent HPV.

CONCLUSIONS

This study presents an available approach to initiate an active surveillance system for adverse events following HPV vaccination, based on a regional health care information platform in China. An extended observation period or the inclusion of additional functional sites is warranted to conduct future hypothesis-generating and hypothesis-confirming studies for vaccine safety concerns.

Improving measurement of binary covariates in claims data: A simulation study

PURPOSE

When investigators have two claims-based definitions for a binary confounder, it is unclear whether to prefer the more sensitive or more specific definition. Our objective was to compare adjusting for the sensitive or specific definition alone vs two novel approaches combining both definitions: a "two-algorithm indicator" and a "two-algorithm restriction" approach.

METHODS

Each simulated patient had a binary exposure, outcome, and confounder. We created two nested, misclassified versions of the confounder using validated heart failure definitions. The sensitive definition had a sensitivity/specificity of 0.98/0.83, while the specific definition had a sensitivity/specificity of 0.77/0.99. Patients were classified into 3 groups: group 0 did not meet either definition, group 1 met the sensitive but not specific definition, and group 2 met both. The two-algorithm indicator approach adjusted using indicators for groups 1 and 2, while the two-algorithm restriction approach excluded patients in group 1 and adjusted using an indicator for group 2. Adjusted exposure odds ratios (ORs) were estimated for each approach using logistic regression.

RESULTS

The crude OR was 1.33 (95% CI, 1.07-1.63). Adjusting for the specific or sensitive definitions resulted in ORs of 1.09 (95% CI, 0.87-1.35) and 1.14 (95% CI, 0.91-1.40). The two-algorithm indicator method returned an OR of 1.07 (95% CI, 0.86-1.33). The two-algorithm restriction approach returned an OR of 1.02 (95% CI, 0.79-1.29) but excluded 20% of the cohort.

CONCLUSIONS

The two-algorithm indicator approach may improve adjustment for claims-based confounders by returning a point estimate at least as unbiased as the better of the two component definitions.

Validation of an administrative claims coding algorithm for serious opioid overdose: A medical chart review

PURPOSE

A standardized definition for serious opioid overdose has not been clearly established for disease surveillance or assessing the impact of risk mitigation strategies. The purpose of this study was to use medical chart review to clinically validate a claims-based algorithm to identify serious opioid overdose events.

METHODS

The algorithm for serious opioid overdose required an opioid poisoning or external cause ICD-9-CM code occurring within 1 day of (a) an adverse effect code for serious central nervous system or respiratory depression or (b) a mechanical ventilation or critical care CPT code. The claims coding algorithm identified a sample of 145 individuals 18 years or older among patients that presented to the emergency department of two large hospitals in metropolitan Atlanta, Georgia from January 2014 to August 2015. Claims-defined cases were evaluated against rigorous clinical definitions for serious opioid overdose using (a) literature-based criteria for typical clinical manifestations of opioid overdose and/or (b) clinical response to the opioid-specific reversal agent naloxone. The positive predictive value (PPV) for a serious opioid overdose was calculated as the percentage of clinically confirmed cases (definite or probable).

RESULTS

Among 140 evaluable claims-defined cases, 107 fulfilled clinical criteria for a serious opioid overdose [95 definite and 12 probable; PPV of 76.4% (95% CI 69.4%, 83.5%)]. Among 30 nonconfirmed cases, 20 were polyintoxications involving one or more nonopioid psychoactive agents.

CONCLUSIONS

An administrative claims coding algorithm for serious opioid overdose had high clinical predictive performance in a medical chart review.

Identification of gastrointestinal perforation based on ICD-10 code in a Japanese administrative medical information database and associated drug exposure risk factors

PURPOSE

The purpose of this study is to evaluate the accuracy of gastrointestinal (GI) perforation ICD-10 coding in the Diagnosis Procedure Combination (DPC) database and to examine drug exposure risk factors for GI perforation.

METHODS

A total of 100 patients with GI perforation ICD-10 codes were selected randomly from Kagawa University Hospital's DPC database between April 2011 and December 2016. Two experienced specialist physicians independently reviewed the medical records and classified cases as "definite A," "definite B," "probable," or "no GI perforation." The positive predictive values (PPVs) of "definite A/B" cases were calculated after stratification by sex, age, ICD-10 code, and diagnostic information in the DPC data. The number of prescribed drugs with side effects of GI perforation according to historical data was compared between "definite A/B" and "no GI perforation" cases.

RESULTS

The overall PPV was 47.0% (95% confidence interval [CI], 36.9-57.2). However, the PPVs for the three categories of diagnostic information in the DPC data ("main diagnosis," "diagnosis causing admission," and "most resource-intensive diagnosis") were each more than 70% after excluding inappropriate patients. Additionally, the PPV focused on these three categories was 76.3% (95% CI, 59.8-88.6). Prescribed drugs with side effects of GI perforation were more frequently detected in "definite A/B" cases (P = .028).

CONCLUSIONS

Although the overall PPV for GI perforation based on ICD-10 code was low, our results suggest that the PPV could be improved by appropriate selection of DPC diagnosis category and that use of multiple medications enhances the risk of GI perforation.

The Systemic Safety of Ranibizumab in Patients 85 Years and Older with Neovascular Age-Related Macular Degeneration

OBJECTIVE

Ranibizumab safety is well established for treatment of neovascular age-related macular degeneration (nAMD), but less is known about the risk of systemic serious adverse events (SAEs), specifically among patients with heightened baseline risk due to age (≥85 years). This analysis examines whether patients ≥85 years of age versus those <85 years experience an increased risk of key systemic SAEs during intravitreal ranibizumab treatment for nAMD.

DESIGN

Retrospective, pooled analysis of safety data from 5 phase III/IIIb multicenter randomized clinical trials in patients with nAMD: ANCHOR, MARINA, PIER, SAILOR, and HARBOR.

PARTICIPANTS

Patients with nAMD receiving ranibizumab (n = 4347) or control (sham/verteporfin photodynamic therapy, n = 441) treatment included in the safety-evaluable set of the 5 trials.

METHODS

The incidence of nonocular SAEs was analyzed stratified by age (<85 years [n = 3795] vs ≥85 years [n = 993]), treatment (control, ranibizumab 0.3 mg, ranibizumab 0.5 mg, ranibizumab 2.0 mg), and injection frequency (monthly, as needed [PRN]).

MAIN OUTCOME MEASURES

Incidence of key systemic SAEs, defined as total nonocular SAEs, deaths, cardiovascular events, cerebrovascular (CBV) events, and Antiplatelet Trialists' Collaboration events.

RESULTS

The MARINA and ANCHOR trials had greater rates of key SAEs for patients ≥85 years versus those <85 years. Ranibizumab exposure did not increase the risk of most SAEs in elderly patients; for CBV events and death, the effect of ranibizumab versus control treatment for age ≥85 years was not interpretable due to small number of events (CBV: n = 2, 2, 5 for control, ranibizumab 0.3 mg, and ranibizumab 0.5 mg, respectively; death: n = 2, 4, 5, respectively). Across all 5 trials, an increased risk was found for age ≥85 years versus <85 years for the marketed dose of ranibizumab 0.5 mg. In the HARBOR trial, increased rates of key SAEs (excluding total nonocular SAEs) for age ≥85 years versus <85 years were observed with monthly dosing but not with PRN dosing; event rates were similar for 2.0 mg versus 0.5 mg.

CONCLUSIONS

Consistent with general trends, the risk of key systemic SAEs was associated with age ≥85 years versus <85 years, but not with ranibizumab drug exposure. The difference between monthly versus PRN was inconclusive. There was no evidence of a dose effect. Interpretation of this retrospective analysis is limited because it was not prospectively powered for statistically definitive conclusions.

Treatment sequence of lenalidomide and hypomethylating agents and the impact on clinical outcomes for patients with myelodysplastic syndromes

Lenalidomide and hypomethylating agents (HMAs) azacitidine and decitabine are approved for treating myelodysplastic syndromes (MDS), but optimal sequencing is unclear. Adults with MDS were identified from a US payer claims database (Inovalon MORE² Registry) to compare outcomes with lenalidomide followed by HMA (LEN-HMA) or HMA followed by lenalidomide (HMA-LEN). There were 96 patients who received LEN-HMA and 89 who received HMA-LEN. LEN-HMA-treated patients had a longer time to second treatment discontinuation (29.0 vs. 19.0 months, p=.009; adjusted hazard ratio [HR] 0.52, 95% confidence interval [CI] 0.29-0.91, p=.023). LEN-HMA-treated patients had a longer median time to insurance disenrollment (22.4 vs. 16.1 months, p<.001; adjusted HR 0.64, 95% CI: 0.44-0.92, p=.017), used as a proxy for survival. Longer treatment duration and survival with LEN-HMA support first-line use of lenalidomide in MDS in sequence with HMAs.

Identifying Patients With Relapsing-Remitting Multiple Sclerosis Using Algorithms Applied to US Integrated Delivery Network Healthcare Data

BACKGROUND

Relapsing-remitting multiple sclerosis (RRMS) has a major impact on affected patients; therefore, improved understanding of RRMS is important, particularly in the context of real-world evidence.

OBJECTIVES

To develop and validate algorithms for identifying patients with RRMS in both unstructured clinical notes found in electronic health records (EHRs) and structured/coded health care claims data.

METHODS

US Integrated Delivery Network data (2010-2014) were queried for study inclusion criteria (possible multiple sclerosis [MS] base cohort): one or more MS diagnosis code, patients aged 18 years or older, 1 year or more baseline history, and no other demyelinating diseases. Sets of algorithms were developed to search narrative text of unstructured clinical notes (EHR clinical notes-based algorithms) and structured/coded data (claims-based algorithms) to identify adult patients with RRMS, excluding patients with evidence of progressive MS. Medical records were reviewed manually for algorithm validation. Positive predictive value was calculated for both EHR clinical notes-based and claims-based algorithms.

RESULTS

From a sample of 5308 patients with possible MS, 837 patients with RRMS were identified using only the EHR clinical notes-based algorithms and 2271 patients were identified using only the claims-based algorithms; 779 patients were identified using both algorithms. The positive predictive value was 99.1% (95% confidence interval [CI], 94.2%-100%) for the EHR clinical notes-based algorithms and 94.6% (95% CI, 89.1%-97.8%) to 94.9% (95% CI, 89.8%-97.9%) for the claims-based algorithms.

CONCLUSIONS

The algorithms evaluated in this study identified a real-world cohort of patients with RRMS without evidence of progressive MS that can be studied in clinical research with confidence.

Measuring prevalence and incidence of chronic conditions in claims and electronic health record databases

Background

Health care databases are natural sources for estimating prevalence and incidence of chronic conditions, but substantial variation in estimates limits their interpretability and utility. We evaluated the effects of design choices when estimating prevalence and incidence in claims and electronic health record databases.

Methods

Prevalence and incidence for five chronic diseases at increasing levels of expected frequencies, from cystic fibrosis to COPD, were estimated in the Clinical Practice Research Datalink (CPRD) and MarketScan databases from 2011 to 2014. Estimates were compared using different definitions of lookback time and contributed person-time.

Results

Variation in lookback time substantially affected estimates. In 2014, for CPRD, use of an all-time vs a 1-year lookback window resulted in 4.3–8.3 times higher prevalence (depending on disease), reducing incidence by 1.9–3.3 times. All-time lookback resulted in strong temporal trends. COPD prevalence between 2011 and 2014 in MarketScan increased by 25% with an all-time lookback but stayed relatively constant with a 1-year lookback. Varying observability did not substantially affect estimates.

Conclusion

This framework draws attention to the underrecognized potential for widely varying incidence and prevalence estimates, with implications for care planning and drug development. Though prevalence and incidence are seemingly straightforward concepts, careful consideration of methodology is required to obtain meaningful estimates from health care databases.

Use of Angiotensin-Converting Enzyme Inhibitors and Angiotensin Receptor Blockers for Geriatric Ischemic Stroke Patients: Are the Rates Right?

BACKGROUND

Our objective is to estimate the effects associated with higher rates of renin-angiotensin system antagonists, angiotensin-converting enzyme inhibitors and angiotensin receptor blockers (ACEI/ARBs), in secondary prevention for geriatric (aged >65 years) patients with new ischemic strokes by chronic kidney disease (CKD) status.

METHODS AND RESULTS

The effects of ACEI/ARBs on survival and renal risk were estimated by CKD status using an instrumental variable (IV) estimator. Instruments were based on local area variation in ACEI/ARB use. Data abstracted from charts were used to assess the assumptions underlying the instrumental estimator. ACEI/ARBs were used after stroke by 45.9% and 45.2% of CKD and non-CKD patients, respectively. ACEI/ARB rate differences across local areas grouped by practice styles were nearly identical for CKD and non-CKD patients. Higher ACEI/ARB use rates for non-CKD patients were associated with higher 2-year survival rates, whereas higher ACEI/ARB use rates for patients with CKD were associated with lower 2-year survival rates. While the negative survival estimates for patients with CKD were not statistically different from zero, they were statistically lower than the estimates for non-CKD patients. Confounders abstracted from charts were not associated with the instrumental variable used.

CONCLUSIONS

Higher ACEI/ARB use rates had different survival implications for older ischemic stroke patients with and without CKD. ACEI/ARBs appear underused in ischemic stroke patients without CKD as higher use rates were associated with higher 2-year survival rates. This conclusion is not generalizable to the ischemic stroke patients with CKD, as higher ACEI/ARBS use rates were associated with lower 2-year survival rates that were statistically lower than the estimates for non-CKD patients.

Identifying Suicide Ideation and Suicidal Attempts in a Psychiatric Clinical Research Database using Natural Language Processing

Research into suicide prevention has been hampered by methodological limitations such as low sample size and recall bias. Recently, Natural Language Processing (NLP) strategies have been used with Electronic Health Records to increase information extraction from free text notes as well as structured fields concerning suicidality and this allows access to much larger cohorts than previously possible. This paper presents two novel NLP approaches - a rule-based approach to classify the presence of suicide ideation and a hybrid machine learning and rule-based approach to identify suicide attempts in a psychiatric clinical database. Good performance of the two classifiers in the evaluation study suggest they can be used to accurately detect mentions of suicide ideation and attempt within free-text documents in this psychiatric database. The novelty of the two approaches lies in the malleability of each classifier if a need to refine performance, or meet alternate classification requirements arises. The algorithms can also be adapted to fit infrastructures of other clinical datasets given sufficient clinical recording practice knowledge, without dependency on medical codes or additional data extraction of known risk factors to predict suicidal behaviour.

Identifying diabetes cases from administrative data: a population-based validation study

Background

Health care data allow for the study and surveillance of chronic diseases such as diabetes. The objective of this study was to identify and validate optimal algorithms for diabetes cases within health care administrative databases for different research purposes, populations, and data sources.

Methods

We linked health care administrative databases from Ontario, Canada to a reference standard of primary care electronic medical records (EMRs). We then identified and calculated the performance characteristics of multiple adult diabetes case definitions, using combinations of data sources and time windows.

Results

The best algorithm to identify diabetes cases was the presence at any time of one hospitalization or physician claim for diabetes AND either one prescription for an anti-diabetic medication or one physician claim with a diabetes-specific fee code [sensitivity 84.2%, specificity 99.2%, positive predictive value (PPV) 92.5%]. Use of physician claims alone performed almost as well: three physician claims for diabetes within one year was highly specific (sensitivity 79.9%, specificity 99.1%, PPV 91.4%) and one physician claim at any time was highly sensitive (sensitivity 93.6%, specificity 91.9%, PPV 58.5%).

Conclusions

This study identifies validated algorithms to capture diabetes cases within health care administrative databases for a range of purposes, populations and data availability. These findings are useful to study trends and outcomes of diabetes using routinely-collected health care data.

Electronic supplementary material

The online version of this article (10.1186/s12913-018-3148-0) contains supplementary material, which is available to authorized users.

Chart validation of inpatient International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) administrative diagnosis codes for venous thromboembolism (VTE) among intravenous immune globulin (IGIV) users in the Sentinel Distributed Database

The Sentinel Distributed Database (SDD) is a database of patient administrative healthcare records, derived from insurance claims and electronic health records, sponsored by the US Food and Drug Administration for evaluation of medical product outcomes. There is limited information on the validity of diagnosis codes for acute venous thromboembolism (VTE) in the SDD and administrative healthcare data more generally.In this chart validation study, we report on the positive predictive value (PPV) of inpatient administrative diagnosis codes for acute VTE-pulmonary embolism (PE) or lower-extremity or site-unspecified deep vein thrombosis (DVT)-within the SDD. As part of an assessment of thromboembolic adverse event risk following treatment with intravenous immune globulin (IGIV), charts were obtained for 75 potential VTE cases, abstracted, and physician-adjudicated.VTE status was determined for 62 potential cases. PPVs for lower-extremity DVT and/or PE were 90% (95% CI: 73-98%) for principal-position diagnoses, 80% (95% CI: 28-99%) for secondary diagnoses, and 26% (95% CI: 11-46%) for position-unspecified diagnoses (originating from physician claims associated with an inpatient stay). Average symptom onset was 1.5 days prior to hospital admission (range: 19 days prior to 4 days after admission).PPVs for principal and secondary VTE discharge diagnoses were similar to prior study estimates. Position-unspecified diagnoses were less likely to represent true acute VTE cases.

Bias from outcome misclassification in immunization schedule safety research

PURPOSE

The Institute of Medicine recommended conducting observational studies of childhood immunization schedule safety. Such studies could be biased by outcome misclassification, leading to incorrect inferences. Using simulations, we evaluated (1) outcome positive predictive values (PPVs) as indicators of bias of an exposure-outcome association, and (2) quantitative bias analyses (QBA) for bias correction.

METHODS

Simulations were conducted based on proposed or ongoing Vaccine Safety Datalink studies. We simulated 4 studies of 2 exposure groups (children with no vaccines or on alternative schedules) and 2 baseline outcome levels (100 and 1000/100 000 person-years), with 3 relative risk (RR) levels (RR = 0.50, 1.00, and 2.00), across 1000 replications using probabilistic modeling. We quantified bias from non-differential and differential outcome misclassification, based on levels previously measured in database research (sensitivity > 95%; specificity > 99%). We calculated median outcome PPVs, median observed RRs, Type 1 error, and bias-corrected RRs following QBA.

RESULTS

We observed PPVs from 34% to 98%. With non-differential misclassification and true RR = 2.00, median bias was toward the null, with severe bias (median observed RR = 1.33) with PPV = 34% and modest bias (median observed RR = 1.83) with PPV = 83%. With differential misclassification, PPVs did not reflect median bias, and there was Type 1 error of 100% with PPV = 90%. QBA was generally effective in correcting misclassification bias.

CONCLUSIONS

In immunization schedule studies, outcome misclassification may be non-differential or differential to exposure. Overall outcome PPVs do not reflect the distribution of false positives by exposure and are poor indicators of bias in individual studies. Our results support QBA for immunization schedule safety research.

Giardiasis Diagnosis and Treatment Practices Among Commercially Insured Persons in the United States

Background

Giardiasis, the most common enteric parasitic infection in the United States, causes an estimated 1.2 million episodes of illness annually. Published clinical recommendations include readily available Giardia-specific diagnostic testing and antiparasitic drugs. We investigated sequences of giardiasis diagnostic and treatment events using MarketScan, a large health insurance claims database.

Methods

We created a longitudinal cohort of 2995 persons diagnosed with giardiasis (International Classification of Diseases, Ninth Revision, Clinical Modification [ICD-9-CM] code 007.1) from 2006 to 2010, and analyzed claims occurring 90 days before to 90 days after initial diagnosis. We evaluated differences in number and sequence of visits, diagnostic tests, and prescriptions by age group (children 1-17 years, adults 18-64 years) using χ2 tests and data visualization software.

Results

Among 2995 patients (212433 claims), 18% had a Giardia-specific test followed by or concurrent with an effective antiparasitic drug, without ineffective antibiotics. Almost two-thirds of patients had an antiparasitic and 27% had an antibiotic during the study window. Compared with children, adults more often had ≥3 visits before diagnosis (19% vs 15%; P = .02). Adults were also less likely to have a Giardia-specific diagnostic test (48% vs 58%; P < .001) and more likely to have an antibiotic prescription (28% vs 25%; P = .04). When Giardia-specific tests and antiparasitic and antibiotic prescriptions were examined, pediatric clinical event sequences most frequently began with a Giardia-specific test, whereas adult sequences most frequently began with an antiparasitic prescription.

Conclusions

Giardiasis care infrequently follows all aspects of clinical recommendations. Multiple differences between pediatric and adult care, despite age-agnostic recommendations, suggest opportunities for provider education or tailored guidance.

Needles in the haystack: Using open-text fields to identify persons with intellectual and developmental disabilities in administrative home care data

BACKGROUND

Use of administrative health data to study populations of interest is becoming more common. Identifying individuals with intellectual and developmental disabilities (IDD) in existing databases can be challenging due to inconsistent definitions and terminologies of IDD over time and across sectors, and the inability to rely on etiologies of IDD as they are frequently unknown.

AIMS

To identify diagnoses related to IDD in an administrative database and create a cohort of persons with IDD.

METHODS

Open-text diagnostic entries related to IDD were identified in an Ontario home care database (2003-2015) and coded as being either acceptable (e.g. Down syndrome) or ambiguous (e.g. intellectually challenged). The cognitive and functional skills of the resulting groups were compared using logistic regressions and standardized differences, and their age distributions were compared to that of the general home care population.

RESULTS

Just under 1% of the home care population had a diagnostic entry related to IDD. Ambiguous terms were most commonly used (61%), and this group tended to be older and less impaired than the group with more acceptable terms used to describe their IDD.

CONCLUSIONS

Open-text diagnostic variables in administrative health records can be used to identify and study individuals with IDD.

IMPLICATIONS

Future work is needed to educate assessors on the importance of using standard, accepted terminology when recording diagnoses related to IDD.

Giardiasis and Subsequent Irritable Bowel Syndrome: A Longitudinal Cohort Study Using Health Insurance Data

Background

Giardia intestinalis is the most commonly reported human intestinal parasite in the United States. Increased incidence of chronic gastrointestinal complaints has been reported after some giardiasis outbreaks. We examined the relationship between giardiasis diagnosis and irritable bowel syndrome (IBS) diagnosis.

Methods

We used the 2006-2010 MarketScan commercial insurance database. Persons with at least 1 giardiasis diagnosis were individually matched on age group, sex, and enrollment length in months to 5 persons without a giardiasis diagnosis. Persons diagnosed with IBS before the date of study entry were excluded. We calculated crude incidence rates (IRs) and developed Cox proportional hazards models.

Results

The matched cohort included 3935 persons with giardiasis and 19663 persons without giardiasis. One-year incidence of IBS was higher in persons with giardiasis (IR = 37.7/1000 person-years vs 4.4/1000 person-years). The unadjusted hazard ratio was 4.8 (95% confidence interval [CI] = 3.6-6.4), attenuated slightly to 3.9 (95% CI = 2.9-5.4) after adjusting for anxiety, depression, and healthcare utilization.

Conclusions

In a large insurance database, individuals diagnosed with giardiasis were more likely to have a subsequent IBS diagnosis, despite accounting for confounders. Future research on risk factors for IBS among giardiasis patients and the pathophysiology of postinfectious IBS is needed.

Successful Comparison of US Food and Drug Administration Sentinel Analysis Tools to Traditional Approaches in Quantifying a Known Drug-Adverse Event Association

The US Food and Drug Administration's Sentinel system has developed the capability to conduct active safety surveillance of marketed medical products in a large network of electronic healthcare databases. We assessed the extent to which the newly developed, semiautomated Sentinel Propensity Score Matching (PSM) tool could produce the same results as a customized protocol-driven assessment, which found an adjusted hazard ratio (HR) of 3.04 (95% confidence interval [CI], 2.81-3.27) comparing angioedema in patients initiating angiotensin-converting enzyme (ACE) inhibitors vs. beta-blockers. Using data from 13 Data Partners between 1 January 2008, and 30 September 2013, the PSM tool identified 2,211,215 eligible ACE inhibitor and 1,673,682 eligible beta-blocker initiators. The tool produced an HR of 3.14 (95% CI, 2.86-3.44). This comparison provides initial evidence that Sentinel analytic tools can produce findings similar to those produced by a highly customized protocol-driven assessment.

Using linked electronic data to validate algorithms for health outcomes in administrative databases

The validity of algorithms used to identify health outcomes in claims-based and administrative data is critical to the reliability of findings from observational studies. The traditional approach to algorithm validation, using medical charts, is expensive and time-consuming. An alternative method is to link the claims data to an external, electronic data source that contains information allowing confirmation of the event of interest. In this paper, we describe this external linkage validation method and delineate important considerations to assess the feasibility and appropriateness of validating health outcomes using this approach. This framework can help investigators decide whether to pursue an external linkage validation method for identifying health outcomes in administrative/claims data.

Pharmacoepidemiologic Methods for Studying the Health Effects of Drug-Drug Interactions

A drug-drug interaction (DDI) occurs when one or more drugs affect the pharmacokinetics (the body's effect on the drug) and/or pharmacodynamics (the drug's effect on the body) of one or more other drugs. Pharmacoepidemiologic studies are the principal way of studying the health effects of potential DDIs. This article discusses aspects of pharmacoepidemiologic research designs that are particularly salient to the design and interpretation of pharmacoepidemiologic studies of DDIs.

Misclassification in administrative claims data: quantifying the impact on treatment effect estimates

Misclassification is present in nearly every epidemiologic study, yet is rarely quantified in analysis in favor of a focus on random error. In this review, we discuss past and present wisdom on misclassification and what measures should be taken to quantify this influential bias, with a focus on bias in pharmacoepidemiologic studies. To date, pharmacoepidemiology primarily utilizes data obtained from administrative claims, a rich source of prescription data but susceptible to bias from unobservable factors including medication sample use, medications filled but not taken, health conditions that are not reported in the administrative billing data, and inadequate capture of confounders. Due to the increasing focus on comparative effectiveness research, we provide a discussion of misclassification in the context of an active comparator, including a demonstration of treatment effects biased away from the null in the presence of nondifferential misclassification. Finally, we highlight recently developed methods to quantify bias and offer these methods as potential options for strengthening the validity and quantifying uncertainty of results obtained from pharmacoepidemiologic research.

Misclassification in administrative claims data: quantifying the impact on treatment effect estimates

Misclassification is present in nearly every epidemiologic study, yet is rarely quantified in analysis in favor of a focus on random error. In this review, we discuss past and present wisdom on misclassification and what measures should be taken to quantify this influential bias, with a focus on bias in pharmacoepidemiologic studies. To date, pharmacoepidemiology primarily utilizes data obtained from administrative claims, a rich source of prescription data but susceptible to bias from unobservable factors including medication sample use, medications filled but not taken, health conditions that are not reported in the administrative billing data, and inadequate capture of confounders. Due to the increasing focus on comparative effectiveness research, we provide a discussion of misclassification in the context of an active comparator, including a demonstration of treatment effects biased away from the null in the presence of nondifferential misclassification. Finally, we highlight recently developed methods to quantify bias and offer these methods as potential options for strengthening the validity and quantifying uncertainty of results obtained from pharmacoepidemiologic research.

Minimizing signal detection time in postmarket sequential analysis: balancing positive predictive value and sensitivity

PURPOSE

Outcome misclassification in retrospective epidemiologic analyses has been well-studied, but little is known about such misclassification with respect to sequential statistical analysis during surveillance of medical product-associated risks, a planned capability of the US Food and Drug Administration's Sentinel System.

METHODS

Using a vaccine example, we model and simulate sequential database surveillance in an observational data network using a variety of outcome detection algorithms. We consider how these algorithms, as characterized by sensitivity and positive predictive value, impact the length of surveillance and timeliness of safety signal detection. We show investigators/users of these networks how they can perform preparatory study design calculations that consider outcome misclassification in sequential database surveillance.

RESULTS

Non-differential outcome misclassification generates longer surveillance times and less timely safety signal detection as compared with the case of no misclassification. Inclusive algorithms characterized by high sensitivity but low positive predictive value outperform more narrow algorithms when detecting rare outcomes. This decision calculus may change considerably if medical chart validation procedures were required.

CONCLUSIONS

These findings raise important questions regarding the design of observational data networks used for pharmacovigilance. Specifically, there are tradeoffs involved when choosing to populate such networks with component databases that are large as compared with smaller integrated delivery system databases that can more easily access laboratory or clinical data and perform medical chart validation.

Commonly used data-collection approaches in clinical research

We provide an overview of the different data-collection approaches that are commonly used in carrying out clinical, public health, and translational research. We discuss several of the factors that researchers need to consider in using data collected in questionnaire surveys, from proxy informants, through the review of medical records, and in the collection of biologic samples. We hope that the points raised in this overview will lead to the collection of rich and high-quality data in observational studies and randomized controlled trials.

What is the effect of area size when using local area practice style as an instrument?

OBJECTIVES

Discuss the tradeoffs inherent in choosing a local area size when using a measure of local area practice style as an instrument in instrumental variable estimation when assessing treatment effectiveness.

STUDY DESIGN

Assess the effectiveness of angiotensin converting-enzyme inhibitors and angiotensin receptor blockers on survival after acute myocardial infarction for Medicare beneficiaries using practice style instruments based on different-sized local areas around patients. We contrasted treatment effect estimates using different local area sizes in terms of the strength of the relationship between local area practice styles and individual patient treatment choices; and indirect assessments of the assumption violations.

RESULTS

Using smaller local areas to measure practice styles exploits more treatment variation and results in smaller standard errors. However, if treatment effects are heterogeneous, the use of smaller local areas may increase the risk that local practice style measures are dominated by differences in average treatment effectiveness across areas and bias results toward greater effectiveness.

CONCLUSION

Local area practice style measures can be useful instruments in instrumental variable analysis, but the use of smaller local area sizes to generate greater treatment variation may result in treatment effect estimates that are biased toward higher effectiveness. Assessment of whether ecological bias can be mitigated by changing local area size requires the use of outside data sources.