Evaluating bias due to data linkage error in electronic healthcare records

Effect of utilizing either a self-reported questionnaire or administrative data alone or in combination on the findings of a randomized controlled trial of the long-term effects of antenatal corticosteroids

INTRODUCTION

A combination of self-reported questionnaire and administrative data could potentially enhance ascertainment of outcomes and alleviate the limitations of both in follow up studies. However, it is uncertain how access to only one of these data sources to assess outcomes impact study findings. Therefore, this study aimed to determine whether the study findings would be altered if the outcomes were assessed by different data sources alone or in combination.

METHODS

At 50-year follow-up of participants in a randomized trial, we assessed the effect of antenatal betamethasone exposure on the diagnosis of diabetes, pre-diabetes, hyperlipidemia, hypertension, mental health disorders, and asthma using a self-reported questionnaire, administrative data, a combination of both, or any data source, with or without adjudication by an expert panel of five clinicians. Differences between relative risks derived from each data source were calculated using the Bland-Altman approach.

RESULTS

There were 424 participants (46% of those eligible, aged 49 years, SD 1, 50% male). There were no differences in study outcomes between participants exposed to betamethasone and those exposed to placebo when the outcomes were assessed using different data sources. When compared to the study findings determined using adjudicated outcomes, the mean difference (limits of agreement) in relative risks derived from other data sources were: self-reported questionnaires 0.02 (-0.35 to 0.40), administrative data 0.06 (-0.32 to 0.44), both questionnaire and administrative data 0.01 (-0.41 to 0.43), and any data source, 0.01 (-0.08 to 0.10).

CONCLUSION

Utilizing a self-reported questionnaire, administrative data, both questionnaire and administrative data, or any of these sources for assessing study outcomes had no impact on the study findings compared with when study outcomes were assessed using adjudicated outcomes.

Comparison of outcomes of the 50-year follow-up of a randomized trial assessed by study questionnaire and by data linkage: The CONCUR study

BACKGROUND/AIMS

Self-reported questionnaires on health status after randomized trials can be time-consuming, costly, and potentially unreliable. Administrative data sets may provide cost-effective, less biased information, but it is uncertain how administrative and self-reported data compare to identify chronic conditions in a New Zealand cohort. This study aimed to determine whether record linkage could replace self-reported questionnaires to identify chronic conditions that were the outcomes of interest for trial follow-up.

METHODS

Participants in 50-year follow-up of a randomized trial were asked to complete a questionnaire and to consent to accessing administrative data. The proportion of participants with diabetes, pre-diabetes, hyperlipidaemia, hypertension, mental health disorders, and asthma was calculated using each data source and agreement between data sources assessed.

RESULTS

Participants were aged 49 years (SD = 1, n = 424, 50% male). Agreement between questionnaire and administrative data was slight for pre-diabetes (kappa = 0.10), fair for hyperlipidaemia (kappa = 0.27), substantial for diabetes (kappa = 0.65), and moderate for other conditions (all kappa >0.42). Administrative data alone identified two to three times more cases than the questionnaire for all outcomes except hypertension and mental health disorders, where the questionnaire alone identified one to two times more cases than administrative data. Combining all sources increased case detection for all outcomes.

CONCLUSIONS

A combination of questionnaire, pharmaceutical, and laboratory data with expert panel review were required to identify participants with chronic conditions of interest in this follow-up of a clinical trial.

Evaluating Linkage Quality of Population-Based Administrative Data for Health Service Research

BACKGROUND

To overcome the limitations of relying on data from a single institution, many researchers have studied data linkage methodologies. Data linkage includes errors owing to legal issues surrounding personal information and technical issues related to data processing. Linkage errors affect selection bias, and external and internal validity. Therefore, quality verification for each connection method with adherence to personal information protection is an important issue. This study evaluated the linkage quality of linked data and analyzed the potential bias resulting from linkage errors.

METHODS

This study analyzed claims data submitted to the Health Insurance Review and Assessment Service (HIRA DATA). The linkage errors of the two deterministic linkage methods were evaluated based on the use of the match key. The first deterministic linkage uses a unique identification number, and the second deterministic linkage uses the name, gender, and date of birth as a set of partial identifiers. The linkage error included in this deterministic linkage method was compared with the absolute standardized difference (ASD) of Cohen's according to the baseline characteristics, and the linkage quality was evaluated through the following indicators: linked rate, false match rate, missed match rate, positive predictive value, sensitivity, specificity, and F1-score.

RESULTS

For the deterministic linkage method that used the name, gender, and date of birth as a set of partial identifiers, the true match rate was 83.5 and the missed match rate was 16.5. Although there was bias in some characteristics of the data, most of the ASD values were less than 0.1, with no case greater than 0.5. Therefore, it is difficult to determine whether linked data constructed with deterministic linkages have substantial differences.

CONCLUSION

This study confirms the possibility of building health and medical data at the national level as the first data linkage quality verification study using big data from the HIRA. Analyzing the quality of linkages is crucial for comprehending linkage errors and generating reliable analytical outcomes. Linkers should increase the reliability of linked data by providing linkage error-related information to researchers. The results of this study will serve as reference data to increase the reliability of multicenter data linkage studies.

A systematic review and meta-analysis of data linkage between motor vehicle crash and hospital-based datasets

Motor vehicle crash data linkage has emerged as a vital tool to better understand the injury outcomes and the factors contributing to crashes. This systematic review and meta-analysis aims to explore the existing knowledge on data linkage between motor vehicle crashes and hospital-based datasets, summarize and highlight the findings of previous studies, and identify gaps in research. A comprehensive and systematic search of the literature yielded 54 studies for a qualitative analysis, and 35 of which were also considered for a quantitative meta-analysis. Findings highlight a range of viable methodologies for linking datasets, including manual, deterministic, probabilistic, and integrative methods. Designing a linkage method that integrates different algorithms and techniques is more likely to result in higher match rate and fewer errors. Examining the results of the meta-analysis reveals that a wide range of linkage rates were reported. There are several factors beyond the approach that affect the linkage rate including the size and coverage of both datasets and the linkage variables. Gender, age, crash type, and roadway geometry at the crash site were likely to be associated with a record's presence in a linked dataset. Linkage rate alone is not the only important metric and when linkage rate is used as a metric in research, both police and hospital rates should be reported. This study also highlights the importance of examining and accounting for population and bias introduced by linking two datasets.

Novel linkage approach to join community-acquired and national data

BACKGROUND

Community optometrists in Scotland have performed regular free-at-point-of-care eye examinations for all, for over 15 years. Eye examinations include retinal imaging but image storage is fragmented and they are not used for research. The Scottish Collaborative Optometry-Ophthalmology Network e-research project aimed to collect these images and create a repository linked to routinely collected healthcare data, supporting the development of pre-symptomatic diagnostic tools.

METHODS

As the image record was usually separate from the patient record and contained minimal patient information, we developed an efficient matching algorithm using a combination of deterministic and probabilistic steps which minimised the risk of false positives, to facilitate national health record linkage. We visited two practices and assessed the data contained in their image device and Practice Management Systems. Practice activities were explored to understand the context of data collection processes. Iteratively, we tested a series of matching rules which captured a high proportion of true positive records compared to manual matches. The approach was validated by testing manual matching against automated steps in three further practices.

RESULTS

A sequence of deterministic rules successfully matched 95% of records in the three test practices compared to manual matching. Adding two probabilistic rules to the algorithm successfully matched 99% of records.

CONCLUSIONS

The potential value of community-acquired retinal images can be harnessed only if they are linked to centrally-held healthcare care data. Despite the lack of interoperability between systems within optometry practices and inconsistent use of unique identifiers, data linkage is possible using robust, almost entirely automated processes.

Virtual patient identifier (vPID): Improving patient traceability using anonymized identifiers in Japanese healthcare insurance claims database

Objective

Japan's national-level healthcare insurance claims database (NDB) is a collective database that contains the entire information on healthcare services being provided to all citizens. However, existing anonymized identifiers (ID1 and ID2) have a poor capability of tracing patients' claims in the database, hindering longitudinal analyses. This study presents a virtual patient identifier (vPID), which we have developed on top of these existing identifiers, to improve the patient traceability.

Methods

vPID is a new composite identifier that intensively consolidates ID1 and ID2 co-occurring in an identical claim to allow to collect claims of each patient even though its ID1 or ID2 may change due to life events or clerical errors. We conducted a verification test with prefecture-level datasets of healthcare insurance claims and enrollee history records, which allowed us to compare vPID with the ground truth, in terms of an identifiability score (indicating a capability of distinguishing a patient's claims from another patient's claims) and a traceability score (indicating a capability of collecting claims of an identical patient).

Results

The verification test has clarified that vPID offers significantly higher traceability scores (0.994, Mie; 0.997, Gifu) than ID1 (0.863, Mie; 0.884, Gifu) and ID2 (0.602, Mie; 0.839, Gifu), and comparable (0.996, Mie) and lower (0.979, Gifu) identifiability scores.

Discussion

vPID is seemingly useful for a wide spectrum of analytic studies unless they focus on sensitive cases to the design limitation of vPID, such as patients experiencing marriage and job change, simultaneously, and same-sex twin children.

Conclusion

vPID successfully improves patient traceability, providing an opportunity for longitudinal analyses that used to be practically impossible for NDB. Further exploration is also necessary, in particular, for mitigating identification errors.

Comparing primary caregivers' reported injury data with routinely recorded injury data to assess predictors of childhood injury

BACKGROUND

Linking self-reported data collected from longitudinal studies with administrative health records is timely and cost-effective, provides the opportunity to augment information contained in each and can offset some of the limitations of both data sources. The aim of this study was to compare maternal-reported child injury data with administrative injury records and assess the level of agreement.

METHODS

A deterministic linkage was undertaken to link injury-related data from the Growing up in New Zealand (GUiNZ) study to routinely collected injury records from New Zealand's Accident Compensation Corporation (ACC) for preschool children. The analyses compared: (i) the characteristics of mothers with linked data vs. those without, (ii) injury incidences from maternal recall with those recorded in ACC injury claims, and (iii) the demographic characteristics of concordant and discordant injury reports, including the validity and reliability of injury records from both data sources.

RESULTS

Of all mothers who responded to the injury questions in the GUiNZ study (n = 5836), more than 95% (n = 5637) agreed to have their child's record linked to routine administrative health records. The overall discordance in injury reports showed an increasing trend as children grew older (9% at 9 M to 29% at 54 M). The mothers of children with discordance between maternal injury reports and ACC records were more likely to be younger, of Pacific ethnicity, with lower educational attainment, and live in areas of high deprivation (p < 0.001). The level of agreement between maternal injury recall and ACC injury record decreased (κ = 0.83 to κ = 0.42) as the cohort moved through their preschool years.

CONCLUSIONS

In general, the findings of this study identified that there was underreporting and discordance of the maternal injury recall, which varied by the demographic characteristics of mothers and their child's age. Therefore, linking the routinely gathered injury data with maternal self-report child injury data has the potential to augment longitudinal birth cohort study data to investigate risk or protective factors associated with childhood injury.

Leveraging Administrative Data to Better Understand and Address Child Maltreatment: A Scoping Review of Data Linkage Studies

BACKGROUND

This scoping review aimed to overview studies that used administrative data linkage in the context of child maltreatment to improve our understanding of the value that data linkage may confer for policy, practice, and research.

METHODS

We searched MEDLINE, Embase, PsycINFO, CINAHL, and ERIC electronic databases in June 2019 and May 2020 for studies that linked two or more datasets (at least one of which was administrative in nature) to study child maltreatment. We report findings with numerical and narrative summary.

RESULTS

We included 121 studies, mainly from the United States or Australia and published in the past decade. Data came primarily from social services and health sectors, and linkage processes and data quality were often not described in sufficient detail to align with current reporting guidelines. Most studies were descriptive in nature and research questions addressed fell under eight themes: descriptive epidemiology, risk factors, outcomes, intergenerational transmission, predictive modelling, intervention/service evaluation, multi-sector involvement, and methodological considerations/advancements.

CONCLUSIONS

Included studies demonstrated the wide variety of ways in which data linkage can contribute to the public health response to child maltreatment. However, how research using linked data can be translated into effective service development and monitoring, or targeting of interventions, is underexplored in terms of privacy protection, ethics and governance, data quality, and evidence of effectiveness.

Comparison of a palm-based biometric solution with a name-based identification system in rural Bangladesh

Background

Unique identifiers are not universal in low- and middle-income countries. Biometric solutions have the potential to augment existing name-based searches used for identification in these settings. This paper describes a comparison of the searching accuracy of a palm-based biometric solution with a name-based database.

Objective

To compare the identification of individuals between a palm-based biometric solution to a name-based District Health Information Software 2 (DHIS2) Android application, in a low-resource setting.

Methods

The study was conducted in Chandpur district, Bangladesh. Trained data collectors enrolled 150 women of reproductive age into two android applications – i) a name-based DHIS2 application, and ii) a palm-based biometric solution – both run on tablets. One week after enrollment, a different research team member attempted to re-identify each enrolled woman using both systems. A single image or text-based name was used for searching at the time of re-identification. We interviewed data collectors at the end of the study.

Results

Significantly more women were successfully identified on the first attempt with a palm-based biometric application (84%) compared with the name-based DHIS2 application (61%). The proportion of identifications that required three or more attempts was similar between name-based (7%, CI 3.7–12.3) and palm-based biometric system (5%, CI: 1.9–9.4). However, the total number of attempts needed was significantly lower with the palm-based solution (mean 1.2 vs. 1.5, p < 0.001). In a group discussion, data collectors reported that the palm-based biometric identification system was both accurate and easy to use.

Conclusion

A palm-based biometric identification system on mobile devices was found to be an easy-to-use and accurate technology for the unique identification of individuals compared to an existing name-based application. Our findings imply that palm-based biometrics on mobile devices may be the next step in establishing unique identifiers in remote and rural settings where they are currently absent.

Estimating the Number of Persons with HIV in Jails via Web Scraping and Record Linkage

This paper presents methods to estimate the number of persons with HIV in North Carolina jails by applying finite population inferential approaches to data collected using web scraping and record linkage techniques. Administrative data are linked with web-scraped rosters of incarcerated persons in a nonrandom subset of counties. Outcome regression and calibration weighting are adapted for state-level estimation. Methods are compared in simulations and are applied to data from the US state of North Carolina. Outcome regression yielded more precise inference and allowed for county-level estimates, an important study objective, while calibration weighting exhibited double robustness under misspecification of the outcome or weight model.

Computer System for the Capture and Preparation of Cytopathological Reports for Cervical Cancer Detection and His Utility in Training for Health Personnel

Health information systems and training are tools that support process management. The current study describes the results of the implementation of technological innovation in the process of the capture and preparation of cytopathological reports. The electronic system was structured based on national standards regarding cervical cancer control. PHP was used to design the software and MYSQL was used for the structure of the database. The total number of health personnel assigned to the cytology department participated, along with a pathologist, who made the records of the patients who came for cervical cytology to a university health center in San Luis Potosi, Mexico. The system was evaluated based on the indicators of structure, process, and results. Structure: comply with the official Mexican regulations for the registration of cervical cancer and electronic health information systems. Process: all records were legible and accurate, with varying percentages of completeness in the patient identification sections (46%) and alternate contact data (80%). Result: percentages above 80% were obtained in the satisfaction of the professionals who used the system. The system was effective as it yielded readable and accurate data that made the process of information capture and delivery of cervical screening results more efficient and faster.

A Comprehensive Systematic Review of Data Linkage Publications on Diabetes in Australia

Aims

Our study aimed to identify the common themes, knowledge gaps and to evaluate the quality of data linkage research on diabetes in Australia.

Methods

This systematic review was developed in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (the PRISMA Statement). Six biomedical databases and the Australian Population Health Research Network (PHRN) website were searched. A narrative synthesis was conducted to comprehensively identify the common themes and knowledge gaps. The guidelines for studies involving data linkage were used to appraise methodological quality of included studies.

Results

After screening and hand-searching, 118 studies were included in the final analysis. Data linkage publications confirmed negative health outcomes in people with diabetes, reported risk factors for diabetes and its complications, and found an inverse association between primary care use and hospitalization. Linked data were used to validate data sources and diabetes instruments. There were limited publications investigating healthcare expenditure and adverse drug reactions (ADRs) in people with diabetes. Regarding methodological assessment, important information about the linkage performed was under-reported in included studies.

Conclusions

In the future, more up to date data linkage research addressing costs of diabetes and its complications in a contemporary Australian setting, as well as research assessing ADRs of recently approved antidiabetic medications, are required.

Data linkage for crash outcome assessment: Linking police-reported crashes, emergency response data, and trauma registry records

INTRODUCTION

Traffic crash reports lack detailed information about emergency medical service (EMS) responses, the injuries, and the associated treatments, limiting the ability of safety analysts to account for that information. Integrating data from other sources can enable a better understanding of characteristics of serious crashes and further explain variance in injury outcomes. In this research, an approach is proposed and implemented to link crash data to EMS run data, patient care reports, and trauma registry data.

METHOD

A heuristic framework is developed to match EMS run reports to crashes through time, location, and other indicators present in both datasets. Types of matches between EMS and crashes were classified. To investigate the fidelity of the match approach, a manual review of a sample of data was conducted. A comparative bias analysis was implemented on several key variables.

RESULTS

72.2% of EMS run reports matched to a crash record and 69.3% of trauma registry records matched with a crash record. Females, individuals between 11 and 20 years old, and individuals involved in single vehicle or head on crashes were more likely to be present in linked data sets. Using the linked data sets, relationships between EMS response time and reported injury in the crash report, and between police-reported injury and injury severity score were examined.

CONCLUSION

Linking data from other sources can greatly enhance the information available to address road safety issues, data quality issues, and more. Linking data has the potential to result in biases that must be investigated as they relate to the use-case for the data.

PRACTICAL IMPLICATIONS

This research resulted in a transferable heuristic approach that can be used to link data sets that are commonly collected by agencies across the world. It also provides guidance on how to check the linked data for biases and errors.

Optimizing the Retrieval of the Vital Status of Cancer Patients for Health Data Warehouses by Using Open Government Data in France

Electronic Medical Records (EMR) and Electronic Health Records (EHR) are often missing critical information about the death of a patient, although it is an essential metric for medical research in oncology to assess survival outcomes, particularly for evaluating the efficacy of new therapeutic approaches. We used open government data in France from 1970 to September 2021 to identify deceased patients and match them with patient data collected from the Institut de Cancérologie de l’Ouest (ICO) data warehouse (Integrated Center of Oncology—the third largest cancer center in France) between January 2015 and November 2021. To meet our objective, we evaluated algorithms to perform a deterministic record linkage: an exact matching algorithm and a fuzzy matching algorithm. Because we lacked reference data, we needed to assess the algorithms by estimating the number of homonyms that could lead to false links, using the same open dataset of deceased persons in France. The exact matching algorithm allowed us to double the number of dates of death in the ICO data warehouse, and the fuzzy matching algorithm tripled it. Studying homonyms assured us that there was a low risk of misidentification, with precision values of 99.96% for the exact matching and 99.68% for the fuzzy matching. However, estimating the number of false negatives proved more difficult than anticipated. Nevertheless, using open government data can be a highly interesting way to improve the completeness of the date of death variable for oncology patients in data warehouses

Linking education and hospital data in England: linkage process and quality

INTRODUCTION

Linkage of administrative data for universal state education and National Health Service (NHS) hospital care would enable research into the inter-relationships between education and health for all children in England.

OBJECTIVES

We aim to describe the linkage process and evaluate the quality of linkage of four one-year birth cohorts within the National Pupil Database (NPD) and Hospital Episode Statistics (HES).

METHODS

We used multi-step deterministic linkage algorithms to link longitudinal records from state schools to the chronology of records in the NHS Personal Demographics Service (PDS; linkage stage 1), and HES (linkage stage 2). We calculated linkage rates and compared pupil characteristics in linked and unlinked samples for each stage of linkage and each cohort (1990/91, 1996/97, 1999/00, and 2004/05).

RESULTS

Of the 2,287,671 pupil records, 2,174,601 (95%) linked to HES. Linkage rates improved over time (92% in 1990/91 to 99% in 2004/05). Ethnic minority pupils and those living in more deprived areas were less likely to be matched to hospital records, but differences in pupil characteristics between linked and unlinked samples were moderate to small.

CONCLUSION

We linked nearly all pupils to at least one hospital record. The high coverage of the linkage represents a unique opportunity for wide-scale analyses across the domains of health and education. However, missed links disproportionately affected ethnic minorities or those living in the poorest neighbourhoods: selection bias could be mitigated by increasing the quality and completeness of identifiers recorded in administrative data or the application of statistical methods that account for missed links.

HIGHLIGHTS

Longitudinal administrative records for all children attending state school and acute hospital services in England have been used for research for more than two decades, but lack of a shared unique identifier has limited scope for linkage between these databases.We applied multi-step deterministic linkage algorithms to 4 one-year cohorts of children born 1 September-31 August in 1990/91, 1996/97, 1999/00 and 2004/05. In stage 1, full names, date of birth, and postcode histories from education data in the National Pupil Database were linked to the NHS Personal Demographic Service. In stage 2, NHS number, postcode, date of birth and sex were linked to hospital records in Hospital Episode Statistics.Between 92% and 99% of school pupils linked to at least one hospital record. Ethnic minority pupils and pupils who were living in the most deprived areas were least likely to link. Ethnic minority pupils were less likely than white children to link at the first step in both algorithms.Bias due to linkage errors could lead to an underestimate of the health needs in disadvantaged groups. Improved data quality, more sensitive linkage algorithms, and/or statistical methods that account for missed links in analyses, should be considered to reduce linkage bias.

Validation of ethnicity in administrative hospital data in women giving birth in England: cohort study

OBJECTIVE

To describe the accuracy of coding of ethnicity in National Health Service (NHS) administrative hospital records compared with self-declared records in maternity booking systems, and to assess the potential impact of misclassification bias.

DESIGN

Secondary analysis of data from records of women giving birth in England (2015-2017).

SETTING

NHS Trusts in England participating in a national audit programme.

PARTICIPANTS

1 237 213 women who gave birth between 1 April 2015 and 31 March 2017. PRIMARY AND SECONDARY OUTCOME MEASURES: (1) Proportion of women with complete ethnicity; (2) agreement on coded ethnicity between maternity (maternity information systems (MIS)) and administrative hospital (Hospital Episode Statistics (HES)) records; (3) rates of caesarean section and obstetric anal sphincter injury by ethnic group in MIS and HES.

RESULTS

91.3% of women had complete information regarding ethnicity in HES. Overall agreement between data sets was 90.4% (κ=0.83); 94.4% when collapsed into aggregate groups of white/South Asian/black/mixed/other (κ=0.86). Most disagreement was seen in women coded as mixed in either data set. Rates of obstetrical events and complications by ethnicity were similar regardless of data set used, with the most differences seen in women coded as mixed.

CONCLUSIONS

Levels of accuracy in ethnicity coding in administrative hospital records support the use of ethnicity collapsed into groups (white/South Asian/black/mixed/other), but findings for mixed and other groups, and more granular classifications, should be treated with caution. Robustness of results of analyses for associations with ethnicity can be improved by using additional primary data sources.

Probabilistic linkage without personal information successfully linked national clinical datasets

Background

Probabilistic linkage can link patients from different clinical databases without the need for personal information. If accurate linkage can be achieved, it would accelerate the use of linked datasets to address important clinical and public health questions.

Objective

We developed a step-by-step process for probabilistic linkage of national clinical and administrative datasets without personal information, and validated it against deterministic linkage using patient identifiers.

Study Design and Setting

We used electronic health records from the National Bowel Cancer Audit and Hospital Episode Statistics databases for 10,566 bowel cancer patients undergoing emergency surgery in the English National Health Service.

Results

Probabilistic linkage linked 81.4% of National Bowel Cancer Audit records to Hospital Episode Statistics, vs. 82.8% using deterministic linkage. No systematic differences were seen between patients that were and were not linked, and regression models for mortality and length of hospital stay according to patient and tumour characteristics were not sensitive to the linkage approach.

Conclusion

Probabilistic linkage was successful in linking national clinical and administrative datasets for patients undergoing a major surgical procedure. It allows analysts outside highly secure data environments to undertake linkage while minimizing costs and delays, protecting data security, and maintaining linkage quality.

Treatment effects in randomised trials using routinely collected data for outcome assessment versus traditional trials: meta-research study

OBJECTIVE

To compare effect estimates of randomised clinical trials that use routinely collected data (RCD-RCT) for outcome ascertainment with traditional trials not using routinely collected data.

DESIGN

Meta-research study.

DATA SOURCE

Studies included in the same meta-analysis in a Cochrane review.

ELIGIBILITY CRITERIA FOR STUDY SELECTION

Randomised clinical trials using any type of routinely collected data for outcome ascertainment, including from registries, electronic health records, and administrative databases, that were included in a meta-analysis of a Cochrane review on any clinical question and any health outcome together with traditional trials not using routinely collected data for outcome measurement.

REVIEW METHODS

Effect estimates from trials using or not using routinely collected data were summarised in random effects meta-analyses. Agreement of (summary) treatment effect estimates from trials using routinely collected data and those not using such data was expressed as the ratio of odds ratios. Subgroup analyses explored effects in trials based on different types of routinely collected data. Two investigators independently assessed the quality of each data source.

RESULTS

84 RCD-RCTs and 463 traditional trials on 22 clinical questions were included. Trials using routinely collected data for outcome ascertainment showed 20% less favourable treatment effect estimates than traditional trials (ratio of odds ratios 0.80, 95% confidence interval 0.70 to 0.91, I2=14%). Results were similar across various types of outcomes (mortality outcomes: 0.92, 0.74 to 1.15, I2=12%; non-mortality outcomes: 0.71, 0.60 to 0.84, I2=8%), data sources (electronic health records: 0.81, 0.59 to 1.11, I2=28%; registries: 0.86, 0.75 to 0.99, I2=20%; administrative data: 0.84, 0.72 to 0.99, I2=0%), and data quality (high data quality: 0.82, 0.72 to 0.93, I2=0%).

CONCLUSIONS

Randomised clinical trials using routinely collected data for outcome ascertainment show smaller treatment benefits than traditional trials not using routinely collected data. These differences could have implications for healthcare decision making and the application of real world evidence.

CMIID: A comprehensive medical information identifier for clinical search harmonization in Data Safe Havens

Over the last decades clinical research has been driven by informatics changes nourished by distinct research endeavors. Inherent to this evolution, several issues have been the focus of a variety of studies: multi-location patient data access, interoperability between terminological and classification systems and clinical practice and records harmonization. Having these problems in mind, the Data Safe Haven paradigm emerged to promote a newborn architecture, better reasoning and safe and easy access to distinct Clinical Data Repositories. This study aim is to present a novel solution for clinical search harmonization within a safe environment, making use of a hybrid coding taxonomy that enables researchers to collect information from multiple repositories based on a clinical domain query definition. Results show that is possible to query multiple repositories using a single query definition based on clinical domains and the capabilities of the Unified Medical Language System, although it leads to deterioration of the framework response times. Participants of a Focus Group and a System Usability Scale questionnaire rated the framework with a median value of 72.5, indicating the hybrid coding taxonomy could be enriched with additional metadata to further improve the refinement of the results and enable the possibility of using this system as data quality tagging mechanism.

Assessing data linkage quality in cohort studies

Background: Linkage of administrative data sources provides an efficient means of collecting detailed data on how individuals interact with cross-sectoral services, society, and the environment. These data can be used to supplement conventional cohort studies, or to create population-level electronic cohorts generated solely from administrative data. However, errors occurring during linkage (false matches/missed matches) can lead to bias in results from linked data.

Aim: This paper provides guidance on evaluating linkage quality in cohort studies.

Methods: We provide an overview of methods for linkage, describe mechanisms by which linkage error can introduce bias, and draw on real-world examples to demonstrate methods for evaluating linkage quality.

Results: Methods for evaluating linkage quality described in this paper provide guidance on (i) estimating linkage error rates, (ii) understanding the mechanisms by which linkage error might bias results, and (iii) information that should be shared between data providers, linkers and users, so that approaches to handling linkage error in analysis can be implemented.

Conclusion: Linked administrative data can enhance conventional cohorts and offers the ability to answer questions that require large sample sizes or hard-to-reach populations. Care needs to be taken to evaluate linkage quality in order to provide robust results.

Fold-stratified cross-validation for unbiased and privacy-preserving federated learning

OBJECTIVE

We introduce fold-stratified cross-validation, a validation methodology that is compatible with privacy-preserving federated learning and that prevents data leakage caused by duplicates of electronic health records (EHRs).

MATERIALS AND METHODS

Fold-stratified cross-validation complements cross-validation with an initial stratification of EHRs in folds containing patients with similar characteristics, thus ensuring that duplicates of a record are jointly present either in training or in validation folds. Monte Carlo simulations are performed to investigate the properties of fold-stratified cross-validation in the case of a model data analysis using both synthetic data and MIMIC-III (Medical Information Mart for Intensive Care-III) medical records.

RESULTS

In situations in which duplicated EHRs could induce overoptimistic estimations of accuracy, applying fold-stratified cross-validation prevented this bias, while not requiring full deduplication. However, a pessimistic bias might appear if the covariate used for the stratification was strongly associated with the outcome.

DISCUSSION

Although fold-stratified cross-validation presents low computational overhead, to be efficient it requires the preliminary identification of a covariate that is both shared by duplicated records and weakly associated with the outcome. When available, the hash of a personal identifier or a patient's date of birth provides such a covariate. On the contrary, pseudonymization interferes with fold-stratified cross-validation, as it may break the equality of the stratifying covariate among duplicates.

CONCLUSION

Fold-stratified cross-validation is an easy-to-implement methodology that prevents data leakage when a model is trained on distributed EHRs that contain duplicates, while preserving privacy.

Infant formula composition and educational performance: a protocol to extend follow-up for a set of randomised controlled trials using linked administrative education records

INTRODUCTION

The effect of infant nutrition on long-term cognition is important for parents and policy makers. However, most clinical trials typically have short follow-up periods, when measures of cognition are poorly predictive of later function. The few trials with longer-term follow-up have high levels of attrition, which can lead to selection bias, and in turn to erroneous interpretation of long-term harms and benefits of infant nutrition. We address the need for unbiased, long-term follow-up, by linking measures of educational performance from administrative education records. Educational performance is a meaningful marker of cognitive function in children and it is strongly correlated with IQ. We aim to evaluate educational performance for children who, as infants, were part of a series of trials that randomised participants to either nutritionally modified infant formula or standard formula. Most trialists anticipated positive effects of these interventions on later cognitive function.

METHODS AND ANALYSIS

Using data from 1923 participants of seven randomised infant formula trials linked to the English National Pupil Database (NPD), this study will provide new insights into the effect of nutrient intake in infancy on school achievement. Our primary outcome will be the mean differences in z-scores between intervention and control groups for a compulsory Mathematics exam sat at age 16. Secondary outcomes will be z-scores for a compulsory English exam at age 16 and z-scores for compulsory Mathematics and English exams at age 11. We will also evaluate intervention effects on the likelihood of receiving special educational needs (SEN) support. All analyses will be performed separately by trial.

ETHICS AND DISSEMINATION

Research ethics approval, and approval from the Health Research Authority Confidentiality Advisory Group, has been obtained for this study. The results of this study will be disseminated to scientific, practitioner, and lay audiences, submitted for publication in peer-reviewed journals, and will contribute towards a PhD dissertation.

Sources of potential bias when combining routine data linkage and a national survey of secondary school-aged children: a record linkage study

BACKGROUND

Linking survey data to administrative records requires informed participant consent. When linkage includes child data, this includes parental and child consent. Little is known of the potential impacts of introducing consent to data linkage on response rates and biases in school-based surveys. This paper assessed: i) the impact on overall parental consent rates and sample representativeness when consent for linkage was introduced and ii) the quality of identifiable data provided to facilitate linkage.

METHODS

Including an option for data linkage was piloted in a sub-sample of schools participating in the Student Health and Wellbeing survey, a national survey of adolescents in Wales, UK. Schools agreeing to participate were randomized 2:1 to receive versus not receive the data linkage question. Survey responses from consenting students were anonymised and linked to routine datasets (e.g. general practice, inpatient, and outpatient records). Parental withdrawal rates were calculated for linkage and non-linkage samples. Multilevel logistic regression models were used to compare characteristics between: i) consenters and non-consenters; ii) successfully and unsuccessfully linked students; and iii) the linked cohort and peers within the general population, with additional comparisons of mental health diagnoses and health service contacts.

RESULTS

The sub-sample comprised 64 eligible schools (out of 193), with data linkage piloted in 39. Parental consent was comparable across linkage and non-linkage schools. 48.7% (n = 9232) of students consented to data linkage. Modelling showed these students were more likely to be younger, more affluent, have higher positive mental wellbeing, and report fewer risk-related behaviours compared to non-consenters. Overall, 69.8% of consenting students were successfully linked, with higher rates of success among younger students. The linked cohort had lower rates of mental health diagnoses (5.8% vs. 8.8%) and specialist contacts (5.2% vs. 7.7%) than general population peers.

CONCLUSIONS

Introducing data linkage within a national survey of adolescents had no impact on study completion rates. However, students consenting to data linkage, and those successfully linked, differed from non-consenting students on several key characteristics, raising questions concerning the representativeness of linked cohorts. Further research is needed to better understand decision-making processes around providing consent to data linkage in adolescent populations.

Reflection on modern methods: five myths about measurement error in epidemiological research

Epidemiologists are often confronted with datasets to analyse which contain measurement error due to, for instance, mistaken data entries, inaccurate recordings and measurement instrument or procedural errors. If the effect of measurement error is misjudged, the data analyses are hampered and the validity of the study's inferences may be affected. In this paper, we describe five myths that contribute to misjudgments about measurement error, regarding expected structure, impact and solutions to mitigate the problems resulting from mismeasurements. The aim is to clarify these measurement error misconceptions. We show that the influence of measurement error in an epidemiological data analysis can play out in ways that go beyond simple heuristics, such as heuristics about whether or not to expect attenuation of the effect estimates. Whereas we encourage epidemiologists to deliberate about the structure and potential impact of measurement error in their analyses, we also recommend exercising restraint when making claims about the magnitude or even direction of effect of measurement error if not accompanied by statistical measurement error corrections or quantitative bias analysis. Suggestions for alleviating the problems or investigating the structure and magnitude of measurement error are given.

Cohort Profile: The Dutch Perined-Lifelines birth cohort

BACKGROUND

Maternal nutrition status (e.g. dietary/nutrient intake) during pregnancy has been associated with pregnancy outcomes including birth weight, infant survival and metabolic health of the offspring during later life. During the past few years, maternal dietary intake, at least three months before conception, has been shown to affect pregnancy outcomes also. However, literature investigating this link is still scarce. The studies that have looked at preconception dietary intake in relation to pregnancy outcome were either animal studies, had small sample sizes or focused on only selected macronutrient intake rather than complete (macro)nutrient composition or dietary intakes (e.g. food groups). Therefore, we aim to investigate the association between preconception diet and pregnancy outcomes in a linked birth cohort. The main objective of this manuscript is to describe the methodology of establishing this birth cohort and to describe both the characteristics of the study population included as well as the representativeness in terms of dietary intake.

METHODS

We created the birth cohort by linking two existing databases; a large population-based cohort study in the Netherlands (The Lifelines Cohort study) and the Dutch national birth registry (Perined), through a 'trusted third party'. The birth cohort contains information on maternal dietary intake during preconception as well as pregnancy outcomes.

RESULTS AND DISCUSSION

In the Lifelines Cohort study, 3,418 pregnancies were available for linking with Perined. In total, 2,368 pregnancies (86.9%) were linked with Perined, resulting in the birth cohort. With this linked cohort we are able to provide insights on the associations between dietary intake before conception and pregnancy outcomes. Such data could potentially improve nutritional care for women of childbearing age. Lifestyle changes in the period preceding pregnancy may be most effective in improving pregnancy outcomes. A focus on this window of opportunity may provide both sufficient time, as well as a period when women are potentially motivated to adopt health optimizing behaviours.

GUILD: GUidance for Information about Linking Data sets

Record linkage of administrative and survey data is increasingly used to generate evidence to inform policy and services. Although a powerful and efficient way of generating new information from existing data sets, errors related to data processing before, during and after linkage can bias results. However, researchers and users of linked data rarely have access to information that can be used to assess these biases or take them into account in analyses. As linked administrative data are increasingly used to provide evidence to guide policy and services, linkage error, which disproportionately affects disadvantaged groups, can undermine evidence for public health. We convened a group of researchers and experts from government data providers to develop guidance about the information that needs to be made available about the data linkage process, by data providers, data linkers, analysts and the researchers who write reports. The guidance goes beyond recommendations for information to be included in research reports. Our aim is to raise awareness of information that may be required at each step of the linkage pathway to improve the transparency, reproducibility, and accuracy of linkage processes, and the validity of analyses and interpretation of results.

Maternal and perinatal outcomes by planned place of birth in Australia 2000 - 2012: a linked population data study

OBJECTIVE

To compare perinatal and maternal outcomes for Australian women with uncomplicated pregnancies according to planned place of birth, that is, in hospital labour wards, birth centres or at home.

DESIGN

A population-based retrospective design, linking and analysing routinely collected electronic data. Analysis comprised χ² tests and binary logistic regression for categorical data, yielding adjusted ORs. Continuous data were analysed using analysis of variance.

SETTING

All eight Australian states and territories.

PARTICIPANTS

Women with uncomplicated pregnancies who gave birth between 2000 and 2012 to a singleton baby in cephalic presentation at between 37 and 41 completed weeks' gestation. Of the 1 251 420 births, 1 171 703 (93.6%) were planned in hospital labour wards, 71 505 (5.7%) in birth centres and 8212 (0.7%) at home.

MAIN OUTCOME MEASURES

Mode of birth, normal labour and birth, interventions and procedures during labour and birth, maternal complications, admission to special care/high dependency or intensive care units (mother or infant) and perinatal mortality (intrapartum stillbirth and neonatal death).

RESULTS

Compared with planned hospital births, the odds of normal labour and birth were over twice as high in planned birth centre births (adjusted OR (AOR) 2.72; 99% CI 2.63 to 2.81) and nearly six times as high in planned home births (AOR 5.91; 99% CI 5.15 to 6.78). There were no statistically significant differences in the proportion of intrapartum stillbirths, early or late neonatal deaths between the three planned places of birth.

CONCLUSIONS

This is the first Australia-wide study to examine outcomes by planned place of birth. For healthy women in Australia having an uncomplicated pregnancy, planned births in birth centres or at home are associated with positive maternal outcomes although the number of homebirths was small overall. There were no significant differences in the perinatal mortality rate, although the absolute numbers of deaths were very small and therefore firm conclusions cannot be drawn about perinatal mortality outcomes.

Using deterministic record linkage to link ambulance and emergency department data: is it possible without patient identifiers? A case study from the UK

INTRODUCTION

Routine linkage of emergency ambulance records with those from the emergency department is uncommon in the UK. Our study, known as the Pre-Hospital Emergency Department Data Linking Project (PHED Data), aimed to link records of all patients conveyed by a single emergency ambulance service to thirteen emergency departments in the UK from 2012-2016.

OBJECTIVES

We aimed to examine the feasibility and resource requirements of collecting de-identified emergency department patient record data and, using a deterministic matching algorithm, linking it to ambulance service data.

METHODS

We used a learning log to record contacts and activities undertaken by the research team to achieve data linkage. We also conducted semi-structured interviews with information management/governance staff involved in the process.

RESULTS

We found that five steps were required for successful data linkage for each hospital trust. The total time taken to achieve linkage was a mean of 65 weeks. A total of 958,057 emergency department records were obtained and, of these, 81% were linked to a corresponding ambulance record. The match rate varied between hospital trusts (50%-94%). Staff expressed strong enthusiasm for data linkage. Barriers to successful linkage were mainly due to inconsistencies between and within acute trusts in the recording of two ambulance event identifiers (CAD and call sign). Further data cleaning was required on emergency department fields before full analysis could be conducted. Ensuring the data was not re-identifiable limited validation of the matching method.

CONCLUSION

We conclude that deterministic record linkage based on the combination of two event identifiers (CAD and call sign) is possible. There is an appetite for data linkage in healthcare organisations but it is a slow process. Developments in standardising the recording of emergency department data are likely to improve the quality of the resultant linked dataset. This would further increase its value for providing evidence to support improvements in health care delivery.

HIGHLIGHTS

Ambulance records are rarely linked to other datasets; this study looks at the feasibility and resource requirement to use deterministic matching to link ambulance and emergency department data for patients conveyed by ambulance to the emergency department.It is possible to link these data, with an average match rate of 81% across 13 emergency departments and one large ambulance trust.All trusts approached provided match-able data and there was an appetite for data linkage; however, it was a long process taking an average of 65 weeks.We conclude that deterministic matching using no patient identifiers can be used in this setting.

Applying an Electronic Health Records Data Quality Framework Across Service Sectors: A Case Study of Juvenile Justice System Data

Context

Integrating electronic health records (EHR) with other sources of administrative data is key to identifying factors affecting the long-term health of traditionally underserved populations, such as individuals involved in the justice system. Linking existing administrative data from multiple sources overcomes many of the limitations of traditional prospective studies of population health, but the linking process assumes high levels of data quality and consistency within administrative data. Studies of EHR, unlike other types of administrative data, have provided guidance to evaluate the utility of big data for population health research.

Case Description

Here, an established EHR data quality framework was applied to identify and describe the potential shortcomings of administrative juvenile justice system data collected by one of four case management systems (CMSs) across 12 counties in a Midwest state. The CMS data were reviewed for logical inconsistencies and compared along the data quality dimensions of plausibility and completeness.

Major Themes

After applying the data quality framework, several patterns of logical inconsistencies within the data were identified. To resolve these inconsistencies, recommendations regarding data entry, review, and extraction are offered.

Conclusion

The recommendations related to achieving quality justice system data can be applied to future efforts to link administrative databases from multiple sources. Increasing trust in administrative data quality related to vulnerable populations ultimately improves knowledge of pressing public health concerns.

Our data, our society, our health: A vision for inclusive and transparent health data science in the United Kingdom and beyond

The last 6 years have seen sustained investment in health data science in the United Kingdom and beyond, which should result in a data science community that is inclusive of all stakeholders, working together to use data to benefit society through the improvement of public health and well-being. However, opportunities made possible through the innovative use of data are still not being fully realised, resulting in research inefficiencies and avoidable health harms. In this paper, we identify the most important barriers to achieving higher productivity in health data science. We then draw on previous research, domain expertise, and theory to outline how to go about overcoming these barriers, applying our core values of inclusivity and transparency. We believe a step change can be achieved through meaningful stakeholder involvement at every stage of research planning, design, and execution and team-based data science, as well as harnessing novel and secure data technologies. Applying these values to health data science will safeguard a social licence for health data research and ensure transparent and secure data usage for public benefit.

Creating the first national linked dataset on perinatal and maternal outcomes in Australia: Methods and challenges

BACKGROUND

Data linkage offers a powerful mechanism for examining healthcare outcomes across populations and can generate substantial robust datasets using routinely collected electronic data. However, it presents methodological challenges, especially in Australia where eight separate states and territories maintain health datasets. This study used linked data to investigate perinatal and maternal outcomes in relation to place of birth. It examined data from all eight jurisdictions regarding births planned in hospitals, birth centres and at home. Data linkage enabled the first Australia-wide dataset on birth outcomes. However, jurisdictional differences in data collection created challenges in obtaining comparable cohorts of women with similar low-risk pregnancies in all birth settings. The objective of this paper is to describe the techniques for managing previously linked data, and specifically for ensuring the resulting dataset contained only low-risk pregnancies.

METHODS

This paper indicates the procedures for preparing and merging linked perinatal, inpatient and mortality data from different sources, providing technical guidance to address challenges arising in linked data study designs.

RESULTS

We combined data from eight jurisdictions linking four collections of administrative healthcare and civil registration data. The merging process ensured that variables were consistent, compatible and relevant to study aims. To generate comparable cohorts for all three birth settings, we developed increasingly complex strategies to ensure that the dataset eliminated women with pregnancies at risk of complications during labour and birth. It was then possible to compare birth outcomes for comparable samples, enabling specific examination of the impact of birth setting on maternal and infant safety across Australia.

CONCLUSIONS

Data linkage is a valuable resource to enhance knowledge about birth outcomes from different settings, notwithstanding methodological challenges. Researchers can develop and share practical techniques to address these challenges. Study findings suggest that jurisdictions develop more consistent data collections to facilitate future data linkage.

Priorities to Overcome Barriers Impacting Data Science Application in Emergency Care Research

For a variety of reasons including cheap computing, widespread adoption of electronic medical records, digitalization of imaging and biosignals, and rapid development of novel technologies, the amount of health care data being collected, recorded, and stored is increasing at an exponential rate. Yet despite these advances, methods for the valid, efficient, and ethical utilization of these data remain underdeveloped. Emergency care research, in particular, poses several unique challenges in this rapidly evolving field. A group of content experts was recently convened to identify research priorities related to barriers to the application of data science to emergency care research. These recommendations included: 1) developing methods for cross-platform identification and linkage of patients; 2) creating central, deidentified, open-access databases; 3) improving methodologies for visualization and analysis of intensively sampled data; 4) developing methods to identify and standardize electronic medical record data quality; 5) improving and utilizing natural language processing; 6) developing and utilizing syndrome or complaint-based based taxonomies of disease; 7) developing practical and ethical framework to leverage electronic systems for controlled trials; 8) exploring technologies to help enable clinical trials in the emergency setting; and 9) training emergency care clinicians in data science and data scientists in emergency care medicine. The background, rationale, and conclusions of these recommendations are included in the present article.

Impact of linkage quality on inferences drawn from analyses using data with high rates of linkage errors in rural Tanzania

BACKGROUND

Studies based on high-quality linked data in developed countries show that even minor linkage errors, which occur when records of two different individuals are erroneously linked or when records belonging to the same individual are not linked, can impact bias and precision of subsequent analyses. We evaluated the impact of linkage quality on inferences drawn from analyses using data with substantial linkage errors in rural Tanzania.

METHODS

Semi-automatic point-of-contact interactive record linkage was used to establish gold standard links between community-based HIV surveillance data and medical records at clinics serving the surveillance population. Automated probabilistic record linkage was used to create analytic datasets at minimum, low, medium, and high match score thresholds. Cox proportional hazards regression models were used to compare HIV care registration rates by testing modality (sero-survey vs. clinic) in each analytic dataset. We assessed linkage quality using three approaches: quantifying linkage errors, comparing characteristics between linked and unlinked data, and evaluating bias and precision of regression estimates.

RESULTS

Between 2014 and 2017, 405 individuals with gold standard links were newly diagnosed with HIV in sero-surveys (n = 263) and clinics (n = 142). Automated probabilistic linkage correctly identified 233 individuals (positive predictive value [PPV] = 65%) at the low threshold and 95 individuals (PPV = 90%) at the high threshold. Significant differences were found between linked and unlinked records in primary exposure and outcome variables and for adjusting covariates at every threshold. As expected, differences attenuated with increasing threshold. Testing modality was significantly associated with time to registration in the gold standard data (adjusted hazard ratio [HR] 4.98 for clinic-based testing, 95% confidence interval [CI] 3.34, 7.42). Increasing false matches weakened the association (HR 2.76 at minimum match score threshold, 95% CI 1.73, 4.41). Increasing missed matches (i.e., increasing match score threshold and positive predictive value of the linkage algorithm) was strongly correlated with a reduction in the precision of coefficient estimate (R2 = 0.97; p = 0.03).

CONCLUSIONS

Similar to studies with more negligible levels of linkage errors, false matches in this setting reduced the magnitude of the association; missed matches reduced precision. Adjusting for these biases could provide more robust results using data with considerable linkage errors.

A guide to evaluating linkage quality for the analysis of linked data

Linked datasets are an important resource for epidemiological and clinical studies, but linkage error can lead to biased results. For data security reasons, linkage of personal identifiers is often performed by a third party, making it difficult for researchers to assess the quality of the linked dataset in the context of specific research questions. This is compounded by a lack of guidance on how to determine the potential impact of linkage error. We describe how linkage quality can be evaluated and provide widely applicable guidance for both data providers and researchers. Using an illustrative example of a linked dataset of maternal and baby hospital records, we demonstrate three approaches for evaluating linkage quality: applying the linkage algorithm to a subset of gold standard data to quantify linkage error; comparing characteristics of linked and unlinked data to identify potential sources of bias; and evaluating the sensitivity of results to changes in the linkage procedure. These approaches can inform our understanding of the potential impact of linkage error and provide an opportunity to select the most appropriate linkage procedure for a specific analysis. Evaluating linkage quality in this way will improve the quality and transparency of epidemiological and clinical research using linked data.

Do GPs accurately record date of death? A UK observational analysis

OBJECTIVE

To examine the concordance between dates of death recorded in UK primary care and national mortality records.

METHODS

UK primary care data from the Clinical Practice Research Datalink were linked to Office for National Statistics (ONS) data, for 118 571 patients who died between September 2010 and September 2015. Logistic regression was used to examine factors associated with discrepancy in death dates between data sets.

RESULTS

Death dates matched in 76.8% of cases with primary care dates preceding ONS date in 2.9%, and following in 20.3% of cases; 92.2% of cases differed by <2 weeks. Primary care date was >4 weeks later than ONS in 1.5% of cases and occurred more frequently with deaths categorised as 'external' (15.8% vs 0.8% for cancer), and in younger patients (15.9% vs 1% for 18-29 and 80-89 years, respectively). General practices with the greatest discrepancies (97.5th percentile) had around 200 times higher odds of recording substantially discordant dates than practices with the lowest discrepancies (2.5th percentile).

CONCLUSION

Dates of death in primary care records often disagree with national records and should be treated with caution. There is marked variation between practices, and studies involving young patients, unexplained deaths and where precise date of death is important are particularly vulnerable to these issues.

Big data: Some statistical issues

A broad review is given of the impact of big data on various aspects of investigation. There is some but not total emphasis on issues in epidemiological research.

Demystifying probabilistic linkage: Common myths and misconceptions

Many of the distinctions made between probabilistic and deterministic linkage are misleading. While these two approaches to record linkage operate in different ways and can produce different outputs, the distinctions between them are more a result of how they are implemented than because of any intrinsic differences. In the way they are generally applied, probabilistic and deterministic procedures can be little more than alternative means to similar ends-or they can arrive at very different ends depending on choices that are made during implementation. Misconceptions about probabilistic linkage contribute to reluctance for implementing it and mistrust of its outputs. We aim to explain how the outputs of either approach can be tailored to suit the intended application, but also to highlight the ways in which probabilistic linkage is generally more flexible, more powerful and more informed by the data. This is accomplished by examining common misconceptions about probabilistic linkage and its difference from deterministic linkage, highlighting the potential impact of design choices on the outputs of either approach. We hope that better understanding of linkage designs will help to allay concerns about probabilistic linkage, and help data linkers to select and tailor procedures to produce outputs that are appropriate for their intended use.

Challenges in administrative data linkage for research

Linkage of population-based administrative data is a valuable tool for combining detailed individual-level information from different sources for research. While not a substitute for classical studies based on primary data collection, analyses of linked administrative data can answer questions that require large sample sizes or detailed data on hard-to-reach populations, and generate evidence with a high level of external validity and applicability for policy making. There are unique challenges in the appropriate research use of linked administrative data, for example with respect to bias from linkage errors where records cannot be linked or are linked together incorrectly. For confidentiality and other reasons, the separation of data linkage processes and analysis of linked data is generally regarded as best practice. However, the 'black box' of data linkage can make it difficult for researchers to judge the reliability of the resulting linked data for their required purposes. This article aims to provide an overview of challenges in linking administrative data for research. We aim to increase understanding of the implications of (i) the data linkage environment and privacy preservation; (ii) the linkage process itself (including data preparation, and deterministic and probabilistic linkage methods) and (iii) linkage quality and potential bias in linked data. We draw on examples from a number of countries to illustrate a range of approaches for data linkage in different contexts.

Heart failure following cancer treatment: characteristics, survival and mortality of a linked health data analysis

BACKGROUND

Cardiotoxicity resulting in heart failure is a devastating complication of cancer therapy. A patient may survive cancer only to develop heart failure (HF), which has a higher mortality rate than some cancers.

AIM

This study aimed to describe the characteristics and outcomes of HF in patients with blood or breast cancer after chemotherapy treatment.

METHODS

Queensland Cancer Registry, Death Registry and Hospital Administration records were linked (1996-2009). Patients were categorised as those with an index HF admission (that occurred after cancer diagnosis) and those without an index HF admission (non-HF).

RESULTS

A total of 15 987 patients was included, and 1062 (6.6%) had an index HF admission. Median age of HF patients was 67 years (interquartile range 58-75) versus 54 years (interquartile range 44-64) for non-HF patients. More men than women developed HF (48.6% vs 29.5%), and a greater proportion in the HF group had haematological cancer (83.1%) compared with breast cancer (16.9%). After covariate adjustment, HF patients had increased mortality risk compared with non-HF patients (hazard ratios 1.67 (95% confidence interval, 1.54-1.81)), and 47% of the index HF admission occurred within 1 year from cancer diagnosis and 70% within 3 years.

CONCLUSION

Cancer treatment may place patients at a greater risk of developing HF. The onset of HF occurred soon after chemotherapy, and those who developed HF had a greater mortality risk.

A prospective cohort study of hospital separations among people who inject drugs in Australia: 2008-2013

OBJECTIVES

Injecting drug use is a persistent behaviour that increases the risk of morbidities and mortality. We assessed the burden of hospital separations among people who inject drugs (PWID), the excess compared to the general population and characteristics of separations associated with frequent use.

DESIGN

Prospective cohort study.

SETTING

All public and private hospitals in Victoria.

PARTICIPANTS

757 community-based PWID with hospital separations between January 2008 and June 2013 identified through record linkage, who contributed over 3729 person-years.

PRIMARY AND SECONDARY OUTCOME MEASURES

Counts, proportions and rates of hospital separations, descriptive administrative data including all diagnoses, comparison of separation rates to the general population, trend in separations and factors associated with frequent separations.

RESULTS

There were 2106 separations in the cohort. The most common principal diagnoses were related to mental and behavioural disorders (31%), but social circumstances influencing health was the most common group of diagnoses (61%) when all contributing diagnoses for each patient were considered. Separation rates were up to three times higher than in the age-matched population, and there was a 12% increase in separations every 6 months. Over a quarter (29%) of the cohort had frequent separations (defined as two or more separations in a calendar year), which were associated with mental health-related diagnoses, being discharged to locations other than a patient's residence, having a medical as opposed to surgical intervention, seasonal patterns, relationship status and gender.

CONCLUSIONS

Mental health conditions and other characteristics associated with separations and frequent separations in particular, emphasise the importance of providing referrals to harm reduction, social services and mental health services at discharge in order to reduce excess hospital separations among PWID.

Evaluation of record linkage of two large administrative databases in a middle income country: stillbirths and notifications of dengue during pregnancy in Brazil

BACKGROUND

Due to the increasing availability of individual-level information across different electronic datasets, record linkage has become an efficient and important research tool. High quality linkage is essential for producing robust results. The objective of this study was to describe the process of preparing and linking national Brazilian datasets, and to compare the accuracy of different linkage methods for assessing the risk of stillbirth due to dengue in pregnancy.

METHODS

We linked mothers and stillbirths in two routinely collected datasets from Brazil for 2009-2010: for dengue in pregnancy, notifications of infectious diseases (SINAN); for stillbirths, mortality (SIM). Since there was no unique identifier, we used probabilistic linkage based on maternal name, age and municipality. We compared two probabilistic approaches, each with two thresholds: 1) a bespoke linkage algorithm; 2) a standard linkage software widely used in Brazil (ReclinkIII), and used manual review to identify further links. Sensitivity and positive predictive value (PPV) were estimated using a subset of gold-standard data created through manual review. We examined the characteristics of false-matches and missed-matches to identify any sources of bias.

RESULTS

From records of 678,999 dengue cases and 62,373 stillbirths, the gold-standard linkage identified 191 cases. The bespoke linkage algorithm with a conservative threshold produced 131 links, with sensitivity = 64.4% (68 missed-matches) and PPV = 92.5% (8 false-matches). Manual review of uncertain links identified an additional 37 links, increasing sensitivity to 83.7%. The bespoke algorithm with a relaxed threshold identified 132 true matches (sensitivity = 69.1%), but introduced 61 false-matches (PPV = 68.4%). ReclinkIII produced lower sensitivity and PPV than the bespoke linkage algorithm. Linkage error was not associated with any recorded study variables.

CONCLUSION

Despite a lack of unique identifiers for linking mothers and stillbirths, we demonstrate a high standard of linkage of large routine databases from a middle income country. Probabilistic linkage and manual review were essential for accurately identifying cases for a case-control study, but this approach may not be feasible for larger databases or for linkage of more common outcomes.

Probabilistic linkage to enhance deterministic algorithms and reduce data linkage errors in hospital administrative data

Background

The pseudonymisation algorithm used to link together episodes of care belonging to the same patient in England [Hospital Episode Statistics ID (HESID)] has never undergone any formal evaluation to determine the extent of data linkage error.

Objective

To quantify improvements in linkage accuracy from adding probabilistic linkage to existing deterministic HESID algorithms.

Methods

Inpatient admissions to National Health Service (NHS) hospitals in England (HES) over 17 years (1998 to 2015) for a sample of patients (born 13th or 28th of months in 1992/1998/2005/2012). We compared the existing deterministic algorithm with one that included an additional probabilistic step, in relation to a reference standard created using enhanced probabilistic matching with additional clinical and demographic information. Missed and false matches were quantified and the impact on estimates of hospital readmission within one year was determined.

Results

HESID produced a high missed match rate, improving over time (8.6% in 1998 to 0.4% in 2015). Missed matches were more common for ethnic minorities, those living in areas of high socio-economic deprivation, foreign patients and those with ‘no fixed abode’. Estimates of the readmission rate were biased for several patient groups owing to missed matches, which were reduced for nearly all groups.

Conclusion

Probabilistic linkage of HES reduced missed matches and bias in estimated readmission rates, with clear implications for commissioning, service evaluation and performance monitoring of hospitals. The existing algorithm should be modified to address data linkage error, and a retrospective update of the existing data would address existing linkage errors and their implications.

Utilising identifier error variation in linkage of large administrative data sources

Background

Linkage of administrative data sources often relies on probabilistic methods using a set of common identifiers (e.g. sex, date of birth, postcode). Variation in data quality on an individual or organisational level (e.g. by hospital) can result in clustering of identifier errors, violating the assumption of independence between identifiers required for traditional probabilistic match weight estimation. This potentially introduces selection bias to the resulting linked dataset. We aimed to measure variation in identifier error rates in a large English administrative data source (Hospital Episode Statistics; HES) and to incorporate this information into match weight calculation.

Methods

We used 30,000 randomly selected HES hospital admissions records of patients aged 0–1, 5–6 and 18–19 years, for 2011/2012, linked via NHS number with data from the Personal Demographic Service (PDS; our gold-standard). We calculated identifier error rates for sex, date of birth and postcode and used multi-level logistic regression to investigate associations with individual-level attributes (age, ethnicity, and gender) and organisational variation. We then derived: i) weights incorporating dependence between identifiers; ii) attribute-specific weights (varying by age, ethnicity and gender); and iii) organisation-specific weights (by hospital). Results were compared with traditional match weights using a simulation study.

Results

Identifier errors (where values disagreed in linked HES-PDS records) or missing values were found in 0.11% of records for sex and date of birth and in 53% of records for postcode. Identifier error rates differed significantly by age, ethnicity and sex (p < 0.0005). Errors were less frequent in males, in 5–6 year olds and 18–19 year olds compared with infants, and were lowest for the Asian ethic group. A simulation study demonstrated that substantial bias was introduced into estimated readmission rates in the presence of identifier errors. Attribute- and organisational-specific weights reduced this bias compared with weights estimated using traditional probabilistic matching algorithms.

Conclusions

We provide empirical evidence on variation in rates of identifier error in a widely-used administrative data source and propose a new method for deriving match weights that incorporates additional data attributes. Our results demonstrate that incorporating information on variation by individual-level characteristics can help to reduce bias due to linkage error.

Automated linkage of patient records from disparate sources

We introduce an automated method of record linkage that has two key features, automated selection of match field interactions to include in the model for estimation and automated threshold determination for classifying record pairs to matches or non-matches. We applied our method to two real-world examples. The first example demonstrated results consistent with our earlier work: When data quality is adequate and the match field discriminating power is high, matching algorithms exhibit similar performance. The second example demonstrated that our method yields a lower false positive rate and higher positive predictive value than the Fellegi-Sunter model in the face of low data quality. When compared to the Fellegi-Sunter model, simulation studies suggest that our method exhibits better overall performance as indicated by higher area under the curve, and less biased estimates for both the match prevalence rate and the m- and u-probabilities over a range of data scenarios, especially when the match prevalence is extreme. Computationally, our method is as efficient as the Fellegi-Sunter model. We recommend this method in situations that an unsupervised linking algorithm is needed.

CATheter Infections in CHildren (CATCH): a randomised controlled trial and economic evaluation comparing impregnated and standard central venous catheters in children

BACKGROUND

Impregnated central venous catheters (CVCs) are recommended for adults to reduce bloodstream infection (BSI) but not for children.

OBJECTIVE

To determine the effectiveness of impregnated compared with standard CVCs for reducing BSI in children admitted for intensive care.

DESIGN

Multicentre randomised controlled trial, cost-effectiveness analysis from a NHS perspective and a generalisability analysis and cost impact analysis.

SETTING

14 English paediatric intensive care units (PICUs) in England.

PARTICIPANTS

Children aged < 16 years admitted to a PICU and expected to require a CVC for ≥ 3 days.

INTERVENTIONS

Heparin-bonded, antibiotic-impregnated (rifampicin and minocycline) or standard polyurethane CVCs, allocated randomly (1 : 1 : 1). The intervention was blinded to all but inserting clinicians.

MAIN OUTCOME MEASURE

Time to first BSI sampled between 48 hours after randomisation and 48 hours after CVC removal. The following data were used in the trial: trial case report forms; hospital administrative data for 6 months pre and post randomisation; and national-linked PICU audit and laboratory data.

RESULTS

In total, 1859 children were randomised, of whom 501 were randomised prospectively and 1358 were randomised as an emergency; of these, 984 subsequently provided deferred consent for follow-up. Clinical effectiveness - BSIs occurred in 3.59% (18/502) of children randomised to standard CVCs, 1.44% (7/486) of children randomised to antibiotic CVCs and 3.42% (17/497) of children randomised to heparin CVCs. Primary analyses comparing impregnated (antibiotic and heparin CVCs) with standard CVCs showed no effect of impregnated CVCs [hazard ratio (HR) 0.71, 95% confidence interval (CI) 0.37 to 1.34]. Secondary analyses showed that antibiotic CVCs were superior to standard CVCs (HR 0.43, 95% CI 0.20 to 0.96) but heparin CVCs were not (HR 1.04, 95% CI 0.53 to 2.03). Time to thrombosis, mortality by 30 days and minocycline/rifampicin resistance did not differ by CVC. Cost-effectiveness - heparin CVCs were not clinically effective and therefore were not cost-effective. The incremental cost of antibiotic CVCs compared with standard CVCs over a 6-month time horizon was £1160 (95% CI -£4743 to £6962), with an incremental cost-effectiveness ratio of £54,057 per BSI avoided. There was considerable uncertainty in costs: antibiotic CVCs had a probability of 0.35 of being dominant. Based on index hospital stay costs only, antibiotic CVCs were associated with a saving of £97,543 per BSI averted. The estimated value of health-care resources associated with each BSI was £10,975 (95% CI -£2801 to £24,751). Generalisability and cost-impact - the baseline risk of BSI in 2012 for PICUs in England was 4.58 (95% CI 4.42 to 4.74) per 1000 bed-days. An estimated 232 BSIs could have been averted in 2012 using antibiotic CVCs. The additional cost of purchasing antibiotic CVCs for all children who require them (£36 per CVC) would be less than the value of resources associated with managing BSIs in PICUs with standard BSI rates of > 1.2 per 1000 CVC-days.

CONCLUSIONS

The primary outcome did not differ between impregnated and standard CVCs. However, antibiotic-impregnated CVCs significantly reduced the risk of BSI compared with standard and heparin CVCs. Adoption of antibiotic-impregnated CVCs could be beneficial even for PICUs with low BSI rates, although uncertainty remains whether or not they represent value for money to the NHS. Limitations - inserting clinicians were not blinded to allocation and a lower than expected event rate meant that there was limited power for head-to-head comparisons of each type of impregnation. Future work - adoption of impregnated CVCs in PICUs should be considered and could be monitored through linkage of electronic health-care data and clinical data on CVC use with laboratory surveillance data on BSI.

TRIAL REGISTRATION

ClinicalTrials.gov NCT01029717.

FUNDING

This project was funded by the NIHR Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 20, No. 18. See the NIHR Journals Library website for further project information.