The RECORD reporting guidelines: meeting the methodological and ethical demands of transparency in research using routinely-collected health data

Validation of algorithms in studies based on routinely collected health data: general principles

Clinicians, researchers, regulators, and other decision-makers increasingly rely on evidence from real-world data (RWD), including data routinely accumulating in health and administrative databases. RWD studies often rely on algorithms to operationalize variable definitions. An algorithm is a combination of codes or concepts used to identify persons with a specific health condition or characteristic. Establishing the validity of algorithms is a prerequisite for generating valid study findings that can ultimately inform evidence-based health care. In this paper, we aim to systematize terminology, methods, and practical considerations relevant to the conduct of validation studies of RWD-based algorithms. We discuss measures of algorithm accuracy, gold/reference standards, study size, prioritization of accuracy measures, algorithm portability, and implications for interpretation. Information bias is common in epidemiologic studies, underscoring the importance of transparency in decisions regarding choice and prioritizing measures of algorithm validity. The validity of an algorithm should be judged in the context of a data source, and one size does not fit all. Prioritizing validity measures within a given data source depends on the role of a given variable in the analysis (eligibility criterion, exposure, outcome, or covariate). Validation work should be part of routine maintenance of RWD sources. This article is part of a Special Collection on Pharmacoepidemiology.

Association between holiday and weekend admissions and mortality outcomes among patients with acute myocardial infarction receiving percutaneous coronary intervention in Taiwan

There is a lack of studies that concurrently differentiate the effect of the holiday season from the weekend effect on mortality risk in patients with acute myocardial infarction (AMI). We evaluated the mortality risk among patients admitted with AMI who underwent percutaneous coronary intervention, using data from the Taiwan National Health Insurance Research Database. Adult AMI patients admitted during January and February between 2013 and 2020 were enrolled and classified into the holiday season (using the Chinese New Year holiday seasons as an indicator) (n = 1729), weekend (n = 4725), and weekday (n = 14,583) groups according to the first day of admission. A multivariable logistic regression model was used to assess the risk. With the weekday group or the weekend group as the reference, the holiday season group did not have increased risks of in-hospital mortality (adjusted odds ratio [aOR] 1.15; 95% confidence intervals [CI] 0.93-1.42 or aOR 1.23; 95% CI 0.96-1.56) and 7-day mortality (aOR 1.20; 95% CI 0.90-1.58 or aOR 1.24; 95% CI 0.90-1.70). Stratified and subgroup analyses showed similar trends. We conclude that holiday season-initiated admissions were not associated with higher mortality risks in AMI admission cases than weekday or weekend admissions.

Dementia-related agitation: a 6-year nationwide characterization and analysis of hospitalization outcomes

OBJECTIVES

To characterize all hospitalizations held in mainland Portugal (2010-2015) with dementia-related agitation based on International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) coding, and to investigate whether there is a relationship between agitation and hospitalization outcomes.

METHODS

A retrospective observational study was conducted using an administrative dataset containing data from all mainland Portuguese public hospitals. Only hospitalization episodes for patients aged over 65 years who have received a dementia diagnosis ascertained by an ICD-9-CM code of dementia with behavioral disturbance (294.11 and 294.21) and dementia without behavioral disturbance (294.10 and 294.20) were selected. Episodes were further grouped according to the presence of an agitation code. For each episode, demographic data and hospitalization outcomes, including length of stay (LoS), in-hospital mortality, discharge destination and all-cause hospital readmissions, were sourced from the dataset. Comparative analyses were performed and multivariable logistic methods were used to estimate the adjusted associations between agitation (exposure) and outcomes.

RESULTS

Overall, 53,156 episodes were selected, of which 6,586 had an agitation code. These were mostly related to male, younger inpatients (mean 81.19 vs. 83.29 years, p < 0.001), had a higher comorbidity burden, stayed longer at the hospital (median 9.00 vs. 8.00 days, p < 0.001) and frequently ended being transferred to another facility with inpatient care. Agitation was shown to independently increase LoS (aOR = 1.385; 95%CI:1.314-1.461), but not the risk of a fatal outcome (aOR = 0.648; 95%CI:0.600-0.700).

CONCLUSION

These results support the importance of detecting and managing agitation early on admission, since its prompt management may prevent lengthy disruptive hospitalizations.

Use of big data from health insurance for assessment of cardiovascular outcomes

Outcome research that supports guideline recommendations for primary and secondary preventions largely depends on the data obtained from clinical trials or selected hospital populations. The exponentially growing amount of real-world medical data could enable fundamental improvements in cardiovascular disease (CVD) prediction, prevention, and care. In this review we summarize how data from health insurance claims (HIC) may improve our understanding of current health provision and identify challenges of patient care by implementing the perspective of patients (providing data and contributing to society), physicians (identifying at-risk patients, optimizing diagnosis and therapy), health insurers (preventive education and economic aspects), and policy makers (data-driven legislation). HIC data has the potential to inform relevant aspects of the healthcare systems. Although HIC data inherit limitations, large sample sizes and long-term follow-up provides enormous predictive power. Herein, we highlight the benefits and limitations of HIC data and provide examples from the cardiovascular field, i.e. how HIC data is supporting healthcare, focusing on the demographical and epidemiological differences, pharmacotherapy, healthcare utilization, cost-effectiveness and outcomes of different treatments. As an outlook we discuss the potential of using HIC-based big data and modern artificial intelligence (AI) algorithms to guide patient education and care, which could lead to the development of a learning healthcare system and support a medically relevant legislation in the future.

Nation-Wide Routinely Collected Health Datasets in China: A Scoping Review

Objectives: The potential for using routinely collected data for medical research in China remains unclear. We sought to conduct a scoping review to systematically characterise nation-wide routinely collected datasets in China that may be of value for clinical research.

Methods: We searched public databases and the websites of government agencies, and non-government organizations. We included nation-wide routinely collected databases related to communicable diseases, non-communicable diseases, injuries, and maternal and child health. Database characteristics, including disease area, data custodianship, data volume, frequency of update and accessibility were extracted and summarised.

Results: There were 70 databases identified, of which 46 related to communicable diseases, 20 to non-communicable diseases, 1 to injury and 3 to maternal and child health. The data volume varied from below 1000 to over 100,000 records. Over half (64%) of the databases were accessible for medical research mostly comprising communicable diseases.

Conclusion: There are large quantities of routinely collected data in China. Challenges to using such data in medical research remain with various accessibility. The potential of routinely collected data may also be applicable to other low- and middle-income countries.

The utilization of a nurse-led clinic by older people with cognitive complaints: Do they follow our advice?

AIM

Cognitive assessment services were found to be beneficial to older people with cognitive complaints but information is limited on adherence to the recommendations provided. This study aimed to identify predictors of clients' adherence to nursing recommendations after attending a community-based nurse-led cognitive assessment service in Hong Kong.

METHODS

A cross-sectional design was adopted. We analysed a dataset routinely collected from clients attending the services between January 2012 and January 2018. Multiple regression analyses were conducted to identify predictors (demographics, functional health status and cognitive performance) of the clients' adherence. Their satisfaction with the services was also examined.

RESULTS

Two hundred and twenty-three clients attended the clinic. Of the 133 who completed the telephone follow-up at the 6 months postconsultation, 44.1% did not adhere to the recommendations. Clients with a higher Clinical Dementia Rating (OR = 4.17), lower Clock Drawing Test score (OR = 1.13) and lower education level (OR = 0.91) were found to be less likely to adhere to the nursing recommendations. Ninety-six percent (190/198) were satisfied with the overall services.

CONCLUSION

Cognitive assessment services in nurse-led clinics could provide a self-referred service for older people with cognitive complaints. However, low adherence to nursing recommendations was found among clients with poorer cognitive functions and lower education levels.

Long-term impact of certain coexisting extrahepatic unisystem and multisystem manifestations on trends in incidence of liver cirrhosis in treatment-naïve patients with chronic hepatitis C: A nested case-control study

Diabetes mellitus (DM) was found to be more common in hepatitis C virus (HCV)-related cirrhotic males. However, the association between DM, or other extrahepatic manifestations (EHMs), and liver cirrhosis is still undetermined. We used a large-scale long-term study to analyze the cirrhosis risk of treatment-naïve HCV patients with EHMs as compared to those without. In this retrospective nested case-control study, we identified 11 872 treatment-naïve patients with chronic HCV between 2001 and 2013 from Taiwan National Health Insurance Research Database and divided them into patients with (cases) and without cirrhosis (controls). All patients were followed up from the index month (exact month of diagnosis) to the end of 2013, death, or study outcome, whichever occurred first. The cases and controls were 1:6 propensity score matched for age, sex, and exact month of diagnosis; finally, 8078 patients (1154 with and 6924 without cirrhosis) were included in the analysis. The presence of coexisting EHMs and a new diagnosis of cirrhosis was analyzed. Adjusted hazard ratios (HRs) and cumulative incidence for cirrhosis were calculated in conditional Cox regression models after propensity score matching. Patients with high-cirrhosis-risk EHMs, such as DM (HR: 1.72, 95% CI: 1.51-1.96, P < .001), HCD (HR: 1.45, 95% CI: 1.27-1.67, P < .007), CKD (HR: 1.21, 95% CI: 1.05-1.38, P < .001), hyperlipidemia (HR: 0.53, 95% CI: 0.46-0.60, P < .001), lichen planus (HR: 2.71, 95% CI: 1.56-4.72, P < .001), and palpable purpura (HR: 2.67, 95% CI: 2.13-3.35, P < .001) exhibited significantly higher risk of liver cirrhosis than those without. Cumulative incidence (P < .001) of liver cirrhosis by pairwise comparisons of multiple high-cirrhosis-risk EHMs, and that of lichen planus was the highest. Our study provided direct estimates of specific HCV-associated EHM time trends of cirrhosis risk, with an upward trend in incidence. Lichen planus was at the top of the list of single-EHM comparisons, and the maximum combination of certain EHMs was the greatest risk factor across a different array of multi-EHM comparisons for liver cirrhosis development.

Use of crystalloids and colloids in Europe per year from 2010 to 2019: Protocol for an international descriptive study

INTRODUCTION

Intravenous (IV) fluids are widely used in everyday clinical practice. In the last two decades, several trials and clinical practice guidelines have been published on the use of different types and dosages of IV fluid. This may have led to a change in clinical practice. Thus, we aim to describe the use of IV crystalloids and colloids in Europe over a 10-year period.

METHODS AND ANALYSIS

We will collect national or regional data on the use of IV crystalloids and colloids in European hospitals through companies and/or federal medical agencies that provide pharmaceutical data concerning the type and amount of fluids used from January 1st, 2010 to December 31st, 2019. We will use run charts to describe time trends in the use of fluids.

CONCLUSION

This study will provide insight into the use of IV fluids in Europe between 2010 and 2019. It will enable the observation of changes over time and among the different European countries and regions. This will provide information on the extent to which landmark trials and clinical practice guidelines on fluids have affected clinical practice in Europe regarding the use of IV fluids.

Record linkage of population-based cohort data from minors with national register data: a scoping review and comparative legal analysis of four European countries

Background: The GDPR was implemented to build an overarching framework for personal data protection across the EU/EEA. Linkage of data directly collected from cohort participants, potentially serving as a prominent tool for health research, must respect data protection rules and privacy rights. Our objective was to investigate law possibilities of linking cohort data of minors with routinely collected education and health data comparing EU/EEA member states. Methods: A legal comparative analysis and scoping review was conducted of openly accessible published laws and regulations in EUR-Lex and national law databases on GDPR's implementation in Portugal, Finland, Norway, and the Netherlands and its connected national regulations purposing record linkage for health research that have been implemented up until April 30, 2021. Results: The GDPR does not ensure total uniformity in data protection legislation across member states offering flexibility for national legislation. Exceptions to process personal data, e.g., public interest and scientific research, must be laid down in EU/EEA or national law. Differences in national interpretation caused obstacles in cross-national research and record linkage: Portugal requires written consent and ethical approval; Finland allows linkage mostly without consent through the national Social and Health Data Permit Authority; Norway when based on regional ethics committee's approval and adequate information technology safeguarding confidentiality; the Netherlands mainly bases linkage on the opt-out system and Data Protection Impact Assessment. Conclusions: Though the GDPR is the most important legal framework, national legislation execution matters most when linking cohort data with routinely collected health and education data. As national interpretation varies, legal intervention balancing individual right to informational self-determination and public good is gravely needed for health research. More harmonization across EU/EEA could be helpful but should not be detrimental in those member states which already opened a leeway for registries and research for the public good without explicit consent.

Assessing the validity of administrative health data for the identification of children and youth with autism spectrum disorder in Ontario

Population‐level identification of children and youth with ASD is essential for surveillance and planning for required services. The objective of this study was to develop and validate an algorithm for the identification of children and youth with ASD using administrative health data. In this retrospective validation study, we linked an electronic medical record (EMR)‐based reference standard, consisting 10,000 individuals aged 1–24 years, including 112 confirmed ASD cases to Ontario administrative health data, for the testing of multiple case‐finding algorithms. Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and corresponding 95% confidence intervals (CI) were calculated for each algorithm. The optimal algorithm was validated in three external cohorts representing family practice, education, and specialized clinical settings. The optimal algorithm included an ASD diagnostic code for a single hospital discharge or emergency department visit or outpatient surgery, or three ASD physician billing codes in 3 years. This algorithm's sensitivity was 50.0% (95%CI 40.7–88.7%), specificity 99.6% (99.4–99.7), PPV 56.6% (46.8–66.3), and NPV 99.4% (99.3–99.6). The results of this study illustrate limitations and need for cautious interpretation when using administrative health data alone for the identification of children and youth with ASD.

Lay Summary

We tested algorithms (set of rules) to identify young people with ASD using routinely collected administrative health data. Even the best algorithm misses more than half of those in Ontario with ASD. To understand this better, we tested how well the algorithm worked in different settings (family practice, education, and specialized clinics). The identification of individuals with ASD at a population level is essential for planning for support services and the allocation of resources. Autism Res 2021, 14: 1037–1045. © 2021 The Authors. Autism Research published by International Society for Autism Research published by Wiley Periodicals LLC.

A retrospective cohort study evaluating correlates of deep tissue infections among patients enrolled in opioid agonist treatment using administrative data in Ontario, Canada

Objective

The objective of this study was to evaluate the relationship between individual characteristics and deep tissue infections in patients enrolled in opioid agonist treatment in Ontario, Canada.

Methods

A retrospective cohort study was conducted on patients in opioid agonist treatment between January 1, 2011, and December 31, 2015 in Ontario, Canada. Patients were identified using data from the Ontario Health Insurance Plan Database, and the Ontario Drug Benefit Plan Database. We identified other study variables including all-cause mortality using data from the Registered Persons Database. Encrypted patient identifiers were used to link across databases. Logistic regression models were used to measure potential correlates of deep tissue infections.

Results

An increase in the incidence of deep tissue infections was observed between 2011 and 2016 for patients on opioid agonist treatment. Additionally, age, sex, positive HIV diagnosis, and all-cause mortality was correlated with deep tissue infection in our study population.

Conclusion

The study indicates factors that are associated with deep tissue infections in the opioid use disorder population and can be used to identify opportunities to reduce the incidence of new infections.

Incidence and time trends of joint surgery in patients with psoriatic arthritis: a register-based time series and cohort study from Denmark

OBJECTIVE

To investigate time-trends and cumulative incidence of joint surgery among patients with psoriatic arthritis (PsA) compared with the general population.

METHODS

In this nationwide register-based cohort study, The Danish National Patient Registry was used to identify incident PsA patients. The 5-year incidence rates (IR) and incidence rate ratios (IRR) of joint surgery were calculated in four calendar-period defined cohorts. Each patient was matched with ten non-PsA individuals from the general population cohort (GPC). The cumulative incidences of any joint and joint-sacrificing surgery, respectively, were estimated using the Aalen-Johansen method.

RESULTS

From 1996 to 2017, 11 960 PsA patients (mean age 50 years; 57% female) were registered. The IRR of any joint surgery was twice as high for PsA patients compared with GPCs across all calendar periods. Among patients with PsA, 2, 10 and 29% required joint surgery at 5, 10 and 15 years after diagnosis. The risk of surgery in PsA patients diagnosed at 18-40 years was higher (22%) than in GPC 60+ year old (20%) after 15 years of follow-up.

CONCLUSIONS

The use of joint surgery among PsA patients remained around twofold higher from 1996 to 2012 compared with GPC. After 15 years of follow-up, nearly 30% of the PsA patients had received any surgery, and even a person diagnosed with PsA at the age of 18-40 years had a higher risk of surgery than GPCs of 60+ year old. Thus, the high surgical rates represent an unmet need in the current treatment of PsA.

The identification of cases of major hemorrhage during hospitalization in patients with acute leukemia using routinely recorded healthcare data

INTRODUCTION

Electronic health care data offers the opportunity to study rare events, although detecting these events in large datasets remains difficult. We aimed to develop a model to identify leukemia patients with major hemorrhages within routinely recorded health records.

METHODS

The model was developed using routinely recorded health records of a cohort of leukemia patients admitted to an academic hospital in the Netherlands between June 2011 and December 2015. Major hemorrhage was assessed by chart review. The model comprised CT-brain, hemoglobin drop, and transfusion need within 24 hours for which the best discriminating cut off values were taken. External validation was performed within a cohort of two other academic hospitals.

RESULTS

The derivation cohort consisted of 255 patients, 10,638 hospitalization days, of which chart review was performed for 353 days. The incidence of major hemorrhage was 0.22 per 100 days in hospital. The model consisted of CT-brain (yes/no), hemoglobin drop of ≥0.8 g/dl and transfusion of ≥6 units. The C-statistic was 0.988 (CI 0.981-0.995). In the external validation cohort of 436 patients (19,188 days), the incidence of major hemorrhage was 0.46 per 100 hospitalization days and the C-statistic was 0.975 (CI 0.970-0.980). Presence of at least one indicator had a sensitivity of 100% (CI 95.8-100) and a specificity of 90.7% (CI 90.2-91.1). The number of days to screen to find one case decreased from 217.4 to 23.6.

INTERPRETATION

A model based on information on CT-brain, hemoglobin drop and need of transfusions can accurately identify cases of major hemorrhage within routinely recorded health records.

Pattern and Outcome of Heart Failure-Related Hospitalization Over 5 Years in a Remote Australian Population: A Retrospective Administrative Data Cohort of 617 Indigenous and non-Indigenous Cases

OBJECTIVE

The aim of this work was to understand the pattern and outcomes for heart failure (HF)-related hospitalization among Indigenous and non-Indigenous patients living in Central Australia.

METHODS AND RESULTS

A retrospective analysis of administrative data for patients presenting with a primary or secondary diagnosis of HF to Central Australia's Alice Springs Hospital during 2008-2012 was performed. The population rate of admission and subsequent outcomes (including mortality and readmission) during the 5-year study period were examined. A total of 617 patients, aged 55.8 ± 17.5 years and 302 (49%) female constituted the study cohort. The 446 Indigenous patients (72%) were significantly younger (50.8 ± 15.9 vs 68.7 ± 14.9; P < .001) and clinically more complex compared with the non-Indigenous patients. Annual prevalence of any HF hospitalization was markedly higher in the Indigenous population (1.9%, 95% CI 1.7-2.1) compared with the non-Indigenous population (0.5%, 95% CI 0.4-0.6); the greatest difference being for women. Overall, non-Indigenous patients had poorer outcomes and were significantly more likely to die (P < .0001), but this was largely driven by age differences. Alternatively, Indigenous patients were significantly more likely to have a higher number of hospitalizations, although indigeneity was not a predictor for 30- or 365-day rehospitalization from the index admission.

CONCLUSION

The pattern of HF among Indigenous Australians in Central Australia is characterized by a younger population with more clinically complex cases and greater health care utilization.

Effect modification, interaction and mediation: an overview of theoretical insights for clinical investigators

We revisited the three interrelated epidemiological concepts of effect modification, interaction and mediation for clinical investigators and examined their applicability when using research databases. The standard methods that are available to assess interaction, effect modification and mediation are explained and exemplified. For each concept, we first give a simple “best-case” example from a randomized controlled trial, followed by a structurally similar example from an observational study using research databases. Our explanation of the examples is based on recent theoretical developments and insights in the context of large health care databases. Terminology is sometimes ambiguous for what constitutes effect modification and interaction. The strong assumptions underlying the assessment of interaction, and particularly mediation, require clinicians and epidemiologists to take extra care when conducting observational studies in the context of health care databases. These strong assumptions may limit the applicability of interaction and mediation assessments, at least until the biases and limitations of these assessments when using large research databases are clarified.