The rise of big clinical databases

Machine learning in heart failure diagnosis, prediction, and prognosis: review

Globally, cardiovascular diseases take the lives of over 17 million people each year, mostly through myocardial infarction, or MI, and heart failure (HF). This comprehensive literature review examines various aspects related to the diagnosis, prediction, and prognosis of HF in the context of machine learning (ML). The review covers an array of topics, including the diagnosis of HF with preserved ejection fraction (HFpEF) and the identification of high-risk patients with HF with reduced ejection fraction (HFrEF). The prediction of mortality in different HF populations using different ML approaches is explored, encompassing patients in the ICU, and HFpEF patients using biomarkers and gene expression. The review also delves into the prediction of mortality and hospitalization rates in HF patients with mid-range ejection fraction (HFmrEF) using ML methods. The findings highlight the significance of a multidimensional approach that encompasses clinical evaluation, laboratory assessments, and comprehensive research to improve our understanding and management of HF. Promising predictive models incorporating biomarkers, gene expression, and consideration of epigenetics demonstrate potential in estimating mortality and identifying high-risk HFpEF patients. This literature review serves as a valuable resource for researchers, clinicians, and healthcare professionals seeking a comprehensive and updated understanding of the role of ML diagnosis, prediction, and prognosis of HF across different subtypes and patient populations.

Leveraging Real-World Evidence and Observational Studies in Treating Multiple Sclerosis

Randomized controlled trials (RCTs) are essential for regulatory approval of disease-modifying therapies (DMTs), yet their strict selection criteria often lead to limited generalizability. Observational studies using real-world data (RWD) allow for more inclusive heterogeneous cohorts resulting in higher external validity to inform treatment practices. As reviewed in this article, well-designed comparative effectiveness studies are an important application of RWD. Although, like RCTs, observational studies have their own set of limitations, including various biases that may confound results, advanced statistical methods can mitigate many of these limitations. A focus on personalized treatment will continue to add value to individualize MS care.

COVID-19 outcomes in persons with hemophilia: results from a US-based national COVID-19 surveillance registry

BACKGROUND

Hypercoagulable state contributing to thrombotic complications worsens COVID-19 severity and outcomes, whereas anticoagulation improves outcomes by alleviating hypercoagulability.

OBJECTIVES

To examine whether hemophilia, an inherent hypocoagulable condition, offers protection against COVID-19 severity and reduces venous thromboembolism (VTE) risk in persons with hemophilia (PwH).

METHODS

A 1:3 propensity score-matched retrospective cohort study used national COVID-19 registry data (January 2020 through January 2022) to compare outcomes between 300 male PwH and 900 matched controls without hemophilia.

RESULTS

Analyses of PwH demonstrated that known risk factors (older age, heart failure, hypertension, cancer/malignancy, dementia, and renal and liver disease) contributed to severe COVID-19 and/or 30-day all-cause mortality. Non-central nervous system bleeding was an additional risk factor for poor outcomes in PwH. Odds of developing VTE with COVID-19 in PwH were associated with pre-COVID VTE diagnosis (odds ratio [OR], 51.9; 95% CI, 12.8-266; p < .001), anticoagulation therapy (OR, 12.7; 95% CI, 3.01-48.6; p < .001), and pulmonary disease (OR, 16.1; 95% CI, 10.4-25.4; p < .001). Thirty-day all-cause mortality (OR, 1.27; 95% CI, 0.75-2.11; p = .3) and VTE events (OR, 1.32; 95% CI, 0.64-2.73; p = .4) were not significantly different between the matched cohorts; however, hospitalizations (OR, 1.58; 95% CI, 1.20-2.10; p = .001) and non-central nervous system bleeding events (OR, 4.78; 95% CI, 2.98-7.48; p < .001) were increased in PwH. In multivariate analyses, hemophilia did not reduce adverse outcomes (OR, 1.32; 95% CI, 0.74-2.31; p = .2) or VTE (OR, 1.14; 95% CI, 0.44-2.67; p = .8) but increased bleeding risk (OR, 4.70; 95% CI, 2.98-7.48; p < .001).

CONCLUSION

After adjusting for patient characteristics/comorbidities, hemophilia increased bleeding risk with COVID-19 but did not protect against severe disease and VTE.

Systems Anesthesiology: Systems of Care Delivery and Optimization in the Operating Room

Anesthesiology presents a challenge to a traditional simplifying approach given the ever-increasing amount of medical data and a more demanding environment. Systems anesthesiology is a modern approach to perioperative care, integrating the complexity of multifactorial knowledge and data to achieve a more adequate representation of reality, while including both patient-related medical aspects as well as economic and organizational challenges. We discuss the value of some innovative technologies such as the emergence of anesthesia information systems, the use of tele-medicine, predictive monitoring, or closed-loop systems as it pertains to the changes in the current standards of care in anesthesiology. Furthermore, we highlight the importance of systems anesthesiology in operating room planning, anesthesia research, and education.

The Journey of Data Within a Global Data Sharing Initiative: A Federated 3-Layer Data Analysis Pipeline to Scale Up Multiple Sclerosis Research

BACKGROUND

Investigating low-prevalence diseases such as multiple sclerosis is challenging because of the rather small number of individuals affected by this disease and the scattering of real-world data across numerous data sources. These obstacles impair data integration, standardization, and analysis, which negatively impact the generation of significant meaningful clinical evidence.

OBJECTIVE

This study aims to present a comprehensive, research question-agnostic, multistakeholder-driven end-to-end data analysis pipeline that accommodates 3 prevalent data-sharing streams: individual data sharing, core data set sharing, and federated model sharing.

METHODS

A demand-driven methodology is employed for standardization, followed by 3 streams of data acquisition, a data quality enhancement process, a data integration procedure, and a concluding analysis stage to fulfill real-world data-sharing requirements. This pipeline's effectiveness was demonstrated through its successful implementation in the COVID-19 and multiple sclerosis global data sharing initiative.

RESULTS

The global data sharing initiative yielded multiple scientific publications and provided extensive worldwide guidance for the community with multiple sclerosis. The pipeline facilitated gathering pertinent data from various sources, accommodating distinct sharing streams and assimilating them into a unified data set for subsequent statistical analysis or secure data examination. This pipeline contributed to the assembly of the largest data set of people with multiple sclerosis infected with COVID-19.

CONCLUSIONS

The proposed data analysis pipeline exemplifies the potential of global stakeholder collaboration and underlines the significance of evidence-based decision-making. It serves as a paradigm for how data sharing initiatives can propel advancements in health care, emphasizing its adaptability and capacity to address diverse research inquiries.

Emergency department performance assessment using administrative data: A managerial framework

Administrative data play an important role in performance monitoring of healthcare providers. Nonetheless, little attention has been given so far to the emergency department (ED) evaluation. In addition, most of existing research focuses on a single core ED function, such as treatment or triage, thus providing a limited picture of performance. The goal of this study is to harness the value of routinely produced records proposing a framework for multidimensional performance evaluation of EDs able to support internal decision stakeholders in managing operations. Starting with the overview of administrative data, and the definition of the desired framework's characteristics from the perspective of decision stakeholders, a review of the academic literature on ED performance measures and indicators is conducted. A performance measurement framework is designed using 224 ED performance metrics (measures and indicators) satisfying established selection criteria. Real-world feedback on the framework is obtained through expert interviews. Metrics in the proposed ED performance measurement framework are arranged along three dimensions: performance (quality of care, time-efficiency, throughput), analysis unit (physician, disease etc.), and time-period (quarter, year, etc.). The framework has been judged as "clear and intuitive", "useful for planning", able to "reveal inefficiencies in care process" and "transform existing data into decision support information" by the key ED decision stakeholders of a teaching hospital. Administrative data can be a new cornerstone for health care operation management. A framework of ED-specific indicators based on administrative data enables multi-dimensional performance assessment in a timely and cost-effective manner, an essential requirement for nowadays resource-constrained hospitals. Moreover, such a framework can support different stakeholders' decision making as it allows the creation of a customized metrics sets for performance analysis with the desired granularity.

Methods for comparative effectiveness based on time to confirmed disability progression with irregular observations in multiple sclerosis

Real-world data sources offer opportunities to compare the effectiveness of treatments in practical clinical settings. However, relevant outcomes are often recorded selectively and collected at irregular measurement times. It is therefore common to convert the available visits to a standardized schedule with equally spaced visits. Although more advanced imputation methods exist, they are not designed to recover longitudinal outcome trajectories and typically assume that missingness is non-informative. We, therefore, propose an extension of multilevel multiple imputation methods to facilitate the analysis of real-world outcome data that is collected at irregular observation times. We illustrate multilevel multiple imputation in a case study evaluating two disease-modifying therapies for multiple sclerosis in terms of time to confirmed disability progression. This survival outcome is derived from repeated measurements of the Expanded Disability Status Scale, which is collected when patients come to the healthcare center for a clinical visit and for which longitudinal trajectories can be estimated. Subsequently, we perform a simulation study to compare the performance of multilevel multiple imputation to commonly used single imputation methods. Results indicate that multilevel multiple imputation leads to less biased treatment effect estimates and improves the coverage of confidence intervals, even when outcomes are missing not at random.

Impact of Large Database Studies on Orthopedic Surgery Literature: Are We Advancing the Field?

Background: While database studies have become more prevalent in the literature, there is concern over their value. In addition, the questions they are suitable to answer are limited. Questions/Purposes: We sought to determine the incidence of database studies in the orthopedic literature and in each subspecialty. In addition, we wanted to assess the impact of database studies on the literature by determining whether citations and Altmetric Attention Scores (AAS) varied by study type (studies using internal or external databases and those not using databases). Methods: We searched PubMed for articles published in impactful orthopedic surgery journals in the year 2018. All articles were discoverable on the Altmetric explorer portal database. Impact was determined by journal impact factor. Study design, subspecialty, number of citations, and AAS were obtained. Univariable analyses were conducted between study type, demographic variables, and the outcome of either citation count or AAS. Multivariable analyses were performed to identify independent predictors of the primary outcomes. Subgroup analyses were performed to differentiate the impact of external and internal database studies compared with non-database studies. Results: A total of 2684 total articles were eligible for inclusion. Of these, 366 studies (13.6%) were database studies. Hip and knee articles had the greatest incidence of database studies. Database studies had significantly more citations (5.9 vs 4.0) and significantly higher AAS (12.8 vs 11.3) compared with non-database studies. External database studies had significantly more citations (6.7 vs 4.8) and significantly higher AAS (14.0 vs 10.7) than internal database studies. Internal database studies had higher traditional citation counts but similar AAS to non-database studies. Conclusions: In 2018, database studies in well-reputed orthopedic journals had a greater number of citations but similar AAS compared with non-database studies. Further studies are warranted.

Leveraging real-world data to improve cochlear implant outcomes: Is the data available?

OBJECTIVES

A small but persistent proportion of individuals do not gain the expected benefit from cochlear implants(CI). A step-change in the understanding of factors affecting outcomes could come through data science. This study evaluates clinical data capture to assess the quality and utility of CI user's health records for data science, by assessing the recording of otitis media. Otitis media was selected as it is associated with the development of sensorineural hearing loss and may affect cochlear implant outcomes.

METHODS

A retrospective service improvement project evaluating the medical records of 594 people with a CI under the care of the University of Southampton Auditory Implant Service between 2014 and 2020.

RESULTS

The clinical records are suitable for data science research. Of the cohort studied 20% of Adults and more than 40% of the paediatric cases have a history of middle ear inflammation.

DISCUSSION

Data science has potential to improve cochlear implant outcomes and improve understanding of the mechanisms underlying poor performance, through retrospective secondary analysis of real-world data.

CONCLUSION

Implant centres and the British Cochlear Implant Group National Hearing Implant Registry are urged to consider the importance of consistently and accurate recording of patient data over time for each CI user. Data where links to hearing loss have been identified, such as middle ear inflammation, may be particularly valuable in future analyses and to inform clinical trials.

Profile of the multicenter cohort of the German Cancer Consortium's Clinical Communication Platform

Treatment concepts in oncology are becoming increasingly personalized and diverse. Successively, changes in standards of care mandate continuous monitoring of patient pathways and clinical outcomes based on large, representative real-world data. The German Cancer Consortium's (DKTK) Clinical Communication Platform (CCP) provides such opportunity. Connecting fourteen university hospital-based cancer centers, the CCP relies on a federated IT-infrastructure sourcing data from facility-based cancer registry units and biobanks. Federated analyses resulted in a cohort of 600,915 patients, out of which 232,991 were incident since 2013 and for which a comprehensive documentation is available. Next to demographic data (i.e., age at diagnosis: 2.0% 0-20 years, 8.3% 21-40 years, 30.9% 41-60 years, 50.1% 61-80 years, 8.8% 81+ years; and gender: 45.2% female, 54.7% male, 0.1% other) and diagnoses (five most frequent tumor origins: 22,523 prostate, 18,409 breast, 15,575 lung, 13,964 skin/malignant melanoma, 9005 brain), the cohort dataset contains information about therapeutic interventions and response assessments and is connected to 287,883 liquid and tissue biosamples. Focusing on diagnoses and therapy-sequences, showcase analyses of diagnosis-specific sub-cohorts (pancreas, larynx, kidney, thyroid gland) demonstrate the analytical opportunities offered by the cohort's data. Due to its data granularity and size, the cohort is a potential catalyst for translational cancer research. It provides rapid access to comprehensive patient groups and may improve the understanding of the clinical course of various (even rare) malignancies. Therefore, the cohort may serve as a decisions-making tool for clinical trial design and contributes to the evaluation of scientific findings under real-world conditions.

Validity of Routinely Reported Rutherford Scores Reported by Clinicians as Part of Daily Clinical Practice

Routinely reported structured data from the electronic health record (EHR) are frequently used for secondary purposes. However, it is unknown how valid routinely reported data are for reuse.

This study aimed to assess the validity of routinely reported Rutherford scores by clinicians as an indicator for the validity of structured data in the EHR.

This observational study compared clinician-reported Rutherford scores with medical record review Rutherford scores for all visits at the vascular surgery department between April 1, 2016 and December 31, 2018. Free-text fields with clinical information for all visits were extracted for the assignment of the medical record review Rutherford score, after which the agreement with the clinician-reported Rutherford score was assessed using Fleiss' Kappa.

A total of 6,633 visits were included for medical record review. Substantial agreement was shown between clinician-reported Rutherford scores and medical record review Rutherford scores for the left (k = 0.62, confidence interval [CI]: 0.60–0.63) and right leg (k = 0.62, CI: 0.60–0.64). This increased to the almost perfect agreement for left (k = 0.84, CI: 0.82–0.86) and right leg (k = 0.85, CI: 0.83–0.87), when excluding missing clinician-reported Rutherford scores. Expert's judgment was rarely required to be the deciding factor (11 out of 6,633).

Substantial agreement between clinician-reported Rutherford scores and medical record review Rutherford scores was found, which could be an indicator for the validity of routinely reported data. Depending on its purpose, the secondary use of routinely collected Rutherford scores is a viable option.

Development and validation of automated electronic health record data reuse for a multidisciplinary quality dashboard

Objective

To describe the development and validation of automated electronic health record data reuse for a multidisciplinary quality dashboard.

Materials and methods

Comparative study analyzing a manually extracted and an automatically extracted dataset with 262 patients treated for HNC cancer in a tertiary oncology center in the Netherlands in 2020. The primary outcome measures were the percentage of agreement on data elements required for calculating quality indicators and the difference between indicators results calculated using manually collected and indicators that used automatically extracted data.

Results

The results of this study demonstrate high agreement between manual and automatically collected variables, reaching up to 99.0% agreement. However, some variables demonstrate lower levels of agreement, with one variable showing only a 20.0% agreement rate. The indicator results obtained through manual collection and automatic extraction show high agreement in most cases, with discrepancy rates ranging from 0.3% to 3.5%. One indicator is identified as a negative outlier, with a discrepancy rate of nearly 25%.

Conclusions

This study shows that it is possible to use routinely collected structured data to reliably measure the quality of care in real-time, which could render manual data collection for quality measurement obsolete. To achieve reliable data reuse, it is important that relevant data is recorded as structured data during the care process. Furthermore, the results also imply that data validation is conditional to development of a reliable dashboard.

The impact of the Covid-19 pandemic on the incidence of diseases and the provision of primary care: A registry-based study

Introduction

The Covid-19 pandemic had a tremendous impact on healthcare but uncertainty remains about the extent to which primary care provision was affected. Therefore, this paper aims to assess the impact on primary care provision and the evolution of the incidence of disease during the first year of the Covid-19 pandemic in Flanders (Belgium).

Methods

Care provision was defined as the number of new entries added to a patient’s medical history. Pre-pandemic care provision (February 1, 2018–January 31, 2020) was compared with care provision during the pandemic (February 1, 2020-January 31, 2021). A large morbidity registry (Intego) was used. Regression models compared the effect of demographic characteristics on care provision and on acute and chronic diagnoses incidence both prior and during the pandemic.

Results

During the first year of the Covid-19 pandemic, overall care provision increased with 9.1% (95%CI 8.5%;9.6%). There was an increase in acute diagnoses of 5.1% (95%CI 4.2%;6.0%) and a decrease in the selected chronic diagnoses of 12.8% (95% CI 7.0%;18.4%). Obesity was an exception with an overall incidence increase. The pandemic led to strong fluctuations in care provision that were not the same for all types of care and all demographic groups in Flanders. Relative to other groups in the population, the pandemic caused a reduction in care provision for children aged 0–17 year and patients from a lower socio-economic situation.

Conclusion

This paper strengthened the claim that Covid-19 should be considered as a syndemic instead of a pandemic. During the first Covid-19 year, overall care provision and the incidence of acute diagnoses increased, whereas chronic diseases’ incidence decreased, except for obesity diagnoses which increased. More granular, care provision and chronic diseases’ incidence decreased during the lockdowns, especially for people with a lower socio-economic status. After the lockdowns they both returned to baseline.

Applications of artificial intelligence and machine learning in heart failure

Machine learning (ML) is a sub-field of artificial intelligence that uses computer algorithms to extract patterns from raw data, acquire knowledge without human input, and apply this knowledge for various tasks. Traditional statistical methods that classify or regress data have limited capacity to handle large datasets that have a low signal-to-noise ratio. In contrast to traditional models, ML relies on fewer assumptions, can handle larger and more complex datasets, and does not require predictors or interactions to be pre-specified, allowing for novel relationships to be detected. In this review, we discuss the rationale for the use and applications of ML in heart failure, including disease classification, early diagnosis, early detection of decompensation, risk stratification, optimal titration of medical therapy, effective patient selection for devices, and clinical trial recruitment. We discuss how ML can be used to expedite implementation and close healthcare gaps in learning healthcare systems. We review the limitations of ML, including opaque logic and unreliable model performance in the setting of data errors or data shift. Whilst ML has great potential to improve clinical care and research in HF, the applications must be externally validated in prospective studies for broad uptake to occur.

Using Structural Equation Modelling in Routine Clinical Data on Diabetes and Depression: Observational Cohort Study

BACKGROUND

Large data sets comprising routine clinical data are becoming increasingly available for use in health research. These data sets contain many clinical variables that might not lend themselves to use in research. Structural equation modelling (SEM) is a statistical technique that might allow for the creation of "research-friendly" clinical constructs from these routine clinical variables and therefore could be an appropriate analytic method to apply more widely to routine clinical data.

OBJECTIVE

SEM was applied to a large data set of routine clinical data developed in East London to model well-established clinical associations. Depression is common among patients with type 2 diabetes, and is associated with poor diabetic control, increased diabetic complications, increased health service utilization, and increased health care costs. Evidence from trial data suggests that integrating psychological treatment into diabetes care can improve health status and reduce costs. Attempting to model these known associations using SEM will test the utility of this technique in routine clinical data sets.

METHODS

Data were cleaned extensively prior to analysis. SEM was used to investigate associations between depression, diabetic control, diabetic care, mental health treatment, and Accident & Emergency (A&E) use in patients with type 2 diabetes. The creation of the latent variables and the direction of association between latent variables in the model was based upon established clinical knowledge.

RESULTS

The results provided partial support for the application of SEM to routine clinical data. Overall, 19% (3106/16,353) of patients with type 2 diabetes had received a diagnosis of depression. In line with known clinical associations, depression was associated with worse diabetic control (β=.034, P<.001) and increased A&E use (β=.071, P<.001). However, contrary to expectation, worse diabetic control was associated with lower A&E use (β=-.055, P<.001) and receipt of mental health treatment did not impact upon diabetic control (P=.39). Receipt of diabetes care was associated with better diabetic control (β=-.072, P<.001), having depression (β=.018, P=.007), and receiving mental health treatment (β=.046, P<.001), which might suggest that comprehensive integrated care packages are being delivered in East London.

CONCLUSIONS

Some established clinical associations were successfully modelled in a sample of patients with type 2 diabetes in a way that made clinical sense, providing partial evidence for the utility of SEM in routine clinical data. Several issues relating to data quality emerged. Data improvement would have likely enhanced the utility of SEM in this data set.

Factors Associated With Nonunion and Infection Following Ankle Arthrodesis Using a Large Claims Database: Who Has Elevated Risk?

Background:

Complications such as nonunion and infection following ankle arthrodesis can lead to increased patient morbidity and financial burden from repeat operations. Improved knowledge of risk factors can improve patient selection and inform post–ankle arthrodesis surveillance protocols.

Methods:

This is a large retrospective, database study with structured query of a national insurance claims database (PearlDiver Technologies) for patients treated with ankle arthrodesis from 2015 to 2019 as identified by International Classification of Diseases, Tenth Revision (ICD-10), codes. Patients with any operation 1 year prior to or following ankle arthrodesis were excluded from analysis to prevent attributing complications to another operation. Likelihoods of nonunion and infection within 1 year and 3 years following ankle arthrodesis were analyzed using Kaplan-Meier estimations. Patient characteristics associated with the identified complications following ankle arthrodesis were analyzed using multivariable logistic regression analyses.

Results:

Our query yielded 2463 patients in the 5-year period who underwent ankle arthrodesis. Nonunion occurred in 11% (95% CI 10-12) of patients within 1 year of ankle arthrodesis and 16% (95% CI 14-17) of patients within 3 years. Infection occurred in 3.9% (95% CI 3.1-4.7) of patients within 1 year of ankle arthrodesis and in 6.2% (95% CI 5.1-7.2) of patients within 3 years. Obese patients increased odds of nonunion on multivariable analysis (OR 1.6, 95% CI 1.3-2.0; P < .001). On multivariable analysis, diabetes (OR 1.7, 95% CI 1.2-2.6; P = .010) and each 1-unit increase in Elixhauser Comorbidity Index scores (OR 1.1, 95% CI 1.1-1.2; P < .001) contributed to increased odds of infection after ankle arthrodesis.

Conclusion:

Nonunion and infection following ankle arthrodesis have a 3-year probability of 16% and 6%, respectively. More than one-quarter of patients with nonunion following ankle arthrodesis experience a delay in diagnosis beyond 1 year. The risk of post–ankle arthrodesis nonunion is highest in patients with obesity; the risk of post–ankle arthrodesis infection is highest in patients with diabetes or an elevated Elixhauser Comorbidity Index score.

Level of Evidence:

Level III, prognostic study.

Best Practices in Large Database Clinical Epidemiology Research in Hepatology: Barriers and Opportunities

With advances in computing and information technology, large health care research databases are becoming increasingly accessible to investigators across the world. These rich, population-level data sources can serve many purposes, such as to generate "real-world evidence," to enhance disease phenotyping, or to identify unmet clinical needs, among others. This is of particular relevance to the study of patients with end-stage liver disease (ESLD), a socioeconomically and clinically heterogeneous population that is frequently under-represented in clinical trials. This review describes the recommended "best practices" in the execution, reporting, and interpretation of large database clinical epidemiology research in hepatology. The advantages and limitations of selected data sources are reviewed, as well as important concepts on data linkages. The appropriate classification of exposures and outcomes is addressed, and the strategies needed to overcome limitations of the data and minimize bias are explained as they pertain to patients with ESLD and/or liver transplantation (LT) recipients. Lastly, selected statistical concepts are reviewed, from model building to analytic decision making and hypothesis testing. The purpose of this review is to provide the practical insights and knowledge needed to ensure successful and impactful research using large clinical databases in the modern era and advance the study of ESLD and LT.

Toward a better understanding about real-world evidence

BACKGROUND

There has been an interest in real-world evidence (RWE) in recent years. RWE is usually generated from data derived from routine healthcare, such as electronic healthcare records and disease registries. While RWE has many advantages, it is often open to various biases, which may distort results. Appropriate understanding and interpretation are critical to the best use of RWE in healthcare decisions.

METHODS

On the basis of a literature review and empirical research experience, we summarised the concept and methodological framework of RWE, and discussed in detail methodological issues specific to routinely collected healthcare data and observational studies using such data.

RESULTS

RWE is derived from a spectrum of data generated from the real-world setting, using two broad study designs including observational studies and pragmatic clinical trials. Real-world data may usually be collected through routine practice or sometimes actively collected with a research purpose. Observational studies using routinely collected data (RCD) are the most common type of RWE, although they are prone to biases. When planning and implementing RWE studies, coherent working steps are warranted, including definition of a clear and answerable research question, development of a research team, selection of a fit-for-purpose data source, choice of state-of-the-art study design, establishing a database with transparent data processing, performing multiple statistical analysis to control bias, and reporting results in accordance with established guidelines.

CONCLUSIONS

RWE has been mounting over the years. The appropriate interpretation and use of such evidence often warrant adequate understanding about methodology. Researchers and policymakers should be aware of the methodological pitfalls when generating and interpreting RWE.

Assessing the quality of clinical and administrative data extracted from hospitals: the General Medicine Inpatient Initiative (GEMINI) experience

OBJECTIVE

Large clinical databases are increasingly used for research and quality improvement. We describe an approach to data quality assessment from the General Medicine Inpatient Initiative (GEMINI), which collects and standardizes administrative and clinical data from hospitals.

METHODS

The GEMINI database contained 245 559 patient admissions at 7 hospitals in Ontario, Canada from 2010 to 2017. We performed 7 computational data quality checks and iteratively re-extracted data from hospitals to correct problems. Thereafter, GEMINI data were compared to data that were manually abstracted from the hospital's electronic medical record for 23 419 selected data points on a sample of 7488 patients.

RESULTS

Computational checks flagged 103 potential data quality issues, which were either corrected or documented to inform future analysis. For example, we identified the inclusion of canceled radiology tests, a time shift of transfusion data, and mistakenly processing the chemical symbol for sodium ("Na") as a missing value. Manual validation identified 1 important data quality issue that was not detected by computational checks: transfusion dates and times at 1 site were unreliable. Apart from that single issue, across all data tables, GEMINI data had high overall accuracy (ranging from 98%-100%), sensitivity (95%-100%), specificity (99%-100%), positive predictive value (93%-100%), and negative predictive value (99%-100%) compared to the gold standard.

DISCUSSION AND CONCLUSION

Computational data quality checks with iterative re-extraction facilitated reliable data collection from hospitals but missed 1 critical quality issue. Combining computational and manual approaches may be optimal for assessing the quality of large multisite clinical databases.

Predicting Postoperative Complications in Cancer Patients: A Survey Bridging Classical and Machine Learning Contributions to Postsurgical Risk Analysis

Simple Summary

Structured survey on the predictive analysis of postoperative complications in oncology, bridging classic risk scores with machine learning advances, and further establishing principles to guide the design of cohort studies and the predictive modeling of postsurgical risks.

Abstract

Postoperative complications can impose a significant burden, increasing morbidity, mortality, and the in-hospital length of stay. Today, the number of studies available on the prognostication of postsurgical complications in cancer patients is growing and has already created a considerable set of dispersed contributions. This work provides a comprehensive survey on postoperative risk analysis, integrating principles from classic risk scores and machine-learning approaches within a coherent frame. A qualitative comparison is offered, taking into consideration the available cohort data and the targeted postsurgical outcomes of morbidity (such as the occurrence, nature or severity of postsurgical complications and hospitalization needs) and mortality. This work further establishes a taxonomy to assess the adequacy of cohort studies and guide the development and assessment of new learning approaches for the study and prediction of postoperative complications.

Effect of tranexamic acid on intracranial haemorrhage and infarction in patients with traumatic brain injury: a pre-planned substudy in a sample of CRASH-3 trial patients

BACKGROUND

Early tranexamic acid (TXA) treatment reduces head injury deaths after traumatic brain injury (TBI). We used brain scans that were acquired as part of the routine clinical practice during the CRASH-3 trial (before unblinding) to examine the mechanism of action of TXA in TBI. Specifically, we explored the potential effects of TXA on intracranial haemorrhage and infarction.

METHODS

This is a prospective substudy nested within the CRASH-3 trial, a randomised placebo-controlled trial of TXA (loading dose 1 g over 10 min, then 1 g infusion over 8 hours) in patients with isolated head injury. CRASH-3 trial patients were recruited between July 2012 and January 2019. Participants in the current substudy were a subset of trial patients enrolled at 10 hospitals in the UK and 4 in Malaysia, who had at least one CT head scan performed as part of the routine clinical practice within 28 days of randomisation. The primary outcome was the volume of intraparenchymal haemorrhage (ie, contusion) measured on a CT scan done after randomisation. Secondary outcomes were progressive intracranial haemorrhage (post-randomisation CT shows >25% of volume seen on pre-randomisation CT), new intracranial haemorrhage (any haemorrhage seen on post-randomisation CT but not on pre-randomisation CT), cerebral infarction (any infarction seen on any type of brain scan done post-randomisation, excluding infarction seen pre-randomisation) and intracranial haemorrhage volume (intraparenchymal + intraventricular + subdural + epidural) in those who underwent neurosurgical haemorrhage evacuation. We planned to conduct sensitivity analyses excluding patients who were severely injured at baseline. Dichotomous outcomes were analysed using relative risks (RR) or hazard ratios (HR), and continuous outcomes using a linear mixed model.

RESULTS

1767 patients were included in this substudy. One-third of the patients had a baseline GCS (Glasgow Coma Score) of 3 (n=579) and 24% had unilateral or bilateral unreactive pupils. 46% of patients were scanned pre-randomisation and post-randomisation (n=812/1767), 19% were scanned only pre-randomisation (n=341/1767) and 35% were scanned only post-randomisation (n=614/1767). In all patients, there was no evidence that TXA prevents intraparenchymal haemorrhage expansion (estimate=1.09, 95% CI 0.81 to 1.45) or intracranial haemorrhage expansion in patients who underwent neurosurgical haemorrhage evacuation (n=363) (estimate=0.79, 95% CI 0.57 to 1.11). In patients scanned pre-randomisation and post-randomisation (n=812), there was no evidence that TXA reduces progressive haemorrhage (adjusted RR=0.91, 95% CI 0.74 to 1.13) and new haemorrhage (adjusted RR=0.85, 95% CI 0.72 to 1.01). When patients with unreactive pupils at baseline were excluded, there was evidence that TXA prevents new haemorrhage (adjusted RR=0.80, 95% CI 0.66 to 0.98). In patients scanned post-randomisation (n=1431), there was no evidence of an increase in infarction with TXA (adjusted HR=1.28, 95% CI 0.93 to 1.76). A larger proportion of patients without (vs with) a post-randomisation scan died from head injury (38% vs 19%: RR=1.97, 95% CI 1.66 to 2.34, p<0.0001).

CONCLUSION

TXA may prevent new haemorrhage in patients with reactive pupils at baseline. This is consistent with the results of the CRASH-3 trial which found that TXA reduced head injury death in patients with at least one reactive pupil at baseline. However, the large number of patients without post-randomisation scans and the possibility that the availability of scan data depends on whether a patient received TXA, challenges the validity of inferences made using routinely collected scan data. This study highlights the limitations of using routinely collected scan data to examine the effects of TBI treatments.

TRIAL REGISTRATION NUMBER

ISRCTN15088122.

Do routine hospital data accurately record comorbidity in advanced kidney disease populations? A record linkage cohort study

BACKGROUND

Routine healthcare datasets capturing clinical and administrative information are increasingly being used to examine health outcomes. The accuracy of such data is not clearly defined. We examine the accuracy of diagnosis recording in individuals with advanced chronic kidney disease using a routine healthcare dataset in England with comparison to information collected by trained research nurses.

METHODS

We linked records from the Access to Transplant and Transplant Outcome Measures study to the Hospital Episode Statistics dataset. International Classification of Diseases (ICD-10) and Office for Population Censuses and Surveys Classification of Interventions and Procedures (OPCS-4) codes were used to identify medical conditions from hospital data. The sensitivity, specificity, positive and negative predictive values were calculated for a range of diagnoses.

RESULTS

Comorbidity information was available in 96% of individuals prior to starting kidney replacement therapy. There was variation in the accuracy of individual medical conditions identified from the routine healthcare dataset. Sensitivity and positive predictive values ranged from 97.7 and 90.4% for diabetes and 82.6 and 82.9% for ischaemic heart disease to 44.2 and 28.4% for liver disease.

CONCLUSIONS

Routine healthcare datasets accurately capture certain conditions in an advanced chronic kidney disease population. They have potential for use within clinical and epidemiological research studies but are unlikely to be sufficient as a single resource for identifying a full spectrum of comorbidities.

Umpire 2.0: Simulating realistic, mixed-type, clinical data for machine learning

The Umpire 2.0 R-package offers a streamlined, user-friendly workflow to simulate complex, heterogeneous, mixed-type data with known subgroup identities, dichotomous outcomes, and time-to-event data, while providing ample opportunities for fine-tuning and flexibility. Here, we describe how we have expanded the core Umpire 1.0 R-package, developed to simulate gene expression data, to generate clinically realistic, mixed-type data for use in evaluating unsupervised and supervised machine learning (ML) methods. As the availability of large-scale clinical data for ML has increased, clinical data has posed unique challenges, including widely variable size, individual biological heterogeneity, data collection and measurement noise, and mixed data types. Developing and validating ML methods for clinical data requires data sets with known ground truth, generated from simulation. Umpire 2.0 addresses challenges to simulating realistic clinical data by providing the user a series of modules to generate survival parameters and subgroups, apply meaningful additive noise, and discretize to single or mixed data types. Umpire 2.0 provides broad functionality across sample sizes, feature spaces, and data types, allowing the user to simulate correlated, heterogeneous, binary, continuous, categorical, or mixed type data from the scale of a small clinical trial to data on thousands of patients drawn from electronic health records. The user may generate elaborate simulations by varying parameters in order to compare algorithms or interrogate operating characteristics of an algorithm in both supervised and unsupervised ML.

Association of baloxavir marboxil prescription with subsequent medical resource utilization among school-aged children with influenza

PURPOSE

Baloxavir marboxil is a novel antiviral agent for influenza, introduced into clinical practice in 2018. A concern remains about the variant virus with reduced susceptibility after baloxavir exposure and its clinical consequences such as healthcare-seeking behavior.

METHODS

Using a healthcare database in Japan, we compared the medical resource use following baloxavir and neuraminidase inhibitors (NAIs) treatment among children aged 7-15 years. The study period was from December 2018 to March 2019. The primary endpoint was the composite of hospitalization, laboratory and radiological tests, and antibiotic use over 1-9 days of antiviral treatment. As exploratory analyses, secondary outcomes being each single component of the primary composite were assessed and subgroup analyses comparing baloxavir with each NAI were done.

RESULTS

Data from 115 867 prescriptions in 115 238 children were analyzed (median age: 10 years; severe influenza risk in 26%; baloxavir accounting for 43%). Overall, baloxavir use did not increase subsequent medical resource utilization in the composite endpoint (adjusted odds ratio [aOR]: 1.04; 95% confidence interval [CI]: 0.99-1.09; P = 0.14), as were likelihoods of other secondary outcomes. In the subgroup analysis, baloxavir use was associated with higher medical resource use than oseltamivir (aOR: 1.21; 95% CI: 1.13-1.31; P < 0.001) and lower resource use than zanamivir (aOR: 0.93; 95% CI 0.86-1.00; P = 0.040).

CONCLUSIONS

Based on a single-year experience in Japan, prescribing baloxavir rather than NAIs did not increase medical resource utilization within 9 days of treatment, except in one exploratory comparison with oseltamivir.

Impact of Recreational Cannabis Legalization on Hospitalizations for Hyperemesis

INTRODUCTION

Chronic cannabis use had been associated with hyperemesis. We sought to determine whether cannabis liberalization contributed to increased hospitalizations for hyperemesis.

METHODS

Cannabis use and admissions for hyperemesis in legalized states were compared with those of nonlegalized states, before and after cannabis legalization, using state inpatient databases.

RESULTS

From 2011 to 2015, cannabis use increased 2.2 times in legalized states and 1.8 times in nonlegalized states. The odds of presentation with hyperemesis were higher in 2015 compared with those of 2011 in all states.

DISCUSSION

Recreational legalization may be contributing to rising cannabis use. Hospitalizations for hyperemesis have also increased but did not seem to be solely due to cannabis legalization.

Patient-centered outcomes in non-small-cell lung cancer: a real-world perspective

Aims: To assess non-small-cell lung cancer (NSCLC) patient-centered outcomes in the real world. Methods: This is a prospective study of NSCLC patients treated at a private cancer care institution in Brazil between 2014 and 2019. Results: The report comprises 337 patients. Advanced stage was associated with higher symptom burden - fatigue (p = 0.03), pain (p < 0.001) and arm pain (p = 0.022) - and worse global, social and physical functioning (all p < 0.001). In the first 2 years, most factors evolved to either improvement or stability: cough (p = 0.02), pain (p = 0.002), global functioning (p < 0.001) and emotional functioning (p < 0.001). Staging (p < 0.001), fatigue (p = 0.001) and gender (p = 0.004) were independently associated with overall survival. Conclusions: Our results demonstrate the feasibility of conducting real-world prospective analysis of patient-centered outcomes.

The Prospective Dutch Colorectal Cancer (PLCRC) cohort: real-world data facilitating research and clinical care

Real-world data (RWD) sources are important to advance clinical oncology research and evaluate treatments in daily practice. Since 2013, the Prospective Dutch Colorectal Cancer (PLCRC) cohort, linked to the Netherlands Cancer Registry, serves as an infrastructure for scientific research collecting additional patient-reported outcomes (PRO) and biospecimens. Here we report on cohort developments and investigate to what extent PLCRC reflects the “real-world”. Clinical and demographic characteristics of PLCRC participants were compared with the general Dutch CRC population (n = 74,692, Dutch-ref). To study representativeness, standardized differences between PLCRC and Dutch-ref were calculated, and logistic regression models were evaluated on their ability to distinguish cohort participants from the Dutch-ref (AU-ROC 0.5 = preferred, implying participation independent of patient characteristics). Stratified analyses by stage and time-period (2013–2016 and 2017–Aug 2019) were performed to study the evolution towards RWD. In August 2019, 5744 patients were enrolled. Enrollment increased steeply, from 129 participants (1 hospital) in 2013 to 2136 (50 of 75 Dutch hospitals) in 2018. Low AU-ROC (0.65, 95% CI: 0.64–0.65) indicates limited ability to distinguish cohort participants from the Dutch-ref. Characteristics that remained imbalanced in the period 2017–Aug’19 compared with the Dutch-ref were age (65.0 years in PLCRC, 69.3 in the Dutch-ref) and tumor stage (40% stage-III in PLCRC, 30% in the Dutch-ref). PLCRC approaches to represent the Dutch CRC population and will ultimately meet the current demand for high-quality RWD. Efforts are ongoing to improve multidisciplinary recruitment which will further enhance PLCRC’s representativeness and its contribution to a learning healthcare system.

Real-time imputation of missing predictor values in clinical practice

Aims

Use of prediction models is widely recommended by clinical guidelines, but usually requires complete information on all predictors, which is not always available in daily practice. We aim to describe two methods for real-time handling of missing predictor values when using prediction models in practice.

Methods and results

We compare the widely used method of mean imputation (M-imp) to a method that personalizes the imputations by taking advantage of the observed patient characteristics. These characteristics may include both prediction model variables and other characteristics (auxiliary variables). The method was implemented using imputation from a joint multivariate normal model of the patient characteristics (joint modelling imputation; JMI). Data from two different cardiovascular cohorts with cardiovascular predictors and outcome were used to evaluate the real-time imputation methods. We quantified the prediction model's overall performance [mean squared error (MSE) of linear predictor], discrimination (c-index), calibration (intercept and slope), and net benefit (decision curve analysis). When compared with mean imputation, JMI substantially improved the MSE (0.10 vs. 0.13), c-index (0.70 vs. 0.68), and calibration (calibration-in-the-large: 0.04 vs. 0.06; calibration slope: 1.01 vs. 0.92), especially when incorporating auxiliary variables. When the imputation method was based on an external cohort, calibration deteriorated, but discrimination remained similar.

Conclusions

We recommend JMI with auxiliary variables for real-time imputation of missing values, and to update imputation models when implementing them in new settings or (sub)populations.

Development and validation of a follow-up methodology for a randomised controlled trial, utilising routine clinical data as an alternative to traditional designs: a pilot study to assess the feasibility of use for the BladderPath trial

Background

Bladder cancer outcomes have not changed significantly in 30 years; the BladderPath trial (Image Directed Redesign of Bladder Cancer Treatment Pathway, ISRCTN35296862) proposes to evaluate a modified pathway for diagnosis and treatment ensuring appropriate pathways are undertaken earlier to improve outcomes. We are piloting a novel data collection technique based on routine National Health Service (NHS) data, with no traditional patient-Health Care Professional contact after recruitment, where trial data are traditionally collected on case report forms. Data will be collected from routine administrative sources and validated via data queries to sites. We report here the feasibility and pre-trial methodological development and validation of the schema proposed for BladderPath.

Methods

Locally treated patient cohorts were utilised for routine data validation (hospital interactions data (HID) and administrative radiotherapy department data (RTD)). Single site events of interest were algorithmically extracted from the 2008–2018 HID and validated against reference datasets to determine detection sensitivity. Survival analysis was performed using RTD and HID data. Hazard ratios and survival statistics were calculated estimating treatment effects and further validating and assessing the scope of routine data.

Results

Overall, 829/1042 (sensitivity 0.80) events of interest were identified in the HID, with varying levels of sensitivity; identifying, 202/206 (sensitivity 0.98; PPV 0.96) surgical events but only 391/568 (sensitivity 0.69; PPV 0.95) radiotherapy regimens. An overall temporal quality improvement trend was present: detecting 41/117 events (35%) in 2011 to 104/109 (95%) in 2017 (all event types). Using the RTD, 5-year survival rates were 43% (95% CI 25–59%) in the chemoradiotherapy group and 30% (95% CI 23–36%) in the radiotherapy group; using the HID, the 5-year radical cystectomy survival rate was 57% (95% CI 50–63%).

Conclusions

Routine data are a feasible method for trial data collection. As long as events of interest are pre-validated, very high sensitivities for trial conduct can be achieved and further improved with targeted data queries. Outcomes can also be produced comparable to clinical trial and national dataset results. Given the real-time, obligatory nature of the HID, which forms the Hospital Episode Statistics (HES) data, alongside other datasets, we believe routine data extraction and validation is a robust way of rapidly collecting datasets for trials.

Supplementary Information

Supplementary information accompanies this paper at 10.1186/s40814-020-00713-y.

40% reoperation rate in adolescents with spondylolisthesis

STUDY DESIGN

Multicenter retrospective.

OBJECTIVE

To determine the long-term complication rate associated with surgical treatment of spondylolisthesis in adolescents. There is limited information on the complication rate associated with posterior spinal fusion (PSF) of spondylolisthesis in the pediatric and adolescent population.

METHODS

Patients who underwent PSF for spondylolisthesis between 2004 and 2015 at four spine centers, < 21 years of age, were included. Exclusion criteria were < 2 years of follow-up or anterior approach. Charts and radiographs were reviewed.

RESULTS

50 patients had PSF for spondylolisthesis, 26 had PSF alone, while 24 had PSF with trans-foraminal lumbar interbody fusion (TLIF). Mean age was 13.9 years (range 9.6-18.4). Mean follow-up was 5.5 years (range 2-15). Mean preoperative slip was 61.2%. 20/50 patients (40%) experienced 23 complications requiring reoperation at a mean of 2.1 years (range 0-9.3) for the following: implant failure (12), persistent radiculopathy (3), infection (3), persistent back pain (2), extension of fusion (2), and hematoma (1). In addition, there were 22 cases of radiculopathy (44%) that were transient. Rate of implant failure was related to preoperative slip angle (p = 0.02). Reoperation rate and rates of implant failure were not associated with preoperative % slip (reoperation: p = 0.42, implant failure: p = 0.15), postoperative % slip (reoperation: p = 0.42, implant failure: p = 0.99), postoperative kyphosis of the lumbosacral angle (reoperation: p = 0.81, implant failure: p = 0.48), change in % slip (reoperation: p = 0.30, implant failure: p = 0.12), change in slip angle (reoperation: p = 0.42, implant failure: p = 0.40), graft used (reoperation: p = 0.22, implant failure: p = 0.81), or addition of a TLIF (reoperation: p = 0.55, implant failure: p = 0.76).

CONCLUSION

PSF of spondylolisthesis in the adolescent population was associated with a 40% reoperation rate and high rate of post-operative radiculopathy. Addition of a TLIF did not impact reoperation rate or rate of radiculopathy.

Cohort profile: he East London Health and Care Partnership Data Repository: using novel integrated data to support commissioning and research

PURPOSE

The East London Health and Care Partnership (ELHCP) Data Repository was established to support commissioning decisions in London. This dataset comprises routine clinical data for the general practitioner (GP)-registered populations of two London boroughs, Tower Hamlets and City and Hackney, and provides a rich source of demographic, clinical and health service use data of relevance to clinicians, commissioners, researchers and policy makers. This paper describes the dataset in its current form, its representativeness and data completeness.

PARTICIPANTS

There were 351 749 and 344 511 members of the GP-registered population in the two boroughs, respectively, for the financial year 2017/2018. Demographic information and prevalence data were available for 9 mental health and 15 physical health conditions. Prevalence rates from the cohort were compared with local and national data. In order to illustrate the health service use data available in the dataset, emergency department use across mental health conditions was described. Information about data completeness was provided.

FINDINGS TO DATE

The ELHCP Data Repository provides a rich source of information about a relatively young, urban, ethnically diverse, population within areas of socioeconomic deprivation. Prevalence data were in line with local and national statistics with some exceptions. Physical health conditions were more common in those with mental health conditions, reflecting that comorbidities are the norm rather than the exception. This has implications for integrated care. Data completeness for risk factors (eg, blood pressure, cholesterol) was high in patients with long-term conditions.

FUTURE PLANS

The data are being further cleaned and evaluated using imputation, Bayesian and economic methods, principally focusing on specific cohorts, including type II diabetes, depression and personality disorder. Data continue to be collected for the foreseeable future to support commissioning decisions, which will also enable more long-term prospective analysis as data become available at the end of each financial year.

Advancing computer-aided drug discovery (CADD) by big data and data-driven machine learning modeling

Advancing a new drug to market requires substantial investments in time as well as financial resources. Crucial bioactivities for drug candidates, including their efficacy, pharmacokinetics (PK), and adverse effects, need to be investigated during drug development. With advancements in chemical synthesis and biological screening technologies over the past decade, a large amount of biological data points for millions of small molecules have been generated and are stored in various databases. These accumulated data, combined with new machine learning (ML) approaches, such as deep learning, have shown great potential to provide insights into relevant chemical structures to predict in vitro, in vivo, and clinical outcomes, thereby advancing drug discovery and development in the big data era.

Data extraction for epidemiological research (DExtER): a novel tool for automated clinical epidemiology studies

The use of primary care electronic health records for research is abundant. The benefits gained from utilising such records lies in their size, longitudinal data collection and data quality. However, the use of such data to undertake high quality epidemiological studies, can lead to significant challenges particularly in dealing with misclassification, variation in coding and the significant effort required to pre-process the data in a meaningful format for statistical analysis. In this paper, we describe a methodology to aid with the extraction and processing of such databases, delivered by a novel software programme; the "Data extraction for epidemiological research" (DExtER). The basis of DExtER relies on principles of extract, transform and load processes. The tool initially provides the ability for the healthcare dataset to be extracted, then transformed in a format whereby data is normalised, converted and reformatted. DExtER has a user interface designed to obtain data extracts specific to each research question and observational study design. There are facilities to input the requirements for; eligible study period, definition of exposed and unexposed groups, outcome measures and important baseline covariates. To date the tool has been utilised and validated in a multitude of settings. There have been over 35 peer-reviewed publications using the tool, and DExtER has been implemented as a validated public health surveillance tool for obtaining accurate statistics on epidemiology of key morbidities. Future direction of this work will be the application of the framework to linked as well as international datasets and the development of standardised methods for conducting electronic pre-processing and extraction from datasets for research purposes.

External validation of a prognostic model for intensive care unit mortality: a retrospective study using the Ontario Critical Care Information System

PURPOSE

To externally validate an intensive care unit (ICU) mortality prediction model that was created using the Ontario Critical Care Information System (CCIS), which includes the Multiple Organ Dysfunction Score (MODS).

METHODS

We applied the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) recommendations to a prospective longitudinal cohort of patients discharged between 1 July 2015 and 31 December 31 2016 from 90 adult level-3 critical care units in Ontario. We used multivariable logistic regression with measures of discrimination, calibration-in-the-large, calibration slope, and flexible calibration plots to compare prediction model performance of the entire data set and for each ICU subtype.

RESULTS

Among 121,201 CCIS records with ICU mortality of 11.3%, the C-statistic for the validation data set was 0.805. The C-statistic ranged from 0.775 to 0.846 among the ICU subtypes. After intercept recalibration to adjust the baseline risk, the mean predicted risk of death matched actual ICU mortality. The calibration slope was close to 1 with all CCIS data and ICU subtypes of cardiovascular and community hospitals with low ventilation rates. Calibration slopes significantly less than 1 were found for ICUs in teaching hospitals and community hospitals with high ventilation rates whereas coronary care units had a calibration slope significantly higher than 1. Calibration plots revealed over-prediction in high risk groups to a varying degree across all cohorts.

CONCLUSIONS

A risk prediction model primarily based on the MODS shows reproducibility and transportability after intercept recalibration. Risk adjusting models that use existing and feasible data collection can support performance measurement at the individual ICU level.

Unsupervised machine learning and prognostic factors of survival in chronic lymphocytic leukemia

OBJECTIVE

Unsupervised machine learning approaches hold promise for large-scale clinical data. However, the heterogeneity of clinical data raises new methodological challenges in feature selection, choosing a distance metric that captures biological meaning, and visualization. We hypothesized that clustering could discover prognostic groups from patients with chronic lymphocytic leukemia, a disease that provides biological validation through well-understood outcomes.

METHODS

To address this challenge, we applied k-medoids clustering with 10 distance metrics to 2 experiments ("A" and "B") with mixed clinical features collapsed to binary vectors and visualized with both multidimensional scaling and t-stochastic neighbor embedding. To assess prognostic utility, we performed survival analysis using a Cox proportional hazard model, log-rank test, and Kaplan-Meier curves.

RESULTS

In both experiments, survival analysis revealed a statistically significant association between clusters and survival outcomes (A: overall survival, P = .0164; B: time from diagnosis to treatment, P = .0039). Multidimensional scaling separated clusters along a gradient mirroring the order of overall survival. Longer survival was associated with mutated immunoglobulin heavy-chain variable region gene (IGHV) status, absent Zap 70 expression, female sex, and younger age.

CONCLUSIONS

This approach to mixed-type data handling and selection of distance metric captured well-understood, binary, prognostic markers in chronic lymphocytic leukemia (sex, IGHV mutation status, ZAP70 expression status) with high fidelity.

Machine learning in infection management using routine electronic health records: tools, techniques, and reporting of future technologies

BACKGROUND

Machine learning (ML) is increasingly being used in many areas of health care. Its use in infection management is catching up as identified in a recent review in this journal. We present here a complementary review to this work.

OBJECTIVES

To support clinicians and researchers in navigating through the methodological aspects of ML approaches in the field of infection management.

SOURCES

A Medline search was performed with the keywords artificial intelligence, machine learning, infection∗, and infectious disease∗ for the years 2014-2019. Studies using routinely available electronic hospital record data from an inpatient setting with a focus on bacterial and fungal infections were included.

CONTENT

Fifty-two studies were included and divided into six groups based on their focus. These studies covered detection/prediction of sepsis (n = 19), hospital-acquired infections (n = 11), surgical site infections and other postoperative infections (n = 11), microbiological test results (n = 4), infections in general (n = 2), musculoskeletal infections (n = 2), and other topics (urinary tract infections, deep fungal infections, antimicrobial prescriptions; n = 1 each). In total, 35 different ML techniques were used. Logistic regression was applied in 18 studies followed by random forest, support vector machines, and artificial neural networks in 18, 12, and seven studies, respectively. Overall, the studies were very heterogeneous in their approach and their reporting. Detailed information on data handling and software code was often missing. Validation on new datasets and/or in other institutions was rarely done. Clinical studies on the impact of ML in infection management were lacking.

IMPLICATIONS

Promising approaches for ML use in infectious diseases were identified. But building trust in these new technologies will require improved reporting. Explainability and interpretability of the models used were rarely addressed and should be further explored. Independent model validation and clinical studies evaluating the added value of ML approaches are needed.

Multi-level models for heart failure patients' 30-day mortality and readmission rates: the relation between patient and hospital factors in administrative data

Background

This study aims at gathering evidence about the relation between 30-day mortality and 30-day unplanned readmission and patient and hospital factors. By definition, we refer to 30-day mortality and 30-day unplanned readmission as the number of deaths and non-programmed hospitalizations for any cause within 30 days after the incident heart failure (HF). In particular, the focus is on the role played by hospital-level factors.

Methods

A multi-level logistic model that combines patient- and hospital-level covariates has been developed to better disentangle the role played by the two groups of covariates. Later on, hospital outliers in term of better-than-expected/worst-than-expected performers have been identified by comparing expected cases vs. observed cases. Hospitals performance in terms of 30-day mortality and 30-day unplanned readmission rates have been visualized through the creation of funnel plots. Covariates have been selected coherently to past literature. Data comes from the hospital discharge forms for Heart Failure patients in the Lombardy Region (Northern Italy). Considering incident cases for HF in the timespan 2010–2012, 78,907 records for adult patients from 117 hospitals have been collected after quality checks.

Results

Our results show that 30-day mortality and 30-day unplanned readmissions are explained by hospital-level covariates, paving the way for the design and implementation of evidence-based improvement strategies. While the percentage of surgical DRG (OR = 1.001; CI (1.000–1.002)) and the hospital type of structure (Research hospitals vs. non-research public hospitals (OR = 0.62; CI (0.48–0.80)) and Non-research private hospitals vs. non-research hospitals OR = 0.75; CI (0.63–0.90)) are significant for mortality, the mean length of stay (OR = 0.96; CI (0.95–0.98)) is significant for unplanned readmission, showing that mortality and readmission rates might be improved through different strategies.

Conclusion

Our results confirm that hospital-level covariates do affect quality of care, and that 30-day mortality and 30-day unplanned readmission are affected by different managerial choices. This confirms that hospitals should be accountable for their “added value” to quality of care.

Validation Of Cancer Diagnoses In Electronic Health Records: Results From The Information System For Research In Primary Care (SIDIAP) In Northeast Spain

Background

Electronic health records are becoming an increasingly valuable resource for epidemiology but their data quality needs to be quantified. We aimed to validate twenty-five types of incident cancer cases in the Information System for Research in Primary Care (SIDIAP) in Catalonia with the population-based cancer registries of Girona and Tarragona as the gold-standard.

Methods

We calculated the sensitivity, positive predictive values (PPV), and the time-difference between the date of diagnosis entered into the SIDIAP and into the registries. We added hospital discharge cancer diagnoses to the SIDIAP to assess sensitivity changes.

Results

We identified 27,046 incident cancer diagnoses in the SIDIAP from 2009–2015 among the 949,841 residents of Girona and Tarragona. The cancer types with the highest sensitivity were breast (89%, 95% CI: 88–90%), colorectal (81%, 95% CI: 80–82%), and prostate (81%, 95% CI: 80–83%). Trachea, bronchus and lung cancers had the highest PPV (76%, 95% CI: 74%-78%) followed by stomach (72%, 95% CI: 68–75%) and pancreas (71%, 95% CI: 67–75%). Most cancer diagnoses were reported with less than three months of difference between the SIDIAP and the registries. More cases were registered first in the registries than in the SIDIAP. By adding cancer diagnoses based on hospital discharge data, sensitivity increased for all cancers, especially for gallbladder and biliary tract for which the sensitivity increased by 21%.

Conclusion

The SIDIAP includes 76% of the cancer diagnoses in the cancer registries but includes a considerable number of cases that are not in the registries. The SIDIAP reports most of the cancer diagnoses within a three-month period difference from the date of diagnosis in the cancer registries. Our results support the use of the SIDIAP cancer diagnoses for epidemiological research when cancer is the outcome of interest. We recommend adding hospital discharge data to the SIDIAP to increase data quality, particularly for less frequent cancer types.

Leveraging real-world data to investigate multiple sclerosis disease behavior, prognosis, and treatment

Randomized controlled clinical trials and real-world observational studies provide complementary information but with different validity. Some clinical questions (disease behavior, prognosis, validation of outcome measures, comparative effectiveness, and long-term safety of therapies) are often better addressed using real-world data reflecting larger, more representative populations. Integration of disease history, clinician-reported outcomes, performance tests, and patient-reported outcome measures during patient encounters; imaging and biospecimen analyses; and data from wearable devices increase dataset utility. However, observational studies utilizing these data are susceptible to many potential sources of bias, creating barriers to acceptance by regulatory agencies and the medical community. Therefore, data standardization and validation within datasets, harmonization across datasets, and application of appropriate analysis methods are important considerations. We review approaches to improve the scope, quality, and analyses of real-world data to advance understanding of multiple sclerosis and its treatment, as an example of opportunities to better support patient care and research.

Congenital Heart Disease and Autism: A Case-Control Study

OBJECTIVES

There has long been an association between congenital heart disease (CHD) and general neurodevelopmental delays. However, the association between CHD and autism spectrum disorders (AuSDs) is less well understood. Using administrative data, we sought to determine the association between CHD and AuSD and identify specific CHD lesions with higher odds of developing AuSD.

METHODS

We performed a 1:3 case-control study of children enrolled in the US Military Health System from 2001 to 2013. Children with International Classification of Disease, Ninth Revision, Clinical Modification codes for AuSD were identified as cases and matched with controls on the basis of date of birth, sex, and enrollment time frame. Each child's records were reviewed for CHD lesions and associated procedures. Conditional logistic regression determined odds ratios (ORs) and 95% confidence intervals (CIs) for comparative associations.

RESULTS

There were 8760 cases with AuSD and 26 280 controls included in the study. After adjustment for genetic syndrome, maternal age, gestational diabetes, short gestation, newborn epilepsy, birth asphyxia, and low birth weight, there were increased odds of AuSD in patients with CHD (OR 1.32; 95% CI 1.10-1.59). Specific lesions with significant OR included atrial septal defects (n = 82; OR 1.72; 95% CI 1.07-2.74) and ventricular septal defects (n = 193; OR 1.65; 95% CI 1.21-2.25).

CONCLUSIONS

Children with CHD have increased odds of developing AuSD. Specific lesions associated with increased risk include atrial septal defects and ventricular septal defects. These findings will be useful for counseling parents of children with CHD.