Assessing the quality of clinical and administrative data extracted from hospitals: the General Medicine Inpatient Initiative (GEMINI) experience

Anaerobic Antibiotic Coverage in Aspiration Pneumonia and the Associated Benefits and Harms: A Retrospective Cohort Study

BACKGROUND

Antibiotics with extended anaerobic coverage are used commonly to treat aspiration pneumonia, which is not recommended by current guidelines.

RESEARCH QUESTION

In patients admitted to hospital for community-acquired aspiration pneumonia, does a difference exist between antibiotic therapy with limited anaerobic coverage (LAC) vs antibiotic therapy with extended anaerobic coverage (EAC) in terms of in-hospital mortality and risk of Clostridioides difficile colitis?

STUDY DESIGN AND METHODS

We conducted a multicenter retrospective cohort study across 18 hospitals in Ontario, Canada, from January 1, 2015, to January 1, 2022. Patients were included if the physician diagnosed aspiration pneumonia and prescribed guideline-concordant first-line community-acquired pneumonia parenteral antibiotic therapy to the patient within 48 h of admission. Patients then were categorized into the LAC group if they received ceftriaxone, cefotaxime, or levofloxacin. Patients were categorized into the EAC group if they received amoxicillin-clavulanate, moxifloxacin, or any of ceftriaxone, cefotaxime, or levofloxacin in combination with clindamycin or metronidazole. The primary outcome was all-cause in-hospital mortality. Secondary outcomes included incident C difficile colitis occurring after admission. Overlap weighting of propensity scores was used to balance baseline prognostic factors.

RESULTS

The LAC and EAC groups included 2,683 and 1,316 patients, respectively. In hospital, 814 patients (30.3%) and 422 patients (32.1%) in the LAC and EAC groups died, respectively. C difficile colitis occurred in five or fewer patients (≤ 0.2%) and 11 to 15 patients (0.8%-1.1%) in the LAC and EAC groups, respectively. After overlap weighting of propensity scores, the adjusted risk difference of EAC minus LAC was 1.6% (95% CI, -1.7% to 4.9%) for in-hospital mortality and 1.0% (95% CI, 0.3%-1.7%) for C difficile colitis.

INTERPRETATION

We found that extended anaerobic coverage likely is unnecessary in aspiration pneumonia because it was associated with no additional mortality benefit, only an increased risk of C difficile colitis.

Characterizing the burden of biliary tract cancers across 28 hospitals in Ontario, Canada

BACKGROUND AND AIMS

The incidence of biliary tract cancers (BTC) appears to be increasing worldwide. We analyzed the characteristics of BTC-related hospitalizations under medical services across 28 hospitals in Ontario, Canada.

METHODS

This study uses data collected by GEMINI, a hospital research data network. BTC-related hospitalizations from 2015 to 2021 under the Department of Medicine or intensive care unit were captured using the International Classification of Diseases, 10th revision, codes for intrahepatic cholangiocarcinoma (iCCA), extrahepatic cholangiocarcinoma, and gallbladder cancers.

RESULTS

A total of 4596 BTC-related hospitalizations (2720 iCCA, 1269 extrahepatic cholangiocarcinoma, 607 gallbladder cancers) were analyzed. The number of unique patients with BTC-related hospitalizations increased over time. For iCCA-related hospitalizations, the total number of hospitalizations increased (from 385 in 2016 to 420 in 2021, p = .005), the hospital length of stay decreased over the study period (mean 10 days [SD, 12] in 2016 to 9 days [SD, 8] in 2021, p = .04), and the number of in-hospital deaths was stable (from 68 [18%] in 2016 to 55 [13%] in 2021, p = .62). Other outcomes such as 30-day readmissions, medical imaging tests, intensive care unit-specific hospitalizations, and length of stay were stable over time for all cohorts. The cost of hospitalization for the BTC cohort increased from median $8203 CAD (interquartile range, 5063-15,543) in 2017 to $8507 CAD (interquartile range, 5345-14,755) in 2021.

CONCLUSIONS

This real-world data analysis showed a rising number of patients with BTC-related hospitalizations and rising number of iCCA-related hospitalizations across 28 hospitals in Ontario between 2015 and 2021.

A simple electronic medical record-based predictors of illness severity in sepsis (sepsis) score

OBJECTIVE

Current scores for predicting sepsis outcomes are limited by generalizability, complexity, and electronic medical record (EMR) integration. Here, we validate a simple EMR-based score for sepsis outcomes in a large multi-centre cohort.

DESIGN

A simple electronic medical record-based predictor of illness severity in sepsis (SEPSIS) score was developed (4 additive lab-based predictors) using a population-based retrospective cohort study.

SETTING

Internal medicine services across four academic teaching hospitals in Toronto, Canada from April 2010-March 2015 (primary cohort) and 2015-2019 (secondary cohort).

PATIENTS

We identified patients admitted with sepsis based upon receipt of antibiotics and positive cultures.

MEASUREMENTS AND MAIN RESULTS

The primary outcome was in-hospital mortality and secondary outcomes were ICU admission at 72 hours, and hospital length of stay (LOS). We calculated the area under the receiver operating curve (AUROC) for the SEPSIS score, qSOFA, and NEWS2. We then evaluated the SEPSIS score in a secondary cohort (2015-2019) of hospitalized patients receiving antibiotics. Our primary cohort included 1,890 patients with a median age of 72 years (IQR: 56-83). 9% died during hospitalization, 18.6% were admitted to ICU, and mean LOS was 12.7 days (SD: 21.5). In the primary and secondary (2015-2019, 4811 patients) cohorts, the AUROCs of the SEPSIS score for predicting in-hospital mortality were 0.63 and 0.64 respectively, which were similar to NEWS2 (0.62 and 0.67) and qSOFA (0.62 and 0.68). AUROCs for predicting ICU admission at 72 hours, and length of stay > 14 days, were similar between scores, in the primary and secondary cohorts. All scores had comparable calibration for predicting mortality.

CONCLUSIONS

An EMR-based SEPSIS score shows a similar ability to predict important clinical outcomes compared with other validated scores (qSOFA and NEWS2). Because of the SEPSIS score's simplicity, it may prove a useful tool for clinical and research applications.

Recent methodological advances in federated learning for healthcare

For healthcare datasets, it is often impossible to combine data samples from multiple sites due to ethical, privacy, or logistical concerns. Federated learning allows for the utilization of powerful machine learning algorithms without requiring the pooling of data. Healthcare data have many simultaneous challenges, such as highly siloed data, class imbalance, missing data, distribution shifts, and non-standardized variables, that require new methodologies to address. Federated learning adds significant methodological complexity to conventional centralized machine learning, requiring distributed optimization, communication between nodes, aggregation of models, and redistribution of models. In this systematic review, we consider all papers on Scopus published between January 2015 and February 2023 that describe new federated learning methodologies for addressing challenges with healthcare data. We reviewed 89 papers meeting these criteria. Significant systemic issues were identified throughout the literature, compromising many methodologies reviewed. We give detailed recommendations to help improve methodology development for federated learning in healthcare.

Real-world use of glucocorticoids and clinical outcomes in adults hospitalized with community-acquired pneumonia on medical wards

BACKGROUND

Little is known about the real-world use of systemic glucocorticoids to treat patients hospitalized with community-acquired pneumonia (CAP) outside of the intensive care unit (ICU).

METHODS

This retrospective cohort study included 11,588 hospitalizations for CAP without chronic pulmonary disease at seven hospitals in Ontario, Canada. We report physician-level variation in the use of glucocorticoids and trends over time. We investigated the association between glucocorticoid prescriptions and clinical outcomes, using propensity score overlap weighting to account for confounding by indication.

RESULTS

Glucocorticoids were prescribed in 1283 (11.1%) patients, increasing over time from 10.0% in 2010 to 11.9% in 2020 (p = .008). Physician glucocorticoid prescribing ranged from 2.9% to 34.6% (median 10.0%, inter quartile range [IQR]: 6.7%-14.6%). Patients receiving glucocorticoids tended to be younger (median age 73 vs. 79), have higher Charlson comorbidity scores (score of 2 or more: 42.4% vs. 31.0%), more cancer (26.6% vs. 13.2%), more renal disease (11.5% vs. 6.6%), and less dementia (7.8% vs. 14.8%). Patients treated with glucocorticoids had higher rates of in-hospital mortality (weighted Risk Difference = 1.72, 95% confidence interval [95% CI]: 0.16-3.3, p = .033). Glucocorticoid use was not associated with ICU admission, hospital length-of-stay, or 30-day readmission.

CONCLUSION

Glucocorticoids were prescribed in 11.1% of patients hospitalized with CAP outside of ICU and one in four physicians prescribed glucocorticoids in more than 14% of patients. Glucocorticoid use was associated with greater in-hospital mortality, although these findings are limited by large selection effects. Clinicians should exercise caution in prescribing glucocorticoids for nonsevere CAP, and definitive trials are needed in this population.

Validation of the Diagnostic Accuracy Levels of International Classification of Diseases, 10th Revision Codes for Diabetic Ketoacidosis: A Multicentre, Cross-sectional Study of Adults

OBJECTIVES

International Classification of Diseases (ICD) codes are commonly used to identify cases of diabetic ketoacidosis (DKA) in health services research, but they have not been validated. Our aim in this study was to assess the accuracy of ICD, 10th revision (ICD-10) diagnosis codes for DKA.

METHODS

We conducted a multicentre, cross-sectional study using data from 5 hospitals in Ontario, Canada. Each hospitalization event has a single most responsible diagnosis code. We identified all hospitalizations assigned diagnosis codes for DKA. A true case of DKA was defined using laboratory values (serum bicarbonate ≤18 mmol/L, arterial pH ≤7.3, anion gap ≥14 mEq/L, and presence of ketones in urine or blood). Chart review was conducted to validate DKA if laboratory values were missing or the diagnosis of DKA was unclear. Outcome measures included positive predictive value (PPV), negative predictive value (NPV), sensitivity, and specificity of ICD-10 codes in patients with laboratory-defined DKA.

RESULTS

We identified 316,517 hospitalizations. Among these, 312,948 did not have an ICD-10 diagnosis code for DKA and 3,569 had an ICD-10 diagnosis code for DKA. Using a combination of laboratory and chart review, we identified that the overall PPV was 67.0%, the NPV was 99.7%, specificity was 99.6%, and sensitivity was 74.9%. When we restricted our analysis to hospitalizations in which DKA was the most responsible discharge diagnosis (n=3,374 [94.5%]), the test characteristics were PPV 69.8%, NPV 99.7%, specificity 99.7%, and sensitivity 71.9%.

CONCLUSION

ICD-10 codes can identify patients with DKA among those admitted to general internal medicine.

Development and external validation of tools for categorizing diagnosis codes in international hospital data

BACKGROUND

The Clinical Classification Software Refined (CCSR) is a tool that groups many thousands of International Classification of Diseases 10th Revision (ICD-10) diagnosis codes into approximately 500 clinically meaningful categories, simplifying analyses. However, CCSR was developed for use in the United States and may not work well with other country-specific ICD-10 coding systems.

METHOD

We developed an algorithm for semi-automated matching of Canadian ICD-10 codes (ICD-10-CA) to CCSR categories using discharge diagnoses from adult admissions at 7 hospitals between Apr 1, 2010 and Dec 31, 2020, and manually validated the results. We then externally validated our approach using inpatient hospital encounters in Denmark from 2017 to 2018.

KEY RESULTS

There were 383,972 Canadian hospital admissions with 5,186 distinct ICD-10-CA diagnosis codes and 1,855,837 Danish encounters with 4,612 ICD-10 diagnosis codes. Only 46.6% of Canadian codes and 49.4% of Danish codes could be directly categorized using the official CCSR tool. Our algorithm facilitated the mapping of 98.5% of all Canadian codes and 97.7% of Danish codes. Validation of our algorithm by clinicians demonstrated excellent accuracy (97.1% and 97.0% in Canadian and Danish data, respectively). Without our algorithm, many common conditions did not match directly to a CCSR category, such as 96.6% of hospital admissions for heart failure.

CONCLUSION

The GEMINI CCSR matching algorithm (available as an open-source package at https://github.com/GEMINI-Medicine/gemini-ccsr) improves the categorization of Canadian and Danish ICD-10 codes into clinically coherent categories compared to the original CCSR tool. We expect this approach to generalize well to other countries and enable a wide range of research and quality measurement applications.

Characterizing medical patients with delirium: A cohort study comparing ICD-10 codes and a validated chart review method

BACKGROUND

Delirium is a major cause of preventable mortality and morbidity in hospitalized adults, but accurately determining rates of delirium remains a challenge.

OBJECTIVE

To characterize and compare medical inpatients identified as having delirium using two common methods, administrative data and retrospective chart review.

METHODS

We conducted a retrospective study of 3881 randomly selected internal medicine hospital admissions from six acute care hospitals in Toronto and Mississauga, Ontario, Canada. Delirium status was determined using ICD-10-CA codes from hospital administrative data and through a previously validated chart review method. Baseline sociodemographic and clinical characteristics, processes of care and outcomes were compared across those without delirium in hospital and those with delirium as determined by administrative data and chart review.

RESULTS

Delirium was identified in 6.3% of admissions by ICD-10-CA codes compared to 25.7% by chart review. Using chart review as the reference standard, ICD-10-CA codes for delirium had sensitivity 24.1% (95%CI: 21.5-26.8%), specificity 99.8% (95%CI: 99.5-99.9%), positive predictive value 97.6% (95%CI: 94.6-98.9%), and negative predictive value 79.2% (95%CI: 78.6-79.7%). Age over 80, male gender, and Charlson comorbidity index greater than 2 were associated with misclassification of delirium. Inpatient mortality and median costs of care were greater in patients determined to have delirium by ICD-10-CA codes (5.8% greater mortality, 95% CI: 2.0-9.5 and $6824 greater cost, 95%CI: 4713-9264) and by chart review (11.9% greater mortality, 95%CI: 9.5-14.2% and $4967 greater cost, 95%CI: 4415-5701), compared to patients without delirium.

CONCLUSIONS

Administrative data are specific but highly insensitive, missing most cases of delirium in hospital. Mortality and costs of care were greater for both the delirium cases that were detected and missed by administrative data. Better methods of routinely measuring delirium in hospital are needed.

Characteristics of Drug Poisonings Seen in the Emergency Department of an Urban Hospital

Background

Drug poisoning, either intentional or non-intentional, is a frequent diagnosis in the emergency department (ED), necessitating patient management from multiple services.

Objective

To describe the drug poisonings seen in the ED of a large academic urban hospital.

Methods

This retrospective descriptive study used 3 years of data (2018-2020) abstracted from the hospital's electronic medical record system and linked to validated, coded extracts from the Canadian Institute for Health Information Discharge Abstract Database. Patients with a diagnosis of acute drug poisoning who presented to the ED were identified on the basis of International Statistical Classification of Diseases and Related Health Problems, 10th revision, Canada (ICD-10-CA) codes, and data were collected for demographic characteristics, the drugs involved, in-hospital management, and inpatient outcomes. Patients with diagnosis of an acute drug reaction, inebriation, or nondrug or in-hospital poisoning were excluded. Data were stratified and analyzed in relation to the intent of drug poisoning.

Results

A total of 2983 visits for drug poisoning, involving 2211 unique patients (mean age 38.3 [standard deviation 16.2] years, 54.7% female), were included, yielding an overall incidence rate of 15.7 drug poisonings per 1000 ED visits (8.1 intentional, 6.4 non-intentional, and 1.3 unknown intent). Among the 1505 intentional drug poisonings, the most prevalent drug sources were antidepressants (n = 405, 26.9%), benzodiazepines (n = 375, 24.9%), and acetaminophen (n = 329, 21.9%); in contrast, opioids (n = 594, 48.1%) were most prevalent for the 1236 non-intentional poisonings. For 716 (24.0%) of the poisoning visits, the patient was admitted to acute care services, and the in-hospital mortality rate was 1.0% (n = 31). In addition, 111 patients (9.0%) with non-intentional drug poisoning left against medical advice. Finally, for 772 (25.9%) of the poisoning visits, the patient returned to the ED after discharge with a subsequent drug poisoning.

Conclusions

Drug poisonings are a common cause of visits to urban EDs. They are rarely fatal but are associated with substantial utilization of hospital resources and considerable recidivism.

Decentralised, collaborative, and privacy-preserving machine learning for multi-hospital data

BACKGROUND

Machine Learning (ML) has demonstrated its great potential on medical data analysis. Large datasets collected from diverse sources and settings are essential for ML models in healthcare to achieve better accuracy and generalizability. Sharing data across different healthcare institutions or jurisdictions is challenging because of complex and varying privacy and regulatory requirements. Hence, it is hard but crucial to allow multiple parties to collaboratively train an ML model leveraging the private datasets available at each party without the need for direct sharing of those datasets or compromising the privacy of the datasets through collaboration.

METHODS

In this paper, we address this challenge by proposing Decentralized, Collaborative, and Privacy-preserving ML for Multi-Hospital Data (DeCaPH). This framework offers the following key benefits: (1) it allows different parties to collaboratively train an ML model without transferring their private datasets (i.e., no data centralization); (2) it safeguards patients' privacy by limiting the potential privacy leakage arising from any contents shared across the parties during the training process; and (3) it facilitates the ML model training without relying on a centralized party/server.

FINDINGS

We demonstrate the generalizability and power of DeCaPH on three distinct tasks using real-world distributed medical datasets: patient mortality prediction using electronic health records, cell-type classification using single-cell human genomes, and pathology identification using chest radiology images. The ML models trained with DeCaPH framework have less than 3.2% drop in model performance comparing to those trained by the non-privacy-preserving collaborative framework. Meanwhile, the average vulnerability to privacy attacks of the models trained with DeCaPH decreased by up to 16%. In addition, models trained with our DeCaPH framework achieve better performance than those models trained solely with the private datasets from individual parties without collaboration and those trained with the previous privacy-preserving collaborative training framework under the same privacy guarantee by up to 70% and 18.2% respectively.

INTERPRETATION

We demonstrate that the ML models trained with DeCaPH framework have an improved utility-privacy trade-off, showing DeCaPH enables the models to have good performance while preserving the privacy of the training data points. In addition, the ML models trained with DeCaPH framework in general outperform those trained solely with the private datasets from individual parties, showing that DeCaPH enhances the model generalizability.

FUNDING

This work was supported by the Natural Sciences and Engineering Research Council of Canada (NSERC, RGPIN-2020-06189 and DGECR-2020-00294), Canadian Institute for Advanced Research (CIFAR) AI Catalyst Grants, CIFAR AI Chair programs, Temerty Professor of AI Research and Education in Medicine, University of Toronto, Amazon, Apple, DARPA through the GARD project, Intel, Meta, the Ontario Early Researcher Award, and the Sloan Foundation. Resources used in preparing this research were provided, in part, by the Province of Ontario, the Government of Canada through CIFAR, and companies sponsoring the Vector Institute.

Grand rounds in methodology: key considerations for implementing machine learning solutions in quality improvement initiatives

Machine learning (ML) solutions are increasingly entering healthcare. They are complex, sociotechnical systems that include data inputs, ML models, technical infrastructure and human interactions. They have promise for improving care across a wide range of clinical applications but if poorly implemented, they may disrupt clinical workflows, exacerbate inequities in care and harm patients. Many aspects of ML solutions are similar to other digital technologies, which have well-established approaches to implementation. However, ML applications present distinct implementation challenges, given that their predictions are often complex and difficult to understand, they can be influenced by biases in the data sets used to develop them, and their impacts on human behaviour are poorly understood. This manuscript summarises the current state of knowledge about implementing ML solutions in clinical care and offers practical guidance for implementation. We propose three overarching questions for potential users to consider when deploying ML solutions in clinical care: (1) Is a clinical or operational problem likely to be addressed by an ML solution? (2) How can an ML solution be evaluated to determine its readiness for deployment? (3) How can an ML solution be deployed and maintained optimally? The Quality Improvement community has an essential role to play in ensuring that ML solutions are translated into clinical practice safely, effectively, and ethically.

Pilot Study of a Software Application to Identify Trauma Registry Inconsistencies

BACKGROUND

Trauma registries are essential to the functioning of modern trauma centers, and high-quality data are necessary to identify patient care issues, develop evidence-based practice, and more. However, institutional experience suggested existing methods to evaluate data quality were insufficient.

OBJECTIVE

This study aims to compare a new software application developed at our trauma center to our existing trauma registry platform on the ability to identify registry inconsistencies (i.e., potential data quality issues).

METHODS

We conducted a pilot retrospective cohort study of patients from September 2019 to August 2020 who underwent chart review during a Level I verification visit and had been audited several times for accuracy. Registry records were processed by both validation systems, and registry inconsistencies were recorded.

RESULTS

In registry data for 63 patients, the new software found 225 registry inconsistencies, and the registry systems found 153 inconsistencies. The most frequent inconsistencies identified by the new software were missing or unknown procedure start times, with 18/63 (28.6%) patients affected and prehospital supplemental oxygen being blank, with 29/53 (54.7%) patients with prehospital care affected. None of the 10 most common inconsistencies detected with the registry systems were true issues.

CONCLUSIONS

This study found the new software application identified 47% more inconsistencies than the standard registry systems, and none of the most frequent inconsistencies detected with the registry systems were true issues pertinent to institutional practice. Centers should consider additional methods to identify registry inconsistencies as existing processes appear insufficient.

Comparative Effectiveness of First-Line and Alternative Antibiotic Regimens in Hospitalized Patients With Nonsevere Community-Acquired Pneumonia: A Multicenter Retrospective Cohort Study

BACKGROUND

There are several antibiotic regimens to treat community-acquired pneumonia (CAP).

RESEARCH QUESTION

In patients hospitalized to a non-ICU ward setting with CAP, is there a difference between first-line and alternative antibiotic regimens (β-lactam plus macrolide [BL+M], β-lactam [BL] alone, respiratory fluoroquinolone [FQ], or β-lactam plus doxycycline [BL+D]) in terms of in-hospital mortality?

STUDY DESIGN AND METHODS

This retrospective cohort study included consecutive patients admitted with CAP at 19 Canadian hospitals from 2015 to 2021. Taking a target trial approach, patients were categorized into the four antibiotic groups based on the initial antibiotic treatment within 48 h of admission. Patients with severe CAP requiring ICU admission in the first 48 h were excluded. The primary outcome was all-cause in-hospital mortality. Secondary outcome included time to being discharged alive. Propensity score and overlap weighting were used to balance covariates.

RESULTS

Of 23,512 patients, 9,340 patients (39.7%) received BL+M, 9,146 (38.9%) received BL, 4,510 (19.2%) received FQ, and 516 (2.2%) received BL+D. The number of in-hospital deaths was 703 (7.5%) for the BL+M group, 888 (9.7%) for the BL group, 302 (6.7%) for the FQ group, and 31 (6.0%) for the BL+D group. The adjusted risk difference for in-hospital mortality when compared with BL+M was 1.5% (95% CI, -0.3% to 3.3%) for BL, -0.9% (95% CI, -2.9% to 1.1%) for FQ, and -1.9% (95% CI, -4.8% to 0.9%) for BL+D. Compared with BL+M, the subdistribution hazard ratio for being discharged alive was 0.90 (95% CI, 0.84-0.96) for BL, 1.07 (95% CI, 0.99-1.16) for FQ, and 1.04 (95% CI, 0.93-1.17) for BL+D.

INTERPRETATION

BL+M, FQ, and BL+D had similar outcomes and can be considered effective regimens for nonsevere CAP. Compared with BL+M, BL was associated with longer time to discharge and the CI for mortality cannot exclude a small but clinically important increase in risk.

An Update to the Kaiser Permanente Inpatient Risk Adjustment Methodology Accurately Predicts In-Hospital Mortality: a Retrospective Cohort Study

BACKGROUND

Methods to accurately predict the risk of in-hospital mortality are important for applications including quality assessment of healthcare institutions and research.

OBJECTIVE

To update and validate the Kaiser Permanente inpatient risk adjustment methodology (KP method) to predict in-hospital mortality, using open-source tools to measure comorbidity and diagnosis groups, and removing troponin which is difficult to standardize across modern clinical assays.

DESIGN

Retrospective cohort study using electronic health record data from GEMINI. GEMINI is a research collaborative that collects administrative and clinical data from hospital information systems.

PARTICIPANTS

Adult general medicine inpatients at 28 hospitals in Ontario, Canada, between April 2010 and December 2022.

MAIN MEASURES

The outcome was in-hospital mortality, modeled by diagnosis group using 56 logistic regressions. We compared models with and without troponin as an input to the laboratory-based acute physiology score. We fit and validated the updated method using internal-external cross-validation at 28 hospitals from April 2015 to December 2022.

KEY RESULTS

In 938,103 hospitalizations with 7.2% in-hospital mortality, the updated KP method accurately predicted the risk of mortality. The c-statistic at the median hospital was 0.866 (see Fig. 3) (25th-75th 0.848-0.876, range 0.816-0.927) and calibration was strong for nearly all patients at all hospitals. The 95th percentile absolute difference between predicted and observed probabilities was 0.038 at the median hospital (25th-75th 0.024-0.057, range 0.006-0.118). Model performance was very similar with and without troponin in a subset of 7 hospitals, and performance was similar with and without troponin for patients hospitalized for heart failure and acute myocardial infarction.

CONCLUSIONS

An update to the KP method accurately predicted in-hospital mortality for general medicine inpatients in 28 hospitals in Ontario, Canada. This updated method can be implemented in a wider range of settings using common open-source tools.

Ceftriaxone and the Risk of Ventricular Arrhythmia, Cardiac Arrest, and Death Among Patients Receiving Lansoprazole

Importance

The combination of ceftriaxone and lansoprazole has been shown to prolong the corrected QT interval on electrocardiogram. However, it is unknown whether this translates to clinically important patient outcomes.

Objective

To compare lansoprazole with another proton pump inhibitor (PPI) during ceftriaxone treatment in terms of risk for ventricular arrhythmia, cardiac arrest, and in-hospital mortality.

Design, Setting, and Participants

A retrospective cohort study including adult medical inpatients receiving ceftriaxone with lansoprazole or another PPI in 13 hospitals in Ontario, Canada, was conducted from January 1, 2015, to December 31, 2021.

Exposure

Lansoprazole during ceftriaxone treatment vs other PPIs during ceftriaxone treatment.

Main Outcomes and Measures

The primary outcome was a composite of ventricular arrhythmia or cardiac arrest that occurred after hospital admission. The secondary outcome was all-cause in-hospital mortality. Propensity-score weighting was used to adjust for covariates including hospital site, demographic characteristics, comorbidities, risk factors for ventricular arrhythmia, illness severity, admitting diagnoses, and concomitant medications.

Results

Of the 31 152 patients hospitalized on internal medicine wards who were treated with ceftriaxone while receiving a PPI, 16 135 patients (51.8%) were male, and the mean (SD) age was 71.7 (16.0) years. The study included 3747 patients in the lansoprazole group and 27 405 patients in the other PPI group. Ventricular arrhythmia or cardiac arrest occurred in 126 patients (3.4%) within the lansoprazole group and 319 patients (1.2%) within the other PPI group. In-hospital mortality occurred in 746 patients (19.9%) within the lansoprazole group and 2762 patients (10.1%) in the other PPI group. After weighting using propensity scores, the adjusted risk difference for the lansoprazole group minus other PPI group was 1.7% (95% CI, 1.1%-2.3%) for ventricular arrhythmia or cardiac arrest and 7.4% (95% CI, 6.1%-8.8%) for in-hospital mortality.

Conclusions and Relevance

The findings of this cohort study suggest that combination therapy with lansoprazole and ceftriaxone should be avoided. More studies are needed to determine whether these findings could be replicated in other populations and settings.

Automated identification of unstandardized medication data: a scalable and flexible data standardization pipeline using RxNorm on GEMINI multicenter hospital data

Objective

Patient data repositories often assemble medication data from multiple sources, necessitating standardization prior to analysis. We implemented and evaluated a medication standardization procedure for use with a wide range of pharmacy data inputs across all drug categories, which supports research queries at multiple levels of granularity.

Methods

The GEMINI-RxNorm system automates the use of multiple RxNorm tools in tandem with other datasets to identify drug concepts from pharmacy orders. GEMINI-RxNorm was used to process 2 090 155 pharmacy orders from 245 258 hospitalizations between 2010 and 2017 at 7 hospitals in Ontario, Canada. The GEMINI-RxNorm system matches drug-identifying information from pharmacy data (including free-text fields) to RxNorm concept identifiers. A user interface allows researchers to search for drug terms and returns the relevant original pharmacy data through the matched RxNorm concepts. Users can then manually validate the predicted matches and discard false positives. We designed the system to maximize recall (sensitivity) and enable excellent precision (positive predictive value) with efficient manual validation. We compared the performance of this system to manual coding (by a physician and pharmacist) of 13 medication classes.

Results

Manual coding was performed for 1 948 817 pharmacy orders and GEMINI-RxNorm successfully returned 1 941 389 (99.6%) orders. Recall was greater than 0.985 in all 13 drug classes, and the F1-score and precision remained above 0.90 in all drug classes, facilitating efficient manual review to achieve 100% precision. GEMINI-RxNorm saved time substantially compared with manual standardization, reducing the time taken to review a pharmacy order row from an estimated 30 to 5 s and reducing the number of rows needed to be reviewed by up to 99.99%.

Discussion and Conclusion

GEMINI-RxNorm presents a novel combination of RxNorm tools and other datasets to enable accurate, efficient, flexible, and scalable standardization of pharmacy data. By facilitating efficient manual validation, the GEMINI-RxNorm system can allow researchers to achieve near-perfect accuracy in medication data standardization.

Electronic health record data quality assessment and tools: a systematic review

OBJECTIVE

We extended a 2013 literature review on electronic health record (EHR) data quality assessment approaches and tools to determine recent improvements or changes in EHR data quality assessment methodologies.

MATERIALS AND METHODS

We completed a systematic review of PubMed articles from 2013 to April 2023 that discussed the quality assessment of EHR data. We screened and reviewed papers for the dimensions and methods defined in the original 2013 manuscript. We categorized papers as data quality outcomes of interest, tools, or opinion pieces. We abstracted and defined additional themes and methods though an iterative review process.

RESULTS

We included 103 papers in the review, of which 73 were data quality outcomes of interest papers, 22 were tools, and 8 were opinion pieces. The most common dimension of data quality assessed was completeness, followed by correctness, concordance, plausibility, and currency. We abstracted conformance and bias as 2 additional dimensions of data quality and structural agreement as an additional methodology.

DISCUSSION

There has been an increase in EHR data quality assessment publications since the original 2013 review. Consistent dimensions of EHR data quality continue to be assessed across applications. Despite consistent patterns of assessment, there still does not exist a standard approach for assessing EHR data quality.

CONCLUSION

Guidelines are needed for EHR data quality assessment to improve the efficiency, transparency, comparability, and interoperability of data quality assessment. These guidelines must be both scalable and flexible. Automation could be helpful in generalizing this process.

Identifying clusters of coexisting conditions and outcomes among adults admitted to hospital with community-acquired pneumonia: a multicentre cohort study

BACKGROUND

Little is known about patterns of coexisting conditions and their influence on clinical care or outcomes in adults admitted to hospital for community-acquired pneumonia (CAP). We sought to evaluate how coexisting conditions cluster in this population to advance understanding of how multimorbidity affects CAP.

METHODS

We studied 11 085 adults admitted to hospital with CAP at 7 hospitals in Ontario, Canada. Using cluster analysis, we identified patient subgroups based on clustering of comorbidities in the Charlson Comorbidity Index. We derived and replicated cluster analyses in independent cohorts (derivation sample 2010-2015, replication sample 2015-2017), then combined these into a total cohort for final cluster analyses. We described differences in medications, imaging and outcomes.

RESULTS

Patients clustered into 7 subgroups. The low comorbidity subgroup (n = 3052, 27.5%) had no comorbidities. The DM-HF-Pulm subgroup had prevalent diabetes, heart failure and chronic lung disease (n = 1710, 15.4%). One disease category defined each remaining subgroup, as follows: pulmonary (n = 1621, 14.6%), diabetes (n = 1281, 11.6%), heart failure (n = 1370, 12.4%), dementia (n = 1038, 9.4%) and cancer (n = 1013, 9.1%). Corticosteroid use ranged from 11.5% to 64.9% in the dementia and pulmonary subgroups, respectively. Piperacillin-tazobactam use ranged from 9.1% to 28.0% in the pulmonary and cancer subgroups, respectively. The use of thoracic computed tomography ranged from 5.7% to 36.3% in the dementia and cancer subgroups, respectively. Adjusting for patient factors, the risk of in-hospital death was greater in the cancer (adjusted odds ratio [OR] 3.12, 95% confidence interval [CI] 2.44-3.99), dementia (adjusted OR 1.57, 95% CI 1.05-2.35), heart failure (adjusted OR 1.66, 95% CI 1.35-2.03) and DM-HF-Pulm subgroups (adjusted OR 1.35, 95% CI 1.12-1.61), and lower in the diabetes subgroup (adjusted OR 0.67, 95% CI 0.50-0.89), compared with the low comorbidity group.

INTERPRETATION

Patients admitted to hospital with CAP cluster into clinically recognizable subgroups based on coexisting conditions. Clinical care and outcomes vary among these subgroups with little evidence to guide decision-making, highlighting opportunities for research to personalize care.

Validity of hospital diagnostic codes to identify SARS-CoV-2 infections in reference to polymerase chain reaction results: a descriptive study

BACKGROUND

In 2020, International Statistical Classification of Diseases and Related Health Problems, 10th Revision (ICD-10) codes were created for laboratory-confirmed SARS-CoV-2 infections. We assessed the operating characteristics of ICD-10 discharge diagnostic code U07.1 within the General Medicine Inpatient Initiative (GEMINI).

METHODS

GEMINI assembles hospitalization data (including administrative ICD-10 discharge diagnostic codes, laboratory results and demographic data) from hospitals in Ontario, Canada. We studied adults (age ≥ 18 yr) admitted during 2020 and tested at least once for SARS-CoV-2 via polymerase chain reaction (PCR) during (or within 48 h before) hospitalization. With PCR results as the reference standard, we calculated sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) for ICD-10 code U07.1 hospital discharge diagnostic codes. Analyses were stratified by demographic data, calendar period and timing of the first test (within or after 48 h of hospital admission).

RESULTS

In 11 852 hospitalizations with at least 1 SARS-CoV-2 PCR test, 444 (3.7%) were positive. The sensitivity of code U07.1 to identify SARS-CoV-2 infection was 97.8%, specificity was 99.5%, PPV was 88.2% and NPV was 99.9%. Operating characteristics were similar in most stratified analyses, but the specificity and PPV were lower if the first SARS-CoV-2 test was done more than 48 hours after admission.

INTERPRETATION

The sensitivity, specificity, PPV and NPV of code U07.1 were high. This supports using code U07.1 to identify SARS-CoV-2 infection in hospitalization data.

Temporal Clustering of Critical Illness Events on Medical Wards

Importance

Recognizing and preventing patient deterioration is important for hospital safety.

Objective

To investigate whether critical illness events (in-hospital death or intensive care unit [ICU] transfer) are associated with greater risk of subsequent critical illness events for other patients on the same medical ward.

Design, Setting, and Participants

Retrospective cohort study in 5 hospitals in Toronto, Canada, including 118 529 hospitalizations. Patients were admitted to general internal medicine wards between April 1, 2010, and October 31, 2017. Data were analyzed between January 1, 2020, and April 10, 2023.

Exposures

Critical illness events (in-hospital death or ICU transfer).

Main Outcomes and Measures

The primary outcome was the composite of in-hospital death or ICU transfer. The association between critical illness events on the same ward across 6-hour intervals was studied using discrete-time survival analysis, adjusting for patient and situational factors. The association between critical illness events on different comparable wards in the same hospital was measured as a negative control.

Results

The cohort included 118 529 hospitalizations (median age, 72 years [IQR, 56-83 years]; 50.7% male). Death or ICU transfer occurred in 8785 hospitalizations (7.4%). Patients were more likely to experience the primary outcome after exposure to 1 prior event (adjusted odds ratio [AOR], 1.39; 95% CI, 1.30-1.48) and more than 1 prior event (AOR, 1.49; 95% CI, 1.33-1.68) in the prior 6-hour interval compared with no exposure. The exposure was associated with increased odds of subsequent ICU transfer (1 event: AOR, 1.67; 95% CI, 1.54-1.81; >1 event: AOR, 2.05; 95% CI, 1.79-2.36) but not death alone (1 event: AOR, 1.08; 95% CI, 0.97-1.19; >1 event: AOR, 0.88; 95% CI, 0.71-1.09). There was no significant association between critical illness events on different wards within the same hospital.

Conclusions and Relevance

Findings of this cohort study suggest that patients are more likely to be transferred to the ICU in the hours after another patient's critical illness event on the same ward. This phenomenon could have several explanations, including increased recognition of critical illness and preemptive ICU transfers, resource diversion to the first event, or fluctuations in ward or ICU capacity. Patient safety may be improved by better understanding the clustering of ICU transfers on medical wards.

Physician antimicrobial prescribing and patient outcomes on general medical wards: a multicentre retrospective cohort study

BACKGROUND

Variability in antimicrobial prescribing may indicate an opportunity for improvement in antimicrobial use. We sought to measure physician-level antimicrobial prescribing in adult general medical wards, assess the contribution of patient-level factors to antimicrobial prescribing and evaluate the association between antimicrobial prescribing and clinical outcomes.

METHODS

Using the General Medicine Inpatient Initiative (GEMINI) database, we conducted a retrospective cohort study of physician-level volume and spectrum of antimicrobial prescribing in adult general medical wards in 4 academic teaching hospitals in Toronto, Ontario, between April 2010 and December 2019. We stratified physicians into quartiles by hospital site based on volume of antimicrobial prescribing (days of therapy per 100 patient-days and antimicrobial-free days) and antibacterial spectrum (modified spectrum score). The modified spectrum score assigns a value to each antibacterial agent based on the breadth of coverage. We assessed patient-level differences among physician quartiles using age, sex, Laboratory-based Acute Physiology Score, discharge diagnosis and Charlson Comorbidity Index. We evaluated the association of clinical outcomes (in-hospital 30-day mortality, length of stay, intensive care unit [ICU] transfer and hospital readmission) with antimicrobial volume and spectrum using multilevel modelling.

RESULTS

The cohort consisted of 124 physicians responsible for 124 158 hospital admissions. The median physician-level volume of antimicrobial prescribing was 56.1 (interquartile range 51.7-67.5) days of therapy per 100 patient-days. We did not find any differences in baseline patient characteristics by physician prescribing quartile. The difference in mean prescribing between quartile 4 and quartile 1 was 15.8 days of therapy per 100 patient-days (95% confidence interval [CI] 9.6-22.0), representing 30% higher antimicrobial prescribing in the fourth quartile than the first quartile. Patient in-hospital deaths, length of stay, ICU transfer and hospital readmission did not differ by physician quartile. In-hospital mortality was higher among patients cared for by prescribers with higher modified spectrum scores (odds ratio 1.13, 95% CI 1.04-1.24).

INTERPRETATION

We found that physician-level variability in antimicrobial prescribing was not associated with differences in patient characteristics or outcomes in academic general medicine wards. These findings provide support for considering the lowest quartile of physician antimicrobial prescribing within each hospital as a target for antimicrobial stewardship.

Using the Hospital Frailty Risk Score to assess mortality risk in older medical patients admitted to the intensive care unit

BACKGROUND

Prognostic information at the time of hospital discharge can help guide goals-of-care discussions for future care. We sought to assess the association between the Hospital Frailty Risk Score (HFRS), which may highlight patients' risk of adverse outcomes at the time of hospital discharge, and in-hospital death among patients admitted to the intensive care unit (ICU) within 12 months of a previous hospital discharge.

METHODS

We conducted a multicentre retrospective cohort study that included patients aged 75 years or older admitted at least twice over a 12-month period to the general medicine service at 7 academic centres and large community-based teaching hospitals in Toronto and Mississauga, Ontario, Canada, from Apr. 1, 2010, to Dec. 31, 2019. The HFRS (categorized as low, moderate or high frailty risk) was calculated at the time of discharge from the first hospital admission. Outcomes included ICU admission and death during the second hospital admission.

RESULTS

The cohort included 22 178 patients, of whom 1767 (8.0%) were categorized as having high frailty risk, 9464 (42.7%) as having moderate frailty risk, and 10 947 (49.4%) as having low frailty risk. One hundred patients (5.7%) with high frailty risk were admitted to the ICU, compared to 566 (6.0%) of those with moderate risk and 790 (7.2%) of those with low risk. After adjustment for age, sex, hospital, day of admission, time of admission and Laboratory-based Acute Physiology Score, the odds of ICU admission were not significantly different for patients with high (adjusted odds ratio [OR] 0.99, 95% confidence interval [CI] 0.78 to 1.23) or moderate (adjusted OR 0.97, 95% CI 0.86 to 1.09) frailty risk compared to those with low frailty risk. Among patients admitted to the ICU, 75 (75.0%) of those with high frailty risk died, compared to 317 (56.0%) of those with moderate risk and 416 (52.7%) of those with low risk. After multivariable adjustment, the risk of death after ICU admission was higher for patients with high frailty risk than for those with low frailty risk (adjusted OR 2.86, 95% CI 1.77 to 4.77).

INTERPRETATION

Among patients readmitted to hospital within 12 months, patients with high frailty risk were similarly likely as those with lower frailty risk to be admitted to the ICU but were more likely to die if admitted to ICU. The HFRS at hospital discharge can inform prognosis, which can help guide discussions for preferences for ICU care during future hospital stays.

Data-driven approach to identifying potential laboratory overuse in general internal medicine (GIM) inpatients

BACKGROUND

Reducing laboratory test overuse is important for high quality, patient-centred care. Identifying priorities to reduce low value testing remains a challenge.

OBJECTIVE

To develop a simple, data-driven approach to identify potential sources of laboratory overuse by combining the total cost, proportion of abnormal results and physician-level variation in use of laboratory tests.

DESIGN, SETTING AND PARTICIPANTS

A multicentre, retrospective study at three academic hospitals in Toronto, Canada. All general internal medicine (GIM) hospitalisations between 1 April 2010 and 31 October 2017.

RESULTS

There were 106 813 GIM hospitalisations during the study period, with median hospital length-of-stay of 4.6 days (IQR: 2.33-9.19). There were 21 tests which had a cumulative cost >US$15 400 at all three sites. The costliest test was plasma electrolytes (US$4 907 775), the test with the lowest proportion of abnormal results was red cell folate (0.2%) and the test with the greatest physician-level variation in use was antiphospholipid antibodies (coefficient of variation 3.08). The five tests with the highest cumulative rank based on greatest cost, lowest proportion of abnormal results and highest physician-level variation were: (1) lactate, (2) antiphospholipid antibodies, (3) magnesium, (4) troponin and (5) partial thromboplastin time. In addition, this method identified unique tests that may be a potential source of laboratory overuse at each hospital.

CONCLUSIONS

A simple multidimensional, data-driven approach combining cost, proportion of abnormal results and physician-level variation can inform interventions to reduce laboratory test overuse. Reducing low value laboratory testing is important to promote high value, patient-centred care.

Osmotic Demyelination Syndrome in Patients Hospitalized with Hyponatremia

BACKGROUND: Osmotic demyelination syndrome (ODS) is a rare but potentially devastating neurologic complication of hyponatremia. The primary objective of this study was to identify the proportion of patients who developed ODS in a large, contemporary, multicenter cohort of patients admitted to the hospital with hyponatremia. METHODS: We conducted a multicenter cohort study of patients admitted with hyponatremia at five academic hospitals in Toronto, Ontario, Canada, between April 1, 2010, and December 31, 2020. All adult patients presenting with hyponatremia (serum sodium level 8 mmol/l in any 24-hour period). RESULTS: Our cohort included 22,858 hospitalizations with hyponatremia. Approximately 50% were women, the average age was 68 years, and mean initial serum sodium was 125 mmol/l (standard deviation, 4.6), including 11.9% with serum sodium from 110 to 119 mmol/l and 1.2% with serum sodium less than 110 mmol/l. Overall, rapid correction of serum sodium occurred in 3632 (17.7%) admissions. Twelve patients developed ODS (0.05%). Seven (58%) patients who developed ODS did not have rapid correction of serum sodium. CONCLUSIONS: In this large multicenter study of patients with hyponatremia, rapid correction of serum sodium was common (n=3632 [17.7%]), but ODS was rare (n=12 [0.05%]). Future studies with a higher number of patients with ODS are needed to better understand potential causal factors for ODS.

Predictive model for early functional outcomes following acute care after traumatic brain injuries: A machine learning-based development and validation study

INTRODUCTION

Few studies on early functional outcomes following acute care after traumatic brain injury (TBI) are available. The aim of this study was to develop and validate a predictive model for functional outcomes at discharge for TBI patients using machine learning methods.

PATIENTS AND METHODS

In this retrospective study, data from 5281 TBI patients admitted for acute care who were identified in the Beijing hospital discharge abstract database were analysed. Data from 4181 patients in 52 tertiary hospitals were used for model derivation and internal validation. Data from 1100 patients in 21 secondary hospitals were used for external validation. A poor outcome was defined as a Barthel Index (BI) score ≤ 60 at discharge. Logistic regression, XGBoost, random forest, decision tree, and back propagation neural network models were used to fit classification models. Performance was evaluated by the area under the receiver operating characteristic curve (AUC), the area under the precision-recall curve (AP), calibration plots, sensitivity/recall, specificity, positive predictive value (PPV)/precision, negative predictive value (NPV) and F1-score.

RESULTS

Compared to the other models, the random forest model demonstrated superior performance in internal validation (AUC of 0.856, AP of 0.786, and F1-score of 0.724) and external validation (AUC of 0.779, AP of 0.630, and F1-score of 0.604). The sensitivity/recall, specificity, PPV/precision, and NPV of the model were 71.8%, 69.2%, 52.2%, and 84.0%, respectively, in external validation. The BI score at admission, age, use of nonsurgical treatment, neurosurgery status, and modified Charlson Comorbidity Index were identified as the top 5 predictors for functional outcome at discharge.

CONCLUSIONS

We established a random forest model that performed well in predicting early functional outcomes following acute care after TBI. The model has utility for informing decision-making regarding patient management and discharge planning and for facilitating health care quality assessment and resource allocation for TBI treatment.

Incorporating economic analysis in interventional cardiology research

Evaluative research in interventional cardiology has focused on clinical and technical outcomes. Inclusion of economic data can enhance evaluative research by quantifying the relative economic burden incurred by different therapies. When combined with clinical outcomes, cost data can provide a measure of value (e.g., marginal cost-effectiveness). In some select situations, cost data can also be used as surrogates for complexity of care and morbidity. In this narrative review, we aim to provide a framework for the application of cost data in clinical trials and observational research, detailing how to incorporate this kind of data into interventional cardiology research.

Variations in long-term care home resident hospitalizations before and during the COVID-19 pandemic in Ontario

OBJECTIVES

To examine how the COVID-19 pandemic affected the demographic and clinical characteristics, in-hospital care, and outcomes of long-term care residents admitted to general medicine wards for non-COVID-19 reasons.

METHODS

We conducted a retrospective cohort study of long-term care residents admitted to general medicine wards, for reasons other than COVID-19, in four hospitals in Toronto, Ontario between January 1, 2018 and December 31, 2020. We used an autoregressive linear model to estimate the change in monthly admission volumes during the pandemic period (March-December 2020) compared to the previous two years, adjusting for any secular trend. We summarized and compared differences in the demographics, comorbidities, interventions, diagnoses, imaging, psychoactive medications, and outcomes of residents before and during the pandemic.

RESULTS

Our study included 2,654 long-term care residents who were hospitalized for non-COVID-19 reasons between January 2018 and December 2020. The crude rate of hospitalizations was 79.3 per month between March-December of 2018-2019 and 56.5 per month between March-December of 2020. The was an adjusted absolute difference of 27.0 (95% CI: 10.0, 43.9) fewer hospital admissions during the pandemic period, corresponding to a relative drop of 34%. Residents admitted during the pandemic period had similar demographics and clinical characteristics but were more likely to be admitted for delirium (pandemic: 7% pre-pandemic: 5%, p = 0.01) and were less likely to be admitted for pneumonia (pandemic: 3% pre-pandemic: 6%, p = 0.004). Residents admitted during the pandemic were more likely to be prescribed antipsychotics (pandemic: 37%, pre-pandemic: 29%, p <0.001) and more likely to die in-hospital (pandemic:14% pre-pandemic: 10%, p = 0.04).

CONCLUSIONS AND IMPLICATIONS

Better integration between long-term care and hospitals systems, including programs to deliver urgent medical care services within long-term care homes, is needed to ensure that long-term care residents maintain equitable access to acute care during current and future public health emergencies.

Predictors of ICU admission associated with gastrointestinal endoscopy in medical inpatients: A retrospective cohort study

BACKGROUND AND AIM

Gastrointestinal (GI) endoscopic procedures are commonly performed in medical inpatients. Limited prior research has examined factors associated with intensive care unit (ICU) admission after GI endoscopy in medical inpatients.

METHODS

This retrospective cohort study was conducted using routinely-collected clinical and administrative data from all general medicine hospitalizations at five academic hospitals in Toronto, Canada between 2010 and 2020. We describe ICU admission and death within 48 h of GI endoscopy in medical inpatients. We examined adjusted associations of patient and procedural factors with ICU admission or death using multivariable logistic regression.

RESULTS

Among 18 290 medical inpatients who underwent endoscopy, 900 (4.9%) required ICU admission or died within 48 h of endoscopy. Following risk adjustment, ICU admission or death were associated with the following procedural factors: endoscopy on the day of hospital admission (aOR 3.16 [2.38-4.21]) or 1 day after admission (aOR 1.92 [1.51-2.44]) and esophagogastroduodenoscopy (EGD) procedures; and the following patient factors: Charlson comorbidity index of two (aOR 1.38 [1.05-1.81]) or three or greater (aOR 1.84 [1.47-2.29]), older age, male sex, lower hemoglobin prior to endoscopy, increased creatinine prior to endoscopy, an admitting diagnosis of liver disease and certain medications (antiplatelet agents and corticosteroids).

CONCLUSIONS

ICU admission or death after endoscopy was associated with procedural factors such as EGD and timing of endoscopy, and patient factors indicative of acute illness and greater comorbidity. These findings can contribute to improved triage and monitoring for patients requiring inpatient endoscopy.

Methodology paper for the General Medicine Inpatient Initiative Medical Education Database (GEMINI MedED): a retrospective cohort study of internal medicine resident case-mix, clinical care and patient outcomes

INTRODUCTION

Unwarranted variation in patient care among physicians is associated with negative patient outcomes and increased healthcare costs. Care variation likely also exists for resident physicians. Despite the global movement towards outcomes-based and competency-based medical education, current assessment strategies in residency do not routinely incorporate clinical outcomes. The widespread use of electronic health records (EHRs) may enable the implementation of in-training assessments that incorporate clinical care and patient outcomes.

METHODS AND ANALYSIS

The General Medicine Inpatient Initiative Medical Education Database (GEMINI MedED) is a retrospective cohort study of senior residents (postgraduate year 2/3) enrolled in the University of Toronto Internal Medicine (IM) programme between 1 April 2010 and 31 December 2020. This study focuses on senior IM residents and patients they admit overnight to four academic hospitals. Senior IM residents are responsible for overseeing all overnight admissions; thus, care processes and outcomes for these clinical encounters can be at least partially attributed to the care they provide. Call schedules from each hospital, which list the date, location and senior resident on-call, will be used to link senior residents to EHR data of patients admitted during their on-call shifts. Patient data will be derived from the GEMINI database, which contains administrative (eg, demographic and disposition) and clinical data (eg, laboratory and radiological investigation results) for patients admitted to IM at the four academic hospitals. Overall, this study will examine three domains of resident practice: (1) case-mix variation across residents, hospitals and academic year, (2) resident-sensitive quality measures (EHR-derived metrics that are partially attributable to resident care) and (3) variations in patient outcomes across residents and factors that contribute to such variation.

ETHICS AND DISSEMINATION

GEMINI MedED was approved by the University of Toronto Ethics Board (RIS#39339). Results from this study will be presented in academic conferences and peer-reviewed journals.

Timing errors and temporal uncertainty in clinical databases-A narrative review

A firm concept of time is essential for establishing causality in a clinical setting. Review of critical incidents and generation of study hypotheses require a robust understanding of the sequence of events but conducting such work can be problematic when timestamps are recorded by independent and unsynchronized clocks. Most clinical models implicitly assume that timestamps have been measured accurately and precisely, but this custom will need to be re-evaluated if our algorithms and models are to make meaningful use of higher frequency physiological data sources. In this narrative review we explore factors that can result in timestamps being erroneously recorded in a clinical setting, with particular focus on systems that may be present in a critical care unit. We discuss how clocks, medical devices, data storage systems, algorithmic effects, human factors, and other external systems may affect the accuracy and precision of recorded timestamps. The concept of temporal uncertainty is introduced, and a holistic approach to timing accuracy, precision, and uncertainty is proposed. This quantitative approach to modeling temporal uncertainty provides a basis to achieve enhanced model generalizability and improved analytical outcomes.

Outcomes for upper gastrointestinal bleeding during the first wave of the COVID-19 pandemic in the Toronto area

BACKGROUND AND AIM

Changes to endoscopy service availability during the COVID-19 pandemic may have affected management of upper gastrointestinal bleeding (UGIB). The aim of this study was to describe the impact of the pandemic on UGIB outcomes in the Toronto area in Canada.

METHODS

We described all adults admitted to general medicine wards or intensive care units at six hospitals in Toronto and Mississauga, Canada, with UGIB during the first wave of the COVID-19 pandemic (March 1 to June 30, 2020) and compared them with a historical cohort (March 1 to June 30, 2018 and 2019). We compared clinical outcomes (in-hospital mortality, length of stay, 30-day readmission, intensive care utilization, receipt of endoscopy, persistent bleeding, receipt of second endoscopy, and need for angiographic or surgical intervention) using multivariable regression models, controlling for demographics, comorbidities, and severity of clinical presentation.

RESULTS

There were 82.5 and 215.5 admissions per month for UGIB during the COVID-19 and control periods, respectively. There were no baseline differences between groups for demographic characteristics, comorbidities, or severity of bleeding. Patients in the COVID-19 group did not have significantly different unadjusted (3.9% vs 4.2%, P = 0.983) or adjusted mortality (adjusted odds ratio [OR] = 0.64, 95% confidence interval [CI] = 0.25-1.48, P = 0.322). Patients in COVID-19 group were less likely to receive endoscopy for UGIB in the unadjusted (61.8% vs 71.0%, P = 0.003) and adjusted (adjusted OR = 0.64, 95% CI = 0.49-0.84, P < 0.01) models. There were no differences between groups for other secondary outcomes.

CONCLUSIONS

While patients admitted for UGIB during the first wave of the pandemic were less likely to receive endoscopy, this had no impact on mortality or any secondary outcomes.

Outcomes in patients with and without disability admitted to hospital with COVID-19: a retrospective cohort study

Background:

Disability-related considerations have largely been absent from the COVID-19 response, despite evidence that people with disabilities are at elevated risk for acquiring COVID-19. We evaluated clinical outcomes in patients who were admitted to hospital with COVID-19 with a disability compared with patients without a disability.

Methods:

We conducted a retrospective cohort study that included adults with COVID-19 who were admitted to hospital and discharged between Jan. 1, 2020, and Nov. 30, 2020, at 7 hospitals in Ontario, Canada. We compared in-hospital death, admission to the intensive care unit (ICU), hospital length of stay and unplanned 30-day readmission among patients with and without a physical disability, hearing or vision impairment, traumatic brain injury, or intellectual or developmental disability, overall and stratified by age (≤ 64 and ≥ 65 yr) using multivariable regression, controlling for sex, residence in a long-term care facility and comorbidity.

Results:

Among 1279 admissions to hospital for COVID-19, 22.3% had a disability. We found that patients with a disability were more likely to die than those without a disability (28.1% v. 17.6%), had longer hospital stays (median 13.9 v. 7.8 d) and more readmissions (17.6% v. 7.9%), but had lower ICU admission rates (22.5% v. 28.3%). After adjustment, there were no statistically significant differences between those with and without disabilities for in-hospital death or admission to ICU. After adjustment, patients with a disability had longer hospital stays (rate ratio 1.36, 95% confidence interval [CI] 1.19–1.56) and greater risk of readmission (relative risk 1.77, 95% CI 1.14–2.75). In age-stratified analyses, we observed longer hospital stays among patients with a disability than in those without, in both younger and older subgroups; readmission risk was driven by younger patients with a disability.

Interpretation:

Patients with a disability who were admitted to hospital with COVID-19 had longer stays and elevated readmission risk than those without disabilities. Disability-related needs should be addressed to support these patients in hospital and after discharge.

Physician Experience Design (PXD): More Usable Machine Learning Prediction for Clinical Decision Making

Delirium is an acute neurocognitive disorder, which is difficult to identify and predict. Using GEMINI, Canada's largest hospital data and analytics study, we had a labeled sample of around 4,000 cases with approximately 25% of cases being labeled as having delirium. Based on this labeled data, we developed machine learning (ML) models and interacted with physicians to interpret the ML models and their predictions. We developed a preliminary Explainable Artificial Intelligence (XAI) framework for physician experience design (PXD) to improve the uptake of ML models by improving the transparency of model results, thereby increasing physician trust in models as well as the uptake of model results for clinical decision making. We developed our PXD approach first with Conceptual Investigation to collect and extract physicians' feedback on ML models and their evaluation requirements. We carried out a case study, working closely with the physicians in a participatory design process to develop a dashboard that presents ML delirium identification results interactively based on physician selections and inputs. In this approach a physician-preferred ML model for clinical decision making is selected through PXD evaluation.

Developing and validating natural language processing algorithms for radiology reports compared to ICD-10 codes for identifying venous thromboembolism in hospitalized medical patients

BACKGROUND

Identifying venous thromboembolism (VTE) from large clinical and administrative databases is important for research and quality improvement.

OBJECTIVE

To develop and validate natural language processing (NLP) algorithms to identify VTE from radiology reports among general internal medicine (GIM) inpatients.

METHODS

This cross-sectional study included GIM hospitalizations between April 1, 2010 and March 31, 2017 at 5 hospitals in Toronto, Ontario, Canada. We developed NLP algorithms to identify pulmonary embolism (PE) and deep venous thrombosis (DVT) from radiologist reports of thoracic computed tomography (CT), extremity compression ultrasound (US), and nuclear ventilation-perfusion (VQ) scans in a training dataset of 1551 hospitalizations. We compared the accuracy of our NLP algorithms, the previously-published "simpleNLP" tool, and administrative discharge diagnosis codes (ICD-10-CA) for PE and DVT to the "gold standard" manual review in a separate random sample of 4000 GIM hospitalizations.

RESULTS

Our NLP algorithms were highly accurate for identifying DVT from US, with sensitivity 0.94, positive predictive value (PPV) 0.90, and Area Under the Receiver-Operating-Characteristic Curve (AUC) 0.96; and in identifying PE from CT, with sensitivity 0.91, PPV 0.89, and AUC 0.96. Administrative diagnosis codes and the simple NLP tool were less accurate for DVT (ICD-10-CA sensitivity 0.63, PPV 0.43, AUC 0.81; simpleNLP sensitivity 0.41, PPV 0.36, AUC 0.66) and PE (ICD-10-CA sensitivity 0.83, PPV 0.70, AUC 0.91; simpleNLP sensitivity 0.89, PPV 0.62, AUC 0.92).

CONCLUSIONS

Administrative diagnosis codes are unreliable in identifying VTE in hospitalized patients. We developed highly accurate NLP algorithms to identify VTE from radiology reports in a multicentre sample and have made the algorithms freely available to the academic community with a user-friendly tool (https://lks-chart.github.io/CHARTextract-docs/08-downloads/rulesets.html#venous-thromboembolism-vte-rulesets).

Assessment of RHIS Quality Assurance Practices in Tarkwa Submunicipal Health Directorate, Ghana

Background. Routine health information system (RHIS) quality assurance has become an important issue, not only because of its significance in promoting high standard of patient care, but also because of its impact on government budgets for the maintenance of health services. Routine health information system comprises healthcare data collection, compilation, storage, analysis, report generation, and dissemination on routine basis at the various healthcare settings. The data from RHIS give a representation of health status, health services, and health resources. The sources of RHIS data are normally individual health records, records of services delivered, and records of health resources. Using reliable information from routine health information systems is fundamental in the healthcare delivery system. Quality assurance practices are measures that are put in places to ensure the health data that are collected meet required quality standards. Routine health information system quality assurance practices ensure that data that are generated from the system are fit for use. This study considered quality assurance practices in the RHIS processes. Methods. A cross-sectional study was conducted in eight health facilities in Tarkwa Submunicipal health service in the western region of Ghana. The study involved routine quality assurance practices among the 90-health staff and management selected from facilities in Tarkwa Submunicipal who collect or use data routinely from 24th December, 2019, to 20th January, 2020. Results. Generally, Tarkwa Submunicipal health service appears to practice quality assurance during data collection, compilation, storage, analysis, and dissemination. The results show some achievement in quality control performance in report dissemination (77.6%), data analysis (68.0%), data compilation (67.4%), report compilation (66.3%), data storage (66.3%), and collection (61.1%). Conclusions. Even though Tarkwa Submunicipal health directorate engages some control measures to ensure data quality, there is the need to strengthen the process to achieve the targeted percentage of performance (90.0%). There was significant shortfall in quality assurance practices performance especially during data collection, with respect to the expected performance.

Internal medicine hospitalisations and liver disease: a comparative disease burden analysis of a multicentre cohort

BACKGROUND

Liver disease is an increasing burden on population health globally.

AIMS

To characterise burden of liver disease among general internal medicine inpatients at seven Toronto-area hospitals and compare it to other common medical conditions.

METHODS

Data from April 2010 to October 2017 were obtained from hospitals participating in the GEMINI collaborative. Using these cohort data from hospital information systems linked to administrative data, we defined liver disease admissions using most responsible discharge diagnoses categorised according to international classification of diseases, 10th Revision-enhanced Canadian version (ICD-10-CA). We identified admissions for heart failure, chronic obstructive pulmonary disease (COPD) and pneumonia as comparators. We calculated standardised mortality ratios (SMRs) as the ratio of observed to expected deaths.

RESULTS

Among 239 018 discharges, liver disease accounted for 1.7% of most responsible discharge diagnoses. Liver disease was associated with marked premature mortality, with SMR of 8.84 (95% CI 8.06-9.67) compared to 1.06 (95% CI 0.99-1.12) for heart failure, 1.05 (95% CI 0.96-1.15) for COPD and 1.28 (95% CI 1.20-1.37) for pneumonia. The majority of deaths were among patients younger than 65 years (57.7%) compared to 3.3% in heart failure, 5.6% in COPD and 10.7% in pneumonia. Liver disease patients presented with worse Laboratory-Based Acute Physiology Scores, were more frequently admitted to the intensive care unit (14.4%), incurred higher average total costs (median $6723 CAD), had higher in-hospital mortality (11.4%), and were more likely to be a readmission from 30 days prior (19.8%). Non-alcoholic fatty liver disease admissions increased from 120 in 2011-2012 to 215 in 2016-2017 (P < 0.01).

CONCLUSION

In Canada's largest urban centre, liver disease admissions resulted in premature morbidity and mortality with higher resource use compared to common cardio-respiratory conditions. Re-evaluation of approaches to caring for inpatients with liver disease is timely and justified.

Caractéristiques et issues des hospitalisations pour les cas de COVID-19 et d’influenza dans la région de Toronto

CONTEXTE:

Les caractéristiques des patients, les soins cliniques, l’utilisation des ressources et les issues cliniques des personnes atteintes de la maladie à coronavirus 2019 (COVID-19) hospitalisées au Canada ne sont pas bien connus.

MÉTHODES:

Nous avons recueilli des données sur tous les adultes hospitalisés atteints de la COVID-19 ou de l’influenza ayant obtenu leur congé d’unités médicales ou d’unités de soins intensifs médicaux et chirurgicaux entre le 1^er novembre 2019 et le 30 juin 2020 dans 7 centres hospitaliers de Toronto et de Mississauga (Ontario). Nous avons comparé les issues cliniques des patients à l’aide de modèles de régression multivariée, en tenant compte des facteurs sociodémographiques et de l’intensité des comorbidités. Nous avons validé le degré d’exactitude de 7 scores de risque mis au point à l’externe pour déterminer leur capacité à prédire le risque de décès chez les patients atteints de la COVID-19.

RÉSULTATS:

Parmi les hospitalisations retenues, 1027 patients étaient atteints de la COVID-19 (âge médian de 65 ans, 59,1 % d’hommes) et 783 étaient atteints de l’influenza (âge médian de 68 ans, 50,8 % d’hommes). Les patients âgés de moins de 50 ans comptaient pour 21,2 % de toutes les hospitalisations dues à la COVID-19 et 24,0 % des séjours aux soins intensifs. Comparativement aux patients atteints de l’influenza, les patients atteints de la COVID-19 présentaient un taux de mortalité perhospitalière (mortalité non ajustée 19,9 % c. 6,1 %; risque relatif [RR] ajusté 3,46 %, intervalle de confiance [IC] à 95 % 2,56–4,68) et un taux d’utilisation des ressources des unités de soins intensifs (taux non ajusté 26,4 % c. 18,0 %; RR ajusté 1,50, IC à 95 % 1,25–1,80) significativement plus élevés, ainsi qu’une durée d’hospitalisation (durée médiane non ajustée 8,7 jours c. 4,8 jours; rapport des taux d’incidence ajusté 1,45; IC à 95 % 1,25–1,69) significativement plus longue. Le taux de réhospitalisation dans les 30 jours n’était pas significativement différent (taux non ajusté 9,3 % c. 9,6 %; RR ajusté 0,98 %, IC à 95 % 0,70–1,39). Trois scores de risque utilisant un pointage pour prédire la mortalité perhospitalière ont montré une bonne discrimination (aire sous la courbe [ASC] de la fonction d’efficacité du récepteur [ROC] 0,72–0,81) et une bonne calibration.

INTERPRÉTATION:

Durant la première vague de la pandémie, l’hospitalisation des patients atteints de la COVID-19 était associée à des taux de mortalité et d’utilisation des ressources des unités de soins intensifs et à une durée d’hospitalisation significativement plus importants que les hospitalisations des patients atteints de l’influenza. De simples scores de risque peuvent prédire avec une bonne exactitude le risque de mortalité perhospitalière des patients atteints de la COVID-19.

Association of diabetes with frequency and cost of hospital admissions: a retrospective cohort study

BACKGROUND

Acute inpatient hospital admissions account for more than half of all health care costs related to diabetes. We sought to identify the most common and costly conditions leading to hospital admission among patients with diabetes compared with patients without diabetes.

METHODS

We used data from the General Internal Medicine Inpatient Initiative (GEMINI) study, a retrospective cohort study, of all patients admitted to a general internal medicine service at 7 Toronto hospitals between 2010 and 2015. The Canadian Institute for Health Information (CIHI) Most Responsible Diagnosis code was used to identify the 10 most frequent reasons for admission in patients with diabetes. Cost of hospital admission was estimated using the CIHI Resource Intensity Weight. Comparisons were made between patients with or without diabetes using the Pearson χ² test for frequency and distribution-free confidence intervals (CIs) for median cost.

RESULTS

Among the 150 499 hospital admissions in our study, 41 934 (27.8%) involved patients with diabetes. Compared with patients without diabetes, hospital admissions because of soft tissue and bone infections were most frequent (2.5% v. 1.9%; prevalence ratio [PR] 1.28, 95% CI 1.19-1.37) and costly (Can$8794 v. Can$5845; cost ratio [CR] 1.50, 95% CI 1.37-1.65) among patients with diabetes. This was followed by urinary tract infections (PR 1.16, 95% CI 1.11-1.22; CR 1.23, 95% CI 1.17-1.29), stroke (PR 1.13, 95% CI 1.07-1.19; CR 1.19, 95% CI 1.14-1.25) and electrolyte disorders (PR 1.11, 95% CI 1.03-1.20; CR 1.20, 95% CI 1.08-1.34).

INTERPRETATION

Soft tissue and bone infections, urinary tract infections, stroke and electrolyte disorders are associated with a greater frequency and cost of hospital admissions in patients with diabetes than in those without diabetes. Preventive strategies focused on reducing hospital admissions secondary to these disorders may be beneficial in patients with diabetes.

Characteristics and outcomes of hospital admissions for COVID-19 and influenza in the Toronto area

BACKGROUND:

Patient characteristics, clinical care, resource use and outcomes associated with admission to hospital for coronavirus disease 2019 (COVID-19) in Canada are not well described.

METHODS:

We described all adults with COVID-19 or influenza discharged from inpatient medical services and medical–surgical intensive care units (ICUs) between Nov. 1, 2019, and June 30, 2020, at 7 hospitals in Toronto and Mississauga, Ontario. We compared patient outcomes using multivariable regression models, controlling for patient sociodemographic factors and comorbidity level. We validated the accuracy of 7 externally developed risk scores to predict mortality among patients with COVID-19.

RESULTS:

There were 1027 hospital admissions with COVID-19 (median age 65 yr, 59.1% male) and 783 with influenza (median age 68 yr, 50.8% male). Patients younger than 50 years accounted for 21.2% of all admissions for COVID-19 and 24.0% of ICU admissions. Compared with influenza, patients with COVID-19 had significantly greater in-hospital mortality (unadjusted 19.9% v. 6.1%, adjusted relative risk [RR] 3.46, 95% confidence interval [CI] 2.56–4.68), ICU use (unadjusted 26.4% v. 18.0%, adjusted RR 1.50, 95% CI 1.25–1.80) and hospital length of stay (unadjusted median 8.7 d v. 4.8 d, adjusted rate ratio 1.45, 95% CI 1.25–1.69). Thirty-day readmission was not significantly different (unadjusted 9.3% v. 9.6%, adjusted RR 0.98, 95% CI 0.70–1.39). Three points-based risk scores for predicting in-hospital mortality showed good discrimination (area under the receiver operating characteristic curve [AUC] ranging from 0.72 to 0.81) and calibration.

INTERPRETATION:

During the first wave of the pandemic, admission to hospital for COVID-19 was associated with significantly greater mortality, ICU use and hospital length of stay than influenza. Simple risk scores can predict in-hospital mortality in patients with COVID-19 with good accuracy.

Modelling resource requirements and physician staffing to provide virtual urgent medical care for residents of long-term care homes: a cross-sectional study

BACKGROUND

The coronavirus disease 2019 (COVID-19) outbreak increases the importance of strategies to enhance urgent medical care delivery in long-term care (LTC) facilities that could potentially reduce transfers to emergency departments. The study objective was to model resource requirements to deliver virtual urgent medical care in LTC facilities.

METHODS

We used data from all general medicine inpatient admissions at 7 hospitals in the Greater Toronto Area, Ontario, Canada, over a 7.5-year period (Apr. 1, 2010, to Oct. 31, 2017) to estimate historical patterns of hospital resource use by LTC residents. We estimated an upper bound of potentially avoidable transfers by combining data on short admissions (≤ 72 h) with historical data on the proportion of transfers from LTC facilities for which patients were discharged from the emergency department without admission. Regression models were used to extrapolate future resource requirements, and queuing models were used to estimate physician staffing requirements to perform virtual assessments.

RESULTS

There were 235 375 admissions to general medicine wards, and residents of LTC facilities (age 16 yr or older) accounted for 9.3% (n = 21 948) of these admissions. Among the admissions of residents of LTC facilities, short admissions constituted 24.1% (n = 5297), and for 99.8% (n = 5284) of these admissions, the patient received laboratory testing, for 86.9% (n = 4604) the patient received plain radiography, for 41.5% (n = 2197) the patient received computed tomography and for 81.2% (n = 4300) the patient received intravenous medications. If all patients who have short admissions and are transferred from the emergency department were diverted to outpatient care, the average weekly demand for outpatient imaging per hospital would be 2.6 ultrasounds, 11.9 computed tomographic scans and 23.9 radiographs per week. The average daily volume of urgent medical virtual assessments would range from 2.0 to 5.8 per hospital. A single centralized virtual assessment centre staffed by 2 or 3 physicians would provide services similar in efficiency (measured by waiting time for physician assessment) to 7 separate centres staffed by 1 physician each.

INTERPRETATION

The provision of acute medical care to LTC residents at their facility would probably require rapid access to outpatient diagnostic imaging, within-facility access to laboratory services and intravenous medication and virtual consultations with physicians. The results of this study can inform efforts to deliver urgent medical care in LTC facilities in light of a potential surge in COVID-19 cases.