Nursing Workflow Change in a COVID-19 Inpatient Unit Following the Deployment of Inpatient Telehealth: Observational Study Using a Real-Time Locating System

BACKGROUND

The COVID-19 pandemic prompted widespread implementation of telehealth, including in the inpatient setting, with the goals to reduce potential pathogen exposure events and personal protective equipment (PPE) utilization. Nursing workflow adaptations in these novel environments are of particular interest given the association between nursing time at the bedside and patient safety. Understanding the frequency and duration of nurse-patient encounters following the introduction of a novel telehealth platform in the context of COVID-19 may therefore provide insight into downstream impacts on patient safety, pathogen exposure, and PPE utilization.

OBJECTIVE

The aim of this study was to evaluate changes in nursing workflow relative to prepandemic levels using a real-time locating system (RTLS) following the deployment of inpatient telehealth on a COVID-19 unit.

METHODS

In March 2020, telehealth was installed in patient rooms in a COVID-19 unit and on movable carts in 3 comparison units. The existing RTLS captured nurse movement during 1 pre- and 5 postpandemic stages (January-December 2020). Change in direct nurse-patient encounters, time spent in patient rooms per encounter, and total time spent with patients per shift relative to baseline were calculated. Generalized linear models assessed difference-in-differences in outcomes between COVID-19 and comparison units. Telehealth adoption was captured and reported at the unit level.

RESULTS

Change in frequency of encounters and time spent per encounter from baseline differed between the COVID-19 and comparison units at all stages of the pandemic (all P<.001). Frequency of encounters decreased (difference-in-differences range -6.6 to -14.1 encounters) and duration of encounters increased (difference-in-differences range 1.8 to 6.2 minutes) from baseline to a greater extent in the COVID-19 units relative to the comparison units. At most stages of the pandemic, the change in total time nurses spent in patient rooms per patient per shift from baseline did not differ between the COVID-19 and comparison units (all P>.17). The primary COVID-19 unit quickly adopted telehealth technology during the observation period, initiating 15,088 encounters that averaged 6.6 minutes (SD 13.6) each.

CONCLUSIONS

RTLS movement data suggest that total nursing time at the bedside remained unchanged following the deployment of inpatient telehealth in a COVID-19 unit. Compared to other units with shared mobile telehealth units, the frequency of nurse-patient in-person encounters decreased and the duration lengthened on a COVID-19 unit with in-room telehealth availability, indicating "batched" redistribution of work to maintain total time at bedside relative to prepandemic periods. The simultaneous adoption of telehealth suggests that virtual care was a complement to, rather than a replacement for, in-person care. However, study limitations preclude our ability to draw a causal link between nursing workflow change and telehealth adoption. Thus, further evaluation is needed to determine potential downstream implications on disease transmission, PPE utilization, and patient safety.

Monitoring Approaches for a Pediatric Chronic Kidney Disease Machine Learning Model

OBJECTIVE

The purpose of this study is to evaluate the ability of three metrics to monitor for a reduction in performance of a chronic kidney disease (CKD) model deployed at a pediatric hospital.

METHODS

The CKD risk model estimates a patient's risk of developing CKD 3 to 12 months following an inpatient admission. The model was developed on a retrospective dataset of 4,879 admissions from 2014 to 2018, then run silently on 1,270 admissions from April to October, 2019. Three metrics were used to monitor its performance during the silent phase: (1) standardized mean differences (SMDs); (2) performance of a "membership model"; and (3) response distribution analysis. Observed patient outcomes for the 1,270 admissions were used to calculate prospective model performance and the ability of the three metrics to detect performance changes.

RESULTS

The deployed model had an area under the receiver-operator curve (AUROC) of 0.63 in the prospective evaluation, which was a significant decrease from an AUROC of 0.76 on retrospective data (p = 0.033). Among the three metrics, SMDs were significantly different for 66/75 (88%) of the model's input variables (p <0.05) between retrospective and deployment data. The membership model was able to discriminate between the two settings (AUROC = 0.71, p <0.0001) and the response distributions were significantly different (p <0.0001) for the two settings.

CONCLUSION

This study suggests that the three metrics examined could provide early indication of performance deterioration in deployed models' performance.

DLMM as a lossless one-shot algorithm for collaborative multi-site distributed linear mixed models

Linear mixed models are commonly used in healthcare-based association analyses for analyzing multi-site data with heterogeneous site-specific random effects. Due to regulations for protecting patients’ privacy, sensitive individual patient data (IPD) typically cannot be shared across sites. We propose an algorithm for fitting distributed linear mixed models (DLMMs) without sharing IPD across sites. This algorithm achieves results identical to those achieved using pooled IPD from multiple sites (i.e., the same effect size and standard error estimates), hence demonstrating the lossless property. The algorithm requires each site to contribute minimal aggregated data in only one round of communication. We demonstrate the lossless property of the proposed DLMM algorithm by investigating the associations between demographic and clinical characteristics and length of hospital stay in COVID-19 patients using administrative claims from the UnitedHealth Group Clinical Discovery Database. We extend this association study by incorporating 120,609 COVID-19 patients from 11 collaborative data sources worldwide.

A lossless, one-shot and privacy-preserving distributed algorithm was revealed for fitting linear mixed models on multi-site data. The algorithm was applied to a study of 120,609 COVID-19 patients using only minimal aggregated data from each of 14 sites.

Building a Learning Health System: Creating an Analytical Workflow for Evidence Generation to Inform Institutional Clinical Care Guidelines

Background  One key aspect of a learning health system (LHS) is utilizing data generated during care delivery to inform clinical care. However, institutional guidelines that utilize observational data are rare and require months to create, making current processes impractical for more urgent scenarios such as those posed by the COVID-19 pandemic. There exists a need to rapidly analyze institutional data to drive guideline creation where evidence from randomized control trials are unavailable.

Objectives  This article provides a background on the current state of observational data generation in institutional guideline creation and details our institution's experience in creating a novel workflow to (1) demonstrate the value of such a workflow, (2) demonstrate a real-world example, and (3) discuss difficulties encountered and future directions.

Methods  Utilizing a multidisciplinary team of database specialists, clinicians, and informaticists, we created a workflow for identifying and translating a clinical need into a queryable format in our clinical data warehouse, creating data summaries and feeding this information back into clinical guideline creation.

Results  Clinical questions posed by the hospital medicine division were answered in a rapid time frame and informed creation of institutional guidelines for the care of patients with COVID-19. The cost of setting up a workflow, answering the questions, and producing data summaries required around 300 hours of effort and $300,000 USD.

Conclusion  A key component of an LHS is the ability to learn from data generated during care delivery. There are rare examples in the literature and we demonstrate one such example along with proposed thoughts of ideal multidisciplinary team formation and deployment.

Predictors of diagnostic transition from major depressive disorder to bipolar disorder: a retrospective observational network study

Many patients with bipolar disorder (BD) are initially misdiagnosed with major depressive disorder (MDD) and are treated with antidepressants, whose potential iatrogenic effects are widely discussed. It is unknown whether MDD is a comorbidity of BD or its earlier stage, and no consensus exists on individual conversion predictors, delaying BD's timely recognition and treatment. We aimed to build a predictive model of MDD to BD conversion and to validate it across a multi-national network of patient databases using the standardization afforded by the Observational Medical Outcomes Partnership (OMOP) common data model. Five "training" US databases were retrospectively analyzed: IBM MarketScan CCAE, MDCR, MDCD, Optum EHR, and Optum Claims. Cyclops regularized logistic regression models were developed on one-year MDD-BD conversion with all standard covariates from the HADES PatientLevelPrediction package. Time-to-conversion Kaplan-Meier analysis was performed up to a decade after MDD, stratified by model-estimated risk. External validation of the final prediction model was performed across 9 patient record databases within the Observational Health Data Sciences and Informatics (OHDSI) network internationally. The model's area under the curve (AUC) varied 0.633-0.745 (µ = 0.689) across the five US training databases. Nine variables predicted one-year MDD-BD transition. Factors that increased risk were: younger age, severe depression, psychosis, anxiety, substance misuse, self-harm thoughts/actions, and prior mental disorder. AUCs of the validation datasets ranged 0.570-0.785 (µ = 0.664). An assessment algorithm was built for MDD to BD conversion that allows distinguishing as much as 100-fold risk differences among patients and validates well across multiple international data sources.

Unsupervised Learning for Automated Detection of Coronary Artery Disease Subgroups

Background The promise of precision population health includes the ability to use robust patient data to tailor prevention and care to specific groups. Advanced analytics may allow for automated detection of clinically informative subgroups that account for clinical, genetic, and environmental variability. This study sought to evaluate whether unsupervised machine learning approaches could interpret heterogeneous and missing clinical data to discover clinically important coronary artery disease subgroups. Methods and Results The Genetic Determinants of Peripheral Arterial Disease study is a prospective cohort that includes individuals with newly diagnosed and/or symptomatic coronary artery disease. We applied generalized low rank modeling and K-means cluster analysis using 155 phenotypic and genetic variables from 1329 participants. Cox proportional hazard models were used to examine associations between clusters and major adverse cardiovascular and cerebrovascular events and all-cause mortality. We then compared performance of risk stratification based on clusters and the American College of Cardiology/American Heart Association pooled cohort equations. Unsupervised analysis identified 4 phenotypically and prognostically distinct clusters. All-cause mortality was highest in cluster 1 (oldest/most comorbid; 26%), whereas major adverse cardiovascular and cerebrovascular event rates were highest in cluster 2 (youngest/multiethnic; 41%). Cluster 4 (middle-aged/healthiest behaviors) experienced more incident major adverse cardiovascular and cerebrovascular events (30%) than cluster 3 (middle-aged/lowest medication adherence; 23%), despite apparently similar risk factor and lifestyle profiles. In comparison with the pooled cohort equations, cluster membership was more informative for risk assessment of myocardial infarction, stroke, and mortality. Conclusions Unsupervised clustering identified 4 unique coronary artery disease subgroups with distinct clinical trajectories. Flexible unsupervised machine learning algorithms offer the ability to meaningfully process heterogeneous patient data and provide sharper insights into disease characterization and risk assessment. Registration URL: https://www.clinicaltrials.gov; Unique identifier: NCT00380185.

Computational drug repositioning of atorvastatin for ulcerative colitis

OBJECTIVE

Ulcerative colitis (UC) is a chronic inflammatory disorder with limited effective therapeutic options for long-term treatment and disease maintenance. We hypothesized that a multi-cohort analysis of independent cohorts representing real-world heterogeneity of UC would identify a robust transcriptomic signature to improve identification of FDA-approved drugs that can be repurposed to treat patients with UC.

MATERIALS AND METHODS

We performed a multi-cohort analysis of 272 colon biopsy transcriptome samples across 11 publicly available datasets to identify a robust UC disease gene signature. We compared the gene signature to in vitro transcriptomic profiles induced by 781 FDA-approved drugs to identify potential drug targets. We used a retrospective cohort study design modeled after a target trial to evaluate the protective effect of predicted drugs on colectomy risk in patients with UC from the Stanford Research Repository (STARR) database and Optum Clinformatics DataMart.

RESULTS

Atorvastatin treatment had the highest inverse-correlation with the UC gene signature among non-oncolytic FDA-approved therapies. In both STARR (n = 827) and Optum (n = 7821), atorvastatin intake was significantly associated with a decreased risk of colectomy, a marker of treatment-refractory disease, compared to patients prescribed a comparator drug (STARR: HR = 0.47, P = .03; Optum: HR = 0.66, P = .03), irrespective of age and length of atorvastatin treatment.

DISCUSSION & CONCLUSION

These findings suggest that atorvastatin may serve as a novel therapeutic option for ameliorating disease in patients with UC. Importantly, we provide a systematic framework for integrating publicly available heterogeneous molecular data with clinical data at a large scale to repurpose existing FDA-approved drugs for a wide range of human diseases.

An informatics consult approach for generating clinical evidence for treatment decisions

BACKGROUND

An Informatics Consult has been proposed in which clinicians request novel evidence from large scale health data resources, tailored to the treatment of a specific patient. However, the availability of such consultations is lacking. We seek to provide an Informatics Consult for a situation where a treatment indication and contraindication coexist in the same patient, i.e., anti-coagulation use for stroke prevention in a patient with both atrial fibrillation (AF) and liver cirrhosis.

METHODS

We examined four sources of evidence for the effect of warfarin on stroke risk or all-cause mortality from: (1) randomised controlled trials (RCTs), (2) meta-analysis of prior observational studies, (3) trial emulation (using population electronic health records (N = 3,854,710) and (4) genetic evidence (Mendelian randomisation). We developed prototype forms to request an Informatics Consult and return of results in electronic health record systems.

RESULTS

We found 0 RCT reports and 0 trials recruiting for patients with AF and cirrhosis. We found broad concordance across the three new sources of evidence we generated. Meta-analysis of prior observational studies showed that warfarin use was associated with lower stroke risk (hazard ratio [HR] = 0.71, CI 0.39-1.29). In a target trial emulation, warfarin was associated with lower all-cause mortality (HR = 0.61, CI 0.49-0.76) and ischaemic stroke (HR = 0.27, CI 0.08-0.91). Mendelian randomisation served as a drug target validation where we found that lower levels of vitamin K1 (warfarin is a vitamin K1 antagonist) are associated with lower stroke risk. A pilot survey with an independent sample of 34 clinicians revealed that 85% of clinicians found information on prognosis useful and that 79% thought that they should have access to the Informatics Consult as a service within their healthcare systems. We identified candidate steps for automation to scale evidence generation and to accelerate the return of results.

CONCLUSION

We performed a proof-of-concept Informatics Consult for evidence generation, which may inform treatment decisions in situations where there is dearth of randomised trials. Patients are surprised to know that their clinicians are currently not able to learn in clinic from data on 'patients like me'. We identify the key challenges in offering such an Informatics Consult as a service.

A survey of extant organizational and computational setups for deploying predictive models in health systems

OBJECTIVE

Artificial intelligence (AI) and machine learning (ML) enabled healthcare is now feasible for many health systems, yet little is known about effective strategies of system architecture and governance mechanisms for implementation. Our objective was to identify the different computational and organizational setups that early-adopter health systems have utilized to integrate AI/ML clinical decision support (AI-CDS) and scrutinize their trade-offs.

MATERIALS AND METHODS

We conducted structured interviews with health systems with AI deployment experience about their organizational and computational setups for deploying AI-CDS at point of care.

RESULTS

We contacted 34 health systems and interviewed 20 healthcare sites (58% response rate). Twelve (60%) sites used the native electronic health record vendor configuration for model development and deployment, making it the most common shared infrastructure. Nine (45%) sites used alternative computational configurations which varied significantly. Organizational configurations for managing AI-CDS were distinguished by how they identified model needs, built and implemented models, and were separable into 3 major types: Decentralized translation (n = 10, 50%), IT Department led (n = 2, 10%), and AI in Healthcare (AIHC) Team (n = 8, 40%).

DISCUSSION

No singular computational configuration enables all current use cases for AI-CDS. Health systems need to consider their desired applications for AI-CDS and whether investment in extending the off-the-shelf infrastructure is needed. Each organizational setup confers trade-offs for health systems planning strategies to implement AI-CDS.

CONCLUSION

Health systems will be able to use this framework to understand strengths and weaknesses of alternative organizational and computational setups when designing their strategy for artificial intelligence.

COVID-19 in patients with autoimmune diseases: characteristics and outcomes in a multinational network of cohorts across three countries

OBJECTIVE

Patients with autoimmune diseases were advised to shield to avoid coronavirus disease 2019 (COVID-19), but information on their prognosis is lacking. We characterized 30-day outcomes and mortality after hospitalization with COVID-19 among patients with prevalent autoimmune diseases, and compared outcomes after hospital admissions among similar patients with seasonal influenza.

METHODS

A multinational network cohort study was conducted using electronic health records data from Columbia University Irving Medical Center [USA, Optum (USA), Department of Veterans Affairs (USA), Information System for Research in Primary Care-Hospitalization Linked Data (Spain) and claims data from IQVIA Open Claims (USA) and Health Insurance and Review Assessment (South Korea). All patients with prevalent autoimmune diseases, diagnosed and/or hospitalized between January and June 2020 with COVID-19, and similar patients hospitalized with influenza in 2017-18 were included. Outcomes were death and complications within 30 days of hospitalization.

RESULTS

We studied 133 589 patients diagnosed and 48 418 hospitalized with COVID-19 with prevalent autoimmune diseases. Most patients were female, aged ≥50 years with previous comorbidities. The prevalence of hypertension (45.5-93.2%), chronic kidney disease (14.0-52.7%) and heart disease (29.0-83.8%) was higher in hospitalized vs diagnosed patients with COVID-19. Compared with 70 660 hospitalized with influenza, those admitted with COVID-19 had more respiratory complications including pneumonia and acute respiratory distress syndrome, and higher 30-day mortality (2.2-4.3% vs 6.32-24.6%).

CONCLUSION

Compared with influenza, COVID-19 is a more severe disease, leading to more complications and higher mortality.

Characteristics and Outcomes of Over 300,000 Patients with COVID-19 and History of Cancer in the United States and Spain

BACKGROUND

We described the demographics, cancer subtypes, comorbidities, and outcomes of patients with a history of cancer and coronavirus disease 2019 (COVID-19). Second, we compared patients hospitalized with COVID-19 to patients diagnosed with COVID-19 and patients hospitalized with influenza.

METHODS

We conducted a cohort study using eight routinely collected health care databases from Spain and the United States, standardized to the Observational Medical Outcome Partnership common data model. Three cohorts of patients with a history of cancer were included: (i) diagnosed with COVID-19, (ii) hospitalized with COVID-19, and (iii) hospitalized with influenza in 2017 to 2018. Patients were followed from index date to 30 days or death. We reported demographics, cancer subtypes, comorbidities, and 30-day outcomes.

RESULTS

We included 366,050 and 119,597 patients diagnosed and hospitalized with COVID-19, respectively. Prostate and breast cancers were the most frequent cancers (range: 5%-18% and 1%-14% in the diagnosed cohort, respectively). Hematologic malignancies were also frequent, with non-Hodgkin's lymphoma being among the five most common cancer subtypes in the diagnosed cohort. Overall, patients were aged above 65 years and had multiple comorbidities. Occurrence of death ranged from 2% to 14% and from 6% to 26% in the diagnosed and hospitalized COVID-19 cohorts, respectively. Patients hospitalized with influenza (n = 67,743) had a similar distribution of cancer subtypes, sex, age, and comorbidities but lower occurrence of adverse events.

CONCLUSIONS

Patients with a history of cancer and COVID-19 had multiple comorbidities and a high occurrence of COVID-19-related events. Hematologic malignancies were frequent.

IMPACT

This study provides epidemiologic characteristics that can inform clinical care and etiologic studies.

Thirty-Day Outcomes of Children and Adolescents With COVID-19: An International Experience

OBJECTIVES

To characterize the demographics, comorbidities, symptoms, in-hospital treatments, and health outcomes among children and adolescents diagnosed or hospitalized with coronavirus disease 2019 (COVID-19) and to compare them in secondary analyses with patients diagnosed with previous seasonal influenza in 2017-2018.

METHODS

International network cohort using real-world data from European primary care records (France, Germany, and Spain), South Korean claims and US claims, and hospital databases. We included children and adolescents diagnosed and/or hospitalized with COVID-19 at age <18 between January and June 2020. We described baseline demographics, comorbidities, symptoms, 30-day in-hospital treatments, and outcomes including hospitalization, pneumonia, acute respiratory distress syndrome, multisystem inflammatory syndrome in children, and death.

RESULTS

A total of 242 158 children and adolescents diagnosed and 9769 hospitalized with COVID-19 and 2 084 180 diagnosed with influenza were studied. Comorbidities including neurodevelopmental disorders, heart disease, and cancer were more common among those hospitalized with versus diagnosed with COVID-19. Dyspnea, bronchiolitis, anosmia, and gastrointestinal symptoms were more common in COVID-19 than influenza. In-hospital prevalent treatments for COVID-19 included repurposed medications (<10%) and adjunctive therapies: systemic corticosteroids (6.8%-7.6%), famotidine (9.0%-28.1%), and antithrombotics such as aspirin (2.0%-21.4%), heparin (2.2%-18.1%), and enoxaparin (2.8%-14.8%). Hospitalization was observed in 0.3% to 1.3% of the cohort diagnosed with COVID-19, with undetectable (n < 5 per database) 30-day fatality. Thirty-day outcomes including pneumonia and hypoxemia were more frequent in COVID-19 than influenza.

CONCLUSIONS

Despite negligible fatality, complications including hospitalization, hypoxemia, and pneumonia were more frequent in children and adolescents with COVID-19 than with influenza. Dyspnea, anosmia, and gastrointestinal symptoms could help differentiate diagnoses. A wide range of medications was used for the inpatient management of pediatric COVID-19.

Treatment and Monitoring Variability in US Metastatic Breast Cancer Care

PURPOSE

Treatment and monitoring options for patients with metastatic breast cancer (MBC) are increasing, but little is known about variability in care. We sought to improve understanding of MBC care and its correlates by analyzing real-world claims data using a search engine with a novel query language to enable temporal electronic phenotyping.

METHODS

Using the Advanced Cohort Engine, we identified 6,180 women who met criteria for having estrogen receptor-positive, human epidermal growth factor receptor 2-negative MBC from IBM MarketScan US insurance claims (2007-2014). We characterized treatment, monitoring, and hospice usage, along with clinical and nonclinical factors affecting care.

RESULTS

We observed wide variability in treatment modality and monitoring across patients and geography. Most women received first-recorded therapy with endocrine (67%) versus chemotherapy, underwent more computed tomography (CT) (76%) than positron emission tomography-CT, and were monitored using tumor markers (58%). Nearly half (46%) met criteria for aggressive disease, which were associated with receiving chemotherapy first, monitoring primarily with CT, and more frequent imaging. Older age was associated with endocrine therapy first, less frequent imaging, and less use of tumor markers. After controlling for clinical factors, care strategies varied significantly by nonclinical factors (median regional income with first-recorded therapy and imaging type, geographic region with these and with imaging frequency and use of tumor markers; P < .0001).

CONCLUSION

Variability in US MBC care is explained by patient and disease factors and by nonclinical factors such as geographic region, suggesting that treatment decisions are influenced by local practice patterns and/or resources. A search engine designed to express complex electronic phenotypes from longitudinal patient records enables the identification of variability in patient care, helping to define disparities and areas for improvement.

Automated model versus treating physician for predicting survival time of patients with metastatic cancer

OBJECTIVE

Being able to predict a patient's life expectancy can help doctors and patients prioritize treatments and supportive care. For predicting life expectancy, physicians have been shown to outperform traditional models that use only a few predictor variables. It is possible that a machine learning model that uses many predictor variables and diverse data sources from the electronic medical record can improve on physicians' performance. For patients with metastatic cancer, we compared accuracy of life expectancy predictions by the treating physician, a machine learning model, and a traditional model.

MATERIALS AND METHODS

A machine learning model was trained using 14 600 metastatic cancer patients' data to predict each patient's distribution of survival time. Data sources included note text, laboratory values, and vital signs. From 2015-2016, 899 patients receiving radiotherapy for metastatic cancer were enrolled in a study in which their radiation oncologist estimated life expectancy. Survival predictions were also made by the machine learning model and a traditional model using only performance status. Performance was assessed with area under the curve for 1-year survival and calibration plots.

RESULTS

The radiotherapy study included 1190 treatment courses in 899 patients. A total of 879 treatment courses in 685 patients were included in this analysis. Median overall survival was 11.7 months. Physicians, machine learning model, and traditional model had area under the curve for 1-year survival of 0.72 (95% CI 0.63-0.81), 0.77 (0.73-0.81), and 0.68 (0.65-0.71), respectively.

CONCLUSIONS

The machine learning model's predictions were more accurate than those of the treating physician or a traditional model.

Learning decision thresholds for risk stratification models from aggregate clinician behavior

Using a risk stratification model to guide clinical practice often requires the choice of a cutoff—called the decision threshold—on the model’s output to trigger a subsequent action such as an electronic alert. Choosing this cutoff is not always straightforward. We propose a flexible approach that leverages the collective information in treatment decisions made in real life to learn reference decision thresholds from physician practice. Using the example of prescribing a statin for primary prevention of cardiovascular disease based on 10-year risk calculated by the 2013 pooled cohort equations, we demonstrate the feasibility of using real-world data to learn the implicit decision threshold that reflects existing physician behavior. Learning a decision threshold in this manner allows for evaluation of a proposed operating point against the threshold reflective of the community standard of care. Furthermore, this approach can be used to monitor and audit model-guided clinical decision making following model deployment.

Characteristics and outcomes of 627 044 COVID-19 patients living with and without obesity in the United States, Spain, and the United Kingdom

BACKGROUND

A detailed characterization of patients with COVID-19 living with obesity has not yet been undertaken. We aimed to describe and compare the demographics, medical conditions, and outcomes of COVID-19 patients living with obesity (PLWO) to those of patients living without obesity.

METHODS

We conducted a cohort study based on outpatient/inpatient care and claims data from January to June 2020 from Spain, the UK, and the US. We used six databases standardized to the OMOP common data model. We defined two non-mutually exclusive cohorts of patients diagnosed and/or hospitalized with COVID-19; patients were followed from index date to 30 days or death. We report the frequency of demographics, prior medical conditions, and 30-days outcomes (hospitalization, events, and death) by obesity status.

RESULTS

We included 627 044 (Spain: 122 058, UK: 2336, and US: 502 650) diagnosed and 160 013 (Spain: 18 197, US: 141 816) hospitalized patients with COVID-19. The prevalence of obesity was higher among patients hospitalized (39.9%, 95%CI: 39.8-40.0) than among those diagnosed with COVID-19 (33.1%; 95%CI: 33.0-33.2). In both cohorts, PLWO were more often female. Hospitalized PLWO were younger than patients without obesity. Overall, COVID-19 PLWO were more likely to have prior medical conditions, present with cardiovascular and respiratory events during hospitalization, or require intensive services compared to COVID-19 patients without obesity.

CONCLUSION

We show that PLWO differ from patients without obesity in a wide range of medical conditions and present with more severe forms of COVID-19, with higher hospitalization rates and intensive services requirements. These findings can help guiding preventive strategies of COVID-19 infection and complications and generating hypotheses for causal inference studies.

ACE: the Advanced Cohort Engine for searching longitudinal patient records

OBJECTIVE

To propose a paradigm for a scalable time-aware clinical data search, and to describe the design, implementation and use of a search engine realizing this paradigm.

MATERIALS AND METHODS

The Advanced Cohort Engine (ACE) uses a temporal query language and in-memory datastore of patient objects to provide a fast, scalable, and expressive time-aware search. ACE accepts data in the Observational Medicine Outcomes Partnership Common Data Model, and is configurable to balance performance with compute cost. ACE's temporal query language supports automatic query expansion using clinical knowledge graphs. The ACE API can be used with R, Python, Java, HTTP, and a Web UI.

RESULTS

ACE offers an expressive query language for complex temporal search across many clinical data types with multiple output options. ACE enables electronic phenotyping and cohort-building with subsecond response times in searching the data of millions of patients for a variety of use cases.

DISCUSSION

ACE enables fast, time-aware search using a patient object-centric datastore, thereby overcoming many technical and design shortcomings of relational algebra-based querying. Integrating electronic phenotype development with cohort-building enables a variety of high-value uses for a learning health system. Tradeoffs include the need to learn a new query language and the technical setup burden.

CONCLUSION

ACE is a tool that combines a unique query language for time-aware search of longitudinal patient records with a patient object datastore for rapid electronic phenotyping, cohort extraction, and exploratory data analyses.

Improving hospital readmission prediction using individualized utility analysis

OBJECTIVE

Machine learning (ML) models for allocating readmission-mitigating interventions are typically selected according to their discriminative ability, which may not necessarily translate into utility in allocation of resources. Our objective was to determine whether ML models for allocating readmission-mitigating interventions have different usefulness based on their overall utility and discriminative ability.

MATERIALS AND METHODS

We conducted a retrospective utility analysis of ML models using claims data acquired from the Optum Clinformatics Data Mart, including 513,495 commercially-insured inpatients (mean [SD] age 69 [19] years; 294,895 [57%] Female) over the period January 2016 through January 2017 from all 50 states with mean 90 day cost of $11,552. Utility analysis estimates the cost, in dollars, of allocating interventions for lowering readmission risk based on the reduction in the 90-day cost.

RESULTS

Allocating readmission-mitigating interventions based on a GBDT model trained to predict readmissions achieved an estimated utility gain of $104 per patient, and an AUC of 0.76 (95% CI 0.76, 0.77); allocating interventions based on a model trained to predict cost as a proxy achieved a higher utility of $175.94 per patient, and an AUC of 0.62 (95% CI 0.61, 0.62). A hybrid model combining both intervention strategies is comparable with the best models on either metric. Estimated utility varies by intervention cost and efficacy, with each model performing the best under different intervention settings.

CONCLUSION

We demonstrate that machine learning models may be ranked differently based on overall utility and discriminative ability. Machine learning models for allocation of limited health resources should consider directly optimizing for utility.

Use of repurposed and adjuvant drugs in hospital patients with covid-19: multinational network cohort study

OBJECTIVE

To investigate the use of repurposed and adjuvant drugs in patients admitted to hospital with covid-19 across three continents.

DESIGN

Multinational network cohort study.

SETTING

Hospital electronic health records from the United States, Spain, and China, and nationwide claims data from South Korea.

PARTICIPANTS

303 264 patients admitted to hospital with covid-19 from January 2020 to December 2020.

MAIN OUTCOME MEASURES

Prescriptions or dispensations of any drug on or 30 days after the date of hospital admission for covid-19.

RESULTS

Of the 303 264 patients included, 290 131 were from the US, 7599 from South Korea, 5230 from Spain, and 304 from China. 3455 drugs were identified. Common repurposed drugs were hydroxychloroquine (used in from <5 (<2%) patients in China to 2165 (85.1%) in Spain), azithromycin (from 15 (4.9%) in China to 1473 (57.9%) in Spain), combined lopinavir and ritonavir (from 156 (<2%) in the VA-OMOP US to 2,652 (34.9%) in South Korea and 1285 (50.5%) in Spain), and umifenovir (0% in the US, South Korea, and Spain and 238 (78.3%) in China). Use of adjunctive drugs varied greatly, with the five most used treatments being enoxaparin, fluoroquinolones, ceftriaxone, vitamin D, and corticosteroids. Hydroxychloroquine use increased rapidly from March to April 2020 but declined steeply in May to June and remained low for the rest of the year. The use of dexamethasone and corticosteroids increased steadily during 2020.

CONCLUSIONS

Multiple drugs were used in the first few months of the covid-19 pandemic, with substantial geographical and temporal variation. Hydroxychloroquine, azithromycin, lopinavir-ritonavir, and umifenovir (in China only) were the most prescribed repurposed drugs. Antithrombotics, antibiotics, H2 receptor antagonists, and corticosteroids were often used as adjunctive treatments. Research is needed on the comparative risk and benefit of these treatments in the management of covid-19.

MINIMAR (MINimum Information for Medical AI Reporting): Developing reporting standards for artificial intelligence in health care

The rise of digital data and computing power have contributed to significant advancements in artificial intelligence (AI), leading to the use of classification and prediction models in health care to enhance clinical decision-making for diagnosis, treatment and prognosis. However, such advances are limited by the lack of reporting standards for the data used to develop those models, the model architecture, and the model evaluation and validation processes. Here, we present MINIMAR (MINimum Information for Medical AI Reporting), a proposal describing the minimum information necessary to understand intended predictions, target populations, and hidden biases, and the ability to generalize these emerging technologies. We call for a standard to accurately and responsibly report on AI in health care. This will facilitate the design and implementation of these models and promote the development and use of associated clinical decision support tools, as well as manage concerns regarding accuracy and bias.

Ontology-driven weak supervision for clinical entity classification in electronic health records

In the electronic health record, using clinical notes to identify entities such as disorders and their temporality (e.g. the order of an event relative to a time index) can inform many important analyses. However, creating training data for clinical entity tasks is time consuming and sharing labeled data is challenging due to privacy concerns. The information needs of the COVID-19 pandemic highlight the need for agile methods of training machine learning models for clinical notes. We present Trove, a framework for weakly supervised entity classification using medical ontologies and expert-generated rules. Our approach, unlike hand-labeled notes, is easy to share and modify, while offering performance comparable to learning from manually labeled training data. In this work, we validate our framework on six benchmark tasks and demonstrate Trove's ability to analyze the records of patients visiting the emergency department at Stanford Health Care for COVID-19 presenting symptoms and risk factors.

Development and validation of phenotype classifiers across multiple sites in the observational health data sciences and informatics network

Objective

Accurate electronic phenotyping is essential to support collaborative observational research. Supervised machine learning methods can be used to train phenotype classifiers in a high-throughput manner using imperfectly labeled data. We developed 10 phenotype classifiers using this approach and evaluated performance across multiple sites within the Observational Health Data Sciences and Informatics (OHDSI) network.

Materials and Methods

We constructed classifiers using the Automated PHenotype Routine for Observational Definition, Identification, Training and Evaluation (APHRODITE) R-package, an open-source framework for learning phenotype classifiers using datasets in the Observational Medical Outcomes Partnership Common Data Model. We labeled training data based on the presence of multiple mentions of disease-specific codes. Performance was evaluated on cohorts derived using rule-based definitions and real-world disease prevalence. Classifiers were developed and evaluated across 3 medical centers, including 1 international site.

Results

Compared to the multiple mentions labeling heuristic, classifiers showed a mean recall boost of 0.43 with a mean precision loss of 0.17. Performance decreased slightly when classifiers were shared across medical centers, with mean recall and precision decreasing by 0.08 and 0.01, respectively, at a site within the USA, and by 0.18 and 0.10, respectively, at an international site.

Discussion and Conclusion

We demonstrate a high-throughput pipeline for constructing and sharing phenotype classifiers across sites within the OHDSI network using APHRODITE. Classifiers exhibit good portability between sites within the USA, however limited portability internationally, indicating that classifier generalizability may have geographic limitations, and, consequently, sharing the classifier-building recipe, rather than the pretrained classifiers, may be more useful for facilitating collaborative observational research.

Assessment of Extractability and Accuracy of Electronic Health Record Data for Joint Implant Registries

IMPORTANCE

Implant registries provide valuable information on the performance of implants in a real-world setting, yet they have traditionally been expensive to establish and maintain. Electronic health records (EHRs) are widely used and may include the information needed to generate clinically meaningful reports similar to a formal implant registry.

OBJECTIVES

To quantify the extractability and accuracy of registry-relevant data from the EHR and to assess the ability of these data to track trends in implant use and the durability of implants (hereafter referred to as implant survivorship), using data stored since 2000 in the EHR of the largest integrated health care system in the United States.

DESIGN, SETTING, AND PARTICIPANTS

Retrospective cohort study of a large EHR of veterans who had 45 351 total hip arthroplasty procedures in Veterans Health Administration hospitals from 2000 to 2017. Data analysis was performed from January 1, 2000, to December 31, 2017.

EXPOSURES

Total hip arthroplasty.

MAIN OUTCOMES AND MEASURES

Number of total hip arthroplasty procedures extracted from the EHR, trends in implant use, and relative survivorship of implants.

RESULTS

A total of 45 351 total hip arthroplasty procedures were identified from 2000 to 2017 with 192 805 implant parts. Data completeness improved over the time. After 2014, 85% of prosthetic heads, 91% of shells, 81% of stems, and 85% of liners used in the Veterans Health Administration health care system were identified by part number. Revision burden and trends in metal vs ceramic prosthetic femoral head use were found to reflect data from the American Joint Replacement Registry. Recalled implants were obvious negative outliers in implant survivorship using Kaplan-Meier curves.

CONCLUSIONS AND RELEVANCE

Although loss to follow-up remains a challenge that requires additional attention to improve the quantitative nature of calculated implant survivorship, we conclude that data collected during routine clinical care and stored in the EHR of a large health system over 18 years were sufficient to provide clinically meaningful data on trends in implant use and to identify poor implants that were subsequently recalled. This automated approach was low cost and had no reporting burden. This low-cost, low-overhead method to assess implant use and performance within a large health care setting may be useful to internal quality assurance programs and, on a larger scale, to postmarket surveillance of implant performance.

The number needed to benefit: estimating the value of predictive analytics in healthcare

Predictive analytics in health care has generated increasing enthusiasm recently, as reflected in a rapidly growing body of predictive models reported in literature and in real-time embedded models using electronic health record data. However, estimating the benefit of applying any single model to a specific clinical problem remains challenging today. Developing a shared framework for estimating model value is therefore critical to facilitate the effective, safe, and sustainable use of predictive tools into the future. We highlight key concepts within the prediction-action dyad that together are expected to impact model benefit. These include factors relevant to model prediction (including the number needed to screen) as well as those relevant to the subsequent action (number needed to treat). In the simplest terms, a number needed to benefit contextualizes the numbers needed to screen and treat, offering an opportunity to estimate the value of a clinical predictive model in action.