Performance of a computable phenotype for identification of patients with diabetes within PCORnet: The Patient-Centered Clinical Research Network

Using electronic health records for clinical pharmacology research: Challenges and considerations

Electronic health records (EHRs) contain a vast array of phenotypic data on large numbers of individuals, often collected over decades. Due to the wealth of information, EHR data have emerged as a powerful resource to make first discoveries and identify disparities in our healthcare system. While the number of EHR-based studies has exploded in recent years, most of these studies are directed at associations with disease rather than pharmacotherapeutic outcomes, such as drug response or adverse drug reactions. This is largely due to challenges specific to deriving drug-related phenotypes from the EHR. There is great potential for EHR-based discovery in clinical pharmacology research, and there is a critical need to address specific challenges related to accurate and reproducible derivation of drug-related phenotypes from the EHR. This review provides a detailed evaluation of challenges and considerations for deriving drug-related data from EHRs. We provide an examination of EHR-based computable phenotypes and discuss cutting-edge approaches to map medication information for clinical pharmacology research, including medication-based computable phenotypes and natural language processing. We also discuss additional considerations such as data structure, heterogeneity and missing data, rare phenotypes, and diversity within the EHR. By further understanding the complexities associated with conducting clinical pharmacology research using EHR-based data, investigators will be better equipped to design thoughtful studies with more reproducible results. Progress in utilizing EHRs for clinical pharmacology research should lead to significant advances in our ability to understand differential drug response and predict adverse drug reactions.

Performance of Phenotype Algorithms for the Identification of Opioid-Exposed Infants

OBJECTIVE

Observational studies examining outcomes among opioid-exposed infants are limited by phenotype algorithms that may under identify opioid-exposed infants without neonatal opioid withdrawal syndrome (NOWS). We developed and validated the performance of different phenotype algorithms to identify opioid-exposed infants using electronic health record data.

METHODS

We developed phenotype algorithms for the identification of opioid-exposed infants among a population of birthing person-infant dyads from an academic health care system (2010-2022). We derived phenotype algorithms from combinations of 6 unique indicators of in utero opioid exposure, including those from the infant record (NOWS or opioid-exposure diagnosis, positive toxicology) and birthing person record (opioid use disorder diagnosis, opioid drug exposure record, opioid listed on medication reconciliation, positive toxicology). We determined the positive predictive value (PPV) and 95% confidence interval for each phenotype algorithm using medical record review as the gold standard.

RESULTS

Among 41 047 dyads meeting exclusion criteria, we identified 1558 infants (3.80%) with evidence of at least 1 indicator for opioid exposure and 32 (0.08%) meeting all 6 indicators of the phenotype algorithm. Among the sample of dyads randomly selected for review (n = 600), the PPV for the phenotype requiring only a single indicator was 95.4% (confidence interval: 93.3-96.8) with varying PPVs for the other phenotype algorithms derived from a combination of infant and birthing person indicators (PPV range: 95.4-100.0).

CONCLUSIONS

Opioid-exposed infants can be accurately identified using electronic health record data. Our publicly available phenotype algorithms can be used to conduct research examining outcomes among opioid-exposed infants with and without NOWS.

Rurality, Cardiovascular Risk Factors, and Early Cardiovascular Disease among Childhood, Adolescent, and Young Adult Cancer Survivors

Background and Aims

Cardiovascular risk factors (CVRFs) later in life potentiate risk for late cardiovascular disease (CVD) from cardiotoxic treatment among survivors. This study evaluated the association of baseline CVRFs and CVD in the early survivorship period.

Methods

This analysis included patients ages 0-29 at initial diagnosis and reported in the institutional cancer registry between 2010 and 2017 (n = 1228). Patients who died within five years (n = 168), those not seen in the oncology clinic (n = 312), and those with CVD within one year of diagnosis (n = 17) were excluded. CVRFs (hypertension, diabetes, dyslipidemia, and obesity) within one year of initial diagnosis, were constructed and extracted from the electronic health record based on discrete observations, ICD9/10 codes, and RxNorm codes for antihypertensives.

Results

Among survivors (n = 731), 10 incident cases (1.4%) of CVD were observed between one year and five years after the initial diagnosis. Public health insurance (p = 0.04) and late effects risk strata (p = 0.01) were positively associated with CVD. Among survivors with public insurance(n = 495), two additional cases of CVD were identified from claims data with an incidence of 2.4%. Survivors from rural areas had a 4.1 times greater risk of CVD compared with survivors from urban areas (95% CI: 1.1-15.3), despite adjustment for late effects risk strata.

Conclusions

Clinically computable phenotypes for CVRFs among survivors through informatics methods were feasible. Although CVRFs were not associated with CVD in the early survivorship period, survivors from rural areas were more likely to develop CVD.

Implications for Survivors

Survivors from non-urban areas and those with public insurance may be particularly vulnerable to CVD.

Body mass index changes and their association with SARS-CoV-2 infection: a real-world analysis

Objective

To study body mass index (BMI) changes among individuals aged 18-99 years with and without SARS-CoV-2 infection.

Subjects/Methods

Using real-world data from the OneFlorida+ Clinical Research Network of the National Patient-Centered Clinical Research Network, we compared changes over time in BMI in an Exposed cohort (positive SARS-CoV-2 test between March 2020 - January 2022), to a contemporary Unexposed cohort (negative SARS-CoV-2 tests), and an age/sex-matched Historical control cohort (March 2018 - January 2020). Body mass index (kg/m2) was retrieved from objective measures of height and weight in electronic health records. We used target trial approaches to estimate BMI at baseline and change per 100 days of follow-up for Unexposed and Historical cohorts relative to the Exposed cohort by categories of sex, race-ethnicity, age, and hospitalization status.

Results

The study sample consisted of 44,436 (Exposed cohort), 164,118 (Unexposed cohort), and 41,189 (Historical cohort). Cumulatively, 62% were women, 21.5% Non-Hispanic Black, 21.4% Hispanic and 5.6% Non-Hispanic Other. Patients had an average age of 51.9 years (SD: 18.9). At baseline, relative to the Exposed cohort (mean BMI: 29.3 kg/m2 [95%CI: 29.0, 29.7]), the Unexposed (-0.07 kg/m2 [95%CI; -0.12, -0.01]) and Historical controls (-0.27 kg/m2 [95%CI; -0.34, -0.20]) had lower BMI. Relative to no change in the Exposed over 100 days (0.00 kg/m2 [95%CI; -0.03,0.03]), the BMI of those Unexposed decreased (-0.04 kg/m2 [95%CI; -0.06, -0.01]) while the Historical cohort's BMI increased (+0.03 kg/m2 [95%CI;0.00,0.06]). BMI changes were consistent between Exposed and Unexposed cohorts for most population groups, except at start of follow-up period among Males and those 65 years or older, and in changes over 100 days among Males and Hispanics.

Conclusions

In a diverse real-world cohort of adults, mean BMI of those with and without SARS-CoV2 infection varied in their trajectories. The mechanisms and implications of weight retention following SARS-CoV-2 infection remain unclear.

Geographic variation and racial disparities in adoption of newer glucose-lowering drugs with cardiovascular benefits among US Medicare beneficiaries with type 2 diabetes

BACKGROUND

Prior studies have shown disparities in the uptake of cardioprotective newer glucose-lowering drugs (GLDs), including sodium-glucose cotranwsporter-2 inhibitors (SGLT2i) and glucagon-like peptide-1 receptor agonists (GLP1a). This study aimed to characterize geographic variation in the initiation of newer GLDs and the geographic variation in the disparities in initiating these medications.

METHODS

Using 2017-2018 claims data from a 15% random nationwide sample of Medicare Part D beneficiaries, we identified individuals diagnosed with type 2 diabetes (T2D), who had ≥1 GLD prescriptions, and did not use SGLT2i or GLP1a in the year prior to the index date,1/1/2018. Patients were followed up for a year. The cohort was spatiotemporally linked to Dartmouth hospital-referral regions (HRRs), with each patient assigned to 1 of 306 HRRs. We performed multivariable Poisson regression to estimate adjusted initiation rates, and multivariable logistic regression to assess racial disparities in each HRR.

RESULTS

Among 795,469 individuals with T2D included in the analyses, the mean (SD) age was 73 (10) y, 53.3% were women, 12.2% were non-Hispanic Black, and 7.2% initiated a newer GLD in the follow-up year. In the adjusted model including clinical factors, compared to non-Hispanic White patients, non-Hispanic Black (initiation rate ratio, IRR [95% CI]: 0.66 [0.64-0.68]), American Indian/Alaska Native (0.74 [0.66-0.82]), Hispanic (0.85 [0.82-0.87]), and Asian/Pacific islander (0.94 [0.89-0.98]) patients were less likely to initiate newer GLDs. Significant geographic variation was observed across HRRs, with an initiation rate spanning 2.7%-13.6%.

CONCLUSIONS

This study uncovered substantial geographic variation and the racial disparities in initiating newer GLDs.

Scalable and interpretable alternative to chart review for phenotype evaluation using standardized structured data from electronic health records

OBJECTIVES

Chart review as the current gold standard for phenotype evaluation cannot support observational research on electronic health records and claims data sources at scale. We aimed to evaluate the ability of structured data to support efficient and interpretable phenotype evaluation as an alternative to chart review.

MATERIALS AND METHODS

We developed Knowledge-Enhanced Electronic Profile Review (KEEPER) as a phenotype evaluation tool that extracts patient's structured data elements relevant to a phenotype and presents them in a standardized fashion following clinical reasoning principles. We evaluated its performance (interrater agreement, intermethod agreement, accuracy, and review time) compared to manual chart review for 4 conditions using randomized 2-period, 2-sequence crossover design.

RESULTS

Case ascertainment with KEEPER was twice as fast compared to manual chart review. 88.1% of the patients were classified concordantly using charts and KEEPER, but agreement varied depending on the condition. Missing data and differences in interpretation accounted for most of the discrepancies. Pairs of clinicians agreed in case ascertainment in 91.2% of the cases when using KEEPER compared to 76.3% when using charts. Patient classification aligned with the gold standard in 88.1% and 86.9% of the cases respectively.

CONCLUSION

Structured data can be used for efficient and interpretable phenotype evaluation if they are limited to relevant subset and organized according to the clinical reasoning principles. A system that implements these principles can achieve noninferior performance compared to chart review at a fraction of time.

A Fair Individualized Polysocial Risk Score for Identifying Increased Social Risk in Type 2 Diabetes

Background

Racial and ethnic minority groups and individuals facing social disadvantages, which often stem from their social determinants of health (SDoH), bear a disproportionate burden of type 2 diabetes (T2D) and its complications. It is crucial to implement effective social risk management strategies at the point of care.

Objective

To develop an electronic health records (EHR)-based machine learning (ML) analytical pipeline to address unmet social needs associated with hospitalization risk in patients with T2D.

Methods

We identified real-world patients with T2D from the EHR data from University of Florida (UF) Health Integrated Data Repository (IDR), incorporating both contextual SDoH (e.g., neighborhood deprivation) and individual-level SDoH (e.g., housing instability). The 2015-2020 data were used for training and validation and 2021-2022 data for independent testing. We developed a machine learning analytic pipeline, namely individualized polysocial risk score (iPsRS), to identify high social risk associated with hospitalizations in T2D patients, along with explainable AI (XAI) and fairness optimization.

Results

The study cohort included 10,192 real-world patients with T2D, with a mean age of 59 years and 58% female. Of the cohort, 50% were non-Hispanic White, 39% were non-Hispanic Black, 6% were Hispanic, and 5% were other races/ethnicities. Our iPsRS, including both contextual and individual-level SDoH as input factors, achieved a C statistic of 0.72 in predicting 1-year hospitalization after fairness optimization across racial and ethnic groups. The iPsRS showed excellent utility for capturing individuals at high hospitalization risk because of SDoH, that is, the actual 1-year hospitalization rate in the top 5% of iPsRS was 28.1%, ~13 times as high as the bottom decile (2.2% for 1-year hospitalization rate).

Conclusion

Our ML pipeline iPsRS can fairly and accurately screen for patients who have increased social risk leading to hospitalization in real word patients with T2D.

Performance Characteristics of a Rule-Based Electronic Health Record Algorithm to Identify Patients with Gross and Microscopic Hematuria

BACKGROUND

Two million patients per year are referred to urologists for hematuria, or blood in the urine. The American Urological Association recently adopted a risk-stratified hematuria evaluation guideline to limit multi-phase computed tomography to individuals at highest risk of occult malignancy.

OBJECTIVES

To understand population-level hematuria evaluations, we developed an algorithm to accurately identify hematuria cases from electronic health records (EHRs).

METHODS

We used International Classification of Diseases (ICD)-9/ICD-10 diagnosis codes, urine color, and urine microscopy values to identify hematuria cases and to differentiate between gross and microscopic hematuria. Using an iterative process, we refined the ICD-9 algorithm on a gold standard, chart-reviewed cohort of 3,094 hematuria cases, and the ICD-10 algorithm on a 300 patient cohort. We applied the algorithm to Geisinger patients ≥35 years (n = 539,516) and determined performance by conducting chart review (n = 500).

RESULTS

After applying the hematuria algorithm, we identified 51,500 hematuria cases and 488,016 clean controls. Of the hematuria cases, 11,435 were categorized as gross, 26,658 as microscopic, 12,562 as indeterminate, and 845 were uncategorized. The positive predictive value (PPV) of identifying hematuria cases using the algorithm was 100% and the negative predictive value (NPV) was 99%. The gross hematuria algorithm had a PPV of 100% and NPV of 99%. The microscopic hematuria algorithm had lower PPV of 78% and NPV of 100%.

CONCLUSION

We developed an algorithm utilizing diagnosis codes and urine laboratory values to accurately identify hematuria and categorize as gross or microscopic in EHRs. Applying the algorithm will help researchers to understand patterns of care for this common condition.

Trends and opportunities in computable clinical phenotyping: A scoping review

Identifying patient cohorts meeting the criteria of specific phenotypes is essential in biomedicine and particularly timely in precision medicine. Many research groups deliver pipelines that automatically retrieve and analyze data elements from one or more sources to automate this task and deliver high-performing computable phenotypes. We applied a systematic approach based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines to conduct a thorough scoping review on computable clinical phenotyping. Five databases were searched using a query that combined the concepts of automation, clinical context, and phenotyping. Subsequently, four reviewers screened 7960 records (after removing over 4000 duplicates) and selected 139 that satisfied the inclusion criteria. This dataset was analyzed to extract information on target use cases, data-related topics, phenotyping methodologies, evaluation strategies, and portability of developed solutions. Most studies supported patient cohort selection without discussing the application to specific use cases, such as precision medicine. Electronic Health Records were the primary source in 87.1 % (N = 121) of all studies, and International Classification of Diseases codes were heavily used in 55.4 % (N = 77) of all studies, however, only 25.9 % (N = 36) of the records described compliance with a common data model. In terms of the presented methods, traditional Machine Learning (ML) was the dominant method, often combined with natural language processing and other approaches, while external validation and portability of computable phenotypes were pursued in many cases. These findings revealed that defining target use cases precisely, moving away from sole ML strategies, and evaluating the proposed solutions in the real setting are essential opportunities for future work. There is also momentum and an emerging need for computable phenotyping to support clinical and epidemiological research and precision medicine.

Impact of Contextual-Level Social Determinants of Health on Newer Antidiabetic Drug Adoption in Patients with Type 2 Diabetes

BACKGROUND

We aimed to investigate the association between contextual-level social determinants of health (SDoH) and the use of novel antidiabetic drugs (ADD), including sodium-glucose cotransporter-2 inhibitors (SGLT2i) and glucagon-like peptide-1 receptor agonists (GLP1a) for patients with type 2 diabetes (T2D), and whether the association varies across racial and ethnic groups.

METHODS

Using electronic health records from the OneFlorida+ network, we assembled a cohort of T2D patients who initiated a second-line ADD in 2015-2020. A set of 81 contextual-level SDoH documenting social and built environment were spatiotemporally linked to individuals based on their residential histories. We assessed the association between the contextual-level SDoH and initiation of SGTL2i/GLP1a and determined their effects across racial groups, adjusting for clinical factors.

RESULTS

Of 28,874 individuals, 61% were women, and the mean age was 58 (±15) years. Two contextual-level SDoH factors identified as significantly associated with SGLT2i/GLP1a use were neighborhood deprivation index (odds ratio [OR] 0.87, 95% confidence interval [CI] 0.81-0.94) and the percent of vacant addresses in the neighborhood (OR 0.91, 95% CI 0.85-0.98). Patients living in such neighborhoods are less likely to be prescribed with newer ADD. There was no interaction between race-ethnicity and SDoH on the use of newer ADD. However, in the overall cohort, the non-Hispanic Black individuals were less likely to use newer ADD than the non-Hispanic White individuals (OR 0.82, 95% CI 0.76-0.88).

CONCLUSION

Using a data-driven approach, we identified the key contextual-level SDoH factors associated with not following evidence-based treatment of T2D. Further investigations are needed to examine the mechanisms underlying these associations.

Comparison of Sodium-Glucose Cotransporter-2 Inhibitor and Glucagon-Like Peptide-1 Receptor Agonist Prescribing in Patients With Diabetes Mellitus With and Without Cardiovascular Disease

Sodium-glucose cotransporter-2 inhibitors (SGLT2is) and glucagon-like peptide-1 receptor agonists (GLP1-RAs) reduce cardiovascular events and mortality in patients with type 2 diabetes mellitus (T2DM). We sought to describe trends in prescribing for SGLT2is and GLP1-RAs in diverse care settings, including (1) the outpatient clinics of a midwestern integrated health system and (2) small- and medium-sized community-based primary care practices and health centers in 3 midwestern states. We included adults with T2DM and ≥1 outpatient clinic visit. The outcomes of interest were annual active prescription rates for SGLT2is and GLP1-RAs (separately). In the integrated health system, 22,672 patients met the case definition of T2DM. From 2013 to 2019, the overall prescription rate for SGLT2is increased from 1% to 15% (absolute difference [AD] 14%, 95% confidence interval [CI] 13% to 15%, p <0.01). The GLP1-RA prescription rate was stable at 10% (AD 0%, 95% CI -1% to 1%, p = 0.9). In community-based primary care practices, 43,340 patients met the case definition of T2DM. From 2013 to 2017, the SGLT2i prescription rate increased from 3% to 7% (AD 4%, 95% CI 3% to 6%, p <0.01), whereas the GLP1-RA prescription rate was stable at 2% to 3% (AD 1%, 95% CI -1 to 1%, p = 0.40). In a fully adjusted regression model, non-Hispanic Black patients had lower odds of SGLT2i or GLP1-RA prescription (odds ratio 0.56, 95% CI 0.34 to 0.89, p = 0.016). In conclusion, the increase in prescription rates was greater for SGLT2is than for GLP1-RAs in patients with T2DM in a large integrated medical center and community primary care practices. Overall, prescription rates for eligible patients were low, and racial disparities were observed.

Developing real-world evidence from real-world data: Transforming raw data into analytical datasets

Development of evidence-based practice requires practice-based evidence, which can be acquired through analysis of real-world data from electronic health records (EHRs). The EHR contains volumes of information about patients-physical measurements, diagnoses, exposures, and markers of health behavior-that can be used to create algorithms for risk stratification or to gain insight into associations between exposures, interventions, and outcomes. But to transform real-world data into reliable real-world evidence, one must not only choose the correct analytical methods but also have an understanding of the quality, detail, provenance, and organization of the underlying source data and address the differences in these characteristics across sites when conducting analyses that span institutions. This manuscript explores the idiosyncrasies inherent in the capture, formatting, and standardization of EHR data and discusses the clinical domain and informatics competencies required to transform the raw clinical, real-world data into high-quality, fit-for-purpose analytical data sets used to generate real-world evidence.

When Less Is More: Identifying Patients With Type 2 Diabetes Engaging in Unnecessary Blood Glucose Monitoring

This study examined whether certain patient characteristics are associated with the prescribing of self-monitoring of blood glucose for patients with type 2 diabetes who are not using insulin and have well-controlled blood glucose. Against recommendations, one-third of the patient sample from a large health network in North Carolina (N = 9,338) received a prescription for testing supplies (i.e., strips or lancets) within the prior 18 months. Women, African Americans, individuals prescribed an oral medication, nonsmokers, and those who were underweight or normal weight all had greater odds of receiving such a prescription. These results indicate that providers may have prescribing tendencies that are potentially biased against more vulnerable patient groups and contrary to guidelines.

Validating a Computable Phenotype for Nephrotic Syndrome in Children and Adults Using PCORnet Data

Background

Primary nephrotic syndromes are rare diseases which can impede adequate sample size for observational patient-oriented research and clinical trial enrollment. A computable phenotype may be powerful in identifying patients with these diseases for research across multiple institutions.

Methods

A comprehensive algorithm of inclusion and exclusion ICD-9 and ICD-10 codes to identify patients with primary nephrotic syndrome was developed. The algorithm was executed against the PCORnet CDM at three institutions from January 1, 2009 to January 1, 2018, where a random selection of 50 cases and 50 noncases (individuals not meeting case criteria seen within the same calendar year and within 5 years of age of a case) were reviewed by a nephrologist, for a total of 150 cases and 150 noncases reviewed. The classification accuracy (sensitivity, specificity, positive and negative predictive value, F1 score) of the computable phenotype was determined.

Results

The algorithm identified a total of 2708 patients with nephrotic syndrome from 4,305,092 distinct patients in the CDM at all sites from 2009 to 2018. For all sites, the sensitivity, specificity, and area under the curve of the algorithm were 99% (95% CI, 97% to 99%), 79% (95% CI, 74% to 85%), and 0.9 (0.84 to 0.97), respectively. The most common causes of false positive classification were secondary FSGS (nine out of 39) and lupus nephritis (nine out of 39).

Conclusion

This computable phenotype had good classification in identifying both children and adults with primary nephrotic syndrome utilizing only ICD-9 and ICD-10 codes, which are available across institutions in the United States. This may facilitate future screening and enrollment for research studies and enable comparative effectiveness research. Further refinements to the algorithm including use of laboratory data or addition of natural language processing may help better distinguish primary and secondary causes of nephrotic syndrome.

Diabetes medication regimens and patient clinical characteristics in the national patient-centered clinical research network, PCORnet

We used electronic medical record (EMR) data in the National Patient-Centered Clinical Research Network (PCORnet) to characterize "real-world" prescription patterns of Type 2 diabetes (T2D) medications. We identified a retrospective cohort of 613,203 adult patients with T2D from 33 datamarts (median patient number: 12,711) from 2012 through 2017 using a validated computable phenotype. We characterized outpatient T2D prescriptions for each patient in the 90 days before and after cohort entry, as well as demographics, comorbidities, non-T2D prescriptions, and clinical and laboratory variables in the 730 days prior to cohort entry. Approximately half of the individuals in the cohort were females and 20% Black. Hypertension (60.3%) and hyperlipidemia (50.5%) were highly prevalent. Most patients were prescribed either a single T2D drug class (42.2%) or had no evidence of a T2D prescription in the EMR (42.4%). A smaller percentage was prescribed multiple T2D drug types (15.4%). Among patients prescribed a single T2D drug type, metformin was the most common (42.6%), followed by insulin (18.2%) and sulfonylureas (13.9%). Newer classes represented approximately 13% of single T2D drug type prescriptions (dipeptidyl peptidase-4 inhibitors [6.6%], glucagon-like peptide-1 receptor agonists [2.5%], thiazolidinediones [2.0%], and sodium-glucose cotransporter-2 inhibitors [1.6%]). Among patients prescribed multiple T2D drug types, the most common combination was metformin and sulfonylureas (63.5%). Metformin-based regimens were highly prevalent in PCORnet's T2D population, whereas newer agents were prescribed less frequently. PCORnet is a novel source for the potential conduct of observational studies among patients with T2D.

Electronic Medical Record-Based Case Phenotyping for the Charlson Conditions: Scoping Review

Background

Electronic medical records (EMRs) contain large amounts of rich clinical information. Developing EMR-based case definitions, also known as EMR phenotyping, is an active area of research that has implications for epidemiology, clinical care, and health services research.

Objective

This review aims to describe and assess the present landscape of EMR-based case phenotyping for the Charlson conditions.

Methods

A scoping review of EMR-based algorithms for defining the Charlson comorbidity index conditions was completed. This study covered articles published between January 2000 and April 2020, both inclusive. Embase (Excerpta Medica database) and MEDLINE (Medical Literature Analysis and Retrieval System Online) were searched using keywords developed in the following 3 domains: terms related to EMR, terms related to case finding, and disease-specific terms. The manuscript follows the Preferred Reporting Items for Systematic reviews and Meta-analyses extension for Scoping Reviews (PRISMA) guidelines.

Results

A total of 274 articles representing 299 algorithms were assessed and summarized. Most studies were undertaken in the United States (181/299, 60.5%), followed by the United Kingdom (42/299, 14.0%) and Canada (15/299, 5.0%). These algorithms were mostly developed either in primary care (103/299, 34.4%) or inpatient (168/299, 56.2%) settings. Diabetes, congestive heart failure, myocardial infarction, and rheumatology had the highest number of developed algorithms. Data-driven and clinical rule–based approaches have been identified. EMR-based phenotype and algorithm development reflect the data access allowed by respective health systems, and algorithms vary in their performance.

Conclusions

Recognizing similarities and differences in health systems, data collection strategies, extraction, data release protocols, and existing clinical pathways is critical to algorithm development strategies. Several strategies to assist with phenotype-based case definitions have been proposed.

Different Strategies to Execute Multi-Database Studies for Medicines Surveillance in Real-World Setting: A Reflection on the European Model

Although postmarketing studies conducted in population‐based databases often contain information on patients in the order of millions, they can still be underpowered if outcomes or exposure of interest is rare, or the interest is in subgroup effects. Combining several databases might provide the statistical power needed. A multi‐database study (MDS) uses at least two healthcare databases, which are not linked with each other at an individual person level, with analyses carried out in parallel across each database applying a common study protocol. Although many MDSs have been performed in Europe in the past 10 years, there is a lack of clarity on the peculiarities and implications of the existing strategies to conduct them. In this review, we identify four strategies to execute MDSs, classified according to specific choices in the execution: (A) local analyses, where data are extracted and analyzed locally, with programs developed by each site; (B) sharing of raw data, where raw data are locally extracted and transferred without analysis to a central partner, where all the data are pooled and analyzed; (C) use of a common data model with study‐specific data, where study‐specific data are locally extracted, loaded into a common data model, and processed locally with centrally developed programs; and (D) use of general common data model, where all local data are extracted and loaded into a common data model, prior to and independent of any study protocol, and protocols are incorporated in centrally developed programs that run locally. We illustrate differences between strategies and analyze potential implications.

Clinical Trial Generalizability Assessment in the Big Data Era: A Review

Clinical studies, especially randomized, controlled trials, are essential for generating evidence for clinical practice. However, generalizability is a long‐standing concern when applying trial results to real‐world patients. Generalizability assessment is thus important, nevertheless, not consistently practiced. We performed a systematic review to understand the practice of generalizability assessment. We identified 187 relevant articles and systematically organized these studies in a taxonomy with three dimensions: (i) data availability (i.e., before or after trial (a priori vs. a posteriori generalizability)); (ii) result outputs (i.e., score vs. nonscore); and (iii) populations of interest. We further reported disease areas, underrepresented subgroups, and types of data used to profile target populations. We observed an increasing trend of generalizability assessments, but < 30% of studies reported positive generalizability results. As a priori generalizability can be assessed using only study design information (primarily eligibility criteria), it gives investigators a golden opportunity to adjust the study design before the trial starts. Nevertheless, < 40% of the studies in our review assessed a priori generalizability. With the wide adoption of electronic health records systems, rich real‐world patient databases are increasingly available for generalizability assessment; however, informatics tools are lacking to support the adoption of generalizability assessment practice.

Towards more Accessible Precision Medicine: Building a more Transferable Machine Learning Model to Support Prognostic Decisions for Micro- and Macrovascular Complications of Type 2 Diabetes Mellitus

Although machine learning models are increasingly being developed for clinical decision support for patients with type 2 diabetes, the adoption of these models into clinical practice remains limited. Currently, machine learning (ML) models are being constructed on local healthcare systems and are validated internally with no expectation that they would validate externally and thus, are rarely transferrable to a different healthcare system. In this work, we aim to demonstrate that (1) even a complex ML model built on a national cohort can be transferred to two local healthcare systems, (2) while a model constructed on a local healthcare system's cohort is difficult to transfer; (3) we examine the impact of training cohort size on the transferability; and (4) we discuss criteria for external validity. We built a model using our previously published Multi-Task Learning-based methodology on a national cohort extracted from OptumLabs® Data Warehouse and transferred the model to two local healthcare systems (i.e., University of Minnesota Medical Center and Mayo Clinic) for external evaluation. The model remained valid when applied to the local patient populations and performed as well as locally constructed models (concordance: .73-.92), demonstrating transferability. The performance of the locally constructed models reduced substantially when applied to each other's healthcare system (concordance: .62-.90). We believe that our modeling approach, in which a model is learned from a national cohort and is externally validated, produces a transferable model, allowing patients at smaller healthcare systems to benefit from precision medicine.