Bringing cohort studies to the bedside: framework for a ‘green button’ to support clinical decision-making

ICD-10 diagnosis codes in electronic health records do not adequately capture fracture complexity for proximal humerus fractures

BACKGROUND

The ability to do comparative effectiveness research (CER) for proximal humerus fractures (PHF) using data in electronic health record (EHR) systems and administrative claims databases was enhanced by the 10th revision of the International Classification of Diseases (ICD-10), which expanded the diagnosis codes for PHF to describe fracture complexity including displacement and the number of fracture parts. However, these expanded codes only enhance secondary use of data for research if the codes selected and recorded correctly reflect the fracture complexity. The objective of this project was to assess the accuracy of ICD-10 diagnosis codes documented during routine clinical practice for secondary use of EHR data.

METHODS

A sample of patients with PHFs treated by orthopedic providers across a large, regional health care system between January 1, 2016, and December 31, 2018, were retrospectively identified from the EHR. Four fellowship-trained orthopedic surgeons reviewed patient radiographs and recorded the Neer Classification characteristics of displacement, number of parts, and fracture location(s). The fracture characteristics were then reviewed by a trained coder, and the most clinically appropriate ICD-10 diagnosis code based on the number of fracture parts was assigned. We assessed congruence between ICD-10 codes documented in the EHR and radiograph-validated codes, and assessed sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for EHR-documented ICD-10 codes.

RESULTS

There were 761 patients with unilateral, closed PHF who met study inclusion criteria. On average, patients were 67 years of age and 77% were female. Based on radiograph review, 37% were 1-part fractures, 42% were 2-part, 11% were 3-part, and 10% were 4-part fractures. Of the EHR diagnosis codes recorded during clinical practice, 59% were "unspecified" fracture diagnosis codes that did not identify the number of fracture parts. Examination of fracture codes revealed PPV was highest for 1-part (PPV = 0.66, 95% confidence interval [CI] 0.60-0.72) and 4-part fractures (PPV = 0.67, 95% CI 0.13-1.00).

CONCLUSIONS

Current diagnosis coding practices do not adequately capture the fracture complexity needed to conduct subgroup analysis for PHF. Conclusions drawn from population studies or large databases using ICD-10 codes for PHF classification should be interpreted within this limitation. Future studies are warranted to improve diagnostic coding to support large observational studies using EHR and administrative claims data.

Scalable Feature Engineering from Electronic Free Text Notes to Supplement Confounding Adjustment of Claims-Based Pharmacoepidemiologic Studies

Natural language processing (NLP) tools turn free-text notes (FTNs) from electronic health records (EHRs) into data features that can supplement confounding adjustment in pharmacoepidemiologic studies. However, current applications are difficult to scale. We used unsupervised NLP to generate high-dimensional feature spaces from FTNs to improve prediction of drug exposure and outcomes compared with claims-based analyses. We linked Medicare claims with EHR data to generate three cohort studies comparing different classes of medications on the risk of various clinical outcomes. We used "bag-of-words" to generate features for the top 20,000 most prevalent terms from FTNs. We compared machine learning (ML) prediction algorithms using different sets of candidate predictors: Set1 (39 researcher-specified variables), Set2 (Set1 + ML-selected claims codes), and Set3 (Set1 + ML-selected NLP-generated features), vs. Set4 (Set1 + 2 + 3). When modeling treatment choice, we observed a consistent pattern across the examples: ML models utilizing Set4 performed best followed by Set2, Set3, then Set1. When modeling the outcome risk, there was little to no improvement beyond models based on Set1. Supplementing claims data with NLP-generated features from free text notes improved prediction of prescribing choices but had little or no improvement on clinical risk prediction. These findings have implications for strategies to improve confounding using EHR data in pharmacoepidemiologic studies.

Evidence synthesis, digital scribes, and translational challenges for artificial intelligence in healthcare

Healthcare has well-known challenges with safety, quality, and effectiveness, and many see artificial intelligence (AI) as essential to any solution. Emerging applications include the automated synthesis of best-practice research evidence including systematic reviews, which would ultimately see all clinical trial data published in a computational form for immediate synthesis. Digital scribes embed themselves in the process of care to detect, record, and summarize events and conversations for the electronic record. However, three persistent translational challenges must be addressed before AI is widely deployed. First, little effort is spent replicating AI trials, exposing patients to risks of methodological error and biases. Next, there is little reporting of patient harms from trials. Finally, AI built using machine learning may perform less effectively in different clinical settings.

Assessing the ability of an instrumental variable causal forest algorithm to personalize treatment evidence using observational data: the case of early surgery for shoulder fracture

Background

Comparative effectiveness research (CER) using observational databases has been suggested to obtain personalized evidence of treatment effectiveness. Inferential difficulties remain using traditional CER approaches especially related to designating patients to reference classes a priori. A novel Instrumental Variable Causal Forest Algorithm (IV-CFA) has the potential to provide personalized evidence using observational data without designating reference classes a priori, but the consistency of the evidence when varying key algorithm parameters remains unclear. We investigated the consistency of IV-CFA estimates through application to a database of Medicare beneficiaries with proximal humerus fractures (PHFs) that previously revealed heterogeneity in the effects of early surgery using instrumental variable estimators.

Methods

IV-CFA was used to estimate patient-specific early surgery effects on both beneficial and detrimental outcomes using different combinations of algorithm parameters and estimate variation was assessed for a population of 72,751 fee-for-service Medicare beneficiaries with PHFs in 2011. Classification and regression trees (CART) were applied to these estimates to create ex-post reference classes and the consistency of these classes were assessed. Two-stage least squares (2SLS) estimators were applied to representative ex-post reference classes to scrutinize the estimates relative to known 2SLS properties.

Results

IV-CFA uncovered substantial early surgery effect heterogeneity across PHF patients, but estimates for individual patients varied with algorithm parameters. CART applied to these estimates revealed ex-post reference classes consistent across algorithm parameters. 2SLS estimates showed that ex-post reference classes containing older, frailer patients with more comorbidities, and lower utilizers of healthcare were less likely to benefit and more likely to have detriments from higher rates of early surgery.

Conclusions

IV-CFA provides an illuminating method to uncover ex-post reference classes of patients based on treatment effects using observational data with a strong instrumental variable. Interpretation of treatment effect estimates within each ex-post reference class using traditional CER methods remains conditional on the extent of measured information in the data.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-022-01663-0.

Evidence-based medicine and machine learning: a partnership with a common purpose

From its origins in epidemiology, evidence-based medicine has promulgated a rigorous approach to assessing the validity, impact and applicability of hypothesis-driven empirical research used to evaluate the utility of diagnostic tests, prognostic tools and therapeutic interventions. Machine learning, a subset of artificial intelligence, uses computer programs to discover patterns and associations within huge datasets which are then incorporated into algorithms used to assist diagnoses and predict future outcomes, including response to therapies. How do these two fields relate to one another? What are their similarities and differences, their strengths and weaknesses? Can each learn from, and complement, the other in rendering clinical decision-making more informed and effective?

Data Consult Service: Can we use observational data to address immediate clinical needs?

OBJECTIVE

A number of clinical decision support tools aim to use observational data to address immediate clinical needs, but few of them address challenges and biases inherent in such data. The goal of this article is to describe the experience of running a data consult service that generates clinical evidence in real time and characterize the challenges related to its use of observational data.

MATERIALS AND METHODS

In 2019, we launched the Data Consult Service pilot with clinicians affiliated with Columbia University Irving Medical Center. We created and implemented a pipeline (question gathering, data exploration, iterative patient phenotyping, study execution, and assessing validity of results) for generating new evidence in real time. We collected user feedback and assessed issues related to producing reliable evidence.

RESULTS

We collected 29 questions from 22 clinicians through clinical rounds, emails, and in-person communication. We used validated practices to ensure reliability of evidence and answered 24 of them. Questions differed depending on the collection method, with clinical rounds supporting proactive team involvement and gathering more patient characterization questions and questions related to a current patient. The main challenges we encountered included missing and incomplete data, underreported conditions, and nonspecific coding and accurate identification of drug regimens.

CONCLUSIONS

While the Data Consult Service has the potential to generate evidence and facilitate decision making, only a portion of questions can be answered in real time. Recognizing challenges in patient phenotyping and designing studies along with using validated practices for observational research are mandatory to produce reliable evidence.

Towards a Responsible Transition to Learning Healthcare Systems in Precision Medicine: Ethical Points to Consider

Learning healthcare systems have recently emerged as a strategy to continuously use experiences and outcomes of clinical care for research purposes in precision medicine. Although it is known that learning healthcare transitions in general raise important ethical challenges, the ethical ramifications of such transitions in the specific context of precision medicine have not extensively been discussed. Here, we describe three levers that institutions can pull to advance learning healthcare systems in precision medicine: (1) changing testing of individual variability (such as genes); (2) changing prescription of treatments on the basis of (genomic) test results; and/or (3) changing the handling of data that link variability and treatment to clinical outcomes. Subsequently, we evaluate how patients can be affected if one of these levers are pulled: (1) patients are tested for different or more factors than before the transformation, (2) patients receive different treatments than before the transformation and/or (3) patients’ data obtained through clinical care are used, or used more extensively, for research purposes. Based on an analysis of the aforementioned mechanisms and how these potentially affect patients, we analyze why learning healthcare systems in precision medicine need a different ethical approach and discuss crucial points to consider regarding this approach.

Personalized treatment options for chronic diseases using precision cohort analytics

To support point-of-care decision making by presenting outcomes of past treatment choices for cohorts of similar patients based on observational data from electronic health records (EHRs), a machine-learning precision cohort treatment option (PCTO) workflow consisting of (1) data extraction, (2) similarity model training, (3) precision cohort identification, and (4) treatment options analysis was developed. The similarity model is used to dynamically create a cohort of similar patients, to inform clinical decisions about an individual patient. The workflow was implemented using EHR data from a large health care provider for three different highly prevalent chronic diseases: hypertension (HTN), type 2 diabetes mellitus (T2DM), and hyperlipidemia (HL). A retrospective analysis demonstrated that treatment options with better outcomes were available for a majority of cases (75%, 74%, 85% for HTN, T2DM, HL, respectively). The models for HTN and T2DM were deployed in a pilot study with primary care physicians using it during clinic visits. A novel data-analytic workflow was developed to create patient-similarity models that dynamically generate personalized treatment insights at the point-of-care. By leveraging both knowledge-driven treatment guidelines and data-driven EHR data, physicians can incorporate real-world evidence in their medical decision-making process when considering treatment options for individual patients.

A scoping review of clinical decision support tools that generate new knowledge to support decision making in real time

OBJECTIVE

A growing body of observational data enabled its secondary use to facilitate clinical care for complex cases not covered by the existing evidence. We conducted a scoping review to characterize clinical decision support systems (CDSSs) that generate new knowledge to provide guidance for such cases in real time.

MATERIALS AND METHODS

PubMed, Embase, ProQuest, and IEEE Xplore were searched up to May 2020. The abstracts were screened by 2 reviewers. Full texts of the relevant articles were reviewed by the first author and approved by the second reviewer, accompanied by the screening of articles' references. The details of design, implementation and evaluation of included CDSSs were extracted.

RESULTS

Our search returned 3427 articles, 53 of which describing 25 CDSSs were selected. We identified 8 expert-based and 17 data-driven tools. Sixteen (64%) tools were developed in the United States, with the others mostly in Europe. Most of the tools (n = 16, 64%) were implemented in 1 site, with only 5 being actively used in clinical practice. Patient or quality outcomes were assessed for 3 (18%) CDSSs, 4 (16%) underwent user acceptance or usage testing and 7 (28%) functional testing.

CONCLUSIONS

We found a number of CDSSs that generate new knowledge, although only 1 addressed confounding and bias. Overall, the tools lacked demonstration of their utility. Improvement in clinical and quality outcomes were shown only for a few CDSSs, while the benefits of the others remain unclear. This review suggests a need for a further testing of such CDSSs and, if appropriate, their dissemination.

Clustering Analysis Supports the Detection of Biological Processes Related to Autism Spectrum Disorder

Genome sequencing has identified a large number of putative autism spectrum disorder (ASD) risk genes, revealing possible disrupted biological pathways; however, the genetic and environmental underpinnings of ASD remain mostly unanswered. The presented methodology aimed to identify genetically related clusters of ASD individuals. By using the VariCarta dataset, which contains data retrieved from 13,069 people with ASD, we compared patients pairwise to build “patient similarity matrices”. Hierarchical-agglomerative-clustering and heatmapping were performed, followed by enrichment analysis (EA). We analyzed whole-genome sequencing retrieved from 2062 individuals, and isolated 11,609 genetic variants shared by at least two people. The analysis yielded three clusters, composed, respectively, by 574 (27.8%), 507 (24.6%), and 650 (31.5%) individuals. Overall, 4187 variants (36.1%) were common to the three clusters. The EA revealed that the biological processes related to the shared genetic variants were mainly involved in neuron projection guidance and morphogenesis, cell junctions, synapse assembly, and in observational, imitative, and vocal learning. The study highlighted genetic networks, which were more frequent in a sample of people with ASD, compared to the overall population. We suggest that itemizing not only single variants, but also gene networks, might support ASD etiopathology research. Future work on larger databases will have to ascertain the reproducibility of this methodology.

Push Button Population Health: The SMART/HL7 FHIR Bulk Data Access Application Programming Interface

The 21st Century Cures Act requires that certified health information technology have an application programming interface (API) giving access to all data elements of a patient's electronic health record, "without special effort". In the spring of 2020, the Office of the National Coordinator of Health Information Technology (ONC) published a rule-21st Century Cures Act Interoperability, Information Blocking, and the ONC Health IT Certification Program-regulating the API requirement along with protections against information blocking. The rule specifies the SMART/HL7 FHIR Bulk Data Access API, which enables access to patient-level data across a patient population, supporting myriad use cases across healthcare, research, and public health ecosystems. The API enables "push button population health" in that core data elements can readily and standardly be extracted from electronic health records, enabling local, regional, and national-scale data-driven innovation.

What You Need to Know Before Implementing a Clinical Research Data Warehouse: Comparative Review of Integrated Data Repositories in Health Care Institutions

Background

Integrated data repositories (IDRs), also referred to as clinical data warehouses, are platforms used for the integration of several data sources through specialized analytical tools that facilitate data processing and analysis. IDRs offer several opportunities for clinical data reuse, and the number of institutions implementing an IDR has grown steadily in the past decade.

Objective

The architectural choices of major IDRs are highly diverse and determining their differences can be overwhelming. This review aims to explore the underlying models and common features of IDRs, provide a high-level overview for those entering the field, and propose a set of guiding principles for small- to medium-sized health institutions embarking on IDR implementation.

Methods

We reviewed manuscripts published in peer-reviewed scientific literature between 2008 and 2020, and selected those that specifically describe IDR architectures. Of 255 shortlisted articles, we found 34 articles describing 29 different architectures. The different IDRs were analyzed for common features and classified according to their data processing and integration solution choices.

Results

Despite common trends in the selection of standard terminologies and data models, the IDRs examined showed heterogeneity in the underlying architecture design. We identified 4 common architecture models that use different approaches for data processing and integration. These different approaches were driven by a variety of features such as data sources, whether the IDR was for a single institution or a collaborative project, the intended primary data user, and purpose (research-only or including clinical or operational decision making).

Conclusions

IDR implementations are diverse and complex undertakings, which benefit from being preceded by an evaluation of requirements and definition of scope in the early planning stage. Factors such as data source diversity and intended users of the IDR influence data flow and synchronization, both of which are crucial factors in IDR architecture planning.

Characterizing physicians' information needs related to a gap in knowledge unmet by current evidence

OBJECTIVE

The study sought to explore information needs arising from a gap in clinicians' knowledge that is not met by current evidence and identify possible areas of use and target groups for a future clinical decision support system (CDSS), which will guide clinicians in cases where no evidence exists.

MATERIALS AND METHODS

We interviewed 30 physicians in a large academic medical center, analyzed transcripts using deductive thematic analysis, and developed a set of themes of information needs related to a gap in knowledge unmet by current evidence. We conducted additional statistical analyses to identify the correlation between clinical experience, clinical specialty, settings of clinical care, and the characteristics of the needs.

RESULTS

This study resulted in a set of themes and subthemes of information needs arising from a gap in current evidence. Experienced physicians and inpatient physicians had more questions and the number of questions did not decline with clinical experience. The main areas of information needs included patients with comorbidities, elderly and children, new drugs, and rare disorders. To address these questions, clinicians most often used a commercial tool, guidelines, and PubMed. While primary care physicians preferred the commercial tool, specialty physicians sought more in-depth knowledge.

DISCUSSION

The current medical evidence appeared to be inadequate in covering specific populations such as patients with multiple comorbidities and elderly, and was sometimes irrelevant to complex clinical scenarios. Our findings may suggest that experienced and inpatient physicians would benefit from a CDSS that generates evidence in real time at the point of care.

CONCLUSIONS

We found that physicians had information needs, which arose from the gaps in current medical evidence. This study provides insights on how the CDSS that aims at addressing these needs should be designed.

Creation and application of virtual patient cohorts of heart models

Patient-specific cardiac models are now being used to guide therapies. The increased use of patient-specific cardiac simulations in clinical care will give rise to the development of virtual cohorts of cardiac models. These cohorts will allow cardiac simulations to capture and quantify inter-patient variability. However, the development of virtual cohorts of cardiac models will require the transformation of cardiac modelling from small numbers of bespoke models to robust and rapid workflows that can create large numbers of models. In this review, we describe the state of the art in virtual cohorts of cardiac models, the process of creating virtual cohorts of cardiac models, and how to generate the individual cohort member models, followed by a discussion of the potential and future applications of virtual cohorts of cardiac models. This article is part of the theme issue 'Uncertainty quantification in cardiac and cardiovascular modelling and simulation'.

A framework to build similarity-based cohorts for personalized treatment advice - a standardized, but flexible workflow with the R package SimBaCo

Along with increasing amounts of big data sources and increasing computer performance, real-world evidence from such sources likewise gains in importance. While this mostly applies to population averaged results from analyses based on the all available data, it is also possible to conduct so-called personalized analyses based on a data subset whose observations resemble a particular patient for whom a decision is to be made. Claims data from statutory health insurance companies could provide necessary information for such personalized analyses. To derive treatment recommendations from them for a particular patient in everyday care, an automated, reproducible and efficiently programmed workflow would be required. We introduce the R-package SimBaCo (Similarity-Based Cohort generation) offering a simple, but modular, and intuitive framework for this task. With the six built-in R-functions, this framework allows the user to create similarity cohorts tailored to the characteristics of particular patients. An exemplary workflow illustrates the distinct steps beginning with an initial cohort selection according to inclusion and exclusion criteria. A plotting function facilitates investigating a particular patient’s characteristics relative to their distribution in a reference cohort, for example the initial cohort or the precision cohort after the data has been trimmed in accordance with chosen variables for similarity finding. Such precision cohorts allow any form of personalized analysis, for example personalized analyses of comparative effectiveness or customized prediction models developed from precision cohorts. In our exemplary workflow, we provide such a treatment comparison whereupon a treatment decision for a particular patient could be made. This is only one field of application where personalized results can directly support the process of clinical reasoning by leveraging information from individual patient data. With this modular package at hand, personalized studies can efficiently weight benefits and risks of treatment options of particular patients.

Stepped-wedge randomised trial to evaluate population health intervention designed to increase appropriate anticoagulation in patients with atrial fibrillation

BACKGROUND

Clinical guidelines recommend anticoagulation for patients with atrial fibrillation (AF) at high risk of stroke; however, studies report 40% of this population is not anticoagulated.

OBJECTIVE

To evaluate a population health intervention to increase anticoagulation use in high-risk patients with AF.

METHODS

We used machine learning algorithms to identify patients with AF from electronic health records at high risk of stroke (CHA₂DS₂-VASc risk score ≥2), and no anticoagulant prescriptions within 12 months. A clinical pharmacist in the anticoagulation service reviewed charts for algorithm-identified patients to assess appropriateness of initiating an anticoagulant. The pharmacist then contacted primary care providers of potentially undertreated patients and offered assistance with anticoagulation management. We used a stepped-wedge design, evaluating the proportion of potentially undertreated patients with AF started on anticoagulant therapy within 28 days for clinics randomised to intervention versus usual care.

RESULTS

Of 1727 algorithm-identified high-risk patients with AF in clinics at the time of randomisation to intervention, 432 (25%) lacked evidence of anticoagulant prescriptions in the prior year. After pharmacist review, only 17% (75 of 432) of algorithm-identified patients were considered potentially undertreated at the time their clinic was randomised to intervention. Over a third (155 of 432) were excluded because they had a single prior AF episode (transient or provoked by serious illness); 36 (8%) had documented refusal of anticoagulation, the remainder had other reasons for exclusion. The intervention did not increase new anticoagulant prescriptions (intervention: 4.1% vs usual care: 4.0%, p=0.86).

CONCLUSIONS

Algorithms to identify underuse of anticoagulation among patients with AF in healthcare databases may not capture clinical subtleties or patient preferences and may overestimate the extent of undertreatment. Changing clinician behaviour remains challenging.

Composer-Visual Cohort Analysis of Patient Outcomes

OBJECTIVE

Visual cohort analysis utilizing electronic health record data has become an important tool in clinical assessment of patient outcomes. In this article, we introduce Composer, a visual analysis tool for orthopedic surgeons to compare changes in physical functions of a patient cohort following various spinal procedures. The goal of our project is to help researchers analyze outcomes of procedures and facilitate informed decision-making about treatment options between patient and clinician.

METHODS

In collaboration with orthopedic surgeons and researchers, we defined domain-specific user requirements to inform the design. We developed the tool in an iterative process with our collaborators to develop and refine functionality. With Composer, analysts can dynamically define a patient cohort using demographic information, clinical parameters, and events in patient medical histories and then analyze patient-reported outcome scores for the cohort over time, as well as compare it to other cohorts. Using Composer's current iteration, we provide a usage scenario for use of the tool in a clinical setting.

CONCLUSION

We have developed a prototype cohort analysis tool to help clinicians assess patient treatment options by analyzing prior cases with similar characteristics. Although Composer was designed using patient data specific to orthopedic research, we believe the tool is generalizable to other healthcare domains. A long-term goal for Composer is to develop the application into a shared decision-making tool that allows translation of comparison and analysis from a clinician-facing interface into visual representations to communicate treatment options to patients.

Paradigm changes for diagnosis: using big data for prediction

Due to profound changes occurring in biomedical knowledge and in health systems worldwide, an entirely new health and social care scenario is emerging. Moreover, the enormous technological potential developed over the last years is increasingly influencing life sciences and driving changes toward personalized medicine and value-based healthcare. However, the current slow progression of adoption, limiting the generation of healthcare efficiencies through technological innovation, can be realistically overcome by fostering convergence between a systems medicine approach and the principles governing Integrated Care. Implicit with this strategy is the multidisciplinary active collaboration of all stakeholders involved in the change, namely: citizens, professionals with different profiles, academia, policy makers, industry and payers. The article describes the key building blocks of an open and collaborative hub currently being developed in Catalonia (Spain) aiming at generation, deployment and evaluation of a personalized medicine program addressing highly prevalent chronic conditions that often show co-occurrence, namely: cardiovascular disorders, chronic obstructive pulmonary disease, type 2 diabetes mellitus; metabolic syndrome and associated mental disturbances (anxiety-depression and altered behavioral patterns leading to unhealthy life styles).

The digital scribe

Current generation electronic health records suffer a number of problems that make them inefficient and associated with poor clinical satisfaction. Digital scribes or intelligent documentation support systems, take advantage of advances in speech recognition, natural language processing and artificial intelligence, to automate the clinical documentation task currently conducted by humans. Whilst in their infancy, digital scribes are likely to evolve through three broad stages. Human led systems task clinicians with creating documentation, but provide tools to make the task simpler and more effective, for example with dictation support, semantic checking and templates. Mixed-initiative systems are delegated part of the documentation task, converting the conversations in a clinical encounter into summaries suitable for the electronic record. Computer-led systems are delegated full control of documentation and only request human interaction when exceptions are encountered. Intelligent clinical environments permit such augmented clinical encounters to occur in a fully digitised space where the environment becomes the computer. Data from clinical instruments can be automatically transmitted, interpreted using AI and entered directly into the record. Digital scribes raise many issues for clinical practice, including new patient safety risks. Automation bias may see clinicians automatically accept scribe documents without checking. The electronic record also shifts from a human created summary of events to potentially a full audio, video and sensor record of the clinical encounter. Digital scribes promisingly offer a gateway into the clinical workflow for more advanced support for diagnostic, prognostic and therapeutic tasks.

Big Data Readiness in Radiation Oncology: An Efficient Approach for Relabeling Radiation Therapy Structures With Their TG-263 Standard Name in Real-World Data Sets

Purpose

To prepare for big data analyses on radiation therapy data, we developed Stature, a tool-supported approach for standardization of structure names in existing radiation therapy plans. We applied the widely endorsed nomenclature standard TG-263 as the mapping target and quantified the structure name inconsistency in 2 real-world data sets.

Methods and Materials

The clinically relevant structures in the radiation therapy plans were identified by reference to randomized controlled trials. The Stature approach was used by clinicians to identify the synonyms for each relevant structure, which was then mapped to the corresponding TG-263 name. We applied Stature to standardize the structure names for 654 patients with prostate cancer (PCa) and 224 patients with head and neck squamous cell carcinoma (HNSCC) who received curative radiation therapy at our institution between 2007 and 2017. The accuracy of the Stature process was manually validated in a random sample from each cohort. For the HNSCC cohort we measured the resource requirements for Stature, and for the PCa cohort we demonstrated its impact on an example clinical analytics scenario.

Results

All but 1 synonym group (“Hydrogel”) was mapped to the corresponding TG-263 name, resulting in a TG-263 relabel rate of 99% (8837 of 8925 structures). For the PCa cohort, Stature matched a total of 5969 structures. Of these, 5682 structures were exact matches (ie, following local naming convention), 284 were matched via a synonym, and 3 required manual matching. This original radiation therapy structure names therefore had a naming inconsistency rate of 4.81%. For the HNSCC cohort, Stature mapped a total of 2956 structures (2638 exact, 304 synonym, 14 manual; 10.76% inconsistency rate) and required 7.5 clinician hours. The clinician hours required were one-fifth of those that would be required for manual relabeling. The accuracy of Stature was 99.97% (PCa) and 99.61% (HNSCC).

Conclusions

The Stature approach was highly accurate and had significant resource efficiencies compared with manual curation.

Privacy-Preserving Patient Similarity Learning in a Federated Environment: Development and Analysis

Background

There is an urgent need for the development of global analytic frameworks that can perform analyses in a privacy-preserving federated environment across multiple institutions without privacy leakage. A few studies on the topic of federated medical analysis have been conducted recently with the focus on several algorithms. However, none of them have solved similar patient matching, which is useful for applications such as cohort construction for cross-institution observational studies, disease surveillance, and clinical trials recruitment.

Objective

The aim of this study was to present a privacy-preserving platform in a federated setting for patient similarity learning across institutions. Without sharing patient-level information, our model can find similar patients from one hospital to another.

Methods

We proposed a federated patient hashing framework and developed a novel algorithm to learn context-specific hash codes to represent patients across institutions. The similarities between patients can be efficiently computed using the resulting hash codes of corresponding patients. To avoid security attack from reverse engineering on the model, we applied homomorphic encryption to patient similarity search in a federated setting.

Results

We used sequential medical events extracted from the Multiparameter Intelligent Monitoring in Intensive Care-III database to evaluate the proposed algorithm in predicting the incidence of five diseases independently. Our algorithm achieved averaged area under the curves of 0.9154 and 0.8012 with balanced and imbalanced data, respectively, in κ-nearest neighbor with κ=3. We also confirmed privacy preservation in similarity search by using homomorphic encryption.

Conclusions

The proposed algorithm can help search similar patients across institutions effectively to support federated data analysis in a privacy-preserving manner.

The forgetting health system

INTRODUCTION

Forgetting shapes learning in two different ways. It impedes learning when important lessons are forgotten. Equally, it can be difficult to enact new lessons if we do not let go of old beliefs and practices that are no longer useful. A learning health system (LHS) that wishes to improve health service delivery will need to find ways to remember processes that shape quality and safety - using data that often resides beyond electronic health records. An LHS will also need to "forget", or programmatically decommission, obsolete practices, whose persistence otherwise leads to unnecessary system complexity and inertia to change.

DISCUSSION

New forms of data needed to improve health services include process metrics extracted from digital systems; human-level metrics that capture workflow patterns and clinician behaviors; and multivariate process patterns that can identify service "syndromes." To avoid inertia to change, system complexity must be reduced by retiring (or forgetting) inefficient or unhelpful work practices. Biological models of programmed cell death provide a rich set of mechanisms to decommission elements of health services. These models suggest health service elements should be able to detect the end of their useful life and should contain internal mechanisms to orchestrate decommissioning-in contrast to current service decommissioning, which is an externally initiated, top-down down-driven process.

CONCLUSIONS

An LHS should take advantage of digital infrastructure to bring together people, sensors, analytics, and quasi-autonomous mechanisms for service adaptation. By drawing inspiration from biology, we can design LHSs that do not just remember but also actively forget.

Feasibility of Population Health Analytics and Data Visualization for Decision Support in the Infectious Diseases Domain: A pilot study

OBJECTIVE

Big data or population-based information has the potential to reduce uncertainty in medicine by informing clinicians about individual patient care. The objectives of this study were: 1) to explore the feasibility of extracting and displaying population-based information from an actual clinical population's database records, 2) to explore specific design features for improving population display, 3) to explore perceptions of population information displays, and 4) to explore the impact of population information display on cognitive outcomes.

METHODS

We used the Veteran's Affairs (VA) database to identify similar complex patients based on a similar complex patient case. Study outcomes measures were 1) preferences for population information display 2) time looking at the population display, 3) time to read the chart, and 4) appropriateness of plans with pre- and post-presentation of population data. Finally, we redesigned the population information display based on our findings from this study.

RESULTS

The qualitative data analysis for preferences of population information display resulted in four themes: 1) trusting the big/population data can be an issue, 2) embedded analytics is necessary to explore patient similarities, 3) need for tools to control the view (overview, zoom and filter), and 4) different presentations of the population display can be beneficial to improve the display. We found that appropriateness of plans was at 60% for both groups (t9=-1.9; p=0.08), and overall time looking at the population information display was 2.3 minutes versus 3.6 minutes with experts processing information faster than non-experts (t8= -2.3, p=0.04).

CONCLUSION

A population database has great potential for reducing complexity and uncertainty in medicine to improve clinical care. The preferences identified for the population information display will guide future health information technology system designers for better and more intuitive display.

Data-Driven Healthcare: Challenges and Opportunities for Interactive Visualization

The healthcare industry's widespread digitization efforts are reshaping one of the largest sectors of the world's economy. This transformation is enabling systems that promise to use ever-improving data-driven evidence to help doctors make more precise diagnoses, institutions identify at risk patients for intervention, clinicians develop more personalized treatment plans, and researchers better understand medical outcomes within complex patient populations. Given the scale and complexity of the data required to achieve these goals, advanced data visualization tools have the potential to play a critical role. This article reviews a number of visualization challenges unique to the healthcare discipline.

Welcome to Volume 5 of the Journal of Comparative Effectiveness Research

Happy New Year to all of our readers, and welcome to the fifth volume of the Journal of Comparative Effectiveness Research. 2014 was another exciting year for the journal, and below you will find some content highlights. I would like to take this opportunity to thank all of our Editorial Board members, readers and contributing authors for their support to date; I look forward to seeing the journal move forward in 2016.

Comparing high-dimensional confounder control methods for rapid cohort studies from electronic health records

Aims:

Electronic health records (EHR), containing rich clinical histories of large patient populations, can provide evidence for clinical decisions when evidence from trials and literature is absent. To enable such observational studies from EHR in real time, particularly in emergencies, rapid confounder control methods that can handle numerous variables and adjust for biases are imperative. This study compares the performance of 18 automatic confounder control methods.

Methods:

Methods include propensity scores, direct adjustment by machine learning, similarity matching and resampling in two simulated and one real-world EHR datasets.

Results & conclusions:

Direct adjustment by lasso regression and ensemble models involving multiple resamples have performance comparable to expert-based propensity scores and thus, may help provide real-time EHR-based evidence for timely clinical decisions.