A knowledge-based, concept-oriented view generation system for clinical data

Ensuring electronic medical record simulation through better training, modeling, and evaluation

OBJECTIVE

Electronic medical records (EMRs) can support medical research and discovery, but privacy risks limit the sharing of such data on a wide scale. Various approaches have been developed to mitigate risk, including record simulation via generative adversarial networks (GANs). While showing promise in certain application domains, GANs lack a principled approach for EMR data that induces subpar simulation. In this article, we improve EMR simulation through a novel pipeline that (1) enhances the learning model, (2) incorporates evaluation criteria for data utility that informs learning, and (3) refines the training process.

MATERIALS AND METHODS

We propose a new electronic health record generator using a GAN with a Wasserstein divergence and layer normalization techniques. We designed 2 utility measures to characterize similarity in the structural properties of real and simulated EMRs in the original and latent space, respectively. We applied a filtering strategy to enhance GAN training for low-prevalence clinical concepts. We evaluated the new and existing GANs with utility and privacy measures (membership and disclosure attacks) using billing codes from over 1 million EMRs at Vanderbilt University Medical Center.

RESULTS

The proposed model outperformed the state-of-the-art approaches with significant improvement in retaining the nature of real records, including prediction performance and structural properties, without sacrificing privacy. Additionally, the filtering strategy achieved higher utility when the EMR training dataset was small.

CONCLUSIONS

These findings illustrate that EMR simulation through GANs can be substantially improved through more appropriate training, modeling, and evaluation criteria.

Association of Health Record Visualizations With Physicians' Cognitive Load When Prioritizing Hospitalized Patients

IMPORTANCE

Current electronic health records (EHRs) contribute to increased physician cognitive workload when completing clinical tasks.

OBJECTIVE

To assess the association of different design features of an EHR-based information visualization tool with the cognitive load of physicians during the clinical prioritization process.

DESIGN, SETTING, AND PARTICIPANTS

This cross-sectional study included a convenience sample of 29 attending physicians at Seattle Children's Hospital, a large tertiary academic pediatric hospital. Data collection took place from August 2017 through October 2017, and analysis occurred from August to October 2018.

EXPOSURE

Physician participants used 3 prototypes with novel visualizations of simulated EHR data that highlighted 1 of 3 key patient characteristics, as follows: (1) acuity, (2) clinical problem list, and (3) clinical change.

MAIN OUTCOMES AND MEASURES

Cognitive workload was measured using the NASA Task Load Index (TLX) scale (range, 1-100, with lower scores indicating lower cognitive workload). Cognitive workload was assessed for the 2 following clinical prioritization tasks: (1) finding information for a specific patient and (2) comparing results among patients for each prototype. Participants ranked 5 hypothetical patients from having the highest to the lowest priority in each design.

RESULTS

A total of 29 physician participants (15 [52%] men; 14 [48%] women; mean [range] age, 43 [35-58] years; mean [range] time in practice, 11 [3-30] years) completed the study. For task 1, the prototype highlighting clinical change was associated with lower median (interquartile range) NASA TLX scores compared with the prototype highlighting acuity (30.3 [15.2-41.6] vs 48.5 [18.7-59.3]; P = .02). For task 2, the prototype highlighting clinical change was associated with lower median (interquartile range) NASA TLX scores compared with the prototype highlighting the clinical problem list (29.1 [16.3-50.8] vs 43.5 [26.6-55.9]; P = .02). The prototype highlighting clinical change had the lowest TLX score in 17 of 29 rankings (59%) for task 1 (χ24 = 24.4; P < .001) and 18 of 29 rankings (62%) for task 2 (χ24 = 17.2; P = .002).

CONCLUSIONS AND RELEVANCE

In this study, well-designed EHR-based information visualizations that highlighted and featured clinically meaningful information patterns significantly reduced physician cognitive workload when prioritizing patient needs.

Patient information organization in the intensive care setting: expert knowledge elicitation with card sorting methods

Introduction

Many electronic health records fail to support information uptake because they impose low-level information organization tasks on users. Clinical concept-oriented views have shown information processing improvements, but the specifics of this organization for critical care are unclear.

Objective

To determine high-level cognitive processes and patient information organization schema in critical care.

Methods

We conducted an open card sort of 29 patient data elements and a modified Delphi card sort of 65 patient data elements. Study participants were 39 clinicians with varied critical care training and experience. We analyzed the open sort with a hierarchical cluster analysis (HCA) and factor analysis (FA). The Delphi sort was split into three initiating groups that resulted in three unique solutions. We compared results between open sort analyses (HCA and FA), between card sorting exercises (open and Delphi), and across the Delphi solutions.

Results

Between the HCA and FA, we observed common constructs including cardiovascular and hemodynamics, infectious disease, medications, neurology, patient overview, respiratory, and vital signs. The more comprehensive Delphi sort solutions also included gastrointestinal, renal, and imaging constructs.

Conclusions

We identified primarily system-based groupings (e.g., cardiovascular, respiratory). Source-based (e.g., medications, laboratory) groups became apparent when participants were asked to sort a longer list of concepts. These results suggest a hybrid approach to information organization, which may combine systems, source, or problem-based groupings, best supports clinicians' mental models. These results can contribute to the design of information displays to better support clinicians' access and interpretation of information for critical care decisions.

Guiding Principles for a Pediatric Neurology ICU (neuroPICU) Bedside Multimodal Monitor: Findings from an International Working Group

BACKGROUND

Physicians caring for children with serious acute neurologic disease must process overwhelming amounts of physiological and medical information. Strategies to optimize real time display of this information are understudied.

OBJECTIVES

Our goal was to engage clinical and engineering experts to develop guiding principles for creating a pediatric neurology intensive care unit (neuroPICU) monitor that integrates and displays data from multiple sources in an intuitive and informative manner.

METHODS

To accomplish this goal, an international group of physicians and engineers communicated regularly for one year. We integrated findings from clinical observations, interviews, a survey, signal processing, and visualization exercises to develop a concept for a neuroPICU display.

RESULTS

Key conclusions from our efforts include: (1) A neuroPICU display should support (a) rapid review of retrospective time series (i.e. cardiac, pulmonary, and neurologic physiology data), (b) rapidly modifiable formats for viewing that data according to the specialty of the reviewer, and (c) communication of the degree of risk of clinical decline. (2) Specialized visualizations of physiologic parameters can highlight abnormalities in multivariable temporal data. Examples include 3-D stacked spider plots and color coded time series plots. (3) Visual summaries of EEG with spectral tools (i.e. hemispheric asymmetry and median power) can highlight seizures via patient-specific "fingerprints." (4) Intuitive displays should emphasize subsets of physiology and processed EEG data to provide a rapid gestalt of the current status and medical stability of a patient.

CONCLUSIONS

A well-designed neuroPICU display must present multiple datasets in dynamic, flexible, and informative views to accommodate clinicians from multiple disciplines in a variety of clinical scenarios.

Integration of an OWL-DL knowledge base with an EHR prototype and providing customized information

When clinicians use electronic health record (EHR) systems, their ability to obtain general knowledge is often an important contribution to their ability to make more informed decisions. In this paper we describe a method by which an external, formal representation of clinical and molecular genetic knowledge can be integrated into an EHR such that customized knowledge can be delivered to clinicians in a context-appropriate manner.Web Ontology Language-Description Logic (OWL-DL) is a formal knowledge representation language that is widely used for creating, organizing and managing biomedical knowledge through the use of explicit definitions, consistent structure and a computer-processable format, particularly in biomedical fields. In this paper we describe: 1) integration of an OWL-DL knowledge base with a standards-based EHR prototype, 2) presentation of customized information from the knowledge base via the EHR interface, and 3) lessons learned via the process. The integration was achieved through a combination of manual and automatic methods. Our method has advantages for scaling up to and maintaining knowledge bases of any size, with the goal of assisting clinicians and other EHR users in making better informed health care decisions.

Data utilization for medical decision making at the time of patient admission to ICU

OBJECTIVES

Information overload in electronic medical records can impede providers' ability to identify important clinical data and may contribute to medical error. An understanding of the information requirements of ICU providers will facilitate the development of information systems that prioritize the presentation of high-value data and reduce information overload. Our objective was to determine the clinical information needs of ICU physicians, compared to the data available within an electronic medical record.

DESIGN

Prospective observational study and retrospective chart review.

SETTING

Three ICUs (surgical, medical, and mixed) at an academic referral center.

SUBJECTS

Newly admitted ICU patients and physicians (residents, fellows, and attending staff).

MEASUREMENTS AND MAIN RESULTS

The clinical information used by physicians during the initial diagnosis and treatment of admitted patients was captured using a questionnaire. Clinical information concepts were ranked according to the frequency of reported use (primary outcome) and were compared to information availability in the electronic medical record (secondary outcome). Nine hundred twenty-five of 1,277 study questionnaires (408 patients) were completed. Fifty-one clinical information concepts were identified as being useful during ICU admission. A median (interquartile range) of 11 concepts (6-16) was used by physicians per patient admission encounter with four used greater than 50% of the time. Over 25% of the clinical data available in the electronic medical record was never used, and only 33% was used greater than 50% of the time by admitting physicians.

CONCLUSIONS

Physicians use a limited number of clinical information concepts at the time of patient admission to the ICU. The electronic medical record contains an abundance of unused data. Better electronic data management strategies are needed, including the priority display of frequently used clinical concepts within the electronic medical record, to improve the efficiency of ICU care.

Information extraction for tracking liver cancer patients' statuses: from mixture of clinical narrative report types

OBJECTIVE

To provide an efficient way for tracking patients' condition over long periods of time and to facilitate the collection of clinical data from different types of narrative reports, it is critical to develop an efficient method for smoothly analyzing the clinical data accumulated in narrative reports.

MATERIALS AND METHODS

To facilitate liver cancer clinical research, a method was developed for extracting clinical factors from various types of narrative clinical reports, including ultrasound reports, radiology reports, pathology reports, operation notes, admission notes, and discharge summaries. An information extraction (IE) module was developed for tracking disease progression in liver cancer patients over time, and a rule-based classifier was developed for answering whether patients met the clinical research eligibility criteria. The classifier provided the answers and direct/indirect evidence (evidence sentences) for the clinical questions. To evaluate the implemented IE module and the classifier, the gold-standard annotations and answers were developed manually, and the results of the implemented system were compared with the gold standard.

RESULTS

The IE model achieved an F-score from 92.40% to 99.59%, and the classifier achieved accuracy from 96.15% to 100%.

CONCLUSIONS

The application was successfully applied to the various types of narrative clinical reports. It might be applied to the key extraction for other types of cancer patients.

The readiness of SNOMED problem list concepts for meaningful use of electronic health records

OBJECTIVE

By 2015, SNOMED CT (SCT) will become the USA's standard for encoding diagnoses and problem lists in electronic health records (EHRs). To facilitate this effort, the National Library of Medicine has published the "SCT Clinical Observations Recording and Encoding" and the "Veterans Health Administration and Kaiser Permanente" problem lists (collectively, the "PL"). The PL is studied in regard to its readiness to support meaningful use of EHRs. In particular, we wish to determine if inconsistencies appearing in SCT, in general, occur as frequently in the PL, and whether further quality-assurance (QA) efforts on the PL are required.

METHODS AND MATERIALS

A study is conducted where two random samples of SCT concepts are compared. The first consists of concepts strictly from the PL and the second contains general SCT concepts distributed proportionally to the PL's in terms of their hierarchies. Each sample is analyzed for its percentage of primitive concepts and for frequency of modeling errors of various severity levels as quality measures. A simple structural indicator, namely, the number of parents, is suggested to locate high likelihood inconsistencies in hierarchical relationships. The effectiveness of this indicator is evaluated.

RESULTS

PL concepts are found to be slightly better than other concepts in the respective SCT hierarchies with regards to the quality measure of the percentage of primitive concepts and the frequency of modeling errors. There were 58% primitive concepts in the PL sample versus 62% in the control sample. The structural indicator of number of parents is shown to be statistically significant in its ability to identify concepts having a higher likelihood of inconsistencies in their hierarchical relationships. The absolute number of errors in the group of concepts having 1-3 parents was shown to be significantly lower than that for concepts with 4-6 parents and those with 7 or more parents based on Chi-squared analyses.

CONCLUSION

PL concepts suffer from the same issues as general SCT concepts, although to a slightly lesser extent, and do require further QA efforts to promote meaningful use of EHRs. To support such efforts, a structural indicator is shown to effectively ferret out potentially problematic concepts where those QA efforts should be focused.

Context-based electronic health record: toward patient specific healthcare

Due to the increasingly data-intensive clinical environment, physicians now have unprecedented access to detailed clinical information from a multitude of sources. However, applying this information to guide medical decisions for a specific patient case remains challenging. One issue is related to presenting information to the practitioner: displaying a large (irrelevant) amount of information often leads to information overload. Next-generation interfaces for the electronic health record (EHR) should not only make patient data easily searchable and accessible, but also synthesize fragments of evidence documented in the entire record to understand the etiology of a disease and its clinical manifestation in individual patients. In this paper, we describe our efforts toward creating a context-based EHR, which employs biomedical ontologies and (graphical) disease models as sources of domain knowledge to identify relevant parts of the record to display. We hypothesize that knowledge (e.g., variables, relationships) from these sources can be used to standardize, annotate, and contextualize information from the patient record, improving access to relevant parts of the record and informing medical decision making. To achieve this goal, we describe a framework that aggregates and extracts findings and attributes from free-text clinical reports, maps findings to concepts in available knowledge sources, and generates a tailored presentation of the record based on the information needs of the user. We have implemented this framework in a system called Adaptive EHR, demonstrating its capabilities to present and synthesize information from neurooncology patients. This paper highlights the challenges and potential applications of leveraging disease models to improve the access, integration, and interpretation of clinical patient data.

A bootstrapping algorithm to improve cohort identification using structured data

Cohort identification is an important step in conducting clinical research studies. Use of ICD-9 codes to identify disease cohorts is a common approach that can yield satisfactory results in certain conditions; however, for many use-cases more accurate methods are required. In this study, we propose a bootstrapping method that supplements ICD-9 codes with lab results, medications, etc. to build classification models that can be used to identify cohorts more accurately. The proposed method does not require prior information about the true class of the patients. We used the method to identify Diabetes Mellitus (DM) and Hyperlipidemia (HL) patient cohorts from a database of 800 thousand patients. Evaluation results show that the method identified 11,000 patients who did not have DM related ICD-9 codes as positive for DM and 52,000 patients without HL codes as positive for HL. A review of 400 patient charts (200 patients for each condition) by two clinicians shows that in both the conditions studied, the labeling assigned by the proposed approach is more consistent with that of the clinicians compared to labeling through ICD-9 codes. The method is reasonably automated and, we believe, holds potential for inexpensive, more accurate cohort identification.

Problem-centric organization and visualization of patient imaging and clinical data

A patient's electronic medical record contains a large amount of unstructured textual information. As patient records become increasingly dense owing to an aging population and increased occurrence of chronic diseases, a tool is needed to help organize and navigate patient data in a way that facilitates a clinician's ability to understand this information and that improves efficiency. A system has been developed for physicians that summarizes clinical information from a patient record. This system provides a gestalt view of the patient's record by organizing information about each disease along four dimensions (axes): time (eg, disease progression over time), space (eg, tumor in left frontal lobe), existence (eg, certainty of existence of a finding), and causality (eg, response to treatment). A display is generated from information provided by radiology reports and discharge summaries. Natural language processing is used to identify clinical abnormalities (problems, symptoms, findings) from these reports as well as associated properties and relationships. This information is presented in an integrated format that organizes extracted findings into a problem list, depicts the information on a timeline grid, and provides direct access to relevant reports and images. The goal of this system is to improve the structure of clinical information and its presentation to the physician, thereby simplifying the information retrieval and knowledge discovery necessary to bridge the gap between acquiring raw data and making an informed diagnosis.

Grand challenges in clinical decision support

There is a pressing need for high-quality, effective means of designing, developing, presenting, implementing, evaluating, and maintaining all types of clinical decision support capabilities for clinicians, patients and consumers. Using an iterative, consensus-building process we identified a rank-ordered list of the top 10 grand challenges in clinical decision support. This list was created to educate and inspire researchers, developers, funders, and policy-makers. The list of challenges in order of importance that they be solved if patients and organizations are to begin realizing the fullest benefits possible of these systems consists of: improve the human-computer interface; disseminate best practices in CDS design, development, and implementation; summarize patient-level information; prioritize and filter recommendations to the user; create an architecture for sharing executable CDS modules and services; combine recommendations for patients with co-morbidities; prioritize CDS content development and implementation; create internet-accessible clinical decision support repositories; use freetext information to drive clinical decision support; mine large clinical databases to create new CDS. Identification of solutions to these challenges is critical if clinical decision support is to achieve its potential and improve the quality, safety and efficiency of healthcare.

TimeLine: visualizing integrated patient records

An increasing amount of data is now accrued in medical information systems; however, the organization of this data is still primarily driven by data source, and does not support the cognitive processes of physicians. As such, new methods to visualize patient medical records are becoming imperative in order to assist physicians with clinical tasks and medical decision-making. The TimeLine system is a problem-centric temporal visualization for medical data: information contained with medical records is reorganized around medical disease entities and conditions. Automatic construction of the TimeLine display from existing clinical repositories occurs in three steps: 1) data access, which uses an eXtensible Markup Language (XML) data representation to handle distributed, heterogeneous medical databases; 2) data mapping and reorganization, reformulating data into hierarchical, problemcentric views; and 3) data visualization, which renders the display to a target presentation platform. Leveraging past work, we describe the latter two components of the TimeLine system in this paper, and the issues surrounding the creation of medical problems lists and temporal visualization of medical data. A driving factor in the development of TimeLine was creating a foundation upon which new data types and the visualization metaphors could be readily incorporated.

Mismatched concepts in a neonatal intensive care unit (NICU): further issues for computer decision support?

BACKGROUND

Common concepts and definitions are important for the effective practice of medicine. In an intensive care unit clear understanding of terminology and communication between different staff groups may be critical for optimal care. If computerised decision support tools are to be successfully deployed in these high intensity environments, all staff must understand the concepts and information that is to be portrayed.

OBJECTIVE

To examine the similarity of language and concepts related to newborn infants and their care in staff groups with different experience.

METHODS

An experimental study in a tertiary regional neonatal intensive care unit involved 32 staff in 4 groups with varying experience (junior and senior / nurses and doctors). A psychologist developed a lexicon of clinical actions and a second lexicon of possible patient descriptors applicable to newborn infants receiving all degrees of neonatal care by conducting interviews with staff on the unit. Card sorting experiments were performed on the terms in the action and descriptor lexicons, the staff being asked to group words that were related to the same or similar concepts. The card sort data were analysed using conventional cluster analysis to produce tree-diagrams or dendrograms and then by distance matrix analysis to give cumulative probability plots.

RESULTS

Differences were shown in the way various classes of staff and staff with different experience mentally map clinical concepts. Clinical actions were grouped more randomly by nurses and by those with less experience with a polarisation between senior doctors and junior nurses. Descriptors were classed more definitively and similarly by junior and senior nurses and senior doctors but more randomly and quite differently by junior doctors. Thus there were differences seen between nurses and doctors with different roles within the unit and differences related to experience.

CONCLUSIONS

Concepts are used differently by various staff groups in a neonatal unit: this may have an impact on the effectiveness of computerised decision aids unless it is taken into account during their development.

User-oriented views in health care information systems

In this paper, we present the methodology we adopted in designing and developing an object-oriented database system for the management of medical records. The designed system provides technical solutions to important requirements of most clinical information systems, such as 1) the support of tools to create and manage views on data and view schemas, offering to different users specific perspectives on data tailored to their needs; 2) the capability to handle in a suitable way the temporal aspects related to clinical information; and 3) the effective integration of multimedia data. Remote data access for authorized users is also considered. As clinical application, we describe here the prototype of a user-oriented clinical information system for the archiving and the management of multimedia and temporally oriented clinical data related to percutaneous transluminal coronary angioplasty (PTCA) patients. Suitable view schemas for various user roles (cath-lab physician, ward nurse, general practitioner) have been modeled and implemented on the basis of a detailed analysis of the considered clinical environment, carried out by an object-oriented approach.

A practical approach to process support in health information systems

This article describes the design of a generator tool for rapid application development. The generator tool is an integral part of a healthcare information system, and newly developed applications are embedded into the healthcare information system from the very beginning. The tool-generated applications are based on a document oriented user interaction paradigm. A significant feature is the support of intra- and interdepartmental clinical processes by means of providing document flow between different user groups. For flexible storage of newly developed applications, a generic EAV-type (Entity-Attribute-Value) database schema is used. Important aspects of a consequent implementation, like database representation of structured documents, document flow, versioning, and synchronization are presented. Applications generated by this approach are in routine use in more than 200 hospitals in Germany.

Providing concept-oriented views for clinical data using a knowledge-based system: an evaluation

OBJECTIVE

Clinical information systems typically present patient data in chronologic order, organized by the source of the information (e.g., laboratory, radiology). This study evaluates the functionality and utility of a knowledge-based system that generates concept-oriented views (organized around clinical concepts such as disease or organ system) of clinical data.

DESIGN

The authors have developed a system that uses a knowledge base of interrelationships between medical concepts to infer relationships between data in electronic medical records. They use these inferences to produce summaries, or views, of the data that are relevant to a specific concept of interest. They evaluated the ability of the system to select relevant information, reduce information overload, and support physician information retrieval.

MEASUREMENTS

The sensitivity and specificity of the system for identifying relevant patient information were calculated. Effect on information overload was assessed by comparing the amount of information in each view with the amount of information in the entire record. Information retrieval accuracy and cost (time) were used to measure the effect of using concept-oriented views on the efficiency and effectiveness of retrievals.

RESULTS

The sensitivity and specificity of the system for identifying relevant clinical information were generally in the range of 70 to 80 percent. Concept-oriented views are effective in reducing the amount of information retrieved (over 80 percent reduction) and, compared with source-oriented views, are able to improve physician retrieval accuracy (p=0.04).

CONCLUSION

Computer-generated, concept-oriented views can be used to reduce clinician information overload and improve the accuracy of clinical data retrieval.