A methodology for interactive mining and visual analysis of clinical event patterns using electronic health record data

The Effect of Alignment on People's Ability to Judge Event Sequence Similarity

Event sequences are central to the analysis of data in domains that range from biology and health, to logfile analysis and people's everyday behavior. Many visualization tools have been created for such data, but people are error-prone when asked to judge the similarity of event sequences with basic presentation methods. This article describes an experiment that investigates whether local and global alignment techniques improve people's performance when judging sequence similarity. Participants were divided into three groups (basic versus local versus global alignment), and each participant judged the similarity of 180 sets of pseudo-randomly generated sequences. Each set comprised a target, a correct choice and a wrong choice. After training, the global alignment group was more accurate than the local alignment group (98 versus 93 percent correct), with the basic group getting 95 percent correct. Participants' response times were primarily affected by the number of event types, the similarity of sequences (measured by the Levenshtein distance) and the edit types (nine combinations of deletion, insertion and substitution). In summary, global alignment is superior and people's performance could be further improved by choosing alignment parameters that explicitly penalize sequence mismatches.

Visualization of Frequent Temporal Patterns in Single or Two Populations

Temporal knowledge discovery in clinical problems, is crucial to investigate problems in the data science era. Meaningful progress has been made computationally in the discovery of frequent temporal patterns, which may store potentially meaningful knowledge. However, for temporal knowledge discovery and acquisition, effective visualization is essential and still stores much room for contributions. While visualization of frequent temporal patterns was relatively under researched, it stores meaningful opportunities in facilitating usable ways to assist domain experts, or researchers, in exploring and acquiring temporal knowledge. In this paper, a novel approach for the visualization of an enumeration tree of frequent temporal patterns is introduced for, whether mined from a single population, or for the comparison of patterns that were discovered in two separate populations. While this approach is relevant to any sequence-based patterns, we demonstrate its use on the most complex scenario of time intervals related patterns (TIRPs). The interface enables users to browse an enumeration tree of frequent patterns, or search for specific patterns, as well as discover the most discriminating TIRPs among two populations. For that a novel visualization of the temporal patterns is introduced using a bubble chart, in which each bubble represents a temporal pattern, and the chart axes represent the various metrics of the patterns, such as their frequency, reoccurrence, and more, which provides a fast overview of the patterns as a whole, as well as access specific ones. We present a comprehensive and rigorous user study on two real-life datasets, demonstrating the usability advantages of the novel approaches.

How can Big Data Analytics Support People-Centred and Integrated Health Services: A Scoping Review

Introduction:

Health systems in high-income countries face a variety of challenges calling for a systemic approach to improve quality and efficiency. Putting people in the centre is the main idea of the WHO model of people-centred and integrated health services. Integrating health services is fuelled by an integration of health data with great potentials for decision support based on big data analytics. The research question of this paper is “How can big data analytics support people-centred and integrated health services?”

Methods:

A scoping review following the recommendations of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses – Scoping Review (PRISMA-ScR) statement was conducted to gather information on how big data analytics can support people-centred and integrated health services. The results were summarized in a role model of a people-centred and integrated health services platform illustrating which data sources might be integrated and which types of analytics might be applied to support the strategies of the people-centred and integrated health services framework to become more integrated across the continuum of care. Additional rapid literature reviews were conducted to generate frequency distributions of the most often used data types and analytical methods in the medical literature. Finally, the main challenges connected with big data analytics were worked out based on a content analysis of the results from the scoping literature review.

Results:

Based on the results from the rapid literature reviews the most often used data sources for big data analytics (BDA) in healthcare were biomarkers (39.3%) and medical images (30.9%). The most often used analytical models were support vector machines (27.3%) and neural networks (20.4%). The people-centred and integrated health services framework defines different strategic interventions for health services to become more integrated. To support all aspects of these interventions a comparably integrated platform of health-related data would be needed, so that a role model labelled as people-centred health platform was developed. Based on integrated data the results of the scoping review (n = 72) indicate, that big data analytics could for example support the strategic intervention of tailoring personalized health plans (43.1%), e.g. by predicting individual risk factors for different therapy options. Also BDA might enhance clinical decision support tools (31.9%), e.g. by calculating risk factors for disease uptake or progression. BDA might also assist in designing population-based services (26.4% by clustering comparable individuals in manageable risk groups e.g. mentored by specifically trained, non-medical professionals. The main challenges of big data analytics in healthcare were categorized in regulatory, (information-) technological, methodological, and cultural issues, whereas methodological challenges were mentioned most often (55.0%), followed by regulatory challenges (43.7%).

Discussion:

The BDA applications presented in this literature review are based on findings which have already been published. For some important components of the framework on people-centred care like enhancing the role of community care or establishing intersectoral partnerships between health and social care institutions only few examples of enabling big data analytical tools were found in the literature. Quite the opposite does this mean that these strategies have less potential value, but rather that the source systems in these fields need to be further developed to be suitable for big data analytics.

Conclusions:

Big data analytics can support people-centred and integrated health services e.g. by patient similarity stratifications or predictions of individual risk factors. But BDA fails to unfold its full potential until data source systems are still disconnected and actions towards a comprehensive and people-centred health-related data platform are politically insufficiently incentivized. This work highlighted the potential of big data analysis in the context of the model of people-centred and integrated health services, whereby the role model of the person-centered health platform can be used as a blueprint to support strategies to improve person-centered health care. Likely because health data is extremely sensitive and complex, there are only few practical examples of platforms to some extent already capable of merging and processing people-centred big data, but the integration of health data can be expected to further proceed so that analytical opportunities might also become reality in the near future.

VBridge: Connecting the Dots Between Features and Data to Explain Healthcare Models

Machine learning (ML) is increasingly applied to Electronic Health Records (EHRs) to solve clinical prediction tasks. Although many ML models perform promisingly, issues with model transparency and interpretability limit their adoption in clinical practice. Directly using existing explainable ML techniques in clinical settings can be challenging. Through literature surveys and collaborations with six clinicians with an average of 17 years of clinical experience, we identified three key challenges, including clinicians' unfamiliarity with ML features, lack of contextual information, and the need for cohort-level evidence. Following an iterative design process, we further designed and developed VBridge, a visual analytics tool that seamlessly incorporates ML explanations into clinicians' decision-making workflow. The system includes a novel hierarchical display of contribution-based feature explanations and enriched interactions that connect the dots between ML features, explanations, and data. We demonstrated the effectiveness of VBridge through two case studies and expert interviews with four clinicians, showing that visually associating model explanations with patients' situational records can help clinicians better interpret and use model predictions when making clinician decisions. We further derived a list of design implications for developing future explainable ML tools to support clinical decision-making.

Interactive Exploration of Longitudinal Cancer Patient Histories Extracted From Clinical Text

PURPOSE

Retrospective cancer research requires identification of patients matching both categorical and temporal inclusion criteria, often on the basis of factors exclusively available in clinical notes. Although natural language processing approaches for inferring higher-level concepts have shown promise for bringing structure to clinical texts, interpreting results is often challenging, involving the need to move between abstracted representations and constituent text elements. Our goal was to build interactive visual tools to support the process of interpreting rich representations of histories of patients with cancer.

METHODS

Qualitative inquiry into user tasks and goals, a structured data model, and an innovative natural language processing pipeline were used to guide design.

RESULTS

The resulting information visualization tool provides cohort- and patient-level views with linked interactions between components.

CONCLUSION

Interactive tools hold promise for facilitating the interpretation of patient summaries and identification of cohorts for retrospective research.

Multispecialty Enterprise Imaging Workgroup Consensus on Interactive Multimedia Reporting Current State and Road to the Future: HIMSS-SIIM Collaborative White Paper

Diagnostic and evidential static image, video clip, and sound multimedia are captured during routine clinical care in cardiology, dermatology, ophthalmology, pathology, physiatry, radiation oncology, radiology, endoscopic procedural specialties, and other medical disciplines. Providers typically describe the multimedia findings in contemporaneous electronic health record clinical notes or associate a textual interpretative report. Visual communication aids commonly used to connect, synthesize, and supplement multimedia and descriptive text outside medicine remain technically challenging to integrate into patient care. Such beneficial interactive elements may include hyperlinks between text, multimedia elements, alphanumeric and geometric annotations, tables, graphs, timelines, diagrams, anatomic maps, and hyperlinks to external educational references that patients or provider consumers may find valuable. This HIMSS-SIIM Enterprise Imaging Community workgroup white paper outlines the current and desired clinical future state of interactive multimedia reporting (IMR). The workgroup adopted a consensus definition of IMR as “interactive medical documentation that combines clinical images, videos, sound, imaging metadata, and/or image annotations with text, typographic emphases, tables, graphs, event timelines, anatomic maps, hyperlinks, and/or educational resources to optimize communication between medical professionals, and between medical professionals and their patients.” This white paper also serves as a precursor for future efforts toward solving technical issues impeding routine interactive multimedia report creation and ingestion into electronic health records.

Review of Temporal Reasoning in the Clinical Domain for Timeline Extraction: Where we are and where we need to be

Understanding a patient's medical history, such as how long symptoms last or when a procedure was performed, is vital to diagnosing problems and providing good care. Frequently, important information regarding a patient's medical timeline is buried in their Electronic Health Record (EHR) in the form of unstructured clinical notes. This results in care providers spending time reading notes in a patient's record in order to become familiar with their condition prior to developing a diagnosis or treatment plan. Valuable time could be saved if this information was readily accessible for searching and visualization for fast comprehension by the medical team. Clinical Natural Language Processing (NLP) is an area of research that aims to build computational methods to automatically extract medically relevant information from unstructured clinical texts. A key component of Clinical NLP is Temporal Reasoning, as understanding a patient's medical history relies heavily on the ability to identify, assimilate, and reason over temporal information. In this work, we review the current state of Temporal Reasoning in the clinical domain with respect to Clinical Timeline Extraction. While much progress has been made, the current state-of-the-art still has a ways to go before practical application in the clinical setting will be possible. Areas such as handling relative and implicit temporal expressions, both in normalization and in identifying temporal relationships, improving co-reference resolution, and building inter-operable timeline extraction tools that can integrate multiple types of data are in need of new and innovative solutions to improve performance on clinical data.

Temporal condition pattern mining in large, sparse electronic health record data: A case study in characterizing pediatric asthma

Objective

This study introduces a temporal condition pattern mining methodology to address the sparse nature of coded condition concept utilization in electronic health record data. As a validation study, we applied this method to reveal condition patterns surrounding an initial diagnosis of pediatric asthma.

Materials and Methods

The SPADE (Sequential PAttern Discovery using Equivalence classes) algorithm was used to identify common temporal condition patterns surrounding the initial diagnosis of pediatric asthma in a study population of 71 824 patients from the Children’s Hospital of Philadelphia. SPADE was applied to a dataset with diagnoses coded using International Classification of Diseases (ICD) concepts and separately to a dataset with the ICD codes mapped to their corresponding expanded diagnostic clusters (EDCs). Common temporal condition patterns surrounding the initial diagnosis of pediatric asthma ascertained by SPADE from both the ICD and EDC datasets were compared.

Results

SPADE identified 36 unique diagnoses in the mapped EDC dataset, whereas only 19 were recognized in the ICD dataset. Temporal trends in condition diagnoses ascertained from the EDC data were not discoverable in the ICD dataset.

Discussion

Mining frequent temporal condition patterns from large electronic health record datasets may reveal previously unknown associations between diagnoses that could inform future research into causation or other relationships. Mapping sparsely coded medical concepts into homogenous groups was essential to discovering potentially useful information from our dataset.

Conclusions

We expect that the presented methodology is applicable to the study of diagnostic trajectories for other clinical conditions and can be extended to study temporal patterns of other coded medical concepts such as medications and procedures.

Visual Analytics for Electronic Health Records: A Review

The increasing use of electronic health record (EHR)-based systems has led to the generation of clinical data at an unprecedented rate, which produces an untapped resource for healthcare experts to improve the quality of care. Despite the growing demand for adopting EHRs, the large amount of clinical data has made some analytical and cognitive processes more challenging. The emergence of a type of computational system called visual analytics has the potential to handle information overload challenges in EHRs by integrating analytics techniques with interactive visualizations. In recent years, several EHR-based visual analytics systems have been developed to fulfill healthcare experts’ computational and cognitive demands. In this paper, we conduct a systematic literature review to present the research papers that describe the design of EHR-based visual analytics systems and provide a brief overview of 22 systems that met the selection criteria. We identify and explain the key dimensions of the EHR-based visual analytics design space, including visual analytics tasks, analytics, visualizations, and interactions. We evaluate the systems using the selected dimensions and identify the gaps and areas with little prior work.

Visual Analytic Tools and Techniques in Population Health and Health Services Research: Scoping Review

Background

Visual analytics (VA) promotes the understanding of data with visual, interactive techniques, using analytic and visual engines. The analytic engine includes automated techniques, whereas common visual outputs include flow maps and spatiotemporal hot spots.

Objective

This scoping review aims to address a gap in the literature, with the specific objective to synthesize literature on the use of VA tools, techniques, and frameworks in interrelated health care areas of population health and health services research (HSR).

Methods

Using the 2018 PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) guidelines, the review focuses on peer-reviewed journal articles and full conference papers from 2005 to March 2019. Two researchers were involved at each step, and another researcher arbitrated disagreements. A comprehensive abstraction platform captured data from diverse bodies of the literature, primarily from the computer and health sciences.

Results

After screening 11,310 articles, findings from 55 articles were synthesized under the major headings of visual and analytic engines, visual presentation characteristics, tools used and their capabilities, application to health care areas, data types and sources, VA frameworks, frameworks used for VA applications, availability and innovation, and co-design initiatives. We found extensive application of VA methods used in areas of epidemiology, surveillance and modeling, health services access, use, and cost analyses. All articles included a distinct analytic and visualization engine, with varying levels of detail provided. Most tools were prototypes, with 5 in use at the time of publication. Seven articles presented methodological frameworks. Toward consistent reporting, we present a checklist, with an expanded definition for VA applications in health care, to assist researchers in sharing research for greater replicability. We summarized the results in a Tableau dashboard.

Conclusions

With the increasing availability and generation of big health care data, VA is a fast-growing method applied to complex health care data. What makes VA innovative is its capability to process multiple, varied data sources to demonstrate trends and patterns for exploratory analysis, leading to knowledge generation and decision support. This is the first review to bridge a critical gap in the literature on VA methods applied to the areas of population health and HSR, which further indicates possible avenues for the adoption of these methods in the future. This review is especially important in the wake of COVID-19 surveillance and response initiatives, where many VA products have taken center stage.

International Registered Report Identifier (IRRID)

RR2-10.2196/14019

Visualization of aggregate perioperative data improves anesthesia case planning: A randomized, cross-over trial

STUDY OBJECTIVE

A challenge in reducing unwanted care variation is effectively managing the wide variety of performed surgical procedures. While an organization may perform thousands of types of cases, privacy and logistical constraints prevent review of previous cases to learn about prior practices. To bridge this gap, we developed a system for extracting key data from anesthesia records. Our objective was to determine whether usage of the system would improve case planning performance for anesthesia residents.

DESIGN

Randomized, cross-over trial.

SETTING

Vanderbilt University Medical Center.

MEASUREMENTS

We developed a web-based, data visualization tool for reviewing de-identified anesthesia records. First year anesthesia residents were recruited and performed simulated case planning tasks (e.g., selecting an anesthetic type) across six case scenarios using a randomized, cross-over design after a baseline assessment. An algorithm scored case planning performance based on care components selected by residents occurring frequently among prior anesthetics, which was scored on a 0-4 point scale. Linear mixed effects regression quantified the tool effect on the average performance score, adjusting for potential confounders.

MAIN RESULTS

We analyzed 516 survey questionnaires from 19 residents. The mean performance score was 2.55 ± SD 0.32. Utilization of the tool was associated with an average score improvement of 0.120 points (95% CI 0.060 to 0.179; p < 0.001). Additionally, a 0.055 point improvement due to the "learning effect" was observed from each assessment to the next (95% CI 0.034 to 0.077; p < 0.001). Assessment score was also significantly associated with specific case scenarios (p < 0.001).

CONCLUSIONS

This study demonstrated the feasibility of developing of a clinical data visualization system that aggregated key anesthetic information and found that the usage of tools modestly improved residents' performance in simulated case planning.

Quantification and visualisation methods of data-driven chronic care delivery pathways: protocol for a systematic review and content analysis

INTRODUCTION

Chronic conditions require long periods of care and often involve repeated interactions with multiple healthcare providers. Faced with increasing illness burden and costs, healthcare systems are currently working towards integrated care to streamline these interactions and improve efficiency. To support this, one promising resource is the information on routine care delivery stored in various electronic healthcare databases (EHD). In chronic conditions, care delivery pathways (CDPs) can be constructed by linking multiple data sources and extracting time-stamped healthcare utilisation events and other medical data related to individual or groups of patients over specific time periods; CDPs may provide insights into current practice and ways of improving it. Several methods have been proposed in recent years to quantify and visualise CDPs. We present the protocol for a systematic review aiming to describe the content and development of CDP methods, to derive common recommendations for CDP construction.

METHODS AND ANALYSIS

This protocol followed the Preferred Reporting Items for Systematic review and Meta-Analysis Protocols. A literature search will be performed in PubMed (MEDLINE), Scopus, IEEE, CINAHL and EMBASE, without date restrictions, to review published papers reporting data-driven chronic CDPs quantification and visualisation methods. We will describe them using several characteristics relevant for EHD use in long-term care, grouped into three domains: (1) clinical (what clinical information does the method use and how was it considered relevant?), (2) data science (what are the method's development and implementation characteristics?) and (3) behavioural (which behaviours and interactions does the method aim to promote among users and how?). Data extraction will be performed via deductive content analysis using previously defined characteristics and accompanied by an inductive analysis to identify and code additional relevant features. Results will be presented in descriptive format and used to compare current CDPs and generate recommendations for future CDP development initiatives.

ETHICS AND DISSEMINATION

Database searches will be initiated in May 2019. The review is expected to be completed by February 2020. Ethical approval is not required for this review. Results will be disseminated in peer-reviewed journals and conference presentations.

PROSPERO REGISTRATION NUMBER

CRD42019140494.

Visual Analysis of High-Dimensional Event Sequence Data via Dynamic Hierarchical Aggregation

Temporal event data are collected across a broad range of domains, and a variety of visual analytics techniques have been developed to empower analysts working with this form of data. These techniques generally display aggregate statistics computed over sets of event sequences that share common patterns. Such techniques are often hindered, however, by the high-dimensionality of many real-world event sequence datasets which can prevent effective aggregation. A common coping strategy for this challenge is to group event types together prior to visualization, as a pre-process, so that each group can be represented within an analysis as a single event type. However, computing these event groupings as a pre-process also places significant constraints on the analysis. This paper presents a new visual analytics approach for dynamic hierarchical dimension aggregation. The approach leverages a predefined hierarchy of dimensions to computationally quantify the informativeness, with respect to a measure of interest, of alternative levels of grouping within the hierarchy at runtime. This information is then interactively visualized, enabling users to dynamically explore the hierarchy to select the most appropriate level of grouping to use at any individual step within an analysis. Key contributions include an algorithm for interactively determining the most informative set of event groupings for a specific analysis context, and a scented scatter-plus-focus visualization design with an optimization-based layout algorithm that supports interactive hierarchical exploration of alternative event type groupings. We apply these techniques to high-dimensional event sequence data from the medical domain and report findings from domain expert interviews.

When order sets do not align with clinician workflow: assessing practice patterns in the electronic health record

BACKGROUND

Order sets are widely used tools in the electronic health record (EHR) for improving healthcare quality. However, there is limited insight into how well they facilitate clinician workflow. We assessed four indicators based on order set usage patterns in the EHR that reflect potential misalignment between order set design and clinician workflow needs.

METHODS

We used data from the EHR on all orders of medication, laboratory, imaging and blood product items at an academic hospital and an itemset mining approach to extract orders that frequently co-occurred with order set use. We identified the following four indicators: infrequent ordering of order set items, rapid retraction of medication orders from order sets, additional a la carte ordering of items not included in order sets and a la carte ordering of items despite being listed in the order set.

RESULTS

There was significant variability in workflow alignment across the 11 762 order set items used in the 77 421 inpatient encounters from 2014 to 2017. The median ordering rate was 4.1% (IQR 0.6%-18%) and median medication retraction rate was 4% (IQR 2%-10%). 143 (5%) medications were significantly less likely while 68 (3%) were significantly more likely to be retracted than if the same medication was ordered a la carte. 214 (39%) order sets were associated with least one additional item frequently ordered a la carte and 243 (45%) order sets contained at least one item that was instead more often ordered a la carte.

CONCLUSION

Order sets often do not align with what clinicians need at the point of care. Quantitative insights from EHRs may inform how order sets can be optimised to facilitate clinician workflow.

Visual Analytic Tools and Techniques in Population Health and Health Services Research: Protocol for a Scoping Review

Background

Visual analytics (VA) promotes the understanding of data using visual, interactive techniques and using analytic and visual engines. The analytic engine includes machine learning and other automated techniques, whereas common visual outputs include flow maps and spatiotemporal hotspots for studying service gaps and disease distribution. The principal objective of this scoping review is to examine the state of science on VA and the various tools, strategies, and frameworks used in population health and health services research (HSR).

Objective

The purpose of this scoping review is to develop an overarching global view of established techniques, frameworks, and methods of VA in population health and HSR. The main objectives are to explore, map, and synthesize the literature related to VA in its application to the two main focus areas of health care.

Methods

We will use established scoping review methods to meet the study objective. As the use of the term visual analytics is inconsistent, one of the major challenges was operationalizing the concepts for developing the search strategy, based on the three main concepts of population health, HSR, and VA. We included peer reviewed and grey literature sources from 2005 till March 2019 in the search. Independent teams of researchers will screen the titles, abstracts and full text articles, whereas an independent researcher will arbiter conflicts. Data will be abstracted and presented using the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews checklist and explanation by two independent researchers.

Results

As of late August 2019, the scoping review is in the full-text screening stage. Data synthesis will follow and the first results are expected to be submitted for publication in December 2019. In this protocol, the methods for undertaking this scoping review are detailed. We present how we operationalized the varied concepts of population health, health services, and VA. The main results of the scoping review will synthesize peer reviewed and grey literature sources on the main methods of VA in the interrelated fields of population health and health services research from January 2005 till March 2019.

Conclusions

VA is being increasingly used and integrated with emerging technologies to support decision making using large data sets. This scoping review of the VA tools, strategies, and frameworks applied to population health and health services aims to increase awareness of this approach for uptake by decision makers working within and toward developing learning health systems globally.

International Registered Report Identifier (IRRID)

DERR1-10.2196/14019

Clinical Information Systems and Artificial Intelligence: Recent Research Trends

OBJECTIVES

This survey aims at reviewing the literature related to Clinical Information Systems (CIS), Hospital Information Systems (HIS), Electronic Health Record (EHR) systems, and how collected data can be analyzed by Artificial Intelligence (AI) techniques.

METHODS

We selected the major journals (11 journals) collecting papers (more than 7,000) over the last five years from the top members of the research community, and read and analyzed the papers (more than 200) covering the topics. Then, we completed the analysis using search engines to also include papers from major conferences over the same five years.

RESULTS

We defined a taxonomy of major features and research areas of CIS, HIS, EHR systems. We also defined a taxonomy for the use of Artificial Intelligence (AI) techniques on healthcare data. In the light of these taxonomies, we report on the most relevant papers from the literature.

CONCLUSIONS

We highlighted some major research directions and issues which seem to be promising and to need further investigations over a medium- or long-term period.

Composer-Visual Cohort Analysis of Patient Outcomes

OBJECTIVE

Visual cohort analysis utilizing electronic health record data has become an important tool in clinical assessment of patient outcomes. In this article, we introduce Composer, a visual analysis tool for orthopedic surgeons to compare changes in physical functions of a patient cohort following various spinal procedures. The goal of our project is to help researchers analyze outcomes of procedures and facilitate informed decision-making about treatment options between patient and clinician.

METHODS

In collaboration with orthopedic surgeons and researchers, we defined domain-specific user requirements to inform the design. We developed the tool in an iterative process with our collaborators to develop and refine functionality. With Composer, analysts can dynamically define a patient cohort using demographic information, clinical parameters, and events in patient medical histories and then analyze patient-reported outcome scores for the cohort over time, as well as compare it to other cohorts. Using Composer's current iteration, we provide a usage scenario for use of the tool in a clinical setting.

CONCLUSION

We have developed a prototype cohort analysis tool to help clinicians assess patient treatment options by analyzing prior cases with similar characteristics. Although Composer was designed using patient data specific to orthopedic research, we believe the tool is generalizable to other healthcare domains. A long-term goal for Composer is to develop the application into a shared decision-making tool that allows translation of comparison and analysis from a clinician-facing interface into visual representations to communicate treatment options to patients.

A roadmap for semi-automatically extracting predictive and clinically meaningful temporal features from medical data for predictive modeling

Predictive modeling based on machine learning with medical data has great potential to improve healthcare and reduce costs. However, two hurdles, among others, impede its widespread adoption in hdealthcare. First, medical data are by nature longitudinal. Pre-processing them, particularly for feature engineering, is labor intensive and often takes 50-80% of the model building effort. Predictive temporal features are the basis of building accurate models, but are difficult to identify. This is problematic. Healthcare systems have limited resources for model building, while inaccurate models produce sub-optimal outcomes and are often useless. Second, most machine learning models provide no explanation of their prediction results. However, offering such explanations is essential for a model to be used in usual clinical practice. To address these two hurdles, this paper outlines: 1) a data-driven method for semi-automatically extracting predictive and clinically meaningful temporal features from medical data for predictive modeling; and 2) a method of using these features to automatically explain machine learning prediction results and suggest tailored interventions. This provides a roadmap for future research.

Identifying, Analyzing, and Visualizing Diagnostic Paths for Patients with Nonspecific Abdominal Pain

BACKGROUND

Diagnosis is complex, uncertain, and error-prone. Symptoms such as nonspecific abdominal pain are especially challenging. A diagnostic path consists of diagnostic steps taken from initial presentation until a diagnosis is obtained or the evaluation ends for other reasons. Analysis of diagnostic paths can reveal patterns associated with more timely and accurate diagnosis. Visual analytics can be used to enhance both analysis and comprehension of diagnostic paths.

OBJECTIVE

This article applies process-mining methods to extract and visualize diagnostic paths from electronic health records (EHRs).

METHODS

Patient features, actions taken (i.e., tests, referrals, etc.), and diagnoses obtained for 501 adult patients (half female, half ≥50 years of age) presenting with abdominal pain were extracted from an EHR database to construct diagnostic paths from a hospital system in suburban Chicago, Illinois, United States. A stable diagnosis was defined as the same diagnosis recorded twice in a 12-month period; a working diagnosis was recorded only once. Three different types of path visualizations were obtained.

RESULTS

A stable diagnosis was obtained in 63 (13%) patients after 12 months. In 271 (54%) patients, a working diagnosis was obtained. Mean path duration was 145.3 days (standard deviation, 195.1 days). These 63 patients received 75 stable diagnoses.

CONCLUSION

Structured EHR data can be used to construct diagnostic paths to gain insight into diagnostic practices for complaints such as abdominal pain.

Timeline representation of clinical data: usability and added value for pharmacovigilance

Background

Pharmacovigilance consists in monitoring and preventing the occurrence of adverse drug reactions (ADR). This activity requires the collection and analysis of data from the patient record or any other sources to find clues of a causality link between the drug and the ADR. This can be time-consuming because often patient data are heterogeneous and scattered in several files. To facilitate this task, we developed a timeline prototype to gather and classify patient data according to their chronology. Here, we evaluated its usability and quantified its contribution to routine pharmacovigilance using real ADR cases.

Methods

The timeline prototype was assessed using the biomedical data warehouse eHOP (from entrepôt de données biomédicales de l’HOPital) of the Rennes University Hospital Centre. First, the prototype usability was tested by six experts of the Regional Pharmacovigilance Centre of Rennes. Their experience was assessed with the MORAE software and a System and Usability Scale (SUS) questionnaire. Then, to quantify the timeline contribution to pharmacovigilance routine practice, three of them were asked to investigate possible ADR cases with the “Usual method” (analysis of electronic health record data with the DxCare software) or the “Timeline method”. The time to complete the task and the data quality in their reports (using the vigiGrade Completeness score) were recorded and compared between methods.

Results

All participants completed their tasks. The usability could be considered almost excellent with an average SUS score of 82.5/100. The time to complete the assessment was comparable between methods (P = 0.38) as well as the average vigiGrade Completeness of the data collected with the two methods (P = 0.49).

Conclusions

The results showed a good general level of usability for the timeline prototype. Conversely, no difference in terms of the time spent on each ADR case and data quality was found compared with the usual method. However, this absence of difference between the timeline and the usual tools that have been in use for several years suggests a potential use in pharmacovigilance especially because the testers asked to continue using the timeline after the evaluation.

Electronic supplementary material

The online version of this article (10.1186/s12911-018-0667-x) contains supplementary material, which is available to authorized users.

Longitudinal Patterns in Clinical and Imaging Measurements Predict Residual Survival in Glioblastoma Patients

The growing amount of longitudinal data for a large population of patients has necessitated the application of algorithms that can discover patterns to inform patient management. This study demonstrates how temporal patterns generated from a combination of clinical and imaging measurements improve residual survival prediction in glioblastoma patients. Temporal patterns were identified with sequential pattern mining using data from 304 patients. Along with patient covariates, the patterns were incorporated as features in logistic regression models to predict 2-, 6-, or 9-month residual survival at each visit. The modeling approach that included temporal patterns achieved test performances of 0.820, 0.785, and 0.783 area under the receiver operating characteristic curve for predicting 2-, 6-, and 9-month residual survival, respectively. This approach significantly outperformed models that used tumor volume alone (p < 0.001) or tumor volume combined with patient covariates (p < 0.001) in training. Temporal patterns involving an increase in tumor volume above 122 mm³/day, a decrease in KPS across multiple visits, moderate neurologic symptoms, and worsening overall neurologic function suggested lower residual survival. These patterns are readily interpretable and found to be consistent with known prognostic indicators, suggesting they can provide early indicators to clinicians of changes in patient state and inform management decisions.

Clinical Data Analytics With Time-Related Graphical User Interfaces: Application to Pharmacovigilance

Pharmacovigilance consists in monitoring and preventing the occurrence of adverse drug reactions. This activity can be time-consuming because it requires the collection of both patient and medication information. In this paper, we present two visualization and data mining applications to make this task easier for the practitioner. These tools have been developed and tested using the biomedical data warehouse eHOP (Hospital Biomedical Data Warehouse) of the Rennes University Hospital Centre. The first application is a tool to visualize the patient electronic health record in the form of a timeline. All patient data is collected and displayed chronologically. The usability test of the timeline has been very positive (SUS score: 82.5) and the tool is now available for practitioners in their daily practice. The second application is a tool to visualize and search the sequences of a patient cohort. The visual interface allow user to quickly visualize sequences. A query builder allows user to search for sequences in relation with a reference sequence, such as a prescription sequence followed by an abnormal biological value. The sequences are then visually aligned with this reference sequence and ranked by similarity. The GSP (Generalized Sequential Pattern) and Apriori algorithms allow us to display a summary of the sequences list by searching for common sequences and associations. The tool was tested on a use case which consisted in detection of inappropriate drug administration. Compared to a random order, we showed this ranking system saved the practitioner time in this task (to analyze one sequence, 3.49 ± 3.54 vs. 2.26 ± 2.86 s, p = 0.0003). These two visualization and data mining applications will help the daily practice of pharmacovigilance.

MAQUI: Interweaving Queries and Pattern Mining for Recursive Event Sequence Exploration

Exploring event sequences by defining queries alone or by using mining algorithms alone is often not sufficient to support analysis. Analysts often interweave querying and mining in a recursive manner during event sequence analysis: sequences extracted as query results are used for mining patterns, patterns generated are incorporated into a new query for segmenting the sequences, and the resulting segments are mined or queried again. To support flexible analysis, we propose a framework that describes the process of interwoven querying and mining. Based on this framework, we developed MAQUI, a Mining And Querying User Interface that enables recursive event sequence exploration. To understand the efficacy of MAQUI, we conducted two case studies with domain experts. The findings suggest that the capability of interweaving querying and mining helps the participants articulate their questions and gain novel insights from their data.

Visual Progression Analysis of Event Sequence Data

Event sequence data is common to a broad range of application domains, from security to health care to scholarly communication. This form of data captures information about the progression of events for an individual entity (e.g., a computer network device; a patient; an author) in the form of a series of time-stamped observations. Moreover, each event is associated with an event type (e.g., a computer login attempt, or a hospital discharge). Analyses of event sequence data have been shown to help reveal important temporal patterns, such as clinical paths resulting in improved outcomes, or an understanding of common career trajectories for scholars. Moreover, recent research has demonstrated a variety of techniques designed to overcome methodological challenges such as large volumes of data and high dimensionality. However, the effective identification and analysis of latent stages of progression, which can allow for variation within different but similarly evolving event sequences, remain a significant challenge with important real-world motivations. In this paper, we propose an unsupervised stage analysis algorithm to identify semantically meaningful progression stages as well as the critical events which help define those stages. The algorithm follows three key steps: (1) event representation estimation, (2) event sequence warping and alignment, and (3) sequence segmentation. We also present a novel visualization system, ET2, which interactively illustrates the results of the stage analysis algorithm to help reveal evolution patterns across stages. Finally, we report three forms of evaluation for ET2: (1) case studies with two real-world datasets, (2) interviews with domain expert users, and (3) a performance evaluation on the progression analysis algorithm and the visualization design.

Correlation Clustering of Stable Angina Clinical Care Patterns for 506 Thousand Patients

Objectives

Our goal was to apply statistical and network science techniques to depict how the clinical pathways of patients can be used to characterize the practices of care providers.

Methods

We included the data of 506,087 patients who underwent procedures related to ischemic heart disease. Patients were assigned to one of the 136 primary health-care centers using a voting scheme based on their residence. The clinical pathways were classified, and the spectrum of the pathway types was computed for each center, then a network was built with the centers as nodes and spectrum correlations as edge weights. Then Louvain clustering was used to group centers with similar pathway spectra.

Results

We identified 3 clusters with rather distinct characteristics that occupy quite compact spatial areas, though no geographical information was used in clustering. Network analysis and hierarchical clustering show the dominance of medical university clinics in each cluster.

Conclusion

Though clinical guidelines provide a uniform regulation for medical decisions, doctors have great freedom in daily clinical practice. This freedom leads to regional preferences of certain clinical pathways, the intercenter professional links, and geographical locality and coupled with quantifiable consequences in terms of care costs and periprocedural risk of patients.

Daehr

Electronic health records (EHR) provide a rich source of temporal data that present a unique opportunity to characterize disease patterns and risk of imminent disease. While many data-mining tools have been adopted for EHR-based disease early detection, linear discriminant analysis (LDA) is one of the most commonly used statistical methods. However, it is difficult to train an accurate LDA model for early disease diagnosis when too few patients are known to have the target disease. Furthermore, EHR data are heterogeneous with significant noise. In such cases, the covariance matrices used in LDA are usually singular and estimated with a large variance.

This article presents Daehr , an extension of the LDA framework using electronic health record data to address these issues. Beyond existing LDA analyzers, we propose Daehr to (1) eliminate the data noise caused by the manual encoding of EHR data and (2) lower the variance of parameter (covariance matrices) estimation for LDA models when only a few patients’ EHR are available for training. To achieve these two goals, we designed an iterative algorithm to improve the covariance matrix estimation with embedded data-noise/parameter-variance reduction for LDA. We evaluated Daehr extensively using the College Health Surveillance Network, a large, real-world EHR dataset. Specifically, our experiments compared the performance of LDA to three baselines (i.e., LDA and its derivatives) in identifying college students at high risk for mental health disorders from 23 U.S. universities. Experimental results demonstrate Daehr significantly outperforms the three baselines by achieving 1.4%--19.4% higher accuracy and a 7.5%--43.5% higher F1-score.

Leveraging electronic health records for predictive modeling of post-surgical complications

Hospital-specific electronic health record systems are used to inform clinical practice about best practices and quality improvements. Many surgical centers have developed deterministic clinical decision rules to discover adverse events (e.g. postoperative complications) using electronic health record data. However, these data provide opportunities to use probabilistic methods for early prediction of adverse health events, which may be more informative than deterministic algorithms. Electronic health record data from a set of 9598 colorectal surgery cases from 2010 to 2014 were used to predict the occurrence of selected complications including surgical site infection, ileus, and bleeding. Consistent with previous studies, we find a high rate of missing values for both covariates and complication information (4-90%). Several machine learning classification methods are trained on an 80% random sample of cases and tested on a remaining holdout set. Predictive performance varies by complication, although an area under the receiver operating characteristic curve as high as 0.86 on testing data was achieved for bleeding complications, and accuracy for all complications compares favorably to existing clinical decision rules. Our results confirm that electronic health records provide opportunities for improved risk prediction of surgical complications; however, consideration of data quality and consistency standards is an important step in predictive modeling with such data.

Temporal electronic phenotyping by mining careflows of breast cancer patients

In this work we present a careflow mining approach designed to analyze heterogeneous longitudinal data and to identify phenotypes in a patient cohort. The main idea underlying our approach is to combine methods derived from sequential pattern mining and temporal data mining to derive frequent healthcare histories (careflows) in a population of patients. This approach was applied to an integrated data repository containing clinical and administrative data of more than 4000 breast cancer patients. We used the mined histories to identify sub-cohorts of patients grouped according to healthcare activities pathways, then we characterized these sub-cohorts with clinical data. In this way, we were able to perform temporal electronic phenotyping of electronic health records (EHR) data.

Hypotension Risk Prediction via Sequential Contrast Patterns of ICU Blood Pressure

Acute hypotension is a significant risk factor for in-hospital mortality at intensive care units. Prolonged hypotension can cause tissue hypoperfusion, leading to cellular dysfunction and severe injuries to multiple organs. Prompt medical interventions are thus extremely important for dealing with acute hypotensive episodes (AHE). Population level prognostic scoring systems for risk stratification of patients are suboptimal in such scenarios. However, the design of an efficient risk prediction system can significantly help in the identification of critical care patients, who are at risk of developing an AHE within a future time span. Toward this objective, a pattern mining algorithm is employed to extract informative sequential contrast patterns from hemodynamic data, for the prediction of hypotensive episodes. The hypotensive and normotensive patient groups are extracted from the MIMIC-II critical care research database, following an appropriate clinical inclusion criteria. The proposed method consists of a data preprocessing step to convert the blood pressure time series into symbolic sequences, using a symbolic aggregate approximation algorithm. Then, distinguishing subsequences are identified using the sequential contrast mining algorithm. These subsequences are used to predict the occurrence of an AHE in a future time window separated by a user-defined gap interval. Results indicate that the method performs well in terms of the prediction performance as well as in the generation of sequential patterns of clinical significance. Hence, the novelty of sequential patterns is in their usefulness as potential physiological biomarkers for building optimal patient risk stratification systems and for further clinical investigation of interesting patterns in critical care patients.

Knowledge Discovery from Biomedical Ontologies in Cross Domains

In recent years, there is an increasing demand for sharing and integration of medical data in biomedical research. In order to improve a health care system, it is required to support the integration of data by facilitating semantic interoperability systems and practices. Semantic interoperability is difficult to achieve in these systems as the conceptual models underlying datasets are not fully exploited. In this paper, we propose a semantic framework, called Medical Knowledge Discovery and Data Mining (MedKDD), that aims to build a topic hierarchy and serve the semantic interoperability between different ontologies. For the purpose, we fully focus on the discovery of semantic patterns about the association of relations in the heterogeneous information network representing different types of objects and relationships in multiple biological ontologies and the creation of a topic hierarchy through the analysis of the discovered patterns. These patterns are used to cluster heterogeneous information networks into a set of smaller topic graphs in a hierarchical manner and then to conduct cross domain knowledge discovery from the multiple biological ontologies. Thus, patterns made a greater contribution in the knowledge discovery across multiple ontologies. We have demonstrated the cross domain knowledge discovery in the MedKDD framework using a case study with 9 primary biological ontologies from Bio2RDF and compared it with the cross domain query processing approach, namely SLAP. We have confirmed the effectiveness of the MedKDD framework in knowledge discovery from multiple medical ontologies.

Progress in Biomedical Knowledge Discovery: A 25-year Retrospective

OBJECTIVES

We sought to explore, via a systematic review of the literature, the state of the art of knowledge discovery in biomedical databases as it existed in 1992, and then now, 25 years later, mainly focused on supervised learning.

METHODS

We performed a rigorous systematic search of PubMed and latent Dirichlet allocation to identify themes in the literature and trends in the science of knowledge discovery in and between time periods and compare these trends. We restricted the result set using a bracket of five years previous, such that the 1992 result set was restricted to articles published between 1987 and 1992, and the 2015 set between 2011 and 2015. This was to reflect the current literature available at the time to researchers and others at the target dates of 1992 and 2015. The search term was framed as: Knowledge Discovery OR Data Mining OR Pattern Discovery OR Pattern Recognition, Automated.

RESULTS

A total 538 and 18,172 documents were retrieved for 1992 and 2015, respectively. The number and type of data sources increased dramatically over the observation period, primarily due to the advent of electronic clinical systems. The period 1992- 2015 saw the emergence of new areas of research in knowledge discovery, and the refinement and application of machine learning approaches that were nascent or unknown in 1992.

CONCLUSIONS

Over the 25 years of the observation period, we identified numerous developments that impacted the science of knowledge discovery, including the availability of new forms of data, new machine learning algorithms, and new application domains. Through a bibliometric analysis we examine the striking changes in the availability of highly heterogeneous data resources, the evolution of new algorithmic approaches to knowledge discovery, and we consider from legal, social, and political perspectives possible explanations of the growth of the field. Finally, we reflect on the achievements of the past 25 years to consider what the next 25 years will bring with regard to the availability of even more complex data and to the methods that could be, and are being now developed for the discovery of new knowledge in biomedical data.

Feasibility of Population Health Analytics and Data Visualization for Decision Support in the Infectious Diseases Domain: A pilot study

OBJECTIVE

Big data or population-based information has the potential to reduce uncertainty in medicine by informing clinicians about individual patient care. The objectives of this study were: 1) to explore the feasibility of extracting and displaying population-based information from an actual clinical population's database records, 2) to explore specific design features for improving population display, 3) to explore perceptions of population information displays, and 4) to explore the impact of population information display on cognitive outcomes.

METHODS

We used the Veteran's Affairs (VA) database to identify similar complex patients based on a similar complex patient case. Study outcomes measures were 1) preferences for population information display 2) time looking at the population display, 3) time to read the chart, and 4) appropriateness of plans with pre- and post-presentation of population data. Finally, we redesigned the population information display based on our findings from this study.

RESULTS

The qualitative data analysis for preferences of population information display resulted in four themes: 1) trusting the big/population data can be an issue, 2) embedded analytics is necessary to explore patient similarities, 3) need for tools to control the view (overview, zoom and filter), and 4) different presentations of the population display can be beneficial to improve the display. We found that appropriateness of plans was at 60% for both groups (t9=-1.9; p=0.08), and overall time looking at the population information display was 2.3 minutes versus 3.6 minutes with experts processing information faster than non-experts (t8= -2.3, p=0.04).

CONCLUSION

A population database has great potential for reducing complexity and uncertainty in medicine to improve clinical care. The preferences identified for the population information display will guide future health information technology system designers for better and more intuitive display.

Visual Analysis of Cloud Computing Performance Using Behavioral Lines

Cloud computing is an essential technology to Big Data analytics and services. A cloud computing system is often comprised of a large number of parallel computing and storage devices. Monitoring the usage and performance of such a system is important for efficient operations, maintenance, and security. Tracing every application on a large cloud system is untenable due to scale and privacy issues. But profile data can be collected relatively efficiently by regularly sampling the state of the system, including properties such as CPU load, memory usage, network usage, and others, creating a set of multivariate time series for each system. Adequate tools for studying such large-scale, multidimensional data are lacking. In this paper, we present a visual based analysis approach to understanding and analyzing the performance and behavior of cloud computing systems. Our design is based on similarity measures and a layout method to portray the behavior of each compute node over time. When visualizing a large number of behavioral lines together, distinct patterns often appear suggesting particular types of performance bottleneck. The resulting system provides multiple linked views, which allow the user to interactively explore the data by examining the data or a selected subset at different levels of detail. Our case studies, which use datasets collected from two different cloud systems, show that this visual based approach is effective in identifying trends and anomalies of the systems.

Predicate Oriented Pattern Analysis for Biomedical Knowledge Discovery

In the current biomedical data movement, numerous efforts have been made to convert and normalize a large number of traditional structured and unstructured data (e.g., EHRs, reports) to semi-structured data (e.g., RDF, OWL). With the increasing number of semi-structured data coming into the biomedical community, data integration and knowledge discovery from heterogeneous domains become important research problem. In the application level, detection of related concepts among medical ontologies is an important goal of life science research. It is more crucial to figure out how different concepts are related within a single ontology or across multiple ontologies by analysing predicates in different knowledge bases. However, the world today is one of information explosion, and it is extremely difficult for biomedical researchers to find existing or potential predicates to perform linking among cross domain concepts without any support from schema pattern analysis. Therefore, there is a need for a mechanism to do predicate oriented pattern analysis to partition heterogeneous ontologies into closer small topics and do query generation to discover cross domain knowledge from each topic. In this paper, we present such a model that predicates oriented pattern analysis based on their close relationship and generates a similarity matrix. Based on this similarity matrix, we apply an innovated unsupervised learning algorithm to partition large data sets into smaller and closer topics and generate meaningful queries to fully discover knowledge over a set of interlinked data sources. We have implemented a prototype system named BmQGen and evaluate the proposed model with colorectal surgical cohort from the Mayo Clinic.

Using network projections to explore co-incidence and context in large clinical datasets: Application to homelessness among U.S. Veterans

INTRODUCTION

Network projections of data can provide an efficient format for data exploration of co-incidence in large clinical datasets. We present and explore the utility of a network projection approach to finding patterns in health care data that could be exploited to prevent homelessness among U.S. Veterans.

METHOD

We divided Veteran ICD-9-CM (ICD9) data into two time periods (0-59 and 60-364days prior to the first evidence of homelessness) and then used Pajek social network analysis software to visualize these data as three different networks. A multi-relational network simultaneously displayed the magnitude of ties between the most frequent ICD9 pairings. A new association network visualized ICD9 pairings that greatly increased or decreased. A signed, subtraction network visualized the presence, absence, and magnitude difference between ICD9 associations by time period.

RESULT

A cohort of 9468 U.S. Veterans was identified as having administrative evidence of homelessness and visits in both time periods. They were seen in 222,599 outpatient visits that generated 484,339 ICD9 codes (average of 11.4 (range 1-23) visits and 2.2 (range 1-60) ICD9 codes per visit). Using the three network projection methods, we were able to show distinct differences in the pattern of co-morbidities in the two time periods. In the more distant time period preceding homelessness, the network was dominated by routine health maintenance visits and physical ailment diagnoses. In the 59days immediately prior to the homelessness identification, alcohol related diagnoses along with economic circumstances such as unemployment, legal circumstances, along with housing instability were noted.

CONCLUSION

Network visualizations of large clinical datasets traditionally treated as tabular and difficult to manipulate reveal rich, previously hidden connections between data variables related to homelessness. A key feature is the ability to visualize changes in variables with temporality and in proximity to the event of interest. These visualizations lend support to cognitive tasks such as exploration of large clinical datasets as a prelude to hypothesis generation.

Coronary artery disease risk assessment from unstructured electronic health records using text mining

Coronary artery disease (CAD) often leads to myocardial infarction, which may be fatal. Risk factors can be used to predict CAD, which may subsequently lead to prevention or early intervention. Patient data such as co-morbidities, medication history, social history and family history are required to determine the risk factors for a disease. However, risk factor data are usually embedded in unstructured clinical narratives if the data is not collected specifically for risk assessment purposes. Clinical text mining can be used to extract data related to risk factors from unstructured clinical notes. This study presents methods to extract Framingham risk factors from unstructured electronic health records using clinical text mining and to calculate 10-year coronary artery disease risk scores in a cohort of diabetic patients. We developed a rule-based system to extract risk factors: age, gender, total cholesterol, HDL-C, blood pressure, diabetes history and smoking history. The results showed that the output from the text mining system was reliable, but there was a significant amount of missing data to calculate the Framingham risk score. A systematic approach for understanding missing data was followed by implementation of imputation strategies. An analysis of the 10-year Framingham risk scores for coronary artery disease in this cohort has shown that the majority of the diabetic patients are at moderate risk of CAD.

An electronic medical record system with treatment recommendations based on patient similarity

As the core of health information technology (HIT), electronic medical record (EMR) systems have been changing to meet health care demands. To construct a new-generation EMR system framework with the capability of self-learning and real-time feedback, thus adding intelligence to the EMR system itself, this paper proposed a novel EMR system framework by constructing a direct pathway between the EMR workflow and EMR data. A prototype of this framework was implemented based on patient similarity learning. Patient diagnoses, demographic data, vital signs and structured lab test results were considered for similarity calculations. Real hospitalization data from 12,818 patients were substituted, and Precision @ Position measurements were used to validate self-learning performance. Our EMR system changed the way in which orders are placed by establishing recommendation order menu and shortcut applications. Two learning modes (EASY MODE and COMPLEX MODE) were provided, and the precision values @ position 5 of both modes were 0.7458 and 0.8792, respectively. The precision performance of COMPLEX MODE was better than that of EASY MODE (tested using a paired Wilcoxon-Mann-Whitney test, p < 0.001). Applying the proposed framework, the EMR data value was directly demonstrated in the clinical workflow, and intelligence was added to the EMR system, which could improve system usability, reliability and the physician's work efficiency. This self-learning mechanism is based on dynamic learning models and is not limited to a specific disease or clinical scenario, thus decreasing maintenance costs in real world applications and increasing its adaptability.

Visual aggregate analysis of eligibility features of clinical trials

OBJECTIVE

To develop a method for profiling the collective populations targeted for recruitment by multiple clinical studies addressing the same medical condition using one eligibility feature each time.

METHODS

Using a previously published database COMPACT as the backend, we designed a scalable method for visual aggregate analysis of clinical trial eligibility features. This method consists of four modules for eligibility feature frequency analysis, query builder, distribution analysis, and visualization, respectively. This method is capable of analyzing (1) frequently used qualitative and quantitative features for recruiting subjects for a selected medical condition, (2) distribution of study enrollment on consecutive value points or value intervals of each quantitative feature, and (3) distribution of studies on the boundary values, permissible value ranges, and value range widths of each feature. All analysis results were visualized using Google Charts API. Five recruited potential users assessed the usefulness of this method for identifying common patterns in any selected eligibility feature for clinical trial participant selection.

RESULTS

We implemented this method as a Web-based analytical system called VITTA (Visual Analysis Tool of Clinical Study Target Populations). We illustrated the functionality of VITTA using two sample queries involving quantitative features BMI and HbA1c for conditions "hypertension" and "Type 2 diabetes", respectively. The recruited potential users rated the user-perceived usefulness of VITTA with an average score of 86.4/100.

CONCLUSIONS

We contributed a novel aggregate analysis method to enable the interrogation of common patterns in quantitative eligibility criteria and the collective target populations of multiple related clinical studies. A larger-scale study is warranted to formally assess the usefulness of VITTA among clinical investigators and sponsors in various therapeutic areas.

DecisionFlow: Visual Analytics for High-Dimensional Temporal Event Sequence Data

Temporal event sequence data is increasingly commonplace, with applications ranging from electronic medical records to financial transactions to social media activity. Previously developed techniques have focused on low-dimensional datasets (e.g., with less than 20 distinct event types). Real-world datasets are often far more complex. This paper describes DecisionFlow, a visual analysis technique designed to support the analysis of high-dimensional temporal event sequence data (e.g., thousands of event types). DecisionFlow combines a scalable and dynamic temporal event data structure with interactive multi-view visualizations and ad hoc statistical analytics. We provide a detailed review of our methods, and present the results from a 12-person user study. The study results demonstrate that DecisionFlow enables the quick and accurate completion of a range of sequence analysis tasks for datasets containing thousands of event types and millions of individual events.