Using Dashboard Networks to Visualize Multiple Patient Histories: A Design Study on Post-Operative Prostate Cancer

MS Pattern Explorer: interactive visual exploration of temporal activity patterns for multiple sclerosis

OBJECTIVES

This article describes the design and evaluation of MS Pattern Explorer, a novel visual tool that uses interactive machine learning to analyze fitness wearables' data. Applied to a clinical study of multiple sclerosis (MS) patients, the tool addresses key challenges: managing activity signals, accelerating insight generation, and rapidly contextualizing identified patterns. By analyzing sensor measurements, it aims to enhance understanding of MS symptomatology and improve the broader problem of clinical exploratory sensor data analysis.

MATERIALS AND METHODS

Following a user-centered design approach, we learned that clinicians have 3 priorities for generating insights for the Barka-MS study data: exploration and search for, and contextualization of, sequences and patterns in patient sleep and activity. We compute meaningful sequences for patients using clustering and proximity search, displaying these with an interactive visual interface composed of coordinated views. Our evaluation posed both closed and open-ended tasks to participants, utilizing a scoring system to gauge the tool's usability, and effectiveness in supporting insight generation across 15 clinicians, data scientists, and non-experts.

RESULTS AND DISCUSSION

We present MS Pattern Explorer, a visual analytics system that helps clinicians better address complex data-centric challenges by facilitating the understanding of activity patterns. It enables innovative analysis that leads to rapid insight generation and contextualization of temporal activity data, both within and between patients of a cohort. Our evaluation results indicate consistent performance across participant groups and effective support for insight generation in MS patient fitness tracker data. Our implementation offers broad applicability in clinical research, allowing for potential expansion into cohort-wide comparisons or studies of other chronic conditions.

CONCLUSION

MS Pattern Explorer successfully reduces the signal overload clinicians currently experience with activity data, introducing novel opportunities for data exploration, sense-making, and hypothesis generation.

HD, Salgado CG, da Silva MB, Barreto JG, da Costa PF, dos Santos AM, Conde GAB, Seruffo MCDR. A study about management of drugs for leprosy patients under medical monitoring: A solution based on AHP-Electre decision-making methods

Leprosy, also known as Hansen's, is one of the listed neglected tropical diseases as a major health problem global. Treatment is one of the main alternatives, however, the scarcity of medication and its poor distribution are important factors that have driven the spread of the disease, leading to irreversible and multi-resistant complications. This paper uses a distribution methodology to optimize medication administration, taking into account the most relevant attributes for the epidemiological profile of patients and the deficit in treatment via Polychemotherapy. Multi-criteria Decision Methods were applied based on AHP-Electre model in a database with information from patients in the state of Para between 2015 and 2020. The results pointed out that 84% of individuals did not receive any treatment and, among these, the method obtained a gain in the distribution of 68% in patients with positive diagnosis for leprosy.

Digital dashboards visualizing public health data: a systematic review

Introduction

Public health is not only threatened by diseases, pandemics, or epidemics. It is also challenged by deficits in the communication of health information. The current COVID-19 pandemic demonstrates that impressively. One way to deliver scientific data such as epidemiological findings and forecasts on disease spread are dashboards. Considering the current relevance of dashboards for public risk and crisis communication, this systematic review examines the state of research on dashboards in the context of public health risks and diseases.

Method

Nine electronic databases where searched for peer-reviewed journal articles and conference proceedings. Included articles (n = 65) were screened and assessed by three independent reviewers. Through a methodological informed differentiation between descriptive studies and user studies, the review also assessed the quality of included user studies (n = 18) by use of the Mixed Methods Appraisal Tool (MMAT).

Results

65 articles were assessed in regards to the public health issues addressed by the respective dashboards, as well as the data sources, functions and information visualizations employed by the different dashboards. Furthermore, the literature review sheds light on public health challenges and objectives and analyzes the extent to which user needs play a role in the development and evaluation of a dashboard. Overall, the literature review shows that studies that do not only describe the construction of a specific dashboard, but also evaluate its content in terms of different risk communication models or constructs (e.g., risk perception or health literacy) are comparatively rare. Furthermore, while some of the studies evaluate usability and corresponding metrics from the perspective of potential users, many of the studies are limited to a purely functionalistic evaluation of the dashboard by the respective development teams.

Conclusion

The results suggest that applied research on public health intervention tools like dashboards would gain in complexity through a theory-based integration of user-specific risk information needs.

Systematic review registration

https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=200178, identifier: CRD42020200178.

Dashboard Design Patterns

This paper introduces design patterns for dashboards to inform dashboard design processes. Despite a growing number of public examples, case studies, and general guidelines there is surprisingly little design guidance for dashboards. Such guidance is necessary to inspire designs and discuss tradeoffs in, e.g., screenspace, interaction, or information shown. Based on a systematic review of 144 dashboards, we report on eight groups of design patterns that provide common solutions in dashboard design. We discuss combinations of these patterns in "dashboard genres" such as narrative, analytical, or embedded dashboard. We ran a 2-week dashboard design workshop with 23 participants of varying expertise working on their own data and dashboards. We discuss the application of patterns for the dashboard design processes, as well as general design tradeoffs and common challenges. Our work complements previous surveys and aims to support dashboard designers and researchers in co-creation, structured design decisions, as well as future user evaluations about dashboard design guidelines. Detailed pattern descriptions and workshop material can be found online: https://dashboarddesignpatterns.github.io.

The Effect of Alignment on People's Ability to Judge Event Sequence Similarity

Event sequences are central to the analysis of data in domains that range from biology and health, to logfile analysis and people's everyday behavior. Many visualization tools have been created for such data, but people are error-prone when asked to judge the similarity of event sequences with basic presentation methods. This article describes an experiment that investigates whether local and global alignment techniques improve people's performance when judging sequence similarity. Participants were divided into three groups (basic versus local versus global alignment), and each participant judged the similarity of 180 sets of pseudo-randomly generated sequences. Each set comprised a target, a correct choice and a wrong choice. After training, the global alignment group was more accurate than the local alignment group (98 versus 93 percent correct), with the basic group getting 95 percent correct. Participants' response times were primarily affected by the number of event types, the similarity of sequences (measured by the Levenshtein distance) and the edit types (nine combinations of deletion, insertion and substitution). In summary, global alignment is superior and people's performance could be further improved by choosing alignment parameters that explicitly penalize sequence mismatches.

Real-time analysis and display of quantitative measures to track and improve clinical workflow

Purpose

Radiotherapy treatment planning is a complex process with multiple, dependent steps involving an interdisciplinary patient care team. Effective communication and real‐time tracking of resources and care path activities are key for clinical efficiency and patient safety.

Materials and Methods

We designed and implemented a secure, interactive web‐based dashboard for patient care path, clinical workflow, and resource utilization management. The dashboard enables visualization of resource utilization and tracks progress in a patient's care path from the time of acquisition of the planning CT to the time of treatment in real‐time. It integrates with the departmental electronic medical records (EMR) system without the creation and maintenance of a separate database or duplication of work by clinical staff. Performance measures of workflow were calculated.

Results

The dashboard implements a standardized clinical workflow and dynamically consolidates real‐time information queried from multiple tables in the EMR database over the following views: (1) CT Sims summarizes patient appointment information on the CT simulator and patient load; (2) Linac Sims summarizes patient appointment times, setup history, and notes, and patient load; (3) Task Status lists the clinical tasks associated with a treatment plan, their due date, status and ownership, and patient appointment details; (4) Documents provides the status of all documents in the patients' charts; and (5) Diagnoses and Interventions summarizes prescription information, imaging instructions and whether the plan was approved for treatment. Real‐time assessment and quantification of progress and delays in a patient's treatment start were achieved.

Conclusions

This study indicates it is feasible to develop and implement a dashboard, tailored to the needs of an interdisciplinary team, which derives and integrates information from the EMR database for real‐time analysis and display of resource utilization and clinical workflow in radiation oncology. The framework developed facilitates informed, data‐driven decisions on clinical workflow management as we seek to optimize clinical efficiency and patient safety.

A Framework for Evaluating Dashboards in Healthcare

In the era of 'information overload', effective information provision is essential for enabling rapid response and critical decision making. In making sense of diverse information sources, dashboards have become an indispensable tool, providing fast, effective, adaptable, and personalized access to information for professionals and the general public alike. However, these objectives place heavy requirements on dashboards as information systems in usability and effective design. Understanding these issues is challenging given the absence of consistent and comprehensive approaches to dashboard evaluation. In this article we systematically review literature on dashboard implementation in healthcare, where dashboards have been employed widely, and where there is widespread interest for improving the current state of the art, and subsequently analyse approaches taken towards evaluation. We draw upon consolidated dashboard literature and our own observations to introduce a general definition of dashboards which is more relevant to current trends, together with seven evaluation scenarios - task performance, behaviour change, interaction workflow, perceived engagement, potential utility, algorithm performance and system implementation. These scenarios distinguish different evaluation purposes which we illustrate through measurements, example studies, and common challenges in evaluation study design. We provide a breakdown of each evaluation scenario, and highlight some of the more subtle questions. We demonstrate the use of the proposed framework by a design study guided by this framework. We conclude by comparing this framework with existing literature, outlining a number of active discussion points and a set of dashboard evaluation best practices for the academic, clinical and software development communities alike.

The DynAIRx Project Protocol: Artificial Intelligence for dynamic prescribing optimisation and care integration in multimorbidity

Background

Structured Medication Reviews (SMRs) are intended to help deliver the NHS Long Term Plan for medicines optimisation in people living with multiple long-term conditions and polypharmacy. It is challenging to gather the information needed for these reviews due to poor integration of health records across providers and there is little guidance on how to identify those patients most urgently requiring review.

Objective

To extract information from scattered clinical records on how health and medications change over time, apply interpretable artificial intelligence (AI) approaches to predict risks of poor outcomes and overlay this information on care records to inform SMRs. We will pilot this approach in primary care prescribing audit and feedback systems, and co-design future medicines optimisation decision support systems.

Design

DynAIRx will target potentially problematic polypharmacy in three key multimorbidity groups, namely, people with (a) mental and physical health problems, (b) four or more long-term conditions taking ten or more drugs and (c) older age and frailty. Structured clinical data will be drawn from integrated care records (general practice, hospital, and social care) covering an ∼11m population supplemented with Natural Language Processing (NLP) of unstructured clinical text. AI systems will be trained to identify patterns of conditions, medications, tests, and clinical contacts preceding adverse events in order to identify individuals who might benefit most from an SMR.

Discussion

By implementing and evaluating an AI-augmented visualisation of care records in an existing prescribing audit and feedback system we will create a learning system for medicines optimisation, co-designed throughout with end-users and patients.

IRVINE: A Design Study on Analyzing Correlation Patterns of Electrical Engines

In this design study, we present IRVINE, a Visual Analytics (VA) system, which facilitates the analysis of acoustic data to detect and understand previously unknown errors in the manufacturing of electrical engines. In serial manufacturing processes, signatures from acoustic data provide valuable information on how the relationship between multiple produced engines serves to detect and understand previously unknown errors. To analyze such signatures, IRVINE leverages interactive clustering and data labeling techniques, allowing users to analyze clusters of engines with similar signatures, drill down to groups of engines, and select an engine of interest. Furthermore, IRVINE allows to assign labels to engines and clusters and annotate the cause of an error in the acoustic raw measurement of an engine. Since labels and annotations represent valuable knowledge, they are conserved in a knowledge database to be available for other stakeholders. We contribute a design study, where we developed IRVINE in four main iterations with engineers from a company in the automotive sector. To validate IRVINE, we conducted a field study with six domain experts. Our results suggest a high usability and usefulness of IRVINE as part of the improvement of a real-world manufacturing process. Specifically, with IRVINE domain experts were able to label and annotate produced electrical engines more than 30% faster.

matExplorer: Visual Exploration on Predicting Ionic Conductivity for Solid-state Electrolytes

Lithium ion batteries (LIBs) are widely used as important energy sources for mobile phones, electric vehicles, and drones. Experts have attempted to replace liquid electrolytes with solid electrolytes that have wider electrochemical window and higher stability due to the potential safety risks, such as electrolyte leakage, flammable solvents, poor thermal stability, and many side reactions caused by liquid electrolytes. However, finding suitable alternative materials using traditional approaches is very difficult due to the incredibly high cost in searching. Machine learning (ML)-based methods are currently introduced and used for material prediction. However, learning tools designed for domain experts to conduct intuitive performance comparison and analysis of ML models are rare. In this case, we propose an interactive visualization system for experts to select suitable ML models and understand and explore the predication results comprehensively. Our system uses a multifaceted visualization scheme designed to support analysis from various perspectives, such as feature distribution, data similarity, model performance, and result presentation. Case studies with actual lab experiments have been conducted by the experts, and the final results confirmed the effectiveness and helpfulness of our system.

VBridge: Connecting the Dots Between Features and Data to Explain Healthcare Models

Machine learning (ML) is increasingly applied to Electronic Health Records (EHRs) to solve clinical prediction tasks. Although many ML models perform promisingly, issues with model transparency and interpretability limit their adoption in clinical practice. Directly using existing explainable ML techniques in clinical settings can be challenging. Through literature surveys and collaborations with six clinicians with an average of 17 years of clinical experience, we identified three key challenges, including clinicians' unfamiliarity with ML features, lack of contextual information, and the need for cohort-level evidence. Following an iterative design process, we further designed and developed VBridge, a visual analytics tool that seamlessly incorporates ML explanations into clinicians' decision-making workflow. The system includes a novel hierarchical display of contribution-based feature explanations and enriched interactions that connect the dots between ML features, explanations, and data. We demonstrated the effectiveness of VBridge through two case studies and expert interviews with four clinicians, showing that visually associating model explanations with patients' situational records can help clinicians better interpret and use model predictions when making clinician decisions. We further derived a list of design implications for developing future explainable ML tools to support clinical decision-making.

DPVis: Visual Analytics With Hidden Markov Models for Disease Progression Pathways

Clinical researchers use disease progression models to understand patient status and characterize progression patterns from longitudinal health records. One approach for disease progression modeling is to describe patient status using a small number of states that represent distinctive distributions over a set of observed measures. Hidden Markov models (HMMs) and its variants are a class of models that both discover these states and make inferences of health states for patients. Despite the advantages of using the algorithms for discovering interesting patterns, it still remains challenging for medical experts to interpret model outputs, understand complex modeling parameters, and clinically make sense of the patterns. To tackle these problems, we conducted a design study with clinical scientists, statisticians, and visualization experts, with the goal to investigate disease progression pathways of chronic diseases, namely type 1 diabetes (T1D), Huntington's disease, Parkinson's disease, and chronic obstructive pulmonary disease (COPD). As a result, we introduce DPVis which seamlessly integrates model parameters and outcomes of HMMs into interpretable and interactive visualizations. In this article, we demonstrate that DPVis is successful in evaluating disease progression models, visually summarizing disease states, interactively exploring disease progression patterns, and building, analyzing, and comparing clinically relevant patient subgroups.

A data science approach to drug safety: Semantic and visual mining of adverse drug events from clinical trials of pain treatments

Clinical trials are the basis of Evidence-Based Medicine. Trial results are reviewed by experts and consensus panels for producing meta-analyses and clinical practice guidelines. However, reviewing these results is a long and tedious task, hence the meta-analyses and guidelines are not updated each time a new trial is published. Moreover, the independence of experts may be difficult to appraise. On the contrary, in many other domains, including medical risk analysis, the advent of data science, big data and visual analytics allowed moving from expert-based to fact-based knowledge. Since 12 years, many trial results are publicly available online in trial registries. Nevertheless, data science methods have not yet been applied widely to trial data. In this paper, we present a platform for analyzing the safety events reported during clinical trials and published in trial registries. This platform is based on an ontological model including 582 trials on pain treatments, and uses semantic web technologies for querying this dataset at various levels of granularity. It also relies on a 26-dimensional flower glyph for the visualization of the Adverse Drug Events (ADE) rates in 13 categories and 2 levels of seriousness. We illustrate the interest of this platform through several use cases and we were able to find back conclusions that were initially found during meta-analyses. The platform was presented to four experts in drug safety, and is publicly available online, with the ontology of pain treatment ADE.

A scoping review of clinical decision support tools that generate new knowledge to support decision making in real time

OBJECTIVE

A growing body of observational data enabled its secondary use to facilitate clinical care for complex cases not covered by the existing evidence. We conducted a scoping review to characterize clinical decision support systems (CDSSs) that generate new knowledge to provide guidance for such cases in real time.

MATERIALS AND METHODS

PubMed, Embase, ProQuest, and IEEE Xplore were searched up to May 2020. The abstracts were screened by 2 reviewers. Full texts of the relevant articles were reviewed by the first author and approved by the second reviewer, accompanied by the screening of articles' references. The details of design, implementation and evaluation of included CDSSs were extracted.

RESULTS

Our search returned 3427 articles, 53 of which describing 25 CDSSs were selected. We identified 8 expert-based and 17 data-driven tools. Sixteen (64%) tools were developed in the United States, with the others mostly in Europe. Most of the tools (n = 16, 64%) were implemented in 1 site, with only 5 being actively used in clinical practice. Patient or quality outcomes were assessed for 3 (18%) CDSSs, 4 (16%) underwent user acceptance or usage testing and 7 (28%) functional testing.

CONCLUSIONS

We found a number of CDSSs that generate new knowledge, although only 1 addressed confounding and bias. Overall, the tools lacked demonstration of their utility. Improvement in clinical and quality outcomes were shown only for a few CDSSs, while the benefits of the others remain unclear. This review suggests a need for a further testing of such CDSSs and, if appropriate, their dissemination.

Trends in Clinical Information Systems Research in 2019

Objective : To give an overview of recent research and to propose a selection of best papers published in 2019 in the field of Clinical Information Systems (CIS).

Method : Each year, we apply a systematic process to retrieve articles for the CIS section of the IMIA Yearbook of Medical Informatics. For six years now, we use the same query to find relevant publications in the CIS field. Each year we retrieve more than 2,000 papers. As CIS section editors, we categorize the retrieved articles in a multi-pass review to distill a pre-selection of 15 candidate best papers. Then, Yearbook editors and external reviewers assess the selected candidate best papers. Based on the review results, the IMIA Yearbook Editorial Committee chooses the best papers during the selection meeting. We used text mining, and term co-occurrence mapping techniques to get an overview of the content of the retrieved articles.

Results : We carried out the query in mid-January 2020 and retrieved a de-duplicated result set of 2,407 articles from 1,023 different journals. This year, we nominated 14 papers as candidate best papers, and three of them were finally selected as best papers in the CIS section. As in previous years, the content analysis of the articles revealed the broad spectrum of topics covered by CIS research.

Conclusions : We could observe ongoing trends, as seen in the last years. Patient benefit research is in the focus of many research activities, and trans-institutional aggregation of data remains a relevant field of work. Powerful machine-learning-based approaches, that use readily available data now often outperform human-based procedures. However, the ethical perspective of this development often comes too short in the considerations. We thus assume that ethical aspects will and should deliver much food for thought for future CIS research.

Cancer Informatics in 2019: Deep Learning Takes Center Stage

Objective : To summarize significant research contributions on cancer informatics published in 2019.

Methods : An extensive search using PubMed/Medline and manual review was conducted to identify the scientific contributions published in 2019 that address topics in cancer informatics. The selection process comprised three steps: (i) 15 candidate best papers were first selected by the two section editors, (ii) external reviewers from internationally renowned research teams reviewed each candidate best paper, and (iii) the final selection of two best papers was conducted by the editorial committee of the Yearbook.

Results : The two selected best papers demonstrate the clinical utility of deep learning in two important cancer domains: radiology and pathology.

Conclusion : Cancer informatics is a broad and vigorous subfield of biomedical informatics. Applications of new and emerging computational technologies are especially notable in 2019.