Interactive visual analysis of perfusion data

DASS Good: Explainable Data Mining of Spatial Cohort Data

Developing applicable clinical machine learning models is a difficult task when the data includes spatial information, for example, radiation dose distributions across adjacent organs at risk. We describe the co-design of a modeling system, DASS, to support the hybrid human-machine development and validation of predictive models for estimating long-term toxicities related to radiotherapy doses in head and neck cancer patients. Developed in collaboration with domain experts in oncology and data mining, DASS incorporates human-in-the-loop visual steering, spatial data, and explainable AI to augment domain knowledge with automatic data mining. We demonstrate DASS with the development of two practical clinical stratification models and report feedback from domain experts. Finally, we describe the design lessons learned from this collaborative experience.

Visual Analytics for Hypothesis-Driven Exploration in Computational Pathology

Recent advances in computational and algorithmic power are evolving the field of medical imaging rapidly. In cancer research, many new directions are sought to characterize patients with additional imaging features derived from radiology and pathology images. The emerging field of Computational Pathology targets the high-throughput extraction and analysis of the spatial distribution of cells from digital histopathology images. The associated morphological and architectural features allow researchers to quantify and characterize new imaging biomarkers for cancer diagnosis, prognosis, and treatment decisions. However, while the image feature space grows, exploration and analysis become more difficult and ineffective. There is a need for dedicated interfaces for interactive data manipulation and visual analysis of computational pathology and clinical data. For this purpose, we present IIComPath, a visual analytics approach that enables clinical researchers to formulate hypotheses and create computational pathology pipelines involving cohort construction, spatial analysis of image-derived features, and cohort analysis. We demonstrate our approach through use cases that investigate the prognostic value of current diagnostic features and new computational pathology biomarkers.

Evaluation of Renal Microperfusion in Diabetic Patients With Kidney Injury by Contrast-Enhanced Ultrasound

OBJECTIVES

To conduct a quantitative analysis of renal microvascular perfusion in diabetic patients with kidney injury using contrast-enhanced ultrasound (CEUS).

METHODS

A total of 172 patients with type 2 diabetes mellitus and kidney injury were recruited from May 2017 to November 2019. After collection of clinical characteristics, a CEUS examination was performed after injection of the contrast agent SonoVue (Bracco SpA, Milan, Italy). Time-intensity curves and renal perfusion parameters were analyzed. Ultrasound-guided renal biopsy was performed. The patients were divided into a diabetic nephropathy (DN) group and a nondiabetic renal disease (NDRD) group according to renal pathologic results. The discrimination of perfusion parameters between the groups was analyzed statistically with SPSS version 19.0 software (IBM Corporation, Armonk, NY). Receiver operating characteristic curves were used to illustrate the diagnostic performance of indicators.

RESULTS

Ninety-eight patients, including 45 with DN (29 male; mean age ± SD, 57.76 ± 10.47 years) and 53 with NDRD (40 male; mean age, 48.7 ± 13.88 years) were included in this study. The peak enhancement (PE), wash-in the area under the curve (AUC), wash-in rate, wash-in perfusion index, wash-out AUC, wash-in and wash-out AUC, and wash-out rate were significantly different between the groups (P < .05). There were no differences in time-related parameters between the DN and NDRD groups (P > .05). The receiver operating characteristic curve analysis showed that the AUC for PE was 0.727, and PE lower than 7712.426 had diagnostic potential, with sensitivity of 81% and specificity of 40% in discriminating between NDRD and DN.

CONCLUSIONS

The quantification of CEUS parameters can discriminate DN in diabetic patients with kidney injury. The PE and AUC may be feasible parameters.

Integrated Dual Analysis of Quantitative and Qualitative High-Dimensional Data

The Dual Analysis framework is a powerful enabling technology for the exploration of high dimensional quantitative data by treating data dimensions as first-class objects that can be explored in tandem with data values. In this article, we extend the Dual Analysis framework through the joint treatment of quantitative (numerical) and qualitative (categorical) dimensions. Computing common measures for all dimensions allows us to visualize both quantitative and qualitative dimensions in the same view. This enables a natural joint treatment of mixed data during interactive visual exploration and analysis. Several measures of variation for nominal qualitative data can also be applied to ordinal qualitative and quantitative data. For example, instead of measuring variability from a mean or median, other measures assess inter-data variation or average variation from a mode. In this work, we demonstrate how these measures can be integrated into the Dual Analysis framework to explore and generate hypotheses about high-dimensional mixed data. A medical case study using clinical routine data of patients suffering from Cerebral Small Vessel Disease (CSVD), conducted with a senior neurologist and a medical student, shows that a joint Dual Analysis approach for quantitative and qualitative data can rapidly lead to new insights based on which new hypotheses may be generated.

Principles of Visualization in Radiation Oncology

BACKGROUND

Medical visualization employs elements from computer graphics to create meaningful, interactive visual representations of medical data, and it has become an influential field of research for many advanced applications like radiation oncology, among others. Visual representations employ the user's cognitive capabilities to support and accelerate diagnostic, planning, and quality assurance workflows based on involved patient data.

SUMMARY

This article discusses the basic underlying principles of visualization in the application domain of radiation oncology. The main visualization strategies, such as slice-based representations and surface and volume rendering are presented. Interaction topics, i.e., the combination of visualization and automated analysis methods, are also discussed. Key Messages: Slice-based representations are a common approach in radiation oncology, while volume visualization also has a long-standing history in the field. Perception within both representations can benefit further from advanced approaches, such as image fusion and multivolume or hybrid rendering. While traditional slice-based and volume representations keep evolving, the dimensionality and complexity of medical data are also increasing. To address this, visual analytics strategies are valuable, particularly for cohort or uncertainty visualization. Interactive visual analytics approaches represent a new opportunity to integrate knowledgeable experts and their cognitive abilities in exploratory processes which cannot be conducted by solely automatized methods.

Facetto: Combining Unsupervised and Supervised Learning for Hierarchical Phenotype Analysis in Multi-Channel Image Data

Facetto is a scalable visual analytics application that is used to discover single-cell phenotypes in high-dimensional multi-channel microscopy images of human tumors and tissues. Such images represent the cutting edge of digital histology and promise to revolutionize how diseases such as cancer are studied, diagnosed, and treated. Highly multiplexed tissue images are complex, comprising 10⁹ or more pixels, 60-plus channels, and millions of individual cells. This makes manual analysis challenging and error-prone. Existing automated approaches are also inadequate, in large part, because they are unable to effectively exploit the deep knowledge of human tissue biology available to anatomic pathologists. To overcome these challenges, Facetto enables a semi-automated analysis of cell types and states. It integrates unsupervised and supervised learning into the image and feature exploration process and offers tools for analytical provenance. Experts can cluster the data to discover new types of cancer and immune cells and use clustering results to train a convolutional neural network that classifies new cells accordingly. Likewise, the output of classifiers can be clustered to discover aggregate patterns and phenotype subsets. We also introduce a new hierarchical approach to keep track of analysis steps and data subsets created by users; this assists in the identification of cell types. Users can build phenotype trees and interact with the resulting hierarchical structures of both high-dimensional feature and image spaces. We report on use-cases in which domain scientists explore various large-scale fluorescence imaging datasets. We demonstrate how Facetto assists users in steering the clustering and classification process, inspecting analysis results, and gaining new scientific insights into cancer biology.

Visual Analytics for the Representation, Exploration, and Analysis of High-Dimensional, Multi-faceted Medical Data

Medicine is among those research fields with a significant impact on humans and their health. Already for decades, medicine has established a tight coupling with the visualization domain, proving the importance of developing visualization techniques, designed exclusively for this research discipline. However, medical data is steadily increasing in complexity with the appearance of heterogeneous, multi-modal, multi-parametric, cohort or population, as well as uncertain data. To deal with this kind of complex data, the field of Visual Analytics has emerged. In this chapter, we discuss the many dimensions and facets of medical data. Based on this classification, we provide a general overview of state-of-the-art visualization systems and solutions dealing with high-dimensional, multi-faceted data. Our particular focus will be on multi-modal, multi-parametric data, on data from cohort or population studies and on uncertain data, especially with respect to Visual Analytics applications for the representation, exploration, and analysis of high-dimensional, multi-faceted medical data.

Comparing Cross-Sections and 3D Renderings for Surface Matching Tasks Using Physical Ground Truths

Within the visualization community there are some well-known techniques for visualizing 3D spatial data and some general assumptions about how perception affects the performance of these techniques in practice. However, there is a lack of empirical research backing up the possible performance differences among the basic techniques for general tasks. One such assumption is that 3D renderings are better for obtaining an overview, whereas cross sectional visualizations such as the commonly used Multi-Planar Reformation (MPR) are better for supporting detailed analysis tasks. In the present study we investigated this common assumption by examining the difference in performance between MPR and 3D rendering for correctly identifying a known surface. We also examined whether prior experience working with image data affects the participant's performance, and whether there was any difference between interactive or static versions of the visualizations. Answering this question is important because it can be used as part of a scientific and empirical basis for determining when to use which of the two techniques. An advantage of the present study compared to other studies is that several factors were taken into account to compare the two techniques. The problem was examined through an experiment with 45 participants, where physical objects were used as the known surface (ground truth). Our findings showed that: 1. The 3D renderings largely outperformed the cross sections; 2. Interactive visualizations were partially more effective than static visualizations; and 3. The high experience group did not generally outperform the low experience group.

Imalytics Preclinical: Interactive Analysis of Biomedical Volume Data

A software tool is presented for interactive segmentation of volumetric medical data sets. To allow interactive processing of large data sets, segmentation operations, and rendering are GPU-accelerated. Special adjustments are provided to overcome GPU-imposed constraints such as limited memory and host-device bandwidth. A general and efficient undo/redo mechanism is implemented using GPU-accelerated compression of the multiclass segmentation state. A broadly applicable set of interactive segmentation operations is provided which can be combined to solve the quantification task of many types of imaging studies. A fully GPU-accelerated ray casting method for multiclass segmentation rendering is implemented which is well-balanced with respect to delay, frame rate, worst-case memory consumption, scalability, and image quality. Performance of segmentation operations and rendering are measured using high-resolution example data sets showing that GPU-acceleration greatly improves the performance. Compared to a reference marching cubes implementation, the rendering was found to be superior with respect to rendering delay and worst-case memory consumption while providing sufficiently high frame rates for interactive visualization and comparable image quality. The fast interactive segmentation operations and the accurate rendering make our tool particularly suitable for efficient analysis of multimodal image data sets which arise in large amounts in preclinical imaging studies.

Renal blood perfusion in GK rats using targeted contrast enhanced ultrasonography

OBJECTIVE

To explore application of targeted contrast enhanced ultrasonography in diagnosis of early stage vascular endothelial injury and diabetic nephropathy.

METHODS

Targeted SonoVue-TM microbubble was prepared by attaching anti-TM monoclonal antibody to the surface of ordinary microbubble SonoVue by biotin - avidin bridge method and ultrasonic instrument was used to evaluate the developing situation of targeted microbubble in vitro. Twenty 12-week-old male GK rats and 20 Wistar rats were enrolled in this study, and were randomly divided into targeted angiography group and ordinary angiography group. Targeted microbubbles SonoVue-TM or general microbubble SonoVue were rapidly injected to the rats via tail vein; the developing situation of the two contrast agents in rats kidneys was dynamically observed. Time-intensity curve was used to analyze rat kidney perfusion characteristics in different groups.

RESULTS

Targeted ultrasound microbubble SonoVue-TM was successfully constructed, and it could be used to develop an external image. Targeted microbubbles SonoVue-TM enabled clear development of experimental rat kidney. Time-intensity curve shapes of rat kidney of the two groups showed as single apex with steep ascending and slowly descending branch. Compared with the control group, the rising slope of the GK rat renal cortex, medulla in targeted angiography group increased (P < 0.05); the peak intensity of medulla increased (P < 0.05), and the total area under the curve of medulla increased (P < 0.05). Compared with control group, the ascending branch of the GK rat in renal cortex, medulla in ordinary angiography group increased (P < 0.05). The peak intensity of the curve increased (P < 0.05), and the total area under the curve increased (P < 0.05). Compared with the ordinary angiography group, the peak of GK rat medulla curve in targeted angiography group intensity increased (P < 0.05), and the total area under the curve increased (P < 0.05).

CONCLUSIONS

Targeted microbubbles SonoVue-TM can make a clear development of experimental rat kidney, its stable performance meet the requirement of ultrasonic observation time limit, and it can reflect early changes of blood perfusion in GK rat kindey.

Interactive Visual Analysis of Image-Centric Cohort Study Data

Epidemiological population studies impose information about a set of subjects (a cohort) to characterize disease-specific risk factors. Cohort studies comprise heterogenous variables describing the medical condition as well as demographic and lifestyle factors and, more recently, medical image data. We propose an Interactive Visual Analysis (IVA) approach that enables epidemiologists to rapidly investigate the entire data pool for hypothesis validation and generation. We incorporate image data, which involves shape-based object detection and the derivation of attributes describing the object shape. The concurrent investigation of image-based and non-image data is realized in a web-based multiple coordinated view system, comprising standard views from information visualization and epidemiological data representations such as pivot tables. The views are equipped with brushing facilities and augmented by 3D shape renderings of the segmented objects, e.g., each bar in a histogram is overlaid with a mean shape of the associated subgroup of the cohort. We integrate an overview visualization, clustering of variables and object shape for data-driven subgroup definition and statistical key figures for measuring the association between variables. We demonstrate the IVA approach by validating and generating hypotheses related to lower back pain as part of a qualitative evaluation.

Visualization and visual analysis of multifaceted scientific data: a survey

Visualization and visual analysis play important roles in exploring, analyzing, and presenting scientific data. In many disciplines, data and model scenarios are becoming multifaceted: data are often spatiotemporal and multivariate; they stem from different data sources (multimodal data), from multiple simulation runs (multirun/ensemble data), or from multiphysics simulations of interacting phenomena (multimodel data resulting from coupled simulation models). Also, data can be of different dimensionality or structured on various types of grids that need to be related or fused in the visualization. This heterogeneity of data characteristics presents new opportunities as well as technical challenges for visualization research. Visualization and interaction techniques are thus often combined with computational analysis. In this survey, we study existing methods for visualization and interactive visual analysis of multifaceted scientific data. Based on a thorough literature review, a categorization of approaches is proposed. We cover a wide range of fields and discuss to which degree the different challenges are matched with existing solutions for visualization and visual analysis. This leads to conclusions with respect to promising research directions, for instance, to pursue new solutions for multirun and multimodel data as well as techniques that support a multitude of facets.

Representative Factor Generation for the Interactive Visual Analysis of High-Dimensional Data

Datasets with a large number of dimensions per data item (hundreds or more) are challenging both for computational and visual analysis. Moreover, these dimensions have different characteristics and relations that result in sub-groups and/or hierarchies over the set of dimensions. Such structures lead to heterogeneity within the dimensions. Although the consideration of these structures is crucial for the analysis, most of the available analysis methods discard the heterogeneous relations among the dimensions. In this paper, we introduce the construction and utilization of representative factors for the interactive visual analysis of structures in high-dimensional datasets. First, we present a selection of methods to investigate the sub-groups in the dimension set and associate representative factors with those groups of dimensions. Second, we introduce how these factors are included in the interactive visual analysis cycle together with the original dimensions. We then provide the steps of an analytical procedure that iteratively analyzes the datasets through the use of representative factors. We discuss how our methods improve the reliability and interpretability of the analysis process by enabling more informed selections of computational tools. Finally, we demonstrate our techniques on the analysis of brain imaging study results that are performed over a large group of subjects.

Cardiac MR perfusion image processing techniques: a survey

First-pass cardiac MR perfusion (CMRP) imaging has undergone rapid technical advancements in recent years. Although the efficacy of CMRP imaging in the assessment of coronary artery diseases (CAD) has been proven, its clinical use is still limited. This limitation stems, in part, from manual interaction required to quantitatively analyze the large amount of data. This process is tedious, time-consuming, and prone to operator bias. Furthermore, acquisition and patient related image artifacts reduce the accuracy of quantitative perfusion assessment. With the advent of semi- and fully automatic image processing methods, not only the challenges posed by these artifacts have been overcome to a large extent, but a significant reduction has also been achieved in analysis time and operator bias. Despite an extensive literature on such image processing methods, to date, no survey has been performed to discuss this dynamic field. The purpose of this article is to provide an overview of the current state of the field with a categorical study, along with a future perspective on the clinical acceptance of image processing methods in the diagnosis of CAD.

Interactive, graph-based visual analysis of high-dimensional, multi-parameter fluorescence microscopy data in toponomics

In Toponomics, the function protein pattern in cells or tissue (the toponome) is imaged and analyzed for applications in toxicology, new drug development and patient-drug-interaction. The most advanced imaging technique is robot-driven multi-parameter fluorescence microscopy. This technique is capable of co-mapping hundreds of proteins and their distribution and assembly in protein clusters across a cell or tissue sample by running cycles of fluorescence tagging with monoclonal antibodies or other affinity reagents, imaging, and bleaching in situ. The imaging results in complex multi-parameter data composed of one slice or a 3D volume per affinity reagent. Biologists are particularly interested in the localization of co-occurring proteins, the frequency of co-occurrence and the distribution of co-occurring proteins across the cell. We present an interactive visual analysis approach for the evaluation of multi-parameter fluorescence microscopy data in toponomics. Multiple, linked views facilitate the definition of features by brushing multiple dimensions. The feature specification result is linked to all views establishing a focus+context visualization in 3D. In a new attribute view, we integrate techniques from graph visualization. Each node in the graph represents an affinity reagent while each edge represents two co-occurring affinity reagent bindings. The graph visualization is enhanced by glyphs which encode specific properties of the binding. The graph view is equipped with brushing facilities. By brushing in the spatial and attribute domain, the biologist achieves a better understanding of the function protein patterns of a cell. Furthermore, an interactive table view is integrated which summarizes unique fluorescence patterns. We discuss our approach with respect to a cell probe containing lymphocytes and a prostate tissue section.

Interactive coordinated multiple-view visualization of biomechanical motion data

We present an interactive framework for exploring space-time and form-function relationships in experimentally collected high-resolution biomechanical data sets. These data describe complex 3D motions (e.g. chewing, walking, flying) performed by animals and humans and captured via high-speed imaging technologies, such as biplane fluoroscopy. In analyzing these 3D biomechanical motions, interactive 3D visualizations are important, in particular, for supporting spatial analysis. However, as researchers in information visualization have pointed out, 2D visualizations can also be effective tools for multi-dimensional data analysis, especially for identifying trends over time. Our approach, therefore, combines techniques from both 3D and 2D visualizations. Specifically, it utilizes a multi-view visualization strategy including a small multiples view of motion sequences, a parallel coordinates view, and detailed 3D inspection views. The resulting framework follows an overview first, zoom and filter, then details-on-demand style of analysis, and it explicitly targets a limitation of current tools, namely, supporting analysis and comparison at the level of a collection of motions rather than sequential analysis of a single or small number of motions. Scientific motion collections appropriate for this style of analysis exist in clinical work in orthopedics and physical rehabilitation, in the study of functional morphology within evolutionary biology, and in other contexts. An application is described based on a collaboration with evolutionary biologists studying the mechanics of chewing motions in pigs. Interactive exploration of data describing a collection of more than one hundred experimentally captured pig chewing cycles is described.

Survey of the visual exploration and analysis of perfusion data

Dynamic contrast-enhanced image data (perfusion data) are used to characterize regional tissue perfusion. Perfusion data consist of a sequence of images, acquired after a contrast agent bolus is applied. Perfusion data are used for diagnostic purposes in oncology, ischemic stroke assessment or myocardial ischemia. The diagnostic evaluation of perfusion data is challenging, since the data is complex and exhibits various artifacts, e.g., motion artifacts. We provide an overview on existing methods to analyze, and visualize CT and MR perfusion data. The integrated visualization of several 2D parameter maps, the 3D visualization of parameter volumes and exploration techniques are discussed. An essential aspect in the diagnosis of perfusion data is the correlation between perfusion data and derived time-intensity curves as well as with other image data, in particular with high resolution morphologic image data. We discuss visualization support with respect to the three major application areas: ischemic stroke diagnosis, breast tumor diagnosis and the diagnosis of coronary heart disease.

Surface extraction from multi-field particle volume data using multi-dimensional cluster visualization

Data sets resulting from physical simulations typically contain a multitude of physical variables. It is, therefore, desirable that visualization methods take into account the entire multi-field volume data rather than concentrating on one variable. We present a visualization approach based on surface extraction from multi-field particle volume data. The surfaces segment the data with respect to the underlying multi-variate function. Decisions on segmentation properties are based on the analysis of the multi-dimensional feature space. The feature space exploration is performed by an automated multi-dimensional hierarchical clustering method, whose resulting density clusters are shown in the form of density level sets in a 3D star coordinate layout. In the star coordinate layout, the user can select clusters of interest. A selected cluster in feature space corresponds to a segmenting surface in object space. Based on the segmentation property induced by the cluster membership, we extract a surface from the volume data. Our driving applications are Smoothed Particle Hydrodynamics (SPH) simulations, where each particle carries multiple properties. The data sets are given in the form of unstructured point-based volume data. We directly extract our surfaces from such data without prior resampling or grid generation. The surface extraction computes individual points on the surface, which is supported by an efficient neighborhood computation. The extracted surface points are rendered using point-based rendering operations. Our approach combines methods in scientific visualization for object-space operations with methods in information visualization for feature-space operations.