Display of fused images: methods, interpretation, and diagnostic improvements

Emergence of "Big Data" and Its Potential and Current Limitations in Medical Imaging

Although electronic imaging was performed in the early 1950s in nuclear medicine, it was the introduction of computed tomography in 1972 that caused a revolution in medical imaging in that it marked the beginning of the inevitable transformation to digital imaging. This transformation is now more or less complete. While initially these CT images were relatively small, comprised of only about 6400 pixels per slice, the steady move toward higher spatial resolution, multislice imaging, digital radiography, and fluoroscopy rapidly increased the size of images and the amount of data required to be stored, processed, displayed, and moved about in a medical imaging department. The more recent introduction of digital pathology with submicron-sized pixels and the need for color further increases these demands. Rising work volumes in hospital, a push for cost containment, and a move toward greater precision in diagnosis and treatment of disease all work together to motivate the development of automated image analysis algorithms and techniques to improve efficiencies in in vivo imaging and pathology. This may require bringing together information from different imaging and nonimaging sources within the institution. While technological development has provided practical means for storage of the burgeoning data load and the use of multiple processors and high-speed networks has enabled more sophisticated analysis locally or in the cloud, challenges remain in terms of the ability to integrate data from different systems, the development of appropriately annotated image bases for training and testing of algorithms, and issues around privacy and ownership in obtaining access to patient-related data.

Glyph-Based Comparative Visualization for Diffusion Tensor Fields

Diffusion Tensor Imaging (DTI) is a magnetic resonance imaging modality that enables the in-vivo reconstruction and visualization of fibrous structures. To inspect the local and individual diffusion tensors, glyph-based visualizations are commonly used since they are able to effectively convey full aspects of the diffusion tensor. For several applications it is necessary to compare tensor fields, e.g., to study the effects of acquisition parameters, or to investigate the influence of pathologies on white matter structures. This comparison is commonly done by extracting scalar information out of the tensor fields and then comparing these scalar fields, which leads to a loss of information. If the glyph representation is kept, simple juxtaposition or superposition can be used. However, neither facilitates the identification and interpretation of the differences between the tensor fields. Inspired by the checkerboard style visualization and the superquadric tensor glyph, we design a new glyph to locally visualize differences between two diffusion tensors by combining juxtaposition and explicit encoding. Because tensor scale, anisotropy type, and orientation are related to anatomical information relevant for DTI applications, we focus on visualizing tensor differences in these three aspects. As demonstrated in a user study, our new glyph design allows users to efficiently and effectively identify the tensor differences. We also apply our new glyphs to investigate the differences between DTI datasets of the human brain in two different contexts using different b-values, and to compare datasets from a healthy and HIV-infected subject.

Comparative exploration of whole-body MR through locally rigid transforms

PURPOSE

Whole-body MRI is seeing increasing use in the study and diagnosis of disease progression. In this, a central task is the visual assessment of the progressive changes that occur between two whole-body MRI datasets, taken at baseline and follow-up. Current radiological workflow for this consists in manual search of each organ of interest on both scans, usually on multiple data channels, for further visual comparison. Large size of datasets, significant posture differences, and changes in patient anatomy turn manual matching in an extremely labor-intensive task that requires from radiologists high concentration for long period of time. This strongly limits the productivity and increases risk of underdiagnosis.

MATERIALS AND METHODS

We present a novel approach to the comparative visual analysis of whole-body MRI follow-up data. Our method is based on interactive derivation of locally rigid transforms from a pre-computed whole-body deformable registration. Using this approach, baseline and follow-up slices can be interactively matched with a single mouse click in the anatomical region of interest. In addition to the synchronized side-by-side baseline and matched follow-up slices, we have integrated four techniques to further facilitate the visual comparison of the two datasets: the "deformation sphere", the color fusion view, the magic lens, and a set of uncertainty iso-contours around the current region of interest.

RESULTS

We have applied our method to the study of cancerous bone lesions over time in patients with Kahler's disease. During these studies, the radiologist carefully visually examines a large number of anatomical sites for changes. Our interactive locally rigid matching approach was found helpful in localization of cancerous lesions and visual assessment of changes between different scans. Furthermore, each of the features integrated in our software was separately evaluated by the experts.

CONCLUSION

We demonstrated how our method significantly facilitates examination of whole-body MR datasets in follow-up studies by enabling the rapid interactive matching of regions of interest and by the explicit visualization of change.

Regions-based illustrative visualization of multimodal datasets

We present as novel method for the exploration of multiple overlapping volumes that provides flexibility to merge data in different ways in different regions. In each region, either one of the modalities is rendered alone or the fusion of two modalities is shown. In the regions where data is fused, the relative weights of each modality are defined with a 2D transfer function depending on the voxel's pair of property values. The regions can be defined interactively by painting on the volume. Alternatively, when one of the modalities has been pre-classified, a graph representation of the dataset is constructed, and regions can be defined as sets of voxels fulfilling a specific combination of classification criteria. In both cases, a different fusion and shading function can be defined for each region. In this way, illustrative images of the dataset can be easily generated applying effects of cutting away, ghosting and modality enhancement.

Hybrid imaging in planar scintigraphy: new implementations and historical precedents

Fusion of tomographic radionuclide studies with anatomical examinations has become standard practice in positron emission tomography (PET) and single photon emission computed tomography (SPECT) imaging. Nonetheless, fusion of planar scintigraphic images with an anatomical modality remains distinctly uncommon, although methods to do so have appeared sporadically in the literature during the past 2 decades. In this article we review several techniques that have been used to combine planar scintigraphic images with radiographs and visual (photographic) images. Rigid or affine transformations have been performed to co-register the planar images with each other using custom, commercial, or public domain software. Display of the hybrid images has been achieved primarily with nonselective color-fusion methods. Promising efforts are underway to develop a technique of fusing planar lymphoscintigraphic images with CT topograms (scout images) obtained on the SPECT-CT camera in a manner that compensates for position-dependent variation in magnification that affects the CT scout. An advantage of this approach is that both of the component images are acquired on the same gantry, without need for repositioning of the patient. It is instructive to note that techniques of fusing rectilinear scans with radiographic and visual images were first developed more than 50 years ago. The revisiting of these methods after many decades reflects a fundamental need for spatial orientation in nuclear medicine that fusion imaging can also bring to planar scintigraphic studies.

Extracellular matrix deposition and scaffold biodegradation in an in vitro three-dimensional model of bone by X-ray computed microtomography

The development of an in vitro model of bone and the optimization of tools for determining the biological processes occurring during bone repair remains a major goal in the field of bone tissue engineering. Recently, a model based on a three-dimensional co-culture of osteoblasts and osteoclast precursors in Skelite(TM) scaffolds was developed. Although induction of osteoblast and osteoclast differentiation was observed, a complete evaluation of bone deposition and biodegradation processes was missing due to technical limitations. In the current study, both X-ray computed microtomography and histological analysis were used to monitor these two key biological processes in the same in vitro model. Either osteoblasts or a combination of osteoblasts and osteoclasts were seeded on Skelite(TM) scaffolds. Scaffold biodegradation and increased bone deposition together with a more organized extracellular matrix were observed in the co-cultures, highlighting the role of osteoclasts in the determination and regulation of bone deposition. Results confirmed the potential and relevance of co-culturing osteoblasts and osteoclasts to resemble native tissue. The combination of X-ray computed microtomography and histology presented in this study could be useful in future studies for the validation and development of new in vitro culture systems for bone tissue engineering.

Comparative Visualization of Ensembles Using Ensemble Surface Slicing

By definition, an ensemble is a set of surfaces or volumes derived from a series of simulations or experiments. Sometimes the series is run with different initial conditions for one parameter to determine parameter sensitivity. The understanding and identification of visual similarities and differences among the shapes of members of an ensemble is an acute and growing challenge for researchers across the physical sciences. More specifically, the task of gaining spatial understanding and identifying similarities and differences between multiple complex geometric data sets simultaneously has proved challenging. This paper proposes a comparison and visualization technique to support the visual study of parameter sensitivity. We present a novel single-image view and sampling technique which we call Ensemble Surface Slicing (ESS). ESS produces a single image that is useful for determining differences and similarities between surfaces simultaneously from several data sets. We demonstrate the usefulness of ESS on two real-world data sets from our collaborators.

High-resolution X-ray microtomography for three-dimensional imaging of cardiac progenitor cell homing in infarcted rat hearts

The recent introduction of stem cells in cardiology provides new tools in understanding the regenerative processes of the normal and pathological heart and has opened a search for new therapeutic strategies. Recent published reports have contributed to identifying possible cellular therapy approaches to generate new myocardium, involving transcoronary and intramyocardial injection of progenitor cells. However, one of the limiting factors in the overall interpretation of clinical results obtained by cell therapy is represented by the lack of three-dimensional (3D) high-resolution methods for the visualization of the injected cells and their fate within the myocardium. This work shows that X-ray computed microtomography may offer the unique possibility of detecting, with high definition and resolution and in ex vivo conditions, the 3D spatial distribution of rat cardiac progenitor cells, labelled with iron oxide nanoparticles, inside the infarcted rat heart early after injection. The obtained 3D images represent a very innovative progress as compared to experimental two-dimensional (2D) histological analysis, which requires time-consuming energies for image reconstruction in order to provide the overall distribution of rat clonogenic cells within the heart. Through microtomography, we were able to observe in 3D the presence of these cells within damaged cardiac tissue, with important structural details that are difficult to visualize by conventional bidimensional imaging techniques. This new 3D-imaging approach appears to be an important way to investigate the cellular events involved in cardiac regeneration and represents a promising tool for future clinical applications.

Comparative visualization for parameter studies of dataset series

This paper proposes comparison and visualization techniques to carry out parameter studies for the special application area of dimensional measurement using 3D X-ray computed tomography (3DCT). A dataset series is generated by scanning a specimen multiple times by varying parameters of an industrial 3DCT device. A high-resolution series is explored using our planar-reformatting-based visualization system. We present a novel multi-image view and an edge explorer for comparing and visualizing gray values and edges of several datasets simultaneously. Visualization results and quantitative data are displayed side by side. Our technique is scalable and generic. It can be effective in various application areas like parameter studies of imaging modalities and dataset artifact detection. For fast data retrieval and convenient usability, we use bricking of the datasets and efficient data structures. We evaluate the applicability of the proposed techniques in collaboration with our company partners.

Kombinierte Hybridsysteme (PET/CT, SPECT/CT) versus multimodale Bildgebung mit getrennten Systemen

With increasing use of combined PET/CT scanners in the last few years, multimodality imaging (Nuclear Medicine/Radiology) found its way into clinical routine diagnostics. In this overview, necessary components for multimodality imaging, strategies for image analysis and image presentation, and diagnostic goals of combined imaging are demonstrated and discussed. A special focus is on the question, whether combined scanners can be replaced by a software approach with separated modalities. Advantages and limitations of multimodality imaging with combined or separated scanners are shown.