Towards Portable Large-Scale Image Processing with High-Performance Computing

High-throughput, large-scale medical image computing demands tight integration of high-performance computing (HPC) infrastructure for data storage, job distribution, and image processing. The Vanderbilt University Institute for Imaging Science (VUIIS) Center for Computational Imaging (CCI) has constructed a large-scale image storage and processing infrastructure that is composed of (1) a large-scale image database using the eXtensible Neuroimaging Archive Toolkit (XNAT), (2) a content-aware job scheduling platform using the Distributed Automation for XNAT pipeline automation tool (DAX), and (3) a wide variety of encapsulated image processing pipelines called “spiders.” The VUIIS CCI medical image data storage and processing infrastructure have housed and processed nearly half-million medical image volumes with Vanderbilt Advanced Computing Center for Research and Education (ACCRE), which is the HPC facility at the Vanderbilt University. The initial deployment was natively deployed (i.e., direct installations on a bare-metal server) within the ACCRE hardware and software environments, which lead to issues of portability and sustainability. First, it could be laborious to deploy the entire VUIIS CCI medical image data storage and processing infrastructure to another HPC center with varying hardware infrastructure, library availability, and software permission policies. Second, the spiders were not developed in an isolated manner, which has led to software dependency issues during system upgrades or remote software installation. To address such issues, herein, we describe recent innovations using containerization techniques with XNAT/DAX which are used to isolate the VUIIS CCI medical image data storage and processing infrastructure from the underlying hardware and software environments. The newly presented XNAT/DAX solution has the following new features: (1) multi-level portability from system level to the application level, (2) flexible and dynamic software development and expansion, and (3) scalable spider deployment compatible with HPC clusters and local workstations.

Prefrontal-Thalamic Anatomical Connectivity and Executive Cognitive Function in Schizophrenia

BACKGROUND

Executive cognitive functions, including working memory, cognitive flexibility, and inhibition, are impaired in schizophrenia. Executive functions rely on coordinated information processing between the prefrontal cortex (PFC) and thalamus, particularly the mediodorsal nucleus. This raises the possibility that anatomical connectivity between the PFC and mediodorsal thalamus may be 1) reduced in schizophrenia and 2) related to deficits in executive function. The current investigation tested these hypotheses.

METHODS

Forty-five healthy subjects and 62 patients with a schizophrenia spectrum disorder completed a battery of tests of executive function and underwent diffusion-weighted imaging. Probabilistic tractography was used to quantify anatomical connectivity between six cortical regions, including PFC, and the thalamus. Thalamocortical anatomical connectivity was compared between healthy subjects and patients with schizophrenia using region-of-interest and voxelwise approaches, and the association between PFC-thalamic anatomical connectivity and severity of executive function impairment was examined in patients.

RESULTS

Anatomical connectivity between the thalamus and PFC was reduced in schizophrenia. Voxelwise analysis localized the reduction to areas of the mediodorsal thalamus connected to lateral PFC. Reduced PFC-thalamic connectivity in schizophrenia correlated with impaired working memory but not cognitive flexibility and inhibition. In contrast to reduced PFC-thalamic connectivity, thalamic connectivity with somatosensory and occipital cortices was increased in schizophrenia.

CONCLUSIONS

The results are consistent with models implicating disrupted PFC-thalamic connectivity in the pathophysiology of schizophrenia and mechanisms of cognitive impairment. PFC-thalamic anatomical connectivity may be an important target for procognitive interventions. Further work is needed to determine the implications of increased thalamic connectivity with sensory cortex.

DAX - The Next Generation: Towards One Million Processes on Commodity Hardware

Large scale image processing demands a standardized way of not only storage but also a method for job distribution and scheduling. The eXtensible Neuroimaging Archive Toolkit (XNAT) is one of several platforms that seeks to solve the storage issues. Distributed Automation for XNAT (DAX) is a job control and distribution manager. Recent massive data projects have revealed several bottlenecks for projects with >100,000 assessors (i.e., data processing pipelines in XNAT). In order to address these concerns, we have developed a new API, which exposes a direct connection to the database rather than REST API calls to accomplish the generation of assessors. This method, consistent with XNAT, keeps a full history for auditing purposes. Additionally, we have optimized DAX to keep track of processing status on disk (called DISKQ) rather than on XNAT, which greatly reduces load on XNAT by vastly dropping the number of API calls. Finally, we have integrated DAX into a Docker container with the idea of using it as a Docker controller to launch Docker containers of image processing pipelines. Using our new API, we reduced the time to create 1,000 assessors (a sub-cohort of our case project) from 65040 seconds to 229 seconds (a decrease of over 270 fold). DISKQ, using pyXnat, allows launching of 400 jobs in under 10 seconds which previously took 2,000 seconds. Together these updates position DAX to support projects with hundreds of thousands of scans and to run them in a time-efficient manner.

Performance Management of High Performance Computing for Medical Image Processing in Amazon Web Services

Adopting high performance cloud computing for medical image processing is a popular trend given the pressing needs of large studies. Amazon Web Services (AWS) provide reliable, on-demand, and inexpensive cloud computing services. Our research objective is to implement an affordable, scalable and easy-to-use AWS framework for the Java Image Science Toolkit (JIST). JIST is a plugin for Medical-Image Processing, Analysis, and Visualization (MIPAV) that provides a graphical pipeline implementation allowing users to quickly test and develop pipelines. JIST is DRMAA-compliant allowing it to run on portable batch system grids. However, as new processing methods are implemented and developed, memory may often be a bottleneck for not only lab computers, but also possibly some local grids. Integrating JIST with the AWS cloud alleviates these possible restrictions and does not require users to have deep knowledge of programming in Java. Workflow definition/management and cloud configurations are two key challenges in this research. Using a simple unified control panel, users have the ability to set the numbers of nodes and select from a variety of pre-configured AWS EC2 nodes with different numbers of processors and memory storage. Intuitively, we configured Amazon S3 storage to be mounted by pay-for-use Amazon EC2 instances. Hence, S3 storage is recognized as a shared cloud resource. The Amazon EC2 instances provide pre-installs of all necessary packages to run JIST. This work presents an implementation that facilitates the integration of JIST with AWS. We describe the theoretical cost/benefit formulae to decide between local serial execution versus cloud computing and apply this analysis to an empirical diffusion tensor imaging pipeline.

Increased myocardial native T1 and extracellular volume in patients with Duchenne muscular dystrophy

BACKGROUND

Duchenne muscular dystrophy (DMD) cardiomyopathy is a progressive disease for which there is no cure. Disease-specific therapies are needed that can be initiated before irreversible myocardial damage ensues. In order to evaluate therapeutic efficacy, surrogate endpoints other than ejection fraction must be found. The hypothesis of this study is that T1 and extracellular volume fraction (ECV) mapping using cardiovascular magnetic resonance (CMR) can detect diffuse extracellular matrix expansion in DMD patients with normal left ventricular ejection fraction (LVEF) and without myocardial late gadolinium enhancement (LGE).

METHODS

Thirty-one DMD and 11 healthy control participants were prospectively enrolled. CMR using a modified Look-Locker (MOLLI) sequence was performed in all participants before and after contrast administration. T1 and ECV maps of the mid left ventricular myocardium were generated and regions of interest were contoured using the standard 6-segment AHA model. Global and segmental values were compared between DMD and controls using a Wilcoxon rank-sum test.

RESULTS

The DMD participants had significantly higher mean native T1 compared with controls (1045 ms vs. 988 ms, p = 0.001). DMD participants with normal LVEF and without evidence of LGE also demonstrated elevated mean native T1 (1039 ms vs. 988 ms, p = 0.002, and 1038 ms vs. 988 ms, p = 0.011). DMD participants had a significantly greater mean ECV than controls (0.31 vs. 0.24, p < 0.001), even in the settings of normal LVEF (0.28 vs. 0.24, p < 0.001) and negative LGE (0.29 vs. 0.24, p = 0.001).

CONCLUSIONS

DMD participants have elevated LV myocardial native T1 and ECV, even in the setting of normal LVEF and in the absence of LGE. T1 and ECV mapping in DMD have potential to serve as surrogate cardiomyopathy outcome measures for clinical trials.

Sex differences in psychophysical and neurophysiological responses to pain in older adults: a cross-sectional study

Background

Neuroimaging studies in younger adults have demonstrated sex differences in brain processing of painful experimental stimuli. Such differences may contribute to findings that women suffer disproportionately from pain. It is not known whether sex-related differences in pain processing extend to older adults.

Methods

This cross-sectional study investigated sex differences in pain reports and brain response to pain in 12 cognitively healthy older female adults and 12 cognitively healthy age-matched older male adults (age range 65–81, median = 67). Participants underwent psychophysical assessments of thermal pain responses, functional MRI, and psychosocial assessment.

Results

When compared to older males, older females reported experiencing mild and moderate pain at lower stimulus intensities (i.e., exhibited greater pain sensitivity; Cohen’s d = 0.92 and 0.99, respectively, p < 0.01) yet did not report greater pain-associated unpleasantness. Imaging results indicated that, despite the lower stimulus intensities required to elicit mild pain detection in females, they exhibited less deactivations than males in regions associated with the default mode network (DMN) and in regions associated with pain affect (bilateral dorsolateral prefrontal cortex, somatomotor area, rostral anterior cingulate cortex (rACC), and dorsal ACC). Conversely, at moderate pain detection levels, males exhibited greater activation than females in several ipsilateral regions typically associated with pain sensation (e.g., primary (SI) and secondary somatosensory cortices (SII) and posterior insula). Sex differences were found in the association of brain activation in the left rACC with pain unpleasantness. In the combined sample of males and females, brain activation in the right secondary somatosensory cortex was associated with pain unpleasantness.

Conclusions

Cognitively healthy older adults in the sixth and seventh decades of life exhibit similar sex differences in pain sensitivity compared to those reported in younger individuals. However, older females did not find pain to be more unpleasant. Notably, increased sensitivity to mild pain in older females was reflected via less brain deactivation in regions associated with both the DMN and in pain affect. Current findings elevate the rACC as a key region associated with sex differences in reports of pain unpleasantness and brain deactivation in older adults. Also, pain affect may be encoded in SII in both older males and females.

Electronic supplementary material

The online version of this article (doi:10.1186/s13293-015-0041-y) contains supplementary material, which is available to authorized users.

Subjective memory complaint only relates to verbal episodic memory performance in mild cognitive impairment

BACKGROUND

A cognitive concern from the patient, informant, or clinician is required for the diagnosis of mild cognitive impairment (MCI); however, the cognitive and neuroanatomical correlates of complaint are poorly understood.

OBJECTIVE

We assessed how self-complaint relates to cognitive and neuroimaging measures in older adults with MCI.

METHOD

MCI participants were drawn from the Alzheimer's Disease Neuroimaging Initiative and dichotomized into two groups based on the presence of self-reported memory complaint (no complaint n = 191, 77 ± 7 years; complaint n = 206, 73 ± 8 years). Cognitive outcomes included episodic memory, executive functioning, information processing speed, and language. Imaging outcomes included regional lobar volumes (frontal, parietal, temporal, cingulate) and specific medial temporal lobe structures (hippocampal volume, entorhinal cortex thickness, parahippocampal gyrus thickness).

RESULTS

Linear regressions, adjusting for age, gender, race, education, Mini-Mental State Examination score, mood, and apolipoprotein E4 status, found that cognitive complaint related to immediate (β = -1.07, p < 0.001) and delayed episodic memory performances assessed on a serial list learning task (β = -1.06, p = 0.001) but no other cognitive measures or neuroimaging markers.

CONCLUSIONS

Self-reported memory concern was unrelated to structural neuroimaging markers of atrophy and measures of information processing speed, executive functioning, or language. In contrast, subjective memory complaint related to objective verbal episodic learning performance. Future research is warranted to better understand the relation between cognitive complaint and surrogate markers of abnormal brain aging, including Alzheimer's disease, across the cognitive aging spectrum.

Vanderbilt University Institute of Imaging Science Center for Computational Imaging XNAT: A multimodal data archive and processing environment

The Vanderbilt University Institute for Imaging Science (VUIIS) Center for Computational Imaging (CCI) has developed a database built on XNAT housing over a quarter of a million scans. The database provides framework for (1) rapid prototyping, (2) large scale batch processing of images and (3) scalable project management. The system uses the web-based interfaces of XNAT and REDCap to allow for graphical interaction. A python middleware layer, the Distributed Automation for XNAT (DAX) package, distributes computation across the Vanderbilt Advanced Computing Center for Research and Education high performance computing center. All software are made available in open source for use in combining portable batch scripting (PBS) grids and XNAT servers.