Development of an automated analysis system for data from flow cytometric intracellular cytokine staining assays from clinical vaccine trials

Eight practices for data management to enable team data science

INTRODUCTION

In clinical and translational research, data science is often and fortuitously integrated with data collection. This contrasts to the typical position of data scientists in other settings, where they are isolated from data collectors. Because of this, effective use of data science techniques to resolve translational questions requires innovation in the organization and management of these data.

METHODS

We propose an operational framework that respects this important difference in how research teams are organized. To maximize the accuracy and speed of the clinical and translational data science enterprise under this framework, we define a set of eight best practices for data management.

RESULTS

In our own work at the University of Rochester, we have strived to utilize these practices in a customized version of the open source LabKey platform for integrated data management and collaboration. We have applied this platform to cohorts that longitudinally track multidomain data from over 3000 subjects.

CONCLUSIONS

We argue that this has made analytical datasets more readily available and lowered the bar to interdisciplinary collaboration, enabling a team-based data science that is unique to the clinical and translational setting.

High-throughput flow cytometry data normalization for clinical trials

Flow cytometry datasets from clinical trials generate very large datasets and are usually highly standardized, focusing on endpoints that are well defined apriori. Staining variability of individual makers is not uncommon and complicates manual gating, requiring the analyst to adapt gates for each sample, which is unwieldy for large datasets. It can lead to unreliable measurements, especially if a template-gating approach is used without further correction to the gates. In this article, a computational framework is presented for normalizing the fluorescence intensity of multiple markers in specific cell populations across samples that is suitable for high-throughput processing of large clinical trial datasets. Previous approaches to normalization have been global and applied to all cells or data with debris removed. They provided no mechanism to handle specific cell subsets. This approach integrates tightly with the gating process so that normalization is performed during gating and is local to the specific cell subsets exhibiting variability. This improves peak alignment and the performance of the algorithm. The performance of this algorithm is demonstrated on two clinical trial datasets from the HIV Vaccine Trials Network (HVTN) and the Immune Tolerance Network (ITN). In the ITN data set we show that local normalization combined with template gating can account for sample-to-sample variability as effectively as manual gating. In the HVTN dataset, it is shown that local normalization mitigates false-positive vaccine response calls in an intracellular cytokine staining assay. In both datasets, local normalization performs better than global normalization. The normalization framework allows the use of template gates even in the presence of sample-to-sample staining variability, mitigates the subjectivity and bias of manual gating, and decreases the time necessary to analyze large datasets.

Automatic B cell lymphoma detection using flow cytometry data

Background

Flow cytometry has been widely used for the diagnosis of various hematopoietic diseases. Although there have been advances in the number of biomarkers that can be analyzed simultaneously and technologies that enable fast performance, the diagnostic data are still interpreted by a manual gating strategy. The process is labor-intensive, time-consuming, and subject to human error.

Results

We used 80 sets of flow cytometry data from 44 healthy donors, 21 patients with chronic lymphocytic leukemia (CLL), and 15 patients with follicular lymphoma (FL). Approximately 15% of data from each group were used to build the profiles. Our approach was able to successfully identify 36/37 healthy donor cases, 18/18 CLL cases, and 12/13 FL cases.

Conclusions

This proof-of-concept study demonstrated that an automated diagnosis of CLL and FL can be obtained by examining the cell capture rates of a test case using the computational method based on the multi-profile detection algorithm. The testing phase of our system is efficient and can facilitate diagnosis of B-lymphocyte neoplasms.

Quality control, analysis and secure sharing of Luminex® immunoassay data using the open source LabKey Server platform

BACKGROUND

Immunoassays that employ multiplexed bead arrays produce high information content per sample. Such assays are now frequently used to evaluate humoral responses in clinical trials. Integrated software is needed for the analysis, quality control, and secure sharing of the high volume of data produced by such multiplexed assays. Software that facilitates data exchange and provides flexibility to perform customized analyses (including multiple curve fits and visualizations of assay performance over time) could increase scientists' capacity to use these immunoassays to evaluate human clinical trials.

RESULTS

The HIV Vaccine Trials Network and the Statistical Center for HIV/AIDS Research and Prevention collaborated with LabKey Software to enhance the open source LabKey Server platform to facilitate workflows for multiplexed bead assays. This system now supports the management, analysis, quality control, and secure sharing of data from multiplexed immunoassays that leverage Luminex xMAP® technology. These assays may be custom or kit-based. Newly added features enable labs to: (i) import run data from spreadsheets output by Bio-Plex Manager™ software; (ii) customize data processing, curve fits, and algorithms through scripts written in common languages, such as R; (iii) select script-defined calculation options through a graphical user interface; (iv) collect custom metadata for each titration, analyte, run and batch of runs; (v) calculate dose-response curves for titrations; (vi) interpolate unknown concentrations from curves for titrated standards; (vii) flag run data for exclusion from analysis; (viii) track quality control metrics across runs using Levey-Jennings plots; and (ix) automatically flag outliers based on expected values. Existing system features allow researchers to analyze, integrate, visualize, export and securely share their data, as well as to construct custom user interfaces and workflows.

CONCLUSIONS

Unlike other tools tailored for Luminex immunoassays, LabKey Server allows labs to customize their Luminex analyses using scripting while still presenting users with a single, graphical interface for processing and analyzing data. The LabKey Server system also stands out among Luminex tools for enabling smooth, secure transfer of data, quality control information, and analyses between collaborators. LabKey Server and its Luminex features are freely available as open source software at http://www.labkey.com under the Apache 2.0 license.

The average IFN-γ secreting capacity of specific CD8(+) T cells is compromised while increasing copies of a single T cell epitope encoded by DNA vaccine

Previous studies suggested that both the frequency and the mean fluorescence intensity (MFI) of cytokine secreting T cells could be of great value for immunogenicity evaluation of a vaccine. In this study, by constructing epitope-based DNA vaccines encoding a previously identified CD8⁺ T cell epitope, we investigated the influence of multiplying epitope copies on both the frequency and the MFI of specific IFN-γ secreting CD8⁺ T cells. We found that frequencies of specific CD8⁺ T cell could be improved by multiplying epitope copies, while the MFI of IFN-γ secreted by epitope-specific CD8⁺ T cells decreased synchronously. And further analysis showed that the decrease of MFI was not caused by the functional avidity variation of CD8⁺ T cell receptor.

QUAliFiER: an automated pipeline for quality assessment of gated flow cytometry data

Background

Effective quality assessment is an important part of any high-throughput flow cytometry data analysis pipeline, especially when considering the complex designs of the typical flow experiments applied in clinical trials. Technical issues like instrument variation, problematic antibody staining, or reagent lot changes can lead to biases in the extracted cell subpopulation statistics. These biases can manifest themselves in non–obvious ways that can be difficult to detect without leveraging information about the study design or other experimental metadata. Consequently, a systematic and integrated approach to quality assessment of flow cytometry data is necessary to effectively identify technical errors that impact multiple samples over time. Gated cell populations and their statistics must be monitored within the context of the experimental run, assay, and the overall study.

Results

We have developed two new packages, flowWorkspace and QUAliFiER to construct a pipeline for quality assessment of gated flow cytometry data. flowWorkspace makes manually gated data accessible to BioConductor’s computational flow tools by importing pre–processed and gated data from the widely used manual gating tool, FlowJo (Tree Star Inc, Ashland OR). The QUAliFiER package takes advantage of the manual gates to perform an extensive series of statistical quality assessment checks on the gated cell sub–populations while taking into account the structure of the data and the study design to monitor the consistency of population statistics across staining panels, subject, aliquots, channels, or other experimental variables. QUAliFiER implements SVG–based interactive visualization methods, allowing investigators to examine quality assessment results across different views of the data, and it has a flexible interface allowing users to tailor quality checks and outlier detection routines to suit their data analysis needs.

Conclusion

We present a pipeline constructed from two new R packages for importing manually gated flow cytometry data and performing flexible and robust quality assessment checks. The pipeline addresses the increasing demand for tools capable of performing quality checks on large flow data sets generated in typical clinical trials. The QUAliFiER tool objectively, efficiently, and reproducibly identifies outlier samples in an automated manner by monitoring cell population statistics from gated or ungated flow data conditioned on experiment–level metadata.

Computational analysis of high-throughput flow cytometry data

INTRODUCTION

Flow cytometry has been around for over 40 years, but only recently has the opportunity arisen to move into the high-throughput domain. The technology is now available and is highly competitive with imaging tools under the right conditions. Flow cytometry has, however, been a technology that has focused on its unique ability to study single cells and appropriate analytical tools are readily available to handle this traditional role of the technology.

AREAS COVERED

Expansion of flow cytometry to a high-throughput (HT) and high-content technology requires both advances in hardware and analytical tools. The historical perspective of flow cytometry operation as well as how the field has changed and what the key changes have been discussed. The authors provide a background and compelling arguments for moving toward HT flow, where there are many innovative opportunities. With alternative approaches now available for flow cytometry, there will be a considerable number of new applications. These opportunities show strong capability for drug screening and functional studies with cells in suspension.

EXPERT OPINION

There is no doubt that HT flow is a rich technology awaiting acceptance by the pharmaceutical community. It can provide a powerful phenotypic analytical toolset that has the capacity to change many current approaches to HT screening. The previous restrictions on the technology, based on its reduced capacity for sample throughput, are no longer a major issue. Overcoming this barrier has transformed a mature technology into one that can focus on systems biology questions not previously considered possible.

DNA and modified vaccinia virus Ankara vaccines encoding multiple cytotoxic and helper T-lymphocyte epitopes of human immunodeficiency virus type 1 (HIV-1) are safe but weakly immunogenic in HIV-1-uninfected, vaccinia virus-naive adults

We evaluated a DNA plasmid-vectored vaccine and a recombinant modified vaccinia virus Ankara vaccine (MVA-mBN32), each encoding cytotoxic and helper T-lymphocyte epitopes of human immunodeficiency virus type 1 (HIV-1) in a randomized, double-blinded, placebo-controlled trial in 36 HIV-1-uninfected adults using a heterologous prime-boost schedule. HIV-1-specific cellular immune responses, measured as interleukin-2 and/or gamma interferon production, were induced in 1 (4%) of 28 subjects after the first MVA-mBN32 immunization and in 3 (12%) of 25 subjects after the second MVA-mBN32 immunization. Among these responders, polyfunctional T-cell responses, including the production of tumor necrosis factor alpha and perforin, were detected. Vaccinia virus-specific antibodies were induced to the MVA vector in 27 (93%) of 29 and 26 (93%) of 28 subjects after the first and second immunizations with MVA-mBN32. These peptide-based vaccines were safe but were ineffective at inducing HIV-1-specific immune responses and induced much weaker responses than MVA vaccines expressing the entire open reading frames of HIV-1 proteins.

Timing of plasmid cytokine (IL-2/Ig) administration affects HIV-1 vaccine immunogenicity in HIV-seronegative subjects

BACKGROUND

To investigate the potential immunostimulatory effect of interleukin (IL) 2 as a human immunodeficiency virus type 1 (HIV-1) vaccine adjuvant, we conducted a study of a plasmid coding for a fusion protein of IL-2 and immunoglobulin (IL-2/Ig).

METHODS

This phase I trial evaluated an HIV-1 DNA vaccine with the plasmid cytokine adjuvant (IL-2/Ig) in 70 HIV-negative adults. Subjects received placebo (group C), adjuvant alone (group A), vaccine alone (group D), increasing doses of adjuvant concurrent with vaccine (groups T1-T4), or adjuvant given 2 days after vaccine (group T5).

RESULTS

No significant differences in adverse events were observed between treatment groups. Cellular immune responses to envelope protein EnvA peptides were detected by interferon (IFN) γ and IL-2 enzyme-linked immunospot (ELISPOT) assays in 50% and 40% of subjects, respectively, in T4, and in 100% and 80% in T5. The median responses for groups T4 and T5, respectively, were 90 and 193 spot-forming cells (SFCs)/10⁶ peripheral blood mononuclear cells (P = .004; T4 vs T5) for the IL-2 ELISPOT assay and 103 and 380 SFCs/10⁶ PBMCs (P = .003; T4 vs T5) for the IFN-γ ELISPOT assay. A trend to more durable cellular immune responses in T5 was observed at 1 year (T5 vs T4/D; P = .07). Higher anti-Env antibody responses were detected with T5 than with T4.

CONCLUSIONS

Plasmid IL-2/Ig significantly increased immune responses when administered 2 days after the DNA vaccine, compared with simultaneous administration. These observations have important implications for the development of cytokine augmentation strategies.

CLINICAL TRIALS REGISTRATION

NCT00069030.

Safety and immunogenicity of an HIV adenoviral vector boost after DNA plasmid vaccine prime by route of administration: a randomized clinical trial

BACKGROUND

In the development of HIV vaccines, improving immunogenicity while maintaining safety is critical. Route of administration can be an important factor.

METHODOLOGY/PRINCIPAL FINDINGS

This multicenter, open-label, randomized trial, HVTN 069, compared routes of administration on safety and immunogenicity of a DNA vaccine prime given intramuscularly at 0, 1 and 2 months and a recombinant replication-defective adenovirus type 5 (rAd5) vaccine boost given at 6 months by intramuscular (IM), intradermal (ID), or subcutaneous (SC) route. Randomization was computer-generated by a central data management center; participants and staff were not blinded to group assignment. The outcomes were vaccine reactogenicity and humoral and cellular immunogenicity. Ninety healthy, HIV-1 uninfected adults in the US and Peru, aged 18-50 were enrolled and randomized. Due to the results of the Step Study, injections with rAd5 vaccine were halted; thus 61 received the booster dose of rAd5 vaccine (IM: 20; ID:21; SC:20). After the rAd5 boost, significant differences by study arm were found in severity of headache, pain and erythema/induration. Immune responses (binding and neutralizing antibodies, IFN-γ ELISpot HIV-specific responses and CD4+ and CD8+ T-cell responses by ICS) at four weeks after the rAd5 booster were not significantly different by administration route of the rAd5 vaccine boost (Binding antibody responses: IM: 66.7%; ID: 70.0%; SC: 77.8%; neutralizing antibody responses: IM: 11.1%; ID: 0.0%; SC 16.7%; ELISpot responses: IM: 46.7%; ID: 35.3%; SC: 44.4%; CD4+ T-cell responses: IM: 29.4%; ID: 20.0%; SC: 35.3%; CD8+ T-cell responses: IM: 29.4%; ID: 16.7%; SC: 50.0%.)

CONCLUSIONS/SIGNIFICANCE

This study was limited by the reduced sample size. The higher frequency of local reactions after ID and SC administration and the lack of sufficient evidence to show that there were any differences in immunogenicity by route of administration do not support changing route of administration for the rAd5 boost.

TRIAL REGISTRATION

ClinicalTrials.gov NCT00384787.

HIV-DNA priming alters T cell responses to HIV-adenovirus vaccine even when responses to DNA are undetectable

Many candidate HIV vaccines are designed to primarily elicit T cell responses. Although repeated immunization with the same vaccine boosts Ab responses, the benefit for T cell responses is ill defined. We compared two immunization regimens that include the same recombinant adenoviral serotype 5 (rAd5) boost. Repeated homologous rAd5 immunization fails to increase T cell responses, but increases gp140 Ab responses 10-fold. DNA prime, as compared with rAd5 prime, directs long-term memory CD8(+) T cells toward a terminally differentiated effector memory phenotype with cytotoxic potential. Based on the kinetics of activated cells measured directly ex vivo, the DNA vaccination primes for both CD4(+) and CD8(+) T cells, despite the lack of detection of the latter until after the boost. These results suggest that heterologous prime-boost combinations have distinct immunological advantages over homologous prime-boosts and suggest that the effect of DNA on subsequent boosting may not be easily detectable directly after the DNA vaccination.

LabKey Server: an open source platform for scientific data integration, analysis and collaboration

BACKGROUND

Broad-based collaborations are becoming increasingly common among disease researchers. For example, the Global HIV Enterprise has united cross-disciplinary consortia to speed progress towards HIV vaccines through coordinated research across the boundaries of institutions, continents and specialties. New, end-to-end software tools for data and specimen management are necessary to achieve the ambitious goals of such alliances. These tools must enable researchers to organize and integrate heterogeneous data early in the discovery process, standardize processes, gain new insights into pooled data and collaborate securely.

RESULTS

To meet these needs, we enhanced the LabKey Server platform, formerly known as CPAS. This freely available, open source software is maintained by professional engineers who use commercially proven practices for software development and maintenance. Recent enhancements support: (i) Submitting specimens requests across collaborating organizations (ii) Graphically defining new experimental data types, metadata and wizards for data collection (iii) Transitioning experimental results from a multiplicity of spreadsheets to custom tables in a shared database (iv) Securely organizing, integrating, analyzing, visualizing and sharing diverse data types, from clinical records to specimens to complex assays (v) Interacting dynamically with external data sources (vi) Tracking study participants and cohorts over time (vii) Developing custom interfaces using client libraries (viii) Authoring custom visualizations in a built-in R scripting environment. Diverse research organizations have adopted and adapted LabKey Server, including consortia within the Global HIV Enterprise. Atlas is an installation of LabKey Server that has been tailored to serve these consortia. It is in production use and demonstrates the core capabilities of LabKey Server. Atlas now has over 2,800 active user accounts originating from approximately 36 countries and 350 organizations. It tracks roughly 27,000 assay runs, 860,000 specimen vials and 1,300,000 vial transfers.

CONCLUSIONS

Sharing data, analysis tools and infrastructure can speed the efforts of large research consortia by enhancing efficiency and enabling new insights. The Atlas installation of LabKey Server demonstrates the utility of the LabKey platform for collaborative research. Stable, supported builds of LabKey Server are freely available for download at http://www.labkey.org. Documentation and source code are available under the Apache License 2.0.

Correlation analysis of intracellular and secreted cytokines via the generalized integrated mean fluorescence intensity

The immune response in humans is usually assessed using immunogenicity assays to provide biomarkers as correlates of protection (CoP). Flow cytometry is the assay of choice to measure intracellular cytokine staining (ICS) of cell-mediated immune (CMI) biomarkers. For CMI analysis, the integrated mean fluorescence intensity (iMFI) was introduced as a metric to represent the total functional CMI response as a CoP. iMFI is computed by multiplying the relative frequency (percent positive) of cells expressing a particular cytokine with the MFI of that population, and correlates better with protection in challenge models than either the percentage or the MFI of the cytokine-positive population. While determination of the iMFI as a CoP can readily be accomplished in animal models that allow challenge/protection experiments, this is not feasible in humans for ethical reasons. As a first step toward extending the iMFI concept to humans, we investigated the correlation of the iMFI derived from a human innate immune response ICS assay with functional cytokine release into the culture supernatant, as innate cytokines need to be released to have a functional impact. Next, we developed a quantitatively more correlative mathematical approach for calculating the functional response of cytokine-producing cells by incorporating the assignment of different weights to the magnitude (frequency of cytokine-positive cells) and the quality (the MFI) of the observed innate immune response. We refer to this model as generalized iMFI.

Data reduction for spectral clustering to analyze high throughput flow cytometry data

Background

Recent biological discoveries have shown that clustering large datasets is essential for better understanding biology in many areas. Spectral clustering in particular has proven to be a powerful tool amenable for many applications. However, it cannot be directly applied to large datasets due to time and memory limitations. To address this issue, we have modified spectral clustering by adding an information preserving sampling procedure and applying a post-processing stage. We call this entire algorithm SamSPECTRAL.

Results

We tested our algorithm on flow cytometry data as an example of large, multidimensional data containing potentially hundreds of thousands of data points (i.e., "events" in flow cytometry, typically corresponding to cells). Compared to two state of the art model-based flow cytometry clustering methods, SamSPECTRAL demonstrates significant advantages in proper identification of populations with non-elliptical shapes, low density populations close to dense ones, minor subpopulations of a major population and rare populations.

Conclusions

This work is the first successful attempt to apply spectral methodology on flow cytometry data. An implementation of our algorithm as an R package is freely available through BioConductor.

Flow cytometry and the future of vaccine development

Vaccine research increasingly aims to understand the fundamental mechanisms of protection afforded by licensed and candidate vaccines. Historically, nearly all licensed vaccines have relied on measures of humoral immunity to provide correlates of protection, but cellular immunity is important for protection afforded by some vaccines and will be required for vaccines against TB and malaria. Common means of assessing vaccine-induced immune responses include measuring the frequency and functions of antigen-specific lymphocytes. While diverse assays can provide this information, flow cytometry is unique in its ability to simultaneously report other features of antigen-specific cellular responses. Here, we review the application of flow cytometry to characterizing three areas of immune responses to vaccines or diseases. First, analysis of cellular (T-cell) responses is more mature: polychromatic flow cytometric analysis of T-cell function has already yielded important insight into correlates of protection. Second, antibody and antigen-specific B-cell detection by flow cytometry are being actively developed; to date, these assays are not yet widely used. Finally, flow cytometry can also be used to analyze the contribution of innate immunity to vaccine efficacy and disease pathogenesis.

HIV-1 vaccine-induced immunity in the test-of-concept Step Study: a case-cohort analysis

BACKGROUND

In the Step Study, the MRKAd5 HIV-1 gag/pol/nef vaccine did not reduce plasma viraemia after infection, and HIV-1 incidence was higher in vaccine-treated than in placebo-treated men with pre-existing adenovirus serotype 5 (Ad5) immunity. We assessed vaccine-induced immunity and its potential contributions to infection risk.

METHODS

To assess immunogenicity, we characterised HIV-specific T cells ex vivo with validated interferon-gamma ELISPOT and intracellular cytokine staining assays, using a case-cohort design. To establish effects of vaccine and pre-existing Ad5 immunity on infection risk, we undertook flow cytometric studies to measure Ad5-specific T cells and circulating activated (Ki-67+/BcL-2(lo)) CD4+ T cells expressing CCR5.

FINDINGS

We detected interferon-gamma-secreting HIV-specific T cells (range 163/10(6) to 686/10(6) peripheral blood mononuclear cells) ex vivo by ELISPOT in 77% (258/354) of people receiving vaccine; 218 of 354 (62%) recognised two to three HIV proteins. We identified HIV-specific CD4+ T cells by intracellular cytokine staining in 58 of 142 (41%) people. In those with reactive CD4+ T cells, the median percentage of CD4+ T cells expressing interleukin 2 was 88%, and the median co-expression of interferon gamma or tumor necrosis factor alpha (TNFalpha), or both, was 72%. We noted HIV-specific CD8+ T cells (range 0.4-1.0%) in 117 of 160 (73%) participants, expressing predominantly either interferon gamma alone or with TNFalpha. Vaccine-induced HIV-specific immunity, including response rate, magnitude, and cytokine profile, did not differ between vaccinated male cases (before infection) and non-cases. Ad5-specific T cells were lower in cases than in non-cases in several subgroup analyses. The percentage of circulating Ki-67+BcL-2(lo)/CCR5+CD4+ T cells did not differ between cases and non-cases.

INTERPRETATION

Consistent with previous trials, the MRKAd5 HIV-1 gag/pol/nef vaccine was highly immunogenic for inducing HIV-specific CD8+ T cells. Our findings suggest that future candidate vaccines have to elicit responses that either exceed in magnitude or differ in breadth or function from those recorded in this trial.