Statistical considerations for the design and analysis of the ELISpot assay in HIV-1 vaccine trials

Comprehensive epitope mapping using polyclonally expanded human CD8 T cells and a two-step ELISpot assay for testing large peptide libraries

The genetic diversity of circulating HIV-1 strains poses a major barrier to the design, development and evaluation of HIV-1 vaccines. The assessment of both vaccine- and natural infection-elicited T cell responses is commonly done with multivalent peptides that are designed to maximally capture the diversity of potential T cell epitopes (PTEs) observed in natural circulating sequences. However, depending on the sequence diversity of viral subtypes and number of the HIV immunogens under investigation, PTE estimates, including HLA-guided computational methods, can easily generate enormous peptide libraries. Evaluation of T cell epitope specificity using such extensive peptide libraries is usually limited by sample availability, even for high-throughput and robust epitope mapping techniques like ELISpot assays. Here we describe a novel, two-step protocol for in-vitro polyclonal expansion of CD8 T cells from a single vial of frozen PBMC, which facilitated the screening 441 HIV-1 Gag peptides for immune responses among 32 HIV-1 positive subjects and 40 HIV-1 negative subjects for peptide qualification. Using a pooled-peptide mapping strategy, epitopes were mapped in two sequential ELISpot assays; the first ELISpot screened 33 large peptide pools using CD8 T cells expanded for 7 days, while the second step tested pool-matrix peptides to identify individual peptides using CD8 T cells expanded for 10 days. This comprehensive epitope screening established the breadth and magnitude of HIV-1 Gag-specific CD8 T cells and further revealed the extent of immune responses to variable/polymorphic epitopes.

The immunologic dominance of an epitope within a rationally designed poly-epitope vaccine is influenced by multiple factors

INTRODUCTION

CD4⁺ T cells are essential for inducing optimal CD8⁺ T cell and antibody-producing B cell responses and maintaining their long-term immunological memory. Therefore, CD4⁺ T cells are a critical component in HIV vaccine development. Due to enormous viral gene variation and significant human host genetic diversity, HIV vaccines may need to be custom-made for different countries.

METHODS

Previously, we designed a CD4⁺ T cell vaccine based on Chinese HIV isolates and HLA-DR alleles using bioinformatics tools and predicted that 20 epitopes could cover 98.1% of the Chinese population. In vivo testing of the poly-epitope antigen in mice only activated specific T cells for some epitopes. To elucidate the mechanism of the observed differential immunogenicity, we examined poly-epitope antigen processing and presentation using in vitro and in vivo analytical methods.

RESULTS

Enzymatic digestion indicated that all 20 epitopes comprising the poly-epitope antigen could be liberated, but MHC II binding assays showed that neither binding affinity nor dissociation rate was associated with the magnitude of T cell immune responses elicited by each peptide epitope in vaccinated mice. Mass spectrometry analysis of MHC II-bound peptides suggested that the abundance of endogenously processed peptides bound to MHC II molecules was significantly associated with the relative immunodominance of these epitopes.

CONCLUSION

These results provide a new rationale for improving the design and testing of poly-epitope vaccines for HIV and other diseases.

Comprehensive definition of human immunodominant CD8 antigens in tuberculosis

Despite widespread use of the Bacillus Calmette-Guerin vaccine, tuberculosis, caused by infection with Mycobacterium tuberculosis, remains a leading cause of morbidity and mortality worldwide. As CD8⁺ T cells are critical to tuberculosis host defense and a phase 2b vaccine trial of modified vaccinia Ankara expressing Ag85a that failed to demonstrate efficacy, also failed to induce a CD8⁺ T cell response, an effective tuberculosis vaccine may need to induce CD8⁺ T cells. However, little is known about CD8, as compared to CD4, antigens in tuberculosis. Herein, we report the results of the first ever HLA allele independent genome-wide CD8 antigen discovery program. Using CD8⁺ T cells derived from humans with latent tuberculosis infection or tuberculosis and an interferon-γ ELISPOT assay, we screened a synthetic peptide library representing 10% of the Mycobacterium tuberculosis proteome, selected to be enriched for Mycobacterium tuberculosis antigens. We defined a set of immunodominant CD8 antigens including part or all of 74 Mycobacterium tuberculosis proteins, only 16 of which are previously known CD8 antigens. Immunogenicity was associated with the degree of expression of mRNA and protein. Immunodominant antigens were enriched in cell wall proteins with preferential recognition of Esx protein family members, and within proteins comprising the Mycobacterium tuberculosis secretome. A validation study of immunodominant antigens demonstrated that these antigens were strongly recognized in Mycobacterium tuberculosis-infected individuals from a tuberculosis endemic region in Africa. The tuberculosis vaccine field will likely benefit from this greatly increased known repertoire of CD8 immunodominant antigens and definition of properties of Mycobacterium tuberculosis proteins important for CD8 antigenicity.

Specific bacterial proteins have been found that drive effective immune responses to tuberculosis, with use in making more effective vaccines. Immunity to tuberculosis (TB) is facilitated by two types of white blood cell; however, most research has focused on one: the CD4⁺ T cell. Deborah A. Lewinsohn and David Lewinsohn, of the Oregon Health & Science University, USA, and collaborators lay out the essential functions of the oft-neglected CD8⁺ T cell, and undertook a broad approach to catalogue and define the bacterial proteins that activate the CD8⁺ T cell response. The team found that TB-infected humans reacted strongly to their protein library, and described several characteristics of CD8⁺ T cell ‘antigens’ (activators of immune cells) that will likely prove highly useful in the design of more protective TB vaccines.

Variability of the IFN-γ ELISpot assay in the context of proficiency testing and bridging studies

Assays that assess cellular mediated immune responses performed under Good Clinical Laboratory Practice (GCLP) guidelines are required to provide specific and reproducible results. Defined validation procedures are required to establish the Standard Operating Procedure (SOP), include pass and fail criteria, as well as implement positivity criteria. However, little to no guidance is provided on how to perform longitudinal assessment of the key reagents utilized in the assay. Through the External Quality Assurance Program Oversight Laboratory (EQAPOL), an Interferon-gamma (IFN-γ) Enzyme-linked immunosorbent spot (ELISpot) assay proficiency testing program is administered. A limit of acceptable within site variability was estimated after six rounds of proficiency testing (PT). Previously, a PT send-out specific within site variability limit was calculated based on the dispersion (variance/mean) of the nine replicate wells of data. Now an overall 'dispersion limit' for the ELISpot PT program within site variability has been calculated as a dispersion of 3.3. The utility of this metric was assessed using a control sample to calculate the within (precision) and between (accuracy) experiment variability to determine if the dispersion limit could be applied to bridging studies (studies that assess lot-to-lot variations of key reagents) for comparing the accuracy of results with new lots to results with old lots. Finally, simulations were conducted to explore how this dispersion limit could provide guidance in the number of replicate wells needed for within and between experiment variability and the appropriate donor reactivity (number of antigen-specific cells) to be used for the evaluation of new reagents. Our bridging study simulations indicate using a minimum of six replicate wells of a control donor sample with reactivity of at least 150 spot forming cells per well is optimal. To determine significant lot-to-lot variations use the 3.3 dispersion limit for between and within experiment variability.

Pooled-Peptide Epitope Mapping Strategies Are Efficient and Highly Sensitive: An Evaluation of Methods for Identifying Human T Cell Epitope Specificities in Large-Scale HIV Vaccine Efficacy Trials

The interferon gamma, enzyme-linked immunospot (IFN-γ ELISpot) assay is widely used to identify viral antigen-specific T cells is frequently employed to quantify T cell responses in HIV vaccine studies. It can be used to define T cell epitope specificities using panels of peptide antigens, but with sample and cost constraints there is a critical need to improve the efficiency of epitope mapping for large and variable pathogens. We evaluated two epitope mapping strategies, based on group testing, for their ability to identify vaccine-induced T-cells from participants in the Step HIV-1 vaccine efficacy trial, and compared the findings to an approach of assaying each peptide individually. The group testing strategies reduced the number of assays required by >7-fold without significantly altering the accuracy of T-cell breadth estimates. Assays of small pools containing 7–30 peptides were highly sensitive and effective at detecting single positive peptides as well as summating responses to multiple peptides. Also, assays with a single 15-mer peptide, containing an identified epitope, did not always elicit a response providing validation that 15-mer peptides are not optimal antigens for detecting CD8+ T cells. Our findings further validate pooling-based epitope mapping strategies, which are critical for characterizing vaccine-induced T-cell responses and more broadly for informing iterative vaccine design. We also show ways to improve their application with computational peptide:MHC binding predictors that can accurately identify the optimal epitope within a 15-mer peptide and within a pool of 15-mer peptides.

Recent developments in clinical trial designs for HIV vaccine research

HIV vaccine strategies are expected to be a crucial component for controlling the HIV epidemic. Despite the large spectrum of potential candidate vaccines for both prophylactic and therapeutic use, the overall development process of an efficacious HIV vaccine strategy is lengthy. The design of clinical trials and the progression of a candidate strategy through the different clinical development stages remain methodologically challenging, mainly due to the lack of validated correlates of protection. In this review, we describe recent advances in clinical trial designs to increase the efficiency of the clinical development of candidate HIV vaccine strategies. The methodological aspects of the designs for early- (phase I and II) and later -stage (phase IIB and III) development are discussed, taking into account the specificities of both prophylactic and therapeutic HIV vaccine development.

Generation of tumor-specific cytotoxic T-lymphocytes from the peripheral blood of colorectal cancer patients for adoptive T-cell transfer

This study designs a strategy for an adoptive cellular therapy (ACT) protocol based on the ex-vivo selection of autologous peripheral blood-derived CD8-enriched T-cells, stimulated with dendritic cells (DCs) that had been pulsed with apoptotic tumor cells to generate cytotoxic T lymphocytes (CTLs) with anti-tumor activity. Seventy-eight colorectal cancer (CRC) patients were enrolled in this study. Tumor tissues and peripheral blood (PB) were obtained at surgery. Tissues were mechanically dissociated and cultured to obtain a primary tumor cell line from each patient. DCs were derived from peripheral blood mononuclear cells (PBMCs) using magnetic positive selection of CD14+ monocytes. Anti-tumor CTLs were elicited in co-/micro-cultures using DCs as antigen-presenting cells, autologous apoptotic tumor cells as a source of antigens, and CD8+ T lymphocytes as effectors. Interferon-γ (IFN-γ) secretion was assessed by ELISpot assays to evaluate the activation of the CTLs against the autologous tumor cells. Primary tumor cell lines were obtained from 20 of 78 patients (25.6%). DCs were generated from 26 patients, and of them, corresponding tumor cell lines were derived from six patients. ELISpot results showed that significant IFN-γ secretion was detected after different numbers of stimulations for two patients, whereas weak secretion was observed for three patients. Despite difficulties due to contamination of several primary tumor cell lines with gut intestinal flora, the results suggest that the generation of tumor-specific CTLs is feasible from patients with CRC, and could be useful for supporting an ACT approach in CRC.

Normal Distribution of CD8+ T-Cell-Derived ELISPOT Counts within Replicates Justifies the Reliance on Parametric Statistics for Identifying Positive Responses

Accurate assessment of positive ELISPOT responses for low frequencies of antigen-specific T-cells is controversial. In particular, it is still unknown whether ELISPOT counts within replicate wells follow a theoretical distribution function, and thus whether high power parametric statistics can be used to discriminate between positive and negative wells. We studied experimental distributions of spot counts for up to 120 replicate wells of IFN-γ production by CD8+ T-cell responding to EBV LMP2A (426 - 434) peptide in human PBMC. The cells were tested in serial dilutions covering a wide range of average spot counts per condition, from just a few to hundreds of spots per well. Statistical analysis of the data using diagnostic Q-Q plots and the Shapiro-Wilk normality test showed that in the entire dynamic range of ELISPOT spot counts within replicate wells followed a normal distribution. This result implies that the Student t-Test and ANOVA are suited to identify positive responses. We also show experimentally that borderline responses can be reliably detected by involving more replicate wells, plating higher numbers of PBMC, addition of IL-7, or a combination of these. Furthermore, we have experimentally verified that the number of replicates needed for detection of weak responses can be calculated using parametric statistics.

Loss of long term protection with the inclusion of HIV pol to a DNA vaccine encoding gag

Traditional vaccine strategies that induce antibody responses have failed to protect against HIV infection in clinical trials, and thus cell-mediated immunity is now an additional criterion. Recent clinical trials that aimed to induce strong T cell responses failed to do so. Therefore, to enhance induction of protective T cell responses, it is crucial that the optimum antigen combination is chosen. Limited research has been performed into the number of antigens selected for an HIV vaccine. This study aimed to compare DNA vaccines encoding either a single HIV antigen or a combination of two antigens, using intradermal vaccination of C57BL/6 mice. Immune assays were performed on splenocytes, and in vivo protection was examined by challenge with a chimeric virus, EcoHIV, able to infect mouse but not human leukocytes, at 10 days (short term) and 60 days (long term) post final vaccination. At 60 days there was significantly lower frequency of induced antigen-specific CD8(+) T cells in the spleens of pCMVgag-pol-vaccinated mice compared with mice which received pCMVgag only. Most importantly, short term viral control of EcoHIV was similar for pCMVgag and pCMVgag-pol-vaccinated mice at day 10, but only the pCMVgag-vaccinated significantly controlled EcoHIV at day 60 compared with pCMV-vaccinated mice, showing that control was reduced with the inclusion of the HIV pol gene.

Statistical estimation & inference of cell counts from ELISPOT limiting dilution assays

The ELISPOT assay is often used for cell count determination in immunological studies. Automated methods are needed to estimate cell concentrations from spot counts obtained from the assay. Three major distributions are assumed for observational cell counts. For each assumed distribution, individual least squares (LS)/ maximum likelihood and/or individual robust least squares (RLS) are applied for parameter estimation. Distributions of study endpoints (derived variables), defined as percentages of antigen-specific memory cell per total immunoglobulin G (IgG), are investigated to provide a basis for hypothesis testing. We show, under some weak conditions, that the distribution of endpoint estimates across subjects is approximately the same within a group. Thus, the t -test or the Wilcoxon Rank Sum test can be applied to detect group differences. These methods are compared through simulations and application to real data.

Statistical methods for the assessment of EQAPOL proficiency testing: ELISpot, Luminex, and Flow Cytometry

In September 2011 Duke University was awarded a contract to develop the National Institutes of Health/National Institute of Allergy and Infectious Diseases (NIH/NIAID) External Quality Assurance Program Oversight Laboratory (EQAPOL). Through EQAPOL, proficiency testing programs are administered for Interferon-γ (IFN-γ) Enzyme-linked immunosorbent spot (ELISpot), Intracellular Cytokine Staining Flow Cytometry (ICS) and Luminex-based cytokine assays. One of the charges of the EQAPOL program was to apply statistical methods to determine overall site performance. We utilized various statistical methods for each program to find the most appropriate for assessing laboratory performance using the consensus average as the target value. Accuracy ranges were calculated based on Wald-type confidence intervals, exact Poisson confidence intervals, or via simulations. Given the nature of proficiency testing data, which has repeated measures within donor/sample made across several laboratories; the use of mixed effects models with alpha adjustments for multiple comparisons was also explored. Mixed effects models were found to be the most useful method to assess laboratory performance with respect to accuracy to the consensus. Model based approaches to the proficiency testing data in EQAPOL will continue to be utilized. Mixed effects models also provided a means of performing more complex analyses that would address secondary research questions regarding within and between laboratory variability as well as longitudinal analyses.

HIV-1 vaccine-induced T-cell responses cluster in epitope hotspots that differ from those induced in natural infection with HIV-1

Several recent large clinical trials evaluated HIV vaccine candidates that were based on recombinant adenovirus serotype 5 (rAd-5) vectors expressing HIV-derived antigens. These vaccines primarily elicited T-cell responses, which are known to be critical for controlling HIV infection. In the current study, we present a meta-analysis of epitope mapping data from 177 participants in three clinical trials that tested two different HIV vaccines: MRKAd-5 HIV and VRC-HIVAD014-00VP. We characterized the population-level epitope responses in these trials by generating population-based epitope maps, and also designed such maps using a large cohort of 372 naturally infected individuals. We used these maps to address several questions: (1) Are vaccine-induced responses randomly distributed across vaccine inserts, or do they cluster into immunodominant epitope hotspots? (2) Are the immunodominance patterns observed for these two vaccines in three vaccine trials different from one another? (3) Do vaccine-induced hotspots overlap with epitope hotspots induced by chronic natural infection with HIV-1? (4) Do immunodominant hotspots target evolutionarily conserved regions of the HIV genome? (5) Can epitope prediction methods be used to identify these hotspots? We found that vaccine responses clustered into epitope hotspots in all three vaccine trials and some of these hotspots were not observed in chronic natural infection. We also found significant differences between the immunodominance patterns generated in each trial, even comparing two trials that tested the same vaccine in different populations. Some of the vaccine-induced immunodominant hotspots were located in highly variable regions of the HIV genome, and this was more evident for the MRKAd-5 HIV vaccine. Finally, we found that epitope prediction methods can partially predict the location of vaccine-induced epitope hotspots. Our findings have implications for vaccine design and suggest a framework by which different vaccine candidates can be compared in early phases of evaluation.

The HIV epidemic is a major global health challenge leading to more than 1.8 million deaths annually, and despite significant efforts, the search for an efficacious and safe vaccine continues. Several candidate vaccines were designed to elicit CD8+ T-cell responses and were based on using recombinant Adenovirus serotype 5 (rAd-5) vector that expresses HIV-derived antigens. While none of these vaccines had protective effects, they provide an opportunity to study vaccine-induced T-cell responses on a population level. Here, we analyze data from the three largest epitope mapping studies performed in three clinical trials testing two rAd-5 vaccines. We find that vaccine-induced responses tend to cluster in “epitope hotspots” and that these hotspots are different for each vaccine and more surprisingly in two different vaccine trials testing the same vaccine. We also compared vaccine-induced hotspots to those elicited by natural infection and found that some of the vaccine-induced hotspots are not observed in natural infection. Finally, we show that epitope prediction methods can be useful for predicting vaccine induced hotspots based on participants HLA alleles.

Defining ELISpot cut-offs from unreplicated test and control wells

In the absence of replication of wells, empirical criteria for enzyme-linked immunospot (ELISpot) positivity use fixed differences or ratios between spot forming units (SFU) counts between test and control. We propose an alternative approach which first identifies the optimally variance-stabilizing transformation of the SFU counts, based on the Bland-Altman plot of the test and control wells. The second step is to derive a positivity threshold from the difference in between-plate distribution functions of the transformed test and control SFU counts. This method is illustrated using 1309 assay results from a cohort study of influenza in Vietnam in which some, but not all, of the peptide pools have clear tendencies for SFU counts to be higher in test than control wells.

Response determination criteria for ELISPOT: toward a standard that can be applied across laboratories

ELISPOT assay readout is often dichomized as positive or negative responses according to prespecified criteria. However, these criteria can vary widely across institutions. The adoption of a common response criterion is a key step toward cross-laboratory comparability. This chapter describes the two main approaches to response determination, identifying the strengths and limitations of each. Nonparametric statistical tests and consideration of data quality are recommended and instructions provided for their ready implementation by nonstatisticians and statisticians alike.

Statistical analysis of ELISPOT assays

Cytokine ELISPOT assays have emerged as a powerful tool for the detection of rare antigen-specific T cells in freshly isolated cell material, such as blood. While ELISPOT assays allow one to directly visualize and count extremely low frequencies of cytokine-secreting T cells among millions of nonsecreting bystander cells, the interpretation of ELISPOT data can become ambiguous when (a) spot numbers in antigen-containing wells are low, (b) spot counts in negative control wells are elevated, and particularly (c) when both of the above occur simultaneously. Thus, the primary task, even before statistics are employed, must be the optimization of the basic assay parameters and reagents such that the assay yields low background signal in the negative-control wells and the maximal number of antigen-induced spots in test wells, i.e., the signal-to-noise ratio is maximized. Furthermore, the use of proper spot-size gating parameters for data analysis is indispensable for screening out irrelevant background spots, and thus increasing the signal-to-noise ratio. The goal of most ELISPOT experiments is to identify positive T-cell responses as defined by a significantly elevated spot count in antigen-stimulated wells over the nonstimulated medium-control or negative-control antigen. In this chapter, we conclude that - with some limitations - the T-Test and related statistical methods which rely on the assumption of normal distribution are suitable for identifying positive ELISPOT results.

Response definition criteria for ELISPOT assays revisited

No consensus has been reached on how to determine if an immune response has been detected based on raw data from an ELISPOT assay. The goal of this paper is to enable investigators to understand and readily implement currently available methods for response determination. We describe empirical and statistical approaches, identifying the strengths and limitations of each approach to allow readers to rationally select and apply a scientifically sound method appropriate to their specific laboratory setting. Five representative approaches were applied to data sets from the CIMT Immunoguiding Program and the response detection and false positive rates were compared. Simulation studies were also performed to compare empirical and statistical approaches. Based on these, we recommend the use of a non-parametric statistical test. Further, we recommend that six medium control wells or four wells each for both medium control and experimental conditions be performed to increase the sensitivity in detecting a response, that replicates with large variation in spot counts be filtered out, and that positive responses arising from experimental spot counts below the estimated limit of detection be interpreted with caution. Moreover, a web-based user interface was developed to allow easy access to the recommended statistical methods. This interface allows the user to upload data from an ELISPOT assay and obtain an output file of the binary responses.

Quantifying HIV-1-specific CD8 (+) T-cell responses using ELISPOT and cytokine flow cytometry

Since the initial description and characterization of the agent that causes AIDS, human immunodeficiency virus (HIV-1), numerous research groups have characterized immune responses to this virus. Much effort has been directed towards identifying potential correlates of protection that may be useful for the development of vaccines and immunotherapies. In addition, several investigations have focused on comparing patients with rapid vs. slow disease progression profiles in an attempt to identify the characteristics of a "successful" immune response. Although many gaps remain in our understanding of the host-pathogen relationship, great progress has been made during the past 20 years in elucidating the adaptive, cell-mediated response to HIV-1. These investigations have benefited in recent years from the development of new approaches to the analysis of antigen-specific CD8+ T-cell function, notably the ELISPOT assay and cytokine flow cytometry. This chapter provides simple protocols for these two methods.

Biomarkers and assessment of vaccine responses

Vaccines for infectious diseases have in the past, and will into the future, relied on a variety of surrogate markers to monitor vaccine efficacy. The primary surrogate markers have been either the antibody titer to vaccine antigens or the measurement of antibody function such as anti-viral neutralizing activity. In recent years, the measurement of T-cell function in conjunction with or independent of antibody measurements have been used to assess vaccine efficacy. ELISPOT, flow cytometry and intra-cellular staining methods are used to determine the impact of vaccines on immune mediators such as interleukins, interferons, MHC expression and pro-inflammatory mediators. The relevant B-cell and T-cell surrogate markers for vaccine efficacy is dependent on the vaccine being used, so that no universal set of surrogate markers can be applied to all vaccines. The use of T-cell surrogate markers can be complicated by the lack of sensitivity to accurately measure intra-cellular mediators. Although typically this is not a problem for infectious disease vaccines, it is a major problem for cancer vaccines.

Peptide impurities in commercial synthetic peptides and their implications for vaccine trial assessment

The advent of T-cell assay methodologies that are amenable to high throughput coupled with the availability of large libraries of overlapping peptides have revolutionized the fields of vaccine efficacy testing and cellular immune response assessment. Since T-cell assay performance is critically dependent upon the quality and specificity of the stimulating peptides, assurance of high-quality and reliable input peptides is an important aspect of assay validation. Herein, we demonstrate that individual peptides from large human immunodeficiency virus (HIV)-based peptide library sets obtained directly from two independent custom peptide suppliers contained contaminating peptides capable of giving false-positive results, which were consistent with nominal antigen-specific CD8+ T-cell responses. In-depth investigation of the cellular response in terms of responding CD8+ T-cell frequency and human leukocyte antigen (HLA) restriction led to the conclusion that one set of HIV type 1 (HIV-1)-derived peptides was contaminated with a peptide from human cytomegalovirus (HCMV), which is commonly used in cellular immunology research applications. Analytical characterization of the original stock of the suspect HIV-1 peptide confirmed the presence of approximately 1% by weight of the HCMV peptide. These observations have critical implications for quality assurance (QA) and quality control (QC) of peptides used in clinical trials where cellular immune-based assays are important end-point determinants. We propose a simple schema of biological QA/QC protocols to augment the standard biochemical QA/QC analyses as a means to circumvent this and other problems that can affect cellular immune-based assay outcome and interpretation.

Detection of HIV vaccine-induced cell-mediated immunity in HIV-seronegative clinical trial participants using an optimized and validated enzyme-linked immunospot assay

An effective vaccine for HIV is likely to require induction of T-cell-mediated immune responses, and the interferon-gamma (IFNgamma) enzyme-linked immunospot (ELISPOT) assay has become the most commonly used assay for measuring these responses in vaccine trials. We optimized and validated the HIV ELISPOT assay using an empirical method to establish positivity criteria that results in a < or =1% false-positive rate. Using this assay, we detected a broad range of HIV-specific ELISPOT responses to peptide pools of overlapping 20mers, 15mers, or 9mers in study volunteers receiving DNA- or adenovirus vector-based HIV vaccines and in HIV-seropositive donors. We found that 15mers generally had higher response magnitudes than 20mers and lower false-positive rates than 9mers. These studies show that our validated ELISPOT assay using 15mer peptide pools and the positivity criteria of > or =55 spots per 10(6) cells and > or =4-fold over mock (negative control) is a sensitive and specific assay for the detection of HIV vaccine-induced cell-mediated immunity.

Defining blood processing parameters for optimal detection of cryopreserved antigen-specific responses for HIV vaccine trials

Interferon-gamma (IFN-gamma) ELISpot and intracellular cytokine staining (ICS) assays are routinely employed in clinical HIV vaccine trials to identify antigen-specific T cells in cryopreserved peripheral blood mononuclear cells (PBMC). Several parameters involved in blood collection, processing and shipping may influence immunological function of the resulting cells, including anticoagulant type, time from venipuncture to PBMC isolation/cryopreservation, method of PBMC isolation and procedure for sample shipping. We examined these parameters in single and multiple site studies, and found the length of time from venipuncture to cryopreservation is the most important parameter affecting performance of T cells in immunological assays. Comparing blood processed at 24 h after venipuncture with that processed within 8 h, we observed on average a modest reduction in PBMC viability ( approximately 8% decrease), a greater loss in cell recovery ( approximately 32%), and between 36-56% loss in IFN-gamma T cell frequencies by ELISpot assay. We also describe three cold shipping methods that maintain immunological function in appropriately cryopreserved PBMC. These data indicate that cryopreservation of PBMC should occur within 8 h of venipuncture for optimal performance. This narrow window for specimen processing has important implications in selecting and monitoring clinical sites with laboratory capacity to perform these procedures in future clinical trials.

Phenotypic and Functional Analysis of Immune CD8+ T Cell Responses Induced by a Single Injection of a HIV DNA Vaccine in Mice

HIV DNA vaccines are potent inducers of cell-mediated immune (CMI) response in mice but elicit poor HIV-specific IFN-gamma-producing T cells in monkeys and humans. In this study, we performed kinetic analyses on splenocytes of BALB/c mice that were immunized by a single injection with a unique DNA vaccine. Using IFN-gamma-ELISPOT and multiparametric FACS analysis, we characterized the induced CMI response. We found that the response was detectable for at least 63 wk. ELISPOT detection of IFN-gamma-producing T cells showed a profile with two waves separated by a long period of minimal response. Multiparametric FACS analysis showed two populations of CD3(+)CD8(+) T cells that were specific for all HIV Ags. These cells had similar robust proliferation abilities and contained granzyme B. However, only a few produced IFN-gamma. Both IFN-gamma-producing and non-IFN-gamma-producing HIV-specific CD8(+) T cells were detected in the early stage (week (W)1 and W2 postimmunization (PI)), in the prolonged intermediate period of minimal response (W4-W26 PI), and in the final late phase of increased response (W30-W63 PI). Our longitudinal characterization showed that both subsets of cells underwent expansion, contraction, and memory generation/maintenance phases throughout the lifespan of the animal. Altogether, these findings bring insight to the heterogeneity of the immune T cell response induced by a single immunization with this DNA and strengthen the concept that used of the IFN-gamma-ELISPOT assay alone may be insufficient to detect critical T cell responses to candidate HIV vaccines.

Phase I Study of Abagovomab in Patients with Epithelial Ovarian, Fallopian Tube, or Primary Peritoneal Cancer

PURPOSE

This open-label study assessed the safety and immunogenicity of two doses and two routes of the anti-idiotypic monoclonal antibody abagovomab (formerly ACA125) in patients with epithelial ovarian, fallopian tube, or primary peritoneal cancer.

EXPERIMENTAL DESIGN

Eligible patients from the three participating institutions were any stage at diagnosis, had relapsed, and had complete or partial response to additional chemotherapy. Patients were randomized to receive abagovomab at 2.0 versus 0.2 mg and i.m. versus s.c. for four immunizations every 2 weeks and then monthly for two additional immunizations. Planned evaluation included interval physical examinations and laboratory assessments with immune assessment, including HLA typing, human anti-mouse antibody, ELISA, and enzyme-linked immunospot. Patients were required to remain on study until week 10 (the first post-baseline Ab3 determination) to be considered for immunologic assessment. The primary end points were safety and immunogenicity primarily determined by Ab3 response.

RESULTS

Forty-two patients received at least one vaccination and were eligible for safety analysis. Thirty-three patients were available for Ab3 analysis (removed for progression of disease, 6; withdrawal of consent, 2; unrelated adverse event, 1). The most common adverse events were self-limited pain at injection site, myalgia, and fever. No hematologic or nonhematologic toxicity grade>2 related to immunization was seen. Ab3 was detectable in all patients (median, 236,794 ng/mL); none of route of administration (P=0.6268), dose (P=0.4602), or cohort (P=0.4944) was statistically significant in terms of effect on maximum post-baseline Ab3 titer. Human anti-mouse antibody was not detectable at baseline but was present in all patients at week 16 (range, 488-45,000 ng/mL).

CONCLUSIONS

Immunization with abagovomab is well tolerated and induced robust Ab3 responses at the two doses and routes tested. A phase III randomized study with abagovomab (2.0 mg s.c.) is warranted.

The automated counting of spots for the ELISpot assay

An automated method for counting spot-forming units in the ELISpot assay is described that uses a statistical model fit to training data that is based on counts from one or more experts. The method adapts to variable background intensities and provides considerable flexibility with respect to what image features can be used to model expert counts. Point estimates of spot counts are produced together with intervals that reflect the degree of uncertainty in the count. Finally, the approach is completely transparent and "open source" in contrast to methods embedded in current commercial software. An illustrative application to data from a study of the reactivity of T-cells from healthy human subjects to a pool of immunodominant peptides from CMV, EBV and flu is presented.

Statistical positivity criteria for the analysis of ELISpot assay data in HIV-1 vaccine trials

The enzyme-linked immunospot (ELISpot) assay is one of the leading technologies used to measure cellular responses in many settings including HIV vaccine clinical trials. HIV-1-specific effector T lymphocyte responses are considered necessary in the control of infection. Accurate measurement and summary of cellular immune responses are critical to HIV research. ELISpot assay readout is often dichotomized into positive/negative responses according to some pre-specified criteria. Given the increase in the number of replicates used for each stimulation with current assay configurations in the HIV Vaccine Trials Network (HVTN) laboratories, a new approach is now possible. We propose an objective criteria based on a hypothesis-driven method that controls the false positive rate while maintaining sensitivity to each of the antigen-specific responses under study. The new approach is compared to other commonly employed criteria using real and simulated data.

Evaluation of the interlaboratory concordance in quantification of human immunodeficiency virus-specific T cells with a gamma interferon enzyme-linked immunospot assay

The gamma interferon (IFN-gamma) enzyme-linked immunospot (ELISPOT) assay is a reference method for the ex vivo monitoring of antigen-specific T cells and a primary tool for assessing clinical trials of human immunodeficiency virus (HIV) or cancer vaccines. Four experienced laboratories in Paris compared their results with this method by exchanging frozen blood samples from eight HIV-seronegative and eight HIV-seropositive subjects. Each laboratory measured the IFN-gamma-producing cells specific for HIV, Epstein-Barr virus, cytomegalovirus, and influenza using the same set of peptides and the same ELISPOT reader but its own ELISPOT technique. The cutoff values for positive responses (50 or 100 spot-forming cells/10(6) peripheral blood mononuclear cells over background) were consistent with the binomial statistic criterion. The global qualitative concordance, as assessed by the kappa index, ranged from 0.38 to 0.92, that is, moderate to excellent, and was better for non-HIV 9-mer peptide pools than for HIV 15-mer peptide pools. The interlaboratory coefficient of variation for the frequency of virus-specific T cells was 18.7% (data are expressed on a log scale). Clustering analysis of HIV-positive subjects showed qualitative agreement for ELISPOT results from all four laboratories. Overall, the good interlaboratory qualitative concordance of IFN-gamma ELISPOT assays with only the peptide source and ELISPOT reader in common suggests that a qualitative comparison of interlaboratory findings is feasible. Nonetheless, a single set of standard operating procedures should be used in multicenter trials to improve standardization.

Excellent safety and tolerability of the human immunodeficiency virus type 1 pGA2/JS2 plasmid DNA priming vector vaccine in HIV type 1 uninfected adults

A vaccine consisting of DNA priming followed by recombinant modified vaccinia Ankara (rMVA) boosting has achieved long-term control of a pathogenic challenge with a chimera of simian and human immunodeficiency viruses (SHIV-89.6P) in rhesus macaques. Based on these results, clade B HIV-1 DNA and rMVA immunogens have been developed for trials in humans. We conducted a first-time in humans phase I safety trial using the pGA2/JS2 (JS2) HIV-1 DNA priming vector expressing Gag, Pol, Env, Tat, Rev, and Vpu. Thirty HIV-uninfected adults were vaccinated with 0.3 or 3 mg of JS2 DNA, or a saline placebo, by intramuscular injection at months 0 and 2. Both doses of DNA were safe and well-tolerated with no differences between the control, 0.3 mg, or 3 mg groups (n = 6, 12, and 12, respectively) through 12 months of postvaccination follow- up. A chromium-release assay using fresh peripheral blood mononuclear cells (PBMCs) and a validated IFN-gamma ELISpot assay with frozen PBMCs failed to detect CD4(+) or CD8(+) HIV-1-specific T cell responses. HIV-specific neutralizing antibodies were also not detected. The vaccine is being further developed as a priming vector for a combined DNA plus rMVA prime/boost HIV vaccination regimen.

Results of an ELISPOT proficiency panel conducted in 11 laboratories participating in international human immunodeficiency virus type 1 vaccine trials

We used an external quality assurance (EQA) panel to assess laboratory competency and comparability when performing ELISPOT assays in support of human immunodeficiency virus type 1 (HIV-1) vaccine trials. Cell recovery, viability, and frequency of interferon-gamma (IFN-gamma)-secreting cells after antigen stimulation were obtained from 11 laboratories on a coded panel of 11 peripheral blood mononuclear cell samples. The median recovery and viability before plating for all samples were 35% and 86%, respectively, with notable interlaboratory and intrasample variability. Empirical as well as statistical analysis methods were used to define positive ELISPOT responses. Remarkable concordance between laboratories was obtained in defining a qualitative assessment of responder/nonresponder status to antigens, but the frequency of responding cells varied among the laboratories. This study highlights the need for better standardization of protocols and reagents to obtain reliable and reproducible data that may support immunogenicity studies, vaccine regulatory submissions, and licensure.