Data analysis as a source of variability of the HLA-peptide multimer assay: from manual gating to automated recognition of cell clusters

Proficiency tests to evaluate the impact on assay outcomes of harmonized influenza-specific Intracellular Cytokine Staining (ICS) and IFN-ɣ Enzyme-Linked ImmunoSpot (ELISpot) protocols

The magnitude and quality of cell-mediated immune responses elicited by natural infection or vaccination are commonly measured by Interferon-ɣ (IFN-ɣ) Enzyme-Linked ImmunoSpot (ELISpot) and Intracellular Cytokine Staining (ICS). To date, laboratories apply a variety of in-house procedures which leads to diverging results, complicates interlaboratory comparisons and hampers vaccine evaluations. During the FLUCOP project, efforts have been made to develop harmonized Standard Operating Procedures (SOPs) for influenza-specific IFN-ɣ ELISpot and ICS assays. Exploratory pilot studies provided information about the interlaboratory variation before harmonization efforts were initiated. Here we report the results of two proficiency tests organized to evaluate the impact of the harmonization effort on assay results and the performance of participating FLUCOP partners. The introduction of the IFN-ɣ ELISpot SOP reduced variation of both background and stimulated responses. Post-harmonization background responses were all lower than an arbitrary threshold of 50 SFU/million cells. When stimulated with A/California and B/Phuket, a statistically significant reduction in variation (p < 0.0001) was observed and CV values were strongly reduced, from 148% to 77% for A/California and from 126% to 73% for B/Phuket. The harmonizing effect of applying an ICS SOP was also confirmed by an increased homogeneity of data obtained by the individual labs. The application of acceptance criteria on cell viability and background responses further enhanced the data homogeneity. Finally, as the same set of samples was analyzed by both the IFN-ɣ ELISpot and the ICS assays, a method comparison was performed. A clear correlation between the two methods was observed, but they cannot be considered interchangeable. In conclusion, proficiency tests show that a limited harmonization effort consisting of the introduction of SOPs and the use of the same in vitro stimulating antigens leads to a reduction of the interlaboratory variation of IFN-ɣ ELISpot data and demonstrate that substantial improvements for the ICS assay are achieved as comparable laboratory datasets could be generated. Additional steps to further reduce the interlaboratory variation of ICS data can consist of standardized gating templates and detailed data reporting instructions as well as further efforts to harmonize reagent and instrument use.

A concise review of flow cytometric methods for minimal residual disease assessment in childhood B-cell precursor acute lymphoblastic leukemia

Minimal residual disease refers to a leukemia cell population that is resistant to chemotherapy or radiotherapy and leads to disease relapse. The assessment of MRD is crucial for making an accurate prognosis of the disease and for the choice of optimal treatment strategy. Here, we review the advantages and disadvantages of the available genetic and phenotypic methods and focus on the multiparametric flow cytometry as a promising method with greater sensitivity, speed, and standardization options. In addition, we discuss how the application of automated data analysis outweighs the use of complex combinations of windows and gates in classical analysis, thus eliminating subjective evaluation.

Unsupervised Analysis of Flow Cytometry Data in a Clinical Setting Captures Cell Diversity and Allows Population Discovery

Data obtained with cytometry are increasingly complex and their interrogation impacts the type and quality of knowledge gained. Conventional supervised analyses are limited to pre-defined cell populations and do not exploit the full potential of data. Here, in the context of a clinical trial of cancer patients treated with radiotherapy, we performed longitudinal flow cytometry analyses to identify multiple distinct cell populations in circulating whole blood. We cross-compared the results from state-of-the-art recommended supervised analyses with results from MegaClust, a high-performance data-driven clustering algorithm allowing fast and robust identification of cell-type populations. Ten distinct cell populations were accurately identified by supervised analyses, including main T, B, dendritic cell (DC), natural killer (NK) and monocytes subsets. While all ten subsets were also identified with MegaClust, additional cell populations were revealed (e.g. CD4⁺HLA-DR⁺ and NKT-like subsets), and DC profiling was enriched by the assignment of additional subset-specific markers. Comparison between transcriptomic profiles of purified DC populations and publicly available datasets confirmed the accuracy of the unsupervised clustering algorithm and demonstrated its potential to identify rare and scarcely described cell subsets. Our observations show that data-driven analyses of cytometry data significantly enrich the amount and quality of knowledge gained, representing an important step in refining the characterization of immune responses.

Introduction to Multiparametric Flow Cytometry and Analysis of High-Dimensional Data

Multiparametric flow cytometry is a technique utilized in translational experiments that utilizes fluorescently tagged antibodies and functional fluorescent dyes to measure proteins on the surface or in the cytoplasm of cells and to measure processes occurring within cells themselves. These fluorescent molecules, or fluorophores, can be tagged to antibodies to measure specific biological molecules such as proteins inside or on the surface of cells. Small organic compounds such as the nucleic acid binding dye propidium iodide (PI) can permeate compromised cell membranes when cells are no longer viable or used to measure DNA content of cycling cells. Successful completion of flow cytometry experiments requires expertise in both the preparation of the samples, acquisition of the samples on instruments, and analyses of the results. This chapter describes the principles needed to conduct a successful multiparameter flow cytometry experiment needed for drug development with references to well established internet resources that are useful to those less experienced in the field. In addition, we provide a brief introduction to data analysis including complex analysis of 10+ parameters simultaneously. These high-dimensional datasets require novel methods for analysis due to the volume of data collected, which are also introduced in this chapter.

Society for Immunotherapy of Cancer clinical and biomarkers data sharing resource document: Volume II-practical challenges

The development of strongly predictive validated biomarkers is essential for the field of immuno-oncology (IO) to advance. The highly complex, multifactorial data sets required to develop these biomarkers necessitate effective, responsible data-sharing efforts in order to maximize the scientific knowledge and utility gained from their collection. While the sharing of clinical- and safety-related trial data has already been streamlined to a large extent, the sharing of biomarker-aimed clinical trial derived data and data sets has been met with a number of hurdles that have impaired the progression of biomarkers from hypothesis to clinical use. These hurdles include technical challenges associated with the infrastructure, technology, workforce, and sustainability required for clinical biomarker data sharing. To provide guidance and assist in the navigation of these challenges, the Society for Immunotherapy of Cancer (SITC) Biomarkers Committee convened to outline the challenges that researchers currently face, both at the conceptual level (Volume I) and at the technical level (Volume II). The committee also suggests possible solutions to these problems in the form of professional standards and harmonized requirements for data sharing, assisting in continued progress toward effective, clinically relevant biomarkers in the IO setting.

From Bivariate to Multivariate Analysis of Cytometric Data: Overview of Computational Methods and Their Application in Vaccination Studies

Flow and mass cytometry are used to quantify the expression of multiple extracellular or intracellular molecules on single cells, allowing the phenotypic and functional characterization of complex cell populations. Multiparametric flow cytometry is particularly suitable for deep analysis of immune responses after vaccination, as it allows to measure the frequency, the phenotype, and the functional features of antigen-specific cells. When many parameters are investigated simultaneously, it is not feasible to analyze all the possible bi-dimensional combinations of marker expression with classical manual analysis and the adoption of advanced automated tools to process and analyze high-dimensional data sets becomes necessary. In recent years, the development of many tools for the automated analysis of multiparametric cytometry data has been reported, with an increasing record of publications starting from 2014. However, the use of these tools has been preferentially restricted to bioinformaticians, while few of them are routinely employed by the biomedical community. Filling the gap between algorithms developers and final users is fundamental for exploiting the advantages of computational tools in the analysis of cytometry data. The potentialities of automated analyses range from the improvement of the data quality in the pre-processing steps up to the unbiased, data-driven examination of complex datasets using a variety of algorithms based on different approaches. In this review, an overview of the automated analysis pipeline is provided, spanning from the pre-processing phase to the automated population analysis. Analysis based on computational tools might overcame both the subjectivity of manual gating and the operator-biased exploration of expected populations. Examples of applications of automated tools that have successfully improved the characterization of different cell populations in vaccination studies are also presented.

Measurable disease evaluation in patients with myeloma

Recent years saw significant breakthroughs in treatment of multiple myeloma. Durable remissions are now seen in a significant proportion of patients with the previously uniformly incurable and progressive disease. Yet because of deep suppression of the neoplastic myeloma clones by the newer therapies, older disease monitoring techniques are insufficient to distinguish between the patients at high risk of imminent relapse and those in whom durable remission is expected. This review briefly describes prognostic and therapeutic implications of measurable disease (MRD) evaluation, explains why deep MRD evaluation is needed for patients without morphologic evidence of disease, and reviews the state of the art of evaluation of myeloma MRD by flow cytometry.

Optimization in Detection of Antigen-Specific T Cells Through Differentially Labeled MHC Multimers

A large variety of fluorescent molecules are used on a regular basis to tag major histocompatibility complex (MHC) multimers for detection of antigen-specific T cells. We have evaluated the way in which the choice of fluorescent label can impact the detection of MHC multimer binding T cells in an exploratory proficiency panel where detection of MHC multimer binding T cells was assessed across 16 different laboratories. We found that the staining index (SI) of the multimer reagent provided the best direct correlation with the value of a given fluorochrome for T cell detection studies. The SI is dependent on flow cytometer settings and chosen antibody panel; hence, the optimal fluorochrome selection may differ from lab to lab. Consequently, we describe a strategy to evaluate performance of the detection channels and optimize the SI for selected fluorescent molecules. This approach can easily be used to test and optimize fluorescence detection in relation to MHC multimer staining and in general, for antibody-based identification of rare cell populations. © 2019 The Authors. Cytometry Part A published by Wiley Periodicals, Inc. on behalf of International Society for Advancement of Cytometry.

Adhering to adhesion: assessing integrin conformation to monitor T cells

Monitoring T cells is of major importance for the development of immunotherapies. Recent sophisticated assays can address particular aspects of the anti-tumor T-cell repertoire or support very large-scale immune screening for biomarker discovery. Robust methods for the routine assessment of the quantity and quality of antigen-specific T cells remain, however, essential. This review discusses selected methods that are commonly used for T-cell monitoring and summarizes the advantages and limitations of these assays. We also present a new functional assay, which specifically detects activated β2 integrins within a very short time following CD8+ T-cell stimulation. Because of its unique and favorable characteristics, this assay could be useful for implementation into our T-cell monitoring toolbox.

Single-cell technologies in reproductive immunology

The maternal-fetal interface represents a unique immune privileged site that maintains the ability to defend against pathogens while orchestrating the necessary tissue remodeling required for placentation. The recent discovery of novel cellular families (innate lymphoid cells, tissue-resident NK cells) suggests that our understanding of the decidual immunome is incomplete. To understand this complex milieu, new technological developments allow reproductive immunologists to collect increasingly complex data at a cellular resolution. Polychromatic flow cytometry allows for greater resolution in the identification of novel cell types by surface and intracellular protein. Single-cell RNA-seq coupled with microfluidics allows for efficient cellular transcriptomics. The extreme dimensionality and size of data sets generated, however, requires the application of novel computational approaches for unbiased analysis. There are now multiple dimensionality reduction (tSNE, SPADE) and visualization tools (SPICE) that allow researchers to efficiently analyze flow cytometry data. Development of computational tools has also been extended to RNA-seq data (including scRNA-seq), which requires specific analytical tools. Here, we provide an overview and a brief primer for the reproductive immunology community on data acquisition and computational tools for the analysis of complex flow cytometry and RNA-seq data.

Validation of Immunomonitoring Methods for Application in Clinical Studies: The HLA-Peptide Multimer Staining Assay

BACKGROUND

Validated assays are essential to generate data with defined specificity, consistency, and reliability. Although the process of validation is required for applying immunoassays in the context of clinical studies, reports on systematic validation of in vitro T cell assays are scarce so far. We recently validated our HLA-peptide multimer staining assay in a systematic manner so as to qualify the method for monitoring antigen-specific T cell responses after immunotherapy.

METHODS

Parameters of the assay, specificity, precision, linearity, sensitivity, and robustness were assessed systematically. Experiments were designed to address specifically each parameter and are detailed.

RESULTS

Nonspecific multimer staining was below the acceptance limit of 0.02% multimer(+) CD8(+) cells. The assay showed acceptable precision in all dimensions it was repeated (CV < 10%) and also demonstrated a linear detection (R2  > 0.99) of antigen specific cells.

CONCLUSIONS

We succeeded in validating the HLA-multimer staining assay in a systematic manner. Additionally, we propose a technical framework and recommendations that can be applied for validating other T cell assessment methods. © 2016 International Clinical Cytometry Society.

Automated Analysis of Flow Cytometry Data to Reduce Inter-Lab Variation in the Detection of Major Histocompatibility Complex Multimer-Binding T Cells

Manual analysis of flow cytometry data and subjective gate-border decisions taken by individuals continue to be a source of variation in the assessment of antigen-specific T cells when comparing data across laboratories, and also over time in individual labs. Therefore, strategies to provide automated analysis of major histocompatibility complex (MHC) multimer-binding T cells represent an attractive solution to decrease subjectivity and technical variation. The challenge of using an automated analysis approach is that MHC multimer-binding T cell populations are often rare and therefore difficult to detect. We used a highly heterogeneous dataset from a recent MHC multimer proficiency panel to assess if MHC multimer-binding CD8+ T cells could be analyzed with computational solutions currently available, and if such analyses would reduce the technical variation across different laboratories. We used three different methods, FLOw Clustering without K (FLOCK), Scalable Weighted Iterative Flow-clustering Technique (SWIFT), and ReFlow to analyze flow cytometry data files from 28 laboratories. Each laboratory screened for antigen-responsive T cell populations with frequency ranging from 0.01 to 1.5% of lymphocytes within samples from two donors. Experience from this analysis shows that all three programs can be used for the identification of high to intermediate frequency of MHC multimer-binding T cell populations, with results very similar to that of manual gating. For the less frequent populations (<0.1% of live, single lymphocytes), SWIFT outperformed the other tools. As used in this study, none of the algorithms offered a completely automated pipeline for identification of MHC multimer populations, as varying degrees of human interventions were needed to complete the analysis. In this study, we demonstrate the feasibility of using automated analysis pipelines for assessing and identifying even rare populations of antigen-responsive T cells and discuss the main properties, differences, and advantages of the different methods tested.

Computational flow cytometry: helping to make sense of high-dimensional immunology data

Recent advances in flow cytometry allow scientists to measure an increasing number of parameters per cell, generating huge and high-dimensional datasets. To analyse, visualize and interpret these data, newly available computational techniques should be adopted, evaluated and improved upon by the immunological community. Computational flow cytometry is emerging as an important new field at the intersection of immunology and computational biology; it allows new biological knowledge to be extracted from high-throughput single-cell data. This Review provides non-experts with a broad and practical overview of the many recent developments in computational flow cytometry.

Toward harmonized phenotyping of human myeloid-derived suppressor cells by flow cytometry: results from an interim study

There is an increasing interest for monitoring circulating myeloid-derived suppressor cells (MDSCs) in cancer patients, but there are also divergences in their phenotypic definition. To overcome this obstacle, the Cancer Immunoguiding Program under the umbrella of the Association of Cancer Immunotherapy is coordinating a proficiency panel program that aims at harmonizing MDSC phenotyping. After a consultation period, a two-stage approach was designed to harmonize MDSC phenotype. In the first step, an international consortium of 23 laboratories immunophenotyped 10 putative MDSC subsets on pretested, peripheral blood mononuclear cells of healthy donors to assess the level of concordance and define robust marker combinations for the identification of circulating MDSCs. At this stage, no mandatory requirements to standardize reagents or protocols were introduced. Data analysis revealed a small intra-laboratory, but very high inter-laboratory variance for all MDSC subsets, especially for the granulocytic subsets. In particular, the use of a dead-cell marker altered significantly the reported percentage of granulocytic MDSCs, confirming that these cells are especially sensitive to cryopreservation and/or thawing. Importantly, the gating strategy was heterogeneous and associated with high inter-center variance. Overall, our results document the high variability in MDSC phenotyping in the multicenter setting if no harmonization/standardization measures are applied. Although the observed variability depended on a number of identified parameters, the main parameter associated with variation was the gating strategy. Based on these findings, we propose further efforts to harmonize marker combinations and gating parameters to identify strategies for a robust enumeration of MDSC subsets.

An Introduction to Automated Flow Cytometry Gating Tools and Their Implementation

Current flow cytometry (FCM) reagents and instrumentation allow for the measurement of an unprecedented number of parameters for any given cell within a homogenous or heterogeneous population. While this provides a great deal of power for hypothesis testing, it also generates a vast amount of data, which is typically analyzed manually through a processing called “gating.” For large experiments, such as high-content screens, in which many parameters are measured, the time required for manual analysis as well as the technical variability inherent to manual gating can increase dramatically, even becoming prohibitive depending on the clinical or research goal. In the following article, we aim to provide the reader an overview of automated FCM analysis as well as an example of the implementation of FLOw Clustering without K, a tool that we consider accessible to researchers of all levels of computational expertise. In most cases, computational assistance methods are more reproducible and much faster than manual gating, and for some, also allow for the discovery of cellular populations that might not be expected or evident to the researcher. We urge any researcher who is planning or has previously performed large FCM experiments to consider implementing computational assistance into their analysis pipeline.