QA/QC issues to aid regulatory acceptance of microarray gene expression data

Application of omics data in regulatory toxicology: report of an international BfR expert workshop

Advances in omics techniques and molecular toxicology are necessary to provide new perspectives for regulatory toxicology. By the application of modern molecular techniques, more mechanistic information should be gained to support standard toxicity studies and to contribute to a reduction and refinement of animal experiments required for certain regulatory purposes. The relevance and applicability of data obtained by omics methods to regulatory purposes such as grouping of chemicals, mode of action analysis or classification and labelling needs further improvement, defined validation and cautious expert judgment. Based on the results of an international expert workshop organized 2014 by the Federal Institute for Risk Assessment in Berlin, this paper is aimed to provide a critical overview of the regulatory relevance and reliability of omics methods, basic requirements on data quality and validation, as well as regulatory criteria to decide which effects observed by omics methods should be considered adverse or non-adverse. As a way forward, it was concluded that the inclusion of omics data can facilitate a more flexible approach for regulatory risk assessment and may help to reduce or refine animal testing.

Optimization of a high-throughput whole blood expression profiling methodology and its application to assess the pharmacodynamics of interferon (IFN) beta-1a or polyethylene glycol-conjugated IFN beta-1a in healthy clinical trial subjects

Background

Clinical trials offer a unique opportunity to study human disease and response to therapy in a highly controlled setting. The application of high-throughput expression profiling to peripheral blood from clinical trial subjects could facilitate the identification of transcripts that function as prognostic or diagnostic markers of disease or treatment. The paramount issue for these methods is the ability to produce robust, reproducible, and timely mRNA expression profiles from peripheral blood. Single-stranded complementary DNA (sscDNA) targets derived from whole blood exhibit improved detection of transcripts and reduced variance as compared to their complementary RNA counterparts and therefore provide a better option for interrogation of peripheral blood on oligonucleotide arrays. High-throughput microarray technologies such as the high-throughput plate array platform offer several advantages compared with slide- or cartridge-based arrays; however, manufacturer’s protocols do not support the use of sscDNA targets.

Results

We have developed a highly reproducible, high-through put, whole blood expression profiling methodology based on sscDNA and used it to analyze human brain reference RNA and universal human reference RNA samples to identify experimental conditions that most highly correlated with a gold standard quantitative polymerase chain reaction reference dataset. We then utilized the optimized method to analyze whole blood samples from healthy clinical trial subjects treated with different versions of interferon (IFN) beta-1a. Analysis of whole blood samples before and after treatment with intramuscular [IM] IFN beta-1a or polyethylene glycol-conjugated IFN (PEG-IFN) beta-1a under optimized experimental conditions demonstrated that PEG-IFN beta-1a induced a more sustained and prolonged pharmacodynamic response than unmodified IM IFN beta-1a. These results provide validation of the utility of this new methodology and suggest the potential therapeutic benefit of a sustained pharmacodynamic response to PEG-IFN beta-1a.

Conclusions

This novel microarray methodology is ideally suited for utilization in large clinical studies to identify expressed transcripts for the elucidation of disease mechanisms of action and as prognostic, diagnostic, or toxicity markers.

Quality assurance of RNA expression profiling in clinical laboratories

RNA expression profiles are increasingly used to diagnose and classify disease, based on expression patterns of as many as several thousand RNAs. To ensure quality of expression profiling services in clinical settings, a standard operating procedure incorporates multiple quality indicators and controls, beginning with preanalytic specimen preparation and proceeding thorough analysis, interpretation, and reporting. Before testing, histopathological examination of each cellular specimen, along with optional cell enrichment procedures, ensures adequacy of the input tissue. Other tactics include endogenous controls to evaluate adequacy of RNA and exogenous or spiked controls to evaluate run- and patient-specific performance of the test system, respectively. Unique aspects of quality assurance for array-based tests include controls for the pertinent outcome signatures that often supersede controls for each individual analyte, built-in redundancy for critical analytes or biochemical pathways, and software-supported scrutiny of abundant data by a laboratory physician who interprets the findings in a manner facilitating appropriate medical intervention. Access to high-quality reagents, instruments, and software from commercial sources promotes standardization and adoption in clinical settings, once an assay is vetted in validation studies as being analytically sound and clinically useful. Careful attention to the well-honed principles of laboratory medicine, along with guidance from government and professional groups on strategies to preserve RNA and manage large data sets, promotes clinical-grade assay performance.

Clinical implementation of RNA signatures for pharmacogenomic decision-making

RNA profiling is increasingly used to predict drug response, dose, or toxicity based on analysis of drug pharmacokinetic or pharmacodynamic pathways. Before implementing multiplexed RNA arrays in clinical practice, validation studies are carried out to demonstrate sufficient evidence of analytic and clinical performance, and to establish an assay protocol with quality assurance measures. Pathologists assure quality by selecting input tissue and by interpreting results in the context of the input tissue as well as the technologies that were used and the clinical setting in which the test was ordered. A strength of RNA profiling is the array-based measurement of tens to thousands of RNAs at once, including redundant tests for critical analytes or pathways to promote confidence in test results. Instrument and reagent manufacturers are crucial for supplying reliable components of the test system. Strategies for quality assurance include careful attention to RNA preservation and quality checks at pertinent steps in the assay protocol, beginning with specimen collection and proceeding through the various phases of transport, processing, storage, analysis, interpretation, and reporting. Specimen quality is checked by probing housekeeping transcripts, while spiked and exogenous controls serve as a check on analytic performance of the test system. Software is required to manipulate abundant array data and present it for interpretation by a laboratory physician who reports results in a manner facilitating therapeutic decision-making. Maintenance of the assay requires periodic documentation of personnel competency and laboratory proficiency. These strategies are shepherding genomic arrays into clinical settings to provide added value to patients and to the larger health care system.

Why is there a lack of consensus on molecular subgroups of glioblastoma? Understanding the nature of biological and statistical variability in glioblastoma expression data

INTRODUCTION

Gene expression patterns characterizing clinically-relevant molecular subgroups of glioblastoma are difficult to reproduce. We suspect a combination of biological and analytic factors confounds interpretation of glioblastoma expression data. We seek to clarify the nature and relative contributions of these factors, to focus additional investigations, and to improve the accuracy and consistency of translational glioblastoma analyses.

METHODS

We analyzed gene expression and clinical data for 340 glioblastomas in The Cancer Genome Atlas (TCGA). We developed a logic model to analyze potential sources of biological, technical, and analytic variability and used standard linear classifiers and linear dimensional reduction algorithms to investigate the nature and relative contributions of each factor.

RESULTS

Commonly-described sources of classification error, including individual sample characteristics, batch effects, and analytic and technical noise make measurable but proportionally minor contributions to inconsistent molecular classification. Our analysis suggests that three, previously underappreciated factors may account for a larger fraction of classification errors: inherent non-linear/non-orthogonal relationships among the genes used in conjunction with classification algorithms that assume linearity; skewed data distributions assumed to be Gaussian; and biologic variability (noise) among tumors, of which we propose three types.

CONCLUSIONS

Our analysis of the TCGA data demonstrates a contributory role for technical factors in molecular classification inconsistencies in glioblastoma but also suggests that biological variability, abnormal data distribution, and non-linear relationships among genes may be responsible for a proportionally larger component of classification error. These findings may have important implications for both glioblastoma research and for translational application of other large-volume biological databases.

Bringing diagnostic technologies to the clinical laboratory: Rigor, regulation, and reality

With the numerous reports of new technologies and biomarkers reported in the literature, it may be surprising that there are not an equal number of new products available to the clinical diagnostic laboratory. Powerful potential tools such as protein microarrays and MS patterns have been extensively published yet commercialization and acceptance of these technologies has yet to happen. The reasons for this are a combination of industry risk avoidance, academic focus on discovery, and a lack of appreciation for the high standards and regulation that surrounds the clinical diagnostic laboratory. The development and validation of a new technology or biomarker ensures that a test is reproducible, controllable, and has a defined accuracy and clinical predictive result but this information is only obtained through somewhat mundane but necessary experimental work. The use of design of experiment principles helps to define material parameters to ensure performance. The organization and documentation of this work through a quality system is both mandated and practical. All of this must be done before a test can reach the market with the safety and effectiveness review of regulatory agencies.

Experimental comparison and cross-validation of Affymetrix HT plate and cartridge array gene expression platforms

The successful use of gene expression microarrays in basic research studies has spawned interest in the use of this technology for clinical trial and population-based studies, but cost, complexity of sample processing and tracking, and limitations of sample throughput have restricted their use for these very large-scale investigations. The Affymetrix GeneChip Plate Array System addresses these concerns and could facilitate larger studies if the data prove to be comparable to industry-standard cartridge arrays. Here we present a comparative evaluation of performance between Affymetrix GeneChip Human 133A cartridge and plate arrays with an emphasis on the assessment of systematic variation and its impact on log ratio data. This study utilized two standardized control RNAs on four independent lots of plate and cartridge arrays. We found that HT plate arrays showed improved specificity and were more reproducible over a wide intensity range, but cartridge arrays exhibit better sensitivity. Not surprisingly, artifactual changes due to positional effects were detectable on plate arrays, but were generally small in number and magnitude and in practice may be removed using standard fold-change and p-value thresholds. Overall, log ratio data between cartridges and plate arrays were remarkably concordant. We conclude that HT arrays offer significant improvements over cartridge arrays for large-scale studies.

Performance comparison of two microarray platforms to assess differential gene expression in human monocyte and macrophage cells

Background

In this study we assessed the respective ability of Affymetrix and Illumina microarray methodologies to answer a relevant biological question, namely the change in gene expression between resting monocytes and macrophages derived from these monocytes. Five RNA samples for each type of cell were hybridized to the two platforms in parallel. In addition, a reference list of differentially expressed genes (DEG) was generated from a larger number of hybridizations (mRNA from 86 individuals) using the RNG/MRC two-color platform.

Results

Our results show an important overlap of the Illumina and Affymetrix DEG lists. In addition, more than 70% of the genes in these lists were also present in the reference list. Overall the two platforms had very similar performance in terms of biological significance, evaluated by the presence in the DEG lists of an excess of genes belonging to Gene Ontology (GO) categories relevant for the biology of monocytes and macrophages. Our results support the conclusion of the MicroArray Quality Control (MAQC) project that the criteria used to constitute the DEG lists strongly influence the degree of concordance among platforms. However the importance of prioritizing genes by magnitude of effect (fold change) rather than statistical significance (p-value) to enhance cross-platform reproducibility recommended by the MAQC authors was not supported by our data.

Conclusion

Functional analysis based on GO enrichment demonstrates that the 2 compared technologies delivered very similar results and identified most of the relevant GO categories enriched in the reference list.