1
|
Pine PS, Lund SP, Stass SA, Kukuruga D, Jiang F, Sorbara L, Srivastava S, Salit M. Cell-based reference samples designed with specific differences in microRNA biomarkers. BMC Biotechnol 2018; 18:17. [PMID: 29554888 PMCID: PMC5859499 DOI: 10.1186/s12896-018-0423-4] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2017] [Accepted: 03/07/2018] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND We demonstrate the feasibility of creating a pair of reference samples to be used as surrogates for clinical samples measured in either a research or clinical laboratory setting. The reference sample paradigm presented and evaluated here is designed to assess the capability of a measurement process to detect true differences between two biological samples. Cell-based reference samples can be created with a biomarker signature pattern designed in silico. Clinical laboratories working in regulated applications are required to participate in proficiency testing programs; research laboratories doing discovery typically do not. These reference samples can be used in proficiency tests or as process controls that allow a laboratory to evaluate and optimize its measurement systems, monitor performance over time (process drift), assess changes in protocols, reagents, and/or personnel, maintain standard operating procedures, and most importantly, provide evidence for quality results. RESULTS The biomarkers of interest in this study are microRNAs (miRNAs), small non-coding RNAs involved in the regulation of gene expression. Multiple lung cancer associated cell lines were determined by reverse transcription (RT)-PCR to have sufficiently different miRNA profiles to serve as components in mixture designs as reference samples. In silico models based on the component profiles were used to predict miRNA abundance ratios between two different cell line mixtures, providing target values for profiles obtained from in vitro mixtures. Two reference sample types were tested: total RNA mixed after extraction from cell lines, and intact cells mixed prior to RNA extraction. MicroRNA profiling of a pair of samples composed of extracted RNA derived from these cell lines successfully replicated the target values. Mixtures of intact cells from these lines also approximated the target values, demonstrating potential utility as mimics for clinical specimens. Both designs demonstrated their utility as reference samples for inter- or intra-laboratory testing. CONCLUSIONS Cell-based reference samples can be created for performance assessment of a measurement process from biomolecule extraction through quantitation. Although this study focused on miRNA profiling with RT-PCR using cell lines associated with lung cancer, the paradigm demonstrated here should be extendable to genome-scale platforms and other biomolecular endpoints.
Collapse
Affiliation(s)
- P Scott Pine
- Joint Initiative for Metrology in Biology, National Institute of Standards and Technology, 443 Via Ortega, Stanford, CA, 94305, USA.
| | - Steven P Lund
- Statistical Engineering Division, National Institute of Standards and Technology, Gaithersburg, MD, 20899, USA
| | - Sanford A Stass
- Department of Pathology, University of Maryland School of Medicine, Baltimore, MD, 21201, USA
| | - Debra Kukuruga
- Department of Pathology, University of Maryland School of Medicine, Baltimore, MD, 21201, USA
| | - Feng Jiang
- Department of Pathology, University of Maryland School of Medicine, Baltimore, MD, 21201, USA
| | - Lynn Sorbara
- Division of Cancer Prevention, National Cancer Institute, Rockville, MD, 20850, USA
| | - Sudhir Srivastava
- Division of Cancer Prevention, National Cancer Institute, Rockville, MD, 20850, USA
| | - Marc Salit
- Joint Initiative for Metrology in Biology, National Institute of Standards and Technology, 443 Via Ortega, Stanford, CA, 94305, USA
| |
Collapse
|
2
|
Pine PS, Lund SP, Parsons JR, Vang LK, Mahabal AA, Cinquini L, Kelly SC, Kincaid H, Crichton DJ, Spira A, Liu G, Gower AC, Pass HI, Goparaju C, Dubinett SM, Krysan K, Stass SA, Kukuruga D, Van Keuren-Jensen K, Courtright-Lim A, Thompson KL, Rosenzweig BA, Sorbara L, Srivastava S, Salit ML. Summarizing performance for genome scale measurement of miRNA: reference samples and metrics. BMC Genomics 2018; 19:180. [PMID: 29510677 PMCID: PMC5838960 DOI: 10.1186/s12864-018-4496-1] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2017] [Accepted: 01/25/2018] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND The potential utility of microRNA as biomarkers for early detection of cancer and other diseases is being investigated with genome-scale profiling of differentially expressed microRNA. Processes for measurement assurance are critical components of genome-scale measurements. Here, we evaluated the utility of a set of total RNA samples, designed with between-sample differences in the relative abundance of miRNAs, as process controls. RESULTS Three pure total human RNA samples (brain, liver, and placenta) and two different mixtures of these components were evaluated as measurement assurance control samples on multiple measurement systems at multiple sites and over multiple rounds. In silico modeling of mixtures provided benchmark values for comparison with physical mixtures. Biomarker development laboratories using next-generation sequencing (NGS) or genome-scale hybridization assays participated in the study and returned data from the samples using their routine workflows. Multiplexed and single assay reverse-transcription PCR (RT-PCR) was used to confirm in silico predicted sample differences. Data visualizations and summary metrics for genome-scale miRNA profiling assessment were developed using this dataset, and a range of performance was observed. These metrics have been incorporated into an online data analysis pipeline and provide a convenient dashboard view of results from experiments following the described design. The website also serves as a repository for the accumulation of performance values providing new participants in the project an opportunity to learn what may be achievable with similar measurement processes. CONCLUSIONS The set of reference samples used in this study provides benchmark values suitable for assessing genome-scale miRNA profiling processes. Incorporation of these metrics into an online resource allows laboratories to periodically evaluate their performance and assess any changes introduced into their measurement process.
Collapse
Affiliation(s)
- P Scott Pine
- Joint Initiative for Metrology in Biology, National Institute of Standards and Technology, 443 Via Ortega, Stanford, CA, 94305, USA.
| | - Steven P Lund
- Statistical Engineering Division, National Institute of Standards and Technology, Gaithersburg, MD, USA
| | - Jerod R Parsons
- Joint Initiative for Metrology in Biology, National Institute of Standards and Technology, 443 Via Ortega, Stanford, CA, 94305, USA
| | - Lindsay K Vang
- Joint Initiative for Metrology in Biology, National Institute of Standards and Technology, 443 Via Ortega, Stanford, CA, 94305, USA
| | - Ashish A Mahabal
- Center for Data Driven Discovery, California Institute of Technology, Pasadena, CA, USA
| | - Luca Cinquini
- Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA, USA
| | - Sean C Kelly
- Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA, USA
| | - Heather Kincaid
- Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA, USA
| | - Daniel J Crichton
- Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA, USA
| | - Avrum Spira
- Section of Computational Biomedicine, Department of Medicine, Boston University School of Medicine, Boston, MA, USA
| | - Gang Liu
- Section of Computational Biomedicine, Department of Medicine, Boston University School of Medicine, Boston, MA, USA
| | - Adam C Gower
- Section of Computational Biomedicine, Department of Medicine, Boston University School of Medicine, Boston, MA, USA
| | - Harvey I Pass
- Department of Cardiothoracic Surgery, NYU Langone Medical Center, New York, NY, USA
| | - Chandra Goparaju
- Department of Cardiothoracic Surgery, NYU Langone Medical Center, New York, NY, USA
| | - Steven M Dubinett
- Department of Medicine, David Geffen School of Medicine at UCLA, Los Angeles, CA, USA.,Veterans Affairs Greater Los Angeles Healthcare System, Los Angeles, CA, USA
| | - Kostyantyn Krysan
- Department of Medicine, David Geffen School of Medicine at UCLA, Los Angeles, CA, USA
| | - Sanford A Stass
- Department of Pathology, University of Maryland School of Medicine, Baltimore, MD, USA
| | - Debra Kukuruga
- Department of Pathology, University of Maryland School of Medicine, Baltimore, MD, USA
| | | | | | - Karol L Thompson
- Center for Drug Evaluation and Research, Food and Drug Administration, Silver Spring, MD, USA
| | - Barry A Rosenzweig
- Center for Drug Evaluation and Research, Food and Drug Administration, Silver Spring, MD, USA
| | - Lynn Sorbara
- Division of Cancer Prevention, National Cancer Institute, Rockville, MD, USA
| | - Sudhir Srivastava
- Division of Cancer Prevention, National Cancer Institute, Rockville, MD, USA
| | - Marc L Salit
- Joint Initiative for Metrology in Biology, National Institute of Standards and Technology, 443 Via Ortega, Stanford, CA, 94305, USA
| |
Collapse
|
5
|
Pine PS, Rosenzweig BA, Thompson KL. An adaptable method using human mixed tissue ratiometric controls for benchmarking performance on gene expression microarrays in clinical laboratories. BMC Biotechnol 2011; 11:38. [PMID: 21486464 PMCID: PMC3103427 DOI: 10.1186/1472-6750-11-38] [Citation(s) in RCA: 11] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2011] [Accepted: 04/12/2011] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Molecular biomarkers that are based on mRNA transcripts are being developed for the diagnosis and treatment of a number of diseases. DNA microarrays are one of the primary technologies being used to develop classifiers from gene expression data for clinically relevant outcomes. Microarray assays are highly multiplexed measures of comparative gene expression but have a limited dynamic range of measurement and show compression in fold change detection. To increase the clinical utility of microarrays, assay controls are needed that benchmark performance using metrics that are relevant to the analysis of genomic data generated with biological samples. RESULTS Ratiometric controls were prepared from commercial sources of high quality RNA from human tissues with distinctly different expression profiles and mixed in defined ratios. The samples were processed using six different target labeling protocols and replicate datasets were generated on high density gene expression microarrays. The area under the curve from receiver operating characteristic plots was calculated to measure diagnostic performance. The reliable region of the dynamic range was derived from log(2) ratio deviation plots made for each dataset. Small but statistically significant differences in diagnostic performance were observed between standardized assays available from the array manufacturer and alternative methods for target generation. Assay performance using the reliable range of comparative measurement as a metric was improved by adjusting sample hybridization conditions for one commercial kit. CONCLUSIONS Process improvement in microarray assay performance was demonstrated using samples prepared from commercially available materials and two metrics - diagnostic performance and the reliable range of measurement. These methods have advantages over approaches that use a limited set of external controls or correlations to reference sets, because they provide benchmark values that can be used by clinical laboratories to help optimize protocol conditions and laboratory proficiency with microarray assays.
Collapse
Affiliation(s)
- P Scott Pine
- Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, MD 20993 USA
| | | | | |
Collapse
|
6
|
Enkemann SA. Standards affecting the consistency of gene expression arrays in clinical applications. Cancer Epidemiol Biomarkers Prev 2010; 19:1000-3. [PMID: 20332273 DOI: 10.1158/1055-9965.epi-10-0044] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022] Open
Abstract
The use of microarray technology to measure gene expression has created optimism for the feasibility of using molecular assessments of tumors routinely in the clinical management of cancer. Gene expression arrays have been pioneers in the development of standards; both for research use and now for clinical application. Some of the existing standards have been driven by the early perception that microarray technology was inconsistent and perhaps unreliable. More recent experimentation has shown that reproducible data can be achieved and clinical standards are beginning to emerge. For the transcriptional assessment of tumors, this means a system that correctly samples a tumor, isolates RNA and processes this for microarray analysis, evaluates the data, and communicates findings in a consistent and timely fashion. The most important standard is to show that a clinically important assessment can be made with microarray data. The standards emerging from work on various parts of the entire process could guide the development of a workable system. However, the final standard for each component of the process depends on the accuracy required when the assay becomes part of the clinical routine: a routine that now includes the molecular evaluation of tumors.
Collapse
Affiliation(s)
- Steven A Enkemann
- Molecular Genomics Laboratory, H. Lee Moffitt Cancer Center and Research Institute, SRB2 12902 Magnolia Drive, Tampa, FL 33612, USA.
| |
Collapse
|
8
|
Thompson KL, Pine PS. Comparison of the diagnostic performance of human whole genome microarrays using mixed-tissue RNA reference samples. Toxicol Lett 2008; 186:58-61. [PMID: 18822358 DOI: 10.1016/j.toxlet.2008.08.018] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2008] [Revised: 08/29/2008] [Accepted: 08/29/2008] [Indexed: 10/21/2022]
Abstract
Universal approaches for assessing the diagnostic performance of microarray assays are essential for the application of microarray technology to clinical and regulatory settings. Reference systems for diagnostic assays in laboratory medicine typically involve the utilization of reference samples, metrics, and reference datasets to ensure that measurements are comparable and true. For microarray performance evaluation and process improvement, reference samples can be composed of mixes of different tissue or cell line RNAs that contain tissue-selective analytes at defined target ratios. The diagnostic accuracy of detected changes in expression, measured as the area under the curve from receiver-operating characteristic plots, can provide a single commutable value for comparing assay specificity and sensitivity. Examples of applying this method for assessing overall performance are provided using public datasets generated on five commercial human whole genome microarray platforms for the MicroArray Quality Control project, a community-wide effort to address issues surrounding microarray data reliability.
Collapse
Affiliation(s)
- Karol L Thompson
- Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, MD, USA.
| | | |
Collapse
|