MSstatsQC: Longitudinal System Suitability Monitoring and Quality Control for Targeted Proteomic Experiments

A Framework for Quality Control in Quantitative Proteomics

A thorough evaluation of the quality, reproducibility, and variability of bottom-up proteomics data is necessary at every stage of a workflow, from planning to analysis. We share vignettes applying adaptable quality control (QC) measures to assess sample preparation, system function, and quantitative analysis. System suitability samples are repeatedly measured longitudinally with targeted methods, and we share examples where they are used on three instrument platforms to identify severe system failures and track function over months to years. Internal QCs incorporated at the protein and peptide levels allow our team to assess sample preparation issues and to differentiate system failures from sample-specific issues. External QC samples prepared alongside our experimental samples are used to verify the consistency and quantitative potential of our results during batch correction and normalization before assessing biological phenotypes. We combine these controls with rapid analysis (Skyline), longitudinal QC metrics (AutoQC), and server-based data deposition (PanoramaWeb). We propose that this integrated approach to QC is a useful starting point for groups to facilitate rapid quality control assessment to ensure that valuable instrument time is used to collect the best quality data possible. Data are available on Panorama Public and ProteomeXchange under the identifier PXD051318.

Quality Assurance for Multiplex Quantitative Clinical Chemistry Proteomics in Large Clinical Trials

BACKGROUND

To evaluate the clinical performance and effectiveness of a multiplex apolipoprotein panel in the context of cardiovascular precision diagnostics, clinical samples of patients with recent acute coronary syndrome in the ODYSSEY OUTCOMES trial were measured by quantitative clinical chemistry proteomics (qCCP). The ISO15189-accredited laboratory setting, including the total testing process (TTP), served as a foundation for this study. Consequently, tailored quality assurance measures needed to be designed and implemented to suit the demands of a multiplex LC-MS/MS test.

METHODS

Nine serum apolipoproteins were measured in 23 376 samples with a laboratory-developed multiplex apolipoprotein test on 4 Agilent 6495 LC-MS/MS systems. A fit-for-purpose process was designed with tailored additions enhancing the accredited laboratory infrastructure and the TTP. Quality assurance was organized in 3 steps: system suitability testing (SST), internal quality control (IQC) evaluation with adjusted Westgard rules to fit a multiplex test, and interpeptide agreement analysis. Data was semi-automatically evaluated with a custom R script.

RESULTS

LC-MS/MS analyses were performed with the following between-run CVs: for apolipoprotein (Apo) (a) 6.2%, Apo A-I 2.3%, Apo A-II 2.1%, Apo A-IV 2.9%, Apo B 1.9%, Apo C-I 3.3%, Apo C-II 3.3%, Apo C-III 2.7%, and for Apo E 3.3% and an average interpeptide agreement Pearson r of 0.981.

CONCLUSIONS

This is the first study of its kind in which qCCP was performed at this scale. This research successfully demonstrates the feasibility of high-throughput LC-MS/MS applications in large clinical trials. ClinicalTrials.gov Registration Number: NCT01663402.

Comprehensive Overview of Bottom-Up Proteomics Using Mass Spectrometry

Proteomics is the large scale study of protein structure and function from biological systems through protein identification and quantification. "Shotgun proteomics" or "bottom-up proteomics" is the prevailing strategy, in which proteins are hydrolyzed into peptides that are analyzed by mass spectrometry. Proteomics studies can be applied to diverse studies ranging from simple protein identification to studies of proteoforms, protein-protein interactions, protein structural alterations, absolute and relative protein quantification, post-translational modifications, and protein stability. To enable this range of different experiments, there are diverse strategies for proteome analysis. The nuances of how proteomic workflows differ may be challenging to understand for new practitioners. Here, we provide a comprehensive overview of different proteomics methods. We cover from biochemistry basics and protein extraction to biological interpretation and orthogonal validation. We expect this Review will serve as a handbook for researchers who are new to the field of bottom-up proteomics.

A framework for quality control in quantitative proteomics

A thorough evaluation of the quality, reproducibility, and variability of bottom-up proteomics data is necessary at every stage of a workflow from planning to analysis. We share vignettes applying adaptable quality control (QC) measures to assess sample preparation, system function, and quantitative analysis. System suitability samples are repeatedly measured longitudinally with targeted methods, and we share examples where they are used on three instrument platforms to identify severe system failures and track function over months to years. Internal QCs incorporated at protein and peptide-level allow our team to assess sample preparation issues and to differentiate system failures from sample-specific issues. External QC samples prepared alongside our experimental samples are used to verify the consistency and quantitative potential of our results during batch correction and normalization before assessing biological phenotypes. We combine these controls with rapid analysis (Skyline), longitudinal QC metrics (AutoQC), and server-based data deposition (PanoramaWeb). We propose that this integrated approach to QC is a useful starting point for groups to facilitate rapid quality control assessment to ensure that valuable instrument time is used to collect the best quality data possible. Data are available on Panorama Public and on ProteomeXchange under the identifier PXD051318.

Monitoring Both Extended and Tryptic Forms of Stable Isotope-Labeled Standard Peptides Provides an Internal Quality Control of Proteolytic Digestion in Targeted Mass Spectrometry-Based Assays

Targeted mass spectrometry (MS)-based proteomic assays, such as multiplexed multiple reaction monitoring (MRM)-MS assays, enable sensitive and specific quantification of proteotypic peptides as stoichiometric surrogates for proteins. Efforts are underway to expand the use of MRM-MS assays in clinical environments, which requires a reliable strategy to monitor proteolytic digestion efficiency within individual samples. Towards this goal, extended stable isotope-labeled standard (SIS) peptides (hE), which incorporate native proteolytic cleavage sites, can be spiked into protein lysates prior to proteolytic (trypsin) digestion, and release of the tryptic SIS peptide (hT) can be monitored. However, hT measurements alone cannot monitor the extent of digestion and may be confounded by matrix effects specific to individual patient samples; therefore, they are not sufficient to monitor sample-to-sample digestion variability. We hypothesized that measuring undigested hE, along with its paired hT, would improve detection of digestion issues compared to only measuring hT. We tested the ratio of the SIS pair measurements, or hE/hT, as a quality control (QC) metric of trypsin digestion for two MRM assays: a direct-MRM (398 targets) and an immuno-MRM (126 targets requiring immunoaffinity peptide enrichment) assay, with extended SIS peptides observable for 54% (216) and 62% (78) of the targets, respectively. We evaluated the quantitative bias for each target in a series of experiments that adversely affected proteolytic digestion (e.g., variable digestion times, pH, and temperature). We identified a subset of SIS pairs (36 for the direct-MRM, 7 for the immuno-MRM assay) for which the hE/hT ratio reliably detected inefficient digestion that resulted in decreased assay sensitivity and unreliable endogenous quantification. The hE/hT ratio was more responsive to a decrease in digestion efficiency than a metric based on hT measurements alone. For clinical-grade MRM-MS assays, this study describes a ready-to-use QC panel and also provides a road map for designing custom QC panels.

Evaluating cPILOT Data toward Quality Control Implementation

Multiplexing enables the monitoring of hundreds to thousands of proteins in quantitative proteomics analyses and increases sample throughput. In most mass-spectrometry-based proteomics workflows, multiplexing is achieved by labeling biological samples with heavy isotopes via precursor isotopic labeling or isobaric tagging. Enhanced multiplexing strategies, such as combined precursor isotopic labeling and isobaric tagging (cPILOT), combine multiple technologies to afford an even higher sample throughput. Critical to enhanced multiplexing analyses is ensuring that analytical performance is optimal and that missingness of sample channels is minimized. Automation of sample preparation steps and use of quality control (QC) metrics can be incorporated into multiplexing analyses and reduce the likelihood of missing information, thus maximizing the amount of usable quantitative data. Here, we implemented QC metrics previously developed in our laboratory to evaluate a 36-plex cPILOT experiment that encompassed 144 mouse samples of various tissue types, time points, genotypes, and biological replicates. The evaluation focuses on the use of a sample pool generated from all samples in the experiment to monitor the daily instrument performance and to provide a means for data normalization across sample batches. Our results show that tracking QC metrics enabled the quantification of ∼7000 proteins in each sample batch, of which ∼70% had minimal missing values across up to 36 sample channels. Implementation of QC metrics for future cPILOT studies as well as other enhanced multiplexing strategies will help yield high-quality data sets.

Mass spectrometry-based high-throughput proteomics and its role in biomedical studies and systems biology

There are multiple reasons why the next generation of biological and medical studies require increasing numbers of samples. Biological systems are dynamic, and the effect of a perturbation depends on the genetic background and environment. As a consequence, many conditions need to be considered to reach generalizable conclusions. Moreover, human population and clinical studies only reach sufficient statistical power if conducted at scale and with precise measurement methods. Finally, many proteins remain without sufficient functional annotations, because they have not been systematically studied under a broad range of conditions. In this review, we discuss the latest technical developments in mass spectrometry (MS)-based proteomics that facilitate large-scale studies by fast and efficient chromatography, fast scanning mass spectrometers, data-independent acquisition (DIA), and new software. We further highlight recent studies which demonstrate how high-throughput (HT) proteomics can be applied to capture biological diversity, to annotate gene functions or to generate predictive and prognostic models for human diseases.

A system suitability testing platform for untargeted, high-resolution mass spectrometry

The broad coverage of untargeted metabolomics poses fundamental challenges for the harmonization of measurements along time, even if they originate from the very same instrument. Internal isotopic standards can hardly cover the chemical complexity of study samples. Therefore, they are insufficient for normalizing data a posteriori as done for targeted metabolomics. Instead, it is crucial to verify instrument’s performance a priori, that is, before samples are injected. Here, we propose a system suitability testing platform for time-of-flight mass spectrometers independent of liquid chromatography. It includes a chemically defined quality control mixture, a fast acquisition method, software for extracting ca. 3,000 numerical features from profile data, and a simple web service for monitoring. We ran a pilot for 21 months and present illustrative results for anomaly detection or learning causal relationships between the spectral features and machine settings. Beyond mere detection of anomalies, our results highlight several future applications such as 1) recommending instrument retuning strategies to achieve desired values of quality indicators, 2) driving preventive maintenance, and 3) using the obtained, detailed spectral features for posterior data harmonization.

A Sensitive and Controlled Data-Independent Acquisition Method for Proteomic Analysis of Cell Therapies

Mass spectrometry (MS)-based proteomic measurements are uniquely poised to impact the development of cell and gene therapies. With the adoption of rigorous instrumental performance qualifications (PQs), large-scale proteomics can move from a research to a manufacturing control tool. Especially suited, data-independent acquisition (DIA) approaches have distinctive qualities to extend multiattribute method (MAM) principles to characterize the proteome of cell therapies. Here, we describe the development of a DIA method for the sensitive identification and quantification of proteins on a Q-TOF instrument. Using the improved acquisition parameters, we defined a control strategy and highlighted some metrics to improve the reproducibility of SWATH acquisition-based proteomic measurements. Finally, we applied the method to analyze the proteome of Jurkat cells that here serves as a model for human T-cells. Raw and processed data were deposited in PRIDE (PXD029780).

Methods for Proteomic Analyses of Mycobacteria

The use of proteomic technologies to characterize and study the proteome of mycobacteria has provided important information in terms of function, diversity, protein-protein interactions, and host-pathogen interactions in Mycobacterium spp. There are many different mass spectrometry methodologies that can be applied to proteomics studies of mycobacteria and microorganisms in general. Sample processing and appropriate study design are critical to generating high-quality data regardless of the mass spectrometry method applied. Appropriate study design relies on statistical rigor and data curation using bioinformatics approaches that are widely applicable regardless of the organism or system studied. Sample processing, on the other hand, is often a niched process specific to the physiology of the organism or system under investigation. Therefore, in this chapter, we will provide protocols for processing mycobacterial protein samples for the specific application of Top-down and Bottom-up proteomic analyses.

The Time Has Come for Quantitative Protein Mass Spectrometry Tests That Target Unmet Clinical Needs

Protein mass spectrometry (MS) is an enabling technology that is ideally suited for precision diagnostics. In contrast to immunoassays with indirect readouts, MS quantifications are multiplexed and include identification of proteoforms in a direct manner. Although widely used for routine measurements of drugs and metabolites, the number of clinical MS-based protein applications is limited. In this paper, we share our experience and aim to take away the concerns that have kept laboratory medicine from implementing quantitative protein MS. To ensure added value of new medical tests and guarantee accurate test results, five key elements of test evaluation have been established by a working group within the European Federation for Clinical Chemistry and Laboratory Medicine. Moreover, it is emphasized to identify clinical gaps in the contemporary clinical pathways before test development is started. We demonstrate that quantitative protein MS tests that provide an additional layer of clinical information have robust performance and meet long-term desirable analytical performance specifications as exemplified by our own experience. Yet, the adoption of quantitative protein MS tests into medical laboratories is seriously hampered due to its complexity, lack of robotization and high initial investment costs. Successful and widespread implementation in medical laboratories requires uptake and automation of this next generation protein technology by the In-Vitro Diagnostics industry. Also, training curricula of lab workers and lab specialists should include education on enabling technologies for transitioning to precision medicine by quantitative protein MS tests.

MSnbase, Efficient and Elegant R-Based Processing and Visualization of Raw Mass Spectrometry Data

We present version 2 of the MSnbase R/Bioconductor package. MSnbase provides infrastructure for the manipulation, processing, and visualization of mass spectrometry data. We focus on the new on-disk infrastructure, that allows the handling of large raw mass spectrometry experiments on commodity hardware and illustrate how the package is used for elegant data processing, method development, and visualization.

Mass Spectrometry Advances and Perspectives for the Characterization of Emerging Adoptive Cell Therapies

Adoptive cell therapy is an emerging anti-cancer modality, whereby the patient's own immune cells are engineered to express T-cell receptor (TCR) or chimeric antigen receptor (CAR). CAR-T cell therapies have advanced the furthest, with recent approvals of two treatments by the Food and Drug Administration of Kymriah (trisagenlecleucel) and Yescarta (axicabtagene ciloleucel). Recent developments in proteomic analysis by mass spectrometry (MS) make this technology uniquely suited to enable the comprehensive identification and quantification of the relevant biochemical architecture of CAR-T cell therapies and fulfill current unmet needs for CAR-T product knowledge. These advances include improved sample preparation methods, enhanced separation technologies, and extension of MS-based proteomic to single cells. Innovative technologies such as proteomic analysis of raw material quality attributes (MQA) and final product quality attributes (PQA) may provide insights that could ultimately fuel development strategies and lead to broad implementation.

Review of Issues and Solutions to Data Analysis Reproducibility and Data Quality in Clinical Proteomics

In any analytical discipline, data analysis reproducibility is closely interlinked with data quality. In this book chapter focused on mass spectrometry-based proteomics approaches, we introduce how both data analysis reproducibility and data quality can influence each other and how data quality and data analysis designs can be used to increase robustness and improve reproducibility. We first introduce methods and concepts to design and maintain robust data analysis pipelines such that reproducibility can be increased in parallel. The technical aspects related to data analysis reproducibility are challenging, and current ways to increase the overall robustness are multifaceted. Software containerization and cloud infrastructures play an important part.We will also show how quality control (QC) and quality assessment (QA) approaches can be used to spot analytical issues, reduce the experimental variability, and increase confidence in the analytical results of (clinical) proteomics studies, since experimental variability plays a substantial role in analysis reproducibility. Therefore, we give an overview on existing solutions for QC/QA, including different quality metrics, and methods for longitudinal monitoring. The efficient use of both types of approaches undoubtedly provides a way to improve the experimental reliability, reproducibility, and level of consistency in proteomics analytical measurements.

Machine Learning Reveals Protein Signatures in CSF and Plasma Fluids of Clinical Value for ALS

We use shotgun proteomics to identify biomarkers of diagnostic and prognostic value in individuals diagnosed with amyotrophic lateral sclerosis. Matched cerebrospinal and plasma fluids were subjected to abundant protein depletion and analyzed by nano-flow liquid chromatography high resolution tandem mass spectrometry. Label free quantitation was used to identify differential proteins between individuals with ALS (n = 33) and healthy controls (n = 30) in both fluids. In CSF, 118 (p-value < 0.05) and 27 proteins (q-value < 0.05) were identified as significantly altered between ALS and controls. In plasma, 20 (p-value < 0.05) and 0 (q-value < 0.05) proteins were identified as significantly altered between ALS and controls. Proteins involved in complement activation, acute phase response and retinoid signaling pathways were significantly enriched in the CSF from ALS patients. Subsequently various machine learning methods were evaluated for disease classification using a repeated Monte Carlo cross-validation approach. A linear discriminant analysis model achieved a median area under the receiver operating characteristic curve of 0.94 with an interquartile range of 0.88–1.0. Three proteins composed a prognostic model (p = 5e-4) that explained 49% of the variation in the ALS-FRS scores. Finally we investigated the specificity of two promising proteins from our discovery data set, chitinase-3 like 1 protein and alpha-1-antichymotrypsin, using targeted proteomics in a separate set of CSF samples derived from individuals diagnosed with ALS (n = 11) and other neurological diseases (n = 15). These results demonstrate the potential of a panel of targeted proteins for objective measurements of clinical value in ALS.

Quality assessment and interference detection in targeted mass spectrometry data using machine learning

Advances in the field of targeted proteomics and mass spectrometry have significantly improved assay sensitivity and multiplexing capacity. The high-throughput nature of targeted proteomics experiments has increased the rate of data production, which requires development of novel analytical tools to keep up with data processing demand. Currently, development and validation of targeted mass spectrometry assays require manual inspection of chromatographic peaks from large datasets to ensure quality, a process that is time consuming, prone to inter- and intra-operator variability and limits the efficiency of data interpretation from targeted proteomics analyses. To address this challenge, we have developed TargetedMSQC, an R package that facilitates quality control and verification of chromatographic peaks from targeted proteomics datasets. This tool calculates metrics to quantify several quality aspects of a chromatographic peak, e.g. symmetry, jaggedness and modality, co-elution and shape similarity of monitored transitions in a peak group, as well as the consistency of transitions’ ratios between endogenous analytes and isotopically labeled internal standards and consistency of retention time across multiple runs. The algorithm takes advantage of supervised machine learning to identify peaks with interference or poor chromatography based on a set of peaks that have been annotated by an expert analyst. Using TargetedMSQC to analyze targeted proteomics data reduces the time spent on manual inspection of peaks and improves both speed and accuracy of interference detection. Additionally, by allowing the analysts to customize the tool for application on different datasets, TargetedMSQC gives the users the flexibility to define the acceptable quality for specific datasets. Furthermore, automated and quantitative assessment of peak quality offers a more objective and systematic framework for high throughput analysis of targeted mass spectrometry assay datasets and is a step towards more robust and faster assay implementation.

Proteomics and Lipidomics in Inflammatory Bowel Disease Research: From Mechanistic Insights to Biomarker Identification

Inflammatory bowel disease (IBD) represents a group of progressive disorders characterized by recurrent chronic inflammation of the gut. Ulcerative colitis and Crohn's disease are the major manifestations of IBD. While our understanding of IBD has progressed in recent years, its etiology is far from being fully understood, resulting in suboptimal treatment options. Complementing other biological endpoints, bioanalytical "omics" methods that quantify many biomolecules simultaneously have great potential in the dissection of the complex pathogenesis of IBD. In this review, we focus on the rapidly evolving proteomics and lipidomics technologies and their broad applicability to IBD studies; these range from investigations of immune-regulatory mechanisms and biomarker discovery to studies dissecting host⁻microbiome interactions and the role of intestinal epithelial cells. Future studies can leverage recent advances, including improved analytical methodologies, additional relevant sample types, and integrative multi-omics analyses. Proteomics and lipidomics could effectively accelerate the development of novel targeted treatments and the discovery of complementary biomarkers, enabling continuous monitoring of the treatment response of individual patients; this may allow further refinement of treatment and, ultimately, facilitate a personalized medicine approach to IBD.

Liquid Chromatography-Mass Spectrometry Education for Clinical Laboratory Scientists

This article describes the need for, stratifies the complexity of, and proposes detailed lists of training competencies for diagnostic laboratory personnel using quantitative liquid chromatography-tandem mass spectrometry (LC-MS/MS) for patient care. Although quantitative LC-MS/MS is evolving toward greater automation with less need for technical expertise, gaps remain in resources for training and assessment.

A large-scale and robust dynamic MRM study of colorectal cancer biomarkers

Over the past 20 years, mass spectrometry (MS) has emerged as a dynamic tool for proteomics biomarker discovery. However, published MS biomarker candidates often do not translate to the clinic, failing during attempts at independent replication. The cause can be shortcomings in study design, sample quality, assay quantitation, and/or quality/process control. To address these shortcomings, we developed an MS workflow in accordance with Tier 2 measurement requirements for targeted peptides, defined by the Clinical Proteomic Tumor Analysis Consortium (CPTAC) "fit-for-purpose" approach, using dynamic multiple reaction monitoring (dMRM), which measures specific peptide transitions during predefined retention time (RT) windows. We describe the development of a robust multipex dMRM assay measuring 641 proteotypic peptides from 392 colorectal cancer (CRC) related proteins, and the procedures to track and handle sample processing and instrument variation over a four-month study, during which the assay measured blood samples from 1045 patients with CRC symptoms. After data collection, transitions were filtered by signal quality metrics before entering receiver operating characteristic (ROC) analysis. The results demonstrated CRC signal carried by 127 proteins in the symptomatic population. The workflow might be further developed to build Tier 1 assays for clinical tests identifying symptomatic individuals at elevated risk of CRC.

SIGNIFICANCE

We developed a dMRM MS method with the rigor of a Tier 2 assay as defined by the CPTAC 'fit for purpose approach' [1]. Using quality and process control procedures, the assay was used to quantify 641 proteotypic peptides representing 392 CRC-related proteins in plasma from 1045 CRC-symptomatic patients. To our knowledge, this is the largest MRM method applied to the largest study to date. The results showed that 127 of the proteins carried univariate CRC signal in the symptomatic population. This large number of single biomarkers bodes well for future development of multivariate classifiers to distinguish CRC in the symptomatic population.

Updates on resources, software tools, and databases for plant proteomics in 2016-2017

Proteomics data processing, annotation, and analysis can often lead to major hurdles in large-scale high-throughput bottom-up proteomics experiments. Given the recent rise in protein-based big datasets being generated, efforts in in silico tool development occurrences have had an unprecedented increase; so much so, that it has become increasingly difficult to keep track of all the advances in a particular academic year. However, these tools benefit the plant proteomics community in circumventing critical issues in data analysis and visualization, as these continually developing open-source and community-developed tools hold potential in future research efforts. This review will aim to introduce and summarize more than 50 software tools, databases, and resources developed and published during 2016-2017 under the following categories: tools for data pre-processing and analysis, statistical analysis tools, peptide identification tools, databases and spectral libraries, and data visualization and interpretation tools. Intended for a well-informed proteomics community, finally, efforts in data archiving and validation datasets for the community will be discussed as well. Additionally, the author delineates the current and most commonly used proteomics tools in order to introduce novice readers to this -omics discovery platform.

QCloud: A cloud-based quality control system for mass spectrometry-based proteomics laboratories

The increasing number of biomedical and translational applications in mass spectrometry-based proteomics poses new analytical challenges and raises the need for automated quality control systems. Despite previous efforts to set standard file formats, data processing workflows and key evaluation parameters for quality control, automated quality control systems are not yet widespread among proteomics laboratories, which limits the acquisition of high-quality results, inter-laboratory comparisons and the assessment of variability of instrumental platforms. Here we present QCloud, a cloud-based system to support proteomics laboratories in daily quality assessment using a user-friendly interface, easy setup, automated data processing and archiving, and unbiased instrument evaluation. QCloud supports the most common targeted and untargeted proteomics workflows, it accepts data formats from different vendors and it enables the annotation of acquired data and reporting incidences. A complete version of the QCloud system has successfully been developed and it is now open to the proteomics community (http://qcloud.crg.eu). QCloud system is an open source project, publicly available under a Creative Commons License Attribution-ShareAlike 4.0.