Target-Decoy with Mass Binning: a simple and effective validation method for shotgun proteomics using high resolution mass spectrometry

False discovery rate: the Achilles' heel of proteogenomics

Proteogenomics refers to the integrated analysis of the genome and proteome that leverages mass-spectrometry (MS)-based proteomics data to improve genome annotations, understand gene expression control through proteoforms and find sequence variants to develop novel insights for disease classification and therapeutic strategies. However, proteogenomic studies often suffer from reduced sensitivity and specificity due to inflated database size. To control the error rates, proteogenomics depends on the target-decoy search strategy, the de-facto method for false discovery rate (FDR) estimation in proteomics. The proteogenomic databases constructed from three- or six-frame nucleotide database translation not only increase the search space and compute-time but also violate the equivalence of target and decoy databases. These searches result in poorer separation between target and decoy scores, leading to stringent FDR thresholds. Understanding these factors and applying modified strategies such as two-pass database search or peptide-class-specific FDR can result in a better interpretation of MS data without introducing additional statistical biases. Based on these considerations, a user can interpret the proteogenomics results appropriately and control false positives and negatives in a more informed manner. In this review, first, we briefly discuss the proteogenomic workflows and limitations in database construction, followed by various considerations that can influence potential novel discoveries in a proteogenomic study. We conclude with suggestions to counter these challenges for better proteogenomic data interpretation.

Comparison of false-discovery rates of various decoy databases

BACKGROUND

The target-decoy strategy effectively estimates the false-discovery rate (FDR) by creating a decoy database with a size identical to that of the target database. Decoy databases are created by various methods, such as, the reverse, pseudo-reverse, shuffle, pseudo-shuffle, and the de Bruijn methods. FDR is sometimes over- or under-estimated depending on which decoy database is used because the ratios of redundant peptides in the target databases are different, that is, the numbers of unique (non-redundancy) peptides in the target and decoy databases differ.

RESULTS

We used two protein databases (the UniProt Saccharomyces cerevisiae protein database and the UniProt human protein database) to compare the FDRs of various decoy databases. When the ratio of redundant peptides in the target database is low, the FDR is not overestimated by any decoy construction method. However, if the ratio of redundant peptides in the target database is high, the FDR is overestimated when the (pseudo) shuffle decoy database is used. Additionally, human and S. cerevisiae six frame translation databases, which are large databases, also showed outcomes similar to that from the UniProt human protein database.

CONCLUSION

The FDR must be estimated using the correction factor proposed by Elias and Gygi or that by Kim et al. when (pseudo) shuffle decoy databases are used.

Noninteger Root Transformations for Preprocessing Nanoelectrospray Ionization High-Resolution Mass Spectra for the Classification of Cannabis

Typically, for measurements with a high dynamic range, the range is reduced by using the square root transform. By using noninteger roots coupled with systematic experimental design, improvements to the measurements may be obtained. The effect of using noninteger root transformation was evaluated using high-resolution mass spectrometry (HRMS) combined with nanoelectrospray ionization (Nano-ESI) to differentiate 23 samples of Cannabis. The mass spectra were evaluated and classified using different mass resolving powers and noninteger root transformations. Classification was achieved by super partial least-squares discriminant analysis (sPLS-DA), support vector machine (SVM), and SVM classification tree type entropy (SVMTreeH). The 2.5 root transformation gave the best overall performance at different resolving powers for chemical profiling from a multilevel factorial experimental design using 2 factors and more than 4 levels. Response surface modeling using a cubic polynomial model of the bootstrapped sPLS-DA average prediction accuracies yielded optima at 0.005 for resolving power and 2.3 for the root transformation. Root transformation is an important spectral preprocessing tool for decreasing the dynamic range so that the relative variance of smaller but more important features may be inflated. For the classification of Cannabis using Nano-ESI, the optimal ranges of root and resolution were broad. The chasing-the-optimum method has been introduced for refining the polynomial response surface model.

Effect of preprocessing high-resolution mass spectra on the pattern recognition of Cannabis, hemp, and liquor

High-resolution mass spectrometry (HRMS) combined with pattern recognition was used to discriminate among twenty-five Cannabis samples, twenty hemp samples, and eight liquor samples. The effects of preprocessing on multivariate data analysis were evaluated for Orbitrap high-resolution mass spectra. Different root transformations were evaluated with respect to the bin width and the average classification rates. In addition, linear binning and proportional binning with various resolving powers were studied with respect to the average classification rates. The proportional binning with the square root transformation gave the best overall performance for chemical profiling or spectral fingerprinting. Six classification methods, fuzzy rule-building expert system (FuRES), linear discriminant analysis (LDA), super partial least squares discriminant analysis (sPLS-DA), support vector machine (SVM), SVM classification tree type gap (SVMTreeG), and SVM classification tree type entropy (SVMTreeH) had similar trends in prediction rate with respect to the resolving power. The optimal proportional mass bin width may depend on the data set, i.e., for the Cannabis samples is resolving power 10^-4, for the hemp samples and the liquor samples are resolving power 10^-3. Hence, data preprocessing methods such as different transformations, binning strategies, and resolving powers are important factors to be optimized for HRMS direct infusion measurements combined with pattern recognition to be an authentication and characterization tool for various products.

Evaluating the effect of database inflation in proteogenomic search on sensitive and reliable peptide identification

Background

Proteogenomics is a promising approach for various tasks ranging from gene annotation to cancer research. Databases for proteogenomic searches are often constructed by adding peptide sequences inferred from genomic or transcriptomic evidence to reference protein sequences. Such inflation of databases has potential of identifying novel peptides. However, it also raises concerns on sensitive and reliable peptide identification. Spurious peptides included in target databases may result in underestimated false discovery rate (FDR). On the other hand, inflation of decoy databases could decrease the sensitivity of peptide identification due to the increased number of high-scoring random hits. Although several studies have addressed these issues, widely applicable guidelines for sensitive and reliable proteogenomic search have hardly been available.

Results

To systematically evaluate the effect of database inflation in proteogenomic searches, we constructed a variety of real and simulated proteogenomic databases for yeast and human tandem mass spectrometry (MS/MS) data, respectively. Against these databases, we tested two popular database search tools with various approaches to search result validation: the target-decoy search strategy (with and without a refined scoring-metric) and a mixture model-based method. The effect of separate filtering of known and novel peptides was also examined. The results from real and simulated proteogenomic searches confirmed that separate filtering increases the sensitivity and reliability in proteogenomic search. However, no one method consistently identified the largest (or the smallest) number of novel peptides from real proteogenomic searches.

Conclusions

We propose to use a set of search result validation methods with separate filtering, for sensitive and reliable identification of peptides in proteogenomic search.

Electronic supplementary material

The online version of this article (doi:10.1186/s12864-016-3327-5) contains supplementary material, which is available to authorized users.

Learning from decoys to improve the sensitivity and specificity of proteomics database search results

The statistical validation of database search results is a complex issue in bottom-up proteomics. The correct and incorrect peptide spectrum match (PSM) scores overlap significantly, making an accurate assessment of true peptide matches challenging. Since the complete separation between the true and false hits is practically never achieved, there is need for better methods and rescoring algorithms to improve upon the primary database search results. Here we describe the calibration and False Discovery Rate (FDR) estimation of database search scores through a dynamic FDR calculation method, FlexiFDR, which increases both the sensitivity and specificity of search results. Modelling a simple linear regression on the decoy hits for different charge states, the method maximized the number of true positives and reduced the number of false negatives in several standard datasets of varying complexity (18-mix, 49-mix, 200-mix) and few complex datasets (E. coli and Yeast) obtained from a wide variety of MS platforms. The net positive gain for correct spectral and peptide identifications was up to 14.81% and 6.2% respectively. The approach is applicable to different search methodologies- separate as well as concatenated database search, high mass accuracy, and semi-tryptic and modification searches. FlexiFDR was also applied to Mascot results and showed better performance than before. We have shown that appropriate threshold learnt from decoys, can be very effective in improving the database search results. FlexiFDR adapts itself to different instruments, data types and MS platforms. It learns from the decoy hits and sets a flexible threshold that automatically aligns itself to the underlying variables of data quality and size.

A survey of computational methods and error rate estimation procedures for peptide and protein identification in shotgun proteomics

This manuscript provides a comprehensive review of the peptide and protein identification process using tandem mass spectrometry (MS/MS) data generated in shotgun proteomic experiments. The commonly used methods for assigning peptide sequences to MS/MS spectra are critically discussed and compared, from basic strategies to advanced multi-stage approaches. A particular attention is paid to the problem of false-positive identifications. Existing statistical approaches for assessing the significance of peptide to spectrum matches are surveyed, ranging from single-spectrum approaches such as expectation values to global error rate estimation procedures such as false discovery rates and posterior probabilities. The importance of using auxiliary discriminant information (mass accuracy, peptide separation coordinates, digestion properties, and etc.) is discussed, and advanced computational approaches for joint modeling of multiple sources of information are presented. This review also includes a detailed analysis of the issues affecting the interpretation of data at the protein level, including the amplification of error rates when going from peptide to protein level, and the ambiguities in inferring the identifies of sample proteins in the presence of shared peptides. Commonly used methods for computing protein-level confidence scores are discussed in detail. The review concludes with a discussion of several outstanding computational issues.