Statistical properties of an early stopping rule for resampling-based multiple testing

Two-sample test with g-modeling and its applications

Many real data analyses involve two-sample comparisons in location or in distribution. Most existing methods focus on problems where observations are independently and identically distributed in each group. However, in some applications the observed data are not identically distributed but associated with some unobserved parameters which are identically distributed. To address this challenge, we propose a novel two-sample testing procedure as a combination of the g $$ g $$ -modeling density estimation introduced by Efron and the two-sample Kolmogorov-Smirnov test. We also propose efficient bootstrap algorithms to estimate the statistical significance for such tests. We demonstrate the utility of the proposed approach with two biostatistical applications: the analysis of surgical nodes data with binomial model and differential expression analysis of single-cell RNA sequencing data with zero-inflated Poisson model.

Uncertainty quantification in ToxCast high throughput screening

High throughput screening (HTS) projects like the U.S. Environmental Protection Agency's ToxCast program are required to address the large and rapidly increasing number of chemicals for which we have little to no toxicity measurements. Concentration-response parameters such as potency and efficacy are extracted from HTS data using nonlinear regression, and models and analyses built from these parameters are used to predict in vivo and in vitro toxicity of thousands of chemicals. How these predictions are impacted by uncertainties that stem from parameter estimation and propagated through the models and analyses has not been well explored. While data size and complexity makes uncertainty quantification computationally expensive for HTS datasets, continued advancements in computational resources have allowed these computational challenges to be met. This study uses nonparametric bootstrap resampling to calculate uncertainties in concentration-response parameters from a variety of HTS assays. Using the ToxCast estrogen receptor model for bioactivity as a case study, we highlight how these uncertainties can be propagated through models to quantify the uncertainty in model outputs. Uncertainty quantification in model outputs is used to identify potential false positives and false negatives and to determine the distribution of model values around semi-arbitrary activity cutoffs, increasing confidence in model predictions. At the individual chemical-assay level, curves with high variability are flagged for manual inspection or retesting, focusing subject-matter-expert time on results that need further input. This work improves the confidence of predictions made using HTS data, increasing the ability to use this data in risk assessment.

Fast approximation of small p-values in permutation tests by partitioning the permutations

Researchers in genetics and other life sciences commonly use permutation tests to evaluate differences between groups. Permutation tests have desirable properties, including exactness if data are exchangeable, and are applicable even when the distribution of the test statistic is analytically intractable. However, permutation tests can be computationally intensive. We propose both an asymptotic approximation and a resampling algorithm for quickly estimating small permutation p-values (e.g., <10-6) for the difference and ratio of means in two-sample tests. Our methods are based on the distribution of test statistics within and across partitions of the permutations, which we define. In this article, we present our methods and demonstrate their use through simulations and an application to cancer genomic data. Through simulations, we find that our resampling algorithm is more computationally efficient than another leading alternative, particularly for extremely small p-values (e.g., <10-30). Through application to cancer genomic data, we find that our methods can successfully identify up- and down-regulated genes. While we focus on the difference and ratio of means, we speculate that our approaches may work in other settings.

CLME: An R Package for Linear Mixed Effects Models under Inequality Constraints

In many applications researchers are typically interested in testing for inequality constraints in the context of linear fixed effects and mixed effects models. Although there exists a large body of literature for performing statistical inference under inequality constraints, user friendly statistical software for implementing such methods is lacking, especially in the context of linear fixed and mixed effects models. In this article we introduce CLME, a package in the R language that can be used for testing a broad collection of inequality constraints. It uses residual bootstrap based methodology which is reasonably robust to non-normality as well as heteroscedasticity. The package is illustrated using two data sets. The package also contains a graphical interface built using the shiny package.

Evaluation of Spectral Counting for Relative Quantitation of Proteoforms in Top-Down Proteomics

Spectral counting is a straightforward label-free quantitation strategy used in bottom-up proteomics workflows. The application of spectral counting in label-free top-down proteomics workflows can be similarly straightforward but has not been applied as widely as quantitation by chromatographic peak areas or peak intensities. In this study, we evaluate spectral counting for quantitative comparisons in label-free top-down proteomics workflows by comparison with chromatographic peak areas and intensities. We tested these quantitation approaches by spiking standard proteins into a complex protein background and comparing relative quantitation by spectral counts with normalized chromatographic peak areas and peak intensities from deconvoluted extracted ion chromatograms of the spiked proteins. Ratio estimates and statistical significance of differential abundance from each quantitation technique are evaluated against the expected ratios and each other. In this experiment, spectral counting was able to detect differential abundance of spiked proteins for expected ratios ≥2, with comparable or higher sensitivity than normalized areas and intensities. We also found that while ratio estimates using peak areas and intensities are usually more accurate, the spectral-counting-based estimates are not substantially worse. Following the evaluation and comparison of these label-free top-down quantitation strategies using spiked proteins, spectral counting, along with normalized chromatographic peak areas and intensities, were used to analyze the complex protein cargo of exosomes shed by myeloid-derived suppressor cells collected under high and low conditions of inflammation, revealing statistically significant differences in abundance for several proteoforms, including the active pro-inflammatory proteins S100A8 and S100A9.

IsoDOT Detects Differential RNA-isoform Expression/Usage with respect to a Categorical or Continuous Covariate with High Sensitivity and Specificity

We have developed a statistical method named IsoDOT to assess differential isoform expression (DIE) and differential isoform usage (DIU) using RNA-seq data. Here isoform usage refers to relative isoform expression given the total expression of the corresponding gene. IsoDOT performs two tasks that cannot be accomplished by existing methods: to test DIE/DIU with respect to a continuous covariate, and to test DIE/DIU for one case versus one control. The latter task is not an uncommon situation in practice, e.g., comparing the paternal and maternal alleles of one individual or comparing tumor and normal samples of one cancer patient. Simulation studies demonstrate the high sensitivity and specificity of IsoDOT. We apply IsoDOT to study the effects of haloperidol treatment on the mouse transcriptome and identify a group of genes whose isoform usages respond to haloperidol treatment.

Quantitative high-throughput screening data analysis: challenges and recent advances

In vitro HTS holds much potential to advance drug discovery and provide cell-based alternatives for toxicity testing. In quantitative HTS, concentration-response data can be generated simultaneously for thousands of different compounds and mixtures. However, nonlinear modeling in these multiple-concentration assays presents important statistical challenges that are not problematic for linear models. The uncertainty of parameter estimates obtained from the widely used Hill equation model can be extremely large when using standard designs. Failure to properly consider standard errors of these parameter estimates would greatly hinder chemical genomics and toxicity testing efforts. In this light, optimal study designs should be developed to improve nonlinear parameter estimation; or alternative approaches with reliable performance characteristics should be used to describe concentration-response profiles.

rSeqDiff: detecting differential isoform expression from RNA-Seq data using hierarchical likelihood ratio test

High-throughput sequencing of transcriptomes (RNA-Seq) has recently become a powerful tool for the study of gene expression. We present rSeqDiff, an efficient algorithm for the detection of differential expression and differential splicing of genes from RNA-Seq experiments across multiple conditions. Unlike existing approaches which detect differential expression of transcripts, our approach considers three cases for each gene: 1) no differential expression, 2) differential expression without differential splicing and 3) differential splicing. We specify statistical models characterizing each of these three cases and use hierarchical likelihood ratio test for model selection. Simulation studies show that our approach achieves good power for detecting differentially expressed or differentially spliced genes. Comparisons with competing methods on two real RNA-Seq datasets demonstrate that our approach provides accurate estimates of isoform abundances and biological meaningful rankings of differentially spliced genes. The proposed approach is implemented as an R package named rSeqDiff.