Adaptive choice of the number of bootstrap samples in large scale multiple testing

Evaluation of statistical approaches for association testing in noisy drug screening data

BACKGROUND

Identifying associations among biological variables is a major challenge in modern quantitative biological research, particularly given the systemic and statistical noise endemic to biological systems. Drug sensitivity data has proven to be a particularly challenging field for identifying associations to inform patient treatment.

RESULTS

To address this, we introduce two semi-parametric variations on the commonly used concordance index: the robust concordance index and the kernelized concordance index (rCI, kCI), which incorporate measurements about the noise distribution from the data. We demonstrate that common statistical tests applied to the concordance index and its variations fail to control for false positives, and introduce efficient implementations to compute p-values using adaptive permutation testing. We then evaluate the statistical power of these coefficients under simulation and compare with Pearson and Spearman correlation coefficients. Finally, we evaluate the various statistics in matching drugs across pharmacogenomic datasets.

CONCLUSIONS

We observe that the rCI and kCI are better powered than the concordance index in simulation and show some improvement on real data. Surprisingly, we observe that the Pearson correlation was the most robust to measurement noise among the different metrics.

Nomogram prediction model for renal anaemia in IgA nephropathy patients

In this study, we focused on the influencing factors of renal anaemia in patients with IgA nephropathy and constructed a nomogram model. We divided 462 patients with IgA nephropathy diagnosed by renal biopsy into anaemic and non-anaemic groups. Then, the influencing factors of renal anaemia in patients with IgA nephropathy were analysed by least absolute shrinkage and selection operator (LASSO) regression and multivariable logistic regression, and a nomogram model for predicting renal anaemia was established. Eventually, nine variables were obtained, which are easy to apply clinically. The areas under the receiver operating characteristic (ROC) curve and precision-recall (PR) curve reached 0.835 and 0.676, respectively, and the C-index reached 0.848. The calibration plot showed that the model had good discrimination, accuracy, and diagnostic efficacy. In addition, the C-index of the model following internal validation reached 0.823. Decision curve analysis suggested that the model had a certain degree of clinical significance. This new nomogram model of renal anaemia combines the basic information, laboratory findings, and renal biopsy results of patients with IgA nephropathy, providing important guidance for predicting and clinically intervening in renal anaemia.

Renal Cell Carcinomas in Vinylidene Chloride-exposed Male B6C3F1 Mice Are Characterized by Oxidative Stress and TP53 Pathway Dysregulation

Vinylidene chloride (VDC) has been widely used in the production of plastics and flame retardants. Exposure of B6C3F1 mice to VDC in the 2-year National Toxicology Program carcinogenicity bioassay resulted in a dose-dependent increases in renal cell hyperplasia, renal cell adenoma, and renal cell carcinomas (RCCs). Among those differentially expressed genes from controls and RCC of VDC-exposed mice, there was an overrepresentation of genes from pathways associated with chronic xenobiotic and oxidative stress as well as c-Myc overexpression and dysregulation of TP53 cell cycle checkpoint and DNA damage repair pathways in RCC. Trend analysis comparing RCC, VDC-exposed kidney, and chamber control kidney showed a conservation of pathway dysregulation in terms of overrepresentation of xenobiotic and oxidative stress, and DNA damage and cell cycle checkpoint pathways in both VDC-exposed kidney and RCC, suggesting that these mechanisms play a role in the pathogenesis of RCC in VDC-exposed mice.

Large-scale multiple testing in genome-wide association studies via region-specific hidden Markov models

BACKGROUND

Identifying genetic variants associated with complex human diseases is a great challenge in genome-wide association studies (GWAS). Single nucleotide polymorphisms (SNPs) arising from genetic background are often dependent. The existing methods, i.e., local index of significance (LIS) and pooled local index of significance (PLIS), were both proposed for modeling SNP dependence and assumed that the whole chromosome follows a hidden Markov model (HMM). However, the fact that SNP data are often collected from separate heterogeneous regions of a single chromosome encourages different chromosomal regions to follow different HMMs. In this research, we developed a data-driven penalized criterion combined with a dynamic programming algorithm to find change points that divide the whole chromosome into more homogeneous regions. Furthermore, we extended PLIS to analyze the dependent tests obtained from multiple chromosomes with different regions for GWAS.

RESULTS

The simulation results show that our new criterion can improve the performance of the model selection procedure and that our region-specific PLIS (RSPLIS) method is better than PLIS at detecting disease-associated SNPs when there are multiple change points along a chromosome. Our method has been used to analyze the Daly study, and compared with PLIS, RSPLIS yielded results that more accurately detected disease-associated SNPs.

CONCLUSIONS

The genomic rankings based on our method differ from the rankings based on PLIS. Specifically, for the detection of genetic variants with weak effect sizes, the RSPLIS method was able to rank them more efficiently and with greater power.

Pooling/bootstrap-based GWAS (pbGWAS) identifies new loci modifying the age of onset in PSEN1 p.Glu280Ala Alzheimer's disease

The literature on GWAS (genome-wide association studies) data suggests that very large sample sizes (for example, 50,000 cases and 50,000 controls) may be required to detect significant associations of genomic regions for complex disorders such as Alzheimer's disease (AD). Because of the challenges of obtaining such large cohorts, we describe here a novel sequential strategy that combines pooling of DNA and bootstrapping (pbGWAS) in order to significantly increase the statistical power and exponentially reduce expenses. We applied this method to a very homogeneous sample of patients belonging to a unique and clinically well-characterized multigenerational pedigree with one of the most severe forms of early onset AD, carrying the PSEN1 p.Glu280Ala mutation (often referred to as E280A mutation), which originated as a consequence of a founder effect. In this cohort, we identified novel loci genome-wide significantly associated as modifiers of the age of onset of AD (CD44, rs187116, P=1.29 × 10⁻¹²; NPHP1, rs10173717, P=1.74 × 10⁻¹²; CADPS2, rs3757536, P=1.54 × 10⁻¹⁰; GREM2, rs12129547, P=1.69 × 10⁻¹³, among others) as well as other loci known to be associated with AD. Regions identified by pbGWAS were confirmed by subsequent individual genotyping. The pbGWAS methodology and the genes it targeted could provide important insights in determining the genetic causes of AD and other complex conditions.

Statistical properties of an early stopping rule for resampling-based multiple testing

Resampling-based methods for multiple hypothesis testing often lead to long run times when the number of tests is large. This paper presents a simple rule that substantially reduces computation by allowing resampling to terminate early on a subset of tests. We prove that the method has a low probability of obtaining a set of rejected hypotheses different from those rejected without early stopping, and obtain error bounds for multiple hypothesis testing. Simulation shows that our approach saves more computation than other available procedures.

How many bootstrap replicates are necessary?

Phylogenetic bootstrapping (BS) is a standard technique for inferring confidence values on phylogenetic trees that is based on reconstructing many trees from minor variations of the input data, trees called replicates. BS is used with all phylogenetic reconstruction approaches, but we focus here on one of the most popular, maximum likelihood (ML). Because ML inference is so computationally demanding, it has proved too expensive to date to assess the impact of the number of replicates used in BS on the relative accuracy of the support values. For the same reason, a rather small number (typically 100) of BS replicates are computed in real-world studies. Stamatakis et al. recently introduced a BS algorithm that is 1 to 2 orders of magnitude faster than previous techniques, while yielding qualitatively comparable support values, making an experimental study possible. In this article, we propose stopping criteria--that is, thresholds computed at runtime to determine when enough replicates have been generated--and we report on the first large-scale experimental study to assess the effect of the number of replicates on the quality of support values, including the performance of our proposed criteria. We run our tests on 17 diverse real-world DNA--single-gene as well as multi-gene--datasets, which include 125-2,554 taxa. We find that our stopping criteria typically stop computations after 100-500 replicates (although the most conservative criterion may continue for several thousand replicates) while producing support values that correlate at better than 99.5% with the reference values on the best ML trees. Significantly, we also find that the stopping criteria can recommend very different numbers of replicates for different datasets of comparable sizes. Our results are thus twofold: (i) they give the first experimental assessment of the effect of the number of BS replicates on the quality of support values returned through BS, and (ii) they validate our proposals for stopping criteria. Practitioners will no longer have to enter a guess nor worry about the quality of support values; moreover, with most counts of replicates in the 100-500 range, robust BS under ML inference becomes computationally practical for most datasets. The complete test suite is available at http://lcbb.epfl.ch/BS.tar.bz2, and BS with our stopping criteria is included in the latest release of RAxML v7.2.5, available at http://wwwkramer.in.tum.de/exelixis/software.html.

Analysis of Correlated Gene Expression Data on Ordered Categories

A bootstrap based methodology is introduced for analyzing repeated measures/longitudinal microarray gene expression data over ordered categories. The proposed non-parametric procedure uses order-restricted inference to compare gene expressions among ordered experimental conditions. The null distribution for determining significance is derived by suitably bootstrapping the residuals. The procedure addresses two potential sources of correlation in the data, namely, (a) correlations among genes within a chip ("intra-chip" correlation), and (b) correlation within subject due to repeated/longitudinal measurements ("temporal" correlation). To make the procedure computationally efficient, the adaptive bootstrap methodology of Guo and Peddada (2008) is implemented such that the resulting procedure controls the false discovery rate (FDR) at the desired nominal level.

Multiple testing in genome-wide association studies via hidden Markov models

MOTIVATION

Genome-wide association studies (GWAS) interrogate common genetic variation across the entire human genome in an unbiased manner and hold promise in identifying genetic variants with moderate or weak effect sizes. However, conventional testing procedures, which are mostly P-value based, ignore the dependency and therefore suffer from loss of efficiency. The goal of this article is to exploit the dependency information among adjacent single nucleotide polymorphisms (SNPs) to improve the screening efficiency in GWAS.

RESULTS

We propose to model the linear block dependency in the SNP data using hidden Markov models (HMMs). A compound decision-theoretic framework for testing HMM-dependent hypotheses is developed. We propose a powerful data-driven procedure [pooled local index of significance (PLIS)] that controls the false discovery rate (FDR) at the nominal level. PLIS is shown to be optimal in the sense that it has the smallest false negative rate (FNR) among all valid FDR procedures. By re-ranking significance for all SNPs with dependency considered, PLIS gains higher power than conventional P-value based methods. Simulation results demonstrate that PLIS dominates conventional FDR procedures in detecting disease-associated SNPs. Our method is applied to analysis of the SNP data from a GWAS of type 1 diabetes. Compared with the Benjamini-Hochberg (BH) procedure, PLIS yields more accurate results and has better reproducibility of findings.

CONCLUSION

The genomic rankings based on our procedure are substantially different from the rankings based on the P-values. By integrating information from adjacent locations, the PLIS rankings benefit from the increased signal-to-noise ratio, hence our procedure often has higher statistical power and better reproducibility. It provides a promising direction in large-scale GWAS.

AVAILABILITY

An R package PLIS has been developed to implement the PLIS procedure. Source codes are available upon request and will be available on CRAN (http://cran.r-project.org/).

CONTACT

zhiwei@njit.edu

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.