Numerical computation of multivariatet-probabilities with application to power calculation of multiple contrasts

Sample size planning for multiple contrast tests

Sample size calculations for two (independent) samples are well established and applied in (pre-)clinical research. When planning several samples, which is common in, for example, preclinical studies, sample size planning tools based on analysis of variance methods are available. Since the underlying effect sizes of these methods are often hard to interpret and to provide for the sample size planning, we employ multiple contrast test procedures for sample size computations in both parametric (under normality assumption) and nonparametric designs using Steel-type tests. Since the exact distributions of the test statistics are unknown under the alternative and variance heterogeneity, we use approximate solutions. Furthermore, since no closed formula for the sample size is available, we use numerical approximations for their computation. Extensive simulation studies are finally conducted to assess the quality of the approximations. It turns out that the methods are accurate in the sense that the multiple contrast test procedures reach the target power to detect the alternative of interest with the sample size computed. The developed procedures are a valuable tool to plan (pre-)clinical trials with several samples and are easily accessible in publicly available software.

Neural specialization to English words in Chinese children: Joint contribution of age and English reading abilities

N1 tuning to words, a neural marker of visual word recognition, develops by an interaction between age and ability. The development of N1 tuning to a second learnt print is unclear. The present study examined the joint contribution of age and English reading abilities to N1 amplitude and tuning to English print in Chinese children in Hong Kong. EEG signals were recorded from 179 children (six to nine years old) while they were performing a repetition detection task comprised of different print stimuli measuring three types of tuning, i.e., coarse tuning (real word versus false font), fine tuning (real versus nonword), and lexicality effect (real versus pseudo word). Children were assessed in English word reading accuracy (EWR) and English sub-lexical orthographic knowledge (EOK). Results indicated that coarse tuning decreased with age but increased with EWR and EOK. Fine tuning uniquely increased with EOK, and the lexicality effect increased with EWR. At last, higher EWR was linked to less right-lateralized coarse tuning in younger children. Taken together, the findings support the visual perceptual expertise account in the L2 context, in that N1 coarse tuning, fine tuning, and lexicality effect are driven by skill improvement.

On dependence assumption in p-value based multiple test procedures

There are various multiple comparison procedures used in confirmatory clinical studies and exploratory research for multiplicity adjustment. Among them are the Hochberg and Benjamini-Hochberg procedures. A common misconception is that these procedures control the type I error rate properly if the test statistics are independent or positively correlated. In fact, a much stronger positive dependence assumption needs to be satisfied to guarantee the type I error rate control. We give a comprehensive review of the dependence conditions used in multiple testing procedures. We show that a weaker positive dependence assumption may result an inflation of type I error rate by a factor of 2 and discuss the type I error rate control under certain negative dependence conditions.

Tractable Bayes of skew-elliptical link models for correlated binary data

Correlated binary response data with covariates are ubiquitous in longitudinal or spatial studies. Among the existing statistical models, the most well-known one for this type of data is the multivariate probit model, which uses a Gaussian link to model dependence at the latent level. However, a symmetric link may not be appropriate if the data are highly imbalanced. Here, we propose a multivariate skew-elliptical link model for correlated binary responses, which includes the multivariate probit model as a special case. Furthermore, we perform Bayesian inference for this new model and prove that the regression coefficients have a closed-form unified skew-elliptical posterior with an elliptical prior. The new methodology is illustrated by an application to COVID-19 data from three different counties of the state of California, USA. By jointly modeling extreme spikes in weekly new cases, our results show that the spatial dependence cannot be neglected. Furthermore, the results also show that the skewed latent structure of our proposed model improves the flexibility of the multivariate probit model and provides a better fit to our highly imbalanced dataset.

An Information Theoretic Approach to Symbolic Learning in Synthetic Languages

An important aspect of using entropy-based models and proposed “synthetic languages”, is the seemingly simple task of knowing how to identify the probabilistic symbols. If the system has discrete features, then this task may be trivial; however, for observed analog behaviors described by continuous values, this raises the question of how we should determine such symbols. This task of symbolization extends the concept of scalar and vector quantization to consider explicit linguistic properties. Unlike previous quantization algorithms where the aim is primarily data compression and fidelity, the goal in this case is to produce a symbolic output sequence which incorporates some linguistic properties and hence is useful in forming language-based models. Hence, in this paper, we present methods for symbolization which take into account such properties in the form of probabilistic constraints. In particular, we propose new symbolization algorithms which constrain the symbols to have a Zipf–Mandelbrot–Li distribution which approximates the behavior of language elements. We introduce a novel constrained EM algorithm which is shown to effectively learn to produce symbols which approximate a Zipfian distribution. We demonstrate the efficacy of the proposed approaches on some examples using real world data in different tasks, including the translation of animal behavior into a possible human language understandable equivalent.

Role of peer support in a hepatitis C elimination programme

Many people with chronic hepatitis C infection don't engage in treatment. To eliminate hepatitis C and avoid health inequalities therapy must be provided to everyone. In other diseases peers with lived experience of the condition have improved care but, for hepatitis C, studies have not shown unequivocal benefit. We completed a retrospective analysis of the English National Health Service treatment registry comparing treatment networks with and without peers using Bayesian Poisson (for count outcomes) or Bayesian Binomial (for proportion outcomes) mixed effects models with time fixed effects. For each outcome, we estimated relative ratio (RR-Poisson model) or odds ratio (Odds Ratio (OR)-Binomial model) between peer and non-peer networks. We analysed 30,729 patients within 20 operational delivery networks. In networks with peers there was an increase in the number of people initiating therapy (RR 1.12 95%, credible interval 1.02-1.21) and an increase in the proportion completing therapy (OR 2.45 95%, credible interval 1.49-3.84). However, we saw no change in proportions of people using drugs who initiated therapy nor any significant change in virological response (OR 1.14 95% credible interval 0.979-1.36). We repeated the analysis looking at the impact of peers two months after they had been introduced, when they had established networks of contacts, and saw an increase in the proportion of people treated in addiction services. In treating patients with chronic hepatitis C infection the inclusion of peer supporters may increase the number of people who initiate and complete antiviral therapy.

A two-stage drop-the-losers design for time-to-event outcome using a historical control arm

Phase II immuno-oncology clinical trials screen for efficacy an increasing number of treatments. In rare cancers, using historical control data is a pragmatic approach for speeding up clinical trials. The drop-the-losers design allows dropping off ineffective arms at interim analyses. We extended the original drop-the-losers design for a time-to-event outcome using a historical control through the one-sample log-rank statistic. Simulated trials featured three arms at the first stage, one at the second stage, nine scenarios, eight sample sizes with 5%- and 10%- nominal family-wise error rate (FWER). A numerical algorithm is provided to solve power calculations at the design stage. Our design was compared with a group of three independent single-arm trials (fixed design) with and without correction for multiplicity. Our design allowed strict control of the FWER at nominal levels while the misspecification of survival distribution and fixed design inflated the FWER up to three times the nominal level. The empirical power of our design increased with the sample size, the treatment effect and the number of effective treatments and dropped when more patients were recruited at the second stage. The fixed design with correction showed comparable power, while our design advantageously included more patients to the most promising arm. Recommendations for future applications are given. By taking advantage of the use of historical control data and a time-to-event outcome, the drop-the-losers design is a promising tool to meet the challenge of improving phase II clinical trials in immuno-oncology.

An optimal kernel-based multivariate U-statistic to test for associations with multiple phenotypes

Set-based analysis that jointly considers multiple predictors in a group has been broadly conducted for association tests. However, their power can be sensitive to the distribution of phenotypes, and the underlying relationships between predictors and outcomes. Moreover, most of the set-based methods are designed for single-trait analysis, making it hard to explore the pleiotropic effect and borrow information when multiple phenotypes are available. Here, we propose a kernel-based multivariate U-statistics (KMU) that is robust and powerful in testing the association between a set of predictors and multiple outcomes. We employed a rank-based kernel function for the outcomes, which makes our method robust to various outcome distributions. Rather than selecting a single kernel, our test statistics is built based on multiple kernels selected in a data-driven manner, and thus is capable of capturing various complex relationships between predictors and outcomes. The asymptotic properties of our test statistics have been developed. Through simulations, we have demonstrated that KMU has controlled type I error and higher power than its counterparts. We further showed its practical utility by analyzing a whole genome sequencing data from Alzheimer's Disease Neuroimaging Initiative study, where novel genes have been detected to be associated with imaging phenotypes.

Questionable research practices may have little effect on replicability

This article examines why many studies fail to replicate statistically significant published results. We address this issue within a general statistical framework that also allows us to include various questionable research practices (QRPs) that are thought to reduce replicability. The analyses indicate that the base rate of true effects is the major factor that determines the replication rate of scientific results. Specifically, for purely statistical reasons, replicability is low in research domains where true effects are rare (e.g., search for effective drugs in pharmacology). This point is under-appreciated in current scientific and media discussions of replicability, which often attribute poor replicability mainly to QRPs.

Testing multiple dose combinations in clinical trials

Drug combination trials are often motivated by the fact that individual drugs target the same disease but via different routes. A combination of such drugs may then have an overall better effect than the individual treatments which has to be verified by clinical trials. Several statistical methods have been explored that discuss the problem of comparing a fixed-dose combination therapy to each of its components. But an extension of these approaches to multiple dose combinations can be difficult and is not yet fully investigated. In this paper, we propose two approaches by which one can provide confirmatory assurance with familywise error rate control, that the combination of two drugs at differing doses is more effective than either component doses alone. These approaches involve multiple comparisons in multilevel factorial designs where the type 1 error can be controlled first, by bootstrapping tests, and second, by considering the least favorable null configurations for a family of union intersection tests. The main advantage of the new approaches is that their implementation is simple. The implementation of these new approaches is illustrated with a real data example from a blood pressure reduction trial. Extensive simulations are also conducted to evaluate the new approaches and benchmark them with existing ones. We also present an illustration of the relationship between the different approaches. We observed that the bootstrap provided some power advantages over the other approaches with the disadvantage that there may be some error rate inflation for small sample sizes.

Comparing a stratified treatment strategy with the standard treatment in randomized clinical trials

The increasing emergence of predictive markers for different treatments in the same patient population allows us to define stratified treatment strategies. We consider randomized clinical trials that compare a standard treatment with a new stratified treatment strategy that divides the study population into subgroups receiving different treatments. Because the new strategy may not be beneficial in all subgroups, we consider in this paper an intermediate approach that establishes a treatment effect in a subset of patients built by joining several subgroups. The approach is based on the simple idea of selecting the subset with minimal p-value when testing the subset-specific treatment effects. We present a framework to compare this approach with other approaches to select subsets by introducing three performance measures. The results of a comprehensive simulation study are presented, and the relative merits of the various approaches are discussed. Copyright © 2016 John Wiley & Sons, Ltd.

Approximate Bayesian inference in semi-mechanistic models

Inference of interaction networks represented by systems of differential equations is a challenging problem in many scientific disciplines. In the present article, we follow a semi-mechanistic modelling approach based on gradient matching. We investigate the extent to which key factors, including the kinetic model, statistical formulation and numerical methods, impact upon performance at network reconstruction. We emphasize general lessons for computational statisticians when faced with the challenge of model selection, and we assess the accuracy of various alternative paradigms, including recent widely applicable information criteria and different numerical procedures for approximating Bayes factors. We conduct the comparative evaluation with a novel inferential pipeline that systematically disambiguates confounding factors via an ANOVA scheme.

An approximate marginal logistic distribution for the analysis of longitudinal ordinal data

Subject-specific and marginal models have been developed for the analysis of longitudinal ordinal data. Subject-specific models often lack a population-average interpretation of the model parameters due to the conditional formulation of random intercepts and slopes. Marginal models frequently lack an underlying distribution for ordinal data, in particular when generalized estimating equations are applied. To overcome these issues, latent variable models underneath the ordinal outcomes with a multivariate logistic distribution can be applied. In this article, we extend the work of O'Brien and Dunson (2004), who studied the multivariate t-distribution with marginal logistic distributions. We use maximum likelihood, instead of a Bayesian approach, and incorporated covariates in the correlation structure, in addition to the mean model. We compared our method with GEE and demonstrated that it performs better than GEE with respect to the fixed effect parameter estimation when the latent variables have an approximately elliptical distribution, and at least as good as GEE for other types of latent variable distributions.

Multiple comparisons for survival data with propensity score adjustment

This article considers the practical problem in clinical and observational studies where multiple treatment or prognostic groups are compared and the observed survival data are subject to right censoring. Two possible formulations of multiple comparisons are suggested. Multiple Comparisons with a Control (MCC) compare every other group to a control group with respect to survival outcomes, for determining which groups are associated with lower risk than the control. Multiple Comparisons with the Best (MCB) compare each group to the truly minimum risk group and identify the groups that are either with the minimum risk or the practically minimum risk. To make a causal statement, potential confounding effects need to be adjusted in the comparisons. Propensity score based adjustment is popular in causal inference and can effectively reduce the confounding bias. Based on a propensity-score-stratified Cox proportional hazards model, the approaches of MCC test and MCB simultaneous confidence intervals for general linear models with normal error outcome are extended to survival outcome. This paper specifies the assumptions for causal inference on survival outcomes within a potential outcome framework, develops testing procedures for multiple comparisons and provides simultaneous confidence intervals. The proposed methods are applied to two real data sets from cancer studies for illustration, and a simulation study is also presented.

Demonstrator skill modulates observational aversive learning

Learning to avoid danger by observing others can be relatively safe, because it does not incur the potential costs of individual trial and error. However, information gained through social observation might be less reliable than information gained through individual experiences, underscoring the need to apply observational learning critically. In order for observational learning to be adaptive it should be modulated by the skill of the observed person, the demonstrator. To address this issue, we used a probabilistic two-choice task where participants learned to minimize the number of electric shocks through individual learning and by observing a demonstrator performing the same task. By manipulating the demonstrator's skill we varied how useful the observable information was; the demonstrator either learned the task quickly or did not learn it at all (random choices). To investigate the modulatory effect in detail, the task was performed under three conditions of available observable information; no observable information, observation of choices only, and observation of both the choices and their consequences. As predicted, our results showed that observable information can improve performance compared to individual learning, both when the demonstrator is skilled and unskilled; observation of consequences improved performance for both groups while observation of choices only improved performance for the group observing the skilled demonstrator. Reinforcement learning modeling showed that demonstrator skill modulated observational learning from the demonstrator's choices, but not their consequences, by increasing the degree of imitation over time for the group that observed a fast learner. Our results show that humans can adaptively modulate observational learning in response to the usefulness of observable information.

Multiplicity adjustments in testing for bioequivalence

Bioequivalence of two drugs is usually demonstrated by rejecting two one-sided null hypotheses using the two one-sided tests for pharmacokinetic parameters: area under the concentration-time curve (AUC) and maximum concentration (Cmax). By virtue of the intersection-union test, there is no need for multiplicity adjustment in testing the two one-sided null hypotheses within each parameter. However, the decision rule for bioequivalence often requires equivalence to be achieved simultaneously on both parameters that contain four one-sided null hypotheses together; without adjusting for multiplicity, the family wise error rate (FWER) could fail to be controlled at the nominal type-I error rate α. The multiplicity issue for bioequivalence in this regard is scarcely discussed in the literature. To address this issue, we propose two approaches including a closed test procedure that controls FWER for the simultaneous AUC and Cmax bioequivalence and requires no adjustment of the type-I error, and an alpha-adaptive sequential testing (AAST) that controls FWER by pre-specifying the significance level on AUC (α1) and obtaining it for Cmax (α2) adaptively after testing of AUC. While both methods control FWER, the closed test requires testing of eight intersection null hypotheses each at α, and AAST is at times accomplished through a slight deduction in α1 and no deduction in α2 relative to α. The latter considers equivalence reached in AUC a higher importance than that in Cmax. Illustrated with published data, the two approaches, although operate differently, can lead to the same substantive conclusion and are better than a traditional method like Bonferroni adjustment.

Sample size determination in two-sided distribution-free treatment versus control multiple comparisons

The problem of power and sample size determination for distribution-free multiple comparison tests of K treatments versus a control group is addressed. We define the power as the probability of correctly rejecting one specified or all K hypotheses, corresponding to the per-pair and all-pairs power, respectively. The power formulas are derived for both joint ranking and pairwise ranking mechanism for general multiple comparison problems, followed by explicit form of these formulas when the single-step, step-down, or step-up adjustments are applied. The proposed power and sample size calculation methods apply to scenarios both when the underlying distributions are known and when they are unknown but a pilot study is available. Numerical methods via quasi-Monte Carlo integration and Monte Carlo integration are assessed. Our simulation studies show the accuracy of the power and sample size calculation formulas. We recommend the Monte Carlo integration as the calculation algorithm. An example from a mouse peritoneal cavity study is used to demonstrate the application of the methods.

Statistical evaluation of toxicological assays: Dunnett or Williams test-take both

The US National Toxicology Program recommends the use of the parametric multiple comparison procedures of Dunnett and Williams for the evaluation of repeated toxicity studies. For endpoints where either increasing or decreasing effects are of toxicological relevance, we recommend the use of the two-sided Dunnett test exclusively. For the many other endpoints, where a priori only one direction is of toxicological relevance, however, we recommend the combination of Dunnett and Williams test. In particular, we recommend the so-called Umbrella-protected Williams test which offers insights for all interesting monotone and non-monotone alternatives while only suffering a marginal loss in power compared to the Dunnett test. We illustrate the power difference analytically and compare the approach for different endpoint types using three real data examples to alternative tests available. Nonparametric tests, which are suitable for the evaluation of skewed distributed or scores data, are also considered. Particular attention is given to the different interpretations of the findings revealed by the different test. R programs used for the analyses are provided.

Resampling in multiple-dose factorial designs

Multiple-dose factorial designs may provide confirmatory evidence that (fixed) combination drugs are superior to either component drug alone. Moreover, a useful and safe range of dose combinations may be identified. In our study, we focus on (A) adjustments of the overall significance level made necessary by multiple testing, (B) improvement of conventional statistical methods with respect to power, distributional assumptions and dimensionality, and (C) construction of corresponding simultaneous confidence intervals. We propose novel resampling algorithms, which in a simple way take the correlation of multiple test statistics into account, thus improving power. Moreover, these algorithms can easily be extended to combinations of more than two component drugs and binary outcome data. Published data summaries from a blood pressure reduction trial are analysed and presented as a worked example. An implementation of the proposed methods is available online as an R package.

A new statistical screening approach for finding pharmacokinetics-related genes in genome-wide studies

Biomedical researchers usually test the null hypothesis that there is no difference of the population mean of pharmacokinetics (PK) parameters between genotypes by the Kruskal-Wallis test. Although a monotone increasing pattern with a number of alleles is expected for PK-related genes, the Kruskal-Wallis test does not consider a monotonic response pattern. For detecting such patterns in clinical and toxicological trials, a maximum contrast method has been proposed. We show how that method can be used with pharmacogenomics data to a develop test of association. Further, using simulation studies, we compare the power of the modified maximum contrast method to those of the maximum contrast method and the Kruskal-Wallis test. On the basis of the results of those studies, we suggest rules of thumb for which statistics to use in a given situation. An application of all three methods to an actual genome-wide pharmacogenomics study illustrates the practical relevance of our discussion.

Partitioning to uncover conditions for permutation tests to control multiple testing error rates

This article discusses specific assumptions necessary for permutation multiple tests to control the Familywise Error Rate (FWER). At issue is that, in comparing parameters of the marginal distributions of two sets of multivariate observations, validity of permutation testing is affected by all the parameters in the joint distributions of the observations. We show the surprising fact that, in the case of a linear model with i.i.d. errors such as in the analysis of Quantitative Trait Loci (QTL), this issue has no impact on control of FWER, if the test statistic is of a particular form. On the other hand, in the analysis of gene expression levels or multiple safety endpoints, unless some assumption connecting the marginal distributions of the observations to their joint distributions is made, permutation multiple tests may not control FWER.

Simultaneous inference in general parametric models

Simultaneous inference is a common problem in many areas of application. If multiple null hypotheses are tested simultaneously, the probability of rejecting erroneously at least one of them increases beyond the pre-specified significance level. Simultaneous inference procedures have to be used which adjust for multiplicity and thus control the overall type I error rate. In this paper we describe simultaneous inference procedures in general parametric models, where the experimental questions are specified through a linear combination of elemental model parameters. The framework described here is quite general and extends the canonical theory of multiple comparison procedures in ANOVA models to linear regression problems, generalized linear models, linear mixed effects models, the Cox model, robust linear models, etc. Several examples using a variety of different statistical models illustrate the breadth of the results. For the analyses we use the R add-on package multcomp, which provides a convenient interface to the general approach adopted here.